[postgis-users] Re: Proposed SQL interface for PGRaster

[Stephen Marshall]

Patrick,

I agree that the PGRaster type should *be* the input/output format, but 
we need to nail down what that statement means.  PostGIS faced a similar 
issue.  They choose to represent GEOMETRY objects in a format based on 
specs developed in the OpenGIS community, i.e. first they used Extended 
WKT, and later a Base-64 encoded version of the OpenGIS WKB format.  
PGRaster will need to make a similar formating choice for its primary 
Input-Output representation. 

Note that the I/O format does not mean the data must be stored on disk 
in this format.  For example, GEOMETRY objects are transformed out of 
base-64 encoding before storage to disk.  As I indicated in the Wiki, 
the storage format could include some reorganization of the data to 
improve performance (e.g. caching the bounding box as a geometry).  The 
only caveat is that any of the derived data must be able to be 
systematically created from the information in the I/O format.

I have been gravitating towards using a known image format as the I/O 
format to avoid having to reinvent so many of their features.  For 
example, image formats tend to be well-compressed and accomodate most of 
the metadata we need to have about a raster, endian-ness, etc.  If we 
invent a our own format for PGRaster, we will have to accomodate all 
those features, and we will have to write our own input/output codecs.  
Why do that when we can use an existing format, with codecs that are 
already written? 

Interchanging data in a well-known image format also feels more open to 
me.  Clients can easily link against libraries like GDAL to decompress 
the data.  If we invent a format, we would have to provide a client-side 
codec, or require all clients to write their own.   Returning the data 
in some internal format used by the server seems like foisting a server 
implementation detail on all the clients.  As I see it, the PGRaster 
data format should be the open interface for interchange between two 
systems: the PostGIS server and a client application.

Using a well-compressed image format will tend to optimize input and 
output of PGRaster data.  Since the formatted raster data will be 
transferred to and from the server in a well-compressed form, there will 
be fewer bytes in the client-server communications.

I can see the advantages of picking a single raster format to be *the* 
PGraster format, rather than being format agnostic.  In fact, when I 
originally floated the PGRaster idea a month or so ago, I suggested 
using JPEG2000 as the single raster format.  However Frank Warmerdam (of 
GDAL fame) argued for an approach based on a more abstract interface 
into different raster formats.  This is similar to how GDAL works, and 
he suggested that GDAL could be used in the PGRaster implementation to 
hide away details of the exact raster format.

I'm still on the fence about the advantages of using a single format vs. 
multiple formats supported through an abstract raster codec.  However, I 
prefer the idea that PGRaster interchanges data in an established 
format, rather than some format we invent.  I think a well-established 
format will be more open, require fewer bytes to be transferred between 
client and server, and avoid the daunting task of designing our own 
image format.

Steve

Patrick wrote:

>"Stephen Marshall" <smarshall at wsi.com> wrote in message 
>news:4575E64E.3020108 at wsi.com...
>  
>
>>Mark,
>>
>>I agree that the user needs to know about the endian-ness of the data. 
>>However, I'm envisioning a system where the data is returned in a 
>>well-known image format, e.g. Geo-TIFF, JPEG2000, etc.
>>    
>>
>
>Hi Stephen,
>
>I am very much against serving up the data in any specific format. PGRaster 
>*is* the format. I would say that PostGIS serves up the data in some 
>generic, described format, just plain ol' tiles AFAIAC, and that client side 
>applications convert this to a format the user is requesting.
>
>There are a number of reasons to do this. First of all, most GIS 
>applications don't work with file-based formats; the first thing they do is 
>convert it to some internal representation. Secondly, the user or 
>application can potentially request a HUGE slab of data and thus cripple the 
>server.
>
>My contention is that if PostGIS supports rasters and does so well, then 
>applications will implement a driver to access the data in PostGIS format. 
>The same happened with the standard geometries now in PostGIS. I don't think 
>any application accessing PostGIS does so by converting WKT to a shapefile 
>and then read in the shapefile.
>
>Cheers,
>Patrick 
>
>
>
>_______________________________________________
>postgis-users mailing list
>postgis-users at postgis.refractions.net
>http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>  
>