[postgis-users] A PostGIS-Raster data proposal

[Marshall, Steve]

Frank,

I really appreciate your comments, and your offers of help.

> As mentioned by others I think the licensing conflicts may be 
> more fundamental then you suggest.

I didn't really understand your comment that there are more fundamental
problems with combining Kakadu in an optional extension to PostGIS.
Essentially, you did the same thing with GDAL, so there is a precedence.
The major difference, as I understand it, is the difference in the
PostGIS and GDAL licenses (GPL vs. X/MIT).  Is there something else here
I am missing?

> I'm not sure I entirely buy into the idea of using jpeg2000 
> as a special native raster data type.  There are a variety
> of limitations, and odd performance characteristics with 
> jpeg2000 which would make me nervous.

I'm not intimately familiar with JPEG 2000 at the low levels, so I don't
understand the limitations.  Is it related to uncertainty about how the
specification should be implemented, vagueness in the format spec, or
something else?

> I would suggest instead the same general approach of treating 
> BLOBs in the database as seekable objects from which components
> could be extracted, but instead of restricting things to 
> jpeg2000, allow any GDAL supported raster format to be in 
> the database, and use GDAL to access them in the database.

In any event, I'm intrigued by the idea of using GDAL as the
implementation package for storing images in the databases.  Using GDAL
would certainly provide more flexibility, but I think the functionality
would require that we can do streaming data extraction from any format
without requiring format-specific code.  Does GDAL provide a
format-neutral interface to get at a pixel subregion of an image, or to
extract a low-res (subsampled) version of an image?

> Actually, when I say "any" format, I really mean any format 
> in GDAL that supports the virtualization interface where 
> all IO is routed through VSIFOpenL() and related functions.  
> This is a lot of the drivers, but by no means all.  This 
> would also allow use of formats like tiled GeoTIFF with 
> overviews, from which chunks or overviews can be 
> efficiently extracted with much better understood 
> performance characteristics.

I did take a look at the VSI subsystem in GDAL.  It looks like
integration to this API would require providing file access semantics
for the underlying relational database storage mechanism.  For TOAST,
there might be a problem with being able to seek to an arbitrary
location, and then write data.  The current interface to TOAST (as I
understand it) allows seeking reads only (combining the read and read
operations into a single function).  By contrast, the PostgreSQL large
object interface provides exactly the needed implementation for VSI
integration, but I have reservation about large objects, as noted in my
response to John Novak.

Using a more flexible set of image format does give me some concern
regarding image geonavigation.  One thing that drew me towards using a
single image format for storage was to have less variation in how
metadata (like image navigation) would be provided.  Assuming a more
flexible GDAL implementation, how would such metadata be extracted from
the image?  Does GDAL provide a generic way to extract image navigation
information from any format, e.g. the location of the image's bounding
box in earth-oriented coordinates?  It seems like a tough thing to do,
given that some image formats store their navigations in files separate
from the image data (e.g. worldfiles).

Finally, I wholeheartedly agree with the advice to test the performance
of random access to TOASTed data.  The whole concept is will fail if the
performance of random access is not up to snuff.

Yours,
Steve Marshall