[Gdal-dev] PostGIS CHIP Support - Tiled Pyramids?

[Chris Hodgson]

Frank and others,

I recently became aware of the work on the PGCHIP driver for gdal:

http://simon.benjamin.free.fr/pgchip/

This has resparked my interest in developing a raster storage system for 
PostGIS. Paul Ramsey is generally opposed to this idea, and I think that 
  I agree with him in principal - however there is a continuous stream 
of people interested in raster support for postgis.

So what I am looking for is a solution that puts the least possible 
additional functionality into the database, but still allows maximal 
functionality by leveraging other tools - I think GDAL may be the right 
place for the additional functionality. The existing PGCHIP driver, as I 
see it, allows a database table to essentially act like a tiff file, or 
any other gdal datasource. This requires very minimal support from 
PostGIS - the CHIP type is already there, with a few improvements and/or 
a metadata table like geometry_columns, it should be able to do all that 
is needed.

For good performance however, we need support for tiling, and 
pyramidding. This COULD be built into the database, with additional 
metadata tables defining the pyramids, but I think it might make more 
sense to put this logic into GDAL.  I think that anything involving 
raster I/O would end up using GDAL anyway, and resampling and mosaicing 
logic really doesn't belong in the database. GDAL already supports tiles 
and pyramids that are internal to a datasource - Tiled TIFFs with 
overviews, and JPEG-2000 or other wavelet based files come to mind. But 
does it, or could it easily be made to, support external pyramids and 
tiles?

As far as I can tell, the external tiling and pyramidding is handled in 
Mapserver (Cheetah?), using gdaltindex and min/max resolutions on 
multiple layers. What I'm thinking of is being able to define pyramid 
sets in gdal-land, similar to the existing virtual raster support. Once 
the pyramid set is defined, data could be loaded into and read from it 
automagically... potentially the different layers of the pyramid could 
even reside in different datasets - I can't see why you would want to do 
that, but once you have that level of abstraction it should be easy.

Does something like this already exist? Have you ever thought about it 
before, Frank? Would it be hard to do? I haven't previously had my 
fingers too deep into the gdal codebase, but I'd be interested in trying 
to implement this, with your guidance, assuming it isn't already there 
and just hiding on me.

Let me know your thoughts,

Chris