[Gdal-dev] GDAL Postgis Raster Driver

Wed Aug 31 15:18:02 EDT 2005

Dave,

I'm definetely not an expert regarding raster formats (I'm just a bit
familiar with png). Any advices concerning the way of optimizing data
storage is thus most welcome. Thank you for all your notes. What I
understood is that you suggest modifying the CHIP structure prior to
doing any extra work on the gdal pgchip driver. If so, I do agree.

Here are some few tips I've used in the current experimental pgchip driver :

- I've used the"future" field to store some meta-information :

future[0] = nBands; 
future[1] = nBitDepth; (8 or 16)
future[2] = nColorType; (RGBA,GREY_SCALE,PALETTE)

- I've used the "compression"  variable to store the number of colors
in case of Palette ColorType. When Palette is used, I place the
palette information between header and data. The CHIP structure then
<header><palette><data>

These modifications you would agree are not acceptable. As you wrote
the original CHIP structure, you might be part of Postgis dev team
(?). Do you know if they plan to improve it ? Are they waiting for
someone to do it ? Shall I ask them directly ;-) ?

Thanks again for all your advices. 

Regards,
Benjamin.

2005/8/31, David Blasby <dblasby at gmail.com>:
> I wrote the original CHIP code.  It was designed to be ultra-simple, and I'm
> assuming that no one's changed it in a while (looking at the code base
> indicates this is true).  Here's a few notes on it:
>  
>  1. Right now you cannot directly index a CHIP.  It has a bounding box and
> an SRID, so you should be able to use the "normal" postgis indexing code &
> only change a few lines.  ie. look for where the indexing code extracts a
> box3d (or box2d) from the GEOMETRY object and change that one line.
>  
>  2. it hasnt been tested well
>  
>  3. its incredibly simple - just a simple header and the data
>  
>  4. consider saving a compressed version (I think you can get postgresql to
> do this for you in the create type command).  Accessing large objects in
> expensive, so the decompressing overhead is probably small compared to the
> cost of pulling extra pages off the disk.
>  
>  5. Also, consider having the header information & the actual data in
> different columns.  Yes, I know this is the opposite of what I said in #1.
> Whenever you run a command on a TOASTed column, postgresql will de-serialize
> the ENTIRE object which is very disk intensive if all you want to know is
> what the image size is or what the SRID is.
>  
>  
>  The entire CHIP code (lwgeom_chip.c in 1.0.x) is only a few hundred lines
> of code.
>  
>  dave
> 
> 
> On 8/31/05, Benjamin Simon <noumayoss at gmail.com> wrote:
> > 
> > Roger,
> > 
> > If you mean "Why use a database engine to store rasters?" then the answer
> is :
> > 
> > - Use spatial queries among tables of rasters images (i.e extracting
> > the right image when given an address or any geocoded object) 
> > - Benefit from the database storage assets  ( -> security, fast access
> > with spatial indexes, backup, ...)
> > - Merge your spatial raster data with any other data you may have in
> > your database
> > 
> > I think one can find many good reasons for using databases, it is just 
> > a very powerfull tool for managing your data, whatever its type or
> > whatever the volume you have. You just need to have the good
> > structures inside your RDBMS to manipulate it. And that's what the
> > CHIP structure aims at. 
> > 
> > On the other hand, if you just mean "Why use the Postgis CHIP C
> > structure instead of redesigning a more approriate new one ?" , then
> > the answer is :
> > 
> > - The Postgis CHIP type is quite well optimized, there could just be 
> > minor improvments to it
> > and that's what i'm working on at the moment. If Postgis development
> > team doesn't want to modify the structure, then we might consider to
> > design a new type in PostgreSQL to store raster data or just cope with 
> > it.
> > 
> > I hope you've found your answer in these words.
> > 
> > Regards,
> > Benjamin.
> > 
> > 
> > 2005/8/31, Roger <roger at sylvanascent.com>:
> > > Ben,
> > > What are the advantages of using this chip scheme? 
> > >
> > > Roger
> > >
> > > -----Original Message-----
> > > From: gdal-dev-bounces at lists.maptools.org
> > > [mailto: gdal-dev-bounces at lists.maptools.org] On Behalf
> Of Benjamin Simon
> > > Sent: Tuesday, August 30, 2005 5:58 AM
> > > To: gdal-dev at lists.maptools.org
> > > Subject: [Gdal-dev] GDAL Postgis Raster Driver 
> > >
> > >
> > > Hi all,
> > >
> > > I've been playing a bit with GDAL driver implementation and as I am
> quite
> > > interested in database development, I 've decided to write a Postgis
> wrapper
> > > for GDAL. I've used the Postgis CHIP datatype to store rasters in 
> > > PostgreSQL.
> > >
> > > Basically, the driver can read files with a pixel definition of 8 or 16
> > > bits. It can deal with 1 or 4 bands (GREY_SCALE, PALETTE and RGBA). I've
> > > only tested to import PNG, BMP, JPEG and TIF files and I think the
> driver is 
> > > not stable yet with any other formats. The driver is supposed to be
> capable
> > > of dealing with projections if Postgis was built with the proj4 library.
> > >
> > > I do think it could be good to move some of the functions from the GDAL 
> > > Driver source code to Postgis source. However, this would require I
> contact
> > > Postgis development team and I prefer to finalize the driver before, as
> it
> > > is really easier to test and develop.
> > > 
> > > I have contacted Frank so that this new experimental driver can soon be
> part
> > > of the global GDAL source tree.
> > >
> > > If you can't wait unil then or if you're interested in testing, you can
> > > download the source code from this address: 
> > > http://simon.benjamin.free.fr/pgchip/ and follow the
> installation
> > > instructions.
> > >
> > > Any suggestions, comments, questions or help are welcome. 
> > >
> > > Regards,
> > > Benjamin.
> > >
> > > _______________________________________________
> > > Gdal-dev mailing list
> > > Gdal-dev at lists.maptools.org 
> > > http://lists.maptools.org/mailman/listinfo/gdal-dev
> > >
> > >
> > 
> > _______________________________________________
> > Gdal-dev mailing list 
> > Gdal-dev at lists.maptools.org
> > http://lists.maptools.org/mailman/listinfo/gdal-dev
> > 
>  
>