[postgis-users] PostGIS WKT Raster

[Pierre Racine]

Hi Imran, thanks for your interest,

>However for building GIST indexes he is also worried 
>from the embedded BBOX information and proposes to store it 
>separately like metadata information in georaster and 
>similarly I can imagine he have to drag out more information 
>out of raster type for query optimization, hierarchal  storage 
>structure etc

Actually, in the PPT presentation, I just ask if it is possible to embed the raster BBOX in a new WKT/WKB type and still build the Gist index with it. My first guess is "yes" it is possible... So I don't really plan yet to store the BBOX outside of the tiles as metadata. Unless someone says it not possible... I really don't like the GeoRaster relational schema. PostGIS geometries are simple to use and successfull because they are self sufficient. I think raster tiles should be viewed the same way.

>1-     Whether this new structure will be according to OGC 
>specification for coverage?

As explained in slide 42, the OGC specification for coverage do not speak about how a coverage should be stored. It speak about how it should be queried. A coverage can also be a vector coverage as stored in PostGIS right now. Unfortunalety this specification does not really propose a seamless integration of topological operator between a raster coverage and a vector coverage. This said, however we store a coverage, the integration of the operators I propose does not exclude the possible addition of queries like the one proposed in the coverage specs. My point of view is "lets see if/how people will implement the coverage specs for a vector coverage and then we could try to do the same for a raster coverage"...

>2-     While adopting this structure are we not limiting 
>ourselves to support only the integrated vector raster 
>operations on the cast there are hundreds of image (raster) 
>processing libraries/algorithms from open source that are 
>supporting Xing Lin (oracle) georaster type  and direct 
>support from DBMS like indexes etc if we have Lin.s georaster 
>type we can extend it directly with hundreds of aforesaid 
>libraries/algorithms available from open source community

First, like I said I don´t really like this design mainly for three reasons:

-it is a copy of Oracle Georaster which (I think) was not originally designed to store georeferenced images... There is also these patents stuff.
-it does not propose any integration with vector,
-it splits the object into two separated tables. I think this is too complicated for what we need. I prefer the PostGIS way of storing a geometry as a self containing object.

Second, I don't know yet any other package/libraries that supports it. Maybe I'm wrong...

Pierre

>best regards
> 
>--imran 
> 
>
>
>
>----- Original Message ----
>From: Paul Ramsey <pramsey at cleverelephant.ca>
>To: Pierre Racine <Pierre.Racine at sbf.ulaval.ca>
>Cc: warmerdam at pobox.com; nicolas.gignac at msp.gouv.qc.ca; 
>smarshall at wsi.com; PostGIS Users Discussion 
><postgis-users at postgis.refractions.net>
>Sent: Monday, July 14, 2008 10:11:05 PM
>Subject: Re: [postgis-users] PostGIS WKT Raster
>
>Pierre,
>
>Firstly, let me commend you for your understanding of the open source
>community process! Not being shy, not taking "no" for an answer, and
>needling the people you need to move are all excellent ways to keep
>the wheels turning, particularly when done with good humor as you have
>shown.  Other first timers could/should learn from your example!
>
>So now I owe you a reply, no doubt:
>
>You have gotten over my first hurdle, in that you have an actual use
>case for raster, that is more involved than "I want to my images,
>because I hear database are faster".
>
>I have long thought that analysis, in particular a fusing of vector
>and raster analysis was the only really compelling use case for raster
>in database, so again, you're on the side of the angels (me).
>
>So, is your *design* good?
>
>Probably as good as possible, but let me point out places of concern
>and see what you think:
>
>- You propose to do this because their is a "great demand for it", but
>the great demand is generally the stupid demand for images-in-database
>"just because". You must either somehow *stop* those people, or ensure
>your solution is capable of meeting their needs.  Since you propose to
>meet the demand, presumably you aim for the latter.  This *will*
>involve a good deal of non-analytical pre-processing infrastructure to
>convert people's multi-resolution, overlapping/underlapping file
>libraries into a uniform resolution coverage.  I suggest you ignore
>these people.
>- You are going to carry a certain amount of information into the
>database and process data into bands, potentially for external
>storage.  Why not store external data in a format like TIFF that can
>hold all the bands and pyramids, etc?
>- Once you have the idea of external storage, you could as easily move
>to internal storage with the BLOB interface.  Not that I recommend
>this, it's easier to back up and restore a filesystem of files than a
>huge database full of BLOBs.
>- If you step back a bit and don't even bother splitting up the
>rasters into tiles, you can use an existing raster access library like
>GDAL to work with serialized data.
>- Or you could use GRASS as your serialization format and hook into
>that.  Added bonus: free algorithms.
>- Basically the less you muck with creating a "disk format for raster"
>(solved problem) and the more you muck with "integrating raster and
>vector analysis in SQL" (unsolved problem) the more leverage you will
>get.
>- I like your external storage idea. What are the implications for
>CREATE TABLE foo AS SELECT...?
>
>Summary: your proposal is better than any I have seen, addresses
>solving problems that if solved will provide actual new functionality
>and benefit to users, and clearly you've thought this through over
>some time.
>
>So, I look forward to seeing your initial plan of attack for 
>development :)
>
>As you begin looking at the code base, you'll find a few stubs of
>raster support that Sandro built at my request for rasterizing vectors
>into CHIPs in the database... basically the first tip toe down the
>path you have written up so completely.
>
>Go with God.
>
>Paul
>
>
>On Mon, Jul 14, 2008 at 12:43 PM, Pierre Racine
><Pierre.Racine at sbf.ulaval.ca> wrote:
>> Paul Ramsey hasn't yet said my design was:
>>
>> -a "meme" (http://blog.cleverelephant.ca/2008/06/x-my-l.html),
>> -a "waste of time" 
>(http://postgis.refractions.net/pipermail/postgis-devel/2007-Ju
>ly/002653.html)
>> -"useless" 
>(http://postgis.refractions.net/pipermail/postgis-users/2007-Ma
>y/015578.html)
>> -that it was "pointless" 
>(http://postgis.refractions.net/pipermail/postgis-users/2007-Oc
>tober/017250.html)
>> -or that I was a "fool" 
>(http://postgis.refractions.net/pipermail/postgis-users/2007-Oc
>tober/017239.html)
>>
>> So I conclude it's a good design!!! :-)
>>
>> He asked for a good design some time ago for raster in the 
>database 
>(http://postgis.refractions.net/pipermail/postgis-users/2006-Oc
>tober/013628.html)
>>
>> Where is the clever elephant?
>>
>> Pierre
>>
>>>-----Message d'origine-----
>>>De : postgis-users-bounces at postgis.refractions.net
>>>[mailto:postgis-users-bounces at postgis.refractions.net] De la
>>>part de Pierre Racine
>>>Envoyé : 7 juillet 2008 11:30
>>>À : postgis-users at postgis.refractions.net
>>>Cc : warmerdam at pobox.com; nicolas.gignac at msp.gouv.qc.ca;
>>>smarshall at wsi.com
>>>Objet : [postgis-users] PostGIS WKT Raster
>>>
>>>Hi raster people,
>>>
>>>I'm starting a 2-3 year project to develop a web-based application to
>>>automate certain GIS tasks commonly needed in ecological 
>research. The
>>>tool will support queries over VERY large extent based on a 
>Canada-wide
>>>assemblages of raster and vector data layers. The queries will
>>>typically
>>>involve intersecting those layers with buffers defined 
>around points or
>>>transects.
>>>
>>>We would like to implement this system with PostGIS, but it currently
>>>does not to support raster data. We could convert all our data to a
>>>single format (raster or vector) and use other tools, however PostGIS
>>>seems to us the best and most powerful vector analysis tool available
>>>and we would prefer to use it. We would like to develop a unified
>>>toolkit so that the application mostly need not worry about
>>>whether base
>>>layers are in vector or raster format. We are then strong 
>proponents of
>>>having raster functionality in PostGIS. I have reviewed 
>every thread on
>>>this list on the subject. I have analysed them and I have 
>compiled them
>>>in the wiki
>>>(http://postgis.refractions.net/support/wiki/index.php?RasterNotes).
>>>
>>>The argument goes like this: The geodatabase paradigm has 
>been a major
>>>recent enhancement in GIS technology, I feel that the seamless
>>>integration of raster and vector analysis should be one of the next.
>>>Spatial analysis is emerging, beside the making and 
>publishing of maps,
>>>as one of the main desktop and web-GIS applications. Rastor/vector
>>>seamless integration is already done for display (in most 
>GIS and with
>>>MapServer or ArcIMS on the web) but definitely not for
>>>analysis. Desktop
>>>analysts must still learn to use two distinct toolsets within most
>>>(all?) GIS packages: one toolset for raster and another for
>>>vector data.
>>>Would not it be easier to build and use applications if we 
>had a unique
>>>data query and analysis toolset independent of the data 
>model? I don't
>>>know any tool having this approach right now. A PostGIS 
>foundation that
>>>addressed this problem would offer better directions to application
>>>developers than the dichotomic one proposed by ESRI since their
>>>beginning. It would provide the necessary abstraction to develop GIS
>>>with ONE set of analysis tools. I feel this is one good reason to
>>>integrate raster in PostGIS. There is "GIS" in the word "PostGIS".
>>>PostGIS should then provide a COMPLETE GIS data foundation (read
>>>"base"!) for geoapplication developers. I think Steve 
>Marshall's design
>>>is a good step in this direction
>>>(http://postgis.refractions.net/pipermail/postgis-users/2006-De
>>>cember/01
>>>4059.html).
>>>
>>>At the same time, we would have a chance to reconsider the 
>raster model
>>>as a coverage instead of a series of images. Spatial 
>databases got rid
>>>of map sheets by allowing complete vector coverages to be stored as
>>>single table (e.g. the entire United States) This approach 
>should works
>>>for raster data as well. Isn't this the approach taken by ArcSDE?
>>>
>>>In summary, I think we must stop thinking about PostGIS as a
>>>mere vector
>>>data repository to support mapping applications. This is the way most
>>>objectors to raster integration seem to view it. We must think about
>>>PostGIS as a powerfull indexed data analysis tool. Seamless
>>>raster/vector analysis in the database could lead to a major
>>>simplification of geoapplications.
>>>
>>>I prepared a PowerPoint presentation with a complete specification of
>>>raster in PostGIS and an analysis of what should be the result of
>>>overlay operation between raster and vector layers. You can download
>>>this PPT at: 
>http://www.cef-cfr.ca/uploads/Membres/WKTRasterPostGIS.ppt
>>>Please have a look at it, feel free to answer questions I ask and
>>>comment.
>>>
>>>Here are the main propositions of this design:
>>>
>>>-Like a vector feature, rasters are stored as a subtype of 
>"geometry"(a
>>>new WKB/WKT geometry type called RASTER instead of POINT or POLYGON)
>>>-There is no distinction between a raster and a tile. A tile
>>>is a raster
>>>and a table of rasters can be seen as a tile coverage. 
>Hence, contrary
>>>to Oracle GeoRaster, only one table is needed to store a coverage and
>>>there is only one type.
>>>-It support raster in the database AND raster out of the 
>database. This
>>>mailing list has shown that both approaches have their pros and cons.
>>>Let's not impose one approach over the other.
>>>-It supports: multi-band, pyramids and variable nodata value, raster
>>>size, pixel size and pixel type for each row.
>>>-It supports import/export from/to Tiff and JPEG.
>>>
>>>But moreover seamless raster/vector integration is materialized by
>>>theses propositions:
>>>
>>>-A lot of existing vector-only functions are adapted to work
>>>with raster
>>>also.
>>>-Most new raster functions are also implemented to work with vector.
>>>Only basic raster derivation functions are proposed.
>>>-Most vector/vector existing functions are adapted to work seamlessly
>>>with raster/vector or raster/raster
>>>
>>>I also want to ask:
>>>
>>>-What is the status of PGRaster? Is any development now underway?
>>>
>>>We have some resources to devote to this project over the next
>>>few years
>>>and are very interested in forming collaborations to move this work
>>>forward.
>>>
>>>Pierre Racine
>>>GIS/Programmer Analyst
>>>University Laval
>>>http://www.cef-cfr.ca/index.php?n=Membres.PierreRacine
>>>_______________________________________________
>>>postgis-users mailing list
>>>postgis-users at postgis.refractions.net
>>>http://postgis.refractions.net/mailman/listinfo/postgis-users
>>>
>>
>_______________________________________________
>postgis-users mailing list
>postgis-users at postgis.refractions.net
>http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
>      
>_______________________________________________
>postgis-users mailing list
>postgis-users at postgis.refractions.net
>http://postgis.refractions.net/mailman/listinfo/postgis-users
>