[postgis-users] PostGIS Raster Performance

Mon Jan 31 08:25:55 PST 2011

Thanks Etienne.

I guess that was the last version (1.8)?

Even if the driver is working fine, we have to check that clients (like MapServer) are using it fine and we have to compare the performance compared with, say, file system raster files.

Pierre

From: postgis-users-bounces at postgis.refractions.net [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of Etienne Bellemare
Sent: 31 janvier 2011 11:14
To: PostGIS Users Discussion
Subject: Re: [postgis-users] PostGIS Raster Performance

Just to mention that I did try the gdal driver for PGRaster and it is working. I can't tell the speed, and like mentionnend Simon it depends of many things. But I did succeed to render (query and write to disk as an ascii file with prj file) a 100x100 pixels 32b float, single band using gdal_translate in the latest release version.

Etienne
On Mon, Jan 31, 2011 at 10:46 AM, Pierre Racine <Pierre.Racine at sbf.ulaval.ca<mailto:Pierre.Racine at sbf.ulaval.ca>> wrote:
Sunny,

There are two planned options for serving raster from PostGIS raster:

The first (1) is to just register the raster as out-of db raster. Only the georeferences of each tile/raster is stored in the database along with the path to the raster itself in the filesystem (you can add any other attribute as additional columns). PostGIS raster in this case act like a quick SQL raster catalog. Actual raster data (pixel values) are not stored in the database. They are store as JPEG or TIFF in the filesystem. It is up to the application to fetch them where they are. In this case their should be no performance lost comparing with any existing raster server solution storing raster coverage in the filesystem. This options still need testing/improvement. For example, the loader (raster2pgsql.py) is not able, right now, to split big raster into multiple JPEG tiles when you use the -k option. It does it only when you store the tiles in-db.

The second (2) option is to store everything in the DB. The raster is then stored in the PostGIS raster format and need to be converted to JPEG before being sent to web client (desktop applications do not have to rely on this conversion as can implement native support for the PostGIS raster WKB format or rely on GDAL). There are two ways to convert PostGIS raster to JPEG or whatever: the first one (2.1) is to rely on the PostGIS raster GDAL driver used by the application (this is how we got almost free support in MapServer). The second one (2.2) is to ask the database to convert PostGIS rasters to JPEG or whatever format with SQL functions like ST_AsJPEG(rast) or ST_AsTIFF(rast) so that the application receive a JPEG or TIFF binary stream. These function are planned but not yet implemented. This option should be a bit slower than the first as raster are converted on the fly.

Option (1), storing out-of-db, targets applications needing VERY faster retrieval not requering raster edit or analysis and not implemented to read native PostGIS or GDAL raster.

Option (2), in-db storage, targets applications needing raster edit or analysis and able to read native PostGIS or GDAL raster.

Options 1 and 2.2 assume that raster tiles can be sent to the web application without having to be blended with other layers. Application (like MapServer) that has to read and blend rasters with other layers before sending them should rely on 2.1 (GDAL). I think this last option has the better future. It is however not very well tested right now and the GDAL driver just got reimplemented in december. We might have to tweak it at some point to get the kind of performance expected by web applications.

I suggest you give a try to MapServer (Regina & Leo explain how to configure it in PostGIS in Action) and, if it is not fast enough, see with Jorge what we can do.

To answer your other questions: raster loading in PostGIS raster is quite fast even if it still need improvement like not passing through a large intermediate SQL file.

Could you elaborate a bit more on the multiband issue? I'm not very familiar with tile caching tech anymore.

Pierre

From: postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net> [mailto:postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net>] On Behalf Of Sunny Teo
Sent: 29 janvier 2011 21:25
To: PostGIS Users Discussion
Subject: Re: [postgis-users] PostGIS Raster Performance

Hi Regina,

Thanks for the lead on this.

It opens up the light and the room for discussion. Well, as far as I know, Oracle Georaster is not good for serving purpose and on top of it, that takes awful amount of time to load the data into Oracle. I assume that its not the case with PG Raster.

The other issue is, even if one dont want to do store the multiband (say 8 band) imagery into the PG Raster for the Speed issue, can we target to link up the Cache into this? If GWC or Tile Cache jpg/png tiles can be kept here and then publish... it will be nice. Again, the issue will be speed ... or some may argue the fact the it defeats the purpose of Cache which is suppose to be non "ram" required serving.

my only wish, that one day we should be able to easily load the images into PG Raster and publish into a web application and get a performance/speed as good as any image servers.

btw, it was a news to me that GeoServer dont support PG Raster yet... I hope they are working on it now...

Thank you;
Sunny

On Sun, Jan 30, 2011 at 6:30 AM, Paragon Corporation <lr at pcorp.us<mailto:lr at pcorp.us>> wrote:
Sunny,

Good question.  I suspect it will be worse at least at the moment (just as PostGIS was worse when it first came out than reading from shapefile).  How much worse I'm not sure.  If you try and see speed issues, we can tackle them, but it hasn't really been our main focus.

 I suspect no one has tested it extensively as far as dishing out large datasets.  The primary focus of PostGIS Raster is for analysis and then secondary is the rendering.  That is not to say it will work badly or we don't care about rendering speed, just that the GDAL drivers etc. needed by Mapserver probably still have a few wrinkles which would make it suboptimal for that kind of work.  The more people use it and find issues, the faster issues will be addressed since we would have data points to go by.

As far as GeoServer goes, I recall reading a thread where Chris Holmes said he would work on a PostGIS Raster support for GeoServer, but not sure where that is down the pipe, but to my knowledge, GeoServer does not YET support PostGIS raster.

Hope that helps,
Regina

________________________________
From: postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net> [mailto:postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net>] On Behalf Of Sunny Teo
Sent: Saturday, January 29, 2011 3:44 AM
To: postgis-users at postgis.refractions.net<mailto:postgis-users at postgis.refractions.net>
Subject: [postgis-users] PostGIS Raster Performance
List,

Can anyone comment on the "Performance" of the PostGIS Rater or WKTRaster ?

If you were to serve this onto a web application, how effective are they? Well, I'm talking of very large datasets like 10GB each and 100 of them. How does it compare with serving the images as files directly.. from a GeoTIFF, ECW, Sid, etc. (obviously using Geoserver/Mapserver or some specialized image serving solution)

Been searching all over the web for the answers, but no luck.

Hope to get some insights from the users.

Thank you;
Sunny.

_______________________________________________
postgis-users mailing list
postgis-users at postgis.refractions.net<mailto:postgis-users at postgis.refractions.net>
http://postgis.refractions.net/mailman/listinfo/postgis-users

_______________________________________________
postgis-users mailing list
postgis-users at postgis.refractions.net<mailto:postgis-users at postgis.refractions.net>
http://postgis.refractions.net/mailman/listinfo/postgis-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20110131/077fe78b/attachment.html>