Re: [OSGeo-Discuss] Raster data on a DBMS

[Lucena, Ivan]

Hi there,

>  Of course, both of the above examples are of the "high number of small
>  size images" variety while the raster/geographic applications are of
>  the "relatively fewer number of very large size images" kind. Still,
>  the case studies are instructive.

I would like to jump on that point and ask for some advice from the community

I need to setup a server for a very large raster dataset. The whole think is about 80 time series of environmental variables of all 
kinds. The average number of single band image files for each series is around 250. The time interval varies and the data source 
goes from EOS satellites to old records of weather stations interpolated to raster. The images are in plain WGS84 -180,180,-
90,90 but there are some regional (=continental) series with similar characteristics. The most important clue, the resolution, 
rows and columns: around 1 to 5 km resulting on 720x360 bands. Data type are typically 1 Byte (uint8), 2 Bytes (int16) and 
4Bytes (single precision float) what results in:

80x250=20,000 files
80x720x360x250=5,184,000,000 pixels
80x720x360x250x2=10,368,000,000 bytes

I can't see much of sense in serve images by image as a in typical web-map application and the requirements are not pointing 
to that direction either.  The main requirements are:

* Query and serve the pixel history in a format suitable for simple statistical and graph presentation. Example: someone selects 
one location that is about 1 km large on the client application and that should produce a graph based on the environmental 
variables selected.

* The other obvious requirement is to provide fast lower resolution frames to produce animation.

* Use OGC standards (WCS?)

I have check several alternatives of data storage and web-maping tool but I still did not decided on any of then.

The most obvious problem I have already noticed is that querying a few pixels on 250 untiled single-band raster files is absurdly 
much slower to query just one multi-band file. 200 times slower if I remember it correctly. Tiles and interleaving is also a important factor, of course. But not so dramatical.

That would reduce the complexity a lot. It would be *just* 80 files. I probably don't even need a separate then in folder. I 
could also generate pyramids overlay on those. That would increase the data storage a little:

80 files
80x720x360x250x2x2=20736000000

Does anybody advocate Geoserver, Mapserver, ArcServer, Imageserver, tile cache, openlayers or others? Or should I use raster 
or database for that?

Best regards,

Ivan