Help on raster optimization

[Frank Warmerdam]

Benoit Andrieu wrote:
> Hi all !
> 
> We do have a problem on using MapServer on raster datas.
...
> We have an high performance degradation when serving WMS clients as WorldWind
> and an home-made client ; more precisely, the degradation is happening on zoomed
> out requests.
...
> Then, I moved forward, in my procmon report and there I saw IT !!
> Mapserver is loading for each of our tiffs files, 6 other files !!
> Those files are :
> pcs.csv = 400 Kb,
> projop_wparm.csv = 165 Kb,
> gcs.csv = 35 Kb,
> unit_of_measure.csv = 20 Kb,
> datum.csv = 70 Kb,
> ellipsoid.csv = 10 Kb
> 
> So, from what I see, for serving my request, mapserver is loading
> approximatively 700Kb of "external" datas and around 50/100 Kb of my 1Mb Tiff
> files. In order to serve a zoomed out request, I have 26 Tiff files. So,
> finally, mapserver is loading 20 Mo.
> 
> So, here is my question : how could I make Mapserv to not load those files again
> and again... ???

Benoit,

Yes, these are all known problems, though most of the time people haven't
done as in depth an analysis as you have.  There are plans to optimize
the EPSG file accessing though it is important to know that the GeoTIFF
driver also looks up EPSG codes in the GDAL pcs.csv (and related files).
The planned optimizations are really just for PROJ4.

In the meantime, stripping unneeded stuff from the epsg, and pcs.csv files
can help, or perhaps as you have done just move the commonly used ones to the
front.

The other issue is that opening a lot of GeoTIFF files is expensive.  You
really ought to build an overview layer if you have a lot of geotiffs,
though 26 does not seem like a terribly large number.

Also, somewhat counter intuitively, having overviews in GeoTIFF files results
in quite a bit of extra seeking around from directory to directory to
identify the available overviews.  I suspect this is a bit expensive in
terms of disk blocking.

This is an area that could stand some careful optimization!

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org