Mapserver WCS memory consumption

[Frank Warmerdam]

On 1/19/06, Bart van den Eijnden (OSGIS) <bartvde at xs4all.nl> wrote:
> Hi list,
>
> we are having a bit of troubles with the memory Mapserver WCS consumes
> when making e.g. a MrSID file of around 15.000 by 15.000 pixels. The
> memory consumption reaches up to 1.5 Gb.
>
> Are there ways around this, or is this to be expected of generating such
> a big raster?

Bart,

It is inherent in the way that MapServer handles WCS (or any
raster rendering) that the result image is completely assembled
in RAM.  As Steve notes, 15000x15000x3 is already 675MB.
If the raster goes through mapresample.c (ie. to resample or
reproject) then a potentially larger copy of the image will be
read from the source file before resampling into the output
array in RAM.

Generally I would assume memory use in MapServer for
rendering a large raster, from a large raster with resampling
will consume up to about 3times the size of the uncompressed
result raster.

Having the source imagery come from many smaller tiled
files will use less memory as each source image will be
no larger than one of the source files, and they are discarded
before the next source image is processed.

I would add that I would expect it very hard to process
such a large request in a reasonble period of time as a web
action.   MapServer won't start streaming back data till
all the processing and compression to mrsid is complete.
I would expect you would need a very long timeout to keep
your connection alive.

Overall, I just don't think MapServer's WCS is very suitable
for processing and returning very large requests.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent