[mapserver-dev] Streaming output from WCS

[Even Rouault]

Le mercredi 10 février 2016 18:29:06, Rahkonen Jukka (MML) a écrit :
> Hi,
> 
> Some WCS users would prefer if large output from WCS (large=a few gigabytes
> or more) could be sent streaming with a relatively short lag between the
> request and the start of response. If the output image is made complete on
> the server side before the transfer starts the lag can be some minutes. Is
> it reasonable idea to stream 1-4 GB GeoTIFFs from Mapserver? What
> restrictions would it put to the possible file formats, compression etc?

There are several issues :

- MapServer currently composes a in-memory image of the result, that is 
limited to MAXSIZE x MAXSIZE pixels (defaults to 2048 if unspecified). If you 
want to generate really big images, then the architecture of the WCS renderer 
should be significantly changed. A possibility would perhaps be to use GDAL VRT 
to have a on-the-fly result raster, not memory bound.

- if the above is not a problem or solved, you also need an output format with 
streaming capabilities. For GeoTIFF, GDAL 2.0 supports creating *uncompressed* 
GeoTIFF ( see http://gdal.org/frmt_gtiff.html ). It could likely be extended to 
generate compressed GeoTIFF in streaming mode, but I didn't go to that point 
as such GeoTIFFs couldn't be read in a streaming way (since the tables with 
tiles/strips offsets and sizes must be written at the end of the file, once all 
offsets and sizes are known) and the objective was to be able to pipe out / 
pipe in the files.
But even if the output format is streamable, MapServer currently either 
generates a in-memory or on-disk temporary output file before sending the 
bytes. This would require changes in it too.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com