[Gdal-dev] Gdalwarp and memory optimization

[Frank Warmerdam]

Nicolas Ribot wrote:
>> -wm memory_in_mb:
>>     Set the amount of memory (in megabytes) that the warp API is 
>> allowed to
>> use for caching.
>>
> 
> Doh !
> I must be really tired !
> This option is clearly indicated in the documentation page...
> 
> Sorry for the noise and thanx for your prompt reply.

Nicolas,

I would also note that setting the GDAL_CACHEMAX parameter higher
can be helpful.  The -wm flag sets the amount of memory used for the
big working buffers (the chunks) when warping.  But down within GDAL
itself the format drivers use a block cache which comes from a seperate
memory pool controlled by GDAL_CACHEMAX.   So on a 1GB system you might
do something like:

   gdalwarp -wm 400 --config GDAL_CACHEMAX 150 ...

This would allocate up to 550MB of RAM between the warping chunk arrays
and the generic cache.  In fact, GDAL_CACHEMAX can also be set as an
environment variable.  And I'd encourage folks with lots of RAM to set
this in their environment normally.   The default small cachesize can
kill performance in some fairly common situations with large files
(such as gdal_translate conversions between line oriented and tiled files).

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org