[gdal-dev] warp memory limit default too low and does not have a
even.rouault at mines-paris.org
Tue Feb 7 18:13:38 EST 2012
Le lundi 06 février 2012 02:41:40, Etienne Tourigny a écrit :
> Hi all,
> The default memory limit for warp operations is 64MB, which is far too
> low using recent hardware and leads to inefficiencies in I/O when
> warping large images.
> This can be changed using the -wm option of gdalwarp, or if using the
> API change the value of GDALWarpOptions::dfWarpMemoryLimit.
> More information here:
> However, it would be better to do both of the following:
> 1) Change the default value of 64 MB to something higher, perhaps
> depending on the machine's configuration (say 1/4 of RAM) - this
> possibility is mentioned in gdalwarpoperation.cpp .
Finding good default values is quite difficult and somehow arbitrary. There's
however something to be aware : in the trac entry you quote above, there's a
mention of a use case where more RAM for dfWarpMemoryLimit will actually
decrease performance. So if you intend increasing it substantially (which
would be the case for 1/4 of RAM on modern configurations), that might make
things significantly worse for that use case. That's definitely something to at
least benchmark and lilely fix in the warping code. The use case if I remember
well is "gdalwarp smallimage bigimage" (the real use case being generally
"gdalwarp smallimage1 smallimage2 ... smallimageN bigmosaicofallsmallimages").
If you increase dfWarpMemoryLimit currently, we will currently read an area of
bigimage much larger than needed to include the extent of smallimage, which
cause excessive read operations before warping, and excessive write
> 2) Add a config option so that the default can be set for all
> operations by the user (which overrides 1, but can be overridden for
> each operation)
Not sure to understand how option 2) if different from the -wm option and how
it would be implemented ...
> Any comments?
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
More information about the gdal-dev