[Gdal-dev] Will increasing the RAM on a server increase the speed of gdalwarp?

[Frank Warmerdam]

Christopher Smith wrote:
> I am newcomer to image warping, and was surprised and disappointed at 
> your direction that extra memory likely would be of no benefit to 
> throughput.  You have suggest manipulation of cache size (your example 
> parameter list including the token "GDAL_CACHEMAX").  You also suggest 
> the adjustment of the size of "working buffers", which I inferred was 
> accomplished by the "-wm 500" argument and value.  Your "operate on 
> 500MB chunks at a time which is better than the default" certainly makes 
> sense.
> 
> My confusion is this:
> Reading the "gdalwarp" documentation  
> (http://www.gdal.org/gdalwarp.html), the "-wm" is advertised to direct 
> the caching size rather than the "operating block" size.  The "GDAL Warp 
> API Tutorial" documents a "GDALWarpOptions" object with a 
> "dfWarpMemoryLimit" attribute allowing the setting of "maximum amount of 
> memory to be used by the GDALWarpOperation 
> <http://www.gdal.org/classGDALWarpOperation.html> when selecting a size 
> of image chunk to operate on".  This gives rise to hope that with 
> computer populated with a large amount of RAM, I might set a very high 
> block size (not allowing block size + cache size + GDAL code segment(s) 
> size to exceed physical memory limit, of course), and improve performance.
> 
> My hopeful suspicions conflict with what you have written.  Further, 
> your example parameter list seems to address cache twice (though 
> admittedly I have not seen "GDAL_CACHEMAX" in written documentation) and 
> block size not at all.  Have I misunderstood something (or many things!)?

Chris,

I must admit my explanation of the two parameters was not all that
illuminating.

The GDAL_CACHEMAX affects the amount of space reserved for the low
level io buffers.  When blocks are read from disk, or written to disk,
they are cached in a global block cache by the GDALRasterBlock class.
Once this cache exceeds GDAL_CACHEMAX old blocks are flushed from the
cache.

You can think of this as primarily an IO cache, and it mostly benefits
you when you might need to read or write file blocks more than once.
This could occur, for instance, in a scanline oriented input file which
is processed in multiple chunks (horizontally) by gdalwarp.

The -wm flag affects the warping algorithm.   The warper will total up
the memory required to hold the input and output image arrays and any
auxilary masking arrays and if they are larger than the "warp memory"
allowed it will subdivide the chunk into smaller chunks and try again.

If the -wm value is very small there is some extra overhead in doing many
small chunks so setting it larger is better but it is a matter of
diminishing returns.

Overall, there is only so much more memory can do for you.

To get a better sense of how things are working, you might want to
try running in debug mode.  For instance:

   gdalwarp --debug on abc.tif def.tif

When things shut down you may see messages like:

   GDAL: 224 block reads on 32 block band 1 of utm.tif.

In this case it is saying that band 1 of utm.tif has 32 blocks, but
that 224 block reads were done, implying that lots of data was
having to be re-read, presumably because of a limited io cache.

You will also see messages like:

GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=0,0,512x512 Dst=0,0,512x512

The Src/Dst windows show you the "chunk size" being used.  In this
case my whole image which is very small.  If you find things are being
broken into a lot of chunks increasing -wm may help somewhat.

But far more important than memory are ensuring you are going through
an optimized path in the warper.  If you ever see it reporting
GDALWarpKernel()::GWKGeneralCase() you know things will be relatively
slow.  Basically, the fastest situations are nearest neighbour resampling
on 8bit data with no nodata or alpha masking in effect.

You also might want to check CPU usage while gdalwarp is running.  If it is
substantially less than 100% then you know things are IO bound.  Otherwise
they are CPU bound.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org