[gdal-dev] How to improve gdal_rasterize perfomance?

[Greg Coats]

I support Frank's suggestion, and note that it is my understanding  
that when the user does not specify a value for --config  
GDAL_CACHEMAX, then the default of (only) 40 MB is used.

In January 2009, working with GDAL version 1.6.0, I did some testing  
and documenting of the effect of GDAL_CACHEMAX. Using as input an  
uncompressed GeoTif 15,000 x 15,000 x 3 = 675 MB, organized into 1024  
x 1024 tiles, I found that using gdal_translate to convert this  
GeoTif to a GeoJP2 with the default --config GDAL_CACHEMAX 40 took  
10473 sec (= 175 minutes), but after increasing GDAL_CACHEMAX by only  
12.5% to --config GDAL_CACHEMAX 45 the processing took only 166 sec  
(= 2.7 minutes), which is 63 times faster!
Greg

GDAL 1.6.0, released 2008/12/04
Input GeoTif 15,000 x 15,000 x 3 = 675 MB
gdal_translate -of JP2KAK -co QUALITY=10 --config GDAL_CACHEMAX
0040  10473.14 sec
0041  10326.75 sec
0042  10492.85 sec
0043   7081.57 sec
0044   3503.33 sec
0045    166.23 sec
0050    166.31 sec
0055    165.89 sec
0060    165.78 sec
0080    165.48 sec
0100    166.53 sec
0125    165.59 sec
0150    165.72 sec
0200    165.65 sec

On Sep 16, 2009, at 9:42 AM, Frank Warmerdam wrote:

> Hermann Peifer wrote:
>> Hi,
>> Are there any hints from the experts about how to improve  
>> gdal_rasterize perfomance?
>> My specific case is to rasterize a 1+GB shapefile into a 26800 x  
>> 23200 GeoTiff. Are there some memory settings or GeoTiff creation  
>> options that could speed up the rasterization?
>
> Hermann,
>
> This sounds like a heap of processing that needs to be done.
>
> You can increase the chunk size on which gdal_rasterize operates by
> increasing the GDAL_CACHEMAX config option.  I would suggest sizing
> it to be roughly 1/3 of your machine RAM.
>
> eg.
>
> gdal_rasterize --config GDAL_CACHEMAX 600 ...
>
> would be appropriate on a machine with 2GB of RAM.
>
> Beyond that I can't think of much that might help.  The above  
> basically
> reduces the number of passes needed through the vector data.  It is  
> all
> processed once for each chunk of the raster that is processed.
>
> Best regards,

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20090916/e7ec1ca4/attachment-0001.html