[gdal-dev] GDAL slow to write GeoTIFF?

[Even Rouault]

Le lundi 27 avril 2015 16:55:24, jramm a écrit :
> I'm writing a custom processing program using GDAL in C.
> 
> I'm processing a raster of roughly 150 000 * 200 000 pixels in windows of
> 256 * 256 pixels.

Is the TIFF tiled ? If it is not, this should help. And/or you could perhaps 
try increasing GDAL_CACHEMAX too.
But for a given organization (tiled vs non-tiled), I'm also surprised you 
would get such a difference between read and write scenarios.

> 
> I'm finding that traversing the raster and applying some basic processing
> to each of the windows takes very little time; about 10 minutes on my
> machine.
> 
> However, as soon as start writing out each window (to the same output
> raster file), the time goes up to about 30 hours....
> 
> The writing is not special...just calling RasterIO like so:
> 
> <code>GDALRasterIO(wt->hBand, GF_Write, xOff, yOff, xSize, ySize, data,
> xSize, ySize, GDT_UInt32, 0, 0);</code>
> 
> 
> In a loop like so:
> 
>   while (movingwindow_next(&win) == 0)
>     {
>     process(winsize, &dg, &cats, &bt, resultBuffer);
>     xOff = movingwindow_xoff();
>     yOff = movingwindow_yoff();
>     //writer_writeWindow(&wt, resultBuffer, win.sizeX, win.sizeY, xOff,
> yOff);
>     }
> 
> ...I would've expected the code to slow down by 2, 3 or even 4 times, but
> 300 times is a bit crazy, no?
> 
> Are their any techniques I can use to optimise writing?
> 
> I am also not particularly limited by the output format if geotiff is not
> the most efficient. I am only constrained by the fact that ESRI ArcGIS must
> be able to read it...
> 
> 
> 
> 
> --
> View this message in context:
> http://osgeo-org.1560.x6.nabble.com/GDAL-slow-to-write-GeoTIFF-tp5203128.h
> tml Sent from the GDAL - Dev mailing list archive at Nabble.com.
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com