[gdal-dev] GDALWarpOperation.ChunkAndWarpImage slow on large tiffs

[Even Rouault]

Le mercredi 09 mars 2011 19:46:26, Radim Blazek a écrit :
> Hi,
> 
> GDALWarpOperation.ChunkAndWarpImage was reported to be extremly slow
> on larger geotiff (over 1GB). Is it possible that it is much slower
> with respect to GDALRasterIO()  even without reprojection? Should I
> return to 'manualy' reading data via GDALRasterIO() + fidling with
> data extents and cells alignement? I thought that GDAL
> ChunkAndWarpImage will do it even faster, possibly knowing better (?)
> various formats nature.

Yes, I wouldn't be surprised that ChunkAndWarpImage is noticeably slower if it 
is used only as a GDALRasterIO() replacement for the no reprojection case. 
There are quite a lot of extra processing that will be done to handle the 
nominal case of reprojection and you'll pay that cost even if you don't need 
it.

By reading the code of GDALCreateGenImgProjTransformer2(), namely 

    if( pszSrcWKT != NULL && strlen(pszSrcWKT) > 0 
        && pszDstWKT != NULL && strlen(pszDstWKT) > 0 
        && !EQUAL(pszSrcWKT,pszDstWKT) )
    {
        CPLString osSrcWKT = pszSrcWKT;
        if (hSrcDS 
            && CSLFetchBoolean( papszOptions, "INSERT_CENTER_LONG", TRUE ) )
            osSrcWKT = InsertCenterLong( hSrcDS, osSrcWKT );
        
        psInfo->pReprojectArg = 
            GDALCreateReprojectionTransformer( osSrcWKT.c_str(), pszDstWKT );
    }

you can see a small optimization in which the reprojection transformer isn't 
instanciated if the 2 SRS are identical. But the WKT strings must really 
match. We should perhaps use OSRIsEqual() instead to be more robust to minor 
differences in WKT. Dunno if you run into this however. And I don't know if 
this optimization accounts for much.

I've just run this non-scientifical benchmark (GDAL built with -O2) that 
confirms your findings :

$ time gdal_translate 
~/gdal/data/bmng/world.topo.bathy.200406.3x21600x21600.B2.tif out.tif
Input file size is 21600, 21600
0...10...20...30...40...50...60...70...80...90...100 - done.
real	0m12.102s
user	0m7.560s
sys	0m1.320s

$ time gdalwarp ~/gdal/data/bmng/world.topo.bathy.200406.3x21600x21600.B2.tif 
out2.tif
Processing input file 
/home/even/gdal/data/bmng/world.topo.bathy.200406.3x21600x21600.B2.tif.
0...10...20...30...40...50...60...70...80...90...100 - done.
real	1m13.387s
user	0m48.610s
sys	0m23.270s

where the source is a 21600x21600x3 JPEG-compressed TIFF with 256x256 tiles.

(with a unoptimized build, it is 14 s versus 3 minutes...)

> 
> Is there way to tune somehow performance? Currently no reprojection
> and GRA_NearestNeighbour.
> 
> Still the same code which I posted before
> http://trac.osgeo.org/qgis/browser/trunk/qgis/src/providers/gdal/qgsgdalpro
> vider.cpp?rev=15400#L625
> 
> Thanks for help.
> Radim
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev