[gdal-dev] performance with VRT dataset containing a large number of raster datasets

[Even Rouault]

Hi,

gdalwarp with a VRT as input dataset should only open the source datasets that
compose the VRT if they intersect the target extent you've specified. Are you
sure that there's only one single tile in your VRT that intersects the target
extent you've specified ?

Could you paste one extract of your all.vrt ?

Even

> Hi all
>
> I am experiencing problems using a VRT dataset built from a large directory
> of .tif files.
>
> While if a build the VRT from just one of these datasets, gdalwarp will
> work very fast with it, if I use the VRT will all of these images, it
> becomes undefinite times slower.
>
> Debugging the gdalwarp command, it seems that gdalwarp accesses every
> single raster composing the vrt datasource, even if that single raster is
> not within the extent defined by the -te gdalwarp option.
>
> Here is the output for a vrt composed by just one single .tif file:
>
> $ time gdalwarp -te -88.88 20.28 -87.88 21.28 single.vrt single.tiff
> ...
> Src=1344,4463,1201x1201 Dst=0,0,1200x1200
> 0...10...20...30...40...50...60...70...80...90...100 - done.
> GDAL: GDALClose(single.vrt, this=0x8ca6ef8)
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x8ca9aa8)
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x8ca7208)
> GDAL: GDALClose(single.tiff, this=0x8ca7e98)
> GDAL: In GDALDestroy - unloading GDAL shared library.
>
> real 0m0.137s
> user 0m0.076s
> sys 0m0.052s
>
> and here is the output for a vrt composed of all of the .tif files:
>
> capooti at ubuntu:~/training/vrt$ time gdalwarp -te -88.88 20.28 -87.88 21.28
> all.vrt all.tiff
> ...
> Src=7344,16464,1201x1201 Dst=0,0,1200x1200
> 0...10...20...30...40...50...60...70...80...90...100 - done.
> GDAL: GDALClose(all.vrt, this=0x9d332b0)
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_18_09.tif, this=0x9d46618)
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x9d467c8)
> ...
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_35_09.tif, this=0x9d4c1b8)
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_35_10.tif, this=0x9d4c3a8)
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x9d30800)
> GDAL: GDALClose(/home/capooti/training/vrt/srtm_35_11.tif, this=0x9d4c630)
> GDAL: GDALClose(all.tiff, this=0x9d33990)
> GDAL: In GDALDestroy - unloading GDAL shared library.
>
> real 61m12.130s
> user 0m6.076s
> sys 0m18.052s
>
> Any hint here to prevent gdalwarp to access the unneeded source raster
> files - the defined extent in -te should just hit a couple of ones?
> thanks in advance
> p
>
> --
> Paolo Corti
> Geospatial software developer
> web: http://www.paolocorti.net
> twitter: @capooti
> skype: capooti
>