[gdal-dev] performance with VRT dataset containing a large number of raster datasets

[Paolo Corti]

Hi all

I am experiencing problems using a VRT dataset built from a large directory
of .tif files.

While if a build the VRT from just one of these datasets, gdalwarp will
work very fast with it, if I use the VRT will all of these images, it
becomes undefinite times slower.

Debugging the gdalwarp command, it seems that gdalwarp accesses every
single raster composing the vrt datasource, even if that single raster is
not within the extent defined by the -te gdalwarp option.

Here is the output for a vrt composed by just one single .tif file:

$ time gdalwarp -te -88.88 20.28 -87.88 21.28 single.vrt single.tiff
...
Src=1344,4463,1201x1201 Dst=0,0,1200x1200
0...10...20...30...40...50...60...70...80...90...100 - done.
GDAL: GDALClose(single.vrt, this=0x8ca6ef8)
GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x8ca9aa8)
GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x8ca7208)
GDAL: GDALClose(single.tiff, this=0x8ca7e98)
GDAL: In GDALDestroy - unloading GDAL shared library.

real 0m0.137s
user 0m0.076s
sys 0m0.052s

and here is the output for a vrt composed of all of the .tif files:

capooti at ubuntu:~/training/vrt$ time gdalwarp -te -88.88 20.28 -87.88 21.28
all.vrt all.tiff
...
Src=7344,16464,1201x1201 Dst=0,0,1200x1200
0...10...20...30...40...50...60...70...80...90...100 - done.
GDAL: GDALClose(all.vrt, this=0x9d332b0)
GDAL: GDALClose(/home/capooti/training/vrt/srtm_18_09.tif, this=0x9d46618)
GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x9d467c8)
...
GDAL: GDALClose(/home/capooti/training/vrt/srtm_35_09.tif, this=0x9d4c1b8)
GDAL: GDALClose(/home/capooti/training/vrt/srtm_35_10.tif, this=0x9d4c3a8)
GDAL: GDALClose(/home/capooti/training/vrt/srtm_19_08.tif, this=0x9d30800)
GDAL: GDALClose(/home/capooti/training/vrt/srtm_35_11.tif, this=0x9d4c630)
GDAL: GDALClose(all.tiff, this=0x9d33990)
GDAL: In GDALDestroy - unloading GDAL shared library.

real 61m12.130s
user 0m6.076s
sys 0m18.052s

Any hint here to prevent gdalwarp to access the unneeded source raster
files - the defined extent in -te should just hit a couple of ones?
thanks in advance
p

-- 
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20131224/3c9ba395/attachment.html>