[gdal-dev] vrt: prevent opening all source files on open?

[Vincent Schut]

On 08/28/2017 12:30 PM, Even Rouault wrote:
>
> Vincent,
>
> http://gdal.org/gdal_vrttut.html should help:
>
> """Some characteristics of the source band can be specified in the 
> optional SourceProperties tag to enable the VRT driver to differ the 
> opening of the source dataset until it really needs to read data from 
> it. This is particularly useful when building VRTs with a big number 
> of source datasets. The needed parameters are the raster dimensions, 
> the size of the blocks and the data type. If the SourceProperties tag 
> is not present, the source dataset will be opened at the same time as 
> the VRT itself.
>
> [...]
>
> <SimpleSource>
>
> <SourceFilename relativeToVRT="1">utm.tif</SourceFilename>
>
> <SourceBand>1</SourceBand>
>
> <SourceProperties RasterXSize="512" RasterYSize="512" DataType="Byte" 
> BlockXSize="128" BlockYSize="128"/>
>
> <SrcRect xOff="0" yOff="0" xSize="512" ySize="512"/>
>
> <DstRect xOff="0" yOff="0" xSize="512" ySize="512"/>
>
> </SimpleSource>
>
> """
>
> Note: in theory, the VRT driver could be improved to implement this 
> lazy opening behaviour without requiring SourceProperties, but 
> until/if this is implemented some day, you have to use SourceProperties.
>
> Even
>
Unfortunately, it's more complex. Both vrt's have SourceProperties tags 
for each source.
I did another experiment to test if it is .hgt versus .tif, and it 
appears to be something else: I removed all but one of the ComplexSource 
items from each. Now both of them appear to try to open the source raster.

The vrt which apparently does not open it's sources is this one: 
https://cloud.sdsc.edu/v1/AUTH_opentopography/Raster/SRTM_GL1/SRTM_GL1_srtm.vrt
Even opening that with /vsicurl/ is fast.
Copying this vrt locally (and thus invalidating the source paths) works 
and does not result in a read error. Remove all but one of the 
ComplexSource elements, and I get an error on running gdalinfo on it 
(because it cannot find the source raster). I'm stumped...

sorry, have to go to a meeting now. More later.

thanks!
Vincent.

> > Hi,
>
> >
>
> > Is there a way (or: what are the prerequisites) to avoid gdal opening
>
> > all source files from a vrt on opening the vrt?
>
> >
>
> > Context: I have created a large vrt, referencing many tif files. Both
>
> > the vrt and tifs are remote (using /vsigs/). It works, but opening (e.g.
>
> > running gdalinfo) is very slow.
>
> >
>
> > I have a comparable vrt where the main difference is that it references
>
> > .hgt files instead of tifs. Running gdalinfo on this vrt is almost
>
> > instantaneous.
>
> >
>
> > As there are no other significant differences otherwise, I wonder if
>
> > this is because if the vrt machinery encounters a tif, it starts to
>
> > check if there are any accompanying overviews (or other important
>
> > metadata), while it does not do that for a .hgt file? Or is there a
>
> > different reason? And how can I prevent this (without, preferably,
>
> > changing to a different source file format)?
>
> >
>
> > Thanks,
>
> > Vincent.
>
> >
>
> > _______________________________________________
>
> > gdal-dev mailing list
>
> > gdal-dev at lists.osgeo.org
>
> > https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> -- 
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20170828/45aff6bb/attachment-0001.html>