[gdal-dev] Reading a VRT file & overview usage

[Angus Dickey]

Even,

Thank you for your response.

Since you can't reproduce this with your test it makes me think there is
something about the input data. I put together some sample data in a public
S3 bucket and some commands. Would you be able to try them out to see if
that reproduces the behaviour? I tried GDAL 2.2.3, GDAL 2.4.2, and GDAL
3.0.1 and saw the same thing. No overlay use with any resampling method
other than nearest.

The COGS are single band float32 with zero indicating NODATA and the VRT is
built with using gdal.BuildVRT('mosaic.vrt', files,
options=gdal.BuildVRTOptions(srcNodata=0, VRTNodata=0)) in Python. The VRT
also has pre-built statistics, if that is a factor.

# v3.0.1
gdalinfo --version

# Output looks normal to me.
gdalinfo /vsicurl/
https://dev-myheat-ca.s3-us-west-2.amazonaws.com/data/gdal/mosaic.vrt
--config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR

# This is actually your tool I think! Reports the COG as valid.
python validate_cloud_optimized_geotiff.py /vsicurl/
https://dev-myheat-ca.s3-us-west-2.amazonaws.com/data/gdal/20.cog.tiff

# ~4 seconds (--debug ON will show GDAL loading overviews)
gdal_translate /vsicurl/
https://dev-myheat-ca.s3-us-west-2.amazonaws.com/data/gdal/mosaic.vrt
/tmp/tile.tiff -projwin -9177335.364031402 5305341.259217514
-9174889.379126277 5302895.274312388 -outsize 256 256 -r nearest --config
GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR

# ~60 seconds (--debug ON will show GDAL loading full resolution data)
gdal_translate /vsicurl/
https://dev-myheat-ca.s3-us-west-2.amazonaws.com/data/gdal/mosaic.vrt
/tmp/tile.tiff -projwin -9177335.364031402 5305341.259217514
-9174889.379126277 5302895.274312388 -outsize 256 256 -r bilinear --config
GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR

Could it have to do with the data being float32 or the way the overviews
are built? GDAL seems to be making the choice to skip the overviews for
some reason.

Thank again,

Angus


On Fri, Oct 30, 2020 at 8:53 AM Even Rouault <even.rouault at spatialys.com>
wrote:

> Angus,
>
> > But the second command ignores the overviews and downloads the full
> > resolution imagery leading to very long download times. Is it expected
> > behaviour that all re-sampling methods except 'nearest neighbour' need to
> > request the full resolution data? Can't they apply the selected
> re-sampling
> > method to the overviews? Is it something to do with the way I created my
> > VRT file?
>
> I can't confirm with the following little test
>
> gdal_translate byte.tif test.tif -outsize 2048 2048
> gdal_translate test.tif left.tif -srcwin 0 0 1024 2048 -b 1 -b 1 -b 1
> gdal_translate test.tif right.tif -srcwin 1024 0 1024 2048  -b 1 -b 1 -b 1
> gdalbuildvrt vrt.vrt left.tif right.tif
> gdaladdo left.tif
> gdaladdo right.tif
> python -c "from osgeo import gdal; ds = gdal.Open('left.tif',
> gdal.GA_Update);
> ds.GetRasterBand(1).GetOverview(2).Fill(0)"
>
> copy left.tif, right.tif and vrt.vrt on S3
>
> gdal_translate /vsis3/mybucket/vrt.vrt out.tif -r bilinear -outsize 128 128
>
> Works fine with GDAL 2.4.4 and master
>
> If you can come up with a minimum reproducing scenario (input files +
> script/
> command to reproduce), please file a ticket on github
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20201103/a4b26459/attachment-0001.html>