[gdal-dev] Faster gdalinfo from COG

[Rahkonen Jukka (MML)]

Hi,

The real issue appears to be in my usage of direct http url instead of using vsicurl in between.

This takes a few seconds for me on Windows with GDAL 3.1.0dev
gdalinfo /vsicurl/http://latuviitta.kapsi.fi/data/jarvi_dem/saimaa_dem.tif
but this takes more than five minutes
gdalinfo http://latuviitta.kapsi.fi/data/jarvi_dem/saimaa_dem.tif

I believe that I used http url directly after reading some QGIS tutorial that suggested that /vsicurl/ is no more needed.

You are right about how I created the file. I used the GDAL 3.1.0dev COG driver and tried to enhance it by computing the statistics with gdal_edit. 

-Jukka-



-----Alkuperäinen viesti-----
Lähettäjä: Even Rouault <even.rouault at spatialys.com> 
Lähetetty: keskiviikko 8. tammikuuta 2020 19.44
Vastaanottaja: gdal-dev at lists.osgeo.org
Kopio: Vincent Sarago <vincent.sarago at gmail.com>; Rahkonen Jukka (MML) <jukka.rahkonen at maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] Faster gdalinfo from COG

On mercredi 8 janvier 2020 12:27:56 CET Vincent Sarago wrote:
> Hi Jukka,
> The time you are seeing is related to your configuration and not 
> specifically to the file itself.
> 
> If you set `GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR` in your env, 
> you’ll see that gdalinfo will be much faster.

The file is also not entirely cloud optimized

$ GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR python swig/python/samples/ validate_cloud_optimized_geotiff.py  /vsicurl/http://latuviitta.kapsi.fi/data/
jarvi_dem/saimaa_dem.tif
/vsicurl/http://latuviitta.kapsi.fi/data/jarvi_dem/saimaa_dem.tif is NOT a valid cloud optimized GeoTIFF.
The following errors were found:
 - The offset of the main IFD should be 192. It is 223880848 instead
 - The offset of the IFD for overview of index 0 is 586, whereas it should be greater than the one of the main image, which is at byte 223880848

How did you generate it ? What surprises me is that it contains the 'ghost' 
area typical of GDAL master COG driver, so the fact that the IFD header is not at the beginning of the file is unexpected.
Would you have edited the file (like adding statistics) after using gdal_translate -of COG ?

After reprocessing it with
gdal_translate saimaa_dem.tif out.tif -of COG -co COMPRESS=DEFLATE I get a 'conformant' COG file

Even

--
Spatialys - Geospatial professional services http://www.spatialys.com