[gdal-dev] Upcoming Cloud Optimized Geotiff (COG) related enhancements

[Even Rouault]

> 
> I find "STRILE" slightly jarring when reading the spec because it is a
> non-word with no obvious meaning. It's also distractingly close to
> "sterile". What would you think about "BLOCK" or "CHUNK"?

Ah, that's lingo used in libtiff, but BLOCK sounds good. Will change to that.

> Older versions of GDAL will remove the COG layout optimizations without any
> warning, no?

Yes (if you edit the file, not on read-only situations of course).
The updated validate_cloud_optimize_geotiff.py script checks if the advertized 
optimizations in the ghost header are valid.

> And will they also copy the ghost region of the file?

Not sure which copy operation you exactly meant, but if you do:

gdal_translate_of_gdal_2.4 cog_of_gdal_3.1.tif cog_of_gdal_2.4.tif
       -co COPY_SRC_OVERVIEWS=YES

the specific layout of GDAL 3.1 and the ghost regions will be lost.

> Is that
> why the existence or absence of the new metadata can't by itself indicate
> that a file is a valid COG?

Well, the definition of what is a valid COG is a bit fuzzy, and evolves over 
time. There are more or less optimized COGs.
The presence of the new metadata by itself doesn't mean that the file is fully 
optimized since non-COG aware TIFF editors could make changes, valid from a 
TIFF point of view, that would break the optimizations. For example if you 
edit a tile in a compressed COG, and that the new compressed size is larger 
than the previous one, TIFF writers will write the new compressed data at the 
end of the file, which will break the STRILE_ORDER=ROW_MAJOR optimization. 
GDAL 3.1 as a TIFF writer will know that it will break the optimization and 
then set KNOWN_INCOMPATIBLE_EDITION=YES. Other writers won't touch that ghost 
area at all.
Or even if the rewritten tile is smaller than the previous version and you can 
rewrite at the same location, a non COG aware writer will not update the 
leader and trailer bytes, which for imagery+mask breaks the optimized reading.
That's why I've added those leader and trailer bytes: with them, GDAL 3.1 can 
double check if the advertized optimizations are really valid.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com