[COG] Fwd: Cloud optimized GeoTIFF configuration specifics [SEC=UNOFFICIAL]

[Even Rouault]

> Each "band" might
> have different bit depth and possibly slightly different geo-registration.
> Yet having them stored together in one file has certain advantages. So let
> me re-phrase the question:
> 
> Is it possible to have several independent full-resolution images in one
> COG and also have overviews for each?
> 
> I see that overviews are marked with TIFF tag:
>   `Subfile Type: reduced-resolution image (1 = 0x1)`
> 
> But that assumes that there is only one main resolution image (possibly
> with multiple samples per pixel).

Interesting question !
Clearly the IFD mechanism of the TIFF 6.0 baseline specification was not meant to
deal with several full-resolution images, each of them having overviews. It is normally
either a sequence of full-resolution images (multiple pages of a scanned documents),
or a full-resolution image + its overviews

In theory, you could imagine putting several IFDs like this chaining:

Image1FullResolution (NewSubFileType = 2 =single page of a multi-page image)
Image1OverviewFactor2 (NewSubFileType = 1 =  reduced-resolution)
Image1OverviewFactor4 (NewSubFileType = 1)
Image2FullResolution  (NewSubFileType = 2)
Image2OverviewFactor2 (NewSubFileType = 1)
Image2OverviewFactor4 (NewSubFileType = 1)

But tiff6.pdf mentions "If multiple subfiles are written, the first one must be the
full-resolution image. Subsequent images, such as reduced-resolution images, may be
in any order in the TIFF file. If a reader wants to use such images, it must scan the
corresponding IFDs before deciding how to proceed"

So a TIFF reader could be confused by my above layout. GDAL cannot currently write such
images (although shouldn't be a bit dig), and on the read side, I guess it would be confused
to figure out which overviews belong to which full resolution image (although that could be
improved to make sure that overviews aren't looked beyond a boundary defined by a NewSubFileType = 2 IFD)
On the read side, multiples pages  are modelled as GDAL subdatasets, that is to
say you must mention the IFD index number you want to open with a syntax like
GTIFF_DIR:1:my.tif

A cleaner solution, that was discussed some time ago on the libtiff mailing list,
https://www.asmail.be/msg0054912161.html, would be to use the TIFF Tree / SubIFD mechanism
that was added per TIFF Tech Note 1:
https://www.awaresystems.be/imaging/tiff/specification/TIFFPM6.pdf
where you can define parent and child IFD

So the layout would become:

Image1FullResolution (NewSubFileType = 2 =single page of a multi-page image, SubIFDS = offset of the 2 below overviews, NextIFD = offset of Image2FullResolution)
	Image1OverviewFactor2 (NewSubFileType = 1 =  reduced-resolution, NextIFD = offset of Image1OverviewFactor4)
	Image1OverviewFactor4 (NewSubFileType = 1, NextIFD = 0)
Image2FullResolution  (NewSubFileType = 2, SubIFDS = offset of the 2 below overviews, NextIFD = 0)
	Image2OverviewFactor2 (NewSubFileType = 1, NextIFD = offset of Image2OverviewFactor4)
	Image2OverviewFactor4 (NewSubFileType = 1, NextIFD = 0)

There is absolutely no support in GDAL on the read and write side for that currently. Shouldn't
be a big deal on the read-side. On the write-side, I'd be more concerned about the state of libtiff
itself regarding this (there is code, but I never exercised it). For example, I've not mentionned transparency
masks that are modelled by separate IFD if you cannot use an alpha band. With SubIFD mechanism,
you could potentially imagine to create a second level of SubIFD to attach the mask to its overview, but libtiff
has explicit code on the write side to prevent that. That said, you would not necessarily be obliged to
do that, and use the current way of putting overviews and masks in the same chain, and the reader
figures out which mask belong to which overview by comparing the image dimensions (GDAL does that
properly for the top-level IFD chaining I presented initially).

The libtiff disccussion thread I mentionned also pointed that few TIFF readers (beyond Photoshop)
are able to deal with SubIFDs.

So, beyond having a single file, the advantages of putting all those "bands" in the same TIFF files aren't
obvious.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com