[Landsat-pds] COG Format discussion

[Frank Warmerdam]

On Mon, Apr 16, 2018 at 12:14 PM, Vincent Sarago <vincent.sarago at gmail.com>
wrote:

> Sorry for hijacking this mailing list but as Landsat dataset is one of the
> biggest open COG dataset, discussion about the evolution of the format here
> made sense to us.
>
> Couple weeks ago we started a discussion internally about COGEO format.
> It’s great to see how many people are using and implementing COG right now
> (e.g planet, digitalglobe…) that’s say we think there is still place for
> improvement.
>
> Here are some of our ideas:
> - add webp compression to libtiff. Even if webp wasn’t a big success in
> term of adoption, the format is still a good option in comparison to JPEG
> and PNG.
>

Vincent, and other folks,

I'd be interested in whether webp is believed to have good enough
performance that it would be a substantial improvement over the Deflate
support already available in libtiff.  I'm also interested in how it
compares to the new zstd support (
https://github.com/OSGeo/gdal/commit/1c60366a193e67ee90856e1008e3c17cb8524f60#diff-ce45424050585add924746240ffc2761).
I'm willing to support new codecs in libtiff if they add significant value,
but I don't want to just add every compression format known.


> - Improve mask storing inside COG. For some technical reason, when you
> create a COG with internal masking, the mask is appended to the end of the
> IFD. Some of improvement could be to append mask TILE data just after the
> imagery tile data.
>

That is interesting.  Even, can you comment on what it would take to ensure
extra mask IFDs are located near their corresponding imagery as part of
GDAL and the implications for COG?

I assume you are suggesting a file with the "nodata" handled as a distinct
IFD like this (our internal serving format):

http://download.osgeo.org/gdal/data/gtiff/20160929_023611_0e0f_Browse.tif

...
TIFF Directory at offset 0x2070 (8304)
  Subfile Type: reduced-resolution image (1 = 0x1)
  Image Width: 857 Image Length: 438
  Tile Width: 128 Tile Length: 128
  Bits/Sample: 8
  Sample Format: unsigned integer
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  YCbCr Subsampling: 2, 2
  Samples/Pixel: 3
  Planar Configuration: single image plane
  Reference Black/White:
     0:     0   255
     1:   128   255
     2:   128   255
  JPEG Tables: (142 bytes)
...
TIFF Directory at offset 0x2a0e (10766)
  Subfile Type: reduced-resolution image/transparency mask (5 = 0x5)
  Image Width: 857 Image Length: 438
  Tile Width: 128 Tile Length: 128
  Bits/Sample: 1
  Sample Format: unsigned integer
  Compression Scheme: AdobeDeflate
  Photometric Interpretation: transparency mask
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Predictor: none 1 (0x1)
...

I consider this format very useful (lossy compression for the imagery, but
losslessly compress nodata masks) for some purposes and I'd be interested
in methods to optimize it for COG use even though it is pretty rare for
applications to properly support it.

Best regards,
Frank


>
> we’d love to hear story or comments from people about the format and how
> they see it moving in the future.
>
> _______________________________________________
> Landsat-pds mailing list
> Landsat-pds at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/landsat-pds
>



-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows |
and watch the world go round - Rush    | Geospatial Software Developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/landsat-pds/attachments/20180416/2c80000d/attachment.html>