[Landsat-pds] COG Format discussion

[Chris Holmes]

Two semi-related things:

- It could be good for us to set up a COG format list? Does anyone have
osgeo list creation powers? If not I can just make a google group. We've
been playing with -stats in COG's to give hints to tile services about
analytic data, would be good to discuss adding it as a cog
'recommendation', and indeed perhaps making more recommendations / possibly
making 'tiers' of COG for some of the more extreme optimizations.

- Anyone up for a cog 'birds of feather' session at foss4g? I'll try to
sign up tomorrow, curious if anyone has preferences for monday vs tuesday.



On Tue, Apr 17, 2018 at 12:54 AM, Even Rouault <even.rouault at spatialys.com>
wrote:

> > I'd be interested in whether webp is believed to have good enough
> > performance that it would be a substantial improvement over the Deflate
> > support already available in libtiff.  I'm also interested in how it
> > compares to the new zstd support (
> > https://github.com/OSGeo/gdal/commit/1c60366a193e67ee90856e1008e3c1
> 7cb8524f6
> > 0#diff-ce45424050585add924746240ffc2761). I'm willing to support new
> codecs
> > in libtiff if they add significant value, but I don't want to just add
> > every compression format known.
>
> You can only compare webp vs deflate or zstd for its lossless profile.
> The WebP website provides this comparison against PNG:
> https://developers.google.com/speed/webp/docs/webp_lossless_alpha_study
> so claiming a 42% size improvement over PNG
>
> For lossy support, if you believe Mozilla (pushing for MozJpeg)
> https://research.mozilla.org/2014/07/15/mozilla-advances-
> jpeg-encoding-with-mozjpeg-2-0/
> rather than Google
> https://developers.google.com/speed/webp/docs/webp_study
> "We consider this study to be inconclusive when it comes to the question
> of whether
> "WebP and/or JPEG XR outperform JPEG by any significant margin"
>
> One potential advantage of webp is that lossy webp for imagery with
> lossless alpha would
> be naturally supported (in comparison to the second point below).
> One drawback of webp is that it is really RGB[A] only (would likely be
> unwise to use
> it for 3 or 4 band image with other photometric interpretations)
>
> >
> > > - Improve mask storing inside COG. For some technical reason, when you
> > > create a COG with internal masking, the mask is appended to the end of
> the
> > > IFD. Some of improvement could be to append mask TILE data just after
> the
> > > imagery tile data.
>
> >
> > I assume you are suggesting a file with the "nodata" handled as a
> distinct
> > IFD like this (our internal serving format):
>
> That was I understood from previous discussion with Vincent's team.
>
> > That is interesting.  Even, can you comment on what it would take to
> ensure
> > extra mask IFDs are located near their corresponding imagery as part of
> > GDAL and the implications for COG?
>
> Such a layout of blocks would apply only for Planar Configuration==single
> image plane
> That's certainly doable by the GDAL GTiff driver, but would require the
> copying
> of imagery to be done in a custom fashion, to properly interleave imagery
> blocks with
> mask blocks, instead of using GDALDatasetCopyWholeRaster() (for imagery)
> and
> GDALRasterBandCopyWholeRaster() (for masks). Actually, implementation
> wise,
> that could be a new option of GDALDatasetCopyWholeRaster
> (INTERLEAVE_MASK=YES).
> For very large images (dimensions > 100,000 pixels) where the size of the
> TileOffsets and TileByteCounts tags is big this constant back and forth
> between IFD
> could be rather costly, as they are reloaded/flushed each time you change
> the active
> IFD in libtiff.
>
> The COG definition would have to be updated for that use case (probably as
> an
> allowed extra optimization, rather than forcing people to use it), and the
> validation
> script as well.
>
> On the GDAL read side, the GTiff driver would also have to be updated to
> detect this
> layout and when reading a block of imagery, it should issue a GET range
> request that
> is big enough to fetch the imagery block and its mask block at once (the
> base logic to
> optimize GET requests for a given IRasterIO() request is already in place)
>
> Even
>
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> Landsat-pds mailing list
> Landsat-pds at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/landsat-pds
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/landsat-pds/attachments/20180513/5aec61ae/attachment.html>