[COG] Mixed Raw and Overview data compression ?

[Vincent Sarago]

> Le 16 sept. 2018 à 21:26, Kirill Kouzoubov <kirill888 at gmail.com> a écrit :
> 
> This would be very useful, reducing storage costs while simultaneously allowing faster preview functionality.


> One issue to consider is `dtype` support/conversion. My understanding is that JPEG is best supported at 8 bit, and can not go beyond 12, while many sensors have more than 8 bits worth of data per band, and even more than 12.

Yeah absolutely, so this would be only for 8bit/12bit RGB dataset (noting that enabling different photogrammetric configuration could save more space too for YCBCR+JPEG). 

>  So one would need bit-depth reduction as well. To complicated things further, one might have 14-bit data stored in 16-bit image, so bit-depth reduction is not a trivial operation.

Yeah I don’t want to overthink this, and create complex dataset (data difference between overview and high res, just thinking about compression) 

> Also `nodata` comes into play as well.

I think using internal bitmask is a better solution than nodata IMO but that’s another subject :D . 

Vincent 

> 
> Kirill 
> 
> On Mon, Sep 17, 2018 at 10:39 AM Vincent Sarago <vincent.sarago at gmail.com <mailto:vincent.sarago at gmail.com>> wrote:
> What do I mean about `mixed` compression ? 
> JPEG (or WEBP) compression are great for storage saving (and also for small http request) but they are `bad` because they are not lossless (WEBP can be set to be lossless but compression is then not as great) so my idea is to package both lossless and non-lossless in one COG. e.g keeping high resolution data to a ZSTD/DEFLATE compression but the overview in JPEG.
> 
> Why ? 
> For web application we mostly want fast tile serving while for processing we will mostly want to read full resolution lossless data, a file with mixed compression could make both world happy ? 
> 
> The gain won’t be astonishing but might be worth a try ? 
> 
> Vincent 
> _______________________________________________
> COG mailing list
> COG at lists.osgeo.org <mailto:COG at lists.osgeo.org>
> https://lists.osgeo.org/mailman/listinfo/cog <https://lists.osgeo.org/mailman/listinfo/cog>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/cog/attachments/20180916/7b97cf5a/attachment.html>