<div dir="ltr">Hi Even,<div><br></div><div>I have some follow up questions on Cloud Optimized GeoTIFFs:</div><div><br></div><div>* Is there a preferred/better INTERLEAVE if there is more than one band?</div><div>* Is there a preferred tile blocksize? You have 512 in your examples. Are there any major trade offs between using 128, 256, 512, or 1024 for x and y block sizes?</div><div>* Should tiles be square? Does it matter?</div><div>* Is it better to skip tiling for small images? If so, at what threshold do you think the switch should happen? 1024?</div><div>* Is DEFLATE preferred for compression type over LZW for lossless compression?</div><div>* If the writer isn't constrained by compute power, are there preferred ZLEVEL and PREDICTOR values? Is there a time cost for decompressing ZLEVEL=9 over 1?</div><div><br></div><div>I'm a little confused by this code from validate_cloud_optimized_geotiff.py:</div><div><br></div><div><div> if main_band.XSize >= 512 or main_band.YSize >= 512:</div><div> if check_tiled:</div><div> block_size = main_band.GetBlockSize()</div><div> if block_size[0] == main_band.XSize and block_size[0] > 1024:</div><div> errors += ["The file is greater than 512xH or Wx512," +</div><div> "but is not tiled"]</div></div><div><br></div><div>Will the above correctly fail an image that is (say) 256x2048 if it is not tiled?</div><div><br></div><div>Thanks!</div><div>-kurt</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 10, 2017 at 1:53 PM, Even Rouault <span dir="ltr"><<a href="mailto:even.rouault@spatialys.com" target="_blank">even.rouault@spatialys.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div style="font-family:'Sans Serif';font-size:9pt;font-weight:400;font-style:normal"><div><div class="h5">
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">On mercredi 10 mai 2017 14:41:44 CEST Peter Schmitt wrote:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> Hi,</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> I ran into something confusing when using gdal-2.2.0 to generate Cloud</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> Optimized GeoTIFFs following the instructions at:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> <a href="https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF" target="_blank">https://trac.osgeo.org/gdal/<wbr>wiki/CloudOptimizedGeoTIFF</a></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> Here's what I did:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> 1. Fetch an image:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> env CPL_VSIL_CURL_ALLOWED_<wbr>EXTENSIONS=tif GDAL_DISABLE_READDIR_ON_OPEN=<wbr>YES</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> VSI_CACHE=TRUE gdal_translate</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> <a href="https://github.com/mapbox/rasterio/raw/master/tests/data/RGB.byte.tif" target="_blank">https://github.com/mapbox/<wbr>rasterio/raw/master/tests/<wbr>data/RGB.byte.tif</a></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> rgb_byte_jpg.tif -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> Not a cloud-optimized GeoTIFF as expected:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> validate_cloud_optimized_<wbr>geotiff.py rgb_byte_jpg.tif</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> rgb_byte_jpg.tif is NOT a valid cloud optimized GeoTIFF : The file should</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> have overviews</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> 2. Add internal overviews:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> gdaladdo -r average rgb_byte_jpg.tif 2 4 8 16 32</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> validate_cloud_optimized_<wbr>geotiff.py rgb_byte_jpg.tif</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> rgb_byte_jpg.tif is NOT a valid cloud optimized GeoTIFF : The offset of the</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> first block of overview of index 3 should be after the one of the overview</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> of index 4</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> 3. Translate the image again copying source overviews added above.</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> gdal_translate rgb_byte_jpg.tif rgb_byte_jpg_trans.tif -co TILED=YES -co</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co COPY_SRC_OVERVIEWS=YES</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> Now the image _is_ cloud-optimized.</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> validate_cloud_optimized_<wbr>geotiff.py rgb_byte_jpg_trans.tif</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> rgb_byte_jpg_trans.tif is a valid cloud optimized GeoTIFF</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> I expected the image to be "cloud optimized" at step 2. Why do I need an</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> additional translate?</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
</div></div><p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Peter,</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Because gdaladdo will add the overview IFD at the end of the file, whereas in the definition of cloud optimized GeoTIFF, they must be all at the beginning of the file so as to be efficiently fetchable. And that can only be done with gdal_translate -co COPY_SRC_OVERVIEWS=YES</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">To avoid quality loss you should only use JPEG compression for the final gdal_translate stage.</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Even</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">-- </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Spatialys - Geospatial professional services</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><a href="http://www.spatialys.com" target="_blank">http://www.spatialys.com</a></p></div><br>______________________________<wbr>_________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/<wbr>mailman/listinfo/gdal-dev</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">--<div><a href="http://schwehr.org" target="_blank">http://schwehr.org</a></div></div>
</div>