<div dir="ltr">(working from pretty close to head...)<div><br></div><div>A follow up question on how to create cogeo files...<div><br></div><div>I appear to need to do additional work to set BLOCK_OFFSET_?_?. How do I go about this?</div><div><br></div><div>I'm trying to create a cogeo without gdal_translate in java by using create and createcopy like this:</div><div><br></div><div><a href="https://gist.github.com/schwehr/39680bec7fd8e3e3f840122ea3bafc65">https://gist.github.com/schwehr/39680bec7fd8e3e3f840122ea3bafc65</a><br></div><div><br></div><div>However, I get this failure when checking the result:</div><div><br></div><div><div>validate_cloud_optimized_geotiff.py cogeo.tif </div><div>Traceback (most recent call last):</div><div> File "validate_cloud_optimized_geotiff.py", line 144, in validate<br></div><div> data_offset = int(main_band.GetMetadataItem('BLOCK_OFFSET_0_0', 'TIFF'))</div><div>TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType</div></div><div><br></div><div>If I use gdal_translate on the result form the java, it validates.</div><div><br></div><div>gdal_translate -co COMPRESS=LZW -co TILED=YES -co COPY_SRC_OVERVIEWS=YES cogeo.tif cogeo2.tif<br></div><div><div>validate_cloud_optimized_geotiff.py cogeo2.tif</div><div>cogeo2.tif is a valid cloud optimized GeoTIFF</div></div><div><br></div><div>gdalinfo -mdd all -listmdd -mm cogeo.tif > <a href="http://cogeo.tif.info">cogeo.tif.info</a><br></div><div>gdalinfo -mdd all -listmdd -mm cogeo2.tif > <a href="http://cogeo2.tif.info">cogeo2.tif.info</a><br></div><div><br></div><div><div>diff -u cogeo{,2}.<a href="http://tif.info">tif.info</a></div><div>--- <a href="http://cogeo.tif.info">cogeo.tif.info</a><span style="white-space:pre"> </span>2018-02-14 10:54:27.050423325 -0800</div><div>+++ <a href="http://cogeo2.tif.info">cogeo2.tif.info</a><span style="white-space:pre"> </span>2018-02-14 10:54:41.862227514 -0800</div><div>@@ -1,5 +1,5 @@</div><div> Driver: GTiff/GeoTIFF</div><div>-Files: cogeo.tif</div><div>+Files: cogeo2.tif</div><div> Size is 743, 372</div><div> Coordinate System is:</div><div> GEOGCS["WGS 84",</div><div>@@ -19,8 +19,8 @@</div><div> Metadata:</div><div> AREA_OR_POINT=Area</div><div> Metadata (DERIVED_SUBDATASETS):</div><div>- DERIVED_SUBDATASET_1_NAME=DERIVED_SUBDATASET:LOGAMPLITUDE:cogeo.tif</div><div>- DERIVED_SUBDATASET_1_DESC=log10 of amplitude of input bands from cogeo.tif</div><div>+ DERIVED_SUBDATASET_1_NAME=DERIVED_SUBDATASET:LOGAMPLITUDE:cogeo2.tif</div><div>+ DERIVED_SUBDATASET_1_DESC=log10 of amplitude of input bands from cogeo2.tif</div><div> Image Structure Metadata:</div><div> COMPRESSION=LZW</div><div> INTERLEAVE=BAND</div></div><div><br></div><div><div>gdalinfo -mdd all -listmdd -mm cogeo.tif</div><div>Driver: GTiff/GeoTIFF</div><div>Files: cogeo.tif</div><div>Size is 743, 372</div><div>Coordinate System is:</div><div>GEOGCS["WGS 84",</div><div> DATUM["WGS_1984",</div><div> SPHEROID["WGS 84",6378137,298.257223563,</div><div> AUTHORITY["EPSG","7030"]],</div><div> AUTHORITY["EPSG","6326"]],</div><div> PRIMEM["Greenwich",0],</div><div> UNIT["degree",0.0174532925199433],</div><div> AUTHORITY["EPSG","4326"]]</div><div>Origin = (116.349975262217256,39.975075059082918)</div><div>Pixel Size = (0.000134747292618,-0.000134747292618)</div><div>Metadata domains:</div><div> IMAGE_STRUCTURE</div><div> (default)</div><div> DERIVED_SUBDATASETS</div><div>Metadata:</div><div> AREA_OR_POINT=Area</div><div>Metadata (DERIVED_SUBDATASETS):</div><div> DERIVED_SUBDATASET_1_NAME=DERIVED_SUBDATASET:LOGAMPLITUDE:cogeo.tif</div><div> DERIVED_SUBDATASET_1_DESC=log10 of amplitude of input bands from cogeo.tif</div><div>Image Structure Metadata:</div><div> COMPRESSION=LZW</div><div> INTERLEAVE=BAND</div><div>Corner Coordinates:</div><div>Upper Left ( 116.3499753, 39.9750751) (116d20'59.91"E, 39d58'30.27"N)</div><div>Lower Left ( 116.3499753, 39.9249491) (116d20'59.91"E, 39d55'29.82"N)</div><div>Upper Right ( 116.4500925, 39.9750751) (116d27' 0.33"E, 39d58'30.27"N)</div><div>Lower Right ( 116.4500925, 39.9249491) (116d27' 0.33"E, 39d55'29.82"N)</div><div>Center ( 116.4000339, 39.9500121) (116d24' 0.12"E, 39d57' 0.04"N)</div><div>Band 1 Block=256x256 Type=Float32, ColorInterp=Gray</div><div> Description = B5</div><div> Computed Min/Max=0.000,0.000</div><div> Overviews: 372x186, 186x93, 93x47, 47x24, 24x12</div></div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Dec 10, 2017 at 5:34 PM, Kurt Schwehr <span dir="ltr"><<a href="mailto:schwehr@gmail.com" target="_blank">schwehr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks Even for the feedback. There have be a few offline discussions going on and Even added some notes to the document on Trac:<div><br></div><div><a href="https://www.google.com/url?sa=D&q=https%3A%2F%2Ftrac.osgeo.org%2Fgdal%2Fwiki%2FCloudOptimizedGeoTIFF%3Faction%3Ddiff%26version%3D11" style="color:rgb(85,26,139);font-family:monospace;font-size:13px;white-space:pre-wrap" target="_blank">https://trac.osgeo.org/gdal/<wbr>wiki/CloudOptimizedGeoTIFF?<wbr>action=diff&version=11</a><br></div><div><br></div><div>This stems from me switching Earth Engine from LZW to DEFLATE when exporting GeoTIFFs (and I added tiling). We have a report from a user that ENVI 5.4 (and 5.1) Classic can't read the resulting images but QGIS & ArcGIS can. I'm reverting exports back to LZW compression.</div></div><div class="gmail_extra"><div><div class="gmail-h5"><br><div class="gmail_quote">On Wed, Nov 15, 2017 at 6:02 AM, Even Rouault <span dir="ltr"><<a href="mailto:even.rouault@spatialys.com" target="_blank">even.rouault@spatialys.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div style="font-family:"Sans Serif";font-size:9pt;font-weight:400;font-style:normal"><span>
<p style="margin:0px;text-indent:0px">On mardi 14 novembre 2017 14:20:58 CET Kurt Schwehr wrote:</p>
<p style="margin:0px;text-indent:0px">> Hi Even,</p>
<p style="margin:0px;text-indent:0px">> </p>
<p style="margin:0px;text-indent:0px">> I have some follow up questions on Cloud Optimized GeoTIFFs:</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">The main constraint of C.O.G is that all the IFD definitions are at the beginning of the file, to avoid seeking at various points in it. Other parameters are pretty much unspecified.</p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">> </p>
<p style="margin:0px;text-indent:0px">> * Is there a preferred/better INTERLEAVE if there is more than one band?</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">Depends on access patterns. If as soon as you process one pixel you need to access the value for all bands, then INTERLEAVE=PIXEL is better, and it will result in smaller sizes of StripOffsets/TileOffsets and StripByteCount/TileByteCount arrays</p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">> * Is there a preferred tile blocksize? You have 512 in your examples. Are</p>
<p style="margin:0px;text-indent:0px">> there any major trade offs between using 128, 256, 512, or 1024 for x and y</p>
<p style="margin:0px;text-indent:0px">> block sizes?</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">Too small blocksizes will result in larger ...Offsets and ...ByteCount arrays. </p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">> * Should tiles be square? Does it matter?</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">No</p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">> * Is it better to skip tiling for small images? If so, at what threshold</p>
<p style="margin:0px;text-indent:0px">> do you think the switch should happen? 1024?</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">I'm not sure if that has an importance. But it is not wise to have an image whose one dimension is larger than the corresponding block dimension (as blocks are not truncated)</p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">> * Is DEFLATE preferred for compression type over LZW for lossless</p>
<p style="margin:0px;text-indent:0px">> compression?</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">Unspecified. DEFLATE is more CPU intensive, but if network times are the limiting factor, it is worth as more eficient</p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">> * If the writer isn't constrained by compute power, are there preferred</p>
<p style="margin:0px;text-indent:0px">> ZLEVEL and PREDICTOR values? Is there a time cost for decompressing</p>
<p style="margin:0px;text-indent:0px">> ZLEVEL=9 over 1?</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">PREDICTOR has neglectable CPU inflence (just a add/diff on integer values), but will not always result in smaller file sizes. Depends on the dataset</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">If I trust <a href="https://github.com/inikep/lzbench" target="_blank">https://github.com/inikep/lzbe<wbr>nch</a> , the time cost for decompression for Deflate/Zlib doesn't seem to vary much with ZLEVEL. So the higher the better.</p>
<p style="margin:0px;text-indent:0px">I don't know for LZW.</p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">> </p>
<p style="margin:0px;text-indent:0px">> I'm a little confused by this code from validate_cloud_optimized_geoti<wbr>ff.py:</p>
<p style="margin:0px;text-indent:0px">> </p>
<p style="margin:0px;text-indent:0px">> if main_band.XSize >= 512 or main_band.YSize >= 512:</p>
<p style="margin:0px;text-indent:0px">> if check_tiled:</p>
<p style="margin:0px;text-indent:0px">> block_size = main_band.GetBlockSize()</p>
<p style="margin:0px;text-indent:0px">> if block_size[0] == main_band.XSize and block_size[0] > 1024:</p>
<p style="margin:0px;text-indent:0px">> errors += ["The file is greater than 512xH or Wx512," +</p>
<p style="margin:0px;text-indent:0px">> "but is not tiled"]</p>
<p style="margin:0px;text-indent:0px">> </p>
<p style="margin:0px;text-indent:0px">> Will the above correctly fail an image that is (say) 256x2048 if it is not</p>
<p style="margin:0px;text-indent:0px">> tiled?</p>
<p style="margin:0px;text-indent:0px"> </p>
</span><p style="margin:0px;text-indent:0px">No, it will pass this test. Since in that case block_size[0] == xsize == 256.</p>
<p style="margin:0px;text-indent:0px">But for such a narrow image, it should probably warn if it is not tiled, as the number of strips, if letting to default strip height, will be larger than really necessary.</p><span>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">Even</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">-- </p>
<p style="margin:0px;text-indent:0px">Spatialys - Geospatial professional services</p>
<p style="margin:0px;text-indent:0px"><a href="http://www.spatialys.com" target="_blank">http://www.spatialys.com</a></p></span></div></blockquote></div><br><br clear="all"><div><br></div></div></div><span class="gmail-HOEnZb"><font color="#888888">-- <br><div class="gmail-m_8261167218496448625gmail_signature">--<div><a href="http://schwehr.org" target="_blank">http://schwehr.org</a></div></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">--<div><a href="http://schwehr.org" target="_blank">http://schwehr.org</a></div></div>
</div></div></div>