<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
Hi,</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
I've encountered a problem using the JP2KAK driver for reading JP2 images.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
I use the following python code to read a specific overview from an image:</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><span style="margin: 0px; background-color: white">-------------------------------------------------------------</span></div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<div style="margin: 0px; font-weight: normal; font-size: 14px; line-height: 19px; font-family: "Droid Sans Mono", monospace, monospace, "Droid Sans Fallback"; background-color: white">
<div style="margin: 0px"><span style="margin: 0px">raster = gdal.Open(path, gdal.GA_ReadOnly)</span></div>
<div style="margin: 0px"><span style="margin: 0px">overview = raster.GetRasterBand(band_num).GetOverview(overview_num)</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px; color: green"># Read a 512X512 buffer at offset (0,0) from the given overview.</span></div>
<div style="margin: 0px"><span style="margin: 0px">buffer = overview.ReadAsArray(</span><span style="margin: 0px; color: rgb(9, 134, 88)">0</span><span style="margin: 0px">,</span><span style="margin: 0px; color: rgb(9, 134, 88)">0</span><span style="margin: 0px">,</span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">,</span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">,</span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">,</span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">)</span></div>
</div>
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<span style="margin: 0px; background-color: white"><br>
</span></div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
I would expect the driver to read exactly the 512X512 buffer I ask for, from the relevant overview.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
In reality, I see the following log:</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<span style="margin: 0px; background-color: white"><br>
</span></div>
<div style="margin: 0px; font-size: 15px; font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; color: rgb(32, 31, 30); background-color: rgb(255, 255, 255)">
<font color="black" face="Calibri,Arial,Helvetica,sans-serif"><span style="margin: 0px; font-size: 12pt">[2020-06-25T13:04:23.159Z] [DEBUG] [image_reader_logger] [1.1.0] Requested buffer for image: /vsis3/***.JP2 : Reads region (0, 0, 512, 512) into 512 X 512.
Overview: 3 : Overview<span> </span></span>dimensions:<span style="margin: 0px; font-size: 12pt"> (802X723)</span></font>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black">
[2020-06-25T13:04:23.177Z] [DEBUG] [image_reader_logger] [1.1.0] Reads band: 1 of overview: 3</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black">
JP2KAK: DirectRasterIO() for 0,0,8192,8192 -> 512x512 (no intermediate)<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black">
<span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black">
<span style="margin: 0px; background-color: white"><br>
</span></div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black">
<span style="margin: 0px; background-color: white">Also, I see VSICURL logs that the driver downloads 26 MB of data.</span></div>
<font color="black" face="Calibri,Arial,Helvetica,sans-serif"><span style="margin: 0px; font-size: 12pt">I've tested the behavior with a pyramidal tiff image, it works as expected. </span></font></div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
I've also written a small C program doing the same. with the same results:</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><span style="margin: 0px; background-color: white">-------------------------------------------------------------</span><br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<span style="margin: 0px; background-color: white"><br>
</span></div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<div style="margin: 0px; font-weight: normal; font-size: 14px; line-height: 19px; font-family: "Droid Sans Mono", monospace, monospace, "Droid Sans Fallback"; background-color: white">
<div style="margin: 0px"><span style="margin: 0px; color: blue">#include<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"gdal.h"</span></div>
<div style="margin: 0px"><span style="margin: 0px; color: blue">#include<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"cpl_conv.h"</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px; color: blue">int</span><span style="margin: 0px"> main()</span></div>
<div style="margin: 0px"><span style="margin: 0px">{</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"AWS_VIRTUAL_HOSTING"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"NO"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"AWS_HTTPS"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"NO"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"AWS_SECRET_ACCESS_KEY"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"***"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"AWS_ACCESS_KEY_ID"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"***"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"AWS_S3_ENDPOINT"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"localhost:9000"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"AWS_SSL_ENABLED"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"NO"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"CPL_DEBUG"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"ON"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"CPL_CURL_VERBOSE"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"NO"</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLSetConfigOption(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"CPL_VSIL_CURL_CHUNK_SIZE"</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(163, 21, 21)">"1000000"</span><span style="margin: 0px">);</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px"> </span></div>
<div style="margin: 0px"><span style="margin: 0px"> GDALDatasetH hDataset;</span></div>
<div style="margin: 0px"><span style="margin: 0px"> GDALAllRegister();</span></div>
<div style="margin: 0px"><span style="margin: 0px"> hDataset = GDALOpen(</span><span style="margin: 0px; color: rgb(163, 21, 21)">"/vsis3/***.JP2"</span><span style="margin: 0px">, GA_ReadOnly);</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px"> GDALRasterBandH hBand;</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px"> hBand = GDALGetRasterBand(hDataset,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">1</span><span style="margin: 0px">);</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px"> hBand = GDALGetOverview(hBand,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">3</span><span style="margin: 0px">);</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px"> u_int16_t *pafScanline;</span></div>
<div style="margin: 0px"><span style="margin: 0px"> pafScanline = (u_int16_t *)CPLMalloc(</span><span style="margin: 0px; color: blue">sizeof</span><span style="margin: 0px">(u_int16_t) *<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px"> *<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">);</span></div>
<div style="margin: 0px"><span style="margin: 0px"> CPLErr err = GDALRasterIO(hBand, GF_Read,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">0</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">0</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">,</span></div>
<div style="margin: 0px"><span style="margin: 0px"> pafScanline,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">512</span><span style="margin: 0px">,
GDT_UInt16,</span></div>
<div style="margin: 0px"><span style="margin: 0px"> <span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">0</span><span style="margin: 0px">,<span> </span></span><span style="margin: 0px; color: rgb(9, 134, 88)">0</span><span style="margin: 0px">);</span></div>
<br>
<div style="margin: 0px"><span style="margin: 0px"> GDALClose(hDataset);</span></div>
<div style="margin: 0px"><span style="margin: 0px">}</span></div>
<br>
<span style="margin: 0px; font-size: 16px; font-family: Calibri, Arial, Helvetica, sans-serif; background-color: white">-------------------------------------------------------------</span><span style="margin: 0px; font-size: 16px; font-family: Calibri, Arial, Helvetica, sans-serif; background-color: white">-------------------------------------------------------------</span><br>
<br>
</div>
The higher the overview, the bigger the request from S3, as if GDAL reads the best resolution and resamples the results...</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<ul>
<li>The input image is a 12 bit, 4 bands, pyramidal (5 internal pyramids) J2K image (image size: 12826X11565).<br>
</li><li>I've built GDAL 3.1.0 with kakadu vs8.0.4 (Also tried with 2.3.1 and kakadu vs7.7).</li></ul>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
Am I missing something?</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
Thanks in advance,</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black; background-color: rgb(255, 255, 255)">
Boris.</div>
<br>
</div>
</body>
</html>