<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">Correct, not a public bucket, which is why the IAM credentials are needed. If I set them manually, it all works fine.<div><br></div><div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">[ u02]$ export AWS_ACCESS_KEY_ID=xxxxx</span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;"> [ u02]$ gdalinfo /vsis3/grid-dev-publiclidar/estonia/dtm/estonia_dtm_5m.tif</span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;"> HTTP: libcurl/7.86.0 OpenSSL/3.0.7 zlib/1.2.13 libssh2/1.10.0 nghttp2/1.47.0 </span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;"> S3: Downloading 0-16383 (https://grid-dev-publiclidar.s3.amazonaws.com/estonia/dtm/estonia_dtm_5m.tif)...</span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;"> S3: Got response_code=206 </span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">GDAL: GDALOpen(/vsis3/grid-dev-publiclidar/estonia/dtm/estonia_dtm_5m.tif, this=0x5645e67d7f30) succeeds as GTiff. </span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">Driver: GTiff/GeoTIFF </span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">GDAL: GDALDefaultOverviews::OverviewScan()</span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">Files: /vsis3/grid-dev-publiclidar/estonia/dtm/estonia_dtm_5m.tif</span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">Size is 75000, 52000</span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">Coordinate System is:</span></font></div><div><font face="Courier New"><span style="font-style: normal; font-size: 15px;">PROJCRS["Estonian Coordinate System of 1997",</span></font></div><div>….</div><br><div dir="ltr">Mike</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><blockquote type="cite">On Nov 19, 2022, at 9:48 AM, Even Rouault <even.rouault@spatialys.com> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<p>Yes, a 206 response code means success here as we are requesting
only bytes 0-16383. So maybe the file is not a valid TIFF ?</p>
<p>( "grid-dev-publiclidar" must not be so public I guess, because
when trying with my credentials, I get a Access Denied)</p>
<p></p>
<div class="moz-cite-prefix">Le 19/11/2022 à 15:40,
<a class="moz-txt-link-abbreviated" href="mailto:michael.smith.erdc@gmail.com">michael.smith.erdc@gmail.com</a> a écrit :<br>
</div>
<blockquote type="cite" cite="mid:8F735EF0-188A-4F70-A9C7-6B3CD9ACD718@gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
I’m seeing that it’s getting a 206 response code, so wouldn’t that
indicate auth is working?
<div><br>
</div>
<div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;"> gdalinfo
/vsis3/grid-dev-publiclidar/estonia/dtm/estonia_dtm_5m.tif</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">HTTP:
Fetch(<a class="moz-txt-link-freetext" href="http://169.254.169.254/latest/api/token">http://169.254.169.254/latest/api/token</a>)</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">HTTP: libcurl/7.86.0 OpenSSL/3.0.7 zlib/1.2.13
libssh2/1.10.0 nghttp2/1.47.0</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">HTTP: These HTTP headers were set:
X-aws-ec2-metadata-token-ttl-seconds: 10</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">HTTP:
Fetch(<a class="moz-txt-link-freetext" href="http://169.254.169.254/latest/meta-data/iam/security-credentials/">http://169.254.169.254/latest/meta-data/iam/security-credentials/</a>)</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">HTTP:
Fetch(<a class="moz-txt-link-freetext" href="http://169.254.169.254/latest/meta-data/iam/security-credentials/iam-grid-s3">http://169.254.169.254/latest/meta-data/iam/security-credentials/iam-grid-s3</a>)</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">AWS: Storing AIM credentials until
2022-11-19T20:42:58Z</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">S3: Downloading 0-16383
(<a class="moz-txt-link-freetext" href="https://grid-dev-publiclidar.s3.amazonaws.com/estonia/dtm/estonia_dtm_5m.tif">https://grid-dev-publiclidar.s3.amazonaws.com/estonia/dtm/estonia_dtm_5m.tif</a>)...</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">S3: Got response_code=206</span></font></div>
<div><font size="3" face="Courier New"><span style="font-style:
normal;">gdalinfo failed - unable to open
'/vsis3/grid-dev-publiclidar/estonia/dtm/estonia_dtm_5m.tif'.</span></font></div>
<div><br>
</div>
<br>
<div dir="ltr">Mike</div>
<div dir="ltr"><br>
</div>
<div dir="ltr"><br>
</div>
<div dir="ltr"><br>
<blockquote type="cite">On Nov 19, 2022, at 9:26 AM, Even
Rouault <a class="moz-txt-link-rfc2396E" href="mailto:even.rouault@spatialys.com"><even.rouault@spatialys.com></a> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr"><span>Hi Mike,</span><br>
<span></span><br>
<span>could you send the output of</span><br>
<span></span><br>
<span>curl
<a class="moz-txt-link-freetext" href="http://169.254.169.254/latest/meta-data/iam/security-credentials/iam-grid-s3">http://169.254.169.254/latest/meta-data/iam/security-credentials/iam-grid-s3</a></span><br>
<span></span><br>
<span>Slightly redacted of course, but with the exact
formatting. This part of thee code currently uses a
"simple JSON parser"
(<a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/blob/c61d116a469821b769630a112dee7f1a61fed885/port/cpl_aws.cpp#L554">https://github.com/OSGeo/gdal/blob/c61d116a469821b769630a112dee7f1a61fed885/port/cpl_aws.cpp#L554</a>),
which is actually just a non JSON-aware string tokenizer,
and I suspect it could be defeated by a new formatting of
S3 or something specific to your credentials.</span><br>
<span></span><br>
<span>It could also be that something unhandled by that
parser appears inside quoted strings, like an escaped
double quote or some other JSON escaped character (like an
escaped forward slash \/ )</span><br>
<span></span><br>
<span>If that was the case we should likely switch to proper
JSON deserialization (that part of the code must predate
libjson-c being a build requirement of GDAL).</span><br>
<span></span><br>
<span>Even</span><br>
<span></span><br>
<span></span><br>
<span>-- </span><br>
<span><a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a></span><br>
<span>My software is free, but my time generally not.</span><br>
<span></span><br>
</div>
</blockquote>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div></blockquote></div></body></html>