[gdal-dev] errors using IAM instance profile auth in s3

[michael.smith.erdc at gmail.com]

I’m seeing that it’s getting a 206 response code, so wouldn’t that indicate auth is working?

 gdalinfo /vsis3/grid-dev-publiclidar/estonia/dtm/estonia_dtm_5m.tif
HTTP: Fetch(http://169.254.169.254/latest/api/token)
HTTP: libcurl/7.86.0 OpenSSL/3.0.7 zlib/1.2.13 libssh2/1.10.0 nghttp2/1.47.0
HTTP: These HTTP headers were set: X-aws-ec2-metadata-token-ttl-seconds: 10
HTTP: Fetch(http://169.254.169.254/latest/meta-data/iam/security-credentials/)
HTTP: Fetch(http://169.254.169.254/latest/meta-data/iam/security-credentials/iam-grid-s3)
AWS: Storing AIM credentials until 2022-11-19T20:42:58Z
S3: Downloading 0-16383 (https://grid-dev-publiclidar.s3.amazonaws.com/estonia/dtm/estonia_dtm_5m.tif)...
S3: Got response_code=206
gdalinfo failed - unable to open '/vsis3/grid-dev-publiclidar/estonia/dtm/estonia_dtm_5m.tif'.


Mike



> On Nov 19, 2022, at 9:26 AM, Even Rouault <even.rouault at spatialys.com> wrote:
> 
> Hi Mike,
> 
> could you send the output of
> 
> curl http://169.254.169.254/latest/meta-data/iam/security-credentials/iam-grid-s3
> 
> Slightly redacted of course, but with the exact formatting. This part of thee code currently uses a "simple JSON parser" (https://github.com/OSGeo/gdal/blob/c61d116a469821b769630a112dee7f1a61fed885/port/cpl_aws.cpp#L554), which is actually just a non JSON-aware string tokenizer, and I suspect it could be defeated by a new formatting of S3 or something specific to your credentials.
> 
> It could also be that something unhandled by that parser appears inside quoted strings, like an escaped double quote or some other JSON escaped character (like an escaped forward slash \/ )
> 
> If that was the case we should likely switch to proper JSON deserialization (that part of the code must predate libjson-c being a build requirement of GDAL).
> 
> Even
> 
> 
> -- 
> http://www.spatialys.com
> My software is free, but my time generally not.
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20221119/4cd70e93/attachment.htm>