[gdal-dev] GDAL vsicurl with query string

[JDzialo John]

Hi Even

Yeah it was just a faked URL that was meant to show the URL format I was trying to use to vsicurl a tif file from AWS S3.

OK so these URLs are time restricted and will terminate after an hour, the maximum I ca set it at.

This is just our test bucket and test user.

If you can try this link...

https://parcel-test.s3.amazonaws.com/test/key/value/string/object.jpg?Signature=zyN053V02Y41VSKA6Yc1%2Bx0fz%2BU%3D&Expires=1408987568&AWSAccessKeyId=AKIAJF3TTVKIEFQXGZ3Q

It will terminate around 1:30pm.



-----Original Message-----
From: Even Rouault [mailto:even.rouault at spatialys.com] 
Sent: Friday, August 22, 2014 6:39 PM
To: gdal-dev at lists.osgeo.org
Cc: JDzialo John
Subject: Re: [gdal-dev] GDAL vsicurl with query string

Le samedi 23 août 2014 00:13:21, JDzialo John a écrit :
> Hi All,
> 
> This is my first post so forgive my noobiness...
> 
> We are running a cropping tool that used to access data from a local 
> hard drive and crop images.
> 
> Now we want to use s3 as our storage in aws and want to access files 
> through http requests.  Great!  Enter vsicurl.
> 
> So when we open our s3 bucket to the world we can easily access our 
> tif images from s3 with a straight url.  For example...
> 
> https://test-bucket.s3.amazonaws.com/test/key/value/string/object.tif
> 
> This works well using the gdalinfo vsicurl cmd...
> 
> gdalinfo
> /vsicurl/https://test-bucket.s3.amazonaws.com/test/key/value/string/ob
> ject
> .tif
> 
> However in s3 it is not secure to allow access to everyone on the 
> internet and would rather use a presigned URL that uses a query string 
> to authenticate access to the object.
> 
> When sending a request with a query string vsicurl and gdal appear to 
> breakdown and I'm not sure why.
> 
> gdalinfo
> /vsicurl/https://test-bucket.s3.amazonaws.com/test/key/value/string/ob
> ject 
> .tif?Signature=sdlfkj88345r9&Expires=12300000&AWSAccessKeyID=AKIALKSDJ
> FOILI
> KJFVOSIDF

I tried the URL in a browser and got a AccessDenied. Was it supposed to work or something faked to just give a sense of the kind of URL ? A working example would be helpful.

> 
> Fails saying it's unable to open the file.
> 
> Is there a specific way to pass the query string to the VSIL API like 
> when using curl with the -d switch?  Any way to run vsicurl with a 
> query string or something I'm missing?
> 
> Thanks for any advice you can throw my way!
> 
> 
> 
> John Dzialo | Linux System Administrator Direct 203.783.8163 | Main 
> 800.352.0050
> 
> Environmental Data Resources, Inc.
> 440 Wheelers Farms Road, Milford, CT 06461 
> www.edrnet.com<http://www.edrnet.com/> | 
> commonground.edrnet.com<http://commonground.edrnet.com/>
> 
> [Description: Description: Description: Description: Description:
> EDR_logo4color_EDR_only_80px2]

--
Spatialys - Geospatial professional services http://www.spatialys.com