[gdal-dev] GDAL vsicurl with query string

Mon Aug 25 13:58:35 PDT 2014

Thanks so much for your help.

Curl works fine it's only the gdalinfo \vsicurl\ that is throwing the error using 7.21.2 of curl and libcurl on a Windows Server 2008 server.

I'm looking through to see what I can find online.

-----Original Message-----
From: Even Rouault [mailto:even.rouault at spatialys.com] 
Sent: Monday, August 25, 2014 3:48 PM
To: JDzialo John
Cc: gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] GDAL vsicurl with query string

Le lundi 25 août 2014 21:40:51, JDzialo John a écrit :
> Weird...
> 
> I'm still having an issue with the certificate using 
> GDAL_HTTP_UNSAFESSL YES.  It seems to try to verify the cert with this set or not.
> 
> I wonder why you are not seeing that error at all?

Not same curl version likely, or not compiled with same options. Mine is curl
7.19.7-1ubuntu1.7

> 
> Is there any other why of setting up gdalinfo to not verify the cert?  

I don't think so right now. Does that work with the curl binary on the command line ? If so, well, no further idea. If it does not work, then you likely have to find the additional curl option needed.

Searching for "error:14090086:SSL
routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"  in your favorite search engine might bring results perhaps.

> Is
> there a list of GDAL config options somewhere I could go through?

Most are documented in
http://www.gdal.org/cpl__http_8h.html#aee8368b7821300f4b81ef4da8a9c6a29 but I see that UNSAFESSL was not.

> 
> 
> 
> -----Original Message-----
> From: Even Rouault [mailto:even.rouault at spatialys.com]
> Sent: Monday, August 25, 2014 2:40 PM
> To: JDzialo John
> Cc: gdal-dev at lists.osgeo.org
> Subject: Re: [gdal-dev] GDAL vsicurl with query string
> 
> Le lundi 25 août 2014 20:23:14, JDzialo John a écrit :
> > Thanks.
> > 
> > I think an hour may be the limit but I set it for 10 hours so if 
> > it's allowed this link should be good until midnight tonight...
> > 
> > https://parcel-test.s3.amazonaws.com/test/key/value/string/object.jpg?
> > Signa
> > ture=7SPVoYI84N2YF5O0vhjvDv%2FU%2FOs%3D&Expires=1409026574&AWSAccess
> > Ke
> > yId=A
> > KIAJF3TTVKIEFQXGZ3Q
> > 
> > Thanks for the debug command I appreciate your help.  I tried the 
> > command and received the following error...
> > 
> > PS C:\Users\jdzialoex> gdalinfo --debug on --config CPL_CURL_VERBOSE 
> > YES "/vsicurl/https://parcel-test.s3.amazonaws.com/
> > test/key/value/string/object.jpg?Signature=7SPVoYI84N2YF5O0vhjvDv%2F
> > U% 2FOs %3D&Expires=1409026574&AWSAccessKeyId=AKIAJF3T TVKIEFQXGZ3Q"
> > VSICURL:
> > GetFileList(/vsicurl/https://parcel-test.s3.amazonaws.com/test/key/v
> > al
> > ue/s
> > tring) * timeout on name lookup is not supported
> > * About to connect() to parcel-test.s3.amazonaws.com port 443 (#0)
> > *   Trying 54.231.2.217... * connected
> > * Connected to parcel-test.s3.amazonaws.com (54.231.2.217) port 443
> > (#0)
> > * libcurl is now using a weak random seed!
> > * SSL certificate problem, verify that the CA cert is OK. Details:
> > error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
> > verify failed * Closing connection #0
> > * timeout on name lookup is not supported
> > * About to connect() to parcel-test.s3.amazonaws.com port 443 (#0)
> > *   Trying 54.231.2.217... * connected
> > * Connected to parcel-test.s3.amazonaws.com (54.231.2.217) port 443
> > (#0)
> > * SSL certificate problem, verify that the CA cert is OK. Details:
> > error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
> > verify failed * Closing connection #0
> > VSICURL:
> > GetFileSize(https://parcel-test.s3.amazonaws.com/test/key/value/stri
> > ng
> > /obj
> > ect.jpg?Signature=7SPVoYI84N2YF5O0vhj
> > vDv%2FU%2FOs%3D&Expires=1409026574&AWSAccessKeyId=AKIAJF3TTVKIEFQXGZ
> > 3Q
> > )=0
> > response_code=0 ERROR 4:
> > `/vsicurl/https://parcel-test.s3.amazonaws.com/test/key/value/string
> > /o
> > bjec
> > t.jpg?Signature=7SPVoYI84N2YF5O0vhjvD
> > v%2FU%2FOs%3D&Expires=1409026574&AWSAccessKeyId=AKIAJF3TTVKIEFQXGZ3Q'
> > does not exist in the file system, and is not recognised as a 
> > supported dataset name.
> > 
> > gdalinfo failed - unable to open
> > '/vsicurl/https://parcel-test.s3.amazonaws.com/test/key/value/string
> > /o
> > bjec
> > t.jpg?Signatu
> > re=7SPVoYI84N2YF5O0vhjvDv%2FU%2FOs%3D&Expires=1409026574&AWSAccessKe
> > yI
> > d=AK
> > IAJF3TTVKIEFQXGZ3Q'.
> 
> Interesting. I don't have that error (but others ;-)). There might be 
> a problem with the certificate, and curl being not able to check it.
> 
> I believe that "--config GDAL_HTTP_UNSAFESSL YES" should bypass 
> certificate checks.
> 
> Assuming this solves this issue, and you'll get the same errors as on 
> my machine, you'll see that the server doesn't apparently accept HEAD 
> requests (this seems to be a constant issue with S3 storage)
> 
> > HEAD
> 
> /test/key/value/string/object.jpg?Signature=7SPVoYI84N2YF5O0vhjvDv%2FU
> %2FOs %3D&Expires=1409026574&AWSAccessKeyId=AKIAJF3TTVKIEFQXGZ3Q 
> HTTP/1.1
> Host: parcel-test.s3.amazonaws.com
> Accept: */*
> 
> < HTTP/1.1 403 Forbidden
> 
> You can solve this by adding --config CPL_VSIL_CURL_USE_HEAD NO
> 
> And, optionnaly, to make it faster, add --config 
> GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR
> 
> With the last 2 applied :
> 
> $ gdalinfo  "/vsicurl/https://parcel-
> test.s3.amazonaws.com/test/key/value/string/object.jpg?Signature=7SPVo
> YI84N 
> 2YF5O0vhjvDv%2FU%2FOs%3D&Expires=1409026574&AWSAccessKeyId=AKIAJF3TTVK
> IEFQX GZ3Q" --config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR --config 
> CPL_VSIL_CURL_USE_HEAD NO Driver: JPEG/JPEG JFIF
> Files: /vsicurl/https://parcel-
> test.s3.amazonaws.com/test/key/value/string/object.jpg?Signature=7SPVo
> YI84N 
> 2YF5O0vhjvDv%2FU%2FOs%3D&Expires=1409026574&AWSAccessKeyId=AKIAJF3TTVK
> IEFQX
> GZ3Q Size is 974, 647
> Coordinate System is `'
> Metadata:
>   EXIF_ExifVersion=0220
>   EXIF_PixelXDimension=974
>   EXIF_PixelYDimension=647
>   EXIF_Software=Google
> Image Structure Metadata:
>   COMPRESSION=JPEG
>   INTERLEAVE=PIXEL
>   SOURCE_COLOR_SPACE=YCbCr
> Corner Coordinates:
> Upper Left  (    0.0,    0.0)
> Lower Left  (    0.0,  647.0)
> Upper Right (  974.0,    0.0)
> Lower Right (  974.0,  647.0)
> Center      (  487.0,  323.5)
> Band 1 Block=974x1 Type=Byte, ColorInterp=Red
>   Overviews: 487x324, 244x162
>   Image Structure Metadata:
>     COMPRESSION=JPEG
> Band 2 Block=974x1 Type=Byte, ColorInterp=Green
>   Overviews: 487x324, 244x162
>   Image Structure Metadata:
>     COMPRESSION=JPEG
> Band 3 Block=974x1 Type=Byte, ColorInterp=Blue
>   Overviews: 487x324, 244x162
>   Image Structure Metadata:
>     COMPRESSION=JPEG
> 
> Even
> 
> --
> Spatialys - Geospatial professional services http://www.spatialys.com

--
Spatialys - Geospatial professional services http://www.spatialys.com