[gdal-dev] debugging ogr/libcurl 302 redirect failure

[snorris at hillcrestgeo.ca]

Hi,

Thanks again for the guidance and fix for below quoting issue several months ago.

The data source has kindly removed the spaces from their filename, quoting is no longer an issue.
Unfortunately with their update I'm now getting a different error that I can't debug (gdal 3.10.1):

ogrinfo \
  /vsizip/{/vsicurl/https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data}/transportation.gdb.zip \
  --debug ON  \
  --config CPL_CURL_VERBOSE=YES
...

ERROR 1: Read-write random access not supported for /vsizip
ogrinfo failed - unable to open '/vsizip/{/vsicurl/https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data}/transportation.gdb.zip'.

# try again as read only

ogrinfo -ro \
  /vsizip/{/vsicurl/https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data}/transportation.gdb.zip \
  --debug ON  \
  --config CPL_CURL_VERBOSE=YES
...

ERROR 4: `/vsizip/{/vsicurl/https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data}/transportation.gdb.zip' does not exist in the file system, and is not recognized as a supported dataset name.
ogrinfo failed - unable to open '/vsizip/{/vsicurl/https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data}/transportation.gdb.zip'.


Downloading the file with curl then reading with ogr works without issue:

curl -L -o transportation.gdb.zip https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data
ogrinfo transportation.gdb.zip
INFO: Open of `transportation.gdb.zip'
      using driver `OpenFileGDB' successful.

Sorry for all the usage questions to the dev list - I'd be happy to take this somewhere else.

thanks

Simon


On Aug 27, 2024, at 3:33 PM, Even Rouault <even.rouault at spatialys.com> wrote:

Hi,

will be fixed per https://github.com/OSGeo/gdal/pull/10663

Workaround with existing versions:

$ ogrinfo "/vsizip/{/vsicurl/https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data}/CoL_TransportationNetwork Jan 3 2024.gdb"
INFO: Open of `/vsizip/{/vsicurl/https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data}/CoL_TransportationNetwork Jan 3 2024.gdb'
      using driver `OpenFileGDB' successful.
Layer: TRN_BRIDGES (Point)
Layer: TRN_DISASTER_RESPONSE_ROUTES (Multi Line String)
Layer: TRN_MEDIANS (Multi Polygon)
Layer: TRN_RAILWAY (Multi Line String)
Layer: TRN_SIDEWALKS (Multi Line String)
Layer: TRN_STREETNAMES (Multi Polygon)
Layer: TRN_BICYCLE_ROUTES (Multi Line String)
Layer: TRN_ROADS (Multi Line String)

Even

Le 27/08/2024 à 23:37, Simon Norris via gdal-dev a écrit :
Hi list,

Using ogrinfo/ogr2ogr, I can't find the right config options to read data at this url: https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data

Example command:
ogrinfo https://governmentofbc.maps.arcgis.com/sharing/rest/content/items/4bba119c2e9042d683cc9378fb1e836e/data \
  --config CPL_VSIL_CURL_USE_HEAD=NO \
  --config CPL_CURL_VERBOSE=YES \
  --debug ON

I've tried a few additional config options, but can't tell what the problem is - the output from CURL_VERBOSE includes a GET to the appropriate new url, and it presumably uses the token without issue because it reports on the correct attachment:

CURL_INFO_HEADER_IN: Content-Disposition: attachment; filename="transportation.gdb.zip"; size=307145

I can download without issue using `curl --location-trusted` but would prefer to use ogr directly.

thanks!

Simon

_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com
My software is free, but my time generally not.