[gdal-dev] vsicurl and an issue with http 302 redirect in the middle of the file
even.rouault at spatialys.com
Wed May 5 13:19:16 PDT 2021
Running with CPL_CURL_VERBOSE=YES shows the following
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Your allowed limit has been reached. Please go to
https://www.weather.gov/abusive-user-block for more info
* no chunk, no close, no size. Assume close to signal end
So it seems that after a number of quickly consecutive GET requests to
the server as here, the server no longer want to see us. The 302 HTTP
error it sends however is not appropriate. 429 "Too many requests" would
be more appropriate
GRIB files with lots of "messages" / bands like that one aren't really
cloud friendly, as you have to skip in many parts of the file to get an
inventory of it.
That said, I see that in that instance, the .grib files are accompanied
by a sidecar file, like
which I believe is produced by the wgrib utility. We could conceptually
try to make use of it, however it doesn't contain all metadata that are
typically retrieved by the driver, so for gdalinfo itself that wouldn't
be sufficient, but for just a Open() call that might be enough.
I've noted that https://github.com/OSGeo/gdal/issues/3799
Le 05/05/2021 à 22:04, Rahkonen Jukka (MML) a écrit :
> Have a look at
> There gdalinfo fails with /vsicurl/ somewhere in the middle of the
> grib2 file when the server sends http 302 as a response for a range
> request. The file is OK as was tested by downloading it first totally
> with curl. Could it be that the http “moved permanently” at that stage
> makes /vsicurl/ to fail?
> I made also a few tests and it seems that the error may happen in
> another http range, but it is always similar:
> VSICURL: Got response_code=302
> GRIB: ERROR: Ran out of file in Section 7
> -Jukka Rahkonen-
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the gdal-dev