<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body>
<p>Jukka,</p>
<p>Running with CPL_CURL_VERBOSE=YES shows the following<br>
</p>
<p>> GET
/data/nccf/com/hrrr/prod/hrrr.20210505/conus/hrrr.t15z.wrfsfcf01.grib2
HTTP/1.1<br>
Host: ftpprd.ncep.noaa.gov<br>
Accept: */*<br>
Range: bytes=84819968-84836351<br>
<br>
* Mark bundle as not supporting multiuse<br>
< HTTP/1.1 302 Your allowed limit has been reached. Please go
to <a class="moz-txt-link-freetext" href="https://www.weather.gov/abusive-user-block">https://www.weather.gov/abusive-user-block</a> for more info <br>
* no chunk, no close, no size. Assume close to signal end<br>
< <br>
</p>
<p>So it seems that after a number of quickly consecutive GET
requests to the server as here, the server no longer want to see
us. The 302 HTTP error it sends however is not appropriate. 429
"Too many requests" would be more appropriate<br>
</p>
<p>GRIB files with lots of "messages" / bands like that one aren't
really cloud friendly, as you have to skip in many parts of the
file to get an inventory of it.</p>
<p>That said, I see that in that instance, the .grib files are
accompanied by a sidecar file, like
<a class="moz-txt-link-freetext" href="https://ftpprd.ncep.noaa.gov/data/nccf/com/hrrr/prod/hrrr.20210505/conus/hrrr.t15z.wrfsfcf01.grib2.idx">https://ftpprd.ncep.noaa.gov/data/nccf/com/hrrr/prod/hrrr.20210505/conus/hrrr.t15z.wrfsfcf01.grib2.idx</a>,
which I believe is produced by the wgrib utility. We could
conceptually try to make use of it, however it doesn't contain all
metadata that are typically retrieved by the driver, so for
gdalinfo itself that wouldn't be sufficient, but for just a Open()
call that might be enough.</p>
<p>I've noted that <a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/issues/3799">https://github.com/OSGeo/gdal/issues/3799</a><br>
</p>
<p>Even<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">Le 05/05/2021 à 22:04, Rahkonen Jukka
(MML) a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:fff6c569f8ea44eabb160664afca3fb8@maanmittauslaitos.fi">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}span.Shkpostityyli17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US">Have a look at <a
href="https://gis.stackexchange.com/questions/395867/opening-a-grib-from-the-web-with-gdal-in-python-using-vsicurl-throws-error-on-m"
moz-do-not-send="true">
https://gis.stackexchange.com/questions/395867/opening-a-grib-from-the-web-with-gdal-in-python-using-vsicurl-throws-error-on-m</a>.
There gdalinfo fails with /vsicurl/ somewhere in the middle
of the grib2 file when the server sends http 302 as a
response for a range request. The file is OK as was tested
by downloading it first totally with curl. Could it be that
the http “moved permanently” at that stage makes /vsicurl/
to fail?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I made also a few tests
and it seems that the error may happen in another http
range, but it is always similar:<br>
VSICURL: Got response_code=302<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">GRIB: ERROR: Ran out of
file in Section 7<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-Jukka Rahkonen-<o:p></o:p></span></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>