<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Michael,</p>
<p>analysis and improvements in
<a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/pull/7737">https://github.com/OSGeo/gdal/pull/7737</a></p>
<p>Even<br>
</p>
<div class="moz-cite-prefix">Le 10/05/2023 à 09:03, Michael Sumner a
écrit :<br>
</div>
<blockquote type="cite"
cite="mid:CAAcGz997pbVhLeJJOyzmZC94epajWHcCKEJ9mHOgWar6cO+uSA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">I'm having trouble with remote dsn from here:
<div><br>
</div>
<div><a
href="https://esgf.nci.org.au/thredds/catalog/esgcet/cmip5/output1/CSIRO-BOM/ACCESS1-3/rcp85/day/ocean/day/r1i1p1/cmip5.output1.CSIRO-BOM.ACCESS1-3.rcp85.day.ocean.day.r1i1p1.v20120413.html#cmip5.output1.CSIRO-BOM.ACCESS1-3.rcp85.day.ocean.day.r1i1p1.v20120413"
moz-do-not-send="true" class="moz-txt-link-freetext">https://esgf.nci.org.au/thredds/catalog/esgcet/cmip5/output1/CSIRO-BOM/ACCESS1-3/rcp85/day/ocean/day/r1i1p1/cmip5.output1.CSIRO-BOM.ACCESS1-3.rcp85.day.ocean.day.r1i1p1.v20120413.html#cmip5.output1.CSIRO-BOM.ACCESS1-3.rcp85.day.ocean.day.r1i1p1.v20120413</a><br>
</div>
<div><br>
</div>
<div>The support tells me I have been blocked for requests from
my IP that exceed 300 in 5min. Apparently, daily you are
cleared from the blocklist but with experiments today I've had
two machines blocked (and now manually cleared). Still, it's
hard to test :0</div>
<div><br>
</div>
<div>I've stripped down to this dsn, I thought limiting the
driver might limit the requests because it won't cascade down
the driver identification (but I'm not sure if -if adds
anymore than the "NETCDF:" is already doing). At any rate I
need to target a subdataset so I need that syntax. </div>
<div><br>
</div>
<div>"vrt://NETCDF:\"/vsicurl/<a
href="https://esgf.nci.org.au/thredds/fileServer/master/CMIP5/output1/CSIRO-BOM/ACCESS1-3/rcp85/day/ocean/day/r1i1p1/v20120413/tos/tos_day_ACCESS1-3_rcp85_r1i1p1_20560101-20651231.nc\"
moz-do-not-send="true">https://esgf.nci.org.au/thredds/fileServer/master/CMIP5/output1/CSIRO-BOM/ACCESS1-3/rcp85/day/ocean/day/r1i1p1/v20120413/tos/tos_day_ACCESS1-3_rcp85_r1i1p1_20560101-20651231.nc\</a>":tos?bands=1&if=NETCDF"</div>
<div><br>
</div>
<div>It takes quite a while to read one band, but the warper and
RasterIO result is good. I think the machine is now blocked
again, I wasn't able to also run gdalinfo after warper and
straight read. </div>
<div><br>
</div>
<div>I'm just doing standalone from-scratch opens, not with a
persistent dataset - I assume that would reduce the request
load, which I'll try in python when I have access again. </div>
<div><br>
</div>
<div>Questions: </div>
<div><br>
</div>
<div>1. Are there other things I can pursue to reduce my request
load on the server? <br>
</div>
<div><br>
</div>
<div>2. Can this actually be faster? RasterIO for one band
360x300 was minutes. (I appreciate this might be better
suited to mdim, but that seems similarly problematic in terms
of requests and performance). </div>
<div><br>
</div>
<div>I've tried the config options GDAL_NETCDF_VERIFY_DIMS /
STRICT and GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECK=/YES and that
does seem to have helped the warper request but RasterIO was
still slow. </div>
<div>. </div>
<div>This was on GDAL 3.8.0dev-414270a94b-dirty, released
2023/05/09 (debug build)<br>
</div>
<div><br>
</div>
<div>I see this repeated warning </div>
<div><br>
</div>
<div>Warning 1: dimension #2 (i) is not a Longitude/X dimension.<br>
Warning 1: dimension #1 (j) is not a Latitude/Y dimension.<br>
Warning 1: dimension #1 (i) is not a Longitude/X dimension.<br>
Warning 1: dimension #0 (j) is not a Latitude/Y dimension.<br>
Warning 1: dimension #1 (i) is not a Longitude/X dimension.<br>
Warning 1: dimension #0 (j) is not a Latitude/Y dimension.<br>
</div>
<div><br>
</div>
<div>Thank you<br clear="all">
<div><br>
</div>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">Michael Sumner<br>
Software and Database Engineer<br>
Australian Antarctic Division<br>
Hobart, Australia<br>
e-mail: <a href="mailto:mdsumner@gmail.com" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">mdsumner@gmail.com</a></div>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>