[gdal-dev] WMTS request error

Tue Sep 3 02:31:41 PDT 2024

Hi,

I would read https://gdal.org/en/latest/user/configoptions.html and try there 3:

GDAL_HTTP_MAX_RETRY=value: Defaults to 0. Set the number of HTTP attempts, when a retry is allowed. (cf GDAL_HTTP_RETRY_CODES for conditions where a retry is attempted.) The default value is 0, meaning no retry.
GDAL_HTTP_RETRY_DELAY=<seconds>: Defaults to 30. Set the delay between HTTP attempts.
GDAL_HTTP_RETRY_CODES=ALL or comma-separated list of codes: (GDAL >= 3.10) Specify which HTTP error codes should trigger a retry attempt. Valid values are ALL or a comma-separated list of HTTP codes. By default, 429, 500, 502, 503 or 504 HTTP errors are considered, as well as other situations with a particular HTTP or Curl error message.

-Jukka Rahkonen-

Lähettäjä: gdal-dev <gdal-dev-bounces at lists.osgeo.org> Puolesta Michal Kowalczuk via gdal-dev
Lähetetty: tiistai 3. syyskuuta 2024 12.08
Vastaanottaja: gdal-dev at lists.osgeo.org
Aihe: [gdal-dev] WMTS request error

Dear GDAL colleagues,

Does GDAL has any http request failure detection, and mechanism for dealing with it different than raising CPL Error.
Look at the following GDAL log. (Problem explanation after the log)
----------------------------------------------------------------------------------------------------
[Tue Sep  3 10:54:47 2024].7930, 1227.3430: HTTP: Requesting [1/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=1
[Tue Sep  3 10:54:47 2024].7930, 1227.3430: HTTP: Requesting [2/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=2
[Tue Sep  3 10:54:49 2024].3630, 1228.9130: HTTP: Requesting [3/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=3
[Tue Sep  3 10:54:50 2024].3630, 1229.9130: HTTP: Requesting [4/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=4
[Tue Sep  3 10:54:50 2024].9300, 1230.4800: HTTP: Requesting [5/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=5
[Tue Sep  3 10:54:52 2024].5640, 1232.1140: HTTP: Requesting [6/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=6
[Tue Sep  3 10:54:52 2024].7680, 1232.3180: HTTP: Requesting [7/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=7
[Tue Sep  3 10:54:53 2024].1190, 1232.6690: HTTP: Requesting [8/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=8
[Tue Sep  3 10:54:53 2024].6050, 1233.1550: HTTP: Requesting [9/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=9
[Tue Sep  3 10:54:53 2024].9630, 1233.5130: HTTP: Requesting [10/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=10
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [0] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=1 : status = 0, type = (null), error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [1] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=2 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [2] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=3 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [3] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=4 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [4] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=5 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [5] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=6 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [6] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=7 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [7] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=8 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [8] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=9 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [9] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=10 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3030, 1233.8530: WMS: ReadBlockFromCache
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: CPLError: C:\Users\Michal\.cache\gdalwmscache\d03651177815877cffe0faf894690352/0/6/069e88521499910bd8763ede3e0cc2c4: No such file or directory
ERROR 4: C:\Users\Michal\.cache\gdalwmscache\d03651177815877cffe0faf894690352/0/6/069e88521499910bd8763ede3e0cc2c4: No such file or directory
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: CPLError: GDALWMS: Unable to open downloaded block.
ERROR 1: GDALWMS: Unable to open downloaded block.
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: WMS: After ReadBlockFromCache
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: CPLError: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
ERROR 1: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
[Tue Sep  3 10:54:54 2024].3050, 1233.8550: CPLError: GDAL_WMS>, band 3: IReadBlock failed at X offset 1, Y offset 4: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
ERROR 1: GDAL_WMS>, band 3: IReadBlock failed at X offset 1, Y offset 4: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
----------------------------------------------------------------------------------------------------

I use GDALRasterIO from C API and as we can see GDAL makes 10 requests. One of these request (bolded) ends with error, which generates the "Unable to open downloaded block." error later.
Is there any method to force a re-request in such situations?
This server error occurs randomly and temporarily, and the next attempt completes without problem.

Regards,
Michał
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240903/037e259a/attachment-0001.htm>