[gdal-dev] WMTS request error

Tue Sep 3 06:27:53 PDT 2024

Hi,

About navigating the documentation, configuration options are needed very often. Now they are somewhat hidden in the front page https://gdal.org/en/latest/index.html. Maybe Configuration options should appear as a title directly in the table of contents, and not under the title "User", that does not feel so important.

-Jukka Rahkonen-

Lähettäjä: Michał Kowalczuk <michkowalczuk at gmail.com>
Lähetetty: tiistai 3. syyskuuta 2024 15.16
Vastaanottaja: Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
Kopio: gdal-dev at lists.osgeo.org
Aihe: Re: [gdal-dev] WMTS request error

Thank you, I will test it.
It's a pity that the WMTS documentation doesn't mention it.
The WMTS docs indicates that only 3 options are available for WMTS: GDAL_MAX_CONNECTIONS, GDAL_ENABLE_WMS_CACHE, GDAL_DEFAULT_WMS_CACHE_PATH (https://gdal.org/en/latest/drivers/raster/wmts.html#configuration-options).
I guess I still don't have enough experience in navigating the documentation and using GDAL;-)

Michał Kowalczuk

wt., 3 wrz 2024 o 11:31 Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi<mailto:jukka.rahkonen at maanmittauslaitos.fi>> napisał(a):
Hi,

I would read https://gdal.org/en/latest/user/configoptions.html and try there 3:

GDAL_HTTP_MAX_RETRY=value: Defaults to 0. Set the number of HTTP attempts, when a retry is allowed. (cf GDAL_HTTP_RETRY_CODES for conditions where a retry is attempted.) The default value is 0, meaning no retry.
GDAL_HTTP_RETRY_DELAY=<seconds>: Defaults to 30. Set the delay between HTTP attempts.
GDAL_HTTP_RETRY_CODES=ALL or comma-separated list of codes: (GDAL >= 3.10) Specify which HTTP error codes should trigger a retry attempt. Valid values are ALL or a comma-separated list of HTTP codes. By default, 429, 500, 502, 503 or 504 HTTP errors are considered, as well as other situations with a particular HTTP or Curl error message.

-Jukka Rahkonen-

Lähettäjä: gdal-dev <gdal-dev-bounces at lists.osgeo.org<mailto:gdal-dev-bounces at lists.osgeo.org>> Puolesta Michal Kowalczuk via gdal-dev
Lähetetty: tiistai 3. syyskuuta 2024 12.08
Vastaanottaja: gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
Aihe: [gdal-dev] WMTS request error

Dear GDAL colleagues,

Does GDAL has any http request failure detection, and mechanism for dealing with it different than raising CPL Error.
Look at the following GDAL log. (Problem explanation after the log)
----------------------------------------------------------------------------------------------------
[Tue Sep  3 10:54:47 2024].7930, 1227.3430: HTTP: Requesting [1/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=1
[Tue Sep  3 10:54:47 2024].7930, 1227.3430: HTTP: Requesting [2/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=2
[Tue Sep  3 10:54:49 2024].3630, 1228.9130: HTTP: Requesting [3/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=3
[Tue Sep  3 10:54:50 2024].3630, 1229.9130: HTTP: Requesting [4/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=4
[Tue Sep  3 10:54:50 2024].9300, 1230.4800: HTTP: Requesting [5/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=5
[Tue Sep  3 10:54:52 2024].5640, 1232.1140: HTTP: Requesting [6/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=6
[Tue Sep  3 10:54:52 2024].7680, 1232.3180: HTTP: Requesting [7/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=7
[Tue Sep  3 10:54:53 2024].1190, 1232.6690: HTTP: Requesting [8/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=8
[Tue Sep  3 10:54:53 2024].6050, 1233.1550: HTTP: Requesting [9/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=9
[Tue Sep  3 10:54:53 2024].9630, 1233.5130: HTTP: Requesting [10/10] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=10
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [0] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=1 : status = 0, type = (null), error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [1] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=2 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [2] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=3 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [3] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=4 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [4] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=5 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [5] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=6 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [6] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=7 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [7] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=8 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [8] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=9 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3000, 1233.8500: HTTP: Request [9] https://mapy.geoportal.gov.pl/wss/service/PZGIK/ORTO/WMTS/StandardResolution?service=WMTS&request=GetTile&version=1.0.0&layer=ORTOFOTOMAPA&style=default&format=image/jpeg&TileMatrixSet=EPSG:4326&TileMatrix=EPSG:4326:5&TileRow=4&TileCol=10 : status = 200, type = image/jpeg, error = (null)
[Tue Sep  3 10:54:54 2024].3030, 1233.8530: WMS: ReadBlockFromCache
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: CPLError: C:\Users\Michal\.cache\gdalwmscache\d03651177815877cffe0faf894690352/0/6/069e88521499910bd8763ede3e0cc2c4: No such file or directory
ERROR 4: C:\Users\Michal\.cache\gdalwmscache\d03651177815877cffe0faf894690352/0/6/069e88521499910bd8763ede3e0cc2c4: No such file or directory
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: CPLError: GDALWMS: Unable to open downloaded block.
ERROR 1: GDALWMS: Unable to open downloaded block.
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: WMS: After ReadBlockFromCache
[Tue Sep  3 10:54:54 2024].3040, 1233.8540: CPLError: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
ERROR 1: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
[Tue Sep  3 10:54:54 2024].3050, 1233.8550: CPLError: GDAL_WMS>, band 3: IReadBlock failed at X offset 1, Y offset 4: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
ERROR 1: GDAL_WMS>, band 3: IReadBlock failed at X offset 1, Y offset 4: GDALWMS: Unable to download block 1, 4.
URL:
  HTTP status code: 0, error: (null).
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
----------------------------------------------------------------------------------------------------

I use GDALRasterIO from C API and as we can see GDAL makes 10 requests. One of these request (bolded) ends with error, which generates the "Unable to open downloaded block." error later.
Is there any method to force a re-request in such situations?
This server error occurs randomly and temporarily, and the next attempt completes without problem.

Regards,
Michał
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240903/a8d3df65/attachment-0001.htm>