[gdal-dev] GDALWMS - error: TCP connection reset by peer

[Rahkonen Jukka (MML)]

Hi,

Large MaxConnections value leads to more concurrent requests and it may cause problems if the WMS server is very busy. Reducing MaxConnections may help if you are the only user of that WMS but if there are thousands of other users as well the service may still run at 100% and drop request either because they reach the server side timeout or because load balancer denies requests. If the overload is temporary then retry after a while could help but I do not believe that GDAL WMS driver can be configured that way  https://gdal.org/drivers/raster/wms.html.  Special tile cache programs like GeoWebCache are better prepared for failing requests https://www.geowebcache.org/docs/current/production/index.html#seed-failure-tolerance

The GDAL WMS configuration options are documented only by example https://gdal.org/drivers/raster/wms.html. I wonder where I could find information about the meaning of options that I can't guess by the name, like "CleanTimeout".

 -Jukka Rahkonen-

Lähettäjä: Roman Breitfuss-Schiffer <roman.schiffer at gmail.com>
Lähetetty: perjantai 19. marraskuuta 2021 10.18
Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahkonen at maanmittauslaitos.fi>; gdal-dev at lists.osgeo.org
Aihe: Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer


Dear Jukka!

Thanks for your answer! I think I'm gonna follow that road you suggested.

We have been thinking a lot and I was just wondering if there could be some GDAL configuration which could cause that behaviour? For example the value for <MaxConnections> in the WMS XML file? Or if more applications are accessing the same WMS XML at the same time which is theoretically possible in our case.

Best regards
Roman
Am 18.11.2021 um 12:12 schrieb Rahkonen Jukka (MML):
Hi,

It seems that the WMS server is closing the door and there is nothing else to do on the GDAL side except to try again later. Contact the WMS service maintainer and report your troubles. They may be able to provide you a more reliable service especially if you are ready to pay for it.

-Jukka Rahkonen-

Lähettäjä: gdal-dev <gdal-dev-bounces at lists.osgeo.org><mailto:gdal-dev-bounces at lists.osgeo.org> Puolesta Roman Breitfuss-Schiffer
Lähetetty: torstai 18. marraskuuta 2021 11.59
Vastaanottaja: gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
Aihe: [gdal-dev] GDALWMS - error: TCP connection reset by peer


Dear all!

We are running a Python application an a linux distribution (RedHat) which creates clips in MBTILES format from a WMS. We are using the Python-API and are basically doing something like this:

src_ds = gdal.Warp(
    tmp_file,
    ds_in,
    format='GTiff',
    outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]],
    dstSRS=output_srs,
)
ds_out = gdal.Translate(
    out_filepath,
    src_ds,
    format='MBTILES',
    creationOptions=creation_options,
)

The application throughs an error from time to time since a few days. We couldn't figure out a pattern yet and therefore we are also not able to reproduce the error. The error is the following:

Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 2765, 602.
URL: TCP connection reset by peer
  HTTP status code: 0, error: TCP connection reset by peer.
Add the HTTP status code to <ZeroBlockHttpCodes> to ignore this error (see http://www.gdal.org/frmt_wms.html).
Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: IReadBlock failed at X offset 2764, Y offset 601
Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block offset 2764, Y block offset 601

The exact same data gets processed successfully in one run and not successfully in another run. We didn't change the code nor the Python or GDAL version. Hence, we are kind of puzzled as the error comes and goes seemingly at random.

Does anyone of you have any hints?

Thanks & best regards!
Roman

_______________________________________________

Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)

--

_______________________________________________

Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)

Veronikagasse 38/20, 1170 Wien

+43 664 5547678
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20211119/9c88cc42/attachment-0001.html>