[mapserver-dev] Memory leak in mapcache

[Patrick Valsecchi]

Hi Thomas, Long time no see!

Every kind of request leaks. Even a WMS GetCapabilities (done by the
health-check) leaks, but at a lesser rate than a GetMap. The web UI does
requests in //, but the ingress limits at 10 connection in //.

And it's not just that it allocates a big chunk of memory and stays there.
It leaks continuously until the OOM wakes up and kills one of apache's
worker. I had to set a limit of 100 requests (every 100 requests the worker
process is killed and a new one is forked) per worker to keep memory usage
at bay.

Thanks

On Fri, Apr 12, 2019 at 7:12 PM thomas bonfort <thomas.bonfort at gmail.com>
wrote:

> Hi Patrick,
> What kind of requests is mapcache responding to, and at what concurrency?
> Serving multiple large wms requests is typically an example where large
> chunks of memory will be allocated.
>
> --
> thomas
>
> On Fri, Apr 12, 2019 at 12:42 AM Patrick Valsecchi <
> patrick.valsecchi at camptocamp.com> wrote:
>
>> Hi,
>>
>> I'm using mapcache v1.6.1 (tested as well with a master build) in docker
>> and I can see huge memory increases in the apache process memory usage when
>> mapcache is used (it's the only thing this apache instance is serving). The
>> thing is configured like that:
>>
>> <mapcache>
>>    <cache name="default" type="memcache">
>>       <server>
>>          <host>prod-2-4-c2cgeoportal-memcached</host>
>>          <port>11211</port>
>>       </server>
>>    </cache>
>>
>>    ...
>>
>>    <tileset name="map">
>>      <source>map</source>
>>      <cache>default</cache>
>>      <grid>swissgrid_005</grid>
>>      <metatile>5 5</metatile>
>>      <metabuffer>128</metabuffer>
>>      <format>image/png</format>
>>      <expires>3600</expires> <!-- 1 hour -->
>>      <auto_expire>13800</auto_expire> <!-- 4 hours -->
>>   </tileset>
>>
>>
>> I can see that memcached is used in it's stats and only mapcache uses it.
>>
>> But the the memory usage of the whole mapcache container looks like that:
>> [image: image.png]
>>
>> Each dip in usage is when the OOM kills one of the apache workers (out of
>> a maximum of 3 workers). I have configured a memory limit in docker at
>> 512MB.
>> Nothing shows up in the apache logs.
>>
>> Are there any known memory leaks? Do I have a bad configuration? I could
>> have the apache workers restarted every 10 queries, but that would be
>> utterly ugly and inefficient...
>>
>> Thanks for your help.
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20190415/1a849ee9/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 21279 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20190415/1a849ee9/attachment-0001.png>