[mapserver-dev] Memory leak in mapcache

[Lime, Steve D (MNIT)]

Not that I saw - nothing from Patrick beyond the original post - at least not to the list.

________________________________
From: mapserver-dev <mapserver-dev-bounces at lists.osgeo.org> on behalf of Stephen Woodbridge <stephenwoodbridge37 at gmail.com>
Sent: Wednesday, May 15, 2019 1:33:08 PM
To: mapserver-dev at lists.osgeo.org
Subject: Re: [mapserver-dev] Memory leak in mapcache

Was there a followup on this? I don't remember seeing one.

Thanks,
   -Steve W

On 4/15/2019 4:36 AM, Patrick Valsecchi wrote:
> Hi Thomas, Long time no see!
>
> Every kind of request leaks. Even a WMS GetCapabilities (done by the
> health-check) leaks, but at a lesser rate than a GetMap. The web UI
> does requests in //, but the ingress limits at 10 connection in //.
>
> And it's not just that it allocates a big chunk of memory and stays
> there. It leaks continuously until the OOM wakes up and kills one of
> apache's worker. I had to set a limit of 100 requests (every 100
> requests the worker process is killed and a new one is forked) per
> worker to keep memory usage at bay.
>
> Thanks
>
> On Fri, Apr 12, 2019 at 7:12 PM thomas bonfort
> <thomas.bonfort at gmail.com <mailto:thomas.bonfort at gmail.com>> wrote:
>
>     Hi Patrick,
>     What kind of requests is mapcache responding to, and at what
>     concurrency? Serving multiple large wms requests is typically an
>     example where large chunks of memory will be allocated.
>
>     --
>     thomas
>
>     On Fri, Apr 12, 2019 at 12:42 AM Patrick Valsecchi
>     <patrick.valsecchi at camptocamp.com
>     <mailto:patrick.valsecchi at camptocamp.com>> wrote:
>
>         Hi,
>
>         I'm using mapcache v1.6.1 (tested as well with a master build)
>         in docker and I can see huge memory increases in the apache
>         process memory usage when mapcache is used (it's the only
>         thing this apache instance is serving). The thing is
>         configured like that:
>
>             <mapcache>
>                <cache name="default" type="memcache">
>                   <server>
>                      <host>prod-2-4-c2cgeoportal-memcached</host>
>                      <port>11211</port>
>                   </server>
>                </cache>
>
>                ...
>
>                <tileset name="map">
>                  <source>map</source>
>                  <cache>default</cache>
>                  <grid>swissgrid_005</grid>
>                  <metatile>5 5</metatile>
>                  <metabuffer>128</metabuffer>
>                  <format>image/png</format>
>                  <expires>3600</expires> <!-- 1 hour -->
>                  <auto_expire>13800</auto_expire> <!-- 4 hours -->
>               </tileset>
>
>
>         I can see that memcached is used in it's stats and only
>         mapcache uses it.
>
>         But the the memory usage of the whole mapcache container looks
>         like that:
>         image.png
>
>         Each dip in usage is when the OOM kills one of the apache
>         workers (out of a maximum of 3 workers). I have configured a
>         memory limit in docker at 512MB.
>         Nothing shows up in the apache logs.
>
>         Are there any known memory leaks? Do I have a bad
>         configuration? I could have the apache workers restarted every
>         10 queries, but that would be utterly ugly and inefficient...
>
>         Thanks for your help.
>         _______________________________________________
>         mapserver-dev mailing list
>         mapserver-dev at lists.osgeo.org
>         <mailto:mapserver-dev at lists.osgeo.org>
>         https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C02d7dd2a7ef94ad9b92908d6d963ca9a%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636935419954290965&sdata=5rn3C0cRHGeVg9yzj0rTpNsrh%2BKnUkqVJzgz2lbipt4%3D&reserved=0
>
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C02d7dd2a7ef94ad9b92908d6d963ca9a%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636935419954300979&sdata=pNYEzd1InZ8gm%2B74ZG%2BgIsCHCXC1L90NDRks6eqcMh0%3D&reserved=0


---
This email has been checked for viruses by Avast antivirus software.
https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.avast.com%2Fantivirus&data=02%7C01%7Csteve.lime%40state.mn.us%7C02d7dd2a7ef94ad9b92908d6d963ca9a%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636935419954300979&sdata=XonCwE%2BUrOoxN1frcUO6sZmDyO0rxGBTkvd27uj9OxA%3D&reserved=0

_______________________________________________
mapserver-dev mailing list
mapserver-dev at lists.osgeo.org
https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C02d7dd2a7ef94ad9b92908d6d963ca9a%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636935419954300979&sdata=pNYEzd1InZ8gm%2B74ZG%2BgIsCHCXC1L90NDRks6eqcMh0%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20190515/f4ab4f12/attachment.html>