[mapserver-dev] Memory leak in mapcache

[Patrick Valsecchi]

Hi,

I'm sorry, I had no time to investigate further. I've just configured
apache to restart mapcache every 100 requests. Poor man's garbage
collection, but that works fine.

Thanks.

On Wed, May 15, 2019 at 10:24 PM Jeff McKenna <jmckenna at gatewaygeomatics.com>
wrote:

> I wonder if we should tag a MapCache release today, there are over 2
> years of changes committed since the last release
> (https://github.com/mapserver/mapcache/compare/rel-1-6-1...master).  -jeff
>
>
>
> On 2019-05-15 4:37 PM, Lime, Steve D (MNIT) wrote:
> > Not that I saw - nothing from Patrick beyond the original post - at
> > least not to the list.
> >
> > ------------------------------------------------------------------------
> > *From:* mapserver-dev <mapserver-dev-bounces at lists.osgeo.org> on behalf
> > of Stephen Woodbridge <stephenwoodbridge37 at gmail.com>
> > *Sent:* Wednesday, May 15, 2019 1:33:08 PM
> > *To:* mapserver-dev at lists.osgeo.org
> > *Subject:* Re: [mapserver-dev] Memory leak in mapcache
> > Was there a followup on this? I don't remember seeing one.
> >
> > Thanks,
> >     -Steve W
> >
> > On 4/15/2019 4:36 AM, Patrick Valsecchi wrote:
> >> Hi Thomas, Long time no see!
> >>
> >> Every kind of request leaks. Even a WMS GetCapabilities (done by the
> >> health-check) leaks, but at a lesser rate than a GetMap. The web UI
> >> does requests in //, but the ingress limits at 10 connection in //.
> >>
> >> And it's not just that it allocates a big chunk of memory and stays
> >> there. It leaks continuously until the OOM wakes up and kills one of
> >> apache's worker. I had to set a limit of 100 requests (every 100
> >> requests the worker process is killed and a new one is forked) per
> >> worker to keep memory usage at bay.
> >>
> >> Thanks
> >>
> >> On Fri, Apr 12, 2019 at 7:12 PM thomas bonfort
> >> <thomas.bonfort at gmail.com <mailto:thomas.bonfort at gmail.com>> wrote:
> >>
> >>     Hi Patrick,
> >>     What kind of requests is mapcache responding to, and at what
> >>     concurrency? Serving multiple large wms requests is typically an
> >>     example where large chunks of memory will be allocated.
> >>
> >>     --
> >>     thomas
> >>
> >>     On Fri, Apr 12, 2019 at 12:42 AM Patrick Valsecchi
> >>     <patrick.valsecchi at camptocamp.com
> >>     <mailto:patrick.valsecchi at camptocamp.com>> wrote:
> >>
> >>         Hi,
> >>
> >>         I'm using mapcache v1.6.1 (tested as well with a master build)
> >>         in docker and I can see huge memory increases in the apache
> >>         process memory usage when mapcache is used (it's the only
> >>         thing this apache instance is serving). The thing is
> >>         configured like that:
> >>
> >>             <mapcache>
> >>                <cache name="default" type="memcache">
> >>                   <server>
> >>                      <host>prod-2-4-c2cgeoportal-memcached</host>
> >>                      <port>11211</port>
> >>                   </server>
> >>                </cache>
> >>
> >>                ...
> >>
> >>                <tileset name="map">
> >>                  <source>map</source>
> >>                  <cache>default</cache>
> >>                  <grid>swissgrid_005</grid>
> >>                  <metatile>5 5</metatile>
> >>                  <metabuffer>128</metabuffer>
> >>                  <format>image/png</format>
> >>                  <expires>3600</expires> <!-- 1 hour -->
> >>                  <auto_expire>13800</auto_expire> <!-- 4 hours -->
> >>               </tileset>
> >>
> >>
> >>         I can see that memcached is used in it's stats and only
> >>         mapcache uses it.
> >>
> >>         But the the memory usage of the whole mapcache container looks
> >>         like that:
> >>         image.png
> >>
> >>         Each dip in usage is when the OOM kills one of the apache
> >>         workers (out of a maximum of 3 workers). I have configured a
> >>         memory limit in docker at 512MB.
> >>         Nothing shows up in the apache logs.
> >>
> >>         Are there any known memory leaks? Do I have a bad
> >>         configuration? I could have the apache workers restarted every
> >>         10 queries, but that would be utterly ugly and inefficient...
> >>
> >>         Thanks for your help.
> >
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20190516/203d3f74/attachment-0001.html>