[mapserver-dev] Mapserver causes too many open files error

[Tamas Szekeres]

We did some investigation in this regard and the following 2 changes could
solve this specific problem on Windows and ASP.NET.

1. Adding the ability to use a single cache instead of a per-thread cache.

2. Increase the number of open stdio file handle limit for Windows. Default
is 512 the upper limit is 2048 accoding to this documentation:
https://msdn.microsoft.com/en-us/library/6e3b887c.aspx


These settings should probably be controlled with compile options or
environment variables.

Any objections to apply such changes?

Best regards,

Tamas






2017-11-27 16:42 GMT+01:00 Lime, Steve D (MNIT) <steve.lime at state.mn.us>:

> The former is certainly simpler. How expensive is setting up and clearing
> out the cache? Probably should look at Thomas’ RFC on the topic.
>
>
>
> *From:* mapserver-dev [mailto:mapserver-dev-bounces at lists.osgeo.org] *On
> Behalf Of *Tamas Szekeres
> *Sent:* Sunday, November 26, 2017 2:01 PM
> *To:* mapserver-dev at lists.osgeo.org
> *Subject:* [mapserver-dev] Mapserver causes too many open files error
>
>
>
> Hi Devs,
>
>
>
> We experienced an issue with mapserver 7.x (comparing to 6.x) that it
> causes too many open file handles in a multi-threaded environment. Actually
> the process is using the ASP.NET backend where the web requests are
> served through the ASP.NET worker process by using a thread pool, and the
> process should be considered as a long running process, where msCleanup is
> not being called regularly.
>
>
>
> By looking into the code the issue should probably be dedicated to the
> global font cache approach implemented in mapserver 7. Since freetype keeps
> the font files open, we get the number of fonts used in the mapfile
> multiplied with the thread pool size which may exceed the number of stdio
> open file handle limit as per https://msdn.microsoft.
> com/en-us/library/kdfaxaay(v=vs.140).aspx
>
>
>
> With the former approach (ie. 6.x) the font cache was implemented in the
> renderers and the cache cleanup was happening on the renderer cleanup
> in msFreeOutputFormat/msFreeMap.
>
>
>
> Do you have any suggestions about a proper workaround to this?
>
>
>
> I might think that we should either bind the font cache to each map
> objects (like earlier) or use a single common font cache for the entire
> process. I would probably prefer the first one as soon as we want to clean
> up every resources what a rendering session allocates.
>
>
>
> Thanks,
>
>
>
> Tamas
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20171201/5c01f352/attachment.html>