[mapserver-dev] Mapserver causes too many open files error

Lime, Steve D (MNIT) steve.lime at state.mn.us
Fri Dec 1 06:48:33 PST 2017


What does #1 entail? The other change is Windows specific so seems ok – I don’t know what the right value should be of course.

From: Tamas Szekeres [mailto:szekerest at gmail.com]
Sent: Friday, December 01, 2017 1:53 AM
To: Lime, Steve D (MNIT) <steve.lime at state.mn.us>
Cc: mapserver-dev at lists.osgeo.org
Subject: Re: [mapserver-dev] Mapserver causes too many open files error

We did some investigation in this regard and the following 2 changes could solve this specific problem on Windows and ASP.NET<http://ASP.NET>.

1. Adding the ability to use a single cache instead of a per-thread cache.

2. Increase the number of open stdio file handle limit for Windows. Default is 512 the upper limit is 2048 accoding to this documentation: https://msdn.microsoft.com/en-us/library/6e3b887c.aspx


These settings should probably be controlled with compile options or environment variables.

Any objections to apply such changes?

Best regards,

Tamas






2017-11-27 16:42 GMT+01:00 Lime, Steve D (MNIT) <steve.lime at state.mn.us<mailto:steve.lime at state.mn.us>>:
The former is certainly simpler. How expensive is setting up and clearing out the cache? Probably should look at Thomas’ RFC on the topic.

From: mapserver-dev [mailto:mapserver-dev-bounces at lists.osgeo.org<mailto:mapserver-dev-bounces at lists.osgeo.org>] On Behalf Of Tamas Szekeres
Sent: Sunday, November 26, 2017 2:01 PM
To: mapserver-dev at lists.osgeo.org<mailto:mapserver-dev at lists.osgeo.org>
Subject: [mapserver-dev] Mapserver causes too many open files error

Hi Devs,

We experienced an issue with mapserver 7.x (comparing to 6.x) that it causes too many open file handles in a multi-threaded environment. Actually the process is using the ASP.NET<http://ASP.NET> backend where the web requests are served through the ASP.NET<http://ASP.NET> worker process by using a thread pool, and the process should be considered as a long running process, where msCleanup is not being called regularly.

By looking into the code the issue should probably be dedicated to the global font cache approach implemented in mapserver 7. Since freetype keeps the font files open, we get the number of fonts used in the mapfile multiplied with the thread pool size which may exceed the number of stdio open file handle limit as per https://msdn.microsoft.com/en-us/library/kdfaxaay(v=vs.140).aspx

With the former approach (ie. 6.x) the font cache was implemented in the renderers and the cache cleanup was happening on the renderer cleanup in msFreeOutputFormat/msFreeMap.

Do you have any suggestions about a proper workaround to this?

I might think that we should either bind the font cache to each map objects (like earlier) or use a single common font cache for the entire process. I would probably prefer the first one as soon as we want to clean up every resources what a rendering session allocates.

Thanks,

Tamas



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20171201/2963b69d/attachment-0001.html>


More information about the mapserver-dev mailing list