[mapguide-internals] std::string not thread safe on Linux

[Trevor Wekel]

Hi Gabriele,

Turning off logging does improve the situation but not 100%.  I have seen other crashes on Linux even with all of the logging disabled.  You may be able to leave the error log on but definitely turn the access log off.
 
Regards,
Trevor


-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Gabriele Monfardini
Sent: August 5, 2010 8:13 AM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] std::string not thread safe on Linux

> What does this mean to MapGuide?  Primarily, this issue comes into play with logging (access, trace, error, etc).  The log writing is performed on a separate thread and we use std::string to propagate information to that thread.   On Linux/GCC, the reference counted structure can be modified by both the logging thread and the worker thread simultaneously causing unexpected behaviour.  Unexpected behaviour takes the form of "glibc double free" and in some cases, a crash of the MapGuide Server process.
>
> This is easily reproducible on servers processing very high operation rates with logging turned on.  For example, I can reproduce the "double free" in under five minutes when serving GETTILEIMAGE requests to 40+ simultaneous users on an 8 core box.

This is somewhat ironic.
We raised log level to the max and enabled Trace log in order to gain
insight on frequent crashes on a Mapguide 2.1 and 2.2 beta server.

Crashes scale with the number of simultaneous users. We've experienced
up to fifty crashes in a day (note that performance was yet
acceptable).
But also uptime of days on servers with a few simultaneous users.

This issue may be the root cause of (some of) the crashes and also
maybe explains why we never find anything useful in the log around the
time of crashes.
In fact, apart from some cases where the last operations were a bulk
of SetResources on first map access (usually with maps that have many
layers), in many more cases the last operations were innocuous and all
terminated with success, as if the real operation that caused the
crash didn't have the time to log anything at all.

I cannot say, however, if the number of crashes has indeed increased
raising log level.

Until a proper solution emerges I need a quick-and-dirty workaround.
Would turning off logging mitigate the issue?

Regards,

Gabriele
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals