MapScript C# Memory Corruption Problem (MapServer 4.10)

[Tamas Szekeres]

Since this issue seems to be quite an essential it would be useful to
keep it tracked through a bug report. Would you submit one by
summarizing the all the information we currently have
(http://mapserver.gis.umn.edu/bugs/enter_bug.cgi)? Assign the bug to
the C# binding.

Here are some further tests that would worth trying:

1. Try to alter the SDE connection pooling behaviour. Adding
PROCESSING "CLOSE_CONNECTION=DEFER"  would set it on a per layer
basis. I don't know exactly which would be the best at this case, but
in fact the connections are retained per threads. Maybe keeping the
connections alive would affect how often the memory allocations
related to the connections take place.

2. I've just added a possibility to use a global lock when invoking
into the mapscript code. To enable this option you should uncomment
#define USE_GLOBAL_LOCK in csmodule.i. and recompile the mapscript
code. This setting would ensure that only one thread will invoke into
libmap.dll in most cases simultaneously, and could help detecting some
threading issues, though this solution is not equal as restricting the
invocation to one thread (like the COM STA model would do for
example).

3. Ensure disposing the cloned mapObj explicitly by the "using"
directive for example, like

using (map = ((mapObj)Session("m_mapObj")).clone() )
{
    // operate on map
}

The child references (if any) should also be controlled by an internal
using blocks. All of the mapscript objects implement IDisposable.


Best regards,

Tamas



2007/2/2, Hong <ljfong at sdf.lonestar.org>:
> On Thu, Feb 01, 2007 at 10:47:00PM +0100, Tamas Szekeres wrote:
> > Hi Hong,
> >
> > Are you still getting the problem at the same function or the location
> > may change in a random fashion? Do you have a stacktrace on it?
>
> I'll summarize the various states of MapServer for me so far:
>
> VS2003 compilation on 32-bit environment: stable (no memory corruption at all)
>
> VS2005 compilation on 32-bit environment: behavior is unpredictable, memory
> corruption at will, be that assigning filter string to layer, creating a new
> classification, etc.
>
> VS2003 compilation on 64-bit environment: 100% memory corruption.
>
> VS2005 compilation on 64-bit environment: behavior is unpredictable, memory
> corruption at will. As a matter of curiosity, this happened:
> With debugger attached to the main process running MapServer, I went to the
> 'immediate window' after a breakpoint and did
> '?p_classObj.getExpressionString' a few times. The first few times, I got the
> expression string, on the 4th time or so, access violation happened. Hmmm...
>
> A chunk of my project requires MapServer to run reliably on 64-bit
> environment, seems like that is a no-go for any foreseeable future. Worst of
> all, when memory corruption happened with MapServer, the memory space of
> ASP.NET worker process is messed up as well. Only through restarting the
> worker process is the process back to its "normal" self.
>
> However, I haven't tried hard compiling MapServer in truly 64-bit way, that
> is, through the VS2005 64-bit command prompt. Tried a few times before, but
> failed miserably. This is what I will do next in my attempt to get it to run
> reliably on 64-bit environment.
>
> > At the moment I can think of 3 issues the memory problem might be related
> > to.
> >
> > 1. Usage of different CRT heaps / using different dll versions
> > You should make sure (once again) that only 1 version of the CRT dll
> > is used in the dependency tree of mapscript.dll. In this case
> > msvcr71.dll would be the expected version. If the same dll is
> > referenced from multiple dll-s make sure that the references point to
> > the same location. Make sure that the same dll has been loaded as you
> > have compiled against.
> > Kicking off the unused dependencies are also a good practice. (May be
> > you have possibly done it)
>
> Double-checked all the dependencies. All the DLLs use the same CRT library.
>
> > 2. Early garbage collection
> > Ensure that the lifespan of the child references (eg. layerObj) are
> > not falling outside of the lifespan of the root object (eg. mapObj).
> > Here is an example may cause potential problems:
> >
> > parentObj parent = new parentObj();
> > childObj child = parent.getChildObj();
> > // at this point there's no more references to 'parent' so it might be
> > garbage collected
> > child.OperateOnChild();  // will cause access violation
> >
> > So as to keep the parent alive until a certain point you might use
> > GC.KeepAlive() as.
> >
> > parentObj parent = new parentObj();
> > childObj child = parent.getChildObj();
> > child.OperateOnChild();
> > GC.KeepAlive(parent);
> >
> > will prevent from garbage collecting the parent until the
> > OperateOnChild() returns. Try to add GC.KeepAlive()
> > at the end of the function.
>
> Tried that before, no help. Now, I have no clear idea how GC.KeepAlive() works
> on object that is maintained across ASP.NET session.
>
> > 3. Multithreading issues.
> > To properly isolate this issue a global lock should be added when
> > invoking the mapscript code (practically at mapscript.dll). I will
> > consider how it can be done easily. Until then you can make a test
> > against the builtin shape or the posgis driver to see whether the
> > problem is SDE related or not.
>
> I'm still yet to test this on a postgis driver or just the builtin shape
> driver. I'll keep you posted on what happens when I do.
>
> Another thing curious: I'm also using ESRI ArcIMS in my project and the
> ActiveX connector seems to run just fine in 64-bit environment, without any
> special set up. Granted the thing is also a COM object, and COM object is
> suppported in .NET framework officially. However, I don't feel it is the same
> with pINVOKE, which is responsible for MapServer on .NET. Maybe pINVOKE is
> buggy under 64-bit of windows. My end goal is to ditch ArcIMS (it has its own
> set of problems) completely in my project and have MapServer in its place. I
> think I'm getting there, but still there is some way to go.
>
> I'll keep this thread updated on my progress.
>
> Thanks.
>
> Hong
>
> > 2007/2/1, Hong <ljfong at sdf.lonestar.org>:
> > >An update for my progress:
> > >
> > >After calling clone() before calling draw() of mapObj, MapServer became
> > >more
> > >stable, but under heavier load (many more dynamic layers and config), the
> > >memory corruption still happened.
> > >
> > >The strange solution I had found was to compile MapServer using Visual
> > >Studio
> > >2003 instead of 2005. After doing so, MapServer no longer suffered memory
> > >corruption. For some reason, the binary produced from VS2003 running under
> > >.NET 2.0 did not have any memory management problem. This was done with the
> > >mapObj still being maintained through session.
> > >
> > >After a lot of brainstorming over this, I have come to the conjecture that
> > >the
> > >garbage collector in the .NET framework might have been the culprit of the
> > >memory corruption. Since MapServer internal objects are managed internally
> > >via
> > >PInvoke, they are unmanaged as far as the garbage collector is concerned.
> > >Somehow between ASP.NET page postbacks, the garbage collector came in and
> > >invalidated the references unmanaged objects prematurely, as they are not
> > >referenced by any managed object. Now, objects compiled for .NET 1.1 (vs
> > >2003)
> > >are somehow treated differently by the garbage collector of .NET 2.0. I
> > >don't
> > >know the exact mechanism. There are ways to control the garbage collector,
> > >however I would rather not explore them.
> > >
> > >I would like to hear any other thoughts/comments.
> > >
> > >Hong
> > >
> > >On Mon, Jan 22, 2007 at 10:39:38PM -0700, Hong wrote:
> > >> I have reopened the bug report.
> > >>
> > >> On Mon, Jan 22, 2007 at 11:12:39PM +0100, Tamas Szekeres wrote:
> > >> > 2007/1/22, Hong <ljfong at sdf.lonestar.org>:
> > >> > >
> > >> > >Now instead of using the mapObj stored in the session, I perform a
> > >clone()
> > >> > >on the mapObj from the session, and use the clone to perform draw().
> > >So
> > >> > >far, I
> > >> > >have not seen any crash after doing this. The only problem I observed
> > >was
> > >> > >that
> > >> > >the symbol transparencies was messed up after subsequent clone + draw.
> > >> > >Maybe
> > >> > >the clone is not perfect, I'm not sure.
> > >> > >
> > >> >
> > >> > That seems strange. You should describe the problem in more detail and
> > >> > reopen the following bug then:
> > >> > http://mapserver.gis.umn.edu/bugs/show_bug.cgi?id=931
> > >> >
> > >> > Please attach the corresponding mapfile, the symbol definition file
> > >> > and the symbol image to this bug.
> > >> >
> > >> > Best Regards,
> > >> >
> > >> > Tamas
> > >
>