MapScript C# Memory Corruption Problem (MapServer 4.10)

[Hong]

On Thu, Feb 01, 2007 at 10:47:00PM +0100, Tamas Szekeres wrote:
> Hi Hong,
> 
> Are you still getting the problem at the same function or the location
> may change in a random fashion? Do you have a stacktrace on it?

I'll summarize the various states of MapServer for me so far:

VS2003 compilation on 32-bit environment: stable (no memory corruption at all)

VS2005 compilation on 32-bit environment: behavior is unpredictable, memory
corruption at will, be that assigning filter string to layer, creating a new
classification, etc.

VS2003 compilation on 64-bit environment: 100% memory corruption.

VS2005 compilation on 64-bit environment: behavior is unpredictable, memory
corruption at will. As a matter of curiosity, this happened:
With debugger attached to the main process running MapServer, I went to the
'immediate window' after a breakpoint and did
'?p_classObj.getExpressionString' a few times. The first few times, I got the
expression string, on the 4th time or so, access violation happened. Hmmm... 

A chunk of my project requires MapServer to run reliably on 64-bit
environment, seems like that is a no-go for any foreseeable future. Worst of
all, when memory corruption happened with MapServer, the memory space of
ASP.NET worker process is messed up as well. Only through restarting the
worker process is the process back to its "normal" self.

However, I haven't tried hard compiling MapServer in truly 64-bit way, that
is, through the VS2005 64-bit command prompt. Tried a few times before, but
failed miserably. This is what I will do next in my attempt to get it to run
reliably on 64-bit environment.

> At the moment I can think of 3 issues the memory problem might be related 
> to.
> 
> 1. Usage of different CRT heaps / using different dll versions
> You should make sure (once again) that only 1 version of the CRT dll
> is used in the dependency tree of mapscript.dll. In this case
> msvcr71.dll would be the expected version. If the same dll is
> referenced from multiple dll-s make sure that the references point to
> the same location. Make sure that the same dll has been loaded as you
> have compiled against.
> Kicking off the unused dependencies are also a good practice. (May be
> you have possibly done it)

Double-checked all the dependencies. All the DLLs use the same CRT library.

> 2. Early garbage collection
> Ensure that the lifespan of the child references (eg. layerObj) are
> not falling outside of the lifespan of the root object (eg. mapObj).
> Here is an example may cause potential problems:
> 
> parentObj parent = new parentObj();
> childObj child = parent.getChildObj();
> // at this point there's no more references to 'parent' so it might be
> garbage collected
> child.OperateOnChild();  // will cause access violation
> 
> So as to keep the parent alive until a certain point you might use
> GC.KeepAlive() as.
> 
> parentObj parent = new parentObj();
> childObj child = parent.getChildObj();
> child.OperateOnChild();
> GC.KeepAlive(parent);
> 
> will prevent from garbage collecting the parent until the
> OperateOnChild() returns. Try to add GC.KeepAlive()
> at the end of the function.

Tried that before, no help. Now, I have no clear idea how GC.KeepAlive() works
on object that is maintained across ASP.NET session.

> 3. Multithreading issues.
> To properly isolate this issue a global lock should be added when
> invoking the mapscript code (practically at mapscript.dll). I will
> consider how it can be done easily. Until then you can make a test
> against the builtin shape or the posgis driver to see whether the
> problem is SDE related or not.

I'm still yet to test this on a postgis driver or just the builtin shape
driver. I'll keep you posted on what happens when I do.

Another thing curious: I'm also using ESRI ArcIMS in my project and the
ActiveX connector seems to run just fine in 64-bit environment, without any
special set up. Granted the thing is also a COM object, and COM object is
suppported in .NET framework officially. However, I don't feel it is the same
with pINVOKE, which is responsible for MapServer on .NET. Maybe pINVOKE is
buggy under 64-bit of windows. My end goal is to ditch ArcIMS (it has its own
set of problems) completely in my project and have MapServer in its place. I
think I'm getting there, but still there is some way to go.

I'll keep this thread updated on my progress.

Thanks.

Hong

> 2007/2/1, Hong <ljfong at sdf.lonestar.org>:
> >An update for my progress:
> >
> >After calling clone() before calling draw() of mapObj, MapServer became 
> >more
> >stable, but under heavier load (many more dynamic layers and config), the
> >memory corruption still happened.
> >
> >The strange solution I had found was to compile MapServer using Visual 
> >Studio
> >2003 instead of 2005. After doing so, MapServer no longer suffered memory
> >corruption. For some reason, the binary produced from VS2003 running under
> >.NET 2.0 did not have any memory management problem. This was done with the
> >mapObj still being maintained through session.
> >
> >After a lot of brainstorming over this, I have come to the conjecture that 
> >the
> >garbage collector in the .NET framework might have been the culprit of the
> >memory corruption. Since MapServer internal objects are managed internally 
> >via
> >PInvoke, they are unmanaged as far as the garbage collector is concerned.
> >Somehow between ASP.NET page postbacks, the garbage collector came in and
> >invalidated the references unmanaged objects prematurely, as they are not
> >referenced by any managed object. Now, objects compiled for .NET 1.1 (vs 
> >2003)
> >are somehow treated differently by the garbage collector of .NET 2.0. I 
> >don't
> >know the exact mechanism. There are ways to control the garbage collector,
> >however I would rather not explore them.
> >
> >I would like to hear any other thoughts/comments.
> >
> >Hong
> >
> >On Mon, Jan 22, 2007 at 10:39:38PM -0700, Hong wrote:
> >> I have reopened the bug report.
> >>
> >> On Mon, Jan 22, 2007 at 11:12:39PM +0100, Tamas Szekeres wrote:
> >> > 2007/1/22, Hong <ljfong at sdf.lonestar.org>:
> >> > >
> >> > >Now instead of using the mapObj stored in the session, I perform a 
> >clone()
> >> > >on the mapObj from the session, and use the clone to perform draw(). 
> >So
> >> > >far, I
> >> > >have not seen any crash after doing this. The only problem I observed 
> >was
> >> > >that
> >> > >the symbol transparencies was messed up after subsequent clone + draw.
> >> > >Maybe
> >> > >the clone is not perfect, I'm not sure.
> >> > >
> >> >
> >> > That seems strange. You should describe the problem in more detail and
> >> > reopen the following bug then:
> >> > http://mapserver.gis.umn.edu/bugs/show_bug.cgi?id=931
> >> >
> >> > Please attach the corresponding mapfile, the symbol definition file
> >> > and the symbol image to this bug.
> >> >
> >> > Best Regards,
> >> >
> >> > Tamas
> >