MapScript C# Memory Corruption Problem (MapServer 4.10)

[Hong]

Yeah, I'll do just that this weekend. Meanwhile I'll try out your suggestions
and I'll update again afterwards.

Thanks.

Hong

On Fri, Feb 02, 2007 at 10:51:21PM +0100, Tamas Szekeres wrote:
> Since this issue seems to be quite an essential it would be useful to
> keep it tracked through a bug report. Would you submit one by
> summarizing the all the information we currently have
> (http://mapserver.gis.umn.edu/bugs/enter_bug.cgi)? Assign the bug to
> the C# binding.
> 
> Here are some further tests that would worth trying:
> 
> 1. Try to alter the SDE connection pooling behaviour. Adding
> PROCESSING "CLOSE_CONNECTION=DEFER"  would set it on a per layer
> basis. I don't know exactly which would be the best at this case, but
> in fact the connections are retained per threads. Maybe keeping the
> connections alive would affect how often the memory allocations
> related to the connections take place.
> 
> 2. I've just added a possibility to use a global lock when invoking
> into the mapscript code. To enable this option you should uncomment
> #define USE_GLOBAL_LOCK in csmodule.i. and recompile the mapscript
> code. This setting would ensure that only one thread will invoke into
> libmap.dll in most cases simultaneously, and could help detecting some
> threading issues, though this solution is not equal as restricting the
> invocation to one thread (like the COM STA model would do for
> example).
> 
> 3. Ensure disposing the cloned mapObj explicitly by the "using"
> directive for example, like
> 
> using (map = ((mapObj)Session("m_mapObj")).clone() )
> {
>    // operate on map
> }
> 
> The child references (if any) should also be controlled by an internal
> using blocks. All of the mapscript objects implement IDisposable.
> 
> 
> Best regards,
> 
> Tamas
> 
> 
> 
> 2007/2/2, Hong <ljfong at sdf.lonestar.org>:
> >On Thu, Feb 01, 2007 at 10:47:00PM +0100, Tamas Szekeres wrote:
> >> Hi Hong,
> >>
> >> Are you still getting the problem at the same function or the location
> >> may change in a random fashion? Do you have a stacktrace on it?
> >
> >I'll summarize the various states of MapServer for me so far:
> >
> >VS2003 compilation on 32-bit environment: stable (no memory corruption at 
> >all)
> >
> >VS2005 compilation on 32-bit environment: behavior is unpredictable, memory
> >corruption at will, be that assigning filter string to layer, creating a 
> >new
> >classification, etc.
> >
> >VS2003 compilation on 64-bit environment: 100% memory corruption.
> >
> >VS2005 compilation on 64-bit environment: behavior is unpredictable, memory
> >corruption at will. As a matter of curiosity, this happened:
> >With debugger attached to the main process running MapServer, I went to the
> >'immediate window' after a breakpoint and did
> >'?p_classObj.getExpressionString' a few times. The first few times, I got 
> >the
> >expression string, on the 4th time or so, access violation happened. 
> >Hmmm...
> >
> >A chunk of my project requires MapServer to run reliably on 64-bit
> >environment, seems like that is a no-go for any foreseeable future. Worst 
> >of
> >all, when memory corruption happened with MapServer, the memory space of
> >ASP.NET worker process is messed up as well. Only through restarting the
> >worker process is the process back to its "normal" self.
> >
> >However, I haven't tried hard compiling MapServer in truly 64-bit way, that
> >is, through the VS2005 64-bit command prompt. Tried a few times before, but
> >failed miserably. This is what I will do next in my attempt to get it to 
> >run
> >reliably on 64-bit environment.
> >
> >> At the moment I can think of 3 issues the memory problem might be related
> >> to.
> >>
> >> 1. Usage of different CRT heaps / using different dll versions
> >> You should make sure (once again) that only 1 version of the CRT dll
> >> is used in the dependency tree of mapscript.dll. In this case
> >> msvcr71.dll would be the expected version. If the same dll is
> >> referenced from multiple dll-s make sure that the references point to
> >> the same location. Make sure that the same dll has been loaded as you
> >> have compiled against.
> >> Kicking off the unused dependencies are also a good practice. (May be
> >> you have possibly done it)
> >
> >Double-checked all the dependencies. All the DLLs use the same CRT library.
> >
> >> 2. Early garbage collection
> >> Ensure that the lifespan of the child references (eg. layerObj) are
> >> not falling outside of the lifespan of the root object (eg. mapObj).
> >> Here is an example may cause potential problems:
> >>
> >> parentObj parent = new parentObj();
> >> childObj child = parent.getChildObj();
> >> // at this point there's no more references to 'parent' so it might be
> >> garbage collected
> >> child.OperateOnChild();  // will cause access violation
> >>
> >> So as to keep the parent alive until a certain point you might use
> >> GC.KeepAlive() as.
> >>
> >> parentObj parent = new parentObj();
> >> childObj child = parent.getChildObj();
> >> child.OperateOnChild();
> >> GC.KeepAlive(parent);
> >>
> >> will prevent from garbage collecting the parent until the
> >> OperateOnChild() returns. Try to add GC.KeepAlive()
> >> at the end of the function.
> >
> >Tried that before, no help. Now, I have no clear idea how GC.KeepAlive() 
> >works
> >on object that is maintained across ASP.NET session.
> >
> >> 3. Multithreading issues.
> >> To properly isolate this issue a global lock should be added when
> >> invoking the mapscript code (practically at mapscript.dll). I will
> >> consider how it can be done easily. Until then you can make a test
> >> against the builtin shape or the posgis driver to see whether the
> >> problem is SDE related or not.
> >
> >I'm still yet to test this on a postgis driver or just the builtin shape
> >driver. I'll keep you posted on what happens when I do.
> >
> >Another thing curious: I'm also using ESRI ArcIMS in my project and the
> >ActiveX connector seems to run just fine in 64-bit environment, without any
> >special set up. Granted the thing is also a COM object, and COM object is
> >suppported in .NET framework officially. However, I don't feel it is the 
> >same
> >with pINVOKE, which is responsible for MapServer on .NET. Maybe pINVOKE is
> >buggy under 64-bit of windows. My end goal is to ditch ArcIMS (it has its 
> >own
> >set of problems) completely in my project and have MapServer in its place. 
> >I
> >think I'm getting there, but still there is some way to go.
> >
> >I'll keep this thread updated on my progress.
> >
> >Thanks.
> >
> >Hong
> >
> >> 2007/2/1, Hong <ljfong at sdf.lonestar.org>:
> >> >An update for my progress:
> >> >
> >> >After calling clone() before calling draw() of mapObj, MapServer became
> >> >more
> >> >stable, but under heavier load (many more dynamic layers and config), 
> >the
> >> >memory corruption still happened.
> >> >
> >> >The strange solution I had found was to compile MapServer using Visual
> >> >Studio
> >> >2003 instead of 2005. After doing so, MapServer no longer suffered 
> >memory
> >> >corruption. For some reason, the binary produced from VS2003 running 
> >under
> >> >.NET 2.0 did not have any memory management problem. This was done with 
> >the
> >> >mapObj still being maintained through session.
> >> >
> >> >After a lot of brainstorming over this, I have come to the conjecture 
> >that
> >> >the
> >> >garbage collector in the .NET framework might have been the culprit of 
> >the
> >> >memory corruption. Since MapServer internal objects are managed 
> >internally
> >> >via
> >> >PInvoke, they are unmanaged as far as the garbage collector is 
> >concerned.
> >> >Somehow between ASP.NET page postbacks, the garbage collector came in 
> >and
> >> >invalidated the references unmanaged objects prematurely, as they are 
> >not
> >> >referenced by any managed object. Now, objects compiled for .NET 1.1 (vs
> >> >2003)
> >> >are somehow treated differently by the garbage collector of .NET 2.0. I
> >> >don't
> >> >know the exact mechanism. There are ways to control the garbage 
> >collector,
> >> >however I would rather not explore them.
> >> >
> >> >I would like to hear any other thoughts/comments.
> >> >
> >> >Hong
> >> >
> >> >On Mon, Jan 22, 2007 at 10:39:38PM -0700, Hong wrote:
> >> >> I have reopened the bug report.
> >> >>
> >> >> On Mon, Jan 22, 2007 at 11:12:39PM +0100, Tamas Szekeres wrote:
> >> >> > 2007/1/22, Hong <ljfong at sdf.lonestar.org>:
> >> >> > >
> >> >> > >Now instead of using the mapObj stored in the session, I perform a
> >> >clone()
> >> >> > >on the mapObj from the session, and use the clone to perform 
> >draw().
> >> >So
> >> >> > >far, I
> >> >> > >have not seen any crash after doing this. The only problem I 
> >observed
> >> >was
> >> >> > >that
> >> >> > >the symbol transparencies was messed up after subsequent clone + 
> >draw.
> >> >> > >Maybe
> >> >> > >the clone is not perfect, I'm not sure.
> >> >> > >
> >> >> >
> >> >> > That seems strange. You should describe the problem in more detail 
> >and
> >> >> > reopen the following bug then:
> >> >> > http://mapserver.gis.umn.edu/bugs/show_bug.cgi?id=931
> >> >> >
> >> >> > Please attach the corresponding mapfile, the symbol definition file
> >> >> > and the symbol image to this bug.
> >> >> >
> >> >> > Best Regards,
> >> >> >
> >> >> > Tamas
> >> >
> >