MapScript C# Memory Corruption Problem (MapServer 4.10)

Tamas Szekeres szekerest at GMAIL.COM
Thu Feb 15 12:32:57 PST 2007 

Currently I have no appropriate ASP.NET application to make tests
with. If you have some sample and test cases I'm ready to try them
out.

Best Regards,

Tamas


2007/2/15, Hong <ljfong at sdf.lonestar.org>:
> Hi Tamas,
>
> Since you have access to a Windows Server 2003 x64 R2 machine, can you tell me
> your experience running MapServer on it? Did you experience random memory
> corruption the way I did?
>
> Hong
>
> On Thu, Feb 15, 2007 at 11:17:45AM +0100, Tamas Szekeres wrote:
> > Hi Hong,
> >
> > Are you sure you've made an x64 build for your dependent libraries?
> > You should start by enabling only the vital components in the build,
> > and adding the new dependencies one by one after a successful build.
> > The gd library is a vital component for mapserver. I would suggest to
> > set up a configuration enabling only the gd component in nmake.opt.
> >
> > I've attached my mapserver nmake.opt and the gd makefile.vc. I've
> > tested these configurations on a Windows Server 2003 x64 R2 and AMD64.
> >
> > You might also compile the SDE related stuff or obtain an x64
> > compatible library version for linking against.
> >
> >
> > Best regards,
> >
> > Tamas
> >
> >
> >
> > 2007/2/15, Hong <ljfong at sdf.lonestar.org>:
> > >Ok, trying to compile everything in pure 64-bit mode, compilation went
> > >fine up
> > >to the moment of linking:
> > >
> > >link /dll   mapbits.obj maphash.obj mapshape.obj mapxbase.obj...
> > >
> > >that ended up with symbol referencing error:
> > >
> > >...
> > >Creating library mapserver_i.lib and object mapserver_i.exp
> > >mapraster.obj : error LNK2001: unresolved external symbol gdImageDestroy
> > >mapgd.obj : error LNK2001: unresolved external symbol gdImageDestroy
> > >...
> > >mapsde.obj : error LNK2019: unresolved external symbol SE_reginfo_free
> > >referenced
> > >in function msSDELayerGetRowIDColumn
> > >mapsde.obj : error LNK2019: unresolved external symbol
> > >SE_reginfo_get_rowid_column referenced in function msSDELayerGetRowIDColumn
> > >...
> > >
> > >All the libs are in correct places, location hard coded, and passed to
> > >LINK.
> > >Searching on google revealed that someone had similar problem compiling
> > >GDAL
> > >on 64-bit as well. The message on gdal-dev mentioned about the way symbol
> > >names are decorated differently in 64-bit windows. I'm thinking bgd.def
> > >might
> > >have trailing "@nn" that needs to be replaced with something else. Does
> > >anyone
> > >have any experience dealing with this issue?
> > >
> > >The original message on gdal-dev:
> > >
> > >http://lists.maptools.org/pipermail/gdal-dev/2006-May/008885.html
> > >
> > >Thanks.
> > >
> > >Hong
> > >
> > >
> > >On Fri, Feb 02, 2007 at 10:51:21PM +0100, Tamas Szekeres wrote:
> > >> Since this issue seems to be quite an essential it would be useful to
> > >> keep it tracked through a bug report. Would you submit one by
> > >> summarizing the all the information we currently have
> > >> (http://mapserver.gis.umn.edu/bugs/enter_bug.cgi)? Assign the bug to
> > >> the C# binding.
> > >>
> > >> Here are some further tests that would worth trying:
> > >>
> > >> 1. Try to alter the SDE connection pooling behaviour. Adding
> > >> PROCESSING "CLOSE_CONNECTION=DEFER"  would set it on a per layer
> > >> basis. I don't know exactly which would be the best at this case, but
> > >> in fact the connections are retained per threads. Maybe keeping the
> > >> connections alive would affect how often the memory allocations
> > >> related to the connections take place.
> > >>
> > >> 2. I've just added a possibility to use a global lock when invoking
> > >> into the mapscript code. To enable this option you should uncomment
> > >> #define USE_GLOBAL_LOCK in csmodule.i. and recompile the mapscript
> > >> code. This setting would ensure that only one thread will invoke into
> > >> libmap.dll in most cases simultaneously, and could help detecting some
> > >> threading issues, though this solution is not equal as restricting the
> > >> invocation to one thread (like the COM STA model would do for
> > >> example).
> > >>
> > >> 3. Ensure disposing the cloned mapObj explicitly by the "using"
> > >> directive for example, like
> > >>
> > >> using (map = ((mapObj)Session("m_mapObj")).clone() )
> > >> {
> > >>    // operate on map
> > >> }
> > >>
> > >> The child references (if any) should also be controlled by an internal
> > >> using blocks. All of the mapscript objects implement IDisposable.
> > >>
> > >>
> > >> Best regards,
> > >>
> > >> Tamas
> > >>
> > >>
> > >>
> > >> 2007/2/2, Hong <ljfong at sdf.lonestar.org>:
> > >> >On Thu, Feb 01, 2007 at 10:47:00PM +0100, Tamas Szekeres wrote:
> > >> >> Hi Hong,
> > >> >>
> > >> >> Are you still getting the problem at the same function or the location
> > >> >> may change in a random fashion? Do you have a stacktrace on it?
> > >> >
> > >> >I'll summarize the various states of MapServer for me so far:
> > >> >
> > >> >VS2003 compilation on 32-bit environment: stable (no memory corruption
> > >at
> > >> >all)
> > >> >
> > >> >VS2005 compilation on 32-bit environment: behavior is unpredictable,
> > >memory
> > >> >corruption at will, be that assigning filter string to layer, creating a
> > >> >new
> > >> >classification, etc.
> > >> >
> > >> >VS2003 compilation on 64-bit environment: 100% memory corruption.
> > >> >
> > >> >VS2005 compilation on 64-bit environment: behavior is unpredictable,
> > >memory
> > >> >corruption at will. As a matter of curiosity, this happened:
> > >> >With debugger attached to the main process running MapServer, I went to
> > >the
> > >> >'immediate window' after a breakpoint and did
> > >> >'?p_classObj.getExpressionString' a few times. The first few times, I
> > >got
> > >> >the
> > >> >expression string, on the 4th time or so, access violation happened.
> > >> >Hmmm...
> > >> >
> > >> >A chunk of my project requires MapServer to run reliably on 64-bit
> > >> >environment, seems like that is a no-go for any foreseeable future.
> > >Worst
> > >> >of
> > >> >all, when memory corruption happened with MapServer, the memory space of
> > >> >ASP.NET worker process is messed up as well. Only through restarting the
> > >> >worker process is the process back to its "normal" self.
> > >> >
> > >> >However, I haven't tried hard compiling MapServer in truly 64-bit way,
> > >that
> > >> >is, through the VS2005 64-bit command prompt. Tried a few times before,
> > >but
> > >> >failed miserably. This is what I will do next in my attempt to get it to
> > >> >run
> > >> >reliably on 64-bit environment.
> > >> >
> > >> >> At the moment I can think of 3 issues the memory problem might be
> > >related
> > >> >> to.
> > >> >>
> > >> >> 1. Usage of different CRT heaps / using different dll versions
> > >> >> You should make sure (once again) that only 1 version of the CRT dll
> > >> >> is used in the dependency tree of mapscript.dll. In this case
> > >> >> msvcr71.dll would be the expected version. If the same dll is
> > >> >> referenced from multiple dll-s make sure that the references point to
> > >> >> the same location. Make sure that the same dll has been loaded as you
> > >> >> have compiled against.
> > >> >> Kicking off the unused dependencies are also a good practice. (May be
> > >> >> you have possibly done it)
> > >> >
> > >> >Double-checked all the dependencies. All the DLLs use the same CRT
> > >library.
> > >> >
> > >> >> 2. Early garbage collection
> > >> >> Ensure that the lifespan of the child references (eg. layerObj) are
> > >> >> not falling outside of the lifespan of the root object (eg. mapObj).
> > >> >> Here is an example may cause potential problems:
> > >> >>
> > >> >> parentObj parent = new parentObj();
> > >> >> childObj child = parent.getChildObj();
> > >> >> // at this point there's no more references to 'parent' so it might be
> > >> >> garbage collected
> > >> >> child.OperateOnChild();  // will cause access violation
> > >> >>
> > >> >> So as to keep the parent alive until a certain point you might use
> > >> >> GC.KeepAlive() as.
> > >> >>
> > >> >> parentObj parent = new parentObj();
> > >> >> childObj child = parent.getChildObj();
> > >> >> child.OperateOnChild();
> > >> >> GC.KeepAlive(parent);
> > >> >>
> > >> >> will prevent from garbage collecting the parent until the
> > >> >> OperateOnChild() returns. Try to add GC.KeepAlive()
> > >> >> at the end of the function.
> > >> >
> > >> >Tried that before, no help. Now, I have no clear idea how GC.KeepAlive()
> > >> >works
> > >> >on object that is maintained across ASP.NET session.
> > >> >
> > >> >> 3. Multithreading issues.
> > >> >> To properly isolate this issue a global lock should be added when
> > >> >> invoking the mapscript code (practically at mapscript.dll). I will
> > >> >> consider how it can be done easily. Until then you can make a test
> > >> >> against the builtin shape or the posgis driver to see whether the
> > >> >> problem is SDE related or not.
> > >> >
> > >> >I'm still yet to test this on a postgis driver or just the builtin shape
> > >> >driver. I'll keep you posted on what happens when I do.
> > >> >
> > >> >Another thing curious: I'm also using ESRI ArcIMS in my project and the
> > >> >ActiveX connector seems to run just fine in 64-bit environment, without
> > >any
> > >> >special set up. Granted the thing is also a COM object, and COM object
> > >is
> > >> >suppported in .NET framework officially. However, I don't feel it is the
> > >> >same
> > >> >with pINVOKE, which is responsible for MapServer on .NET. Maybe pINVOKE
> > >is
> > >> >buggy under 64-bit of windows. My end goal is to ditch ArcIMS (it has
> > >its
> > >> >own
> > >> >set of problems) completely in my project and have MapServer in its
> > >place.
> > >> >I
> > >> >think I'm getting there, but still there is some way to go.
> > >> >
> > >> >I'll keep this thread updated on my progress.
> > >> >
> > >> >Thanks.
> > >> >
> > >> >Hong
> > >> >
> > >> >> 2007/2/1, Hong <ljfong at sdf.lonestar.org>:
> > >> >> >An update for my progress:
> > >> >> >
> > >> >> >After calling clone() before calling draw() of mapObj, MapServer
> > >became
> > >> >> >more
> > >> >> >stable, but under heavier load (many more dynamic layers and config),
> > >> >the
> > >> >> >memory corruption still happened.
> > >> >> >
> > >> >> >The strange solution I had found was to compile MapServer using
> > >Visual
> > >> >> >Studio
> > >> >> >2003 instead of 2005. After doing so, MapServer no longer suffered
> > >> >memory
> > >> >> >corruption. For some reason, the binary produced from VS2003 running
> > >> >under
> > >> >> >.NET 2.0 did not have any memory management problem. This was done
> > >with
> > >> >the
> > >> >> >mapObj still being maintained through session.
> > >> >> >
> > >> >> >After a lot of brainstorming over this, I have come to the conjecture
> > >> >that
> > >> >> >the
> > >> >> >garbage collector in the .NET framework might have been the culprit
> > >of
> > >> >the
> > >> >> >memory corruption. Since MapServer internal objects are managed
> > >> >internally
> > >> >> >via
> > >> >> >PInvoke, they are unmanaged as far as the garbage collector is
> > >> >concerned.
> > >> >> >Somehow between ASP.NET page postbacks, the garbage collector came in
> > >> >and
> > >> >> >invalidated the references unmanaged objects prematurely, as they are
> > >> >not
> > >> >> >referenced by any managed object. Now, objects compiled for .NET 1.1
> > >(vs
> > >> >> >2003)
> > >> >> >are somehow treated differently by the garbage collector of .NET
> > >2.0. I
> > >> >> >don't
> > >> >> >know the exact mechanism. There are ways to control the garbage
> > >> >collector,
> > >> >> >however I would rather not explore them.
> > >> >> >
> > >> >> >I would like to hear any other thoughts/comments.
> > >> >> >
> > >> >> >Hong
> > >> >> >
> > >> >> >On Mon, Jan 22, 2007 at 10:39:38PM -0700, Hong wrote:
> > >> >> >> I have reopened the bug report.
> > >> >> >>
> > >> >> >> On Mon, Jan 22, 2007 at 11:12:39PM +0100, Tamas Szekeres wrote:
> > >> >> >> > 2007/1/22, Hong <ljfong at sdf.lonestar.org>:
> > >> >> >> > >
> > >> >> >> > >Now instead of using the mapObj stored in the session, I
> > >perform a
> > >> >> >clone()
> > >> >> >> > >on the mapObj from the session, and use the clone to perform
> > >> >draw().
> > >> >> >So
> > >> >> >> > >far, I
> > >> >> >> > >have not seen any crash after doing this. The only problem I
> > >> >observed
> > >> >> >was
> > >> >> >> > >that
> > >> >> >> > >the symbol transparencies was messed up after subsequent clone +
> > >> >draw.
> > >> >> >> > >Maybe
> > >> >> >> > >the clone is not perfect, I'm not sure.
> > >> >> >> > >
> > >> >> >> >
> > >> >> >> > That seems strange. You should describe the problem in more
> > >detail
> > >> >and
> > >> >> >> > reopen the following bug then:
> > >> >> >> > http://mapserver.gis.umn.edu/bugs/show_bug.cgi?id=931
> > >> >> >> >
> > >> >> >> > Please attach the corresponding mapfile, the symbol definition
> > >file
> > >> >> >> > and the symbol image to this bug.
> > >> >> >> >
> > >> >> >> > Best Regards,
> > >> >> >> >
> > >> >> >> > Tamas
> > >> >> >
> > >> >
> > >
>

More information about the MapServer-users mailing list