[mapguide-internals] Memory leakage in WebTier (Java)

Walt Welton-Lair walt.welton-lair at autodesk.com
Thu Sep 2 10:05:52 EDT 2010


Why should the GC order matter at all?

- case 1: src gets GC'ed first, followed by layerDefContent
        When the src gets GC'ed the ref count of the associated C++ object drops back to one - no C++ objects are yet destroyed.  Then when the layerDefContent object gets GC'ed the ref count for its C+ object goes to zero and it gets destroyed, which in turn will release the final reference to the src C++ object and it will also get destroyed.

- case 2: layerDefContent gets GC'ed first, followed by src
        When the layerDefContent gets GC'ed the ref count for its C+ object goes to zero and it gets destroyed, and the ref count of the src C++ object drops back to one.  Then when the src object gets GC'ed the ref count for the src C+ object goes to zero and it also gets destroyed.


It seems like some of the managed objects are not getting GC'ed, which in turn is causing the underlying C++ objects to not be destroyed.

Walt


-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Bruce Dechant
Sent: Wednesday, September 01, 2010 5:38 PM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] Memory leakage in WebTier (Java)

Andreas,

Isn't Gc fun :)

You could try altering your code below to control the scope better - this in turn may force the MgByteReader to be GCed 1st.

Ie:

try { 
	MgByteSource src = new MgByteSource(b, b.length);
      {
	    MgByteReader layerDefContent = src.GetReader();
	    resourceService.SetResource(newLayerResId, layerDefContent, null);
      }
      layer.SetLayerDefinition(newLayerResId, resourceService);
	layer.ForceRefresh();
} catch
...

It has been awhile since I have tackled these issues so the above may make no difference.

Thanks,
Bruce

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Andreas Morf
Sent: Wednesday, September 01, 2010 3:31 PM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] Memory leakage in WebTier (Java)

Bruce,
I think this is done within the same scope:

try { 
	MgByteSource src = new MgByteSource(b, b.length);
	MgByteReader layerDefContent = src.GetReader();
	resourceService.SetResource(newLayerResId, layerDefContent, null);
	layer.SetLayerDefinition(newLayerResId, resourceService);
	layer.ForceRefresh();
} catch
...

But with java-gc i cannot guess which object is gc'd first and when and IMHO
there is a problem of not having references between mgjavaobjects resulting
in gc'ing them without disposing all underlying c++ objects (and of course
not having a gc on c++ heap cleaning up all orphaned crap after a while).

Andreas

-----Ursprüngliche Nachricht-----
Von: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] Im Auftrag von Bruce
Dechant
Gesendet: Mittwoch, 1. September 2010 22:16
An: MapGuide Internals Mail List
Betreff: RE: [mapguide-internals] Memory leakage in WebTier (Java)

Andreas,

In the case you have below if you Release the MgByteReader object before the
src object gets GCed it will reduce the reference count to 0 and call the
Dispose method to delete the object properly. It looks like you are getting
the MgByteReader object and hanging on to it past the lifetime of the src
object.
Is it possible to alter your code so that you change the sequence to finish
using the MgByteReader object before you go out of scope for your src
object?

In general readers should be cleaned up as soon as you are finished with
them.

Thanks,
Bruce

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Andreas
Morf
Sent: Wednesday, September 01, 2010 2:00 PM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] Memory leakage in WebTier (Java)

Bruce,
I am using a 2.2 (trunk) compiled with vc10 - the only difference is that i
updated to the newest versions of ACE, dbxml and geos. Indeed, mgserver is
really stable and fast (plus 20-30% compared to 2.1) with all optimisations
activated in vc project options.

Back to the example mentioned in my first email: can you explain me, how the
orphaned c++ objects (which are not on the java heap) are freed/deleted
after the java (SWIG) objects are gc'd?

Andreas



-----Ursprüngliche Nachricht-----
Von: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] Im Auftrag von Bruce
Dechant
Gesendet: Mittwoch, 1. September 2010 21:34
An: MapGuide Internals Mail List
Betreff: RE: [mapguide-internals] Memory leakage in WebTier (Java)

Andreas,

That is odd - under load testing without the setting below I would easily
see the java heap get dangerously close to 2GB, but with the change it stays
around 100MB or so.
So it fixed the memory leak issues I was seeing with MGOS 2.2

My next question is what version of MGOS are you using?
The 2.2 release is really stable compared to 2.1 and prior releases.

Thanks,
Bruce


-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Andreas
Morf
Sent: Wednesday, September 01, 2010 1:27 PM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] Memory leakage in WebTier (Java)

Bruce,
I recently installed tomcat 6.0.29 and inserted your suggested entry -
unfortunately leakage remains the same. The java heap is not increasing and
the memory profiler shows that the orphaned MgXXX objects are allocated by
Mg-dll's.

Thanks,
Andreas

-----Ursprüngliche Nachricht-----
Von: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] Im Auftrag von Bruce
Dechant
Gesendet: Mittwoch, 1. September 2010 18:00
An: MapGuide Internals Mail List
Betreff: RE: [mapguide-internals] Memory leakage in WebTier (Java)

Andreas,

What version of Tomcat are you using? 
I ask because there is a known Tomcat issue with memory leak prevention not
being enabled by default.

Regardless, please check that the following entry is in the Tomcat
server.xml file. This entry fixes lots of Java memory leaks.

  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />

Thanks,
Bruce


-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Andreas
Morf
Sent: Wednesday, September 01, 2010 7:12 AM
To: mapguide-internals at lists.osgeo.org
Subject: [mapguide-internals] Memory leakage in WebTier (Java)

Hi all
I'm intensively using the java webtier and with more users accessing
mapguide this is getting a real problem:

After every call to a servlet using MgApi the memory consumption of
javaw.exe (tomcat) is increasing even after GC. I used a memory
profiler on javaw and saw that almost all the MgXXX objects remain in
memory.
After some more investigations with the debugger I remarked the following
pattern which I can describe on an example:

...
MgByteSource src = new MgByteSource(b, b.length);
MgByteReader layerDefContent = src.GetReader();
...

When gc deletes the src-object the MgGuardDisposable::Release() is called by
JNI. Because of the 'src.GetReader()' call m_refCount
has a value of 2, so 'Release()' just decreases it by 1 and returns to JAVA
without calling the Dispose() method of MgByteSource. So
the java-object is deleted while the C++ object remains orphaned in memory.
This pattern is happening with almost every MgApi usage and my
LoadMap-Servlet which creates a new MgMap and saves all Layers with a
filter to the Session-Repository every run leads to a leakage of around 5MB
(MgMap has quite a large cascade of (not just small)
child objects).

Can anyone confirm my diagnosis and maybe give a hint where/how to fix this
problem and is this maybe a general memory management
issue?

Greets, Andreas


_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals


More information about the mapguide-internals mailing list