[MAPSERVER-USERS] leaking memory in java mapscript (was Re: thread safety)

[rich.fromm]

rich.fromm wrote:
> 
> Other than calling delete() on every mapserver object as applicable, is
> there
> anything else that I can be doing?  I fear that there may be a real
> genuine
> leak within mapserver, and that that might be hard to track down.
> 

I have narrowed this down to a smaller testcase, that only has two mapscript
objects.

I have a fixed map file, and I'm doing barely anything, just creating the
map
object from the map file, drawing to the image object, and getting the
bytes.
I am calling both map.delete() and img.delete() at the end.  Then I loop
over
all of this repeatedly.

This is a little bit of a simplification, but essentially what I'm doing is
this:

      for (int i = 0; i < loop; i++) {
         mapObj map = null;
         imageObj img = null;

         try {
            map = new mapObj(filename);
            img = map.draw();
            byte[] bytes = img.getBytes();
         } catch (Exception e) {
            System.err.println("Caught exception: " + e);
            e.printStackTrace(System.err);
         } finally {
            if (img != null) {
               img.delete();
            }
            if (map != null) {
               map.delete();
            }
         }
         System.gc();
      } // end for

I have tried a number of variants of this (including outputting the bytes to
a
file), and the offending line seems to be the call to imageObj.getBytes().
With it, my program leaks.  Without it, it does not:

[rich at peyote mapserver_test 17:53:31]$ <mapserver_test> diff -u
java/src/TestMapscript.java.bak1 java/src/TestMapscript.java.bak3
--- java/src/TestMapscript.java.bak1	2008-02-20 17:37:43.000000000 -0800
+++ java/src/TestMapscript.java.bak3	2008-02-20 17:53:27.000000000 -0800
@@ -1,4 +1,4 @@
-// this code does NOT leak
+// this code DOES leak
 
 import edu.umn.gis.mapscript.mapObj;
 import edu.umn.gis.mapscript.imageObj;
@@ -47,7 +47,7 @@
             System.err.println("Drawing map to image object");
             img = map.draw();
 
-//             byte[] bytes = img.getBytes();
+            byte[] bytes = img.getBytes();
 //             int width = img.getWidth();
 //             int height = img.getHeight();
 //             System.err.println("The image is of size " + width + " x " +
height + " and contains " + bytes.length + " bytes");
[rich at peyote mapserver_test 17:53:53]$ <mapserver_test> 

I am calling System.gc() at the end of each iteration.  And by examining the
memory from java's perspective (Runtime.getRuntime().freeMemory() and
Runtime.getRuntime().totalMemory()) I can see that it is reasonably stable.
However, the memory as reported by top gradually grows.

The source for imageObj.getBytes() is as follows:

mapscript/java/edu/umn/gis/mapscript/imageObj.java

  public byte[] getBytes() {
    return mapscriptJNI.imageObj_getBytes(swigCPtr);
  }

I don't totally understand how all of the SWIG stuff works, but I can follow
it a bit.  The source for this is:

  getBytes() (mapscript/swiginc/image.i)

which notes the following:

    /*
   
-------------------------------------------------------------------------
    getBytes returns a gdBuffer structure (defined in mapscript.i) which
must
    be typemapped to an object appropriate to the target language.  This
    typemap must also gdFree the data member of the gdBuffer.  See the type-
    maps in java/javamodule.i and python/pymodule.i for examples.

    contributed by Jerry Pisk, jerry.pisk at gmail.com
   
-------------------------------------------------------------------------
    */

And all of the following:

  msSaveImageBuffer() (maputil.c)
  msSaveImageBufferAGG() (mapagg.cpp)
  msSaveImageBufferGD() (mapgd.c)

confirm that the caller must free the returned array (with gdFree()).

What's not entirely clear to me is whether "the caller" means the java
mapscript code, or the application writer.  In other words, am I responsible
for doing this, or should this happen when I call img.delete() ?  Here is
the
code for imageObj.delete():

mapscript/java/edu/umn/gis/mapscript/imageObj.java

  public void delete() {
    if(swigCPtr != 0 && swigCMemOwn) {
      swigCMemOwn = false;
      mapscriptJNI.delete_imageObj(swigCPtr);
    }
    swigCPtr = 0;
  }

But even though I don't see a call to gdFree() there, it's not clear to me
that it's not happening somewhere.  I don't understand the SWIG path enough,
and I haven't yet tried following it in gdb.  (Although that might be worth
trying.)

If I am responsible for doing this, how do I do it?  It appears to me that
gdFree() is not part of the mapscript API (it doesn't appear in the docs),
and
just that it is just a mapscript internal function.

Right now my test case is using a fairly complicated mapfile from a large
data
source located in a postgis database.  I will try to see if I can replicate
this problem with a simpler map file and/or synthetic data.

But in the mean time I wanted to post this in case anyone might have some
advice based on these observations.

Once again, thanks for any help that anyone might be able to provide.

- Rich

p.s. This is all based on mapserver 5.0.0

-- 
View this message in context: http://www.nabble.com/thread-safety-tp15514019p15603603.html
Sent from the Mapserver - User mailing list archive at Nabble.com.