Antwort: Re: [MAPSERVER-USERS] leaking memory in java mapscript (was Re: thread safety)

umn-ms at hydrotec.de umn-ms at hydrotec.de
Wed Feb 20 23:36:54 PST 2008


Hi

> I don't totally understand how all of the SWIG stuff works, but I can 
follow
> it a bit.  The source for this is:
It's easier to watch the SWIG-generated JNI-Code in 
mapscript/java/mapscript_wrap.c 
Example: 
  mapscriptJNI.delete_imageObj(swigCPtr);
is implemented by the C-function
  SWIGEXPORT jbyteArray JNICALL 
Java_edu_umn_gis_mapscript_mapscriptJNI_imageObj_1getBytes 

> confirm that the caller must free the returned array (with gdFree()).
At the end of the long-named C-function you'll find a call to gdFree

Can you estimate how much memory is lost in each step of your
testprogram? Is it approx. the size of the image?

Benedikt Rothe


mapserver-users-bounces at lists.osgeo.org schrieb am 21.02.2008 03:31:09:

> 
> 
> rich.fromm wrote:
> > 
> > Other than calling delete() on every mapserver object as applicable, 
is
> > there
> > anything else that I can be doing?  I fear that there may be a real
> > genuine
> > leak within mapserver, and that that might be hard to track down.
> > 
> 
> I have narrowed this down to a smaller testcase, that only has two 
mapscript
> objects.
> 
> I have a fixed map file, and I'm doing barely anything, just creating 
the
> map
> object from the map file, drawing to the image object, and getting the
> bytes.
> I am calling both map.delete() and img.delete() at the end.  Then I loop
> over
> all of this repeatedly.
> 
> This is a little bit of a simplification, but essentially what I'm doing 
is
> this:
> 
>       for (int i = 0; i < loop; i++) {
>          mapObj map = null;
>          imageObj img = null;
> 
>          try {
>             map = new mapObj(filename);
>             img = map.draw();
>             byte[] bytes = img.getBytes();
>          } catch (Exception e) {
>             System.err.println("Caught exception: " + e);
>             e.printStackTrace(System.err);
>          } finally {
>             if (img != null) {
>                img.delete();
>             }
>             if (map != null) {
>                map.delete();
>             }
>          }
>          System.gc();
>       } // end for
> 
> I have tried a number of variants of this (including outputting the 
bytes to
> a
> file), and the offending line seems to be the call to 
imageObj.getBytes().
> With it, my program leaks.  Without it, it does not:
> 
> [rich at peyote mapserver_test 17:53:31]$ <mapserver_test> diff -u
> java/src/TestMapscript.java.bak1 java/src/TestMapscript.java.bak3
> --- java/src/TestMapscript.java.bak1   2008-02-20 17:37:43.000000000 
-0800
> +++ java/src/TestMapscript.java.bak3   2008-02-20 17:53:27.000000000 
-0800
> @@ -1,4 +1,4 @@
> -// this code does NOT leak
> +// this code DOES leak
> 
>  import edu.umn.gis.mapscript.mapObj;
>  import edu.umn.gis.mapscript.imageObj;
> @@ -47,7 +47,7 @@
>              System.err.println("Drawing map to image object");
>              img = map.draw();
> 
> -//             byte[] bytes = img.getBytes();
> +            byte[] bytes = img.getBytes();
>  //             int width = img.getWidth();
>  //             int height = img.getHeight();
>  //             System.err.println("The image is of size " + width + " x 
" +
> height + " and contains " + bytes.length + " bytes");
> [rich at peyote mapserver_test 17:53:53]$ <mapserver_test> 
> 
> I am calling System.gc() at the end of each iteration.  And by examining 
the
> memory from java's perspective (Runtime.getRuntime().freeMemory() and
> Runtime.getRuntime().totalMemory()) I can see that it is reasonably 
stable.
> However, the memory as reported by top gradually grows.
> 
> The source for imageObj.getBytes() is as follows:
> 
> mapscript/java/edu/umn/gis/mapscript/imageObj.java
> 
>   public byte[] getBytes() {
>     return mapscriptJNI.imageObj_getBytes(swigCPtr);
>   }
> 
> I don't totally understand how all of the SWIG stuff works, but I can 
follow
> it a bit.  The source for this is:
> 
>   getBytes() (mapscript/swiginc/image.i)
> 
> which notes the following:
> 
>     /*
> 
> 
-------------------------------------------------------------------------
>     getBytes returns a gdBuffer structure (defined in mapscript.i) which
> must
>     be typemapped to an object appropriate to the target language.  This
>     typemap must also gdFree the data member of the gdBuffer.  See the 
type-
>     maps in java/javamodule.i and python/pymodule.i for examples.
> 
>     contributed by Jerry Pisk, jerry.pisk at gmail.com
> 
> 
-------------------------------------------------------------------------
>     */
> 
> And all of the following:
> 
>   msSaveImageBuffer() (maputil.c)
>   msSaveImageBufferAGG() (mapagg.cpp)
>   msSaveImageBufferGD() (mapgd.c)
> 
> confirm that the caller must free the returned array (with gdFree()).
> 
> What's not entirely clear to me is whether "the caller" means the java
> mapscript code, or the application writer.  In other words, am I 
responsible
> for doing this, or should this happen when I call img.delete() ?  Here 
is
> the
> code for imageObj.delete():
> 
> mapscript/java/edu/umn/gis/mapscript/imageObj.java
> 
>   public void delete() {
>     if(swigCPtr != 0 && swigCMemOwn) {
>       swigCMemOwn = false;
>       mapscriptJNI.delete_imageObj(swigCPtr);
>     }
>     swigCPtr = 0;
>   }
> 
> But even though I don't see a call to gdFree() there, it's not clear to 
me
> that it's not happening somewhere.  I don't understand the SWIG path 
enough,
> and I haven't yet tried following it in gdb.  (Although that might be 
worth
> trying.)
> 
> If I am responsible for doing this, how do I do it?  It appears to me 
that
> gdFree() is not part of the mapscript API (it doesn't appear in the 
docs),
> and
> just that it is just a mapscript internal function.
> 
> Right now my test case is using a fairly complicated mapfile from a 
large
> data
> source located in a postgis database.  I will try to see if I can 
replicate
> this problem with a simpler map file and/or synthetic data.
> 
> But in the mean time I wanted to post this in case anyone might have 
some
> advice based on these observations.
> 
> Once again, thanks for any help that anyone might be able to provide.
> 
> - Rich
> 
> p.s. This is all based on mapserver 5.0.0
> 
> -- 
> View this message in context: http://www.nabble.com/thread-safety-
> tp15514019p15603603.html
> Sent from the Mapserver - User mailing list archive at Nabble.com.
> 
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20080221/fd0b46e8/attachment.htm>


More information about the MapServer-users mailing list