[mapserver] Java VM Crashes using the Mapscript API
Benedikt Rothe
umn-ms at HYDROTEC.DE
Fri Nov 18 08:51:56 EST 2005
Hallo Umberto, Fernando, Donovan and ...
Thanks for coming back to this point. This area is also still a big
problem to me. (Especially I'm happy about the
Java/Oracle-guru-combination
Fernando/Umberto.)
It's maybe usefull to figure out where we agree and where we disagree:
- I disagree to Umberto with the use of the Java-gc in Mapserver 4.6. I
state:
As long as Java/Mapserver is at is is now (Version 4.6.1)
one must not rely on gc only. One has to call delete.
To the question of Fernando: "So the question is, where I can call
delete? "
is my rule of thumb:
"Call delete on Mapserver-Objects as soon as you can whenever a delete
exists."
As a matter of fact: By doing so, delete is called more often then
necessary, because
in many cases the finaliser would do a good job. But this doesn't
matter.
- I agree to Umberto that there is a bug in Java/Mapserver behaviour.
My suggestion is obvioulsy not a solution but a way to live with
the software as it is.
If we would come to the question "How should Java/Mapserver behave"
my personal opinion is to look at the way the Eclipse-People made SWT.
They also had the problem, that Java-Objects enclose ressources which are
not managed by the Java-VM:
http://www.eclipse.org/articles/swt-design-2/swt-design-2.html
(Rule 1: If you created it, you dispose it. )
Following this approache would leave to only small changes in
Java/Mapserver-Code
but to a different documentation of the Java/Mapscript-API and it should
be clearer
when a objected has to be delete by the way an object is created.
(Actually I suppose, that Umberto will disagree in this point ... :-) )
At last it's maybe usefull to share experiences in stability:
We wrote an Oracle-Spatial/Tomcat/Mapserver-Application (Linux-Suse 9.3)
- On our testsystem it runs very fine for weeks.
The testsystem even survives stress-tests, where we simulate some
browsers making
many requests.
- On the customers (production-)machine we still have Tomcat crashes about
one or
two times the day. We didn't find a solution (except a
watchdog-software which restarts tomcat in
the crash-case :-( )
This machine also survives the one- or two-hour stress-tests!
This happens although I'm quite shure, that we followed the "delete as
soon as possible"-rule.
Crashes occure often somewhere in the OCI.
But I'm not able to be more precise for the moment, because I have very
restricted access to
the production-machine and crashes which occure only one or two times a
day
are obviously difficult to analyse.
(Fernando: That does not necessarily mean, that the Mapserver-Oracle-Code
is buggy. This
could be a consecutive fault from some other memory-management-error ...)
Benedikt
Umberto Nicoletti <umberto.nicoletti at gmail.com> schrieb am 18.11.2005
13:09:38:
> On 11/18/05, Fernando Simon <fsimon at univali.br> wrote:
> > Hi Benedikt and Umberto,
> > Here I'm working with a project that uses JavaMapscript and I'm
with
> > some stability problems. For the project I use Mapserver (I compiled
> > myself, with multi thread support) with just these supports:
> > JavaMapscript, Oracle Spatial (I develop the maporaclespatial.c),
Raster.
> > Sometimes the server (TomCat) crashes in randomly parts of the
code,
> > I suspect that the issue is with Java garbage collector and the delete
> > function. In the code I did as Benedict suggested, if I create an
object
> > (like layer, class, style....), after I finished all the operations
with
> > this object I delete it (using delete).
> > But Umberto, wrote, correct me if I'm wrong, that he don't use the
> > delete function. So the question is, where I can call delete? For what
> > objects can I use delete?
>
> To be exact I wrote:
>
> /quote
>
> [...] this is why I never attempt to
> release objects on my own, but I always leave it up to the gc.
> Even in that case a segfault is indeed possible and that's why we need
> to open an issue on bugzilla so that the developers know.
>
> quote/
>
> So my suggestion is: do not use delete at all, since the jvm will
> handle that for you automatically, but remember that this will not
> prevent random crashes from happening. The issue is unfortunately
> deeply rooted within mapserver, which was not coded for long
> multi-threaded operations, but rather for short-lived independent
> processes.
>
> To date I have found that Java mapscript can be used to provide basic
> mapping services with reasonable stability and performance when only
> the following components are involved:
>
> - shape support
> - raster support (via gdal)
> - proj
>
> In the future I will have to support postgis and that frankly worries me
a bit.
>
> HTH,
> Umberto
>
> > Regards.
> >
> >
------------------------------------------------------------------------
> > Fernando Simon
> > Mapserver and Oracle Spatial developer
> > G10 - Laboratorio de Computacao Aplicada - Brazil
> > http://www.univali.br/g10 - UNIVALI/CTTMAR
> >
------------------------------------------------------------------------
> >
> > Benedikt Rothe wrote:
> >
> > >
> > > Hallo Umberto and Christian
> > >
> > > Christian reported a crash in layerObj.delete() which I
> > > also detected.
> > > I think I understood the following:
> > >
> > > Let's say, we have an initialized mapObj mO;
> > >
> > > layerObj lO = new layerObj(mO);
> > > // Say lO is the 5th layer now.
> > > // Now lO.swigCMemOwn == true and
> > > // lO.swigCPtr points onto the mO.swigCPtr->layers[4]
> > >
> > > ...
> > > // now the mapObj is deleted but the Java-layerObj continues to
live.
> > > // the C-layer-Object mO.swigCPtr->layers[4] will deleted and freed
also!
> > > mO.delete()
> > > // From now on O.swigCPtr points into invalid piece of C-memory.
> > > ...
> > >
> > > Later on the Java-garbage-collector finalizes the layerObj.
> > >
> > > Because of "lO.swigCMemOwn == true" and "lO.swigCPtr!=0" the
> > > finalize-method calls
> > > layerObj.delete(). This method frees lO.swigCPtr which is invalid
memory.
> > >
> > > Same story about layerObj/classObj.
> > >
> > > Benedikt Rothe
> > >
> > >
> > >
> > > UMN MapServer Users List <MAPSERVER-USERS at LISTS.UMN.EDU> schrieb am
> > > 28.06.2005 12:16:01:
> > >
> > > > Hello Umberto,
> > > >
> > > > Umberto Nicoletti wrote:
> > > >
> > > > >Hi Christian,
> > > > >I do not think the error you are getting is due to threading
issues,
> > > > >but trying what Mario suggests will at least narrow the search
field.
> > > > >
> > > > >In case you are still getting errors even with synchronized code
> > > > >blocks I have a few questions for you to help me understand your
> > > > >setup:
> > > > >Do you always get errors in delete_classObj or do the segfaults
happen
> > > > >in different functions?
> > > > >
> > > > >
> > > > Yes, the segfaults alway happen in the delete_classObj function.
> > > >
> > > > >If the segfault is always caused by
> > > > >delete_ClassObj then I suppose you are creating class objects in
your
> > > > >java code. Is that true?
> > > > >
> > > > >
> > > > Yes, that's true, I need to create classObjects.
> > > >
> > > > >If that is the case then we can setup a very simple test to
reproduce
> > > > >the problem: modify one of the examples so that it will load a
map and
> > > > >then start adding classes in similar fashion to your code, but in
a
> > > > >tight loop. Classes should be made eligible for gc by dropping
all
> > > > >references to them. As soon as gc kicks in the vm should crash.
At
> > > > >that point it will be clear that the problem is in
delete_ClassObj and
> > > > >the hunting season will be open.
> > > > >
> > > > >
> > > > I have tested a proposed solution by Benedikt Rothe yet, which
said that
> > > > one should call the delete method of the classObject after it is
not
> > > > needed any more.
> > > > This confirms my assumption that this problem occours when java's
> > > > garbage-collector destroys the null-referenced objects. Now it
seems to
> > > > work properly.
> > > >
> > > > The mapserver version we actually use is 4.4.1, we have tested
4.4.2,
> > > > 4.2.4 and 4.6.0, too before.
> > > >
> > > > Thank you all for your help!
> > > >
> > > > Best Regards,
> > > > Christian :-)
> > > >
> > > >
> > > > >BTW can you report the mapserver version you are using (I have
4.4.2
> > > > >and I know it works, so if you can use that).
> > > > >
> > > > >Best Regards,
> > > > >Umberto
> > > > >
> > > > >On 6/21/05, Christian Schroeder <mailings at abiwood99.de> wrote:
> > > > >
> > > > >
> > > > >>Hello Umberto,
> > > > >>
> > > > >>thank your for your immediate answer.
> > > > >>I do not call the delete_ methody directly and don't think I am
using
> > > > >>special gc parameters.
> > > > >>
> > > > >>And... I have read the README file before compiling the
mapserver :-)
> > > > >>
> > > > >>I will try to get it to work with "synchronized"-flags als Mario
Basa
> > > > >>supposed.
> > > > >>
> > > > >>Thank you!
> > > > >>
> > > > >>Christian
> > > > >>
> > > > >>
> > > > >>Umberto Nicoletti schrieb:
> > > > >>
> > > > >>
> > > > >>
> > > > >>>Christian,
> > > > >>>are you calling the delete_ methods directly in your code or
are you
> > > > >>>using some special gc paramaters?
> > > > >>>
> > > > >>>As a side note the --use-threads option to configure is
*absolutely*
> > > > >>>necessary, as are brakes on your car. I think we should write
it in
> > > > >>>the README (as if someone actually cared to read it :-( ).
> > > > >>>
> > > > >>>Best regards,
> > > > >>>Umberto
> > > > >>>
> > > > >>>
> > > > >>>On 6/21/05, Sean Gillies <sgillies at frii.com> wrote:
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>>I'm forwarding this to the users list. Hopefully, Umberto will
> > > be able
> > > > >>>>to provide some insight.
> > > > >>>>
> > > > >>>>cheers,
> > > > >>>>Sean
> > > > >>>>
> > > > >>>>On Jun 16, 2005, at 5:08 PM, Christian Schröder wrote:
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>>Dear Mr. Gillies,
> > > > >>>>>
> > > > >>>>>some weeks ago me and Florian Pepping contacted you because
we had
> > > > >>>>>problems using the Java Mapscript API. Thanks to you we could
solve
> > > > >>>>>these problems :-)
> > > > >>>>>
> > > > >>>>>Now we got our program doing what it's supposed to do but
there is
> > > > >>>>>still a big problem left which we were not able to solve yet:
> > > > >>>>>We created a simple servlet which is created inside a Tomcat
5.0
> > > > >>>>>Webserver. This servlet created a map image (png/jpg) and
displays
> > > > >>>>>some specified objects on the map. (We use it for location
based
> > > > >>>>>services --> "show me the position of the next printer").
> > > > >>>>>After an irregular number of calls of our servlet which uses
> > > the Java
> > > > >>>>>Mapscript API the complete Java VM and with it the Tomcat
> > > crashes. I
> > > > >>>>>attached the error report below. The program works properly
for a
> > > > >>>>>number of calls (between 5 and 1000 :-) ) and after that it
> > > crashes.
> > > > >>>>>We have tried several versions of the mapserver (4.4.1,
4.4.2,
> > > 4.2.4,
> > > > >>>>>4.6.1 RC1) and compiled the Java Mapscript Module with JDK
> > > 1.4.2 and
> > > > >>>>>1.5.0. We also configured mapserver using the --with-threads
> > > option,
> > > > >>>>>but all this did not help. By the way the mapserv cgi-module
works
> > > > >>>>>properly.
> > > > >>>>>
> > > > >>>>>Do you have an idea for this?
> > > > >>>>>
> > > > >>>>>Thanks for your anxiety
> > > > >>>>>
> > > > >>>>>Christian & Florian
> > > > >>>>>University of Paderborn, Germany
> > > > >>>>>
> > > > >>>>>
> > > >
>>>>>-------------------------------------------------------------------
> > > > >>>>>
> > > > >>>>>JavaMapscriptLoader: mapscript native library has been
loaded.
> > > > >>>>>* mapscript native library loaded *
> > > > >>>>>
> > > > >>>>>An unexpected exception has been detected in native code
> > > outside the
> > > > >>>>>VM.
> > > > >>>>>Unexpected Signal : 11 occurred at PC=0x3338268
> > > > >>>>>Function=delete_classObj+0x8
> > > > >>>>>Library=/usr/lib/libmapscript.so
> > > > >>>>>
> > > > >>>>>Current Java thread:
> > > > >>>>> at
edu.umn.gis.mapscript.mapscriptJNI.delete_classObj(Native
> > > > >>>>>Method)
> > > > >>>>> at
edu.umn.gis.mapscript.classObj.delete(classObj.java:32)
> > > > >>>>> at
edu.umn.gis.mapscript.classObj.finalize(classObj.java:26)
> > > > >>>>> at java.lang.ref.Finalizer.invokeFinalizeMethod(Native
> > > Method)
> > > > >>>>> at
java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
> > > > >>>>> at
java.lang.ref.Finalizer.access$100(Finalizer.java:14)
> > > > >>>>> at
> > > >
>>>>>java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)
> > > > >>>>>
> > > > >>>>>Dynamic libraries:
> > > > >>>>>Can not get information for pid = 10558
> > > > >>>>>
> > > > >>>>>Heap at VM Abort:
> > > > >>>>>Heap
> > > > >>>>>def new generation total 1152K, used 135K [0x08ae0000,
> > > 0x08c20000,
> > > > >>>>>0x08fc0000)
> > > > >>>>> eden space 1024K, 0% used [0x08ae0000, 0x08ae4078,
0x08be0000)
> > > > >>>>> from space 128K, 100% used [0x08c00000, 0x08c20000,
0x08c20000)
> > > > >>>>> to space 128K, 0% used [0x08be0000, 0x08be0000,
0x08c00000)
> > > > >>>>>tenured generation total 15048K, used 13624K [0x08fc0000,
> > > > >>>>>0x09e72000,
> > > > >>>>>0x0cae0000)
> > > > >>>>> the space 15048K, 90% used [0x08fc0000, 0x09d0e1c0,
0x09d0e200,
> > > > >>>>>0x09e72000)
> > > > >>>>>compacting perm gen total 18432K, used 18373K [0x0cae0000,
> > > > >>>>>0x0dce0000,
> > > > >>>>>0x10ae0000)
> > > > >>>>> the space 18432K, 99% used [0x0cae0000, 0x0dcd1618,
0x0dcd1800,
> > > > >>>>>0x0dce0000)
> > > > >>>>>
> > > > >>>>>Local Time = Tue Jun 14 15:32:19 2005
> > > > >>>>>Elapsed Time = 246
> > > > >>>>>#
> > > > >>>>># The exception above was detected in native code outside the
VM
> > > > >>>>>#
> > > > >>>>># Java VM: Java HotSpot(TM) Client VM (1.4.2_08-b03 mixed
mode)
> > > > >>>>>#
> > > > >>>>># An error report file has been saved as
/tmp/hs_err_pid10558.log.
> > > > >>>>># Please refer to the file for further information.
> > > > >>>>>#
> > > > >>>>>
> > > >
>>>>>-------------------------------------------------------------------
> > > > >>>>>
> > > > >>>>>On Mar 22, 2005, at 12:02 PM, Florian Pepping wrote:
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>>Dear Mr. Gillies,
> > > > >>>>>>
> > > > >>>>>>I'm a student of the University of Paderborn in Germany and
> > > member of
> > > > >>>>>>the project group "Location Based Services for Wireless
> > > Devices". In
> > > > >>>>>>this project we try to position laptops and other
WLAN-enabled
> > > > >>>>>>devices using the signal strength of the WLAN. According to
their
> > > > >>>>>>position, we want to offer location based services to the
persons
> > > > >>>>>>using the devices (where I am; where's the next printer; is
> > > there a
> > > > >>>>>>friend nearby)
> > > > >>>>>>
> > > > >>>>>>In order to do this, we want to use your mapserver and the
Java
> > > > >>>>>>Mapscript API to generate maps according to the actual
> > > position and
> > > > >>>>>>situation. We like to customize the map of our building and
add
> > > > >>>>>>points, lines and so on.
> > > > >>>>>>
> > > > >>>>>>We have been able to compile the whole mapserver and the
Java
> > > > >>>>>>Mapscript API. A small Java example also works, which
presents an
> > > > >>>>>>unchanged map of our building.
> > > > >>>>>>
> > > > >>>>>>[...]
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >
> > > > >
> > > > >
> > > > >
> >
> >
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapserver-users/attachments/20051118/bbce56c0/attachment.html
More information about the mapserver-users
mailing list