[gdal-dev] Java bindings to gdal/ogr - looking for more developer documentation

[Tom Moore]

Hi Evan

I seem to still be having problems closing datasets in Java resulting in a JVM crash, using gdal-3-13-3_x64 downloaded from the GISInternals site.  Below is the stack trace information. 

---------------  T H R E A D  ---------------

Current thread (0x000001ffd39932c0):  JavaThread "Finalizer" daemon [_thread_in_native, id=25140, stack(0x000000ba6eb00000,0x000000ba6ec00000) (1024K)]

Stack: [0x000000ba6eb00000,0x000000ba6ec00000],  sp=0x000000ba6ebfed60,  free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [gdal.dll+0xc86927]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.gdal.gdal.gdalJNI.delete_Dataset(J)V+0
j  org.gdal.gdal.Dataset.delete()V+25
j  org.gdal.gdal.Dataset.finalize()V+1
j  java.lang.System$2.invokeFinalize(Ljava/lang/Object;)V+1 java.base at 21.0.6
j  java.lang.ref.Finalizer.runFinalizer(Ljdk/internal/access/JavaLangAccess;)V+115 java.base at 21.0.6
j  java.lang.ref.Finalizer$FinalizerThread.run()V+29 java.base at 21.0.6
v  ~StubRoutines::call_stub 0x000001ffa369100d

siginfo: EXCEPTION_ACCESS_VIOLATION (0xc0000005), reading address 0xffffffffffffffff

The crash is happening when the swig delete_Dataset method, called by the finalizer, calls into the gdal.dll.

I tried calling Open() and Close() several times in a row to see if this would trigger the problem, like this:
gdal.*AllRegister*();
 Dataset ds = gdal.Open(filename); 
 ds.Close();
 ds.Close();  

but that seemed to work ok.

I also tried open and close sequences, like this:
 Dataset ds = gdal.Open(filename);
 ds.Close();
 ds = gdal.Open(filename);
 ds.Close();  
 ds = gdal.Open(filename);
 System.gc();
 System.runFinalization();

This didn't trigger a problem either.  I did a few more tests, repeating steps of opening a dataset, reading a band, closing the dataset, and this didn't do it either.

If I enable CPL_DEBUG then my pattern of access that causes the crash is:
GDAL: GDALDriver::Create(GTiff,c:/tmp/test1.tif,10,10,1,Int32,000001BCB9893680)
GDAL: GDALClose(c:/tmp/test1.tif, this=000001BCB9872A90)
GDAL: GDALOpen(c:/tmp/test1.tif, this=000001BCB98784D0) succeeds as GTiff.
GDAL: GDALClose(c:/tmp/test1.tif, this=000001BCB98784D0)
GDAL: GDALOpen(c:/tmp/test1.tif, this=000001BCBE486550) succeeds as GTiff.
GDAL: GDALOpen(c:/tmp/test1.tif.vat.dbf, this=000001BCBE10D070) succeeds as ESRI Shapefile.
GDAL: GDALClose(c:/tmp/test1.tif.vat.dbf, this=000001BCBE10D070)
GDAL: GDALClose(c:/tmp/test1.tif, this=000001BCBE486550)

Around this point the finalizer kicks in and the JVM crashes.

This is a hard one for me to track down, because I don't know the tooling to debug native cpp code from within Java.  What I could do is insert some debugging statements in to the code to try to see what is going on.  If you have any suggestions about this I would be happy to try it.

Tom


On Fri, May 30, 2025, at 5:53 PM, Even Rouault wrote:
> 
> Le 30/05/2025 à 23:39, Tom Moore a écrit :
> > Hi Even
> >
> > I just wanted to update you and provide a record for posterity with 
> > the results of me playing around with gdal/java and resource management.
> >
> > It appears to me that Dataset objects should not be closed from Java 
> > client code.  If you do then often there will be an access violation 
> > (native null pointer) that will crash the JVM. From the dump file the 
> > following stack trace shows the Java code being executed when the 
> > exception occurs:
> 
> I believe this issue should be fixed in 3.11.0 per 
> https://github.com/OSGeo/gdal/commit/ec4ca7930b48653bb0fac27b59c6c1bf883c45f2
> 
> Actually historically this was the .delete() method that should be 
> called. The exposition of .Close() is quite recent and was broken (until 
> the above mentioned fix)
> 
> >
> > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> > j  org.gdal.gdal.gdalJNI.delete_Dataset(J)V+0
> > j  org.gdal.gdal.Dataset.delete()V+25
> > j  org.gdal.gdal.Dataset.finalize()V+1
> > j  java.lang.System$2.invokeFinalize(Ljava/lang/Object;)V+1 
> > java.base at 21.0.6
> > j 
> > java.lang.ref.Finalizer.runFinalizer(Ljdk/internal/access/JavaLangAccess;)V+115 
> > java.base at 21.0.6
> > j  java.lang.ref.Finalizer$FinalizerThread.run()V+29 java.base at 21.0.6
> >
> > So it looks like the swig bindings implement a finalizer to dispose of 
> > the native object when the reference is no longer reachable (nice!).  
> > However it looks like there might not be any protection to prevent bad 
> > things from happening when the Delete method is called more than 
> > once.  It appears that in this case that when the finalizer calls 
> > Delete on the Dataset the jvm crashes because the Dataset has already 
> > been closed manually and the resources have already been released.
> >
> > If I am correct in this conclusions then this is not a problem.  If 
> > correct I suggest that either the docs be updated to indicate do not 
> > manually call Delete on the Dataset object, or change the Delete 
> > method code to gracefully handle multiple calls (better choice).
> >
> > By the way, something to note for the future is that finalizers have 
> > been deprecated since Java 9 (2017).  This is described in 
> > https://openjdk.org/jeps/421. Although deprecated, finalizers are 
> > still allowed in modern jdk's and probably will be for a while yet. At 
> > some future time they will be removed.  The suggested replacement for 
> > finalizers are cleaners, but they are only available in Java 9+.  When 
> > finalizers are removed there will need to be a new set of bindings 
> > that use cleaners.  You can probably ignore this problem for a few 
> > more years, but at some point you will need to provide two sets of 
> > Java bindings (one required for Java 8 and earlier, and one required 
> > for some future Java and later).
> 
> Thanks for the heads up. Seizing the opportunity to remind interesting 
> parties in the GDAL Java bindings that they should not be shy and are 
> welcome to be proactive. On my side, they are very very minimilastically 
> maintained.
> 
> Even
> 
> -- 
> http://www.spatialys.com
> My software is free, but my time generally not.
> 
> 

--
Tom Moore
Spatial Planning Systems
960 Burkes Bluff Lane
Deep River ON  K0J 1P0
Canada

Phone: +1 613 584 9354
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250716/58d98163/attachment.htm>