[gdal-dev] Sharing GDAL Dataset objects between threads (JNI)

[Markus Schneider]

Hi Even,

thanks for you help so far.

I now observed that handing around a (ECW) Dataset between threads works
fine, as long as I open it using:

- gdal.Open(String name)

However, when I open it using:

- gdal.OpenShared(String name)

I see strange effects and VM crashes.

Does this make sense? What do you believe is preferrable with regard to
performance and memory consumption:

1. Using a ThreadPool and obtain a Dataset using gdal.OpenShared for
each thread
2. Using gdal.Open and simply pool the obtained Dataset objects (and
hand them around between threads)

Best regards,
Markus

Am 14.10.2013 13:26, schrieb Even Rouault:
> Selon Markus Schneider <schneider at occamlabs.de>:
> 
>> Hi Even,
>>
>> Am 14.10.2013 13:10, schrieb Even Rouault:
>>> This is surprising. Are you seeing that with a particular driver or all
>> drivers
>>> ?
>>
>> I am experiencing this with ECW driver.
> 
> Ah, well, anything can happen within the beast then... Which SDK version : 3.3
> or 5 ?
> 
>>
>>> I don't see any use of TLS on the Java side. There might be some use of TLS
>>> objects on the C side, for functions in cpl_path.cpp for example, but their
>> use
>>> should be limited to the lifetime of the execution of a particular
>> function, so
>>> that shouldn't cause problems in the scenario you describe.
>>
>> I will try to prepare a minimal example to reproduce the issue. I will
>> also check if it occurs with a GeoTIFF.
> 
> Yes that's definnitely something to check.
> 
>>
>>>> Would it be hard to eliminate this behaviour (as it
>>>> would make pooling Dataset objects much easier and more efficient)?
>>>
>>> Difficult to tell until we have understood what really happens...
>>>
>>> Do you manage to reproduce that by running under Valgrind ? If so, that
>> might
>>> help understanding what is going wrong.
>>
>> I never used Valgrind, but with some help, I may be able to check
>> behaviour and report.
> 
> $ apt-get/yum install valgrind
> $ valgrind [valgrind_options] the_binary [the_binary_options] 2>valgrind_log.txt
> 
> where the_binary will be your servlet server. Not sure how well it run within
> Valgrind. You need to use a recent enough version of Valgrind to run the JVM
> with it (at least for 64 bit builds). And things will run at the minimum 10x
> slower
> 
> Running Java with Valgrind tends to generate a lot of warnings. You'll probably
> have to check the Valgrind option that increases the maximum number of warning
> messages.
> 
>>
>> Best regards,
>> Markus
>>
>> --
>> Markus Schneider
>> CEO
>>
>> Occam Labs UG (haftungsbeschränkt)
>> Godesberger Allee 139
>> 53175 Bonn, Germany
>>
>> +49 228 93798874
>>
>> http://www.occamlabs.de
>>
>>


-- 
Markus Schneider
CEO

Occam Labs UG (haftungsbeschränkt)
Godesberger Allee 139
53175 Bonn, Germany

+49 228 93798874

http://www.occamlabs.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 263 bytes
Desc: OpenPGP digital signature
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20131017/401e3b12/attachment.pgp>