[gdal-dev] HDF5 Multi-threading Errors

[andrewparadis]

I am working in a .NET (4.5.2) environment, using GDAL v2.3.3 (along with the
Native and Plugins packages). The HDF5 driver is version 1.8.14. The
elevation source I'm using is dbdbv6_level0c.h5, supplied by NAVOCEANO, and
is version 6.2. There are 1,700 subdatasets (each 600x600 float-32) in the
.h5.

In our application, a map engine makes asynchronous requests for slippy map
tiles. At small zoom levels, these requests are passed to our ETOPO1
elevation source (which uses the ESRI driver, source file
etopo1_ice_g_f4.flt), but at larger zoom levels, we poll the DBDB-V source
summarized above.

As mentioned, these requests are asynchronous, and I've run into
multi-threading issues with the HDF5 driver. Everything "works as expected"
for the ESRI driver and ETOPO1 elevation data, but making the same requests
of DBDB-V / HDF5 results in various exceptions.

As a quick test application, I wrote this:



The output shows that the ETOPO1 source handles the parallel calls just
fine, but that the DBDB-V source fails:



The originating exception is thrown from the HDF5 library:



Finally, I should note that if the Parallel.For loop around the DBDB-V is
changed to a normal for-loop, then nothing bad happens... the application
outputs a sequential success story:



However, it takes *almost four seconds* (average is 3.8s) for every
Gdal.Open(...) call to DBDB-V! Compare this to the very quick response of
ETOPO1 (average 1 millisecond)... and it explains why we can't realistically
even consider synchronous calls to DBDB-V, and really need to understand why
it is failing in parallel.

I know that other questions have been asked about HDF5 multi-threading, but
they seemed to focus on building the HDF5 source locally. We don't have that
option - our data is supplied by NAVOCEANO "as is."

Very anxious to learn if anyone has a solution, or perhaps this is an issue
that could be fixed in an upcoming branch. Thanks!



--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html