[gdal-dev] HDF5 Multi-threading Errors

[Even Rouault]

Hi,

Part of your message didn't make it. See as it appears in
https://lists.osgeo.org/pipermail/gdal-dev/2019-February/049832.html

Do you have a link to that dbdbv6_level0c.h5 file ?

Regarding multi-threading, do you use a different GDAL dataset handle for each 
thread ? This is critical for proper behaviour. Some drivers might, by luck, 
work correctly with the same handle, but this is absolutely not guaranteed.

Even

> I am working in a .NET (4.5.2) environment, using GDAL v2.3.3 (along with
> the Native and Plugins packages). The HDF5 driver is version 1.8.14. The
> elevation source I'm using is dbdbv6_level0c.h5, supplied by NAVOCEANO, and
> is version 6.2. There are 1,700 subdatasets (each 600x600 float-32) in the
> .h5.
> 
> In our application, a map engine makes asynchronous requests for slippy map
> tiles. At small zoom levels, these requests are passed to our ETOPO1
> elevation source (which uses the ESRI driver, source file
> etopo1_ice_g_f4.flt), but at larger zoom levels, we poll the DBDB-V source
> summarized above.
> 
> As mentioned, these requests are asynchronous, and I've run into
> multi-threading issues with the HDF5 driver. Everything "works as expected"
> for the ESRI driver and ETOPO1 elevation data, but making the same requests
> of DBDB-V / HDF5 results in various exceptions.
> 
> As a quick test application, I wrote this:
> 
> 
> 
> The output shows that the ETOPO1 source handles the parallel calls just
> fine, but that the DBDB-V source fails:
> 
> 
> 
> The originating exception is thrown from the HDF5 library:
> 
> 
> 
> Finally, I should note that if the Parallel.For loop around the DBDB-V is
> changed to a normal for-loop, then nothing bad happens... the application
> outputs a sequential success story:
> 
> 
> 
> However, it takes *almost four seconds* (average is 3.8s) for every
> Gdal.Open(...) call to DBDB-V! Compare this to the very quick response of
> ETOPO1 (average 1 millisecond)... and it explains why we can't realistically
> even consider synchronous calls to DBDB-V, and really need to understand
> why it is failing in parallel.
> 
> I know that other questions have been asked about HDF5 multi-threading, but
> they seemed to focus on building the HDF5 source locally. We don't have that
> option - our data is supplied by NAVOCEANO "as is."
> 
> Very anxious to learn if anyone has a solution, or perhaps this is an issue
> that could be fixed in an upcoming branch. Thanks!
> 
> 
> 
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com