[gdal-dev] [EXTERNAL] Re: HDF5 and CF conventions

[Agram, Piyush S (US 334K)]

Hi Even,


My main question is :

Why... ? Why the netCDF driver on a netCDF v4 / HDF5 file doesn't do the job already ? I'm quite concerned by the CF logic being duplicated in several drivers.

Ans: Unfortunately, we have a data volume problem with handling large SAR datasets. HDF5 allows us to work with float16 and one-bit/ two-bit masks while using a single file with hierarchical organization within it. Currently, we just append the .nc extension and test the code with the netcdf driver when not using any of the custom HDF5 features. We would have to use the advanced HDF5 features for managing data volumes moving ahead.


I love your optimism :-)

Ans: Fingers crossed. I could be digging myself a hole here. But thanks to Julien and David B., we were able to get netcdf group support in. Hoping that the amount of effort is roughly the same.



Constants are only one part of the equation. The whole logic to build the OGRSpatialReference object from attributes is also something that would be great to have coded just once. To reuse it in multiple drivers would require to abstract the way to query attributes. Which brings me to thinking that the new multidim API could potentially be of use to have a driver-independent CF CRS decoding logic.

Ans: I agree. A simplified structure that holds the current dataset id, group id and root id could potentially be specified for netcdf and HDF5? This simplified structure could reside within CF.h and be specialized  in the netcdf and hdf5 dataset cpp files. The logic for searching the groups / traversing the tree could all be local to the CF.h / CF.cpp file then.

Will issue a separate PR for “crs_wkt”. This will make the datasets generated using pyproj.CRS.to_cf() readable with GDAL.

Piyush


From: Even Rouault <even.rouault at spatialys.com>
Date: Tuesday, March 31, 2020 at 10:34 AM
To: "gdal-dev at lists.osgeo.org" <gdal-dev at lists.osgeo.org>
Cc: "Agram, Piyush S (US 334K)" <piyush.agram at jpl.nasa.gov>
Subject: [EXTERNAL] Re: [gdal-dev] HDF5 and CF conventions


Piyush,



> We are considering extending the HDF5 driver to support CF conventions.



My main question is :

Why... ? Why the netCDF driver on a netCDF v4 / HDF5 file doesn't do the job already ? I'm quite concerned by the CF logic being duplicated in several drivers.



> All the logic exists as part of the netcdf driver (pretty much) and it should not take a really long time to add that support to HDF5 driver as well.



I love your optimism :-)



> 1. Working towards this, would it be ok to move the “CF_*” defines / constants to its own CF.h header file? That will then let us use it in other drivers as well.



Constants are only one part of the equation. The whole logic to build the OGRSpatialReference object from attributes is also something that would be great to have coded just once. To reuse it in multiple drivers would require to abstract the way to query attributes. Which brings me to thinking that the new multidim API could potentially be of use to have a driver-independent CF CRS decoding logic.



> 2. CF conventions 1.7+ supports an attribute

> called “crs_wkt”. GDAL has traditionally referred to this attribute as

> “spatial_ref”. Would it be ok to update netcdf driver to look for crs_wkt

> first and then fall back to usual spatial_ref attribute when it is not

> found?



Sounds good to me. Could/should be a separate pull request.



> 3. If one were to add CF support for HDF5, would it be preferable

> to make this its own driver like KEA/BAG (say H5CF) or part of the

> HDF5imagedataset core like CosmoSkymed / KompSAT products?



I'd say more in the existing HDF5 driver to limit the amount of new code.

KEA & BAG are really very specialized profiles of HDF5.



Even





--

Spatialys - Geospatial professional services

http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20200331/0bd5c3d7/attachment.html>