[gdal-dev] HDF5 and geolocation arrays
Michael Sumner
mdsumner at gmail.com
Mon Apr 29 13:42:02 PDT 2024
Oh wow, that's fantastic - thanks so much. Lots for me to learn from there
too, with the HDF5EOS GRID details.
Cheers, Mike
On Tue, Apr 30, 2024 at 6:08 AM Even Rouault <even.rouault at spatialys.com>
wrote:
> Michael,
>
> actually support for those products was already there at 95% since they
> use the HDF5EOS GRID formalism which we support since 3.7. But that
> particular type of products has an oddity. They have an empty
> GROUP=Dimension within the GROUP=GridStructure in the HDF5EOS metadata,
> which our parser didn't like as we use the information in this group to map
> dimension names to dimension index and size. Fixed/worked around in
> https://github.com/OSGeo/gdal/pull/9807 . Hard to tell if those products
> are in fault given that the HDF5EOS specification at
> https://www.earthdata.nasa.gov/s3fs-public/imported/ESDS-RFC-008-v1.1.pdf
> , appendix A, is extremely minimum regarding the HDF5EOS metadata itself
> (for once, I complain for a spec to be too minimal !), and the example they
> give for grids doesn't make sense to me (the declared dimension names and
> sizes have nothing to do with the ones use in the grid)
>
> I now get:
>
> $ gdalinfo
> HDF5:"AMSR_U2_L3_SeaIce12km_B04_20120702.he5"://HDFEOS/GRIDS/SpPolarGrid12km/Data_Fields/SI_12km_SH_18H_ASC
> Driver: HDF5Image/HDF5 Dataset
> Files: AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> Size is 632, 664
> Coordinate System is:
> PROJCRS["unnamed",
> BASEGEOGCRS["Unknown datum based upon the custom spheroid",
> DATUM["Not specified (based on custom spheroid)",
> ELLIPSOID["Custom spheroid",6378273,0,
> LENGTHUNIT["metre",1,
> ID["EPSG",9001]]]],
> PRIMEM["Greenwich",0,
> ANGLEUNIT["degree",0.0174532925199433,
> ID["EPSG",9122]]]],
> CONVERSION["Polar Stereographic (variant B)",
> METHOD["Polar Stereographic (variant B)",
> ID["EPSG",9829]],
> PARAMETER["Latitude of standard parallel",-70,
> ANGLEUNIT["degree",0.0174532925199433],
> ID["EPSG",8832]],
> PARAMETER["Longitude of origin",0,
> ANGLEUNIT["degree",0.0174532925199433],
> ID["EPSG",8833]],
> PARAMETER["False easting",0,
> LENGTHUNIT["metre",1],
> ID["EPSG",8806]],
> PARAMETER["False northing",0,
> LENGTHUNIT["metre",1],
> ID["EPSG",8807]]],
> CS[Cartesian,2],
> AXIS["(E)",north,
> MERIDIAN[90,
> ANGLEUNIT["degree",0.0174532925199433,
> ID["EPSG",9122]]],
> ORDER[1],
> LENGTHUNIT["Meter",1]],
> AXIS["(N)",north,
> MERIDIAN[0,
> ANGLEUNIT["degree",0.0174532925199433,
> ID["EPSG",9122]]],
> ORDER[2],
> LENGTHUNIT["Meter",1]]]
> Data axis to CRS axis mapping: 1,2
> Origin = (-3950000.000000000000000,4350000.000000000000000)
> Pixel Size = (12500.000000000000000,-12500.000000000000000)
> Metadata:
> Conventions=CF-1.6
> HDFEOS_INFORMATION_HDFEOSVersion=HDFEOS_5.1.15
> history=This version of the Sea Ice processing code contains updates
> provided by the science team on September 16, 2019. For details on these
> updates, see the release notes provided in the DAP.
> institution=NASA's AMSR Science Investigator-led Processing System (SIPS)
> references=Please cite these data as: Markus, T., J. C. Comiso, and W.
> N. Meier. 2018. AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness
> Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids,
> Version 1. [Indicate subset used]. Boulder, Colorado USA. NASA National
> Snow and Ice Data Center Distributed Active Archive Center. doi:
> https://doi.org/10.5067/RA1MIJOYPK3P.
> source=satellite observation
> title=AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, Sea
> Ice Concentration, Motion & Snow Depth Polar Grids
> Corner Coordinates:
> Upper Left (-3950000.000, 4350000.000) ( 42d14'27.21"W, 39d11'27.54"S)
> Lower Left (-3950000.000,-3950000.000) (135d 0' 0.00"W, 41d23'59.41"S)
> Upper Right ( 3950000.000, 4350000.000) ( 42d14'27.21"E, 39d11'27.54"S)
> Lower Right ( 3950000.000,-3950000.000) (135d 0' 0.00"E, 41d23'59.41"S)
> Center ( 0.000, 200000.000) ( 0d 0' 0.01"E, 88d 8'51.76"S)
> Band 1 Block=632x1 Type=Int32, ColorInterp=Undefined
> NoData Value=0
> Metadata:
> add_offset=0
> coordinates=lon lat
> long_name=18.7 GHz horizontal daily average ascending Tbs
> packing_convention=netCDF
> packing_convention_description=unpacked = scale_factor x packed +
> add_offset
> scale_factor=0.1
> standard_name=brightness_temperature
> units=degree_kelvin
> _FillValue=0
>
> Even
> Le 29/04/2024 à 21:10, Michael Sumner via gdal-dev a écrit :
>
> This HDF5 (requires earthdata credentials your "Authorization: Bearer
> <token>" in GDAL_HTTP_HEADERS, or equiv) presents without geolocation
> arrays.
>
> gdalinfo "/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5"
> -sd 26
> Driver: HDF5Image/HDF5 Dataset
> Files: /vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> Size is 608, 896
> Metadata:
> Conventions=CF-1.6
> HDFEOS_INFORMATION_HDFEOSVersion=HDFEOS_5.1.15
> history=This version of the Sea Ice processing code contains updates
> provided by the science team on September 16, 2019. For details on these
> updates, see the release notes provided in the DAP.
> institution=NASA's AMSR Science Investigator-led Processing System (SIPS)
> references=Please cite these data as: Markus, T., J. C. Comiso, and W.
> N. Meier. 2018. AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness
> Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids,
> Version 1. [Indicate subset used]. Boulder, Colorado USA. NASA National
> Snow and Ice Data Center Distributed Active Archive Center. doi:
> https://doi.org/10.5067/RA1MIJOYPK3P.
> source=satellite observation
> title=AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, Sea
> Ice Concentration, Motion & Snow Depth Polar Grids
> Corner Coordinates:
> Upper Left ( 0.0, 0.0)
> Lower Left ( 0.0, 896.0)
> Upper Right ( 608.0, 0.0)
> Lower Right ( 608.0, 896.0)
> Center ( 304.0, 448.0)
> Band 1 Block=608x1 Type=Int32, ColorInterp=Undefined
> Metadata:
> comment=data value meaning: 0 -- Open Water, 110 -- missing/not
> calculated, 120 -- Land
> coordinates=lon lat
> long_name=Sea ice concentration daily average
> units=percent
>
>
>
> gdalinfo --version
> GDAL 3.9.0dev-cb4d30f56d, released 2024/04/15
>
> The geolocation arrays are sds 33 and 32 respectively:
>
> HDF5:"/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> "://HDFEOS/GRIDS/NpPolarGrid12km/lon
>
> HDF5:"/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> "://HDFEOS/GRIDS/NpPolarGrid12km/lat
>
> And things work when lining those up in VRT with warp. Can the HDF5 driver
> be made to auto-detect these geolocation arrays?
>
> I see that the NETCDF driver actually does:
>
> gdalinfo "NetCDF:/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5"
> -sd 26
>
> I'm asking as an email rather than pursuing the fix because, these data
> are actually a regular grid on the north and south poles, and so
> geolocation by arrays is sub-optimal the specification is listed in
>
> https://nsidc.org/sites/default/files/au_si12-v001-userguide_1.pdf
>
> and the two parameter sets are
>
> Np-north: -te -3850000, -5350000, 3750000, 5850000 -t_srs EPSG:3411
> Sp-south: -te -3950000, -3950000, 3950000, 4350000 -t_srs EPSG:3412
>
> Is this generally something we should pursue within GDAL? It seems like an
> endless task to detect-on-open exactly this situation and assign the easy
> fix, but this is a pretty fundamental data stream and it's very common so
> the longlat/arrays might be numerically detectable with other
> heuristics hinting that it's polar (??) and there are plenty of other
> sources that present equivalents in the right way e.g. this one:
>
> "/vsicurl/
> https://noaadata.apps.nsidc.org/NOAA/G02135/north/daily/geotiff/2012/07_Jul/N_20120702_concentration_v3.0.tif
> "
>
> The right approach is probably to inform the providers and get the right
> metadata baked in ... but there's pros and cons to either. I'm not sure
> there's even enough information in these files to clearly detect the
> situation, it would be a bit like the NSIDCbin driver with its very strict
> requirements.
>
> Cheers, Mike
>
>
> --
>
> Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> Hobart, Australia
> e-mail: mdsumner at gmail.com
>
> _______________________________________________
> gdal-dev mailing listgdal-dev at lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsumner at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240430/c347716d/attachment.htm>
More information about the gdal-dev
mailing list