[gdal-dev] HDF5 and geolocation arrays

Mon Apr 29 17:53:29 PDT 2024

you need

gdalinfo --config GDAL_HTTP_HEADERS="Authorization: Bearer <long token>"
 "/vsicurl/https://n5eil01u.ecs.nsidc.org/…

Where "<long token>" is an Earthdata access token from your account.

https://urs.earthdata.nasa.gov/documentation/for_users/user_token

HTH   (there are other methods like setting GDAL_HTTP_HEADER_FILE to an
actual file with that "Authorization: Bearer ..." content.

On Tue, Apr 30, 2024 at 7:37 AM Joaquim Manuel Freire Luís <jluis at ualg.pt>
wrote:

> > This HDF5 (requires earthdata credentials your "Authorization: Bearer
> <token>" in GDAL_HTTP_HEADERS, or equiv) presents without geolocation
> arrays.
>
>
>
> gdalinfo "/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5"
> -sd 26
>
>
>
> Excuse for this little derail, but how do we do that? I mean, the
> credentials. I tried with both:
>
>
>
> gdalinfo --config GDAL_HTTP_HEADERS=login:passw "/vsicurl/
> https://n5eil01u.ecs.nsidc.org/…
>
>
>
> and
>
>
>
> gdalinfo  "/vsicurl/https://login:passwd@n5eil01u.ecs.nsidc.org…
>
>
>
> but both errored with
>
>
>
> ERROR 3: Cannot read 4029 bytes
>
> gdalinfo failed - unable to open '/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> '.
>
>
>
> and
>
>
>
> ERROR 3: Cannot read 4029 bytes
>
> gdalinfo failed - unable to open '/vsicurl/
> https://login:passwd@n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> '.
>
>
>
>
>
> Thanks
>
>
>
> Joaquim
>
>
>
> *From:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> *On Behalf Of *Michael
> Sumner via gdal-dev
> *Sent:* Monday, April 29, 2024 8:11 PM
> *To:* gdal-dev <gdal-dev at lists.osgeo.org>
> *Subject:* [gdal-dev] HDF5 and geolocation arrays
>
>
>
> This HDF5 (requires earthdata credentials your "Authorization: Bearer
> <token>" in GDAL_HTTP_HEADERS, or equiv) presents without geolocation
> arrays.
>
>
>
> gdalinfo "/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5"
> -sd 26
> Driver: HDF5Image/HDF5 Dataset
> Files: /vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> Size is 608, 896
> Metadata:
>   Conventions=CF-1.6
>   HDFEOS_INFORMATION_HDFEOSVersion=HDFEOS_5.1.15
>   history=This version of the Sea Ice processing code contains updates
> provided by the science team on September 16, 2019. For details on these
> updates, see the release notes provided in the DAP.
>   institution=NASA's AMSR Science Investigator-led Processing System (SIPS)
>   references=Please cite these data as: Markus, T., J. C. Comiso, and W.
> N. Meier. 2018. AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness
> Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids,
> Version 1. [Indicate subset used]. Boulder, Colorado USA. NASA National
> Snow and Ice Data Center Distributed Active Archive Center. doi:
> https://doi.org/10.5067/RA1MIJOYPK3P.
>   source=satellite observation
>   title=AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, Sea
> Ice Concentration, Motion & Snow Depth Polar Grids
> Corner Coordinates:
> Upper Left  (    0.0,    0.0)
> Lower Left  (    0.0,  896.0)
> Upper Right (  608.0,    0.0)
> Lower Right (  608.0,  896.0)
> Center      (  304.0,  448.0)
> Band 1 Block=608x1 Type=Int32, ColorInterp=Undefined
>   Metadata:
>     comment=data value meaning: 0 -- Open Water, 110 -- missing/not
> calculated, 120 -- Land
>     coordinates=lon lat
>     long_name=Sea ice concentration daily average
>     units=percent
>
>
>
>
>
>
>
> gdalinfo --version
> GDAL 3.9.0dev-cb4d30f56d, released 2024/04/15
>
>
>
> The geolocation arrays are sds 33 and 32 respectively:
>
>
>
> HDF5:"/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> "://HDFEOS/GRIDS/NpPolarGrid12km/lon
>
>
>
> HDF5:"/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5
> "://HDFEOS/GRIDS/NpPolarGrid12km/lat
>
>
>
> And things work when lining those up in VRT with warp. Can the HDF5 driver
> be made to auto-detect these geolocation arrays?
>
>
>
> I see that the NETCDF driver actually does:
>
>
>
> gdalinfo "NetCDF:/vsicurl/
> https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5"
> -sd 26
>
>
>
> I'm asking as an email rather than pursuing the fix because, these data
> are actually a regular grid on the north and south poles, and so
> geolocation by arrays is sub-optimal  the specification is listed in
>
>
>
> https://nsidc.org/sites/default/files/au_si12-v001-userguide_1.pdf
>
>
>
> and the two parameter sets are
>
>
>
> Np-north: -te -3850000,  -5350000, 3750000, 5850000 -t_srs EPSG:3411
>
> Sp-south: -te -3950000,  -3950000, 3950000, 4350000 -t_srs EPSG:3412
>
>
>
> Is this generally something we should pursue within GDAL? It seems like an
> endless task to detect-on-open exactly this situation and assign the easy
> fix, but this is a pretty fundamental data stream and it's very common so
> the longlat/arrays might be numerically detectable with other
> heuristics hinting that it's polar (??) and there are plenty of other
> sources that present equivalents in the right way e.g. this one:
>
>
>
> "/vsicurl/
> https://noaadata.apps.nsidc.org/NOAA/G02135/north/daily/geotiff/2012/07_Jul/N_20120702_concentration_v3.0.tif
> "
>
>
>
> The right approach is probably to inform the providers and get the right
> metadata baked in ... but there's pros and cons to either. I'm not sure
> there's even enough information in these files to clearly detect the
> situation, it would be a bit like the NSIDCbin driver with its very strict
> requirements.
>
>
>
> Cheers, Mike
>
>
>
>
>
> --
>
>
>
> Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> Hobart, Australia
> e-mail: mdsumner at gmail.com
>

-- 
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsumner at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240430/c971a35d/attachment-0001.htm>