<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.gmailsignatureprefix
{mso-style-name:gmail_signature_prefix;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">></span> This HDF5 (requires earthdata credentials your "Authorization: Bearer <token>" in GDAL_HTTP_HEADERS, or equiv) presents without geolocation arrays. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">gdalinfo "/vsicurl/<a href="https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5">https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5</a>" -sd 26<br>
<br>
<span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Excuse for this little derail, but how do we do that? I mean, the credentials. I tried with both:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gdalinfo --config GDAL_HTTP_HEADERS=login:passw "/vsicurl/https://n5eil01u.ecs.nsidc.org/…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">and<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="NO-BOK" style="font-size:11.0pt">gdalinfo "/vsicurl/https://login:passwd@n5eil01u.ecs.nsidc.org…<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NO-BOK" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="NO-BOK" style="font-size:11.0pt">but both errored with<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NO-BOK" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">ERROR 3: Cannot read 4029 bytes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gdalinfo failed - unable to open '/vsicurl/https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5'.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">and<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">ERROR 3: Cannot read 4029 bytes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gdalinfo failed - unable to open '/vsicurl/https://login:passwd@n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5'.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Joaquim<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> gdal-dev <gdal-dev-bounces@lists.osgeo.org>
<b>On Behalf Of </b>Michael Sumner via gdal-dev<br>
<b>Sent:</b> Monday, April 29, 2024 8:11 PM<br>
<b>To:</b> gdal-dev <gdal-dev@lists.osgeo.org><br>
<b>Subject:</b> [gdal-dev] HDF5 and geolocation arrays<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">This HDF5 (requires earthdata credentials your "Authorization: Bearer <token>" in GDAL_HTTP_HEADERS, or equiv) presents without geolocation arrays. <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">gdalinfo "/vsicurl/<a href="https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5">https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5</a>" -sd 26<br>
Driver: HDF5Image/HDF5 Dataset<br>
Files: /vsicurl/<a href="https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5">https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5</a><br>
Size is 608, 896<br>
Metadata:<br>
Conventions=CF-1.6<br>
HDFEOS_INFORMATION_HDFEOSVersion=HDFEOS_5.1.15<br>
history=This version of the Sea Ice processing code contains updates provided by the science team on September 16, 2019. For details on these updates, see the release notes provided in the DAP.<br>
institution=NASA's AMSR Science Investigator-led Processing System (SIPS)<br>
references=Please cite these data as: Markus, T., J. C. Comiso, and W. N. Meier. 2018. AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids, Version 1. [Indicate subset used]. Boulder, Colorado
USA. NASA National Snow and Ice Data Center Distributed Active Archive Center. doi:
<a href="https://doi.org/10.5067/RA1MIJOYPK3P">https://doi.org/10.5067/RA1MIJOYPK3P</a>.<br>
source=satellite observation<br>
title=AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids<br>
Corner Coordinates:<br>
Upper Left ( 0.0, 0.0)<br>
Lower Left ( 0.0, 896.0)<br>
Upper Right ( 608.0, 0.0)<br>
Lower Right ( 608.0, 896.0)<br>
Center ( 304.0, 448.0)<br>
Band 1 Block=608x1 Type=Int32, ColorInterp=Undefined<br>
Metadata:<br>
comment=data value meaning: 0 -- Open Water, 110 -- missing/not calculated, 120 -- Land<br>
coordinates=lon lat<br>
long_name=Sea ice concentration daily average<br>
units=percent<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">gdalinfo --version<br>
GDAL 3.9.0dev-cb4d30f56d, released 2024/04/15 <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The geolocation arrays are sds 33 and 32 respectively: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">HDF5:"/vsicurl/<a href="https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5">https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5</a>"://HDFEOS/GRIDS/NpPolarGrid12km/lon<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">HDF5:"/vsicurl/<a href="https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5">https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5</a>"://HDFEOS/GRIDS/NpPolarGrid12km/lat<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">And things work when lining those up in VRT with warp. Can the HDF5 driver be made to auto-detect these geolocation arrays? <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I see that the NETCDF driver actually does: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">gdalinfo "NetCDF:/vsicurl/<a href="https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5">https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5</a>"
-sd 26<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm asking as an email rather than pursuing the fix because, these data are actually a regular grid on the north and south poles, and so geolocation by arrays is sub-optimal the specification is listed in <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://nsidc.org/sites/default/files/au_si12-v001-userguide_1.pdf">https://nsidc.org/sites/default/files/au_si12-v001-userguide_1.pdf</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">and the two parameter sets are<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Np-north: -te -3850000, -5350000, 3750000, 5850000 -t_srs EPSG:3411<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Sp-south: -te -3950000, -3950000, 3950000, 4350000 -t_srs EPSG:3412<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Is this generally something we should pursue within GDAL? It seems like an endless task to detect-on-open exactly this situation and assign the easy fix, but this is a pretty fundamental data stream and it's very common so the longlat/arrays
might be numerically detectable with other heuristics hinting that it's polar (??) and there are plenty of other sources that present equivalents in the right way e.g. this one: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">"/vsicurl/<a href="https://noaadata.apps.nsidc.org/NOAA/G02135/north/daily/geotiff/2012/07_Jul/N_20120702_concentration_v3.0.tif">https://noaadata.apps.nsidc.org/NOAA/G02135/north/daily/geotiff/2012/07_Jul/N_20120702_concentration_v3.0.tif</a>"<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The right approach is probably to inform the providers and get the right metadata baked in ... but there's pros and cons to either. I'm not sure there's even enough information in these files to clearly detect the situation, it would be
a bit like the NSIDCbin driver with its very strict requirements. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers, Mike<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><span class="gmailsignatureprefix">--</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Michael Sumner<br>
Software and Database Engineer<br>
Australian Antarctic Division<br>
Hobart, Australia<br>
e-mail: <a href="mailto:mdsumner@gmail.com" target="_blank">mdsumner@gmail.com</a><o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>