<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:tax="http://schemas.microsoft.com/sharepoint/taxonomy/soap/" xmlns:tns="http://schemas.microsoft.com/sharepoint/soap/recordsrepository/" xmlns:spsup="http://microsoft.com/webservices/SharePointPortalServer/UserProfileService" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Texte brut Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.TextebrutCar
{mso-style-name:"Texte brut Car";
mso-style-priority:99;
mso-style-link:"Texte brut";
font-family:"Calibri","sans-serif";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.PlainText, li.PlainText, div.PlainText
{mso-style-name:"Plain Text";
mso-style-link:"Plain Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:Consolas;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Piyush,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Thanks for your help proposal!<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">I am working on porting changes to latest GDAL, I think it will be available tomorrow on GitHub.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">For this step I prefer finish it myself because I have already worked a lot on the NetCDF driver and it’s a big piece of code (10k lines!) not always easy to navigate...
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">My modifications are “generic” and CF-oriented so they should handle all sort of NetCDF4 files. Can I have access to some examples of the NC4 files you want to read?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Julien<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Agram, Piyush S (334D) [mailto:Piyush.Agram@jpl.nasa.gov]
<br>
<b>Envoyé :</b> dimanche 14 octobre 2018 03:01<br>
<b>À :</b> Julien Demaria; gdal-dev@lists.osgeo.org<br>
<b>Objet :</b> Re: parsing of netcdf4 no-data values in hdf5 reader<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US">Hi Julien,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> If you want to put up your code on github, I can volunteer to help port it to 2.x. Generating unittests would not be a problem. Will probably take existing datasets in the test suite and add a group around the datasets
to begin with. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Not very familiar with MODIS/VIIRS but looks like its nc4 with groups but all the dimensions are datasets under root. Maybe we can generalize this too?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Piyush<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:12.0pt;color:black">From:
</span></b><span lang="EN-US" style="font-size:12.0pt;color:black">Julien Demaria <julien.demaria@acri-st.fr><br>
<b>Date: </b>Saturday, October 13, 2018 at 4:09 PM<br>
<b>To: </b>"Agram, Piyush S (334D)" <Piyush.Agram@jpl.nasa.gov>, "gdal-dev@lists.osgeo.org" <gdal-dev@lists.osgeo.org><br>
<b>Subject: </b>RE: parsing of netcdf4 no-data values in hdf5 reader<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<p class="MsoPlainText"><span lang="EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">I have already already implemented NetCDF4 groups support for GDAL in 2015 and use it to read for example NASA OceanColor MODIS/VIIRS L2 products, see the attached email.<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">Unfortunately I never found enough time to add corresponding GDAL autotests (covering all my modifications) to be able to provide these improvements to the GDAL community.<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">The other problem is that I implemented this in GDAL 1.11 so porting to latest GDAL need some work, I started to do that in 2016 but never finished...<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">But I think my code is good and it will be a pity if not reused...<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">I think I can try to finish the porting to the latest GDAL and provide it to GitHub, but I'm still not sure to have enough time to implement all needed corresponding GDAL autotests. If you want to contribute you are
welcome! ;-)<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">Best Regards,<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">Julien<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">-----Message d'origine-----<br>
De : gdal-dev [mailto:gdal-dev-bounces@lists.osgeo.org] De la part de Agram, Piyush S (334D)<br>
Envoyé : samedi 13 octobre 2018 21:00<br>
À : gdal-dev@lists.osgeo.org<br>
Objet : Re: [gdal-dev] parsing of netcdf4 no-data values in hdf5 reader<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">Hi Even,<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> Looking into the issue, I think the NETCDF reader does not read datasets located insides groups for Rasters. There are 2 specific parts of the reader that seems to be the issue:<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">1. If the drivers does not find 2 arrays that are dimensions under root it bails. This would need to be updated if data is organized within groups and there are no dims/vars at the root level.<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">2. It only looks into the groups when used for reading vectors and when it does it only parses the string types for geometry definitions. This would need to be updated.
<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">Some related questions to see if anybody is already planning to contribute these capabilities:<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">1. Update to NETCDF raster driver to look into sub-groups as well? The spec does not seem to talk about this. Assuming they are trying to be backward compatible with formats that did not support groups - is this acceptable?<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">2. Support for CF conventions in HDF5 reader? NASA/ESRI seem to be happy working with CF-compliant HDF5.<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">Adding either support will let people use GDAL with xarray etc.
<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">Piyush <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">_______________________________________________<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">gdal-dev mailing list<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">gdal-dev@lists.osgeo.org<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US">https://lists.osgeo.org/mailman/listinfo/gdal-dev<o:p></o:p></span></p>
</div>
</body>
</html>