[gdal-dev] Re: discussion on improvements to the NetCDF driver
and CF-1 convention
Brian Case
rush at winkey.org
Fri Sep 2 11:49:03 EDT 2011
Etienne
I can read it with ogr. I am wondering if others have interest too
http://svn.osgeo.org/gdal/sandbox/winkey/ogr-netcdf/ogrsf_frmts/netcdf/
Brian
On Fri, 2011-09-02 at 10:57 -0300, Etienne Tourigny wrote:
> There has been recent work on incorporating netcdf support for point
> and trajectory data with Talend, maybe that can help?
>
>
> http://www.neogeo-online.net/blog/archives/1350/
> http://talendforge.org/svn/sdi/trunk/org.talend.sdi.designer.components.sandbox/components/sNetCDFInput/
>
>
> Etienne
>
> On Tue, Aug 30, 2011 at 10:18 AM, Etienne Tourigny
> <etiennesky at yahoo.com> wrote:
> Brian,
>
> I am not familiar with madis data. I assume that it is
> point-data for meteorological stations?
>
> If you like, please post your ideas and a short description in
> a new topic in the wiki entry at
> http://trac.osgeo.org/gdal/wiki/NetCDF_Improvements .
>
> Etienne
>
>
>
>
> ----- Original Message -----
> From: Brian Case <rush at winkey.org>
> To: Etienne Tourigny <etiennesky at yahoo.com>
> Cc: "gdal-dev at lists.osgeo.org" <gdal-dev at lists.osgeo.org>
> Sent: Sunday, August 28, 2011 7:54:13 PM
> Subject: Re: [gdal-dev] Re: discussion on improvements to the
> NetCDF driver and CF-1 convention
>
> any thoughts on a ogr netcdf driver? I have a simple one for
> madis data
> that needs work in my sandbox
>
> Brian
>
>
>
> On Sun, 2011-08-28 at 14:56 -0700, Etienne Tourigny wrote:
> > I have created a new wiki page at
> http://trac.osgeo.org/gdal/wiki/NetCDF_Improvements ,
> following Even's suggestion (Thanks Even).
> >
> > The following topics have been included:
> >
> >
> > Topics:
> >
> > 1 - UNIDATA NetCDF Conventions
> >
> > 2 - Metadata
> > 3 - Datum issues
> > 4 - Dimension issues
> > 5 - Compatibility with other software
> > 6 - Test files
> > 7 - Contribution
> > Discussion
> >
> > I encourage interested parties in participating in this
> discussion in the wiki and in the mailing list.
> >
> >
> > In particular, it would be useful to collect information on
> contributors and obtain test files with issues and the
> expected outcome.
> >
> > regards, Etienne
> >
> >
> > ----- Original Message -----
> > From: Even Rouault <even.rouault at mines-paris.org>
> > To: gdal-dev at lists.osgeo.org; Etienne Tourigny
> <etiennesky at yahoo.com>
> > Cc:
> > Sent: Friday, August 26, 2011 4:20:06 PM
> > Subject: Re: [gdal-dev] Re: discussion on improvements to
> the NetCDF driver and CF-1 convention
> >
> > Le vendredi 26 août 2011 17:15:47, Etienne Tourigny a
> écrit :
> >
> > My thoughts, just from a process point of view, not being
> myself deeply
> > involved in the NetCDF format :
> >
> > * A wiki page (I'd suggest either a new page -
> > http://trac.osgeo.org/gdal/wiki/NetCDF_Improvements - or at
> least a clearly
> > identified section in the current page, to avoid confusion
> for outsiders about
> > what is currently implemented and what is not) can be a
> convenient way of
> > structuring and editing the content by anyone with a osgeo
> id. But discussions
> > on the mailing list are also OK, at least to notify new
> content has been added
> > (provided that it doesn't cause flooding on the mailing
> list ;-))
> >
> > * A http://download.osgeo.org/gdal/data/netcdf directory
> would be the right
> > place to store NetCDF sample files, provided they are free
> under the terms of
> > http://download.osgeo.org/gdal/data/COPYING . As write
> access to it is
> > restricted to people with appropriate rights, having those
> rights, I can help
> > uploading files over there. Having small (typically < or ~
> 10 KB ) sample files
> > is also good so they can be directly included in SVN under
> the
> > autotest/gdrivers/data directory and thus easily used by
> regression tests.
> >
> > * Tickets can be used to track the implementation of
> specific items and
> > bugfixes. But I've found they are not generally suitable for
> brainstorming when
> > the discussion is lasting, as you cannot edit already posted
> content. Wiki is
> > clearly better for this.
> >
> > * Looking at past history, I see we haven't yet used the RFC
> formalism for
> > work specific to one driver, while it doesn't require
> changes or extensions to
> > the GDAL API. But it wouldn't hurt to create one if the need
> arises. Anyway, a
> > point that might require care and some consensus among the
> interested parties
> > is about how backward compatibility issues, in particular
> with respect to files
> > produced by older GDAL versions, are adressed.
> > http://trac.osgeo.org/gdal/wiki/rfc1_pmc details when a vote
> is required.
> >
> > Best regards,
> >
> > Even
> >
> > > Hi all,
> > >
> > > Since a few people have shown an interest in helping out,
> I would suggest
> > > to set up space where we can post ideas, goals and test
> files. This could
> > > take the form of one of the following:
> > >
> > > - a new wiki entry in trac, or the existing one at
> > > http://trac.osgeo.org/gdal/wiki/NetCDF - a file repository
> at
> > > http://download.osgeo.org/gdal/data/netcdf as suggested
> some time ago by
> > > warmerdam
> > >
> > > and/or
> > >
> > > - a new ticket in trac (which allows to post files)
> > >
> > > What are you thoughts about this? Would it make sense to
> eventually
> > > prepare an RFC?
> > >
> > >
> > >
> > > I would suggest that each group identify the problems with
> the current
> > > driver in terms of importing and exporting, post the files
> that they would
> > > like to be supported, which other software they use with
> the netcdf files,
> > > and their proposed contribution (code, test, ideas, etc.).
> > >
> > > Those that cannot participate in code development can help
> out in testing
> > > and discussion. Personally, I can write and test code,
> given example files
> > > and expected outcome.
> > >
> > >
> > > Now to your specific comments, Knut-Frode:
> > >
> > > 1) Yes CF-1.5 should be the goal, although I'm not sure we
> need to support
> > > upgrading/downgrading.
> > >
> > > 5) The WGS84 issue needs further research. Weather and
> ocean data are in
> > > lat/lon, but what is the sphere radius assumed by the
> various data
> > > providers and the models which output/import netcdf
> files? I agree that a
> > > small error does not make a difference with low-resolution
> dataset, but
> > > I'd like to be sure in any case. We also need a means to
> translate the
> > > CF-1.x datum information into EPSG/Proj.4 values.
> > >
> > >
> > > 7) There is some support of a z-axis right now, in fact
> they driver assumes
> > > that a third dimension is vertical if I understand
> correctly. However, I
> > > have not tested nor looked into that. This issue
> definitely needs come
> > > planning and an RFC, IMHO.
> > >
> > >
> > > Kyle, thanks for you help and interest!
> > >
> > >
> > > cheers,
> > > Etienne
> > >
> > >
> > > ________________________________
> > > From: Knut-Frode Dagestad <knutfrodesoppel at hotmail.com>
> > > To: gdal-dev at lists.osgeo.org
> > > Sent: Friday, August 26, 2011 4:41:02 AM
> > > Subject: [gdal-dev] Re: discussion on improvements to the
> NetCDF driver and
> > > CF-1 convention
> > >
> > > Etienne,
> > >
> > > My group is also very much in support of the suggested
> improvements, as
> > > NetCDF/CF is becoming the most important standard for data
> > > storage/exchange in our community (climate, meteorology,
> oceanography). My
> > > group is not much into GDAL driver development (at least
> not yet), but we
> > > are now building our future very much around GDAL, so we
> are interested to
> > > join the discussion around improved support for NetCDF/CF
> reading and
> > > writing.
> > >
> > > About some of the mentioned issues:
> > >
> > > 1) The latest version of CF is 1.5, so it would be better
> to target 1.5 (or
> > > 1.x) than 1.0. Ideally the driver could be made generic
> for smooth upgrade
> > > to newer (or downgrade to older) versions of the CF
> conventions.
> > >
> > > 5) It sounds meaningful to assume WGS84 datum if not
> specified. I believe
> > > at least 99% of data typically stored in NetCDF/CF will
> have this datum,
> > > and even when that is not the case, positioning errors
> less than 100 m
> > > should normally have little impact for the types of data.
> Precision on
> > > datum level is probably more relevant for more typical
> GIS-data with
> > > roads, borders etc. A warning could however be issued each
> time the driver
> > > has to assume WGS84.
> > >
> > > 7) We agree in particular that proper handling of time
> axis is of high
> > > value. This is one of the issues which is highly relevant
> for climate-data
> > > (often 4-dimensional), but not well supported by typical
> GIS-software or
> > > standards designed with 2-dimensional data in mind. A
> general
> > > representation of a time-axis (or even vertical axis) in
> GDAL would be
> > > ideal, but that would probably be too much an effort for
> anyone to
> > > develop.
> > >
> > >
> > > Knut-Frode
> > > Nansen Environmental and Remote Sensing Center
> > >
> > > On 24/08/2011 22:37, Etienne Tourigny wrote:
> > > > Hi all,
> > > >
> > > > I would like to start a discussion with those interested
> about fixing
> > > > various issues in the NetCDF driver.
> > > >
> > > > As it stands, it is not fully CF-1.0 compliant, and
> produces
> > > > geographical grids that are not valid for other
> software.
> > > > Also, the infamous "up-side down" problem has been
> addressed for
> > > > importing netcdfs, but needs to be addressed for
> exporting.
> > > >
> > > > Related ticket: http://trac.osgeo.org/gdal/ticket/2129
> > > > I have been submitting patches for some time to fix the
> related issues.
> > > >
> > > > I have identified a number of issues, of which the
> following have been
> > > > fixed partially in my proposed patch.
> > > >
> > > > 1- conform to the cf-1.0 standard for geographical grids
> (creating lat
> > > > and lon dimensions and variables)
> > > > 2- make netcdf grids "bottom-up" by default, as all
> software out there
> > > > uses that convention
> > > > 3- fix problems related to floating-point / string
> conversions (also
> > > > reported in issue #4200 )
> > > > 4- fix metadata handling and duplication (as reported in
> issue #4204),
> > > > and keep band (variable) metadata outside of the global
> metadata
> > > >
> > > > Pending issues:
> > > >
> > > > 5- how to deal with netcdfs which have no datum? Can we
> assume WGS84?
> > > > These files are extremely common, and CF-1.0 has
> provisions for
> > > > identifying the key variables, but no Proj/Wkt codes.
> > > > 6- fix proper export of projected CRS (this is not fully
> necessary
> > > > though) with lat/lon values respecting CF-1.0
> > > > 7- handle time axis properly (at least when creating a
> netcdf) - that
> > > > would be great for import/export of netcdf
> > > >
> > > > Regards,
> > > > Etienne
> > > >
> > > >
> > > > _______________________________________________
> > > > gdal-dev mailing list
> > > > gdal-dev at lists.osgeo.org
> > > > http://lists.osgeo.org/mailman/listinfo/gdal-dev
> > >
> > > _______________________________________________
> > > gdal-dev mailing list
> > > gdal-dev at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/gdal-dev
> > > _______________________________________________
> > > gdal-dev mailing list
> > > gdal-dev at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/gdal-dev
> > _______________________________________________
> > gdal-dev mailing list
> > gdal-dev at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/gdal-dev
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
More information about the gdal-dev
mailing list