[gdal-dev] Re: discussion on improvements to the NetCDF driver and CF-1 convention

[Etienne Tourigny]

Hi all,

Since a few people have shown an interest in helping out, I would suggest to set up space where we can post ideas, goals and test files.  This could take the form of one of the following:

- a new wiki entry in trac, or the existing one at http://trac.osgeo.org/gdal/wiki/NetCDF
- a file repository at http://download.osgeo.org/gdal/data/netcdf as suggested some time ago by warmerdam

and/or

- a new ticket in trac (which allows to post files)

What are you thoughts about this?  Would it make sense to eventually prepare an RFC?



I would suggest that each group identify the problems with the current driver in terms of importing and exporting, post the files that they would like to be supported, which other software they use with the netcdf files, and their proposed contribution (code, test, ideas, etc.).

Those that cannot participate in code development can help out in testing and discussion. Personally, I can write and test code, given example files and expected outcome. 


Now to your specific comments, Knut-Frode:

1) Yes CF-1.5 should be the goal, although I'm not sure we need to support upgrading/downgrading.

5) The WGS84 issue needs further research. Weather and ocean data are in lat/lon, but what is the sphere radius assumed by the various data providers and  the models which output/import netcdf files? I agree that a small error does not make a difference with low-resolution dataset, but I'd like to be sure in any case. We also need a means to translate the CF-1.x datum information into EPSG/Proj.4 values.


7) There is some support of a z-axis right now, in fact they driver assumes that a third dimension is vertical if I understand correctly. However, I have not tested nor looked into that.  This issue definitely needs come planning and an RFC, IMHO.


Kyle, thanks for you help and interest!


cheers,
Etienne


________________________________
From: Knut-Frode Dagestad <knutfrodesoppel at hotmail.com>
To: gdal-dev at lists.osgeo.org
Sent: Friday, August 26, 2011 4:41:02 AM
Subject: [gdal-dev] Re: discussion on improvements to the NetCDF driver and CF-1 convention

Etienne,

My group is also very much in support of the suggested improvements, as NetCDF/CF is becoming the most important standard for data storage/exchange in our community (climate, meteorology, oceanography). My group is not much into GDAL driver development (at least not yet), but we are now building our future very much around GDAL, so we are interested to join the discussion around improved support for NetCDF/CF reading and writing.

About some of the mentioned issues:

1) The latest version of CF is 1.5, so it would be better to target 1.5 (or 1.x) than 1.0. Ideally the driver could be made generic for smooth upgrade to newer (or downgrade to older) versions of the CF conventions.

5) It sounds meaningful to assume WGS84 datum if not specified. I believe at least 99% of data typically stored in NetCDF/CF will have this datum, and even when that is not the case, positioning errors less than 100 m should normally have little impact for the types of data. Precision on datum level is probably more relevant for more typical GIS-data with roads, borders etc.
A warning could however be issued each time the driver has to assume WGS84.

7) We agree in particular that proper handling of time axis is of high value. This is one of the issues which is highly relevant for climate-data (often 4-dimensional), but not well supported by typical GIS-software or standards designed with 2-dimensional data in mind.
A general representation of a time-axis (or even vertical axis) in GDAL would be ideal, but that would probably be too much an effort for anyone to develop.


Knut-Frode
Nansen Environmental and Remote Sensing Center



On 24/08/2011 22:37, Etienne Tourigny wrote:
> Hi all,
> 
> I would like to start a discussion with those interested about fixing
> various issues in the NetCDF driver.
> 
> As it stands, it is not fully CF-1.0 compliant, and produces
> geographical grids that are not valid for other software.
> Also, the infamous "up-side down" problem has been addressed for
> importing netcdfs, but needs to be addressed for exporting.
> 
> Related ticket: http://trac.osgeo.org/gdal/ticket/2129
> I have been submitting patches for some time to fix the related issues.
> 
> I have identified a number of issues, of which the following have been
> fixed partially in my proposed patch.
> 
> 1- conform to the cf-1.0 standard for geographical grids (creating lat
> and lon dimensions and variables)
> 2- make netcdf grids "bottom-up" by default, as all software out there
> uses that convention
> 3- fix problems related to floating-point / string conversions (also
> reported in issue #4200 )
> 4- fix metadata handling and duplication (as reported in issue #4204),
> and keep band (variable) metadata outside of the global metadata
> 
> Pending issues:
> 
> 5- how to deal with netcdfs which have no datum? Can we assume WGS84?
> These files are extremely common, and CF-1.0 has provisions for
> identifying the key variables, but no Proj/Wkt codes.
> 6- fix proper export of projected CRS (this is not fully necessary
> though) with lat/lon values respecting CF-1.0
> 7- handle time axis properly (at least when creating a netcdf) - that
> would be great for import/export of netcdf
> 
> Regards,
> Etienne
> 
> 
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev