[Gdal-dev] Possible enhanced NetCDF support in GDAL.

Fri Mar 30 02:04:54 EDT 2007

Hello,

I've been using the gdal-1.2.6 distribtion as part of a MapServer build
and have had need to access netCDF files. I found the netCDF driver too
inflexible to read the files I needed to access and have customised the
driver code to be more flexible. The implementation I have currently
works very well and I'm happy to make it available for adapatation /
inclusion into future releases of gdal. I don't know the correct
protocol for this nor whether the gdal team are interested so I'm asking
you now. What should I do. 

To illustrate the changes I have made to the gdal netCDF driver and to
possibly gain your interest in it I'll now attempt to describe it. The
original driver assumes a particular netCDF file structure and variable
names. For my modified driver I have chosen to explicitely specify how
the driver should read the file in a generic fashion. I have achieved
this by having a xml file that specifies which nc file and how to
interpret it that you construct for every nc file that you wish to read.
Then when the driver opens a file it accepts either the properly
constructed xml file or the traditional nc file as a valid format. If an
nc file then it interprets it in identical fashion to the original
driver for backward compatibility. The xml file format required is as
follows and is taken directly from a comment block within my modified
code:

// This is a more advanced means of opening netCDF files that allows
raster 
// information to be extracted from files whose data is not held in the 
// structure assumed by the original netCDF driver code. The XML file
referenced 
// by poOpenInfo contains meta data instructing this driver on which cdf
file to 
// open and how it should interpret the data within.
// 
// XML file format for specification as follows:
//
//  <CDF>
//    <FileName>{name}</FileName>
//    <DimNameLon>{name}</DimNameLon>
//    <DimNameLat>{name}</DimNameLat>
//    <Variable>{name}</Variable>
//    <Units>{name}</Units>
//    <NumberOfBands>{value}</NumberOfBands>
//    <NoDataValue>{value}</NoDataValue>
//
//    <Extent>
//      <Top>{value}</Top>
//      <Left>{value}</Left>
//      <Bottom>{value}</Bottom>
//      <Right>{value}</Right>
//    </Extent>
//
//    <Range>
//      <Minimum>{value}</Minimum>
//      <Maximum>{value}</Maximum>
//
//      <ColorTable>
//        <NumberOfEntries>{value}</NumberOfEntries>
//
//        <Scheme>
//          <Hue>{Start value} {End value}</Hue>
//          <Lightness>{Start value} {End value}</Lightness>
//          <Saturation>{Start value} {End value}</Saturation>
//        </Scheme>
//
//        <Type>{name}</Type>
//        <Entry1>{Value 1} {Value 2} {Value3} {Value4}<Entry1>
//        <Entry2>{Value 1} {Value 2} {Value3} {Value4}<Entry2>
//        <Entry3>{Value 1} {Value 2} {Value3} {Value4}<Entry3>
//          .
//          .
//          .
//        <EntryX>{Value 1} {Value 2} {Value3} {Value4}<EntryX>
//      </ColorTable>
//    </Range> 
//
//    <Scaling>
//      <Scale>{value}</Scale>
//      <Offset>{value}</Offset>
//    </Scaling>
//
//    <Band1>
//      <{ExtraDimName1}>{value}</{ExtraDimName1}>
//      <{ExtraDimName2}>{value}</{ExtraDimName2}>
//        .
//        .
//        .
//      <{ExtraDimNameM}>{value}</{ExtraDimNameM}>
//    </Band1>
//
//    <Band2>
//      <{ExtraDimName1}>{value}</{ExtraDimName1}>
//      <{ExtraDimName2}>{value}</{ExtraDimName2}>
//        .
//        .
//        .
//      <{ExtraDimNameM}>{value}</{ExtraDimNameM}>
//    </Band2>
//      .
//      .
//      .
//    <BandN>
//      <{ExtraDimName1}>{value}</{ExtraDimName1}>
//      <{ExtraDimName2}>{value}</{ExtraDimName2}>
//        .
//        .
//        .
//      <{ExtraDimNameM}>{value}</{ExtraDimNameM}>
//    </BandN>
//  </CDF>
//
// FileName is the fully qualified name and path of the netCDF file you
are
// processing. DimNameLon and DimNameLat name the dimensions in the CDF
variable
// that correspond to longitudes and latitudes. If variable is a 2
dimension 
// array then NumberOfBands is implicitely 1. In such case NumberOfBands
is 
// irrelevent and so too are the <BandN> sections. 
//
// <Extent> is optional and specifies the co-ordinate extent of the
data. If 
// specified this is used to create the mapping for this raster data. 
//
// <Range> is optional and specifies the maximum and minimum data values
in 
// variable to consider to be usable data. 
//
// <Scaling> is optional and specifies the scale and offset parameters
to 
// apply to the variable data using simple linear scaling. 
//
// <ColourTable> is optional and if specified along with <Range> will
result in
// the driver colourising the image over the specified range using the
specified
// colour table. The colour entries are ordered from lowest to highest.
The 
// number of values required in each colour entry depends on the <Type>
which 
// may be one of Gray, RGB, CMYK or HLS. For Gray {Value1} is only
required. For
// RGB {Value1} is the red value, {Value2} is the green value, {Value3}
is the blue 
// value and {Value4}, which is optional, is the alpha value (alpha
blending?). 
// For CMYK {Value1} is the cyan value, {Value2} is the magenta value,
{Value3} 
// is the yellow value and {Value4} is the black value. For HLS {Value1}
is the 
// hue value, {Value2} is the lightness value and {Value3} is the
saturation value.
// The last index in the colour table is used to represent transparent
by setting
// the no data value to that index.  
//
// If <Scheme> is defined then the colour table colour entries and
<type> field
// are ignored and the colour table entries synthesised according to the
<Scheme>.
// The HLS colour type is used and the colour table entries are
generated by a 
// linear interpolation between the start and end HLS parameters. If
only one 
// value is given then the component is assumed constant with that
value.
//
// In the cases where the number of dimensions in Variable exceeds 2 you
must 
// specify the number of bands explicitely. Furthermore, each band must
have 
// a number of dimension definitions to specify the fixed dimension
values to
// use in accessing the Variable data. The entry names correspond to the

// dimension names in Variable and the values user specified. For an N
dimension 
// Variable you must have N-2 ExtraDimName entries whose names are
unique and 
// not duplicated in DimNameLon and DimNameLat. 

Modifications to the code were only made in the netcdfdataset.cpp and
gdalcolortable.cpp files and the changes are only ansi C/C++ without
reliance on other libraries. The changes I made to gdalcolortable.cpp
were ones to implement the HLS and CMYK colour models which had
skeletons but no implementations. 

Our typical application of this enhanced driver is to access and map
oceanographic data such as SST contained in annual or six monthly
aggregate files. These files typically have the data stored in 3d arrays
(lat, lon, time) and sometimes 4d arrays (lat, lon, time, depth). With
this modified driver it is simple to access a particular time slice for
viewing as map data. The ability to colourise the data with a
customisable colour table is a nice bonus. 

If you are interested in including it or a modified version of it into
new releases of gdal I'll gladly offer the code to you and bring it up
to date to the latest development version, though I can't promise much
commitment for furture maintenance. It would seem that this approach to
generalisation would possbily work well with other file formats such as
HDF.

Regards,

Paavo.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20070330/00ec0be1/attachment.html