[Gdal-dev] Anyone heard of ESML? (maybe off-topic)

[Frank Warmerdam]

Pushkar Pradhan wrote:
> I'm working on a project which requires us to read some of the scientific
> data formats like HDF-EOS, netCDF etc.
> Some colleagues suggested the use of Earth Science Markup Language
> (http://esml.itsc.uah.edu) which extends XML for this very task. They claim
> that with this user can define how the data is stored so that any format can
> be read.
> I think GDAL is able to read/write HDF/netCDF etc. and many other popular
> formats (can it read HDF-EOS a variant of HDF?).
> I'm trying to understand how ESML works. If anybody has heard about it, can
> you tell how it compares with GDAL? Would it be of any help to the GDAL
> project?

Pushkar,

I have quickly review the esml web site.  It seems ESML is an XML format
for describing the data in a seperate data file.  The file might be ascii,
"raw binary", HDF-EOS or GRIB format.   In the case of ascii or binary formats
the xml file itself has enough information to described the format such that
the raw data can be extracted.  In the case of HDF-EOS or GRIB it is assumed
that the application still "knows" about those formats, but the ESML file is
(in theory at least) providing a place to hang extra metadata.

EMSL also comes with a C++ and Python library for accessing ESML desribed files.
I assume this library must in turn link in HDF-EOS and GRIB libraries.  It
provides read (and write?) access to the files desribed by an ESML file.

In a sense then, the ESML libraries provide multi-format access to scientific
datasets, much like GDAL does, though only HDF-EOS, GRIB, and any format that
can be described by the ascii or binary data descriptions *and* that has an
esml file associated with it.

Now, the document and examples seem a bit thin.  It may be that esml and the
associated libraries do more than is immediately obvious.  But to me it seems
that they haven't really taken advantage of this XML file to allow alot of
extra metadata and semantic information to be associated with the datasets
as I would have expected.

I would not be adverse to implementing support for reading (and perhaps
writing) ESML decribed data, and perhaps even using ESML as the preferred GDAL
format for desrcibing raw data files.  I am not that keen on adding a dependence
on their libraries though that too is an option.   However, any such effort is
non-trivial and I'm not in a great hurry to do it.  If someone else wanted to
build a driver for it, that would be great.

I would also be more interested in using it as the preferred format for
GDAL raw files if it supportted arbitrary extra metadata (stuff like
colortables, coordinate systems and so forth) in some practical fashion.

Best regards,

-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent