[Gdal-dev] Anyone heard of ESML? (maybe off-topic)

[Chris G. Nicholas]

I'm know I'm dense, but can someone please explain what inherent advantage this newfangled XML shtuff has over plain 'ole looking at the file suffix (.tif, .bil, .jp2, .hdf, .shp, .ppm, .sid, etc), which one will have to do anyway?

and then again, (for you old timers!) there was (and still is!) a file's "magic" number.  'man magic' on a *NIX system yields:

"NAME
       magic - file command’s magic number file
 
DESCRIPTION
       This  manual page documents the format of the magic file as used by the
       file(1) command, version 3.39. The file command identifies the type  of
       a  file  using,  among  other tests, a test for whether the file begins
       with a certain magic number.  The file /usr/share/magic specifies  what
       magic numbers are to be tested for, what message to print if a particu-
       lar magic number is found, and additional information to  extract  from
       the file.       

       Each  line  of  the file specifies a test to be performed.  A test com-
       pares the data starting at a particular  offset  in  the  file  with  a
       1-byte,  2-byte, or 4-byte numeric value or a string..... 
"

> but the ESML file is (in theory at least) providing a place to hang extra metadata.

OK, ok...now I understand...more "catch-all" mechanisms.  I must confess that here at GlobeXplorer we extract all relevant header items and put the info into an external database, with hooks to collection-specific metadata schemas. I understand that geojp2 intends to 'put it all' into the headers as well.

but that's the nice thing with standards - you get to choose which one you like. :-)

I am facing something similar with my OpenGIS WMS service - when one clicks on an image for "FeatureInfo" - what do I return?

At this point, I am inclined to return a GlobeXplorer tracking_id, with XML schema scoped to com.globexplorer, with hooks to "drill-down" into that tracking_id for schema and values.

comments...suggestions...flames welcomed

Chris

-----Original Message-----
From:	Frank Warmerdam [mailto:warmerdam at pobox.com]
Sent:	Mon 1/26/2004 7:04 PM
To:	gdal-dev at remotesensing.org
Cc:	
Subject:	Re: [Gdal-dev] Anyone heard of ESML? (maybe off-topic)
Pushkar Pradhan wrote:
> I'm working on a project which requires us to read some of the scientific
> data formats like HDF-EOS, netCDF etc.
> Some colleagues suggested the use of Earth Science Markup Language
> (http://esml.itsc.uah.edu) which extends XML for this very task. They claim
> that with this user can define how the data is stored so that any format can
> be read.
> I think GDAL is able to read/write HDF/netCDF etc. and many other popular
> formats (can it read HDF-EOS a variant of HDF?).
> I'm trying to understand how ESML works. If anybody has heard about it, can
> you tell how it compares with GDAL? Would it be of any help to the GDAL
> project?

Pushkar,

I have quickly review the esml web site.  It seems ESML is an XML format
for describing the data in a seperate data file.  The file might be ascii,
"raw binary", HDF-EOS or GRIB format.   In the case of ascii or binary formats
the xml file itself has enough information to described the format such that
the raw data can be extracted.  In the case of HDF-EOS or GRIB it is assumed
that the application still "knows" about those formats, but the ESML file is
(in theory at least) providing a place to hang extra metadata.

EMSL also comes with a C++ and Python library for accessing ESML desribed files.
I assume this library must in turn link in HDF-EOS and GRIB libraries.  It
provides read (and write?) access to the files desribed by an ESML file.

In a sense then, the ESML libraries provide multi-format access to scientific
datasets, much like GDAL does, though only HDF-EOS, GRIB, and any format that
can be described by the ascii or binary data descriptions *and* that has an
esml file associated with it.

Now, the document and examples seem a bit thin.  It may be that esml and the
associated libraries do more than is immediately obvious.  But to me it seems
that they haven't really taken advantage of this XML file to allow alot of
extra metadata and semantic information to be associated with the datasets
as I would have expected.

I would not be adverse to implementing support for reading (and perhaps
writing) ESML decribed data, and perhaps even using ESML as the preferred GDAL
format for desrcibing raw data files.  I am not that keen on adding a dependence
on their libraries though that too is an option.   However, any such effort is
non-trivial and I'm not in a great hurry to do it.  If someone else wanted to
build a driver for it, that would be great.

I would also be more interested in using it as the preferred format for
GDAL raw files if it supportted arbitrary extra metadata (stuff like
colortables, coordinate systems and so forth) in some practical fashion.

Best regards,

-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent


_______________________________________________
Gdal-dev mailing list
Gdal-dev at remotesensing.org
http://remotesensing.org/mailman/listinfo/gdal-dev