[gdal-dev] Re: [gfc-dev] GFC

[Frank Warmerdam]

Liujian Qian wrote:
> Have you thought about providing GDALDriver-based interface to things like HDF
> library, which is very complex, but already has a well-developed programming
> interface (it's own driver)? (All this gets me thinking that OGC should push on
> a raster specification RSN, so that you can make GDALDriver mimic that, and
> authors of proprietary formats can easily come up with their subclasses of
> GDALDriver and register it with your GDALDriverManager.)

L.J.

I have written a GeoGateway interface to the HDF EOS library in the past, 
and hope to do so again for GDAL. I would also like to support HDF datasets
that are not EOS as part of that effort. In short, yes, I do imagine there
being an GDALHDFDriver class which acts as a sort of bridge to the HDF
(and HDF EOS) libraries. 

This is a challenging one to do because HDF allows for very complex 
datasets, with many grids of different sizes, different regions and so
forth. 

> Frank> The GDALDriver derived driver class is also responsible for
> Frank> dataset opening, and creation. Once the dataset is instantiated
> Frank> all further data access is the responsibility of the dataset
> Frank> (GDALDataset).
> 
> I assume that there will be many interactions between a GDALDriver and the
> GDALDatasets it creats? Say if a dataset wants to check out a particular
> data item it will use the corresponding GDALDriver to get the actual data off a
> formated file or something like? Do you have a protocol for the message
> passing between these two types of objects?

I actually see the GDALDriver's as being fairly simple, basically providing
some boot strapping support for opening, and creating datasets, but that
virtually all the work would be done by the GDALDataset itself. In that
sense, I don't see alot of interactions between the classes. 

Note that the GDALDriver and GDALDataset are both simple objects in the
current process space. Message passing is via normal object method calls. 

> Frank> In theory one GDALDriver might be responsible for handling a
> Frank> more than one GDALDataset derived class, though it isn't clear
> Frank> why this would be needed.
> 
> Do you mean one GDALDriver actually *owns* those datasets? I thought it probably
> is a good idea to maintain only a loose connection between GDALDatasets and
> GDALDrivers, without each knowing too much about the other. But I could be wrong.

It seems permissible for a particular GDALDriver to have some intimate 
knowledge of the GDALDataset derived class it is implemented to create. 
In general though (at the base class level for instance) I don't see any
special permissions existing. 

> Do you think a cache for blocks/tiles of recently processed raster will be useful? I
> was thinking about doing that for my GFC, where each grid will have a tag/key string
> and the tile cache will look it up there first for a desired tile using the
> key string. If it founds it there there is no need for physical I/O; otherwise the
> normal disk read/write occur.

Within GeoGateway I implemented a simple raster tile cache a couple of 
years ago. Recently this was extended to a sophisticated tiled caching
system with global memory managers to control strategy.

It isn't clear to me too what extent I want to build caching into GDAL, 
but it does seem obvious that some minimal caching will eventually be
needed. However, one criteria would be that application code can easily
turn all caching off to avoid extra memory overhead in situations where
the applications are providing their own caching, or know that they don't
need any. 

> Yeah I agree with you. That's why I have the customHeader() methods in the importer
> class. The idea is that GGridImporter puts whatever specialized information it deems
> not of general interest in a custom header (an opaque block of data) of the GGrid.
> It's the specialized GGrid subclasses, such as GDem or GRMImage, that can interpret
> the custom header properly.

While this approach has some utility, it really gives very little opportunity
for code not specifically aware of that format (or class) to utilize the
data.

Within OGDI the updateDictionary() routine allows you to fetch a (potentially
long) string associated with a particular name. Some names are specific
to that format, and might not have meaning to much application code. 
Other names can be used to share extension data by convention between
a variety of drivers, and application code.

A similar approach was used with GeoGateway and the MetaDataIO() method. 

In GeoSoft DATs they also have a similar extension mechanism. 

My suggestion then is that you modify your customHeader() method to take
a name parameter, and perhaps to give a clue whether the returned data
can be interpreted as printable text, or only binary. 

In GeoGateway the Metadata mechanism was used for a variety of things
but well known conventions were established for some of them. For
instance a metadata item named "NO_DATA_VALUE" would contain as it's
value a string representation of the raster number which was considered
to be a no-data, or dummy value.

Metadata also provided a mechanism to return arbitrary text name value
pairs for specific formats. For instance the SpotSAR file format contained
a variety of interesting information about the acquisition parameters of
the dataset. They were all returned as metadata items where the name 
portion was prefixed by ``SpotSAR_''. The data didn't mean anything specific
to most applications, but a user could easily review it.

I haven't developed a metadata interface for GDAL yet. I have briefly
reviewed FDGC metadata standards from the states, and a couple of different
metadata representations, including the USGS/NASA ODL (Object Definition
Language?) for metadata. I don't know exactly how this will be handled
in GDAL, but I am thinking of going beyond the GeoGateway concept of a 
flag metadata dictionary to a (potentially) heirarchical set of information
and also to carry some typing information with the metadata. 

Best regards,

-----------------------------------+---------------------------------------
Who can give them back their lives | Frank Warmerdam, Software Developer
and all those wasted years? - Rush | URL http://members.home.com/warmerda
| warmerda at h...
------------------------------------------------------------------------
Free Web-based e-mail groups -- http://www.eGroups.com