[gdal-dev] Re: [gfc-dev] GFC

Liujian Qian qian at g...
Sat Dec 5 15:30:41 EST 1998


>>>>> "Frank" == Frank Warmerdam <warmerda at h...> writes:

[details on GGridImporter cut]

Frank> The above approach is good, and fairly analygous to the approach I am 
Frank> planning within GDAL. I will briefly explain that:

Nice scheme!

Frank> GDALDriver:
Frank> This is the base class for each of the format drivers. The 
Frank> driver itself is mostly just responsible for reporting some 
Frank> general information about the format (or gateway). For instance
Frank> the kind of information you would need to populate dialogs allowing
Frank> dataset creation before an instance of the format exists. 

Frank> Some GDALDrivers will actually be drivers for another underlying 
Frank> multi-format API (such as an OGDIDriver) and so I want to ensure 
Frank> that would driver can represent a variety of underlying formats
Frank> effectively. I am not sure what all this will entail. 

Have you thought about providing GDALDriver-based interface to things like HDF 
library, which is very complex, but already has a well-developed programming 
interface (it's own driver)? (All this gets me thinking that OGC should push on
a raster specification RSN, so that you can make GDALDriver mimic that, and 
authors of proprietary formats can easily come up with their subclasses of 
GDALDriver and register it with your GDALDriverManager.)

Frank> The GDALDriver derived driver class is also responsible for 
Frank> dataset opening, and creation. Once the dataset is instantiated
Frank> all further data access is the responsibility of the dataset
Frank> (GDALDataset). 

I assume that there will be many interactions between a GDALDriver and the 
GDALDatasets it creats? Say if a dataset wants to check out a particular 
data item it will use the corresponding GDALDriver to get the actual data off a 
formated file or something like? Do you have a protocol for the message
passing between these two types of objects?

Frank> In theory one GDALDriver might be responsible for handling a
Frank> more than one GDALDataset derived class, though it isn't clear
Frank> why this would be needed.

Do you mean one GDALDriver actually *owns* those datasets? I thought it probably
is a good idea to maintain only a loose connection between GDALDatasets and 
GDALDrivers, without each knowing too much about the other. But I could be wrong.


Frank> GDALDriverManager: 
Frank> This singleton class maintains a list of GDALDriver's. 
Frank> In addition to the ability to register a driver with it
Frank> there are (or will be) methods to manipulate the order in
Frank> which they are tried on unknown datasets. 

Frank> The GDALOpen() function essentially passes the dataset to each 
Frank> GDALDriver derived driver in turn till one recognises it. 

Frank> GDALDataset:
Frank> Particular format classes are derived from this base class, which
Frank> is a close analog to your GGridImporter. It contains a variety of
Frank> virtual methods, some with default implementations and others that
Frank> are pure. The derived class of course fills these in. 

Frank> It is my intention to use the GDALDataset baseclass for both raster
Frank> and vector formats. In part this comes from my grounding in PCIDSK
Frank> format which held both raster and vector information. This really
Frank> doesn't seem to happen with many other formats, so I don't know if
Frank> keeping this commonality is important. 

I think to derive both raster and vector dataset classes from GDALDataset is a good
idea. It doesn't necessarily mean that both type of data will physically sit next to
each other in a single dataset. That is indeed rare. But to develop applications 
based on a single uniform notion of GDALDataset is a good idea. 

Frank> Comments on GGridImporter:

Frank> Multiple Bands:
Frank> Do you intend that applications would instantiate a GGridImporter for
Frank> each band (channel) of multi-band raster files? Would there be a class
Frank> to to relate the different bands? In fact a GDALDataset represents the
Frank> entire dataset, and a GDALRasterBand is instantiated for each band. At
Frank> the least you might consider extending the GGridImporter to allow 
Frank> reporting of the number of bands, and access to a requested band. 

My intention is to make GGrid minimum in terms of functionalities and data model.
As such I want GGrid to represent only a single shear/band of grided data, upon
which I can define very generic methods such as boolean operations. Later on there
will be more specific ADTs such as GRMImage (for remote sensing images), which will
maintain a list of GGrid objects each representing a band when necessary. But you
are right I should probably add the interface about band information reporting in
to the base GGridImporter class.

Frank> Blocking:
Frank> In GDALDataset I have provided three core methods to access raster data.
Frank> The simple ReadBlock(), and WriteBlock() methods are intended to simply
Frank> read, or write a block of raster data in the ``optimal'' blocking
Frank> organization of the underlying dataset. The idea is that when a band is
Frank> initially accessed it would return information on it's optimal blocking. 
Frank> For many formats the block size would be one scanline. For tiled 
Frank> formats the optimal block size would be the tile size. Within GDAL I
Frank> am assuming a uniform block size. A format like TIFF might report a
Frank> whole strip (usually a number of scanlines amounting to no more than 8K
Frank> of data) as the natural block size. 

Frank> In order to facilitate efficient access to modern tiled formats, I think
Frank> this could be important.

Frank> RasterIO:
Frank> The third raster access method on the GDALRasterBand is the RasterIO()
Frank> method. This is a complex call that allows for rectangular window
Frank> resampling, and alteration of datatypes on the fly. The caller expresses
Frank> the size of the dataset window, the size of their buffer, and the datatype
Frank> of their buffer. The method takes care of resampling, and type conversion.

This is excellent. Keep it rolling guys.

Frank> This method will have a default implementation on the base GDALRasterBand
Frank> class, but the intention is that formats with overviews, or that have other
Frank> specialized ways of satisfying such requests can override the method to 
Frank> to do it more efficiently than the default implementation. 

Do you think a cache for blocks/tiles of recently processed raster will be useful? I
was thinking about doing that for my GFC, where each grid will have a tag/key string
and the tile cache will look it up there first for a desired tile using the
key string. If it founds it there there is no need for physical I/O; otherwise the
normal disk read/write occur.

Frank> API Bridges:
Frank> Finally, I am thinking about how I would build a single GGridImporter 
Frank> interface for GDAL. In effect this wouldn't be very hard, but I think it
Frank> would be difficult to make much specialized information about the real
Frank> underlying format available. In essence, I would like to be able to report
Frank> back some information about the underlying database, such as a short and
Frank> long format name that is not fixed by the class (derived from GGridImporter).

Yeah I agree with you. That's why I have the customHeader() methods in the importer
class. The idea is that GGridImporter puts whatever specialized information it deems
not of general interest in a custom header (an opaque block of data) of the GGrid. 
It's the specialized GGrid subclasses, such as GDem or GRMImage, that can interpret 
the custom header properly. 

>> This seems to provide a clean interface for making use of raster data format
>> converters that you guys are developing. Comments and suggestions? Is such
>> a uniform interface possible? What about vector data formats?

Frank> I do think such a clean interface is possible, though certainly the complexity
Frank> depends in part on the objectives. It is an objective of GDAL that 
Frank> applications be able to operate on any GDAL supported file as if it were
Frank> a native file (within the restrictions of the target file format). 

Frank> The class you have described is a grid _importer_. Does this mean that 
Frank> within your system you are intending to convert data to your own internal
Frank> format for further access, or do you intend that the user be able to 
Frank> do operations directly with data in other formats? Either is a reasonable
Frank> goal, but if you want to be able to do operations directly on foreign 
Frank> format data you will need to substantially complicate the methodology. 

Good question. For now I have my own grid format (I know, I know, it's probably is
a bad idea), because my generic grid data operations depending on knowing the
exact layout of the grid cells. I will give it some hard thinking about where to
draw the line...

>> One last thing, in your shapelib 1.2, I found that you use a for loop to
>> swap the byte order of shorts/ints/doubles, this seems to be a bit slow
>> compared to macros that are based on bit-shifting, particularly on 64-bit
>> machines or for the case of shorts/ints (sometimes the macro version is
>> 4 or 6 times faster). If you are interested in testing my macro-based
>> byte order swaping routine, I can certainly send it to you.

Frank> I would be interested in getting these macros for incorporation into the 
Frank> CPL (Common Portability Library) (currently in gdal/port). When Shapelib
Frank> is brought into the GDAL tree I will update it to use them. I have at least
Frank> one client interested in having Shapelib be as efficient as reasonably 
Frank> possible. 

I will send it to you in another message...

Cheers,

L.J.

------------------------------------------------------------------------
Free Web-based e-mail groups -- http://www.eGroups.com





More information about the Gdal-dev mailing list