[gdal-dev] GDAL WKT Raster cache: Problem, options to solve and doubts

[Frank Warmerdam]

2009/8/14 Jorge Arévalo <jorge.arevalo at gmail.com>:
> Problem: In basic GDAL WKT Raster driver, each row of a raster table
> (one block, in regularly blocked rasters) means one server round. This
> is slow, and "sub-optimal".

Jorge,

I would note that one round trip per block is not necessarily so terrible
if the block size is reasonably large (128x128 or larger for instance).

> How to solve it?: IReadBlock executes an spatial query to get all the
> rows of a table that fits in a natural block, defined by nXOff, nYOff
> and the values fetched by GetBlockSize. In many raster formats, one
> natural block is a scanline, of size (nXRasterSize, 1). In WKT Raster
> format, if we have a regularly blocked raster,  "natural" block size
> will be equal to RASTER_COLUMNS-defined block size, and the query will
> return one block. To avoid one spatial query for each block petition,
> we should "force" the driver to get all the blocks covering the area
> requested in a IRasterIO call.
>
> How to implement it? My approach is based on implementing
> WKTRasterDataset::IRasterIO method (overriding GDALDataset::IRasterIO
> method). This method executes a spatial query that returns all the
> raster rows covering the area requested. Now, I have all the data of
> an image region. If the requested region dimensions match the buffer
> dimensions, I can copy all the pixels fetched in this way:
>
> pImage = {b1b1b1b1b1b1b1b1b1b1...|b2b2b2b2b2b2b2b2b2...|...|bnbnbnbnbnbnbnbnbn}
>
> Where bi are the bytes of the band i. A WKT Raster image has a
> non-interleaved format, all the band are consecutive. Is it correct? I
> mean, copy the data in pImage with this format.

Some of the arguments to IRasterIO() are the values indicating
how the imagery should be interleaved into the target buffer.
You might want to check them and only implemented the direct
copy if the interleaving matches what is convenient for you.

> And if the region dimensions don't match the buffer dimensiones,
> should I raise an error and finish or delegate in base
> GDALDataset::IRasterIO implementation?

Definately do not raise an error.  If for any reason it is not
convenient to process a request in an optimized fashion then
just call the underlying IRasterIO() method (on whatever your
base class is, possible GDALDataset).

> Another question: After copying the data from fetched rows in pImage
> buffer, should I do anything more?

You need to ensure it is in local machine byte order if it is not
GDT_Byte.

> Other part of my implementation is overriding
> GDALRasterBand::IRasterIO method. My method will simply call the
> WKTRasterDataset::IRasterIO method with only one band to read. Is it
> correct?

This is acceptable I think.  My only concern is that there may be
situations in which WKTRasterDataset::IRasterIO() will not
process the request in an optimized fashion and will fallback to
GDALDataset::IRasterIO() which in turn might call IRasterIO
on the band again.  This may become obvious in use. :-)

I would note that the optimized IRasterIO() implementation is
not really necessary for a successful project though it would
certainly be a crowning achievement for the summer.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent