[Gdal-dev] Two Level Raster Caches

[Frank Warmerdam]

Folks,

Traditionally GDALRasterBands maintain a block cache.  Internally 
this is an array of pointers to GDALRasterBlock objects.  If a block hasn't
been read, the pointer is NULL. 

However, for very large images with relatively small tile sizes it is
possible to have *alot* of block pointers to store.  For instance, a 
1 million by 1 million image with 64x64 tiles would be 244 million
blocks.  The block pointer array alone for this image would be 1GB.

While this is a pretty big image, I have had real-world performance
issues due to the size of the cache pointer array.   

With Peng's encouragement, I have changed to a two-level cache
in some cases.  Basically the first level cache becomes an array 
of pointers to secondary grids which are 64x64 blocks. 

I have done *some* testing of this change, and committed it today.  It
shouldn't cause any problems, but if folks have an issue please let me
know. 

I would add the two level caching only kicks in for images at least
32 blocks wide. 

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent