[gdal-dev] New RFC - Fast Open - for discussion

[Frank Warmerdam]

Even Rouault wrote:
> Hi
> 
> Although I agree that the goal of fast opening is a good idea, as discussed on 
> IRC, I'm not sure we need this RFC for the current work plan it contains :
> * listing the files in the directory can already be skipped by defining 
> GDAL_DISABLE_READDIR_ON_OPEN=YES. But perhaps this could benefit from the 
> mechanism of thread local configuration option
> * for the GeoTIFF driver, I think the code could be reworked so that 
> collecting the coordinate system is done only when GetProjectionRef() is 
> called. The .aux.xml file could be opened only when GetProjectionRef(), 
> GetGeoTransform() or GetMetadata() are called. 

Even,

Based on issues raised by you and Howard, I have withdrawn this RFC - at
least for the time being.  In the meantime I have created a ticket on the
slow behavior of the GeoTIFF driver and applied a few improvements including
the suggested "delayed coordinate system processing" change.  This does help
dramatically for GeoTIFF.  With these changes it does not seem worth
pursuing RFC 25 at this time, though I might revisit it in the future.

   http://trac.osgeo.org/gdal/ticket/2957

> * since GDAL 1.6.0, the GDAL VRT driver can avoid opening the underlying 
> datasets if the SourceProperties elements is defined for each SimpleSource. 
> This is done by the gdalbuildvrt driver. So, opening a VRT dataset should be 
> really quick. The underlying datasets are opened only when real data must be 
> read from them.

Understood, but there are still cases (such as requesting an overview of a
VRT mosaic) where many small files might be accessed.

> Currently, one area where we cannot differ the processing is the instanciation 
> of raster bands. The GetRasterBand() method in GDALDataset is not virtual, so 
> they must be instanciated at the dataset opening. But this is probably only 
> an issue for datasets with millions of bands.

Correct.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent