[gdal-dev] does gdal can read large file faster?

[Frank Warmerdam]

chen xiaodong wrote:
> On our company's platform ,we use gdal to read/write geo raster 
> files.There is a problem I acount with is that when process a little 
> larger file beyond 1G ,the performance is poor that a  file with a 1.5G 
> compacity and tiff format resampling can consume more then 20 minutes!It 
> troubles me for a long time,so appreciate for any idea!

Hi,

One possible reason for this might be "cache thrashing" in the GDAL block
cache.  This occurs in a number of circumstances.  One example would be
when one whole row of tiles from an input file will not fit in the block
cache at the same time, and when data is fetched one scanline at a time.

Then, for each scanline the full set of intersecting tiles will get discarded
and re-read.

The most direct solution is to increase the block cache size with the
GDALSetCacheMax() function:

   http://www.gdal.org/gdal_8h.html#dfb1e95703ee577f012935869852d96c

It can also be helpful to enable debug output, and if you see a message about
cache thrashing when a dataset is closed that indicates that there are signs
of cache thrashing.

Best regards,
-
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent