[GRASS-dev] r.in.gdal: how to speed-up import with huge amount of bands?

[Markus Neteler]

Hi,

I have received temperature map time series of 50 years of daily data
in single Geotiff files (Tmin, Tmean, Tmax). Each GeoTIFF has around
21550 bands, 4GB file size.

Problem: the import takes "forever" despite using a superfast disk, i.e.
120 seconds per band (size is only 464 x 201), so 29 DAYS for each file.

The problem will be that it has to run through the entire 4GB over and over
again for each channel. I guess I want to do heavy caching - but how?

Looking at ImportBand() in main.c of r.in.gdal, I see that GDALRasterIO()
is used:
 http://www.gdal.org/gdal_tutorial.html#gdal_tutorial_read
but I don't see hints to tell the IO function to keep more in cache.

The original files are in netCDF format, I used gdalwarp to generate GeoTIFF.
First I thought to write it out as 21550 files but didn't find an
option to do so.

Clueless,
Markus