[gdal-dev] reading large image files

[Even Rouault]

Stephen,

My first answer was "Maybe you have not waited long enough... ;-)"

But I could also reproduce the slowness with GDAL <= 1.5.2 while reading such 
a huge amount of data into one call.
There was an inefficient reading strategy in GDALRasterBand::IRasterIO that 
your dataset precisely hits, that is to say raster with big raster dimensions 
and tiled data. This is bug #2457 and it has been fixed recently in the SVN 
branch of 1.5 (in r14863) and trunk. I can confirm that updating to r14863 of 
branches/1.5 fixes the slowlyness.

So several solutions come to mind :
* try to read your file piece by piece (ideally by adjusting your requests on 
block boundaries)
* update to the SVN version 
* wait for the next release 
* try with GDAL_CACHEMAX=54, because 54 MB = 56 623 104 bytes > (1024 * 1024 * 
2) * (27552 / 1024). See #2457 for the explanation of that formula ;-) 

(Side note, I see one error in your code. You ask for GDT_Byte data but you 
allocate a float array. )

Best regards

Le Wednesday 16 July 2008 17:53:21 stephen tan, vous avez écrit :
> I have a nitf image that I am trying to read into an array using
> gdal-1.5.1.  The file is about 1.5G .
>
> gdalinfo returns the following information:
>     Size is 27552, 25196
>     ...
>     Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
>
> I have tried to read the image into an array using the following code:
>     GDALDataset  *poDataset;
>
>     GDALAllRegister();
>
>     poDataset = (GDALDataset *) GDALOpenShared( fname.c_str(),
> GA_ReadOnly );
>     if( poDataset == NULL )
>     {
>         cerr << "Could not open file: " << fname << endl;
>         exit(1);
>     }
>
>     int width = poDataset->GetRasterXSize();
>     int height = poDataset->GetRasterYSize();
>     int bands = poDataset->GetRasterCount();
>
>     GDALRasterBand  *poBand;
>     int nBlockXSize, nBlockYSize;
>
>     poBand = poDataset->GetRasterBand( 1 );
>
>     poBand->GetBlockSize( &nBlockXSize, &nBlockYSize );
>
>     //float* data;
>     //data = (float *) CPLMalloc(sizeof(float)*width*height*bands);
>
>     float* data;
>     data = new float[width*height*bands];
>
>     //GByte* data;
>     //data = (GByte *) CPLMalloc(sizeof(float)*width*height*bands);
>
>     //GByte data[width*height*bands];
>
>     if(poBand->RasterIO( GF_Read, 0, 0, width, height, data, width,
> height, GDT_Byte, 0, 0 ))
>     {
>         cerr << "Error: Could not read image data." << endl;
>         exit(1);
>     }
>     GDALClose(poDataset);
>
> No errors are returned.  I let it run for several mins before I hit CTRL+C.
> Any C++ suggestions on getting the image into a 1D array and/or 2D array
> would be most helpful.  The above code works for smaller images.  I have
> also tried different GDALSetCacheMax with no success.
>
> Thanks
>
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev