[gdal-dev] GDALRasterBand::RasterIO and FlushCache

[Frank Warmerdam]

David,

I see I misunderstood about the free and it is already documented.  So I
think the docs are good now and that most users should never even need to
think about the caching aspect.

Best regards,
Frank

On Fri, May 31, 2013 at 4:38 PM, Frank Warmerdam <warmerdam at pobox.com>wrote:

> David,
>
> Note that the image cache will start discarding blocks on it's own when it
> is full.  I think the default cache is about 64MB.    I don't think that
> flushing the cache should be part of normal applications operations, though
> if you are very tight on memory you might want to alter the default size.
>  Note that the cached blocks are discarded when the dataset is closed.
>
> I will update the example to free the buffer at the end.
>
> Best regards,
> Frank
>
> On Fri, May 31, 2013 at 4:24 PM, David Strip <gdal at stripfamily.net> wrote:
>
>>  I'm reading a fairly large (at least to me) geotif (about 2GB) a line at
>> a time. I noticed that memory usage increases as I read the image. It
>> appears that what's going on is that the driver hangs onto these lines of
>> the image, even though I'm providing the same buffer for it to read into
>> each time. At this point I noticed the FlushCache method. Calling this
>> after each reading each scan line solves the problem of the growing memory
>> usage.
>>
>> I assume this behavior of caching whatever is read is deliberate. I think
>> it would be useful in the API examples to make this behavior clear and
>> perhaps show the use of the FlushCache method.
>> For example, the API Tutorial page shows this for how to read a line into
>> a buffer:
>>
>> *Reading Raster Data
>> *There are a few ways to read raster data, but the most common is via
>> the GDALRasterBand::RasterIO() method. This method will automatically take
>> care of data type conversion, up/down sampling and windowing. The following
>> code will read the first scanline of data into a similarly sized buffer,
>> converting it to floating point as part of the operation.
>>
>> In C++:
>>
>>         float *pafScanline;
>>         int   nXSize = poBand->GetXSize();
>>
>>         pafScanline = (float *) CPLMalloc(sizeof(float)*nXSize);
>>         poBand->RasterIO( GF_Read, 0, 0, nXSize, 1,
>>                           pafScanline, nXSize, 1, GDT_Float32,
>>                           0, 0 );
>>
>> The pafScanline buffer should be freed with CPLFree() when it is no
>> longer used.
>>
>> I think it would be useful to point out, in addition to the need to
>> CPLFree the buffer that if you are using this example in a loop to read the
>> image a line at a time, you should FlushCache every line (or periodically)
>> to keep from accumulating the entire image in the cache (which is invisible
>> to the user). It seems (to me) that it would be better to put this in the
>> API Tutorial rather than as a code snippet in the Wiki, since the tutorial
>> is where most people would look when starting out.
>>
>>
>>
>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>
>
>
> --
>
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Software Developer
>



-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20130531/f0e79d78/attachment-0001.html>