[gdal-dev] Memory usage for GetProjectionRef

[Frank Warmerdam]

Livneh Yehiyam wrote:
> Frank,
> I agree that the amount is small per file, but in our application we 
> need to keep thousands of files open, and keep memory consumption to a 
> certain limit.

Livneh,

Ah, I'm starting to grasp your issue.  I will humbly suggest that the idea
of keeping thousands of files open at the same time may be unwise.  If it
really is critical you might need a lighter weight approach than GDAL
or libtiff.

> I have another question regarding the stripped file. When I open a large 
> stripped file, the memory rises by almost 1 MB. I assume this is caused 
> by some sort of caching. When I open hundreds of such files, the amount 
> of memory is multiplied. Can I limit this amount?

Amoung other things the current active directory in a TIFF file will hold
it's strip offsets and strip size values in RAM.  For a large file
(say 50000x50000 with one scanline strips) this would amount to
50000 x 16 or about 800K if the offsets and sizes are 64bit.  This is
all within libtiff.

Also, GDALRasterBand builds arrays of block pointers are creation time
which will initially all be null.  In the above case this would be another
50000 pointers - another 400K on a 64bit system.  There is a special
"subblocking" mechanism in place to prevent this block pointer array
from initially being too large for tiled files but for scanline oriented
files this mechanism does not kick in.  The GDALRasterBand block pointer
memory use will apply to any GDAL raster format that uses the block
cache.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent