[gdal-dev] Memory usage for GetProjectionRef

[Livneh Yehiyam]

Hi
I'm sorry if I was misunderstood, but these are not leaks I'm worried about, but rather I'm trying to understand the memory consumption of the various GDAL calls I use.
I didn't notice any leak, and I trust you test for leaks all the time.
Take for example the first two examples below. These two are the same image, with and without the pyramid (overviews). Why should the call to GetProjectionRef raise the memory of the application by a different amount?

Thanks 
Yehiyam
________________________________________
From: Frank Warmerdam [warmerdam at pobox.com]
Sent: Tuesday, October 05, 2010 11:32 PM
To: Livneh Yehiyam
Cc: gdal-dev
Subject: Re: [gdal-dev] Memory usage for GetProjectionRef

Livneh Yehiyam wrote:
> Hi
> After further investigation I think the situation is more complex, as the results I'm getting depends on the file opened.
> I ran the test with several tiff files and I'm getting completly different results, as described below.
> Also, to rest my mind that this is not a problem with the C# binding, I'v created an identical test using the c interface.
>
> case 1: 580MB tiff, tiled RGB with pyramid:
> GDALOpen increased the memory by 17KB.
> GetProjectionRef increased the memory by an additional 35KB.
> After calling GDALClose, the memory was still 2KB more than before the call to GDALOpen.
>
> case 1: 434MB tiff, tiled RGB without pyramid:
> GDALOpen increased the memory by 40KB.
> GetProjectionRef increased the memory by an additional 0.6KB.
> After calling GDALClose, the memory was still 2KB more than before the call to GDALOpen.
>
> case 3: 420MB tiff, one row per strip RGB without pyramid:
> GDALOpen increased the memory by 186KB.
> GetProjectionRef increased the memory by an additional 0.5KB.
> After calling GDALClose, the memory was still 4KB more than before the call to GDALOpen.
>
> case 4: 2GB tiff, one row per strip palette packbits compression without pyramid:
> GDALOpen increased the memory by 867KB.
> GetProjectionRef increased the memory by an additional 0.1KB.
> After calling GDALClose, the memory was still 9KB more than before the call to GDALOpen.
>
> Beside being very inconsistent (or at least I fail to see the pattern), a few things worry me, and might send me searching for another solution:
> 1. opening large stripped files increases the memory by almost 1MB. Multiplied by hundreds of files this amount to hundreds of MBs.
> 2. for a big tiled pyramid file, even the 35KB increase for just reading the projection might be too much.

Livneh,

I can run "gdalinfo utm.tif" under valgrind, a memory corruption and
leak checker, and confirm that no memory is leaked.  I'm not sure what
you are using to check memory use but I have no reason to trust it.

Note that there are memory buffers allocated and kept around for a
variety of purposes.  Some of these are not freed until cleanup functions
like GDALDestroyDriverManager() are called.  These may include cached
copies of the coordinate system .csv files.

You are welcome to go looking for another solution, but I have little
reason to believe GDAL's GeoTIFF driver leaks memory.  If it does
I would be pleased to fix it if you can demonstrate it in a credible way.

I hope you will forgive my disbelief, but I've received so many half
baked claims of memory leaks over the years from folks who don't
understand how heaps work in C programs, how to use leak checking
tools and so forth that my first reaction is usually to disregard
them now.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent
**********************************************************************************************
This message (including any attachments) issued by RAFAEL- ADVANCED DEFENSE SYSTEMS LTD. 
(hereinafter "RAFAEL") contains confidential information intended for a specific individual and purpose, may 
constitute information that is privileged or confidential or otherwise protected from disclosure. If you are not 
the intended recipient, you should contact us immediately and thereafter delete this message from your 
system. You are hereby notified that any disclosure, copying, dissemination, distribution or forwarding of this 
message, or the taking of any action based on it, is strictly prohibited. If you have received this e-mail in error, 
please notify us immediately by e-mail mailto:lawraf at rafael.co.il and completely delete or destroy any and all 
electronic or other copies of the original message and any attachments thereof.
**********************************************************************************************