[Gdal-dev] memory leak in MapInfo layer

[Richard Matsunaga]

Sorry if this is a duplicate. I never saw this message come up.


After days of pain and suffering and ultimately, barking up all the wrong
trees, I found the source of the memory leak.

I created a stripped-down-to-basics C# version of a program that writes
point symbols to a MapInfo file. I also created an identical C program.

I used DevPartner 8.1 to analyze both programs. It detects numerous leaks in
gdal, but based on Mateusz's comments and other odd results reported by
DevPartner, I am not putting too much faith in those results.

After all this, it turns out that OGR_F_SetStyleString() is the source of
the leak (or more accurately, when the feature is created on the layer
itself - hence the lack leak if the geometry is never added to the feature).
This is evident in both C and the C# versions of the programs, but for some
reason is greatly exaggerated in C#. For example, when I generate the same
layer repeatedly, it consumes a 100+ kB for each iteration in C. But in C#,
it consumes 2+ MB of memory for each iteration. I can't explain that, but
maybe has something to do with strdup and how the new buffer size is
computed when passed a reference to a C# string type? Doesn't sound right.

The root cause appears to be *poStyleMGr not being deleted (there are other
methods in the same file that do the same thing):

void ITABFeatureSymbol::SetSymbolFromStyleString(const char *pszStyleString)

{
 ...
	OGRStyleMgr *poStyleMgr = new OGRStyleMgr(NULL); 
 ...
}

Cheers,
Richard

-----Original Message-----
From: gdal-dev-bounces at lists.maptools.org
[mailto:gdal-dev-bounces at lists.maptools.org] On Behalf Of Mateusz Loskot
Sent: June 14, 2006 8:33 AM
To: gdal-dev at lists.maptools.org
Subject: Re: [Gdal-dev] memory leak in MapInfo layer

Richard Matsunaga wrote:
> Hi Mateusz,
> 
> Here is a sample of the code I am using. I have removed non-OGR 
> related details. For those who may not know, the 'using' statement is 
> used to force Dispose to be called once out of scope of the using 
> statement.


Yes, but OgrSpatialReference class is not disposable, so how did you manage
to put it in using?

VC# 2005 throws me this message:
error CS1674: 'GDALWrapper.OGR.OgrSpatialReference': type used in a using
statement must be implicitly convertible to 'System.IDisposable'

Or I have old version (from the beginning of May 2006) of your GDALWrapper?


> The C# wrapper is basically the same as the one I sent out some time  
> ago, although there have been changes - extended functionality, bug  
> fixes and some unrelated memory issues caused by the wrapper itself.
>  I can send this out again if that would help. I am fairly confident  
> at this point that the wrapper is not the cause of the leak, but since 
> I am far from being a platform invoke expert, there is a, of course, 
> chance that some unexpected behaviour with the wrapper is the  cause.


The best way to check is to implement the same functionality in C++
correctly and see if this also will leak.
It should be not very hard to get ESRI Shapefile/MapInfo File driver, create
datasource, create one layer.


> I am not using any high tech means to detect memory leaks. I run the  
> code below in a loop and watch the system memory climb and climb.
> The .NET garbage collection can be a finicky thing to an outside 
> observer such as myself, but it will not allow the system to run out 
> of memory unless there is a leak of some kind, and that can be a 
> problem with managed resources or unmanaged code. I have run a few 
> .NET memory profilers and they have not found any problems. There is 
> another commercial product that I have not tried (yet) that is able to 
> instrument both managed and unmanaged code for memory profiling.


I checked the creation of data source  with .NET Memory Profiler and also
watching the memory usage in the tasks list.
I don't see huge increase of memory consuming.
I mean, 1000 layers (1000 x data source, driver, layer objects) gave me
about 2 MB memory usage.
Also the .NET Profiler reports the same number of objects created and
disposed of all classes I used.

I got problems with CreateLayer for Shapefile.
Calling OGR OGR_DS_CreateLayer gave me memory corruption.
I tried various parameters, null checking, etc.
I have no idea what's going wrong there.

But I've shown you, Valgrind on Linux does not show any memory issues with
ogr2ogr - a C++ program using OGR.
Unfortunately, I don't have any other bounds checker on Windows than
debuggers coming with VC++ and VC# 2005.

May be you could do some tests as simple as possible:
- create and add single field
- create and add single point (geometry) to feature etc.

This way you should be able to find where is the problem in C# wrapper, if
any.

Cheers
--
Mateusz Loskot
http://mateusz.loskot.net

_______________________________________________
Gdal-dev mailing list
Gdal-dev at lists.maptools.org
http://lists.maptools.org/mailman/listinfo/gdal-dev