[gdal-dev] Memory fragmentation and memory consumption when warping with GCPs

[Frank Warmerdam]

Joaquim Luis wrote:
> So one question is, why warping with GCPs takes such large amount of 
> memory (apparently a chunk
> of 500 Mb was not enough to process 5000 GCPs)?

Joaquim,

Is the memory fragmentation really related to the use of many GCPs?  Does
the same problem occur with 4 GCPs?

> I also tried with the command line GDAL gdaltransform and here I had no 
> problems. So I'm a bit puzzled.
> Do all programs depend on the memory fragmentation issue, or is it just 
> a Matlab limitation?
> (Ah, Windows XP here)
> 
> As a side note, I also have MEXs that link with the OpenCV library, but 
> those do not sensibly
> fragment the memory.

It is possible that OpenCV uses a private heap thereby minimising the
amount of fragmentation in the "master heap".  It might also be that OpenEV
just doesn't use that much memory.

I suspect you are seeing some heap fragmentation and consumption due to
GDAL's block caching mechanism but it is really hard to know.  You can
influence the amount of memory available for the block cache using
GDALSetCacheMax(). I'm not in the habit of monitoring heap fragmentation
on Win32.  I will say that GDAL normally uses malloc() and free() for
memory allocation so that may have some implications for heap effects.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent