[gdal-dev] Memory fragmentation and memory consumption when warping with GCPs

[Joaquim Luis]

Hi,

I managed to solve my problem with the GCPs in MEM driver (and sorry if I bothered some with
that thread), which turned up to be due to a bad initialization of the dataset geotransform.

After that I wrote a MEX version of gdaltransform to be able to reproject points using GCPs.
This MEX works but it revealed serious problems. At least under Matlab.
Matlab requires not only LOTS of memory but also that the available memory is continuous.
For example, if one needs a variable of 100 Mb the system needs to have a chunk of free memory
not less than 100 Mb. Two chunks of 99 and 1 Mb are not enough as we'll have an out memory error.

Now, I just realize that when I call a MEX that links against gdal.dll the memory immediately becomes
highly fragmented. As an example of my current case, I had the largest chunk of free memory
of 1070 Mb and after loading gdal.dll the largest chunk become 540 Mb
Eventually due to this memory fragmentation, I could not reproject a file having a large number of
GCPs (> 5000 GCPs). By decreasing the number of GCPs to half I was able to perform the operation.

So one question is, why warping with GCPs takes such large amount of memory (apparently a chunk
of 500 Mb was not enough to process 5000 GCPs)?

I also tried with the command line GDAL gdaltransform and here I had no problems. So I'm a bit
puzzled.
Do all programs depend on the memory fragmentation issue, or is it just a Matlab limitation?
(Ah, Windows XP here)

As a side note, I also have MEXs that link with the OpenCV library, but those do not sensibly
fragment the memory.

Thanks

Joaquim Luis