[Gdal-dev] Moving window for image filters & memory allocation

[Jochen Krueger]

I'm programming a filter that uses a moving window to calculate a new 
image. As my images are very big (> 10000x10000 pixels) processing speed 
  is essential and I don't want to load each window seperately from the 
file.

To limit the file access while keeping memory needs acceptable I planned 
to load a number of lines equal to the height of my window into a buffer 
and then read the window data from that buffer while processing the 
scanline. In a second step I might want to load additional lines to 
reduce the file access even further, but for the start that is sufficient.

So, following the example on the website, I use the following lines to 
load the data:

     int bufferSize = xsize * winsize;
     lowresBand = lowresDataset->GetRasterBand( i );
     lowresBuffer  = (float *) CPLMalloc(sizeof(float)*bufferSize);
     lowresBand->RasterIO( GF_Read, 0, firstline, xsize, winsize,
                           lowresBuffer, xsize, winsize, GDT_Float32,
                           0, 0 );

'xsize' is the width of the dataset, 'winsize' the size of the window 
and 'firstline' the number of the first line to load. After doing this I 
have the data loaded into the buffer 'lowresBuffer', right? This is 
where I want to move my window through. Unfortunately that's where my 
two big problems start:

1st: I seem to be unable to actually access a specific pixel within the 
buffer. I first assumed it was just the lines written in a straight 
sequence into the buffer (so after xsize pixels the next line would 
start) and thus tried to access the values using lowresBuffer[i]. But 
that only seemed to access the first line, so I gathered the buffer must 
be actually a two dimensional array. I tried using lowresBuffer[i][j], 
but to no avail either.

Now I'm not very proficient in C++ yet, and I'm sure the solution is 
very simple, but I simply can't find it...

2nd: As I rewrite the window buffer over and over and also the initial 
buffer is rewritten many times I first had a problem with excessive 
memory usage (deleteing the pointer without freeing the reserved 
memory). Then I learned that I was allocating new memory with each loop 
instead of simply overwriting the old buffer. So I tried to use delete 
in its various incarnations and also CPLRealloc, which seemed to be the 
exact function I was looking for. But somehow this doesn't really help 
me. Either the memory need is not contained - or I suffer from extreme 
memory corruption.

As my tool will be used in a processing chain memory leaks are not 
acceptable band I have to do everything to avoid excessive memory usage.

I know these are absolute beginners problems, but if someone could point 
me to where I can find a good tutorial/help on this, I'd appreciate this 
very much.