[gdal-dev] Raster algebra with Gdal and Python

[Vincent Schut]

On 02/02/2010 08:01 AM, Alexander Bruy wrote:
> Hi again,
>
> with your help I've fix problem with "integer overflow" and now I get
> correct results.
>
> Also I clean up my code for reduce memory usage but problem with large files is
> remain. I have raster with size 11779*10663 and 5 bands. When I calculate NDVI
> on Linux box (2 Gb RAM, 2Gb swap, GDAL 1.6.3) all works fine and I get correct
> results.Peak memory consumption during this operation is 1900 Mb RAM and
> 300 Mb swap (monitored with htop).
>
> When I run same code with same raster on Windows box (2 Gb RAM, swap size is
> selected by OS ~1.3 Gb, GDAL 1.6.3) I get MemoryError. Peak memory consumption
> during this operation is 600 Mb RAM and 700 Mb swap. I've try smaller
> raster with size
> 8500*7600 pixels but get MemoryError too.
>
>    
Alexander,

if you want your raster algebra to be independent of raster size, you'll
have to resort to do the processing in a loop, each time processing a
part of the input(s), writing the result, and so on.
Easiest would be (to program) to loop over the image lines. Read a line
for each input, calculate the resulting line, write that line to the
output, and repeat.
Performance-wise, it could be better to calculate in blocks of lines,
for example 50 lines per iteration.
This of course gets more complicated when your calculations include
neighbourhood expressions.

Vincent.