[GRASSLIST:8372] Re: r.terraflow, or system, out of memory

[Hamish]

> > > i'm running r.terraflow (grass6.0.1) on a large:
> > >
> > > nsres:      2
> > > ewres:      2
> > > rows:       10050
> > > cols:       7760
> > >
> > > raster and the process seems to stop out of  the blue, i
> > >know that this module works on the same machine on an
> > >subset of the data i'm working on.
> > > Anyway i don't understand if this is an r.terraflow
> > >problem (I've searched in the bug/whishlist and didn't
> > >find anything related) or a kernel problem, indeed i
> > >found in  /var/log/messages
> > >
> > > kernel: __alloc_pages: 0-order allocation failed
> > >(gfp=0xf0/0)
> > >
> > > Do someone experienced somethig like that and knows how
> > >to fix?
..
> > Do you have enough space on your tmp file?
> > terraflow use temp file that can result quite big...
> > it happened to me that was the cause of crashing the
> > command and leaving trash on the tmp dir (i can't remember
> > now but there's a way to define which dir use as temp)....
> > Don't know if it is related...
..
> It could also be that you have exceeded 2 Gig on one of the temp
> files. r.terraflow needs to be compiled with special flags to exceed
> this amount...I don't think this is done be default yet,
> unfortunately.
>
> Look at the GRASSLIST archives for this message and the follow-ups:
> [GRASSLIST:7445]

here's that thread:
  http://thread.gmane.org/gmane.comp.gis.grass.user/8868


>From the r.terraflow help page:

"One of the techniques used by r.terraflow is the space-time trade-off.
In particular, in order to avoid searches, which are I/O-expensive,
r.terraflow computes and works with an augmented elevation raster in
which each cell stores relevant information about its 8 neighbors, in
total up to 80B per cell. As a result r.terraflow works with
intermediate temporary files that may be up to 80N bytes, where N is the
number of cells (rows x columns) in the elevation raster (more
precisely, 80K bytes, where K is the number of valid (not nodata) cells
in the input elevation raster). All this intermediate temporary files
are stored in the path specified by STREAM_DIR. Note: STREAM_DIR must
contain enough free disk space in order to store up to 2 x 80N bytes."


So 10050 x 7760 cells * 80 bytes * 2 = ~12gb disk space needed, and as
David mentions this is bigger than 2GB so you will need to have GRASS
built with large file support*. If you did build GRASS with large file
support, then this is a bug as the r.terraflow Makefiles aren't checking
for that flag and making the needed adjustments. I'm pretty sure that
this has not been fixed but don't know enough about LFS and the
inner workings of the configure system to fix it myself.

[*] ./configure --with-cxx --enable-largefile



Hamish