[GRASSLIST:7455] Re: r.terraflow failured

[Andrew Danner]

A couple of things:

 1) creating and accessing files larger than 2GB requires large file
support under Linux. Typically you can do this by including the compile
flags -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE when compiling
r.terraflow. If you do not see these flags when compiling, it will not
be able to read/write files bigger than 2GB (a 32-bit file offset)

 2) The application must use off_t data types for file offsets and not
int/long/whatever... The compiler flags above make off_t a 64-bit type
and allows files upto something like 8 billion GB. There may be some
offsets in r.terraflow that are still int/long and need to convert to
off_t

 3) I have some familiarity with the r.terraflow code and I'll be
looking at it for some clean-up starting next week. I can keep you
posted.

-Andy
 
On Tue, 2005-07-05 at 01:36 -0700, David Finlayson wrote:
> I've been having trouble with this too. It seems that on my system,
> r.terraflow cannot make files larger than 2Gb. I CAN create large
> files on my system. For example:
> 
> dd if=/dev/zero of=test.tmp bs=1024k count=4000
> 
> creates a 4 GB file without problems. So it must be either something
> intrinsic to the code in r.terraflow or a library it is sharing.
> That's were my skills run out. I'm not a real programmer.
> 
> Interestingly enough, I met Dr. Toma when she was writing the original
> version of this code for her PhD (I supplied her a big DEM to test the
> algorithm). If we can't make progress on our own, I can send her an
> email and see if she has any thoughts on the source of the problem.
> 
> David
> 
> 
> 
> 
> 
> 
> 
> On 7/4/05, Hamish <hamish_nospam at yahoo.com> wrote:
> > orkun wrote:
> > > >>At first time I tried to run r.terraflow in a small test area. I
> > > >>failured. It gave this error:
> > > >>
> > > >>r.terraflow elev=el_5  filled=el5-filled  dir=el5-mfdir
> > > >>swatershed=el5-watershed accumulation=el5-accu tci=el5-tci
> > > >>
> > > >>assigning directions on plateaus
> > > >>File size limit exceeded
> > > >>
> > > >>
> > > >>Could you tell me why ?
> > 
> > David Finlayson wrote:
> > > >r.terraflow makes large temp files during the operation. Check the
> > > >STREAM_DIR (defaults to /var/tmp) you assigned I would bet that one
> > > >of those temp files exceeded 2 Gb. That seems to be the filesize
> > > >limit on my distribution of linux.
> > > >
> > > >Davids
> > 
> > orkun wrote:
> > > thank you
> > > You are right.
> > > that file made my disk full.
> > > Although I used a small region for testing purposes. It produced
> > > giantic files.
> > > I do not know what will happen if I use whole region ?
> > >
> > > what do you suggest ?
> > 
> > 
> > Try it and see what happens! (It has always worked for me)
> > Worst case you run out of disk space again. Maybe the resolution was
> > set way too high?
> > 
> > How big was the region that failed? (output of 'g.region -p'?)
> > 
> > 
> > 
> > Hamish
> > 
> 
>