[GRASS-dev] r.watershed and swap memory

Tue Mar 18 13:22:39 EDT 2008

Dear Helena,
Thank you very much for your answer...
my problems are:

* is the flowdirection output produced by r.terraflow suitable for basin
creation (like I can do with the drainage map created by r.watershed and
given as input to r.water.outlet)? Because this is my final target...
It seems to me that r.terraflow outputs aren't useful for basin
delineation... or I'm wrong? 

* I'm really interested in testing TerraSTREAM but, probably due to my
fault, I wasn't able to obtain a login and password...

many many thanks

Ivan

Il giorno mar, 18/03/2008 alle 09.33 -0400, Helena Mitasova ha scritto:
> Ivan - you may talk to Yann on this list before you buy more memory -  
> he is trying to do the same as you , but with a bigger DEM and 8GB of  
> memory (Yann I hope it is OK with you that I am revealing this here).
> My experience with large DEMs (up to 10,000x10,000) has been that I  
> had to split the area into sections that were about
> 2000x2000 on 1GB memory computer) to get it done (I was able to do  
> that for Panama because of its shape - many small watersheds rather  
> than a single big one) - it took me several days to do that.
> Then I ran r.terraflow and I got it in 3 hours.
> 
> Yann says that r.terraflow did not work for him - now I remember what  
> the problem was when I tried to run it recently -
> it needs a LOT of hard drive space which is not a problem these days,  
> BUT the default has been changed
> to /tmp which for my linux box is only 2GB or so. But when running  
> r.terraflow you can define where
> you want the temporary files to be written - so give it something  
> with a lot of space (tens of gigabytes at least)
> and it should run. I think that the default should be changed to  
> where it was - I think it is the regular grass tmp where people  
> usually have a lot of space for the data.
> 
> If even that does not work you can give a try to brand new  
> TerraSTREAM - see the link below
> (and let me know whether it works for you),
> 
> Helena
> 
>   TerraSTREAM provides a series of components that
> perform flow modeling and terrain analysis tasks on very large digital
> elevation models and works equally well on TIN and grid DEMs. The  
> algorithms
> used in the libraries have provable efficient performance in the  
> worst case,
> even on very large terrains that do not fit in the main memory of the  
> computer.
> TerraSTREAM 0.2 comes with direct GRASS and ArcGIS support as well as  
> a simple
> standalone graphical user interface and powerful command line tools  
> that can be
> used alone or integrated into most GIS environments by scripting. For  
> more
> information about this release and for contact information, visit
> http://madalgo.au.dk/Trac-TerraSTREAM/.
> The TerraSTREAM 0.2 users guide is available here:
> http://madalgo.au.dk/Trac-TerraSTREAM/wiki/UsersGuide .
> 
> 
> 
> On Mar 18, 2008, at 5:32 AM, ivan marchesini wrote:
> 
> > Dear Grass Users and Developers, sorry for cross posting but we  
> > hope the
> > argument can be of interest for all and we hope someone can give us a
> > solution to this problem..
> >
> > We have this kind of problem:
> > * a large DEM (150000000 cells)
> > * an ordinary computer (2 GB ram)
> > * we must obtain the drainage map using r.watershed (without changing
> > resolution), because then we need to be able to calculate the upstream
> > basin for each cell (r.water.outlet).
> > * we have tried r.watershed straigth (but after few seconds a memory
> > allocation problem crashed the program)
> > * we have tried the "-m" option but after 4 days of work it is  
> > still at
> > 0%.
> > * giving up the last option, because it takes too long, we have
> > monitored the ram usage by means of "free -m" and we have seen that
> > r.watershed rapidly saturate the ram and then, after a little usage of
> > swap (20 mb) crashes..  so it seems that r.watershed doesn't use swap
> > memory... (and is then unuseful, as we did, to increase the swap
> > memory)
> > * we have tried to modify
> > "Swappiness" (http://www.gentoo.it/doc/memory.html#doc_chap5) but
> > without success... the error is still the same
> >
> >
> > so at this point:
> > * is adding ram to the computer the only solution?
> > * if yes, how can we estimate the ram to buy!
> > * can we obtain some better results compiling grass with option
> > --enable-largefile
> > * does someone solved a similar problem in some other way?
> >
> > Thank you for your suggestions!!!
> > we hope really in your help
> >
> > Luca & Ivan
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > -- 
> > Ti prego di cercare di non inviarmi files .dwg, .doc, .xls, .ppt.
> > Preferisco formati liberi.
> > Please try to avoid to send me .dwg, .doc, .xls, .ppt files.
> > I prefer free formats.
> > http://it.wikipedia.org/wiki/Formato_aperto
> > http://en.wikipedia.org/wiki/Open_format
> >
> > Ivan Marchesini
> > Department of Civil and Environmental Engineering
> > University of Perugia
> > Via G. Duranti 93/a
> > 06125
> > Perugia (Italy)
> > Socio fondatore GFOSS "Geospatial Free and Open Source Software"  
> > http://www.gfoss.it
> > e-mail: marchesini at unipg.it
> >         ivan.marchesini at gmail.com
> > tel: +39(0)755853760
> > fax (university): +39(0)755853756
> > fax (home): +39(0)5782830887
> > jabber: geoivan73 at jabber.org
> >
> > _______________________________________________
> > grass-dev mailing list
> > grass-dev at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/grass-dev
> 
> 
-- 
Ti prego di cercare di non inviarmi files .dwg, .doc, .xls, .ppt.
Preferisco formati liberi.
Please try to avoid to send me .dwg, .doc, .xls, .ppt files.
I prefer free formats.
http://it.wikipedia.org/wiki/Formato_aperto
http://en.wikipedia.org/wiki/Open_format

Ivan Marchesini
Department of Civil and Environmental Engineering
University of Perugia
Via G. Duranti 93/a 
06125
Perugia (Italy)
Socio fondatore GFOSS "Geospatial Free and Open Source Software" http://www.gfoss.it
e-mail: marchesini at unipg.it
        ivan.marchesini at gmail.com
tel: +39(0)755853760
fax (university): +39(0)755853756
fax (home): +39(0)5782830887
jabber: geoivan73 at jabber.org