[GRASS-user] r.stream.extract error

[Giuseppe Amatulli]

Thanks Markus!!
I will test and I will let you know how it works.

I have few  more questions
1) now how much is the upper limit matrix cell number that r.stream.extract
can handle?
2) is the r.stream.basins add-on subjects to the same limitation? In case
would be possible to update also for  r.stream.basins?
3) is r.stream.extract support the use of multi-threaded through openMP?
Would be difficult implement?

Best
Giuseppe







On 31 October 2017 at 15:54, Markus Metz <markus.metz.giswork at gmail.com>
wrote:

>
>
> On Mon, Oct 30, 2017 at 1:42 PM, Giuseppe Amatulli <
> giuseppe.amatulli at gmail.com> wrote:
> >
> > Hi,
> > I'm using the r.stream.extract grass command
> >
> > r.stream.extract elevation=elv  accumulation=upa threshold=0.5
>  depression=dep     direction=dir  stream_raster=stream memory=35000 --o
> --verbose
> >
> > where the elv is raster of 142690 *  80490 = 11,485,118,100 cell
> >
> > and I get this error
> >
> > 12.97% of data are kept in memory
> > Will need up to 293.52 GB (300563 MB) of disk space
> > Creating temporary files...
> > Loading input raster maps...
> > 0..3..6..9..12..15..18..21..24..27..30..33..36..39..42..
> 45..48..51..54..57..60..63..66..69..72..75..78..81..84..
> 87..90..93..96..99..100
> > ERROR: Unable to load input raster map(s)
>
> This error is caused by integer overflow because not all variables
> necessary to support such large maps were 64 bit integer.
>
> Fixed in trunk and relbr72 with r71620,1, and tested with a DEM with
> 172800 * 67200 = 11,612,160,000 cells: r.stream.extract finished
> successfully in 18 hours (not a HPC, a standard desktop maschine with 32 GB
> of RAM and a 750 GB SSD).
> >
> > According to the help manual the memory=35000 should be set in according
> to the overall memory available. I set the HPC upper memory limit to 40G.
> >
> > I try several combination of these parameters  but i still get the same
> error.
> > If the r.stream.extract is based on r.watershed than the segmentation
>  library should be able to handle a huge raster.
>
> r.stream.extract is based on a version of r.watershed that did not support
> yet such huge raster maps, therefore support for such huge raster maps
> needed to be added to r.stream.extract separately.
>
> >
> > Anyone know how to over pass this limitation/error ?
>
> Please use the latest GRASS 7.2 or GRASS 7.3 version from svn.
>
> Markus M
>
> >
> > Thank you
> > Best
> > --
> > Giuseppe Amatulli, Ph.D.
> >
> > Research scientist at
> > Yale School of Forestry & Environmental Studies
> > Yale Center for Research Computing
> > Center for Science and Social Science Information
> > New Haven, 06511
> > Teaching: http://spatial-ecology.org
> > Work:  https://environment.yale.edu/profile/giuseppe-amatulli/
> >
> > _______________________________________________
> > grass-user mailing list
> > grass-user at lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/grass-user
>
>


-- 
Giuseppe Amatulli, Ph.D.

Research scientist at
Yale School of Forestry & Environmental Studies
Yale Center for Research Computing
Center for Science and Social Science Information
New Haven, 06511
Teaching: http://spatial-ecology.org
Work:  https://environment.yale.edu/profile/giuseppe-amatulli/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20171101/ca21fd26/attachment.html>