[GRASS-dev] R.terraflow on massive grids

[Doug_Newcomb at fws.gov]

Markus Metz <markus.metz.giswork at googlemail.com> 
08/15/2011 06:28 AM

To
Doug_Newcomb at fws.gov
cc
grass-dev at lists.osgeo.org
Subject
Re: [GRASS-dev] R.terraflow on massive grids






><Doug_Newcomb at fws.gov> wrote:
>>
>> Hi Folks,
>> I have an fcell grid of elevations for the state of North Carolina 
(51000
>> rows 133000 columns 6783000000 cells) .  I tried to run r.terraflow in
>> GRASS7 ( 8/8/2011 svn snapshot) and ran into the dimension limits.  So 
I
>> patched them according to Glynn's email ,
>> http://www.osgeo.org/pipermail/grass-user/2004-February/024722.html 
 and
>> tried again ( Would it be better to change the  dimension variable to 
int
>> instead of  short int?) .
>>
>> This time my Streams file builds to about 26 GB and then r.terraflow 
bombs
>> with :
>>
>> MFD flow direction
>> D8CUT=999999986991104.000000
>> Memory size: 808.00M (847249408) bytes
>> Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
>> r.terraflow: grass2str.h:145: AMI_STREAM<T>*
>> cell2stream(char*, elevation_type, long int*) [with T =
>> float, elevation_type = float]: Assertion `nrows * ncols ==
>> str->stream_len()' failed.
>>
>> The memory size is interesting, because I'm giving it 8GB of RAM out of 
16
>> GB in the command.

>Note that the memory manager of the iostream lib used by r.terraflow
>can only handle up to 2047 MB of RAM because the memory option in MB
>is internally converted to Byte and stored as int, thus the max
>recognized value is 2^31 - 1 Byte.

>Markus M

This appears to be kind of like pulling on a corner of a spiderweb. :-) 

Should the limit to memory be indicated in the gui?
Would there be any benefit to other parts of GRASS in bumping up this 
memory limit?  I can see the reason on 32 bit windows XP, but that is 
starting to go away.

Doug


Doug Newcomb 
USFWS
Raleigh, NC
919-856-4520 ext. 14 doug_newcomb at fws.gov
---------------------------------------------------------------------------------------------------------
The opinions I express are my own and are not representative of the 
official policy of the U.S.Fish and Wildlife Service or Dept. of the 
Interior.   Life is too short for undocumented, proprietary data formats.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/grass-dev/attachments/20110815/e9bd4b97/attachment.html