[GRASS-user] r.watershed / r.terraflow - huge area (amazon)

[Markus Metz]

On Wed, Jan 11, 2017 at 3:58 PM, Carlos Grohmann <carlos.grohmann at gmail.com>
wrote:

> Hi,
>
> Sorry for the bad math :)
>
> I mixed up things. With 03-sec resolution, I have 1.4 billion cells, and
> r.watershed would need 45.5 GB.
> With 01-sec resolution, I have 12.8 billion cells and r.watershed need
> about 415 GB.
>
> The Location is in a drive with 1TB free. So this shouldn't be a problem.
> I'm running it now and it seems it's going to work. I'll report on this
> tomorrow.
>
>
> I think I didn't expressed correctly the r,terraflow issue. I didn't meant
> if the rows/columns limit is needed in the sense of it being a required
> parameter, but it was more of a general question, in the sense of 'do we
> need to have that limit hardcoded?' I can open a ticket on this issue.
>
> The limit of 32,767 is indirectly hard-coded because r.terraflow uses
short integer to store row and column indices. The authors of the module
assumed that massive grids are not larger than 32,767 x 32,767 cells. This
limit can be changed by using 32 bit integers to store row and column
indices, but this will increase disk space requirements which are already
higher than for r.watershed.

Markus M



> best
>
> Carlos
>
>
>
> On Tue, Jan 10, 2017 at 2:51 PM, Markus Neteler <neteler at osgeo.org> wrote:
>
>> On Tue, Jan 10, 2017 at 12:30 PM, Carlos Grohmann
>> <carlos.grohmann at gmail.com> wrote:
>> > Hello all
>> >
>> > I'm working on deriving a nice drainage network for the Amazon basin.
>> The
>> > problem is that the area is too large.
>> >
>> > The region settings are these:
>> > projection: 3 (Latitude-Longitude)
>> > zone:       0
>> > datum:      wgs84
>> > ellipsoid:  wgs84
>> > north:      6:17N
>> > south:      21:30S
>> > west:       80:37W
>> > east:       44:50W
>> > nsres:      0:00:03
>> > ewres:      0:00:03
>> > rows:       33340
>> > cols:       42940
>> > cells:      1431619600
>> >
>> > With 14.3 billion cels,
>>
>> Isn't it 1431619600 = 1.431.619.600 = 1.4 billion cells?
>>
>> > r.watershed would need about 450 GiB of disk space
>> > running in seg mode, is that correct? (from the manual, 31 MB for 1
>> million
>> > cells).
>>
>> https://grass.osgeo.org/grass72/manuals/r.watershed.html#
>> large-regions-with-many-cells
>> " The upper limit of the ram version is 2 billion (2^31 - 1) cells,
>> whereas the upper limit for the seg version is 9 billion-billion (2^63
>> - 1 = 9.223372e+18) cells."
>>
>> So that would be supported by r.watershed.
>>
>> > I do have that disk space available, but it's in a secondary drive. If
>> I set
>> > TMPDIR to that drive, will r/watershed use it? I ask because the
>> > documentation (https://grass.osgeo.org/grass72/manuals/variables.html)
>> only
>> > says that this environmental variable is used by "[Various GRASS GIS
>> > commands and wxGUI]".
>>
>> AFAIK the TMPDIR variable is not relevant here. The tmp dir is
>>
>> location/mapset/.tmp/
>>
>> which you could link to the extra drive. This should be simplified for
>> sure:
>>
>> Involved code:
>> https://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/file_name.c
>> https://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/tempfile.c
>>
>> Please open a ticket with the relevant info in it.
>>
>> > On a side note, r.terraflow should be an alternative for such a large
>> > dataset, but from the manual:
>> > "r.terraflow has a limit on the number of rows and columns (max 32,767
>> > each)".
>> > That's the size of a positive short integer. Is this limit still needed?
>>
>> I have no idea here, did you try it?
>>
>> Markus
>>
>>
>> --
>> Markus Neteler
>> http://www.mundialis.de - free data with free software
>> http://grass.osgeo.org
>> http://courses.neteler.org/blog
>>
>
>
>
> --
> Prof. Carlos Henrique Grohmann
> Institute of Energy and Environment - Univ. of São Paulo, Brazil
> - Digital Terrain Analysis | GIS | Remote Sensing -
>
> http://carlosgrohmann.com
> http://orcid.org/0000-0001-5073-5572
> ________________
> Can’t stop the signal.
>
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20170112/8c694810/attachment.html>