<p>Dear Grass users and developers,</p><div>The region is only 42001 * 42001 i.e. smaller than the limit of 46341 x 46341.<br>The module r.watershed didn't work without -m due to memory requirement.<br>So, the next best, I thought was to assign RAM as much as possible.</div>
<div>And I began trying with 15500 MB downwards.<br>I understand that it's really difficult to isolate Memory overflow errors in any program code.<br>Both the modules have worked ( r.watershed and r.stream.extract) after allocating approximately half the available RAM.<br>
I am really interested in investigating the reasons of r.stream.extract not creating the vector table in postgres with large regions. </div><div>Please note that for smaller regions, everything is perfect.</div><div>I have observed that the initial calculation of memory requirement is not perfect but it's a good indicator.<br>
Almost two years back with quadcore AMD, 2 gb RAM on GRASSv6 , I had observed this.</div><div>For a much smaller region and 1 kM resolution data, the memory required as reported by r.watershed was 60gb (without RAM i.e. -m flag).<br>
With usual swap space of 4gb, it didn't worked.</div><div>However, it worked when I added a temporary swap space of 38 gb and used -m flag.</div><div>I ran my computer uninterrupted for a week to get the results.</div>
<p>The GRASSv7 is in much better shape and moving in right direction to utilise fully the expected hardware development (processing power, cheaper RAM, multiple cores)<br> and better quality of DEM data. ( Eight years back, it was a big pain digitising the contours for the fun of watching rivers flow on your desktop.)</p>
<div>Now, in the coming month, what I plan that I will format the computer.<br>Reload everything from scratch.<br>And on 1 tb swap space and 1 tb disk space , try everything afresh.</div><div>This will highlight the issue related with disk space, if any.</div>
<div>I will additionally try to do things advised by GRASS developers.</div><div><br><br> </div><div class="gmail_quote">On Sun, Nov 20, 2011 at 3:38 AM, Glynn Clements <span dir="ltr"><<a href="mailto:glynn@gclements.plus.com">glynn@gclements.plus.com</a>></span> wrote:<br>
<blockquote style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;" class="gmail_quote"><div class="im"><br>
Hamish wrote:<br>
<br>
> note that at a region size of 46341 x 46341 we<br>
> get to 2^31 cells, and to the point where a signed<br>
> 32bit integer overflows and wraps backwards on<br>
> itself. If that were the case I'd suspect a malloc<br>
> error or a segfault not SIGKILL, but it may be a<br>
> clue.<br>
<br>
</div>The value of 9 indicates that the child process (etc/r.watershed.seg)<br>
terminated either with exit(9) or due to signal 9 (SIGKILL). The<br>
latter seems more likely (i.e. SIGKILL from the kernel's "OOM-killer"<br>
is quite likely for a process which uses too much memory, while I<br>
can't see any mechanism by which exit(9) would occur).<br>
<br>
If the kernel runs low on either physical or virtual memory, it<br>
identifies a process which is using a lot of memory and kills it as if<br>
by sending SIGKILL (although SIGKILL isn't actually "sent"; it can't<br>
be blocked, ignored or caught, so the kernel just deletes the process;<br>
if the parent calls wait() etc on the process, it is reported that the<br>
process was terminated via SIGKILL).<br>
<br>
AFAICT, r.watershed requires far more memory than just the size of the<br>
underlying raster data. It's possible that it isn't interpreting the<br>
memory= parameter correctly. It's also possible that the kernel is<br>
including the memory used for caching the segment file in deciding<br>
which process to kill.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Glynn Clements <<a href="mailto:glynn@gclements.plus.com">glynn@gclements.plus.com</a>><br>
</font></span></blockquote></div><br>