[GRASS-user] GRASS 7 error code 9

[Pankaj Kr Sharma]

Dear  Grass users and developers,
The region is only 42001 * 42001 i.e. smaller than the limit of 46341 x
46341.
The module r.watershed didn't work without -m due to memory requirement.
So, the next best, I thought was to assign RAM as much as possible.
And I began trying with 15500 MB downwards.
I understand that it's really difficult to isolate Memory overflow errors
in any program code.
Both the modules have worked ( r.watershed and r.stream.extract) after
allocating approximately half the available RAM.
I am really interested in investigating the reasons of r.stream.extract not
creating the vector table in postgres with large regions.
Please note that for smaller regions, everything is perfect.
I have observed that the initial calculation of memory requirement is not
perfect but it's a good indicator.
Almost two years back with quadcore AMD, 2 gb RAM on GRASSv6 , I had
observed this.
For a much smaller region and 1 kM resolution data, the memory required as
reported by r.watershed was 60gb (without RAM i.e. -m flag).
With usual swap space of 4gb, it didn't worked.
However, it worked when I added a temporary swap space of 38 gb and used -m
flag.
I ran my computer uninterrupted for a week to get the results.

The GRASSv7 is in much better shape and moving in right direction to
utilise fully the expected hardware development (processing power, cheaper
RAM, multiple cores)
 and better quality of DEM data. ( Eight years back, it was a big pain
digitising the contours for the fun of watching rivers flow on your
desktop.)
Now, in the coming month, what I plan that I will format the computer.
Reload everything from scratch.
And on 1 tb swap space and 1 tb disk space , try everything afresh.
This will highlight the issue related with disk space, if any.
I will additionally try to do things advised by GRASS  developers.



On Sun, Nov 20, 2011 at 3:38 AM, Glynn Clements <glynn at gclements.plus.com>wrote:

>
> Hamish wrote:
>
> > note that at a region size of 46341 x 46341 we
> > get to 2^31 cells, and to the point where a signed
> > 32bit integer overflows and wraps backwards on
> > itself. If that were the case I'd suspect a malloc
> > error or a segfault not SIGKILL, but it may be a
> > clue.
>
> The value of 9 indicates that the child process (etc/r.watershed.seg)
> terminated either with exit(9) or due to signal 9 (SIGKILL). The
> latter seems more likely (i.e. SIGKILL from the kernel's "OOM-killer"
> is quite likely for a process which uses too much memory, while I
> can't see any mechanism by which exit(9) would occur).
>
> If the kernel runs low on either physical or virtual memory, it
> identifies a process which is using a lot of memory and kills it as if
> by sending SIGKILL (although SIGKILL isn't actually "sent"; it can't
> be blocked, ignored or caught, so the kernel just deletes the process;
> if the parent calls wait() etc on the process, it is reported that the
> process was terminated via SIGKILL).
>
> AFAICT, r.watershed requires far more memory than just the size of the
> underlying raster data. It's possible that it isn't interpreting the
> memory= parameter correctly. It's also possible that the kernel is
> including the memory used for caching the segment file in deciding
> which process to kill.
>
> --
> Glynn Clements <glynn at gclements.plus.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/grass-user/attachments/20111120/b6767346/attachment.html