[GRASS-user] g_malloc on r.watershed with 30GB free ram + r.fill.dir crash

[Markus Neteler]

On Thu, Sep 10, 2015 at 9:00 PM, mael <mael.moreni at onf.fr> wrote:
> I am working on a 13407 lines * 11050 columns, 565 MB, raster file (SRTM
DEM,
> 30m) with a script that calls a series of hydrological modules in GRASS
> 7.0.0.
> The system I am using is a Windows server 2012 R2, 64 bit Intel Xeon 2.5
GHz
> processor, 32GB of RAM.
>
> The first module called in the script is r.watershed. When lunched on that
> data set, it generates the following error : G_malloc: impossible to
> allocate 1185910608 bytes to raster/r.watershed/ram/init_vars.c:149
> It apears to be running out of memory even though there is a lot more
> available than what it's asking for.

Very likely your computational region is wrong, i.e. too large.
Check it with

g.region -p

I just used the EU DEM 25m to make a test on my tiny ASUS laptop (4GB RAM,
Intel i3), using Fedora 22, 64bit:

> 13407  * 11050  <<--- your DEM
[1] 148147350
>
> 12880 * 16370  <<-- my DEM
[1] 210845600

(I just had this DEM ready here to play with)


GRASS 7.1.svn (eu_laea):~ > g.region -p
projection: 99 (Lambert Azimuthal Equal Area)
zone:       0
datum:      etrs89
ellipsoid:  grs80
north:      2699750
south:      2377750
west:       4126750
east:       4536000
nsres:      25
ewres:      25
rows:       12880
cols:       16370
cells:      210845600


Results:

RAM: it used some swap memory since I have browser etc open at the same
time.

[neteler at oboe ~]$ free
              total        used        free      shared  buff/cache
available
Mem:        3930508     3600648       32408       93164      297452
 165584
Swap:       3932156     2735164     1196992

Timing:

GRASS 7.1.svn (eu_laea):~ > time -p r.watershed elevation=eu_dem_25_TN
accumulation=eu_dem_25_TN.acc basin=eu_dem_25_TN.watersheds threshold=10000
SECTION 1a (of 5): Initiating Memory.
SECTION 1b (of 5): Determining Offmap Flow.
 100%
SECTION 2: A* Search.
 100%
SECTION 3a: Accumulating Surface Flow with MFD.
 100%
SECTION 3b: Adjusting drainage directions.
 100%
SECTION 4: Watershed determination.
 100%
SECTION 5: Closing Maps.

real 1270.30
user 1048.68
sys 54.00

... 21 minutes.



> I managed to pass that first hurdle using the -m flag on r.watershed. The
> scripts now requires 7 GB of disk space; even if it's slow and I don't
> understand why it can't use the ram in the first place, r.watershed
finishes
> correctly.

Given my test on a small laptop, it is very likely that your actual
computational region
does not match the input map.

> However, when the script gets to the next module, r.fill.dir, GRASS
crashes
> at 'Reading input elevation raster map...' with the following message:

Why do you use the r.fill.dir module?

Markus

-- 
http://consulting.neteler.org
http://gis.cri.fmach.it/neteler/
http://courses.neteler.org/blog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150910/3832d1b5/attachment.html>