[GRASS-user] execGRASS("r.diversity") Done. No rasters created (Large rasters)

[Markus Neteler]

Dear Sandra,

On Fri, Jun 20, 2014 at 7:49 AM, Markus Neteler <neteler at osgeo.org> wrote:
> On Sat, Jun 14, 2014 at 9:21 AM, Sandra MacFadyen <sandramf at live.co.za> wrote:
>> I am using r.diversity (GRASS GIS 7.0.0svn build 60785 win32) through R
>> (R version 3.0.2 win32) on Windows 7 64bit.
...
>> However, when running the same code on a larger image (cells=6746328) from my
>> own location, although it reports Done, no rasters are created. If I subset the image
>> (cells=1632830) and run it again its works (see # sub2Kruger # code and results below).
>> So I'm guessing it is a memory issue?
...
> ... it consumes a lot of memory... will check on a bigger machine,
> perhaps a memory leak.

The assumption turned out to be right and I think we got it today!

Vaclav Petras checked it and discovered an "unfortunate" memory
allocation which he fixed in r.li.* in
revision http://trac.osgeo.org/grass/changeset/61812 ("r.li: fix
memory handling (memory leak in avl_to_array function))".

Now r.li has become very fast, on my laptop:

GRASS 7.1.svn (nc_spm_08_grass7):~ >  g.region -p rast=lsat5_1987_10 res=10 -a
...
rows:       1355
cols:       1503
cells:      2036565

GRASS 7.1.svn (nc_spm_08_grass7):~ > time -p r.li.simpson --o
input=lsat5_1987_10 at landsat conf=conf_diversity_5.0
output=lsat5_1987_div__simpson_size_5.0
r.li.simpson complete. Raster map <lsat5_1987_div__simpson_size_5.0>
created.
--> 29.32 seconds

or with a simulated higher resolution:

GRASS 7.1.svn (nc_spm_08_grass7):~ >  g.region -p rast=lsat5_1987_10
res=5 -aprojection: 99 (Lambert Conformal Conic)
...
rows:       2708
cols:       3005
cells:      8137540

GRASS 7.1.svn (nc_spm_08_grass7):~ > time -p r.li.simpson --o
input=lsat5_1987_10 at landsat conf=conf_diversity_5.0
output=lsat5_1987_div__simpson_size_5.0
r.li.simpson complete. Raster map <lsat5_1987_div__simpson_size_5.0>
created.
--> 227.37 seconds (used to be > 2 hours)

So, to grab this improvement for Windows, grab the version from here:
http://wingrass.fsv.cvut.cz/grass71/

(or via OSGeo4W installer). Be sure that the revision is at least
r61812 which is indicated in the file name.

Please let us know if all works to avoid that the change has any
negative impact.
Tests here did not show any changes in the output except for the speed
improvement and solved memory leak.

Subsequently also r.diversity should behave now.

I'll backport it to GRASS 7.0 release branch after some testing.

Markus