[GRASS-user] Solved: execGRASS("r.diversity") Done. No rasters created (Large rasters)

Tue Sep 30 00:39:39 PDT 2014

This is great Sandra. Now I am curious to see final ecological
relationships you are searching for!
D

2014-09-30 3:24 GMT+02:00 Sandra MacFadyen <sandramf at live.co.za>:

> Dear Vaclav,
>
>
>
> I did a quick manual test to see if the results were correct for a few 5x5
> windows and yes, everything appears 100%:
>
>
>
> # Example data from one 5x5 window
>
> test1 = c(572,617,661,638,616,
>
>           662,617,593,638,616,
>
>           662,639,572,661,616,
>
>           662,662,639,662,616,
>
>           594,572,594,639,593)
>
>
>
> # Calculate info needed for diversity indices
>
> freq1 = as.data.frame(table(test1)) # Calculate frequency table
>
> S = length(unique(test1)) # Total number of species (S)
>
> N = length(test1) # Total number of individuals (N)
>
> ln.S = log(S) #Natural log of species (ln S)
>
> ln.N = log(N) # Natural log of individuals (ln N)
>
> Pi = freq1$Freq/N # Proportion of individuals that belong to each species
>
> Pi2 = (Pi ^ 2)
>
> ln.pi = log(Pi)
>
> pi.ln.pi = Pi * ln.pi
>
> pi.ln.pi2 = (ln.pi^2)*Pi
>
>
>
> # Calculate different diversity indices
>
> (M = (S-1)/ln.N) # Margalef's index (M)
>
> (i.Di = 1/sum(Pi2)) # Simpson's index [Inverse] (i.Di)
>
> (i.Dc = 1 - (sum(Pi2))) # Simpson's index [Complement] (i.Dc) * This one
> calculated with r.diversity
>
> (H = sum(pi.ln.pi)*-1) # Shannon-Wiener index (H')
>
> (J = H/ln.S) # Pielou's index (J)
>
>
>
> Thanks again
>
>
>
> Cheers
>
> Sandra
>
>
>
> *From:* Vaclav Petras [mailto:wenzeslaus at gmail.com <wenzeslaus at gmail.com>]
>
> *Sent:* 18 September 2014 02:46 AM
> *To:* Sandra MacFadyen
> *Cc:* Markus Neteler; GRASS user list; Duccio Rocchini; Luca Delucchi
> *Subject:* Re: [GRASS-user] Solved: execGRASS("r.diversity") Done. No
> rasters created (Large rasters)
>
>
>
>
>
>
>
> On Wed, Sep 17, 2014 at 7:40 PM, Sandra MacFadyen <sandramf at live.co.za>
> wrote:
>
> Dear Markus,
>
> Excellent! Thanks to everyone for their help.
> I tested r.diversity in the new revision (61840) on a raster with 12933032
> cells and it took under 3 minutes to complete r.li.simpson, r.li.shannon,
> r.li.pielou and r.li.renyi outputs.
> Amazing :)
>
> > execGRASS("r.diversity",flags="overwrite", parameters=list(input="flow",
> prefix="flow_div", alpha=0.5, size=3))
> > r.li.simpson complete. Raster map <flow_div_simpson_size_3.0> created.
> > r.li.shannon complete. Raster map <flow_div_shannon_size_3.0> created.
> > r.li.pielou complete. Raster map <flow_div_pielou_size_3.0> created.
> > r.li.renyi complete. Raster map <flow_div_renyi_size_3.0_alpha_0.5>
> > created.
> > Done.
>
> Thanks again and keep well.
>
> Good to hear that. Can you say if the results are correct? I computed
> difference of one map before and after and it was OK. But it would be great
> to have more tests. Let's start with: does the result make sense?
>
>
>
> Vaclav
>
>
> Cheers
> Sandra
>
> -----Original Message-----
> From: neteler.osgeo at gmail.com [mailto:neteler.osgeo at gmail.com] On Behalf
> Of Markus Neteler
> Sent: 06 September 2014 05:15 AM
> To: Sandra MacFadyen
> Cc: GRASS user list; Duccio Rocchini; Luca Delucchi
> Subject: Re: [GRASS-user] execGRASS("r.diversity") Done. No rasters
> created (Large rasters)
>
> Dear Sandra,
>
> On Fri, Jun 20, 2014 at 7:49 AM, Markus Neteler <neteler at osgeo.org> wrote:
> > On Sat, Jun 14, 2014 at 9:21 AM, Sandra MacFadyen <sandramf at live.co.za>
> wrote:
> >> I am using r.diversity (GRASS GIS 7.0.0svn build 60785 win32) through
> >> R (R version 3.0.2 win32) on Windows 7 64bit.
> ...
> >> However, when running the same code on a larger image (cells=6746328)
> >> from my own location, although it reports Done, no rasters are
> >> created. If I subset the image
> >> (cells=1632830) and run it again its works (see # sub2Kruger # code and
> results below).
> >> So I'm guessing it is a memory issue?
> ...
> > ... it consumes a lot of memory... will check on a bigger machine,
> > perhaps a memory leak.
>
> The assumption turned out to be right and I think we got it today!
>
> Vaclav Petras checked it and discovered an "unfortunate" memory allocation
> which he fixed in r.li.* in revision
> http://trac.osgeo.org/grass/changeset/61812 ("r.li: fix memory handling
> (memory leak in avl_to_array function))".
>
> Now r.li has become very fast, on my laptop:
>
> GRASS 7.1.svn (nc_spm_08_grass7):~ >  g.region -p rast=lsat5_1987_10
> res=10 -a ...
> rows:       1355
> cols:       1503
> cells:      2036565
>
> GRASS 7.1.svn (nc_spm_08_grass7):~ > time -p r.li.simpson --o
> input=lsat5_1987_10 at landsat conf=conf_diversity_5.0
> output=lsat5_1987_div__simpson_size_5.0
> r.li.simpson complete. Raster map <lsat5_1987_div__simpson_size_5.0>
> created.
> --> 29.32 seconds
>
> or with a simulated higher resolution:
>
> GRASS 7.1.svn (nc_spm_08_grass7):~ >  g.region -p rast=lsat5_1987_10
> res=5 -aprojection: 99 (Lambert Conformal Conic) ...
> rows:       2708
> cols:       3005
> cells:      8137540
>
> GRASS 7.1.svn (nc_spm_08_grass7):~ > time -p r.li.simpson --o
> input=lsat5_1987_10 at landsat conf=conf_diversity_5.0
> output=lsat5_1987_div__simpson_size_5.0
> r.li.simpson complete. Raster map <lsat5_1987_div__simpson_size_5.0>
> created.
> --> 227.37 seconds (used to be > 2 hours)
>
> So, to grab this improvement for Windows, grab the version from here:
> http://wingrass.fsv.cvut.cz/grass71/
>
> (or via OSGeo4W installer). Be sure that the revision is at least
> r61812 which is indicated in the file name.
>
> Please let us know if all works to avoid that the change has any negative
> impact.
> Tests here did not show any changes in the output except for the speed
> improvement and solved memory leak.
>
> Subsequently also r.diversity should behave now.
>
> I'll backport it to GRASS 7.0 release branch after some testing.
>
> Markus
>
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user
>
>
>

-- 
Duccio Rocchini, PhD

http://gis.cri.fmach.it/rocchini/

Fondazione Edmund Mach
Research and Innovation Centre
Department of Biodiversity and Molecular Ecology
GIS and Remote Sensing Unit
Via Mach 1, 38010 San Michele all'Adige (TN) - Italy
Phone +39 0461 615 570
ducciorocchini at gmail.com
duccio.rocchini at fmach.it
skype: duccio.rocchini
Google Scholar Profile: scholar.google.it/citations?hl=it&user=OJtw7agAAAAJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20140930/00931dd1/attachment.html>