[GRASS-user] zonal statistics/metrics - beyond simple statistics

Fri Apr 13 05:34:14 PDT 2018

 Hi Stephan,
nice points you raised. I didn't know about the Unix philosophy, and indeed it seems as a more solid / simple way of building things...
And yeah, I agree that these metrics/statistics I mentioned are not 'general' at all.What I meant by general is that we can write any other function that has one or more raster/vector maps as input, makes some calculation and returns a value, and just use this function as input for this 'GeneralizedZonalStats', without having to change anything else...
But yes, I agree that calling that 'Generalized' or 'General' may not be the best option...
Best,Bernardo
    Em quinta-feira, 12 de abril de 2018 12:18:20 BRT, Stefan Blumentrath <Stefan.Blumentrath at nina.no> escreveu:  

 #yiv8564277711 #yiv8564277711 -- _filtered #yiv8564277711 {font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv8564277711 {panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv8564277711 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv8564277711 {panose-1:0 0 0 0 0 0 0 0 0 0;}#yiv8564277711 #yiv8564277711 p.yiv8564277711MsoNormal, #yiv8564277711 li.yiv8564277711MsoNormal, #yiv8564277711 div.yiv8564277711MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:New serif;}#yiv8564277711 a:link, #yiv8564277711 span.yiv8564277711MsoHyperlink {color:blue;text-decoration:underline;}#yiv8564277711 a:visited, #yiv8564277711 span.yiv8564277711MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv8564277711 p.yiv8564277711msonormal0, #yiv8564277711 li.yiv8564277711msonormal0, #yiv8564277711 div.yiv8564277711msonormal0 {margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New serif;}#yiv8564277711 p.yiv8564277711msonormal0, #yiv8564277711 li.yiv8564277711msonormal0, #yiv8564277711 div.yiv8564277711msonormal0 {margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New serif;}#yiv8564277711 p.yiv8564277711msonormal, #yiv8564277711 li.yiv8564277711msonormal, #yiv8564277711 div.yiv8564277711msonormal {margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New serif;}#yiv8564277711 p.yiv8564277711msochpdefault, #yiv8564277711 li.yiv8564277711msochpdefault, #yiv8564277711 div.yiv8564277711msochpdefault {margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New serif;}#yiv8564277711 span.yiv8564277711msohyperlink {}#yiv8564277711 span.yiv8564277711msohyperlinkfollowed {}#yiv8564277711 span.yiv8564277711emailstyle18 {}#yiv8564277711 p.yiv8564277711msonormal1, #yiv8564277711 li.yiv8564277711msonormal1, #yiv8564277711 div.yiv8564277711msonormal1 {margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:serif;}#yiv8564277711 span.yiv8564277711msohyperlink1 {color:blue;text-decoration:underline;}#yiv8564277711 span.yiv8564277711msohyperlinkfollowed1 {color:purple;text-decoration:underline;}#yiv8564277711 p.yiv8564277711msonormal01, #yiv8564277711 li.yiv8564277711msonormal01, #yiv8564277711 div.yiv8564277711msonormal01 {margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:serif;}#yiv8564277711 span.yiv8564277711emailstyle181 {font-family:sans-serif;color:#1F497D;}#yiv8564277711 p.yiv8564277711msochpdefault1, #yiv8564277711 li.yiv8564277711msochpdefault1, #yiv8564277711 div.yiv8564277711msochpdefault1 {margin-right:0cm;margin-left:0cm;font-size:10.0pt;font-family:New serif;}#yiv8564277711 span.yiv8564277711EmailStyle30 {font-family:sans-serif;color:#1F497D;}#yiv8564277711 .yiv8564277711MsoChpDefault {font-size:10.0pt;} _filtered #yiv8564277711 {margin:70.85pt 70.85pt 70.85pt 70.85pt;}#yiv8564277711 div.yiv8564277711WordSection1 {}#yiv8564277711 
Hi Bernado,

It depends what you consider general. Even if I work with landscape ecology myself, I would assume landscape metrics or functional connectivity to be rather “special” than “general” statistics.

Putting a lot of functionality into one module can be very useful for some cases, but can come with a number of drawbacks as well.

A general Unix concept is to do one thing and do it well, and rather write modular programs that work together than very complex single tools…

Cheers

Stefan

1:https://en.wikipedia.org/wiki/Unix_philosophy

From: Bernardo Santos <bernardo_brandaum at yahoo.com.br>
Sent: torsdag 12. april 2018 16.57
To: Helmut Kudrnovsky <hellik at web.de>; grass-user at lists.osgeo.org; Stefan Blumentrath <Stefan.Blumentrath at nina.no>
Subject: Re: RE: [GRASS-user] zonal statistics/metrics - beyond simple statistics

Hi Stephan,

Some minutes before you sent me this e-mails, I'd just found out about v.db.select -r. It worked and made the process much faster. Thanks for that anyway.

In my case, the polygons do not overlap. Indeed, many things could be done with r.univar, r.stats or a combination of them and their output info. However, my aim was to think about a way of generalizing it, so that we would be able to use many other functions from the raster maps using the same architecture for calculating the zonal statistics/metrics.

For example, I am still not sure how I would use a combination of r.univar/r.stats and some kind of input raster map to produce, say, zonal info of number of habitat patches, structural or functional connectivity, isolation, or other more complex metrics used in landscape ecology.

That's way I was building something like that, so that we can just write an alternative function using any raster input map that returns the value of one of such metrics, and then use it to calculate them over sets of zones.

Best,

B

PS: thanks for sharing your code! This v.rast.bufferstats seems very useful, and I can think of some very interesting applications of that. I helped a friend to write something similar but using ArcPy as a ArcMap toolbox, and we applied it to some interesting cases in the delimitation of Protected Areas' buffer zones (but still did not published it). But doing that with a free and open source tool would be much more interesting! 

Em quarta-feira, 11 de abril de 2018 18:32:18 BRT, Stefan Blumentrath <Stefan.Blumentrath at nina.no> escreveu:

Hi Bernado,

Please find attached the «work-in-progress» version of v.rast.bufferstats for some more inspiration.

v.db.select -r does what you are looking for:

https://grass.osgeo.org/grass74/manuals/v.db.select.html

However, if the polygons you are working with don`t overlap, looping should not be required, and you should be able to use r.univar with zones or r.stats for all polygons at once…

Cheers

Stefan

From: Bernardo Santos <bernardo_brandaum at yahoo.com.br>
Sent: onsdag 11. april 2018 20.59
To: Helmut Kudrnovsky <hellik at web.de>; grass-user at lists.osgeo.org; Stefan Blumentrath <Stefan.Blumentrath at nina.no>
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

Hi Helmut and Stefan,

thanks for sharing that with me, it is good to know that there are similarongoing approaches to solve related issues.

Helmut,

very nice addon. However, it seems to me that the part on calculating statistics is still based on v.rast.stats, am I right?

Stefan, the question you pose is indeed very related. I took a better look at my code and I noticed that what really takes time is not the process of making a mask, but what comes before: the rasterization of one of the polygons of the input vector.

What I do is:

1) g.region using the vector layer, aligning it to the first of the input rasters

2) v.to.rast selecting only one of the polygons at a time from the vector layer

3) r.mask using the rasterized polygon

4) g.region zoom=rasterized_polygon

5) calculate any function for all raster maps (in principle using a loop), using this small region and the mask, and attaching the results to the attribute table

6) repeat 1-5 for all polygons in the input vector

However, I noticed now that what takes time is v.rast.stats, since it depends on the whole region selected, which in my case is much bigger than a single polygon (the vector is a set of ~6,000 small polygons).

Do you guys know if there is a way of using a single polygon from a vector to define the region using something like g.region?

[this would be really great!!]

something like

g.region vector=vector_of_interest cat=1

Best,

Bernardo

Em terça-feira, 10 de abril de 2018 06:01:34 BRT, Stefan Blumentrath <Stefan.Blumentrath at nina.no> escreveu: 

Hei Bernardo,

Just for the record: I am currently working on something similar / related.

First of all I would like to improve speed of v.rast.stats for multiple inputs:
https://www.mail-archive.com/grass-dev@lists.osgeo.org/msg52562.htmland 
https://trac.osgeo.org/grass/ticket/3523

Adding a functionality to "tabulate areas" or "count categories" is a next step I have in mind which I am working on in v.rast.bufferstats (will be a port of v.what.rast.buffer [1] to GRASS 7 / pygrass (with some enhancements)). Once I have a good solution there I would like to see if it can be moved to v.rast.stats. Support for dbf driver is however a challenge...

I will add my "work in progress" script to github, so you can have a look if you like.

Cheers
Stefan

1: https://grass.osgeo.org/grass64/manuals/addons/

-----Original Message-----
From: grass-user <grass-user-bounces at lists.osgeo.org> On Behalf Of Helmut Kudrnovsky
Sent: mandag 9. april 2018 23.19
To: grass-user at lists.osgeo.org
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

Bernardo Santos wrote
> Dear GRASS list,
> I am developing a Python script to be able to calculate (virtually 
> any) metrics or statistics for zones/polygons in a vector - in analogy 
> to zonal statistics (such as v.rast.stats).The idea is that one can 
> calculate raster-based metrics (such as proportion of habitat, number 
> of patches, or any metric that can be formalized as a function that 
> takes some information from the input raster and returns a value) for 
> each polygon in the vector, and this is updated as a value in a newly 
> created column in the attribute table of this vector.
> Is there already anything like that (some addon/module) that I am 
> missing, just to avoid re-doing something already created?
> If not, what I am doing is to create a loop over all the features in a 
> vector, and for each one I zoom and use the polygon to define a mask 
> (using r.mask), so that the calculation of the selected metric is 
> performed only over that polygon, and this process is repeated.The 
> script allows one to calculate metrics/statistics for multiple raster 
> maps at once, and to incorporate other function for statistics also. 
> It may be found here:https://github.com/LEEClab/GeneralizedZonalStats
> 
> For small vectors this works nicely and I believe it has a great 
> potential. However, when I try to calculate metrics for a large 
> dataset (e.g. the Brazilian map of cities, with almost 6,000 polygons) 
> - and that is when the tool would be interesting -, the process of 
> creating each mask takes too long (387 steps), and the tool becomes kind of useless.
> Then I have two questions:- First, what drives the number of steps 
> GRASS takes to create a mask? Why it is very small for some maps but 
> very large for others? I quite don't understand that yet.- Do you 
> think of a easier or faster way of doing the same thing (instead of using masks)?
> v.rast.stats seems to use r.univar and the option 'zones' for doing 
> so, but then one gets restricted to the statistics calculated by this module.
> Any help or comment would be very welcome!
> Best,Bernardo Niebuhr
> _______________________________________________
> grass-user mailing list

> grass-user at .osgeo

> https://lists.osgeo.org/mailman/listinfo/grass-user

have look at

https://grass.osgeo.org/grass74/manuals/addons/v.habitat.dem.html

where I've coded some DEM derived stats and characteristics for polygones.

-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html
_______________________________________________
grass-user mailing list
grass-user at lists.osgeo.org

https://lists.osgeo.org/mailman/listinfo/grass-user
_______________________________________________
grass-user mailing list
grass-user at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20180413/1f0bad4a/attachment-0001.html>