[GRASS-dev] matplotlib example script

[Michael Barton]

On Jul 24, 2008, at 6:03 PM, Glynn Clements wrote:

> Even if it takes just as long, you're less likely to have it fail
> because "plotlist" consumed all available RAM. As it stands, plotlist
> will have one entry for every non-null cell in the raster.
>
> When processing "bulk" data, anything that uses a fixed amount of
> memory (e.g. one integer per bin) is preferable to using memory
> proportional to the size of the input.
>
> Hence the recommendation to iterate over the lines of r.stats' output
> rather than read it all into a list then iterate over the list.

To do a histogram, I need to send ax.hist a list of values. So I don't  
know how I can get away without creating that list unless I use a  
completely different algorithm (something from numpy?).

This probably isn't a very good example, as there may be other more  
efficient ways to get the data to actually create the histogram. It's  
more a way to try this out and see if it is useful--and what needs to  
be done to use it.

On the other hand, in spite of recent improvements, d.hist is still  
pretty ugly, with formatting issues like varying font sizes on a  
single axis. And it is not very flexible. It would be nice to see  
where standard deviations lie, customize axis formatting, etc. For  
this module in particular, it is probably better to bin the data in  
another way than I have, and input it into one of the matplotlib plot  
methods.

Here are the results of trying this on a larger image (Terra/ASTER)  
with almost 30 million cells. The script takes about 2.5X as long as  
d.histogram (which surprising to me is also using r.stats internally).  
It still isn't bad at 15 seconds for an unoptimized first shot.


GRASS 7.0.svn (Spain_wgs84z30):~ > r.info ASTER.0536.vnir.3   
+ 
----------------------------------------------------------------------------+
  | Layer:    ASTER.0536.vnir.3              Date: Sun Aug 17 23:08:30  
2003    |
  | Mapset:   aster                          Login of Creator:  
cmbarton        |
  | Location:  
Spain_wgs84z30                                                   |
  | DataBase: /Users/Shared/ 
grassdata                                          |
  | Title:     ( ASTER.vnir. 
0536.3 )                                           |
  | Timestamp:  
none                                                            |
   
| 
----------------------------------------------------------------------------|
   
| 
                                                                             |
  |   Type of Map:  raster               Number of Categories:  
252             |
  |   Data Type:     
CELL                                                       |
  |   Rows:          
5175                                                       |
  |   Columns:       
5787                                                       |
  |   Total Cells:   
29947725                                                   |
  |        Projection: UTM (zone  
30)                                           |
  |            N:  4423719.5    S:  4346094.5   Res:     
15                     |
  |            E:   774409.5    W:   687604.5   Res:     
15                     |
  |   Range of data:    min = 5  max =  
252                                     |
   
| 
                                                                             |
  |   Data  
Description:                                                        |
  |    generated by  
i.in.erdas                                                 |
   
| 
                                                                             |
   
| 
                                                                             |
   
+ 
----------------------------------------------------------------------------+


GRASS 7.0.svn (Spain_wgs84z30):~ > time r.stats -Acn ASTER.0536.vnir. 
3  >/dev/null
  100%
r.stats complete.

real	0m2.774s
user	0m2.418s
sys	0m0.122s
GRASS 7.0.svn (Spain_wgs84z30):~ > time histogram_mpldemo.py  
input=ASTER.0536.vnir.3 output=testaster
  100%
r.stats complete.

real	0m14.957s
user	0m11.938s
sys	0m1.443s

GRASS 7.0.svn (Spain_wgs84z30):~ > time d.histogram ASTER.0536.vnir.3  
 >/dev/null 100%
r.stats complete.

real	0m5.811s
user	0m2.440s
sys	0m0.144s

Michael