[GRASS-dev] big speedups for i.landsat.rgb

[Hamish]

Hi,

I just committed some changes to i.landsat.rgb(.sh) in 6.5svn
which may be of interest.. For my test map what used to take
40 sec to run now completes in 6.8 sec.

 * combine two repeated r.univar calls into a single run now
   that r.univar supports multiple percentile=s.
  -- this easily cuts the module run-time in half at little cost.

 * (optionally) run r.univar on the three RGB bands in parallel.
   This uses 3x the RAM but completes in 1/3rd of the wall time.


Experimental, but seems to work fine. I've done a similar, but
more advanced, poor man's parallelization with r3.in.xyz in
grass6 addons, which seems to work well. I've been slowly coming
to the realization that the higher level the parallelization
happens at, the more efficient it will be. (at the cost of
narrowed influence)

If the approach is well received, i.oif(.sh) is also low-hanging
fruit, and we can also port the method over to python scripts in
grass7, as described here:
  http://lists.osgeo.org/pipermail/grass-user/2012-February/063627.html



I explored using r.univar's multiple-input map capability
to simplify the serial-mode version of the code, but it uses 3x
the RAM as a single map and actually runs slightly slower than
calling r.univar 3 times + using awk to combine the results.
I guess the large malloc() are to blame for the slight slowdown.
(one reason to keep serial mode around is for a low-RAM mode)

However-- this indicates that r.univar's multiple input map
capability is an easy place to parallelize the code, by processing
each map in its own thread. Not sure if pthreads or OpenMP would
be better for that; OpenMP includes some automatic management
code to only create as many new threads as you have cores (e.g.
if you try to process 20 maps at once it will only spawn 4
threads at a time on a quad-core CPU). Maybe pthreads is smart
like that too or just goes by what you hardcode or set with the
enviro variables?


Any other multi-band imagery modules which jump out as easy
pickings ...? (r.in.gdal when importing interleaved multibands?)


Hamish