[GRASS-dev] Addons to Parallelize

[Hamish]

Fernando wrote:
> my name is Fernando Sierra, I'm working with Grass7 and I want to
> parallelize a few commands. 

see http://grass.osgeo.org/wiki/OpenMP
    http://grass.osgeo.org/wiki/Category:Parallelization

> I have a ToDoList:

how's you experience with programming in C, C++, OpenMP, OpenCL, and/or
pthreads?

>    r.cva. 

(go for it, but fwiw not part of grass)

>    r.mapcalc

r.mapcalc already uses pthreads

>   r.surf.rst

Some work has been done for the RST library wrt its use of the gmath
library's LU decomposition. From that I get the feeling that it may be
better to parallelize the modules rather than the inner loops of the
libraries, counter-intuitive as that may sound. i.e. it is more efficient
to spawn a process or thread for 2-24 CPUs than it is to spawn 2000 of
them for e.g. each row of data. Parallelizing imagery modules for R,G,B
bands only scales to 3 in parallel, but is an easy win. (e.g. see i.*
python scripts in grass7).

i.e. for the next few years # of CPUs on typical deployments will be <<256,
so many more threads than that just cause wasted overhead. So far the most
effective MP I've seen in grass is the "poor man's" method of just launching
multiple modules at the same time.  see the r3.in.xyz python script for
an example.

For v.surf.rst, I would suggest an approach of perhaps handling the
quadtree elements in their own threads and running the LU decomposition
many times, rather than try to make the individual LU be parallelized.
The v.surf.bspline module may be an easier win, it's already segmenting
its job into ~ 20-100 chunks, but running them serially. See the code
for existing comments about where the parallelization could go.

>   r.los

r.los's current method doesn't scale, better to focus on r.viewshed.
Having said that, r.los works by splitting up the circular pie into
a number of slices, which might be easy to send to their own threads.

> I don't know which commands are parallelized or could be
> parallelized.
> Do you have any idea or suggestion about how to start?

the wiki pages have a lot of info already, hope it helps.

I think it is important to identify which modules are CPU bound, and
focus on them first.


best,
Hamish