[GRASS5] Parallel GRASS modules for high performance

[Sorokine, Alexandre]

Helena,

The kind of parallelization (by subregion) you are talking below can be
easily achieved by [ab]using my pd-GRASS module.  pd-GRASS uses shell
scripts and ssh to distribute GRASS on a cluster.  Each node gets its
own subregion.  pd.run will run any GRASS command in parallel on the
cluster nodes thus each regions will be processed on a separate node.
The only function that is missing in pd-GRASS is to assemble results
into a single map.  Everything else is pretty much taken care of by
pd-GRASS.

--
Alex Sorokine, Ph.D. <sorokina at ornl.gov>
Oak Ridge National Lab
 

> -----Original Message-----
> From: grass5-admin at grass.itc.it 
> [mailto:grass5-admin at grass.itc.it] On Behalf Of Helena Mitasova
> Sent: Friday, November 25, 2005 2:38 PM
> To: Muzaffer Ayvaz
> Cc: grass5 at grass.itc.it
> Subject: Re: [GRASS5] Parallel GRASS modules for high performance
> 
> [...skip...]
> 
> The problem with these implementations is that unless the 
> developer is committed to keeping them up to date they die 
> pretty quickly (e.g. the rst works with GRASS5 but not GRASS6).
> So I have been begging everybody who tries to do parallel 
> stuff for grass to do the parallelization on top of the 
> modules rather than within the modules so that they are 
> minimally dependent on changes within the modules. For 
> example, v.surf.rst can be run efficiently by splitting 
> region into smaller overlapping subregions and sending each 
> subregion to a different processor and then patch the results 
> together. Same can be done for r.mapcalc , r.slope.aspect and 
> many other modules (there are some exceptions such as modules 
> that include flow routing). This may have its own problems 
> but it is definitely more general and has much better chance 
> of surviving beyond one release cycle than writing a parallel 
> version of a module.
> 
> [...skip...]