[GRASS-dev] multi-threaded r.mapcalc

[Dylan Beaudette]

On Friday 21 November 2008, Glynn Clements wrote:
> I have added experimental multi-threading support to r.mapcalc in 7.0
> (r34440). It isn't enabled by default; to enable it, use
> "make USE_PTHREAD=1". It only uses a handful of pthread functions, so
> there's a reasonable chance of it working on MacOSX and/or Windows
> (with the pthread-win32 library).
>
> You can change the number of worker threads by setting the WORKERS
> environment variable.
>
> The parallelism is limited to evaluation of argument expressions to
> functions and operators. It doesn't attempt to calculate multiple rows
> concurrently, or multiple output maps.
>
> Also, only one thread can call get_map_row() at a time, so commands
> which are I/O bound (i.e. which only perform simple calculations, so
> most of the time is spent performing I/O), won't see much of an
> improvement.
>
> This is mainly due to the fact that the libgis raster I/O functions
> aren't re-entrant, due to the use of pre-allocated row buffers (and
> possibly other issues).
>
> [There are also static row buffers in r.mapcalc itself, but those
> could easily be eliminated if there was any point.]
>
> Finally, eval() is excluded from parallelisation, as it is used for
> variable assignments, and a variable needs to be assigned before it is
> used. If this is significant, we can have separate parallel and
> sequential versions. If you perform variable assignments other than in
> eval(), you lose.
>
> Mostly, I'm interested in discovering whether this actually has any
> practical benefit. IOW, whether anyone is actually using scripts which
> do enough computation to make use of additional cores.

This type of built-in parallelisation would be nice for CPU-bound operations 
such as r.sun-- I often split my script across multiple cores via several 
background jobs. The proposed approach would be much cleaner.

Nice work!

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341