[GRASS-user] Parallel processes

[patrick s.]

Dylan

A small sidenote on your issue. I also use GNU parallel for operations 
that have to run on very large scales. Never had problems with it when 
running it on different mapsets, i.e. I create a temporary mapset in my 
scripts that are wrapped  by the command. Storing the data back to a 
main mapset or PostgreSQL allows to delete the temporary mapsets at the 
end of each process. Maybe a kind of hack, but works well for me. Always 
happy on feedback to optimize these ;-)

Greetz,
Patrick

On 20.10.2015 19:09, grass-user-request at lists.osgeo.org wrote:
> Thank you Glynn, your advice confirms some empirical notes:
>
> 1. parallel processes that use data from external USB disks quickly
> saturate the capacity of the bus or mechanism of the drive
>
> 2. parallel processes that use data from an internal SSD can generally
> saturate all 8 cores of my Intel i7
>
>
> My main motivation for asking this question was to determine instances
> where parallel operations in GRASS are_not_  safe. From my reading of
> the wiki, manual pages, and your recent comments on GRASS-dev, it
> would appear that the following operations may not be safe:
>
> 1. region-altering
>
> 2. calculations in the presence of a MASK
>
> 3. reading "external" (r.external) GDAL sources (?)
>
> 4. some mapcalc expressions
>
> In order to simplify my testing, I have disabled pthread support and
> invoke "parallelization" via backgrounding or GNU parallel. My
> examples with GNU parallel stem from the tremendous (apparent) utility
> of this tool, in that most "bash for loops" can be directly converted
> into "smart" parallel jobs.
>
> Thanks,
> Dylan