[GRASS-user] "Parallelization" of Python Script

[Daniel Lee]

Hi Johannes,

Certainly it's possible, it'd just be a question of how you make your loop.
Inside the loop of your jobs you could evaluate whether it's an odd or even
job and check if the previous job has been finished, then start the next
one. I'm not sure what your code looks like, but I've condensed Hamish's
script for my own reference into an example gist so that I can remember
some of the cool tricks. I don't know if it would help you to look at it
since you're already seen the original script, but if you want to take a
look feel free to:

https://gist.github.com/3282580

I think you'll basically have to work with the modulo operator a bit. If
you use that, you can possibly reduce a lot of nested for loops into a for
loop with an if-elif-else query.

Best,
Daniel

--

B.Sc. Daniel Lee
Geschäftsführung für Forschung und Entwicklung
ISIS - International Solar Information Solutions GbR
Vertreten durch: Daniel Lee, Nepomuk Reinhard und Nils Räder

Softwarecenter 3
35037 Marburg
Festnetz: +49 6421 379 6256
Mobil: +49 176 6127 7269
E-Mail: Lee at isi-solutions.org
Web: http://www.isi-solutions.org




2012/8/7 Johannes Radinger <johannesradinger at gmail.com>

> Hi,
>
> for an example of grass.start_command() for parallelizing a bunch
>> of r.cost runs, see v.surf.icw(.py) in grass7 addons:
>>
>>
>> https://trac.osgeo.org/grass/browser/grass-addons/grass7/vector/v.surf.icw/v.surf.icw.py
>>
>> thank you for that example. I think it explains it very well how it works
> to assign
> multiple r.cost runs to single processes with grass.start_command.
> I am just wondering how it is done when there are multiple consecutive
> processes
> in the for loop. In your example (v.surf.icw.py) for each step (e.g.
> r.cost (line 271), r.mapcalc (298))
> an separate for loop is started...Is there a way to combine the steps etc.
> in a function (e.g. combination
> of r.cost and mapcalc) and launch that function in a way like
> grass.start_command in a single loop?
> If possible that would probably save code lines and might be a little more
> clear (at least to me).
> I am just asking because one of my skripts which is still in "serial mode"
> involves lots of steps inside the for loop.
> This would create in parallel at least a dozen for loops which might
> appear very unclear.
>
> Anyway I think the parallelization can really save computation time of my
> skripts... :)
>
> /Johannes
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20120807/b439c1d4/attachment.html>