[GRASS-dev] multiprocessing problem

[Luca Delucchi]

On Fri, 8 Apr 2022 at 11:17, Stefan Blumentrath
<Stefan.Blumentrath at nina.no> wrote:
>
> Ciao Luca,
>

Ciao Stefan

> Yes, you could also consider looping over e.g. rows (maybe in combination with "np.apply_along_axis") so you could put results easier back together to a map if needed at a later stage.
>
> In addition, since you use multiprocessing.Manager, you may try to use multiprocessing.Array: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Array
>
> E.g. here:
> https://github.com/lucadelu/grass-addons/blob/5ca56bdb8b3394ebeed23aa5b3240bf6690e51bf/src/raster/r.raoq.area/r.raoq.area.py#L81
>
> According to the post here: https://medium.com/analytics-vidhya/using-numpy-efficiently-between-processes-1bee17dcb01
> multiprocessing.Array is needed to put the numpy array into shared memory and avoid pickling.
>
> I have not tried or investigated myself, but maybe worth a try...
>

Yes I saw it but I didn't try before. I tried last days but I didn't
get any improvements, I will try in the coming days

> Cheers
> Stefan
>

-- 
ciao
Luca

www.lucadelu.org