[GRASS-user] Parallel proccess with pygrass and mapcalc

[Pietro]

Ciao Lorenzo,

On Wed, Mar 2, 2016 at 6:10 PM, Lorenzo Bottaccioli
<lorenzo.bottaccioli at gmail.com> wrote:
> I have written this code from the example provided in
> (https://grass.osgeo.org/grass70/manuals/libpython/pygrass.modules.interface.html?highlight=parallelmodulequeue#pygrass.modules.interface.module.ParallelModuleQueue)
> but there is no difference in terms of time between setting nproces=1 or
> nproces=8. How is possible? How can I achieve improvements?

Actually I can not reproduce your issue on my system I obtain:

--- 2.03981089592 seconds with 1 process ---
--- 1.10217881203 seconds with 4 process ---

here the code that I used:

{{{
import copy
import os
import time

import grass.script.setup as gsetup
from grass.pygrass.modules import Module, ParallelModuleQueue


def test_parallell(rinput, routput, ncalcs, nprocs=1):
    start_time = time.time()
    mapcalc_list = []
    mapcalc = Module("r.mapcalc", overwrite=True, run_=False)
    queue = ParallelModuleQueue(nprocs)
    for n in range(ncalcs):
        new_mapcalc = copy.deepcopy(mapcalc)
        mapcalc_list.append(new_mapcalc)
        m = new_mapcalc(expression="{o}_{n} = {i} * {n}".format(i=rinput,
                                                                o=routput,
                                                                n=n))
        queue.put(m)
    queue.wait()
    print("--- %s seconds with %d process ---" % (time.time() - start_time,
                                                  nprocs))


gsetup.init(os.environ['GISBASE'],
            os.path.abspath("/home/pietro/docdat/gis"),
            'nc_basic_spm_grass7', 'user1')


test_parallell('elevation', 'tmptest', 16, nprocs=1)
test_parallell('elevation', 'tmptest', 16, nprocs=4)
Module('g.remove', type='raster', pattern='tmptest' + '*', flags='f')

}}}

Please next time try to use the north Carolina mapset, make your
problem simpler, remove all the unnecessary parts and reduce the code
to the essential.

Hope it helps.

Happy hacking!

Pietro