[GRASS-user] multiprocessing in python

[Leonardo Hardtke]

Dear all,
I am working on a module to extract the phenological parameters (like
timesat) from a time series implemented in python/cython and making use of
gscript and other grass stuff.
It works great on a 256x256 and as the plan is applying it over Australia
at 250m over 17 years, I need to split the process in small tiles. The idea
is to run this processes in parallel and I am having issues implementing
it.

This would be the first part of the process that runs on each tile:

def tile_process(tile_index):
    '''
    Function for every worker:
    Applies any function to the sub_region corresponding to the tile_index.
    '''
    global Rows
    global Cols
    global RowBlockSize
    global ColBlockSize
    global full_region
    global dates
    global years
    global indices
    global data_serie
    global yr_limits_extra
    global yr_limits
    global dbif

    sub_name='block'
    TileRow, TileCol, sr =
sub_region(tile_index,full_region,RowBlockSize,ColBlockSize)
    # # Define a temporary region based on the parameters caluculated with
the
    start_row = TileRow * RowBlockSize
    start_col = TileCol * ColBlockSize
    n_rows = sr['rows']
    n_cols = sr['cols']

    strds = tgis.SpaceTimeRasterDataset(data_serie)
    strds.select(dbif=dbif)
    maps = strds.get_registered_maps_as_objects(dbif=dbif)

    # Numer of time steps
    steps = len(maps)
    # Make an empty array
    #print(steps)
    EVI = np.empty([steps,n_rows,n_cols])
    # fill the array
    for step, map in enumerate(maps):
         map.select(dbif=dbif)
         image_name = map.get_name()+'@'+data_serie.split('@')[1]
         #print("reading: {}".format(image_name))
         EVI[step,:] =
raster2numpy_sub(image_name,start_row,n_rows,start_col,n_cols)
    mean = EVI.mean()
    print(mean)
    ....
    ....
    ....


and this is how I start the multiprocess pool.

    pool.map(tile_process, xrange(RowBlockN*ColBlockN))
    pool.close()
    pool.join()

and it gives me:

AssertionError: can only test a child process


of course if I do: tile_process(0) or tile_process(1) etc ,the right result
comes out.

Does any of you have experience with this? Any suggestion would be welcome!
Sorry for the messy code. Is still in early stage.


-- 
Dr. Leonardo A. Hardtke
C3 UTS, Scientific Officer
CB04.06.315.06
Email:leonardoandres.hardtke at uts.edu.au or leohardtke at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20180206/3725c856/attachment.html>