[QGIS-Developer] Running grass algorithms in threads
Rudi von Staden
rudivs at gmail.com
Tue Aug 14 04:42:25 PDT 2018
Hi all,
The bottleneck in my script at the moment is the calculation of zonal stats
using 'grass7:r.stats.zonal'. I thought I might speed things up by using
QgsTask.fromFunction() or QgsProcessingAlgRunnerTask() to run these
calculations in parallel. In my tests of both approaches the tasks seem to
complete (task.status() == QgsTask.Complete), but the output file is only
generated for 1 of 4 parallel tasks (the task that finishes first).
I'm assuming this is because grass algorithms are not thread safe? Or am I
missing something in my implementation that could make this work?
Thanks,
Rudi
My code for the QgsTask approach is as below:
def getZonal(task, habitatModelFile, cover):
tempFile = QgsProcessingUtils.generateTempFilename("output.tif")
processing.run("grass7:r.stats.zonal", {
'base':habitatModelFile,
'cover':cover,
'method':5,
'-c':False,
'-r':False,
'output':tempFile,
'GRASS_REGION_PARAMETER':None,
'GRASS_REGION_CELLSIZE_PARAMETER':0,
'GRASS_RASTER_FORMAT_OPT':'',
'GRASS_RASTER_FORMAT_META':''},context=context,feedback=algFeedback)
if task.isCanceled():
deleteFile(tempFile)
return
return tempFile
ls90Task = QgsTask.fromFunction('LS90', getZonal, habitatModelFile=hm1,
cover=ls90Layer)
QgsApplication.taskManager().addTask(ls90Task)
feedback.pushInfo("Calculating LS14 mean...")
ls14Task = QgsTask.fromFunction('LS14 ', getZonal, habitatModelFile=hm2,
cover=ls14Layer)
QgsApplication.taskManager().addTask(ls14Task)
hs90Task = QgsTask.fromFunction('HS90 ', getZonal, habitatModelFile=hm3,
cover=hs90Layer)
QgsApplication.taskManager().addTask(hs90Task)
hs14Task = QgsTask.fromFunction('HS14 ', getZonal, habitatModelFile=hm4,
cover=hs14Layer)
QgsApplication.taskManager().addTask(hs14Task)
while (len([t for t in [ls90Task.status(), ls14Task.status(),
hs90Task.status(),
hs14Task.status()] if t in [QgsTask.Running, QgsTask.Queued]])
> 0)
and not feedback.isCanceled():
sleep(1)
if feedback.isCanceled():
# some cleanup code (send task.cancel() and wait for tasks to terminate)
break
ls90Result = ls90Task.returned_values
ls14Result = ls14Task.returned_values
hs90Result = hs90Task.returned_values # only this file exists
hs14Result = hs14Task.returned_values
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20180814/533413c6/attachment.html>
More information about the QGIS-Developer
mailing list