[Qgis-user] Iterative calls of processing algorithm fails
Gabriel De Luca
pablogabrieldeluca at gmail.com
Fri Oct 11 07:54:26 PDT 2024
Hi,
I'm probably not the right user to answer, because I don't master the
processing framework, but I get the impression that the problem is in its
multithreaded nature.
I think it's likely that you'll access result['OUTPUT'] before it is
returned. I'm also interested in this type of cases, so a response from a
more experienced user would be welcome.
In the meantime, you can try something like:
import threading
lock = threading.Lock()
result = {}
with lock:
try:
result = processing.run("qgis:shortestpathpointtolayer", parameters,
context=context, feedback=feedback, is_child_algorithm=True)
except Exception as e:
raise e # Handle it properly
shortest_path = QgsProcessingContext.takeResultLayer(context,
result['OUTPUT'])
Regards,
Gabriel
El vie, 11 oct 2024 a la(s) 6:01 a.m., Morten Storm via QGIS-User (
qgis-user at lists.osgeo.org) escribió:
> Hi all,
>
> I have written a processing script for QGIS 3.x, which itself calls a
> native
> processing function (qgis:shortestpathpointtolayer) in a for loop.
>
> ---------
> from qgis.PyQt.QtCore import QCoreApplication
> from qgis.core import (QgsProcessing,
> QgsFeatureRequest,
> QgsExpression,
> QgsProcessingContext,
> QgsProcessingException,
> QgsProcessingAlgorithm,
> QgsProcessingParameterVectorLayer,
> QgsProcessingFeatureSourceDefinition)
> from qgis import processing
>
>
> class findDownstreamPathAlgorithm(QgsProcessingAlgorithm):
> """
> This is an algorithm that calculates downstream paths on
> a simplified network.
>
> """
>
> INPUT_NETWORK = 'INPUT_NETWORK'
> INPUT_VALVES = 'INPUT_VALVES'
>
> def tr(self, string):
> return QCoreApplication.translate('Processing', string)
>
> def createInstance(self):
> return findDownstreamPathAlgorithm()
>
> def name(self):
> return 'iterateclusters'
>
> def displayName(self):
> return self.tr('Iterate clusters')
>
> def group(self):
> return self.tr('Thvilum')
>
> def groupId(self):
> return 'lukkelister'
>
> def shortHelpString(self):
> return self.tr("Iterate clusters in a simplified network")
>
> def initAlgorithm(self, config=None):
>
> self.addParameter(
> QgsProcessingParameterVectorLayer(
> self.INPUT_NETWORK,
> self.tr('Netværkslag'),
> [QgsProcessing.SourceType.TypeVectorLine]
> )
> )
>
> self.addParameter(
> QgsProcessingParameterVectorLayer(
> self.INPUT_VALVES,
> self.tr('Ventillag'),
> [QgsProcessing.SourceType.TypeVectorPoint]
> )
> )
>
> def processAlgorithm(self, parameters, context, feedback):
>
> # Retrieve the feature sources
> network = self.parameterAsVectorLayer(
> parameters,
> self.INPUT_NETWORK,
> context
> )
>
> valves = self.parameterAsVectorLayer(
> parameters,
> self.INPUT_VALVES,
> context
> )
>
> if network is None:
> raise
> QgsProcessingException(self.invalidSourceError(parameters,
> self.INPUT_NETWORK))
>
> if valves is None:
> raise
> QgsProcessingException(self.invalidSourceError(parameters,
> self.INPUT_VALVES))
>
> # Store inlets from valves layer in separate scractch layer
> inlets =
> valves.materialize(QgsFeatureRequest(QgsExpression("situation =
> 'IsInlet'")))
>
> # Collect clusters to evaluate
> clusterList = []
> for feat in network.getFeatures():
> clusterList.append(feat["cluster"])
>
> # Remove duplicates and sort list
> clusterList = sorted(set(clusterList))
> if len(clusterList) > 0:
> total = 100.0 / len(clusterList)
> else:
> total = 0
>
> isCanceled = False
> for current, cluster in enumerate(clusterList):
> # Update the progress bar
> feedback.setProgress(int(current * total))
>
> valves.selectByExpression("cluster = {} and situation =
> 'Flow'".format(cluster))
> for feat in valves.selectedFeatures():
> # Stop the algorithm if cancel button has been clicked
> if feedback.isCanceled():
> isCanceled = True
> break
>
> vertex_id = feat["vertexId"]
> valve_geom = feat.geometry()
> network.selectByExpression("cluster != {}".format(cluster))
>
> parameters = {
> 'DEFAULT_DIRECTION' : 2,
> 'DEFAULT_SPEED' : 50,
> 'DIRECTION_FIELD' : '',
> 'END_POINTS' : inlets,
> 'INPUT' : QgsProcessingFeatureSourceDefinition(
> network.id(), selectedFeaturesOnly=True),
> 'OUTPUT' : 'TEMPORARY_OUTPUT',
> #'OUTPUT_NON_ROUTABLE' : 'TEMPORARY_OUTPUT',
> 'SPEED_FIELD' : '',
> 'START_POINT' : valve_geom,
> 'STRATEGY' : 0,
> 'TOLERANCE' : 0,
> 'VALUE_BACKWARD' : '',
> 'VALUE_BOTH' : '',
> 'VALUE_FORWARD' : ''
> }
>
> result = processing.run("qgis:shortestpathpointtolayer",
> parameters, context=context, feedback=feedback, is_child_algorithm=True)
> shortest_path =
> QgsProcessingContext.takeResultLayer(context, result['OUTPUT'])
>
> # Do something with the path
>
> del shortest_path
>
> if isCanceled:
> break
>
>
> return {self.INPUT_NETWORK: network}
> ---------
>
> When running this script on QGIS 3.28.4 it runs a couple of iterations and
> then raises an exception (access violation) with the following description:
>
> **Python Stack Trace**
> ```
> Windows fatal exception: access violation
>
> Thread 0x00005d8c (most recent call first):
> File
>
> "C:\PROGRA~1/QGIS32~2.4/apps/qgis-ltr/./python/plugins\processing\gui\Algori
> thmExecutor.py", line 72 in execute
> results, ok = alg.run(parameters, context, feedback, {}, False)
> File
>
> "C:\PROGRA~1/QGIS32~2.4/apps/qgis-ltr/./python/plugins\processing\core\Proce
> ssing.py", line 187 in runAlgorithm
> ret, results = execute(alg, parameters, context, feedback,
> catch_exceptions=False)
> File
>
> "C:\PROGRA~1/QGIS32~2.4/apps/qgis-ltr/./python/plugins\processing\tools\gene
> ral.py", line 116 in run
> return Processing.runAlgorithm(algOrName, parameters,
> onFinish=post_process, feedback=feedback, context=context)
> File
>
> "C:\Users\morte\AppData\Roaming\QGIS\QGIS3\profiles\default\processing\scrip
> ts\iterateClusters.py", line 156 in processAlgorithm
> result = processing.run("qgis:shortestpathpointtolayer", parameters,
> context=context, feedback=feedback, is_child_algorithm=True)
>
> Current thread 0x000075e0 (most recent call first):
> File
>
> "C:\PROGRA~1/QGIS32~2.4/apps/qgis-ltr/./python/plugins\processing\Processing
> Plugin.py", line 394 in executeAlgorithm
> dlg.exec_()
> File
>
> "C:\PROGRA~1/QGIS32~2.4/apps/qgis-ltr/./python/plugins\processing\gui\Proces
> singToolbox.py", line 234 in executeAlgorithm
> self.executeWithGui.emit(alg.id(), self, self.in_place_mode, False)
> ```
>
> On QGIS 3.38.3 the process also runs for a couple of iteration and then
> just
> freezes the whole application.
>
> Running a simple (=non processing) version of this script from the Python
> console works beautifully on both QGIS versions!
>
> What am I missing in my processing script?
>
>
>
> Best regards,
> Morten
>
> --
> Morten Storm
> Styrke 10 ApS
>
> House of Innovation | Jernbanegade 27 | 6000 Kolding | Denmark
>
> www: https://styrke10.dk
> mobile: +45 51 51 09 29
>
>
> _______________________________________________
> QGIS-User mailing list
> QGIS-User at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20241011/744b6f82/attachment.htm>
More information about the QGIS-User
mailing list