[QGIS-Developer] Custom processing script instabilities (Nzl_T)
Nazli Turini [BBA]
nazli.turini at broadband-academy.de
Fri Jun 25 03:09:02 PDT 2021
Hi Evereybody,
I am facing the same issue.
I am creating a custom processing algorithm in which it loads 1, the google satellite 2. get my study_region.shp file 3. loop through the features in study_region.shp 4. and for each feature create the mbtiles using processing.run('qgis:tilesxyzmbtiles'...)5. Save the result on the predefined folder.
The script runs when the ZOOM_MAX and ZOOM_MIN are 12. When I change the ZOOM_MAX to 19, the processing window vanished and qgis crashed. Maybe it gets some hint to @ Alexis
Here is my code
import os
from os import makedirs
from qgis.PyQt.QtCore import QCoreApplication
from PyQt5.QtGui import QColor
from qgis.core import (QgsProcessing,
QgsFeatureSink,
QgsProcessingException,
QgsProcessingAlgorithm,
QgsProcessingParameterFeatureSource,
QgsProcessingParameterFeatureSink,QgsRasterLayer,QgsAbstractPropertyCollection,QgsProject,QgsLayerTreeGroup, QgsLayerTreeLayer, QgsLayerTree)
from qgis import processing
from qgis.core import QgsProcessingFeedback
from qgis.core import (QgsProcessingParameterFile,
QgsProcessingParameterString,
)
from qgis.utils import iface
class crea_mbtiles_MERGIN(QgsProcessingAlgorithm):
# Constants used to refer to parameters and outputs. They will be
# used when calling the algorithm from another algorithm, or when
# calling from the QGIS console.
def tr(self, string):
"""
Returns a translatable string with the self.tr() function.
"""
return QCoreApplication.translate('Processing', string)
def createInstance(self):
# Must return a new copy of your algorithm.
return crea_mbtiles_MERGIN()
def name(self):
"""
Returns the unique algorithm name.
"""
return 'Create Mbtiles'
def displayName(self):
"""
Returns the translated algorithm name.
"""
return self.tr('Create Mbtiles')
def flags(self):
return super().flags() | QgsProcessingAlgorithm.FlagNoThreading
Ausbaugebiet = 'Ausbaugebiet'
Projektnummer= 'Projekt-Nummer'
OUT_FOLDER = 'OUT_FOLDER' # abändern für andere Dateien
def initAlgorithm(self, config=None):
self.addParameter(
QgsProcessingParameterFeatureSource(
self.Ausbaugebiet,
'Ausbaugebiet',
[QgsProcessing.TypeVectorAnyGeometry]
)
)
self.addParameter(
QgsProcessingParameterString(
self.Projektnummer,
'Projekt-Nummer',
optional=False
)
)
self.addParameter(
QgsProcessingParameterFile(
self.OUT_FOLDER,
f'Zentralen MERGIN Ordner',
behavior=QgsProcessingParameterFile.Folder,
defaultValue='V:\99_Planungsdaten\Mergin',
optional=False
)
)
def processAlgorithm(self, parameters, context, feedback):
results = {}
Ausbaugebiet = self.parameterAsSource(
parameters,
self.Ausbaugebiet,
context
)
out_folder: str = self.parameterAsString(
parameters,
self.OUT_FOLDER,
context
)
Projektnummer = self.parameterAsString(
parameters,
self.Projektnummer,
context
)
#if Projektnummer.find(SEPARATOR) > 0:
output_path = os.path.join(out_folder, Projektnummer)
#Projektnummer = path.basename(Projektnummer)
# Create folder if not present
if not os.path.exists(output_path):
makedirs(output_path)
#Load the google xyz data
urlWithParams = 'type=xyz&url=https://mt1.google.com/vt/lyrs%3Ds%26x%3D%7Bx%7D%26y%3D%7By%7D%26z%3D%7Bz%7D&zmax=19&zmin=0'
rlayer = QgsRasterLayer(urlWithParams, 'Google', 'wms')
QgsProject.instance().addMapLayer(rlayer, False)
layerTree = iface.layerTreeCanvasBridge().rootGroup()
layerTree.insertChildNode(0, QgsLayerTreeLayer(rlayer))
if feedback.isCanceled():
return {}
input_featuresource = self.parameterAsSource(parameters, 'Ausbaugebiet', context)
for feature in input_featuresource.getFeatures():
attrs = feature.attributes()
name=attrs[1]
bbox = feature.geometry().boundingBox()
bbox_extent = '%f,%f,%f,%f' % (bbox.xMinimum(), bbox.xMaximum(), bbox.yMinimum(), bbox.yMaximum())
alg_params = {
'BACKGROUND_COLOR': QColor(0, 0, 0, 0),
'DPI': 300,
'EXTENT': bbox_extent,
'METATILESIZE': 6,
'QUALITY': 75,
'TILE_FORMAT': 0,
'ZOOM_MAX': 12,
'ZOOM_MIN': 12,
'OUTPUT_FILE': output_path+'/googlestallite_%s.mbtiles' %(name)
}
google=processing.run('qgis:tilesxyzmbtiles', alg_params,is_child_algorithm=True,context=context, feedback=feedback)['OUTPUT_FILE']
# Check for cancelation
if feedback.isCanceled():
return {}
#Remove the layer
QgsProject.instance().removeMapLayers([rlayer.id()])
del rlayer
return result
Thank you so much and best regards,
Nazli
-----Ursprüngliche Nachricht-----
Von: QGIS-Developer <qgis-developer-bounces at lists.osgeo.org> Im Auftrag von qgis-developer-request at lists.osgeo.org
Gesendet: Freitag, 25. Juni 2021 09:15
An: qgis-developer at lists.osgeo.org
Betreff: QGIS-Developer Digest, Vol 188, Issue 36
Send QGIS-Developer mailing list submissions to
qgis-developer at lists.osgeo.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.osgeo.org/mailman/listinfo/qgis-developer
or, via email, send a message with subject or body 'help' to
qgis-developer-request at lists.osgeo.org
You can reach the person managing the list at
qgis-developer-owner at lists.osgeo.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of QGIS-Developer digest..."
Today's Topics:
1. Custom processing script instabilities (Alexis R.L.)
2. Re: Custom processing script instabilities (Luigi Pirelli)
3. Re: Custom processing script instabilities (Alexis R.L.)
4. Re: Custom processing script instabilities (Luigi Pirelli)
----------------------------------------------------------------------
Message: 1
Date: Thu, 24 Jun 2021 15:50:54 -0400
From: "Alexis R.L." <alroyliz0 at gmail.com>
To: qgis-developer <qgis-developer at lists.osgeo.org>
Subject: [QGIS-Developer] Custom processing script instabilities
Message-ID:
<CAOM0XWDiy-WUXQrL4fEFXKxUyzBb-K9Mix3kBTRwK4nnRAhHww at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Greetings,
I have one custom script that uses PIL to get the exif tag, and when I execute the script for the first time, the processing window vanishes and qgis stall and then crashes.
To get it to run properly I had to set the return before opening the files with PIL to run a neutered version and then subsequent runs were flawless during that qgis instance.
I ended up using a standalone exif reader, though slower , but there are no more crashes.
Has anyone else had similar issues and is anyone aware of a way to prevent such crashes or handle them more gracefully, as no trace is left currently.
Thanks and have a nice day,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20210624/d49ff6b4/attachment-0001.html>
------------------------------
Message: 2
Date: Thu, 24 Jun 2021 22:34:01 +0200
From: Luigi Pirelli <luipir at gmail.com>
To: "Alexis R.L." <alroyliz0 at gmail.com>
Cc: qgis-developer <qgis-developer at lists.osgeo.org>
Subject: Re: [QGIS-Developer] Custom processing script instabilities
Message-ID:
<CAFO80_qOK0pBn1KajrPN0paAF=+SrsObeLs=QGqoY_vs=Oq3XQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
hard to say something without reading the code... btw generally you do not have to interact with main GUi from a processing alg that by default run in a separated thread => this can cause crash if you do not use standard processing libs (e.g. feedback) to send message to the user.
No idea if this could be the source of the problem., IMHO sharing the code would help and also help to verify if it can be replicable.
regards
Luigi Pirelli
**************************************************************************************************
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Book: Mastering QGIS3 - 3rd Edition
<https://www.packtpub.com/eu/application-development/mastering-geospatial-development-qgis-3x-third-edition>
* Hire a team: http://www.qcooperative.net
**************************************************************************************************
On Thu, 24 Jun 2021 at 21:51, Alexis R.L. <alroyliz0 at gmail.com> wrote:
> Greetings,
>
> I have one custom script that uses PIL to get the exif tag, and when I
> execute the script for the first time, the processing window vanishes
> and qgis stall and then crashes.
>
> To get it to run properly I had to set the return before opening the
> files with PIL to run a neutered version and then subsequent runs were
> flawless during that qgis instance.
>
> I ended up using a standalone exif reader, though slower , but there
> are no more crashes.
>
> Has anyone else had similar issues and is anyone aware of a way to
> prevent such crashes or handle them more gracefully, as no trace is left currently.
>
> Thanks and have a nice day,
>
> Alex
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20210624/084a8256/attachment-0001.html>
------------------------------
Message: 3
Date: Thu, 24 Jun 2021 17:05:09 -0400
From: "Alexis R.L." <alroyliz0 at gmail.com>
To: Luigi Pirelli <luipir at gmail.com>
Cc: qgis-developer <qgis-developer at lists.osgeo.org>
Subject: Re: [QGIS-Developer] Custom processing script instabilities
Message-ID:
<CAOM0XWB22aCNvEZ8O04yi-VATkjotqN_rHJ26rBpGC87Asv7BA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Greetings Luigi,
There were no interaction, I used glob to gather all the images in a folder, used Image.Open and then used _getexif(). But returning before this part was the only way to get a stable first run.
Sadly I can share the code (work and IP).
The rest of it looped through datapoints and correlated the images to the points using the exif timestamp and the geolocalisation.
Alex
Le jeu. 24 juin 2021 ? 16:34, Luigi Pirelli <luipir at gmail.com> a ?crit :
> hard to say something without reading the code... btw generally you do
> not have to interact with main GUi from a processing alg that by
> default run in a separated thread => this can cause crash if you do
> not use standard processing libs (e.g. feedback) to send message to the user.
> No idea if this could be the source of the problem., IMHO sharing the
> code would help and also help to verify if it can be replicable.
>
> regards
>
> Luigi Pirelli
>
>
> **********************************************************************
> ****************************
> * LinkedIn: https://www.linkedin.com/in/luigipirelli
> * Stackexchange:
> http://gis.stackexchange.com/users/19667/luigi-pirelli
> * GitHub: https://github.com/luipir
> * Book: Mastering QGIS3 - 3rd Edition
> <https://www.packtpub.com/eu/application-development/mastering-geospat
> ial-development-qgis-3x-third-edition>
> * Hire a team: http://www.qcooperative.net
>
> **********************************************************************
> ****************************
>
>
> On Thu, 24 Jun 2021 at 21:51, Alexis R.L. <alroyliz0 at gmail.com> wrote:
>
>> Greetings,
>>
>> I have one custom script that uses PIL to get the exif tag, and when
>> I execute the script for the first time, the processing window
>> vanishes and qgis stall and then crashes.
>>
>> To get it to run properly I had to set the return before opening the
>> files with PIL to run a neutered version and then subsequent runs
>> were flawless during that qgis instance.
>>
>> I ended up using a standalone exif reader, though slower , but there
>> are no more crashes.
>>
>> Has anyone else had similar issues and is anyone aware of a way to
>> prevent such crashes or handle them more gracefully, as no trace is
>> left currently.
>>
>> Thanks and have a nice day,
>>
>> Alex
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20210624/415ff84b/attachment-0001.html>
------------------------------
Message: 4
Date: Fri, 25 Jun 2021 09:14:43 +0200
From: Luigi Pirelli <luipir at gmail.com>
To: "Alexis R.L." <alroyliz0 at gmail.com>
Cc: qgis-developer <qgis-developer at lists.osgeo.org>
Subject: Re: [QGIS-Developer] Custom processing script instabilities
Message-ID:
<CAFO80_oh3NPcZE9oOaoB+0+FfhGLrKU74t71+PsYGXCw=736dw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
strange that a library call exit, so I would inspect if the lib faces a raise condition. I would check the code outside qgis checking if it terminates for some reason.
Luigi Pirelli
**************************************************************************************************
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Book: Mastering QGIS3 - 3rd Edition
<https://www.packtpub.com/eu/application-development/mastering-geospatial-development-qgis-3x-third-edition>
* Hire a team: http://www.qcooperative.net
**************************************************************************************************
On Thu, 24 Jun 2021 at 23:05, Alexis R.L. <alroyliz0 at gmail.com> wrote:
> Greetings Luigi,
>
> There were no interaction, I used glob to gather all the images in a
> folder, used Image.Open and then used _getexif(). But returning before
> this part was the only way to get a stable first run.
>
> Sadly I can share the code (work and IP).
>
> The rest of it looped through datapoints and correlated the images to
> the points using the exif timestamp and the geolocalisation.
>
> Alex
>
> Le jeu. 24 juin 2021 ? 16:34, Luigi Pirelli <luipir at gmail.com> a ?crit :
>
>> hard to say something without reading the code... btw generally you
>> do not have to interact with main GUi from a processing alg that by
>> default run in a separated thread => this can cause crash if you do
>> not use standard processing libs (e.g. feedback) to send message to the user.
>> No idea if this could be the source of the problem., IMHO sharing the
>> code would help and also help to verify if it can be replicable.
>>
>> regards
>>
>> Luigi Pirelli
>>
>>
>> *********************************************************************
>> *****************************
>> * LinkedIn: https://www.linkedin.com/in/luigipirelli
>> * Stackexchange:
>> http://gis.stackexchange.com/users/19667/luigi-pirelli
>> * GitHub: https://github.com/luipir
>> * Book: Mastering QGIS3 - 3rd Edition
>> <https://www.packtpub.com/eu/application-development/mastering-geospa
>> tial-development-qgis-3x-third-edition>
>> * Hire a team: http://www.qcooperative.net
>>
>> *********************************************************************
>> *****************************
>>
>>
>> On Thu, 24 Jun 2021 at 21:51, Alexis R.L. <alroyliz0 at gmail.com> wrote:
>>
>>> Greetings,
>>>
>>> I have one custom script that uses PIL to get the exif tag, and when
>>> I execute the script for the first time, the processing window
>>> vanishes and qgis stall and then crashes.
>>>
>>> To get it to run properly I had to set the return before opening the
>>> files with PIL to run a neutered version and then subsequent runs
>>> were flawless during that qgis instance.
>>>
>>> I ended up using a standalone exif reader, though slower , but there
>>> are no more crashes.
>>>
>>> Has anyone else had similar issues and is anyone aware of a way to
>>> prevent such crashes or handle them more gracefully, as no trace is
>>> left currently.
>>>
>>> Thanks and have a nice day,
>>>
>>> Alex
>>> _______________________________________________
>>> QGIS-Developer mailing list
>>> QGIS-Developer at lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20210625/fcd0affd/attachment.html>
------------------------------
Subject: Digest Footer
_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
------------------------------
End of QGIS-Developer Digest, Vol 188, Issue 36
***********************************************
More information about the QGIS-Developer
mailing list