[QGIS-Developer] Memory Leak / Slowdown?

[Drexel Peter]

Hi Matthias,
Thanks to your hints I found a way:
The script also works if I do not add the Layer to the Project but directly save the Sld-Style. So I can also skip clearing the QgsProject.instance.

vlayer=QgsVectorLayer (pfad ,sfileOhneExt,"ogr")
#QgsProject.instance().addMapLayer(vlayer)
vlayer.saveSldStyle (sldpfad)
#QgsProject.instance().clear()

But besides my personal “success”: including the 2 Lines should not lead to rising time /memory consumption or am I wrong?
Should I make a ticket?

Peter

Von: Matthias <matthias at opengis.ch>
Gesendet: Montag, 20. April 2020 15:40
An: Drexel Peter <Peter.Drexel at vorarlberg.at>; qgis-user at lists.osgeo.org; qgis-developer at lists.osgeo.org
Betreff: Re: AW: [QGIS-Developer] Memory Leak / Slowdown?

Hi Peter,

It's possible that a part of the cleanup only happens on the next event loop run.

Questions / things to try:

- Do you really need to add them to the project or is a layer without adding it to the project good enough for your purpose?
- Can you run this in batches of ~100 (or even 1) each and then trigger the next batch on a new event loop invocation, e.g. by using QTimer.singleShot()
- The easiest thing would be calling QgsApplication.processEvents(), but be warned that using this is known for introducing instabilities, depending on the context in which it is called

Matthias



20 April 2020 15:11 Drexel Peter <Peter.Drexel at vorarlberg.at<mailto:Peter.Drexel at vorarlberg.at>> wrote:
Hi Matthias,
yes, the same with a clean new profile.
4 iterations per second at the begining, 1 very 6 seconds at the end…

<QgsMapLayer: 'nix 1 13:08:41' (ogr)>
<QgsMapLayer: 'nix 2 13:08:41' (ogr)>
<QgsMapLayer: 'nix 3 13:08:41' (ogr)>
<QgsMapLayer: 'nix 4 13:08:42' (ogr)>
<QgsMapLayer: 'nix 5 13:08:42' (ogr)>
<QgsMapLayer: 'nix 6 13:08:42' (ogr)>
<QgsMapLayer: 'nix 7 13:08:42' (ogr)>
<QgsMapLayer: 'nix 8 13:08:43' (ogr)>
…
<QgsMapLayer: 'nix 397 13:25:38' (ogr)>
<QgsMapLayer: 'nix 398 13:25:45' (ogr)>
<QgsMapLayer: 'nix 399 13:25:51' (ogr)>






QGIS-Version


3.12.1-București


QGIS-Codeversion


121cc00ff0<https://github.com/qgis/QGIS/commit/121cc00ff0>


Kompiliert gegen Qt


5.11.2


Laufendes Qt


5.11.2


Kompiliert mit GDAL/OGR


3.0.4


Läuft mit GDAL/OGR


3.0.4


Kompiliert mit GEOS


3.8.1-CAPI-1.13.3


Läuft mit GEOS


3.8.1-CAPI-1.13.3


Kompiliert mit SQLite


3.29.0


Läuft mit SQLite


3.29.0


PostgreSQL-Client-Version


11.5


SpatiaLite-Version


4.3.0


QWT-Version


6.1.3


QScintilla2-Version


2.10.8


Kompiliert mit PROJ


6.3.1


Läuft mit PROJ


Rel. 6.3.1, February 10th, 2020


BS-Version


Windows 10 (10.0)


Aktive Python-Erweiterungen


db_manager;
MetaSearch;
processing





Von: Matthias Kuhn <matthias at opengis.ch<mailto:matthias at opengis.ch>>
Gesendet: Montag, 20. April 2020 12:50
An: Drexel Peter <Peter.Drexel at vorarlberg.at<mailto:Peter.Drexel at vorarlberg.at>>; qgis-user at lists.osgeo.org<mailto:qgis-user at lists.osgeo.org>
Betreff: Re: [QGIS-Developer] Memory Leak / Slowdown?


Hi Peter,

Did you try this on a clean new profile with no plugins installed?

Bests

Matthias
On 4/20/20 11:08 AM, Drexel Peter wrote:
Hi,
I use a Python-Script to load all shapefiles in a directory tree one by one and export the sld-files.
It works fine, but slows down incredible after ~100 files.
So I tried the following barebone script with just the essentials  (load shapefile, add it to the map canvas, remove it again and then clear the project) in a loop and I get the same behavior…

from datetime import datetime
for i in range(1, 1000):
                vlayer=QgsVectorLayer ("aShapeFile.shp" ,"run  " + str(i) + " at " + datetime.now().strftime('%H:%M:%S') ,"ogr")
                QgsProject.instance().addMapLayer(vlayer)
                QgsProject.instance().removeMapLayer(vlayer.id())
                QgsProject.instance().clear()

Same result,  the Script slows down after every iteration in the loop AND QGIS-memory consumption goes up!
Do I need to do some garbage cleaning / what am I doing wrong / is this a Memory leak?

Thanks everybody for any hints / help!

Peter


_______________________________________________ QGIS-Developer mailing listQGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20200420/431d73b2/attachment-0001.html>