[QGIS-Developer] Memory Leak / Slowdown?

Mon Apr 20 08:54:15 PDT 2020

Hi Matthias,
they both do not work…
So I will make a Ticket I think…
Thanks anyway Matthias for your great support!

Best greetings
Peter

Von: Matthias <matthias at opengis.ch>
Gesendet: Montag, 20. April 2020 17:21
An: Drexel Peter <Peter.Drexel at vorarlberg.at>; qgis-user at lists.osgeo.org; qgis-developer at lists.osgeo.org
Betreff: Re: AW: AW: [QGIS-Developer] Memory Leak / Slowdown?

Hi Peter,

Happy you found a way.

Can you try proposals number 2 and 3 from my previous email? If they do not help, please open a ticket.

Bests
Matthias

20 April 2020 17:14 Drexel Peter <Peter.Drexel at vorarlberg.at<mailto:Peter.Drexel at vorarlberg.at>> wrote:
Hi Matthias,
Thanks to your hints I found a way:
The script also works if I do not add the Layer to the Project but directly save the Sld-Style. So I can also skip clearing the QgsProject.instance.

vlayer=QgsVectorLayer (pfad ,sfileOhneExt,"ogr")
#QgsProject.instance().addMapLayer(vlayer)
vlayer.saveSldStyle (sldpfad)
#QgsProject.instance().clear()

But besides my personal “success”: including the 2 Lines should not lead to rising time /memory consumption or am I wrong?
Should I make a ticket?

Peter

Von: Matthias <matthias at opengis.ch<mailto:matthias at opengis.ch>>
Gesendet: Montag, 20. April 2020 15:40
An: Drexel Peter <Peter.Drexel at vorarlberg.at<mailto:Peter.Drexel at vorarlberg.at>>; qgis-user at lists.osgeo.org<mailto:qgis-user at lists.osgeo.org>; qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>
Betreff: Re: AW: [QGIS-Developer] Memory Leak / Slowdown?

Hi Peter,

It's possible that a part of the cleanup only happens on the next event loop run.

Questions / things to try:

- Do you really need to add them to the project or is a layer without adding it to the project good enough for your purpose?
- Can you run this in batches of ~100 (or even 1) each and then trigger the next batch on a new event loop invocation, e.g. by using QTimer.singleShot()
- The easiest thing would be calling QgsApplication.processEvents(), but be warned that using this is known for introducing instabilities, depending on the context in which it is called

Matthias

20 April 2020 15:11 Drexel Peter <Peter.Drexel at vorarlberg.at<mailto:Peter.Drexel at vorarlberg.at>> wrote:
Hi Matthias,
yes, the same with a clean new profile.
4 iterations per second at the begining, 1 very 6 seconds at the end…

<QgsMapLayer: 'nix 1 13:08:41' (ogr)>
<QgsMapLayer: 'nix 2 13:08:41' (ogr)>
<QgsMapLayer: 'nix 3 13:08:41' (ogr)>
<QgsMapLayer: 'nix 4 13:08:42' (ogr)>
<QgsMapLayer: 'nix 5 13:08:42' (ogr)>
<QgsMapLayer: 'nix 6 13:08:42' (ogr)>
<QgsMapLayer: 'nix 7 13:08:42' (ogr)>
<QgsMapLayer: 'nix 8 13:08:43' (ogr)>
…
<QgsMapLayer: 'nix 397 13:25:38' (ogr)>
<QgsMapLayer: 'nix 398 13:25:45' (ogr)>
<QgsMapLayer: 'nix 399 13:25:51' (ogr)>

QGIS-Version

3.12.1-București

QGIS-Codeversion

121cc00ff0<https://github.com/qgis/QGIS/commit/121cc00ff0>

Kompiliert gegen Qt

5.11.2

Laufendes Qt

5.11.2

Kompiliert mit GDAL/OGR

3.0.4

Läuft mit GDAL/OGR

3.0.4

Kompiliert mit GEOS

3.8.1-CAPI-1.13.3

Läuft mit GEOS

3.8.1-CAPI-1.13.3

Kompiliert mit SQLite

3.29.0

Läuft mit SQLite

3.29.0

PostgreSQL-Client-Version

11.5

SpatiaLite-Version

4.3.0

QWT-Version

6.1.3

QScintilla2-Version

2.10.8

Kompiliert mit PROJ

6.3.1

Läuft mit PROJ

Rel. 6.3.1, February 10th, 2020

BS-Version

Windows 10 (10.0)

Aktive Python-Erweiterungen

db_manager;
MetaSearch;
processing

Von: Matthias Kuhn <matthias at opengis.ch<mailto:matthias at opengis.ch>>
Gesendet: Montag, 20. April 2020 12:50
An: Drexel Peter <Peter.Drexel at vorarlberg.at<mailto:Peter.Drexel at vorarlberg.at>>; qgis-user at lists.osgeo.org<mailto:qgis-user at lists.osgeo.org>
Betreff: Re: [QGIS-Developer] Memory Leak / Slowdown?

Hi Peter,

Did you try this on a clean new profile with no plugins installed?

Bests

Matthias
On 4/20/20 11:08 AM, Drexel Peter wrote:
Hi,
I use a Python-Script to load all shapefiles in a directory tree one by one and export the sld-files.
It works fine, but slows down incredible after ~100 files.
So I tried the following barebone script with just the essentials  (load shapefile, add it to the map canvas, remove it again and then clear the project) in a loop and I get the same behavior…

from datetime import datetime
for i in range(1, 1000):
                vlayer=QgsVectorLayer ("aShapeFile.shp" ,"run  " + str(i) + " at " + datetime.now().strftime('%H:%M:%S') ,"ogr")
                QgsProject.instance().addMapLayer(vlayer)
                QgsProject.instance().removeMapLayer(vlayer.id())
                QgsProject.instance().clear()

Same result,  the Script slows down after every iteration in the loop AND QGIS-memory consumption goes up!
Do I need to do some garbage cleaning / what am I doing wrong / is this a Memory leak?

Thanks everybody for any hints / help!

Peter

_______________________________________________ QGIS-Developer mailing listQGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20200420/998e0b79/attachment-0001.html>