<div>Hi Peter,<br></div><div><br></div><div>It's possible that a part of the cleanup only happens on the next event loop run.<br></div><div><br></div><div>Questions / things to try:<br></div><div><br></div><div>- Do you really need to add them to the project or is a layer without adding it to the project good enough for your purpose?<br></div><div>- Can you run this in batches of ~100 (or even 1) each and then trigger the next batch on a new event loop invocation, e.g. by using QTimer.singleShot()<br></div><div>- The easiest thing would be calling QgsApplication.processEvents(), but be warned that using this is known for introducing instabilities, depending on the context in which it is called<br></div><div><br></div><div>Matthias<br></div><div id="editorUserSignature" style="display:none;"><br></div><div id="editorUserSignature" style="display:none;"><br></div><div><br></div><div><i>20 April 2020 15:11 Drexel Peter <Peter.Drexel@vorarlberg.at> wrote:</i><br></div><blockquote class="infmailquote"><div class="WordSection1"><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)">Hi Matthias,</span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)">yes, the same with a clean new profile.</span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)">4 iterations per second at the begining, 1 very 6 seconds at the end…</span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"> </span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 1 13:08:41' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 2 13:08:41' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 3 13:08:41' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 4 13:08:42' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 5 13:08:42' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 6 13:08:42' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 7 13:08:42' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 8 13:08:43' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)">…</span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 397 13:25:38' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 398 13:25:45' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"><QgsMapLayer: 'nix 399 13:25:51' (ogr)></span><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"> </span><br></p><div><div align="center"><table class="MsoNormalTable" style="width:100.0%" width="100%" cellpadding="0" border="0"><tbody><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt"> </span></span><br></p><p style="margin:0cm;margin-bottom:.0001pt"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt"> </span></span><br></p><p style="margin:0cm;margin-bottom:.0001pt"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">QGIS-Version</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">3.12.1-București</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">QGIS-Codeversion</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt"><a href="https://github.com/qgis/QGIS/commit/121cc00ff0">121cc00ff0</a></span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Kompiliert gegen Qt</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">5.11.2</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Laufendes Qt</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">5.11.2</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Kompiliert mit GDAL/OGR</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">3.0.4</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Läuft mit GDAL/OGR</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">3.0.4</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Kompiliert mit GEOS</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">3.8.1-CAPI-1.13.3</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Läuft mit GEOS</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">3.8.1-CAPI-1.13.3</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Kompiliert mit SQLite</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">3.29.0</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Läuft mit SQLite</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">3.29.0</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">PostgreSQL-Client-Version</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">11.5</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">SpatiaLite-Version</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">4.3.0</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">QWT-Version</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">6.1.3</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">QScintilla2-Version</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">2.10.8</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Kompiliert mit PROJ</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">6.3.1</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Läuft mit PROJ</span></span><br></p></td><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Rel. 6.3.1, February 10th, 2020</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">BS-Version</span></span><br></p></td><td colspan="3" style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Windows 10 (10.0)</span></span><br></p></td></tr><tr><td style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">Aktive Python-Erweiterungen</span></span><br></p></td><td colspan="3" style="padding:0cm 0cm 0cm 0cm"><p style="margin:0cm;margin-bottom:.0001pt;-qt-block-indent:0"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt">db_manager;<br> MetaSearch;<br> processing</span></span></p></td></tr></tbody></table></div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:"Courier New""><span class="size" style="font-size:8pt"> </span></span></span></b><br></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span class="colour" style="color:rgb(31, 73, 125)"> </span></b><br></p><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"> </span><br></p></div><p class="MsoNormal"><span class="colour" style="color:rgb(31, 73, 125)"> </span><br></p><div><div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span style="mso-fareast-language:DE-AT" lang="DE">Von:</span></b> <span style="mso-fareast-language:DE-AT" lang="DE">Matthias Kuhn <matthias@opengis.ch><br> <b>Gesendet:</b> Montag, 20. April 2020 12:50<br> <b>An:</b> Drexel Peter <Peter.Drexel@vorarlberg.at>; qgis-user@lists.osgeo.org<br> <b>Betreff:</b> Re: [QGIS-Developer] Memory Leak / Slowdown?</span></p></div></div><p class="MsoNormal"> <br></p><p>Hi Peter,<br></p><p>Did you try this on a clean new profile with no plugins installed?<br></p><p>Bests<br></p><p>Matthias<br></p><div><p class="MsoNormal">On 4/20/20 11:08 AM, Drexel Peter wrote:<br></p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><p class="MsoNormal"><span lang="EN-US">Hi,</span><br></p><p class="MsoNormal"><span lang="EN-US">I use a Python-Script to load all shapefiles in a directory tree one by one and export the sld-files.</span><br></p><p class="MsoNormal"><span lang="EN-US">It works fine, but slows down incredible after ~100 files.</span><br></p><p class="MsoNormal"><span lang="EN-US">So I tried the following barebone script with just the essentials  (load shapefile, add it to the map canvas, remove it again and then clear the project) in a loop and I get the same behavior…</span><br></p><p class="MsoNormal"><span lang="EN-US"> </span><br></p><p class="MsoNormal"><span lang="EN-US">from datetime import datetime</span><br></p><p class="MsoNormal"><span lang="EN-US">for i in range(1, 1000):</span><br></p><p class="MsoNormal"><span lang="EN-US">                vlayer=QgsVectorLayer ("aShapeFile.shp" ,"run  " + str(i) + " at " + datetime.now().strftime('%H:%M:%S') ,"ogr")</span><br></p><p class="MsoNormal"><span lang="EN-US">                QgsProject.instance().addMapLayer(vlayer)</span><br></p><p class="MsoNormal"><span lang="EN-US">                QgsProject.instance().removeMapLayer(vlayer.id())</span><br></p><p class="MsoNormal"><span lang="EN-US">                QgsProject.instance().clear()</span><br></p><p class="MsoNormal"><span lang="EN-US"> </span><br></p><p class="MsoNormal"><span lang="EN-US">Same result,  the Script slows down after every iteration in the loop AND QGIS-memory consumption goes up!</span><br></p><p class="MsoNormal"><span lang="EN-US">Do I need to do some garbage cleaning / what am I doing wrong / is this a Memory leak?</span><br></p><p class="MsoNormal"><span lang="EN-US"> </span><br></p><p class="MsoNormal"><span lang="EN-US">Thanks everybody for any hints / help!</span><br></p><p class="MsoNormal"><span lang="EN-US"> </span><br></p><p class="MsoNormal"><span lang="EN-US">Peter</span><br></p><p class="MsoNormal"><span lang="EN-US"> </span><br></p><p class="MsoNormal"><span class="font" style="font-family:"Times New Roman", serif"><span class="size" style="font-size:12pt"><br></span></span></p><div>_______________________________________________ QGIS-Developer mailing list<a href="mailto:QGIS-Developer@lists.osgeo.org">QGIS-Developer@lists.osgeo.org</a> List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-developer">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a> Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-developer">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br></div></blockquote></div></blockquote><div><br></div>