[Qgis-developer] Possible memory leak in QGIS redraws?

[HAUBOURG]

Sweeeeet!
I'm out of office right now. I will test next week. Anyway, thanks a lot for that impressive work. I love that community.
Cheers
Regis

Régis Haubourg

Le 22 oct. 2015 09:35, Nyall Dawson <nyall.dawson at gmail.com> a écrit :

On 21 Oct 2015 6:49 PM, "Nyall Dawson" <nyall.dawson at gmail.com<mailto:nyall.dawson at gmail.com>> wrote:
>
>
> On 21 Oct 2015 3:20 PM, "Nyall Dawson" <nyall.dawson at gmail.com<mailto:nyall.dawson at gmail.com>> wrote:
> >
> > On 9 October 2015 at 01:45, Régis Haubourg
> > <regis.haubourg at eau-adour-garonne.fr<mailto:regis.haubourg at eau-adour-garonne.fr>> wrote:
> > > Hi all,
> > > I try to narrow down a production issue here.
> > > In long sessions with lot's of redraws on complex projects, memory seems to
> > > increase regularly.
> > >
> > > In a more simple environnement with no plugins, just sending multiple
> > > redraws using F5 key triggers it.
> > > It grows quite fast on 2.6 up to 4Go sometimes, quite slower in master and
> > > 2.10. Every 5-6 redraws, QGIS adds 1Mo of RAM in 2.6, less often in recent
> > > versions.
> > >
> > > If confirmed, and if nothing else is to be blamed (windows?, QT?) this is a
> > > serious issue for enterprise uses, especially if we want to build
> > > application that run all day long with many canvas redraws.
> > >
> >
> > I've done some more testing, using valgrind and massif. Here's what I've found:
> >
> > - there's a big leak coming from Qt's font database. Fixed in Qt 5.1
> > (https://bugreports.qt.io/browse/QTBUG-26861<http://cp.mcafee.com/d/5fHCNEi418SyMM-MqejhOO-OrKrhKejpd7bb1EVodET79ICzBBNdUSzssCOqemm3hO-rhKejd78VVYQsI6U-hBjyuJAaJN8wHZrbWxlK9szIqMx_eNd_HpvkaJNbAtzm4fVS9IswyqenAT7-LOab3dTPhPRXBQQmnKjj78If9Zh5BqWqJXFYG7DR8OJMddECQjt-jppd7bbPBS6jtPpesRG9px1Iz7RGNDjxqviHx2dj5fM1Micxc95QT7D4js2SDDYxFyBo0Ey9YKrd7arPVI5-Aq83iS0V6l2W7CvfGJYjiWq80JmB943_d42LGRymd41_Wld44E6y01MA6z1EwzW4tck96TTzrCnIGKk3PqFA>). About 50% of the memory
> > leaked is caused by this one. We could possibly work around it by
> > having a singleton so we only ever use one QFontDatabase in QGIS, or
> > just get on with the move to Qt5 and get the proper fix for free. ;)
> > - I also see a lot of memory used by Python objects. Specifically, a
> > lot coming from QgsPythonUtilsImpl::evalString and the PyRun_String
> > call in that function. I don't think these are leaks though (at least,
> > not within QGIS). But that's the second largest memory allocator I see
> > reported by massif
> > - There was a small leak of a GEOS object in labeling... don't think
> > this would cause too much memory usage though. I'll submit a fix after
> > giving it some more testing.
> > - Lastly, the remaining large chunk of allocated memory is used by the
> > images stored during rendering. I'm 90% sure these are all properly
> > cleaned, but I still can't throw the niggling suspicion that just
> > maybe there's some special condition in which these images aren't
> > properly deleted.
> >
> > So yeah... no magic fixes to this one unfortunately. (At least, not
> > until QGIS 3.0!)
>
> Actually... Scratch that. Just found some huge leaks in certain circumstances. Fixes in progress.

Try with latest master. It's made a huge difference for me. Exporting my most complex atlas (300 pages, with shapeburst, layer effects, some pretty hefty dynamic labeling):

Initially memory usage is about 320mb.
2.8: memory usage grows to 1.5 gb
2.10: memory usage to about 2 gb
Master: memory usage grows to 480 mb

Win! I suspect Qt5 will make another big difference too.

Nyall

>
> Nyall
>
>
>
> >
> > Nyall
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20151022/e047f93b/attachment-0001.html>