[Qgis-developer] Memory leak in QGIS 2.14 with a Shapefile

[kimaidou]

Hi Nyall,

I have seen you have just backported the commits into release-2_14. Thanks
a lot !

Michaël

2016-05-14 14:57 GMT+02:00 kimaidou <kimaidou at gmail.com>:

> Ok, thanks a lot !
>
> 2016-05-14 12:41 GMT+02:00 Nyall Dawson <nyall.dawson at gmail.com>:
>
>>
>> On 14 May 2016 19:39, "kimaidou" <kimaidou at gmail.com> wrote:
>> >
>> > Hi Nyall,
>> >
>> > I have just tested it with today's master, and there is no memory leak
>> any more. Thanks a lot for helping solving this nasty one.
>> > Could you please backport the related "memory leak" commits to the 2.14
>> release branch ?
>>
>> Thanks for the confirmation. I'll backport sometime next week (got a
>> bunch of commits I need to do this for)
>>
>> Nyall
>>
>> >
>> > Cheers,
>> > Michaël
>> >
>> > 2016-05-10 14:10 GMT+02:00 kimaidou <kimaidou at gmail.com>:
>> >>
>> >> Thanks Nyall,
>> >>
>> >> I will build last master and check it asap.
>> >>
>> >> Cheers,
>> >> Michaël
>> >>
>> >> 2016-05-10 6:25 GMT+02:00 Nyall Dawson <nyall.dawson at gmail.com>:
>> >>>
>> >>> On 10 May 2016 at 03:38, kimaidou <kimaidou at gmail.com> wrote:
>> >>> > Hi
>> >>> >
>> >>> > I just tested with a freshly built QGIS release-2_14 , and the issue
>> >>> > remains.
>> >>> > The easiest way to reproduce is to zoom back and forth from big
>> scales ( ~
>> >>> > 1:25 000 ) to layer extent scale ( ~ 1:600 000 )
>> >>>
>> >>> Should be fixed by
>> >>>
>> https://github.com/qgis/QGIS/commit/6b80518a5b55ab30fb19493266d5626d0f89d26b
>> >>>
>> >>> There was a leak in the OGR provider when a feature had null geometry
>> >>> and a filter rect was set on a feature request (such as when rendering
>> >>> a map).
>> >>>
>> >>> I also fixed another (potentially huge?) memory leak in the python
>> code.
>> >>>
>> >>> Would appreciate testing and feedback on whether this has made a big
>> >>> difference for you!
>> >>>
>> >>> Nyall
>> >>>
>> >>> >
>> >>> > Good evening/morning
>> >>> > Michaël
>> >>> >
>> >>> > 2016-05-09 15:02 GMT+02:00 kimaidou <kimaidou at gmail.com>:
>> >>> >>
>> >>> >> Hi Matthias,
>> >>> >>
>> >>> >> Not yet, I need to build QGIS master before being able to report
>> back
>> >>> >> Thanks for your answer
>> >>> >>
>> >>> >> 2016-05-09 14:59 GMT+02:00 Matthias Kuhn <matthias at opengis.ch>:
>> >>> >>>
>> >>> >>> Have you tested with either the latest 2.14 nightly or master
>> nightly?
>> >>> >>> An issue with OGR connection pooling has been resolved recently.
>> And I
>> >>> >>> wonder if that could be related.
>> >>> >>>
>> >>> >>>  -- Matthias
>> >>> >>>
>> >>> >>>
>> >>> >>> On 05/09/2016 02:28 PM, kimaidou wrote:
>> >>> >>>
>> >>> >>> I created an issue about it : http://hub.qgis.org/issues/14792
>> >>> >>>
>> >>> >>> 2016-05-09 11:48 GMT+02:00 kimaidou <kimaidou at gmail.com>:
>> >>> >>>>
>> >>> >>>> Hi again,
>> >>> >>>>
>> >>> >>>> I also forgot to mention that I have no memory leak when using
>> the same
>> >>> >>>> dataset in PostGIS ( tested by importing the Shapefile in
>> PostGIS with
>> >>> >>>> shape2pgsql )
>> >>> >>>>
>> >>> >>>> Michaël
>> >>> >>>>
>> >>> >>>> 2016-05-09 11:07 GMT+02:00 kimaidou <kimaidou at gmail.com>:
>> >>> >>>>>
>> >>> >>>>> I forgot to mention the steps to reproduce the bug, and the
>> precautions
>> >>> >>>>> to take :
>> >>> >>>>>
>> >>> >>>>> * open the project in QGIS 2.14 ( I have not tested with older
>> >>> >>>>> versions)
>> >>> >>>>>
>> >>> >>>>> * open a terminal and use this kind of command to monitor QGIS
>> memory
>> >>> >>>>> consumption : htop, or ps with
>> >>> >>>>> watch -n1 'ps -aux --sort -pmem | head -5'
>> >>> >>>>>
>> >>> >>>>> * Navigate through the project by zooming in and out, pan, etc.
>> I think
>> >>> >>>>> you emphasize the memory leak by displaying the layer at small
>> scale ( with
>> >>> >>>>> many buildings displayed)
>> >>> >>>>>
>> >>> >>>>> * Be aware that this will increase the memory, so keep an eye
>> on the
>> >>> >>>>> memory consumption so that your system has enough free memory
>> to work
>> >>> >>>>> properly.
>> >>> >>>>>
>> >>> >>>>> Michaël
>> >>> >>>>>
>> >>> >>>>> 2016-05-09 11:00 GMT+02:00 kimaidou <kimaidou at gmail.com>:
>> >>> >>>>>>
>> >>> >>>>>> Hi devs,
>> >>> >>>>>>
>> >>> >>>>>> I have just found a memory leak in QGIS Desktop (and server)
>> with the
>> >>> >>>>>> project available in [1] ( a 20Mo tar.bz2 )
>> >>> >>>>>>
>> >>> >>>>>> This is a very simple project with only one buildings
>> Shapefile ( SHP
>> >>> >>>>>> 68Mo). This layer has been exported from PostGIS from a query
>> wich aimed to
>> >>> >>>>>> simplify the building for fast rendering.
>> >>> >>>>>>
>> >>> >>>>>> The geometries have been simplified ( I used SnapToGrid, then
>> >>> >>>>>> (St_Dump(St_Union(a.geom)))  to remove as many nodes as
>> possible )
>> >>> >>>>>>
>> >>> >>>>>> The layers has ~ 700 000 features.
>> >>> >>>>>>
>> >>> >>>>>> Has anyone encoutered such a leak ?
>> >>> >>>>>>
>> >>> >>>>>> [1]
>> >>> >>>>>>
>> https://framadrop.org/r/_7VsZtvzdU#+eJHHPPOIdVcgj/AUp8V91T9m6gmO7/XCh6uOhzP7ro=
>> >>> >>>>>>
>> >>> >>>>>> Cheers,
>> >>> >>>>>>
>> >>> >>>>>> Michaël
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>> _______________________________________________
>> >>> >>> Qgis-developer mailing list
>> >>> >>> Qgis-developer at lists.osgeo.org
>> >>> >>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >>> >>> Unsubscribe:
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >>> >>>
>> >>> >>>
>> >>> >>> --
>> >>> >>> Matthias Kuhn
>> >>> >>> OPENGIS.ch - https://www.opengis.ch
>> >>> >>> Spatial • (Q)GIS • PostGIS • Open Source
>> >>> >>>
>> >>> >>>
>> >>> >>> _______________________________________________
>> >>> >>> Qgis-developer mailing list
>> >>> >>> Qgis-developer at lists.osgeo.org
>> >>> >>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >>> >>> Unsubscribe:
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>> > _______________________________________________
>> >>> > Qgis-developer mailing list
>> >>> > Qgis-developer at lists.osgeo.org
>> >>> > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >>> > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >>
>> >>
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20160516/8d6def74/attachment-0001.html>