[Qgis-psc] Fixing a performance regression in rule based renderer with many rules

[Neumann, Andreas]

Dear PSC members, 

There had been major performance regressions in QGIS since 2.8 for
layers that use many categories or rules.  See issues
http://hub.qgis.org/issues/15369 and http://hub.qgis.org/issues/15404 

Both Nyall and Martin had a brief look at the issues. 

One issue is that the layer tree gets more and more powerful over the
releases and may result in performance issues, but the much bigger issue
is the SQL compiler that translates QGIS client side filters to SQL code
to be executed on the database server, in order to reduce the number of
features that are actually needed for rendering. While for a low number
of rules, this works fine, it fails big on many rules. The slow down can
be a factor of 20-50, depending on the number of symbology/labeling
rules and may even bring QGIS/and/or the database to a crash. Seems like
for each rule, an SQL "OR" combination is added, which can result in
massive SQL statements that may even cause troubles on the database, and
the result is much slower than without the "optimized" statement. 

Seems like Martin is able to fix this in approx. 2 dev days, with
profiling and some heuristics deciding whether a serverside or
clientside filtering would be better. He will look at the SQL compiler
issues and also on potential performance issues in the layer tree, for
bigger legends. 

I am asking PSC for approval that we can pay him the two dev days from
the QGIS.ORG funds to work on this issue. Fixes would go into 3x and the
various relevant 2x versions that still get updates. 

Can you please provide feedback if you would agree with this proposal or
not? 

Thank you, 

Andreas 

  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20161021/9ecf4763/attachment.html>