[Qgis-developer] Perfs: a lot of WKB conversions

[Patrick Valsecchi]

Hi,

I was looking at the perfs of qgis server and I did a small profiling of a
GetMap on a road layer (linestring) with QGIS configured with defaults. We
spend a lot of time converting to and from WKB. For example, for rending
67k features (3 GetMaps), we do this for each feature:

   - When reading the features from the DB, *we parse the WKB* and convert
   it to a QgsLineStringV2 (array of x and array of y)
   - Then, in QgsSymbolV2, we get the WKB from the QgsLineStringV2 (OK, no
   conversion, we kept the WKB)
   - Then QgsClipper::clippedLineWKB uses the QgsConstWkbSimplifierPtr
   which uses QgsMapToPixelSimplifier::simplifyPoints:
      - *Parse the WKB* to get the bounding box
      - *Parse the WKB* to read the points, simplify them and *write a new
      WKB* with the simplified geometry
      - *Parse the simplified WKB* to read the points into a QPolygonF

So that is 4 parsing of WKB and 1 write WKB instead of the single parse
that is really needed. I have trouble getting exact numbers but we spend
roughly 15% of our time doing just that. For example, we spend globally
4.3% of our time calling 1M times QgsConstWkbPtr::verifyBound.

A few numbers (% are relative to the caller and I show only the major
stuff):

   - QgsWMSServer::executeRequest
      - 83% in getMap ->QgsVectorLayerRenderer::drawRendererV2
         - 69% in renderFeature -> QgsSymbolV2::renderFeature
         - 52% in QgsLineSymbolV2::renderFeature (mostly Qt rendering stuff)
            - 35% in QgsSymbolV2::_getLineString ->
            QgsMapToPixelSimplifier::simplifyPoints
               - 46% simplifyWkbGeometry
               - 36% calculateBoundingBox
               - 11% QgsConstWkbPtr::operator>>
               - 25% in QgsFeatureIterator::nextFeature ->
         QgsPostgresFeatureIterator::fetchFeature
         - 79% in QgsPostgresFeatureIterator
            - 4% QgsPostgresConn::PGgetResult
            - others...
            - 17% in QgsHttpRequestHandler::setGetMapResponse

Now, simplifying all the WKB parse/write will gain maybe 10-15% in global
perfs for this case. But there are maybe reasons for that (memory usage or
I don't know)?
Why not avoiding simplifying a linestring if its bbox is in the same
ballpark as the bbox we are rendering? Or maybe this logic: if bboxLS.w /
nbPoints > pixelW*2 or bboxLS.h / nbPoints > pixelH*2, then we don't do
simplification. The idea being points are roughly equally spaced in a
linestring.

Why storing the points as two arrays in QgsLineStringV2 instead of directly
a QPolygonF?

I've been focusing only on linestrings for the moment. But I guess the same
applies to the other types of geometry.

I know it's a lot of questions for a single email. But I'm new to this code
and have no knowledge of its history and the choices/tradeoffs that were
made.

Thanks.


PS:

I think I've found what I'll do at Bonn's Hackfest ;-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20160719/121117b2/attachment-0001.html>