[Qgis-developer] Perfs: a lot of WKB conversions

[Nyall Dawson]

On 19 July 2016 at 17:58, Patrick Valsecchi
<patrick.valsecchi at camptocamp.com> wrote:
> Hi,
>
> I was looking at the perfs of qgis server and I did a small profiling of a
> GetMap on a road layer (linestring) with QGIS configured with defaults. We
> spend a lot of time converting to and from WKB. For example, for rending 67k
> features (3 GetMaps), we do this for each feature:
>
> When reading the features from the DB, we parse the WKB and convert it to a
> QgsLineStringV2 (array of x and array of y)
> Then, in QgsSymbolV2, we get the WKB from the QgsLineStringV2 (OK, no
> conversion, we kept the WKB)
> Then QgsClipper::clippedLineWKB uses the QgsConstWkbSimplifierPtr which uses
> QgsMapToPixelSimplifier::simplifyPoints:
>
> Parse the WKB to get the bounding box
> Parse the WKB to read the points, simplify them and write a new WKB with the
> simplified geometry
> Parse the simplified WKB to read the points into a QPolygonF
>
> So that is 4 parsing of WKB and 1 write WKB instead of the single parse that
> is really needed. I have trouble getting exact numbers but we spend roughly
> 15% of our time doing just that. For example, we spend globally 4.3% of our
> time calling 1M times QgsConstWkbPtr::verifyBound.
>
> A few numbers (% are relative to the caller and I show only the major
> stuff):
>
> QgsWMSServer::executeRequest
>
> 83% in getMap ->QgsVectorLayerRenderer::drawRendererV2
>
> 69% in renderFeature -> QgsSymbolV2::renderFeature
>
> 52% in QgsLineSymbolV2::renderFeature (mostly Qt rendering stuff)
> 35% in QgsSymbolV2::_getLineString ->
> QgsMapToPixelSimplifier::simplifyPoints
>
> 46% simplifyWkbGeometry
> 36% calculateBoundingBox
> 11% QgsConstWkbPtr::operator>>
>
> 25% in QgsFeatureIterator::nextFeature ->
> QgsPostgresFeatureIterator::fetchFeature
>
> 79% in QgsPostgresFeatureIterator
> 4% QgsPostgresConn::PGgetResult
> others...
>
> 17% in QgsHttpRequestHandler::setGetMapResponse
>
> Now, simplifying all the WKB parse/write will gain maybe 10-15% in global
> perfs for this case. But there are maybe reasons for that (memory usage or I
> don't know)?
>
> Why not avoiding simplifying a linestring if its bbox is in the same
> ballpark as the bbox we are rendering? Or maybe this logic: if bboxLS.w /
> nbPoints > pixelW*2 or bboxLS.h / nbPoints > pixelH*2, then we don't do
> simplification. The idea being points are roughly equally spaced in a
> linestring.

See https://github.com/qgis/qgis3.0_api/issues/15

>
> Why storing the points as two arrays in QgsLineStringV2 instead of directly
> a QPolygonF?

We can't use a QPolygonF, as it uses qreal for storage. For certain
architectures this is float, not double, and so is unsuitable for
storing coordinates.

I think the first logical step would be converting the clipper to work
directly with QgsAbstractGeometryV2, so that QgsSymbol can avoid one
of the to/from conversions at least.

Nyall


> I've been focusing only on linestrings for the moment. But I guess the same
> applies to the other types of geometry.
>
> I know it's a lot of questions for a single email. But I'm new to this code
> and have no knowledge of its history and the choices/tradeoffs that were
> made.
>
> Thanks.
>
>
> PS:
>
> I think I've found what I'll do at Bonn's Hackfest ;-)
>
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer