[Qgis-psc] (QGIS 2020 Grant Report) QGIS Server and performance monitoring

[Nyall Dawson]

On Thu, 12 Nov 2020 at 20:17, Paul Blottiere <blottiere.paul at gmail.com> wrote:
>
> Dear PSC,
>
>
> Here is the report about the QEP
> https://github.com/qgis/QGIS-Enhancement-Proposals/issues/185.
>
> The whole QGIS-Server-PerfSuite has been upgraded to use 3.10 and 3.14 releases
> side by side with 2.18 and master branch. Performances may be now monitored
> daily with the latest releases. Moreover, a simple anomalies detection mechanism
> has been implemented and a mail is sent if a regression is detected.
>
> Several scenarios have also been added to compare performance with the same
> data but relying on different providers (PostGIS, Spatialite, Geopackage and
> Shapefile).
>
> Finally, a simple mechanism based on multiprocessing has been implemented to
> simulate multi-clients situation. However, we don't have scenario based on this
> mechanism for now.
>
> For those interested, the last report is available here:
> http://test.qgis.org/perf_test/graffiti/2020_11_12_01_00/report.html.
>
> In spite of the limited scope of scenarios, there're very interesting results.
> Here is not the place to talk about it, but I'll communicate about it later.

This is great work, and very valuable!! Thank you!

If I can make a few suggestions for improving the charts when you're
next working on this platform, can I suggest:

- change the color for the "database" tests - the duplicate colors
mean it's not possible to see which results are for the database vs
non database results
- Splitting the "with trust" tests out from the without trust tests:
there's such a difference in the timings of the tests with/without
this setting that the axis choices just don't work well for the "with
trust" results.
- split the 1 core vs 4 core tests into separate charts, so it's
easier to compare like for like
- change the rounding to 0.01 second instead of 0.1 seconds. Eg.
results for "GetMap: Lines" are tricky to interpret with the 0.1
second rounding
- GetMap: Polygons and datasource influence -- clarify whether these
results are for master?
- add the git sha somewhere so we can tell exactly what master refers to

Hope that helps!
Nyall