[Qgis-psc] Performance tests QGIS Server

Sat Jun 9 08:07:16 PDT 2018

Hi PSC ,

Yes we are aware of the MS perf platform since the 2016 QGIS server code
sprint.

We have been discussing quite a few times about coding with performance and
waited for some investment from C2C during 2017. We are very happy they now
publish a daily report, however publishing static tests on static datasets
is not what we seek. Yves showed some results in the QGIS Fr user days two
years back, and I saw the expectations, frustrations and debates it rose
that time.
Our goal is NOT to compare QGIS server with Mapserver or GeoServer. We
don't want to fall into these of debates that in my opinion are dangerous
for everyone because it's almost impossible to reach absolute measuring for
tools that runs in different environnement and render data differenly.

We are talking of a semi scientific approach here, and being able to
reproduce and confirm issues with different tools is probably a very good
thing, if we just can take time to analyse things.

Our task was to build a very light platform dedicated to be integrated in
continuous integration, and that is really easy to enrich with new tests.
So we build a comparison between framework between ltr, release and dev
version that can be run in many contexts (web server type, mutlithread,
multiprocess options, rendering options, data providers, etc..).

We also want to keep an history of the developpement version performance
for each test.

As you see, this can lead to massive amount of computing and logs, so we
really need something easy to set up and as light as possible. Moreover,
measuring performance needs light tools to avoid influencing the measure
themselves.

We believe that QGIS renders data extremely fast, but it has some glitches
due to the desktop design oringin that can be tackled if we have permanent
feedback.

Paul just finished the platform this week, and we now dedicating efforts on
running it on a dedicated server to be certain of not being influenced by
any external load.

I hope next week we'll publish it with the first reports.

In the end, the question of hosting the platform on dedicated server still
remains, as much as administrating it correctly so that no tool run in
parallel at the same time, with the risk of altering the measures.  I hope
you are still ok with the fact that we need this kind of tool.

Best regards,
Régis

2018-06-09 15:10 GMT+02:00 Richard 🌍 Duivenvoorde <richard at duif.net>:

> Hi PSC,
>
> On my TODO list there was to ask Yves/Camptocamp about their QGISserver
> tests and make sure Oslandia was eventually aware of that.
>
> See below:
>
> - results are available
> https://gmf-test.sig.cloud.camptocamp.net/ms_perfs/
> and apparenlty updated daily?
> Yves promises to do some cleanup and upload to test.qgis.org
>
> - he thinks Oslandia is aware of this work (to be sure, I bcc Regis :-)
>
> Regards,
>
> Richard Duivenvoorde
>
>
> -------- Forwarded Message --------
> Subject:        Re: Performance tests QGIS Server
> Date:   Wed, 6 Jun 2018 10:30:39 +0200
>
> Le 04/06/2018 à 20:44, Richard 🌍 Duivenvoorde a écrit :
> > Hi Yves,
> >
> > During PSC meeting we were talking about some QGIS-Server OWS
> > performance-tests/service that Oslandia is doing currently.
> I heard something about this indeed.
> > Andreas mentioned that CampToCamp also did something (for Andreas) last
> > year. And that you asked me to put it on the website.
> > So Question: did you ask me? And did I answer? :-)
> Camptocamp did it something and we discussed about how to improve it and
> share to the community. The topic was "QGIS benchmark" (07/11/2017). See
> https://gmf-test.sig.cloud.camptocamp.net/ms_perfs/. Here a quote of
> your answer:
>
> """
> What is the idea? That we (as qgis.org <http://qgis.org>) ourselves run
> the benchmark
> (docker?) every now and then?
>
> If NOT, then it is easiest when I give you credentials on the
> test.qgis.org <http://test.qgis.org> (virtual)Webserver, so you can
> rsync/scp the result to a
> directory 'benchmarks' at:
>
> http://test.qgis.org/
>
> As you can see Paul (of Oslandia) also pushes the QGISServer CITE test
> results there into a directory 'ogc_cite':
>
> http://http://test.qgis.org/ogc_cite/ <http://test.qgis.org/ogc_cite/>
>
> The idea is to either add an index.html to test.qgis.org
> <http://test.qgis.org> which then
> sents you to individual test directories (or the latest in that
> directory), OR we create a (translatable) page in the qgis.org
> <http://qgis.org> website
> which does some description of the different tests, and then links to
> the html-output pages on test.qgis.org <http://test.qgis.org>
> I think there should really be some explanation at the performance tests...
>
> Both is possible, just need some body/time to do it :-)
> """
> I am ok with a rsync on the QGIS server. Result need some love to
> improve some graphics (null value gives no graph at all).
>
> And I should add picture of layer to illustrate the layer complexity.
>
> > Andreas was also wondering in how much of the Oslandia work is actually
> > already done by C2C and if it is still handy to communicatie about this.
> I don't know, I have no idea what Oslandia is working on, well not more
> than what they shared on QGIS mailing list. They are aware of our
> project for sure, as we discussed about this at the QGIS server hackfest
> and 3Liz pushed a pull request.
>
> Y.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20180609/d43d19be/attachment.html>