[Qgis-developer] [Server] performance questions

[Jörg Habenicht]

Hello Andreas,

I did forward the question to our internal development, returned some
answers for clarity:

Am 22.04.2016 um 16:45 schrieb Neumann, Andreas:
> Hi Jörg,
> 
> Unfortunately I am not the suitable person to ask. In only occasionally
> develop Python and don't know C/C++. However, I am a long-time QGIS
> server user. I have big projects as well (with 80/90 layers). They
> usually render in 1-2 seconds, not 20 seconds like you cite. So maybe
> there are other issues/bottlenecks?

It seems so.

> Do you have an exceptional number of features in your layers?

Yes.

> Do you use scale thresholds to filter features.

No.

> Typically it doesn't make any sense to render too many features at small
> scales, because you can't depict them anyway if there are too many.
> 
> I assume you use Postgis as a data source - since you talk about DB
> queries.

Ah yes, correct.

> Do you have the proper spatial index and other indexes?

We do have indices installed along the tables, sometimes more than one.
We did not analyse each and every table, but the handling should perform
sufficiently.

> Do you have a recent version of QGIS server?

Fairly recent, version 9.4

> I think version 2.10 and later
> introduced filtering inside the DB (instead of filtering in QGIS), which
> should come with a performance improvement.

No, we dont use DB server filtering.

> 20 seconds is definitely too long a wait for a map rendering, but I
> guess you can bring this down to 1-2 seconds if properly optimized (both
> in DB and in QGIS).
> 
> It may well be that there is room for improvements in rendering speed of
> QGIS server, but multi-threading is probaby the most complicated way of
> improving performance in QGIS server. When I asked Marco Hugentobler,
> the main QGIS Server developer, he told me it would be almost impossible
> to make QGIS server multi-threaded. 3LIZ is also into QGIS server
> development since some time. So these are the guys to ask.
> 
> And then - the other issue is that you want to reserve your other CPU
> cores for parallel requests as well. You usually don't want to dedicate
> all your CPU power to a single http request.

Oh yes certainly I want to.
If an eight core server uses only one core to calculate the result, we
could have a speed improvement factor 2-6 (my guess) for the single
answer. With negligible slowdown of the application because of the
additional handling of multi threading. So in this case the server
delivers the resulting picture faster than before.
And if the same server already has a multi threaded request running on
eight cores, then the linux kernel divides the cpu calculation. So that
a single request still gains an estimated improvement of 1-3.
So in the result we have no significant speed loss in case of a fully
loaded server. But much gain in case of an idle server.


> 
> But if there is a way to improve speed in QGIS server I am all for it ...
> 
> Greetings,
> 
> Andreas
> 
[snip]

cu
Jörg

-- 

mWerk GmbH
Dipl.-Ing. Jörg Habenicht
Landwehrstr. 76
30519 Hannover
(T) +49 511 9999 8033
(F) +49 511 9999 8041
(E) jh at mwerk.net
Amtsgericht Hannover HRB 206522
Geschäftsführer
Reiner Brachvogel
Dennis Kornehl