<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /></head><body style='font-size: 10pt; font-family: Verdana,Geneva,sans-serif'>
<p>Hi Jörg,</p>
<p>Unfortunately I am not the suitable person to ask. In only occasionally develop Python and don't know C/C++. However, I am a long-time QGIS server user. I have big projects as well (with 80/90 layers). They usually render in 1-2 seconds, not 20 seconds like you cite. So maybe there are other issues/bottlenecks? Do you have an exceptional number of features in your layers? Do you use scale thresholds to filter features. Typically it doesn't make any sense to render too many features at small scales, because you can't depict them anyway if there are too many.</p>
<p>I assume you use Postgis as a data source - since you talk about DB queries. Do you have the proper spatial index and other indexes? Do you have a recent version of QGIS server? I think version 2.10 and later introduced filtering inside the DB (instead of filtering in QGIS), which should come with a performance improvement.</p>
<p>20 seconds is definitely too long a wait for a map rendering, but I guess you can bring this down to 1-2 seconds if properly optimized (both in DB and in QGIS).</p>
<p>It may well be that there is room for improvements in rendering speed of QGIS server, but multi-threading is probaby the most complicated way of improving performance in QGIS server. When I asked Marco Hugentobler, the main QGIS Server developer, he told me it would be almost impossible to make QGIS server multi-threaded. 3LIZ is also into QGIS server development since some time. So these are the guys to ask.</p>
<p>And then - the other issue is that you want to reserve your other CPU cores for parallel requests as well. You usually don't want to dedicate all your CPU power to a single http request.</p>
<p>But if there is a way to improve speed in QGIS server I am all for it ...</p>
<p>Greetings,</p>
<p>Andreas</p>
<p>On 2016-04-22 15:20, Jörg Habenicht wrote:</p>
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">Hi Andreas, dear list,<br /><br /> sorry for the warm up of this old topic.<br /><br /> There is in deed a need to enable multi threading in Qgis-Server.<br /><br /> In our company we use around 60 different layer to lay some topics on<br /> the map. This may be an extreme example to read, but quite common in our<br /> environment, i.e. we can not lower this number significantly.<br /><br /> A warmed up QGis-Server process which already read and cached the<br /> Configuration file qgis-xy.conf still needs around 20 seconds to<br /> calculate the complete resulting map picture.<br /><br /> During this calculation there is only one thread working, serializing<br /> the DB-requests, calculation the layer picture and mapping the picture<br /> on the resulting picture.<br /> The process is running on a multi core server. If we can utilize the<br /> idle cores to do useful work, this can be an easy example to speed up<br /> the process (significant).<br /><br /> I'd like to share my developing experience to do this change, but I<br /> would need further information and some support to do some useful patches.<br /><br /> I have scanned the git-Qgis sources (commit<br /> 81744ecf90a8f6c1e2e94fdacb07e3dca6987dcc) for a suitable multithreading<br /> entry. Found QgsMapRenderer::render() because of the line "while (<br /> li.hasPrevious() )". It seems in this loop the layers are painted in<br /> serial onto the resulting picture.<br /> If I can build an auto parallel computing with QtConcurrent in this<br /> place there seems to be the most gain. I mean to parallelize the loop<br /> with concurrent DB requests, concurrent painting in separate pictures<br /> and in serial map the single pictures onto the result picture.<br /><br /> Right now I don't know if I can do DB requests in parallel, or where the<br /> code for database requests is buried.<br /> And how about the calls inside the loop, are there side effects<br /> preventing multi threading?<br /><br /><br /> Do you got hints, flames or suggestions?<br /><br /><br /> cu<br /> Jörg<br /><br /><br /> Am 23.02.2016 um 16:39 schrieb Neumann, Andreas:
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0">Hi Régis,<br /><br /> QGIS server is single-threaded. You can't simply turn on multi-threaded<br /> rendering in QGIS server. The other issue is that each Apache FCGI<br /> process has it's own set of cache. So if you have 5-10 parallel Apache<br /> threads or processes each thread/process has it's own cache. If your<br /> client hits a thread that has the cache unititialized you will have wait<br /> longer until this cache is fillled.<br /><br /> I once asked Marco whether one could<br /><br /> a) have a shared cache for all Apache threads or processes</blockquote>
<br /> There is no shared cache, because Apache spawns QGis-Processes with its<br /> own memory layout. these Processes know nothing about its neighbor, and<br /> you have a hard time to exchange data between all processes.<br /> How about a scheduler, which assigns requests to warmed up Qgis-processes?<br /><br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><br /> b) support multi-threaded rendering<br /><br /> But he said it would be very complicated to implement a) and b), if not<br /> impossible. Maybe other devs have other views. Note that was about 2<br /> years ago, when I last discussed this with Marco.<br /><br /> ---------------------<br /><br /> Nevertheless I believe there would be room for performance improvements<br /> in QGIS server - if interested parties share their efforts (be it dev<br /> time or financial resources).<br /><br /></blockquote>
[snip]
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><br /> Andreas<br /><br /></blockquote>
[snip]
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><br /> <br /><br /><br /> _______________________________________________<br /> Qgis-developer mailing list<br /><a href="mailto:Qgis-developer@lists.osgeo.org">Qgis-developer@lists.osgeo.org</a><br /> List info: <a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br /> Unsubscribe: <a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br /><br /></blockquote>
<br /> -- <br /><br /> mWerk GmbH<br /> Dipl.-Ing. Jörg Habenicht<br /> Landwehrstr. 76<br /> 30519 Hannover<br /> (T) +49 511 9999 8033<br /> (F) +49 511 9999 8041<br /> (E) <a href="mailto:jh@mwerk.net">jh@mwerk.net</a><br /> Amtsgericht Hannover HRB 206522<br /> Geschäftsführer<br /> Reiner Brachvogel<br /> Dennis Kornehl<br /> _______________________________________________<br /> Qgis-developer mailing list<br /><a href="mailto:Qgis-developer@lists.osgeo.org">Qgis-developer@lists.osgeo.org</a><br /> List info: <a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br /> Unsubscribe: <a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a></div>
</blockquote>
<p> </p>
<div> </div>
</body></html>