[Qgis-developer] QGIS Server does not release files after each request - OGROpenShared instead of OGROpen ?

[kimaidou]

Hi !

Paolo : the bug has been backported in branches "release-2_8" and
"release-2_10", so it would be included in 2.8.4 whenever it is released.

Andreas : This fix is only relevant for WMS clients using a non empty
STYLES parameter in GetMap. In this cases, this would greatly improve the
speed if :

* the WMS client do use a non empty STYLES parameter. If not, QGIS Server
already used the cached layer.
* the WMS client sends a lot of GetMap queries, for example if you have
many users or if you use tiles.
* the underlying data are big or complex, because the queries "SELECT
extent()" and "SELECT count(id)=count(DISTINCT id) will not be sent, and
theses queries can be slow.

In Lizmap client for example, we use STYLES with 'default' for each layer,
and we usually have map built with tiles. So it will be significant here.

Cheers,
Michaël


2015-09-16 10:54 GMT+02:00 Andreas Neumann <a.neumann at carto.net>:

> Thanks René-Luc and Kimaidou,
>
> This sounds like a useful fix. I will test and report if I run into issues.
>
> Do you have any numbers/observations how much this will speed up services
> that use Postgis as storage? I mean very rough numbers only.
>
> Anyway - I will test.
>
> Andreas
>
>
> On 16.09.2015 10:19, kimaidou wrote:
>
> Hi all
>
> René-Luc has found why this appeared, essentially with Lizmap. Lizmap send
> GetMap request to QGIS Server with a non empty STYLES parameter (ex:
> STYLES=default). This is to let the user choose the style.
>
> In this case, QGIS Server did not use the maplayer cache, so it had to get
> all information from the provider to build the maplayer objet for each
> GETMAP request ! This is why in this case, a new opening was made on the
> file.
> The following commit fixes it :
>
> https://github.com/qgis/QGIS/commit/a5450a78ffcbba9a75afa8be43460ad741fea050
>
> It has been backported in branches release-2_8 and release-2_10
>
> A very good side effect is with PostGIS layers :
> * before, QGIS Server sent many many queries to build the whole maplayer
> for each GetMAP, such as : [1] if metadata were available, and worse like
> [2] if metadata were not (for example for PostGIS views or if the user had
> disabled "use metadata" )
> * now only the main "get data" queries are sent for each GetMap, like [3]
>
> In [2], you can see some performance kiling queries (on big datasets),
> such as
>
> SELECT count(distinct ""nb"")=count(""nb"") FROM (SELECT row_number()
> OVER() AS nb, * FROM ""2013"".v_geo_parcelle
> ) AS ""subQuery_0"""
>
> SELECT st_extent(""geom"") FROM (SELECT row_number() OVER() AS nb, * FROM
> ""2013"".v_geo_parcelle
> ) AS ""subQuery_0""
>
> [1] https://paste.sh/yguqyRWL#EPcHHVH3uuijGgEtzH0pAlHb
> [2] https://paste.sh/frCglZ9A#A1nQ6BXtdBeEiZWQmTJP8v9l
> [3] https://paste.sh/Rkvqf757#K1MCRitovIxMblBVils1VF7r
>
>
> Thanks a lot to René-Luc for fixing this !
>
> Cheers
> Michaël
>
> 2015-09-14 8:39 GMT+02:00 Luca Manganelli <luca76 at gmail.com>:
>
>> On Fri, Sep 11, 2015 at 5:09 PM, kimaidou < <kimaidou at gmail.com>
>> kimaidou at gmail.com> wrote:
>> > I tested a quick and dirty patch, by replacing all occurences of
>> OGROpen byt
>> > OGROpenShared in QGIS source, then rebuild and test server side --> No
>> locks
>> > anymore ! QGIS Server works like a charm here, without increasing the
>> number
>> > ...
>>
>> This is a great fix! It's important to optimize resources, expecially
>> with qgis server.
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>
>
>
> _______________________________________________
> Qgis-developer mailing listQgis-developer at lists.osgeo.orghttp://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20150916/cba26f9a/attachment.html>