[Qgis-developer] QGIS Server does not release files after each request - OGROpenShared instead of OGROpen ?

[kimaidou]

Hi

I tested a quick and dirty patch, by replacing all occurences of OGROpen
byt OGROpenShared in QGIS source, then rebuild and test server side --> No
locks anymore ! QGIS Server works like a charm here, without increasing the
number of opening for the same file.

The only (but big) problem, is that this seems to be not thread safe, so it
makes QGIS Desktop segfault. For example, I just started QGIS 2.8.3 with my
patch, and it segfaults when "Browser" pannel read the content of my
favorite folders (and find Shapefiles). I am pretty sure that the
multithreaded rendering would also cause it to segfault.

No worries for the Server side, since it is not multithreaded by conception.

We are investigating this issue and will report back here.

Cheers
Michaël




2015-09-11 9:39 GMT+02:00 kimaidou <kimaidou at gmail.com>:

> Hi all
>
> We are trying to use QGIS Server with nginx / supervisor instead of
> classical Apache / mod_fcgid.
>
> We have seen that for each server request, such as a GetMap responding an
> image rendered from a single Shapefile (or Tiff, of whatever), QGIS Server
> opens a new access to the file.
>
> Exemple number found via the lsof command, after some requests
>
> lsof /tmp/someShapefile.shp | wc -l
> 126
>
> which means here 126 requests has been processed by QGIS Server, but the
> access to Shapfile has never been released. Each requests leads to a new
> file handler.
>
> Imagine a project with several layers, and a web map with many users /
> requests --> this lead rapidly to many opening of the same files open for
> nothing, and can cause issues.
>
> In Apache configuration, this was not sensible, as mod_fcgid and Apache
> kill QGIS Server worker after N completed requests, and it seems the ulimit
> used is the system limit.
>
> In our configuration, supervisor has by default a ulimit of 1024 files
> opened. This limit is vers fast reached ( 1024 requests only ). We could
> tune this configuration, but it seems to walk aournd the issue without
> solving its origin.
>
>
> We have 2 ways of dealing with it :
>
> * Make sure QGIS Server does not open a data file each time a request
> needs it --> we will investigate if the cache is well used
>
> * We could also use GDALOpenShared instead of GDALOpen , and OGROpenShared
> instead of OGROpen in QGIS provider code:
>
> GDALOpenShared :
> http://www.gdal.org/gdal_8h.html#a46a02c21047d78f88f8abb1ff6b14ae2
> OGROpen in QGIS :
> https://github.com/qgis/QGIS/search?utf8=%E2%9C%93&q=ogropen&type=Code
>
>
> Any idea, comment appreciated !
>
> Regards,
> Michaël
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20150911/ec8ab72f/attachment.html>