[QGIS-Developer] QGIS Server plugins and thread safety

[G. Allegri]

Python GIL  doess't guarantee thread safety, and it only applies to python
code. It only protects the internal state of the interpreter.
Moreover The C API offers the opportunity to release the GIL (see for
example numpy [1]).

Giovanni

[1] https://stackoverflow.com/a/36480941

Il giorno gio 25 ott 2018 alle ore 13:10 Alessandro Pasotti <
apasotti at gmail.com> ha scritto:

>
> I would go for multiprocess and stick to a single thread per-process.
>
> But David may have a better advice.
>
> Btw, it's nice to see a use case for Python Vector data providers :)
>
>
>
>
> On Thu, Oct 25, 2018 at 1:02 PM Radim Blazek <radim.blazek at gmail.com>
> wrote:
>
>> If we narrow down the environment to uWSGI (with multiple threads
>> enabled) running a Python application, which means that more instances
>> of the same application may run in threads but never at the same time
>> due to Python GIL (if I understand it well), is it still a problem? If
>> we ensure that there are no static variables and singletons used by
>> QGIS server? Is it then a problem to create/delete QObject on a non
>> main thread?
>>
>> Radim
>> On Sun, Nov 19, 2017 at 5:39 PM Martin Dobias <wonder.sk at gmail.com>
>> wrote:
>> >
>> > Hi Alessandro
>> >
>> > On Sun, Nov 19, 2017 at 10:34 AM, Alessandro Pasotti <
>> apasotti at gmail.com> wrote:
>> > > Hi,
>> > >
>> > > mi recent experiments with multi-threaded python wrappers for QGIS
>> server
>> > > showed a critical flaw in my original implementation of the server
>> plugins.
>> > >
>> > > First, I want to stress that this is not a problem in FCGI server
>> > > implementation but only if the server is used directly from python in
>> a
>> > > multi threaded server implementation.
>> > >
>> > > The problem is that the server interface that exposes the request
>> handler is
>> > > a static property of the interface, that is changed on every request.
>> > > This means that there is a race condition in setting/accessing the
>> handler
>> > > from a plugin filter, and that a plugin filter might access to the
>> handler
>> > > for a wrong request.
>> >
>> > I think this is probably just the tip of the iceberg: most of the
>> > classes in qgis_core are not thread-safe, and I believe the
>> > implementation of server's services is not expected to be run in
>> > multiple threads at once either. So even though things in theory could
>> > work if requests are handled in multiple threads, most likely there
>> > will be crashes. Even though map rendering was made to run safe in
>> > multiple worker threads, there is still the assumption that the map
>> > rendering is started from the main thread where all the objects live
>> > (I'm talking about the QObject-based classes). Dealing with objects
>> > like map layers or project in non-main thread is likely to end up
>> > badly.
>> >
>> > I would say for now we should simply stick with handling only one
>> > request per server process at a time - just like how it is done with
>> > the FastCGI implementation.
>> >
>> > Cheers
>> > Martin
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > QGIS-Developer at lists.osgeo.org
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>
>
> --
> Alessandro Pasotti
> w3:   www.itopen.it
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20181025/3f4afc98/attachment.html>