[QGIS-Developer] QGIS Server plugins and thread safety

[Radim Blazek]

OK, single thread only for now.

On Sun, Nov 19, 2017 at 5:39 PM Martin Dobias <wonder.sk at gmail.com> wrote:

> Dealing with objects like map layers or project in non-main thread is likely
> to end up badly.

Just as a theoretical question: Why, if there are no static variables
and everything (except layer rendering, which works in threads
already) will happen on the same thread?

Radim

On Thu, Oct 25, 2018 at 4:28 PM G. Allegri <giohappy at gmail.com> wrote:
>
> Python GIL  doess't guarantee thread safety, and it only applies to python code. It only protects the internal state of the interpreter.
> Moreover The C API offers the opportunity to release the GIL (see for example numpy [1]).
>
> Giovanni
>
> [1] https://stackoverflow.com/a/36480941
>
> Il giorno gio 25 ott 2018 alle ore 13:10 Alessandro Pasotti <apasotti at gmail.com> ha scritto:
>>
>>
>> I would go for multiprocess and stick to a single thread per-process.
>>
>> But David may have a better advice.
>>
>> Btw, it's nice to see a use case for Python Vector data providers :)
>>
>>
>>
>>
>> On Thu, Oct 25, 2018 at 1:02 PM Radim Blazek <radim.blazek at gmail.com> wrote:
>>>
>>> If we narrow down the environment to uWSGI (with multiple threads
>>> enabled) running a Python application, which means that more instances
>>> of the same application may run in threads but never at the same time
>>> due to Python GIL (if I understand it well), is it still a problem? If
>>> we ensure that there are no static variables and singletons used by
>>> QGIS server? Is it then a problem to create/delete QObject on a non
>>> main thread?
>>>
>>> Radim
>>> On Sun, Nov 19, 2017 at 5:39 PM Martin Dobias <wonder.sk at gmail.com> wrote:
>>> >
>>> > Hi Alessandro
>>> >
>>> > On Sun, Nov 19, 2017 at 10:34 AM, Alessandro Pasotti <apasotti at gmail.com> wrote:
>>> > > Hi,
>>> > >
>>> > > mi recent experiments with multi-threaded python wrappers for QGIS server
>>> > > showed a critical flaw in my original implementation of the server plugins.
>>> > >
>>> > > First, I want to stress that this is not a problem in FCGI server
>>> > > implementation but only if the server is used directly from python in a
>>> > > multi threaded server implementation.
>>> > >
>>> > > The problem is that the server interface that exposes the request handler is
>>> > > a static property of the interface, that is changed on every request.
>>> > > This means that there is a race condition in setting/accessing the handler
>>> > > from a plugin filter, and that a plugin filter might access to the handler
>>> > > for a wrong request.
>>> >
>>> > I think this is probably just the tip of the iceberg: most of the
>>> > classes in qgis_core are not thread-safe, and I believe the
>>> > implementation of server's services is not expected to be run in
>>> > multiple threads at once either. So even though things in theory could
>>> > work if requests are handled in multiple threads, most likely there
>>> > will be crashes. Even though map rendering was made to run safe in
>>> > multiple worker threads, there is still the assumption that the map
>>> > rendering is started from the main thread where all the objects live
>>> > (I'm talking about the QObject-based classes). Dealing with objects
>>> > like map layers or project in non-main thread is likely to end up
>>> > badly.
>>> >
>>> > I would say for now we should simply stick with handling only one
>>> > request per server process at a time - just like how it is done with
>>> > the FastCGI implementation.
>>> >
>>> > Cheers
>>> > Martin
>>> > _______________________________________________
>>> > QGIS-Developer mailing list
>>> > QGIS-Developer at lists.osgeo.org
>>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>>
>>
>> --
>> Alessandro Pasotti
>> w3:   www.itopen.it
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer