[Qgis-developer] pyqgis: parallel QThreads to services possible?

[Martin Dobias]

Hi Richard

On Wed, May 4, 2016 at 4:36 AM, Richard Duivenvoorde
<rdmailings at duif.net> wrote:
> Hi Devs,
>
> my plugin needs to fire as quick as possible:
> - 24 (hourly) WPS requests to service A, resulting in a dir with shp/zips
> - >3 (slow) paging WFS requests (30000 features) to service B resulting
> in 3 gml files
> If all this is finished, show all the results in QGIS
>
> mostly using this page:
>
> https://snorfalorpagus.net/blog/2013/12/07/multithreading-in-qgis-python-plugins/
>
> I created 'workers' for the services, and two different QThreads to be
> able to move the workers to.
>
> BUT when testing/running: I do not see 2 parallel service processes: I
> see first all 24 WPS request, and then the 3 slow WFS request (or the
> other way around if I change order of starting the threads...

Hmm maybe your problem is with the Global Interpreter Lock (GIL) in
Python. Even though Python supports multi-threading, it does not allow
multiple threads to execute Python code at the same time. In various
cases this is not a problem really. For example, thread #1 asks
operating system to do some I/O that may take some time, so Python
releases GIL from thread #1, and thread #2 can be resumed and acquire
GIL to run Python code, while thread #1 is waiting for the OS.

Now the question is how things really work in your code. If you use
some libraries for WFS/WPS access, they should probably handle GIL
correctly... it is a bit difficult to tell the exact reason without
seeing the code.

If things start to look complicated with threads, maybe you could try
python's 'multiprocessing' module instead:
https://docs.python.org/2/library/multiprocessing.html

Cheers
Martin