[QGIS-Developer] Multiprocessing QGIS

[Sebastian M. Ernst]

Hi João, all,

really interesting question.

> however when i run the plugin a new instance of application is open
> and give me an error.

I assume that you are using `multiprocessing` [1] from Python's standard
library. Its default behavior is to use process-based parallelism (as
the title in the documentation says). I am assuming that you have not
altered its default behavior (e.g. by configuring it to use threads
instead). This is why you get at least another "worker" process if you
e.g. create a process pool. I have successfully used this method with
QGIS before, but always on Linux and OS X. I am just guessing that you
are doing this on Windows, right?

Depending on the operating system, `multiprocessing` uses different
methods to create worker processes. On Linux and OS X, it simply forks
[2] the main process. This way, you do not see a second application
window popping up. You "simply" get a second, third, fourth, ... process
"in the background". On Windows, the `fork` syscall does not exist,
neither does something similar [3]. `multiprocessing` therefore starts a
complete new process from scratch. If it was pure Python, it would
simply start a new Python interpreter process. Because it is QGIS, it
opens another instance of QGIS.

I am not entirely sure what QGIS' (intended) behavior is in a case like
this. From the top of my head, having read the relevant portion of its
source code, I'd expect at least a second application window likely
followed by non-deterministic behavior (again, depending on the
use-case, likely resulting in a crash). Can someone elaborate?

Best regards,
Sebastian


1: https://docs.python.org/3/library/multiprocessing.html
2: https://en.wikipedia.org/wiki/Fork_(system_call)
3: https://stackoverflow.com/q/985281/1672565


Am 23.03.21 um 13:08 schrieb Joao Folgado:
> Hi everyone,
> 
> I had created  a plugin for Qgis and I trying to optimize it. So i use
> in my code the library python that have Queue whit multiprocessing
> however when i run the plugin a new instance of application is open and
> give me an error.
> I had early use this type of multiprocessing before in simple projects
> in university and it works ok.
> Can someone help ? I researched in many forums, i see people whit the
> same question but not with the answer.
> Thank you very much.
> 
> Best regards,
> João Folgado
> Portugal
> -- 
> JOÃO FOLGADO
> 
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>