[Qgis-developer] Fwd: Threads in sextante

[Martin Dobias]

Hi Camilo!

On Thu, May 10, 2012 at 4:32 PM, Camilo Polymeris <cpolymeris at gmail.com> wrote:
> Hello all,
> one of the things I will be implementing for sextante during GSoC will
> be multithreading. I have experimented a bit with the possibilities
> during the last few days, but it would be good to know your opinion on
> what would be the simplest, cleanest solution.
>
> At this point I feel the obvious place to introduce this functionality
> is the AlgorithmExecutor class, as suggested by Victor, converting
> that to an instantiable class, inheriting from QThread.
> I tend to favor QThread over the python standard threads, because its
> signaling system is familiar to all working with Qt and would interact
> better with the existing GUI code.

What should be the benefit of multithreading here? To run algorithms
outside from main thread so that GUI does not get blocked? Or mainly
to allow running several algorithms in parallel at once? Or both?

Multithreading always comes with some cost... you need to make sure
that data you are accessing in one thread are not being accessed also
in some other thread... and this can be a lot of trouble to do it
properly.

One more thing about threading and Python - there is the infamous
global interpreter lock which prevents native threads to execute
Python code at once. So even if you had several threads, only one of
them can run Python at one time. Most of the time this probably won't
cause problems since lots of algorithms would just spawn some routines
in other native libraries, but it is good to know about this
limitation.

Finally, apart from QThread there is a Qt framework called
QtConcurrent that brings high-level API for multithreading. With
QtConcurrent you do not have to manually spawn and watch threads, it
limits the amount of running threads to the number of cores, it
supports some common multithreading patterns like map-reduce etc. This
framework is however available only for C++, so this means that the
core sextante interfaces would have to be moved to c++ (which would be
good IMO).

Cheers
Martin