[Qgis-developer] Multi threading and background processing

[Tom Moore]

Greetings everyone!  I am new to the list and the QGIS community, and I am
looking for a bit of strategic direction from the experts.  I am building a
landscape-level wildfire simulation model.  The model will simulate change
and development of a natural forested landscape over centuries of time as
forest grow and wildfires occur.  The mechanics of the model are fairly
simple, and I will likely implement these in a c module for reasonable
performance.  The model simulates wildfire ignition and spread on a raster
surface, with generations representing future time periods.  As the model
runs it will gather statistics and save information about conditions of
each generation.

I am considering using QGIS as a platform for the simulation model.  I
think there would be some nice benefits of making my model be a
python-based plugin that could leverage a lot of the data handling involved
with the spatial data sets, plus it looks like I could build an attractive
and function ui quite painlessly.  Here are some of the features that I
would like that I am pretty sure I can get from the QGIS platform:
- easy access to raster data sets via gdal.
- easy way to capture input parameters (data sources, regions of interest,
simulation model options) using pyQt and qgis platfrom.
- nice way to display final results from simulation, by save results out to
raster data set and then displaying in map canvas
- quite likely easy ways to display summary results as charts and
histograms.
So far +++ for QGIS.

I have done a little bit of research to find out what it will take, and I
think the pieces are there. But there are a couple of things that I am not
sure about and I would like to ask the experts before I dig myself in to a
hole.

The simulation model will take a long time to run and I don't want it to
stall the GUI as it is performing.
1) Does qg support background processing?  Where can I read about the
synchronization mechanism?  If not, what is the best way to handle this?

It would be really nice to be able to have a dynamic display of the ongoing
simulation as it is running.  I was thinking that I could use a GDAL
in-memory driver (http://www.gdal.org/frmt_mem.html) to hold simulated
conditions.
2) If I did this, could I display this type of raster in the qg map
canvas?  What kind of synchronization is required so the simulation pauses
when the map is redrawn due to user interaction (zooming or panning for
example).

3) How will I be able to signal the map canvas when it is time for an
update?  Is QgsMapLayer.dataChanged() sufficient?   I see that someone else
has tried this and had difficulty:
http://article.gmane.org/gmane.comp.gis.qgis.devel/21481.  I assume that
the problem there was that the method was stalling the event loop.  What
are the rules for event loop interactions?

4) Can I update my raster map layer without having to re-draw the other
layers that may be on the the map canvas?  Does the map canvas support
compositing?  I see that QgsMapLayer supports an image cache, but I have
not found any explanation of how to use it.

At this stage I'm looking for some clues.  I think there may be some
multi-threading issues to my design that require careful attention.  I see
that multi threading has been discussed in the context of rendering (
http://hub.qgis.org/issues/2037) but I am not sure about the current
status, or if this is the solution to my problem (perhaps not).  Any
suggestions, opinions or pointers to code or documentation would be greatly
appreciated.  If this is just a case of rtfm then I would be happy to hear
that also.

Cheers,

Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20131120/d7fda488/attachment-0001.html>