<div dir="ltr"><div><div><div><div><div>Hi Angus<br><br></div>We have met for sure. How ya doing? Yes, the simulation model will be open source :). In the longer term it would be nice to make this be part of a larger forest simulation, including links to a spatial management model, but that is not the immediate priority.<br>
<br></div>Thanks for the tips about multiple processes and shared memory. Hmmmm... I suspect that it might be slightly tidier to keep this all within one process, at least I would like to give it a try to start with. Once I have built up enough callous tissue on my forehead maybe I'll come back to this.<br>
<br>As you can imagine from my newbie questions I'm not a python guy. While I really like a lot of the modern mechanisms in python I have never been able to look at a python script without being puzzled by the retro/hipster nature of using whitespace as syntax. As someone who once lined up fortran statements on punch cards you will have to pry whitespace indifferent languages from my cold hunt-n-peck fingers. Anyway, I'm expecting to use python for the ui and glue code, and do all of the number crunching in a C module. Horses for courses eh?<br>
<br>Good point about the global interpreter lock. This might not be a problem because I should be able to instantiate the simulator in the main interpreter, and then kick it off into its own compute thread (PyQt QThread I guess?). It looks like the GIL gets released when entering compiled code (<a href="http://stackoverflow.com/questions/1595649/threading-in-a-pyqt-application-use-qt-threads-or-python-threads" target="_blank">http://stackoverflow.com/questions/1595649/threading-in-a-pyqt-application-use-qt-threads-or-python-threads)</a>,and I should be able to use the Qt QueuedConnnection signal/slot mechanism from within C to safely communicate back and forth between the worker thread and the app. But regardless, the simulation code wont need to interact with python once it has been initialized other than a few housekeeping bits that can probably be handled using signals and handlers in the C code (send status to app, accept pause/continue/terminate in the worker thread, etc). I think I'm OK here.<br>
<br>With respect to my previous point 2) I'm hoping that I can subclass the raster data source, and add a QMutex/critical section around any calls to fetch data through GDALDataset::RasterIO. The simulator would use the same critical section around the places where it is modifying shared data structures.<br>
<br></div>I assume that I handle 3) using a signal/slot mechanism: set the dataChanged() state on the map layer in the simulation thread and then send a qt signal to the app that it needs to refresh the map canvas. Can anyone confirm if this is the way to do it?<br>
<br></div>As for my previous point 4) (map layer image caching and map canvas compositing), I was hoping that someone could point me towards some documentation about this apparent feature. Does anyone know any more about this one? Although I can just give it a try, it would be nice to know what I'm getting in to before I get too far down the road.<br>
<br></div><div>Thanks,<br><br></div>Tom<br><div><div><div><br></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Nov 20, 2013 at 1:43 PM, Angus Carr <span dir="ltr"><<a href="mailto:angus.carr@gmail.com" target="_blank">angus.carr@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div>Hi, Tom. Your PyQT plugin would stall the main thread with a long process. You can launch threads, but that would still cause problems because of the python interpreter lock, which only allows a single thread access at once, so stall other threads.<br>
<br></div>You will need to launch a process to calculate the results. <br><br></div>There are mechanisms in python to chat between the processes in shared memory. I've played with some of them, and they can vary between fast and slow, depending on the details- it can still lock your main thread if you do it wrong.<br>
<br></div>I had a multi-process app filling four cores with analysis, but it would slow down once the individual runs started bringing data back to the central collator. It would block the queue of all process, bringing performance to a crawl. <br>
<br></div>If you are doing what I hope you are doing, then I would love to participate in bringing an FBP fire simulator to QGIS in open code. Particularly if it can then be used to do crazy things like be a process step in a forest simulation...<br>
<br></div>Yes, I am a fellow Canadian QGIS user, and I believe we have met, although it's been a while... :-). I think we had a beer in Montana's in Thunder Bay in 2008, if that helps... :-)<br><br></div>If you were interesting in collaborating, drop me a direct email. As long as it is open source, I can play. If it's tight to your company, I will have to defer.<br>
<br></div>Cheers,<br>Angus Carr.<br><div><div><br><br><br></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Wed, Nov 20, 2013 at 1:01 PM, Tom Moore <span dir="ltr"><<a href="mailto:moortom@gmail.com" target="_blank">moortom@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr"><div><div><div><div><div><div>Greetings everyone! I am new to the list and the QGIS community, and I am looking for a bit of strategic direction from the experts. I am building a landscape-level wildfire simulation model. The model will simulate change and development of a natural forested landscape over centuries of time as forest grow and wildfires occur. The mechanics of the model are fairly simple, and I will likely implement these in a c module for reasonable performance. The model simulates wildfire ignition and spread on a raster surface, with generations representing future time periods. As the model runs it will gather statistics and save information about conditions of each generation.<br>
<br></div>I am considering using QGIS as a platform for the simulation model. I think there would be some nice benefits of making my model be a python-based plugin that could leverage a lot of the data handling involved with the spatial data sets, plus it looks like I could build an attractive and function ui quite painlessly. Here are some of the features that I would like that I am pretty sure I can get from the QGIS platform:<br>
</div><div>- easy access to raster data sets via gdal.<br></div><div>- easy way to capture input parameters (data sources, regions of interest, simulation model options) using pyQt and qgis platfrom.<br></div><div>- nice way to display final results from simulation, by save results out to raster data set and then displaying in map canvas<br>
- quite likely easy ways to display summary results as charts and histograms.<br></div><div>So far +++ for QGIS.<br><br></div>I have done a little bit of research to find out what it will take, and I think the pieces are there. But there are a couple of things that I am not sure about and I would like to ask the experts before I dig myself in to a hole.<br>
<br></div>The simulation model will take a long time to run and I don't want it to stall the GUI as it is performing. <br>1) Does qg support background processing? Where can I read about the synchronization mechanism? If not, what is the best way to handle this?<br>
<br></div>It would be really nice to be able to have a dynamic display of the ongoing simulation as it is running. I was thinking that I could use a GDAL in-memory driver (<a href="http://www.gdal.org/frmt_mem.html" target="_blank">http://www.gdal.org/frmt_mem.html</a>) to hold simulated conditions.<br>
</div>2) If I did this, could I display this type of raster in the qg map canvas? What kind of synchronization is required so the simulation pauses when the map is redrawn due to user interaction (zooming or panning for example).<br>
<br></div>3) How will I be able to signal the map canvas when it is time for an update? Is QgsMapLayer.dataChanged() sufficient? I see that someone else has tried this and had difficulty: <a href="http://article.gmane.org/gmane.comp.gis.qgis.devel/21481" target="_blank">http://article.gmane.org/gmane.comp.gis.qgis.devel/21481</a>. I assume that the problem there was that the method was stalling the event loop. What are the rules for event loop interactions?<br>
<div><div><div><div><div><br></div><div>4) Can I update my raster map layer without having to re-draw the other layers that may be on the the map canvas? Does the map canvas support compositing? I see that QgsMapLayer supports an image cache, but I have not found any explanation of how to use it.<br>
<br></div><div>At this stage I'm looking for some clues. I think there may be some multi-threading issues to my design that require careful attention. I see that multi threading has been discussed in the context of rendering (<a href="http://hub.qgis.org/issues/2037" target="_blank">http://hub.qgis.org/issues/2037</a>) but I am not sure about the current status, or if this is the solution to my problem (perhaps not). Any suggestions, opinions or pointers to code or documentation would be greatly appreciated. If this is just a case of rtfm then I would be happy to hear that also.<br>
<br></div><div>Cheers,<br><br>Tom<br></div></div></div></div></div></div>
<br></div></div>_______________________________________________<br>
Qgis-developer mailing list<br>
<a href="mailto:Qgis-developer@lists.osgeo.org" target="_blank">Qgis-developer@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer" target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br></blockquote></div><br></div>
</blockquote></div><br></div>