[Qgis-developer] A roadmap to QGIS 2.0

[Marco Hugentobler]

Hi Martin

> - identify which parts of the rendering stack are in python and
> explicitly run rendering of these layers in main branch. This would
> actually avoid the whole problem and might work fine. I haven't
> investigated this option yet, I'm mainly interested in an simple way
> how to find out if a class is implemented in python.
> I would appreciate any comments for that.

Yes, difficult problem with the GIL.
A simple solution would be to check the loaded python plugins and see if there 
are any with type RENDERER or MAPLAYER (maybe it would need an additional type 
for symbollayer plugins too). If so, do the rendering single threaded.

I know, there are disvantages of this approach. E.g. people could forget to 
unload the openlayers plugin even if they don't use such a layer in the 
current project. Or a plugin could use the wrong declaration and block QGIS 
(but plugins can also block QGIS because of other bugs).

However, it seems much simpler to me compared to force the python rendering 
components to the main branch.

Regards,
Marco

Am Freitag, 4. Februar 2011, um 13.32:38 schrieb Martin Dobias:
> Hi Juergen
> 
> On Fri, Feb 4, 2011 at 11:27 AM, Jürgen E. <jef at norbit.de> wrote:
> > Did I miss Martins comments on that?  I didn't find any.
> 
> Sorry for not joining the discussion, first I was too busy, then I've
> been away, so after 14 days I thought it's too late to join the
> discussion :-)
> Anyway I'm going to share my thoughts now.
> 
> I agree with Tim's goals for the 2.0 release. I also agree with the
> point that we should do few releases (e.g. 1.9.x) before 2.0 that
> would incrementally throw out old stuff and bring a lot of polish, so
> that 2.0 will be merely a renamed 1.9.x version - in order to have a
> solid release, not a feature-packed unstable release. (Probably many
> people will give 2.0 a try and if it's buggy they'll wait again for
> 3.0). At the same time I would be a bit more modest regarding the
> timeline, but as Tim noted, it looks like everyone has a different
> opinion on that and his plan is quite reasonable.
> 
> One of the many things that I would like to see in 2.0 is a reworked
> geometry class. I hope everyone who knows the inner working of
> QgsGeometry will agree with me :-) I think boost.geometry library
> (former GGL) is a strong candidate for the backend, since it starts to
> mature and has several advantages over GEOS:
> - allows to use our own geometry classes - no need to copy geometries
> back and forth
> - it's pure header library - so it's only build dependency, not
> runtime dependency
> It could miss some functionality that is available in geos, though - I
> haven't done an extensive comparison yet.
> 
> One of the important things is how/when to merge the threading branch.
> I know that just few people tried it, so I didn't get much feedback
> apart from Marco's and Pirmin's comments. As of now, I am aware of
> just one important blocker that prevents me to merge the branch (there
> will be probably few bugs, but I don't think of them as critical).
> 
> The blocker is the issue of threading vs python. Python in general is
> not a great friend with threading - it has an infamous Global
> Interpreter Lock (GIL) which ensures that only one thread in
> application can run Python code. To enable threaded applications, GIL
> is unlocked periodically (when running python code) and locked again
> to give other threads a chance to run  The problem in QGIS shows up if
> any part of the rendering stack is in Python. That could be a custom
> symbol layer instance, custom renderer or a custom "plugin" layer.
> Rendering freezes at that point, because Python code is waiting for
> GIL, but it doesn't receive it since no Python code is running either
> (in main thread) that could unlock the GIL. The question is how to
> handle this situation. I had few ideas how to solve this:
> - run periodically some python code in main thread, so GIL could be
> acquired by rendering thread. This is lame: if called often, it eats a
> lot of cpu, if called rarely the code in rendering thread progresses
> slowly.
> - unlock GIL manually when rendering starts and lock it again when
> rendering finished. Until now I have failed to produce a result which
> wouldn't crash/deadlock when tried running python code in main thread
> while rendering thread is running python code. (As someone noted on
> one mailing list, python documentation for this topic is clear as
> mud).
> - identify which parts of the rendering stack are in python and
> explicitly run rendering of these layers in main branch. This would
> actually avoid the whole problem and might work fine. I haven't
> investigated this option yet, I'm mainly interested in an simple way
> how to find out if a class is implemented in python.
> I would appreciate any comments for that.
> 
> Among the smaller issues, there should be more aggressive locking to
> prevent data corruption when rendering and modifying the data at the
> same time (e.g. vector edit buffer, raster/vector layer properties). I
> also had to remove connection pooling for postgis layers from one
> server, since it's not possible to read data for more than one layer
> at once from one connection. But these things can be fixed as the time
> goes.
> 
> > Did anyone try merges yet?
> 
> I haven't tried to merge the code yet.
> 
> I would recommend to merge Radim's raster providers branch prior to
> the threading branch, since it's much simpler. However then we will
> have to make sure that rasters work correctly with the threading.
> 
> > I've was going to try - but I all I managed so far is to get git (and
> > that took quite some amount of persuasiveness to get there - git rookie
> > here) to checkout trunk, rasterproviders, threading and table branch
> > into one repository.
> > 
> > I'd like to work more on the forms stuff and I think the table join
> > branch could be a prerequisite for that.
> > 
> > Anything I can do to help with the merges?
> 
> It will be useful to merge the table join branch to threading branch
> first, and then update it with the zillion of commits that happened in
> trunk in meanwhile. With that help, I think threading branch could be
> merged to trunk much sooner. I would really appreciate that!
> 
> Regards
> Martin
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer


-- 
Dr. Marco Hugentobler
Sourcepole - Linux & Open Source Solutions
Churerstr. 22, CH-8808 Pfäffikon SZ, Switzerland
marco.hugentobler at sourcepole.ch http://www.sourcepole.ch
Technical Advisor QGIS Project Steering Committee