[Qgis-developer] How to debug and unit test Python plugins outside QGIS (e.g. in Ecplise IDE + PyDev)

Wed Mar 9 05:17:06 EST 2011

On Mon, 2011-03-07 at 23:46 +0100, Marco Bernasocchi wrote:
> Hi Stefan, All,
> coming from webdesign I use Aptana (eclipse derivation) with pydev with
> success. Eclipse itself always managed to drive me crazy for some
> reasons... I tried as well spe and erich and didn't like the feeling.
> Ggeany and gedit with many plugins are very light but Aptana/pydev is
> what suits my needs best.

Hi all,

I added "Python plugin development tools" among topics of next code
sprint. I'll try to get some debugging/testing tools working for that
moment. Here are what we have so far:

http://www.qgis.org/wiki/Writing_Python_Plugins
http://www.qgis.org/wiki/How_to_debug_QGIS_Plugins

regards,
Anne

> ciao
> Marco
> 
> On 03/06/2011 06:08 PM, Stefan Keller wrote:
> > Hi Alex
> > 
> > You wrote:
> >> I tried Eclipse with Pydev years back and felt that it was a dead end
> >> for Python programming (also a bit of a resource hog). Partly because
> >> all the useful stuff wasn't in the open source version, and partly
> >> because Eclipse was a huge memory hole for something as simple as python.
> > 
> > => Which useful stuff so you mean?
> > 
> > Eclipse and PyDev are open source and my Eclipse starts in about 5
> > seconds on my laptop (Win XP with an Intel Centrino and uses 200MB
> > memory i.e. less than Firefox).
> > 
> >> Eric, SPE, and Spyder all seem like better options - 2 of the 3 are
> >> written with PyQt also and I've gotten at least 2 of them to work with
> >> OSGeo4W installs (Requires launching via a batch file).
> > 
> > The requirements of my "modern SW developer environment" are syntax
> > highlighting, autocompletion, unit testing und debugging. Eclipse
> > seems to deliver that. We also tested Eric4, SPE, PythonWin (Windows
> > only), IDLE and PyPE where Eric and SPE seem to be most promising.
> > Spyder, JEdit etc. are to me rather text programming editors, not
> > IDEs.
> > 
> > Eclipse and probably all other IDEs/Editors can set PATHs to external
> > libraries.
> > => With which software did you get problems?
> > 
> > @all => What Python IDEs/Editors are other QGIS developers using?
> > 
> > As Martin said, it must be possible to get external compiling,
> > debugging and unit testing with QGIS to work since there QGIS
> > libraries can als be used for developing custom applications based on
> > QGIS API. I'll report on that.
> > 
> > Yours, S.
> > 
> > 
> > 2011/3/5 Alex Mandel <tech_dev at wildintellect.com>:
> >> I tried Eclipse with Pydev years back and felt that it was a dead end
> >> for Python programming (Also a bit of a resource hog). Partly because
> >> all the useful stuff wasn't in the open source version, and partly
> >> because Eclipse was a huge memory hole for something as simple as python.
> >>
> >> Eric, SPE, and Spyder all seem like better options - 2 of the 3 are
> >> written with PyQt also and I've gotten at least 2 of them to work with
> >> OSGeo4W installs (Requires launching via a batch file).
> >>
> >> I'm not so sure this would be a QGIS SOC project, however if it's just a
> >> task in say creating unit tests that would make sense.
> >>
> >> Thanks,
> >> Alex
> >>
> >> On 03/03/2011 12:26 AM, Stefan Keller wrote:
> >>> I still hope this should be an issue to be solved within reasonable
> >>> time given Eclipse as IDE.
> >>> However, yes, it could be that this turns out to be a larger endeavour.
> >>> I can imagine that one solution could be that Python stubs have to be
> >>> generated out of PyGt4/QGIS libs?
> >>>
> >>> Yours, S.
> >>>
> >>> 2011/3/3 Andreas Neumann <a.neumann at carto.net>:
> >>>> Couldn't this be a GSoC idea?
> >>>>
> >>>> Creating a developer-friendly Development environment for Python Developers?
> >>>> With a nice debugging solution, code completion, possibility to do unit
> >>>> tests, integration with the plugin repository, etc.?
> >>>>
> >>>> I would guess a lot of devs would gain productivity through such improved
> >>>> developer tools and a student could learn quite a bit in putting these
> >>>> together.
> >>>>
> >>>> Andreas
> >>>>
> >>>> On Thu, 3 Mar 2011 08:38:36 +0100, Stefan Keller wrote:
> >>>>>
> >>>>> Many thanks also for the hint to standalone apps.!
> >>>>> I still hope there is a solution to unit test QGIS plugins.
> >>>>> This would make QGIS programmers even more happy :->
> >>>>>
> >>>>> Yours, S.
> >>>>>
> >>>>>
> >>>>> 2011/3/3 Alex Mandel <tech_dev at wildintellect.com>:
> >>>>>>
> >>>>>> It might be possible, I'm not super familiar with units tests yet.
> >>>>>> The big issue I see is that most plugins rely on the main QGIS app
> >>>>>> providing data to the plugin or interacting with the plugin. If you can
> >>>>>> design the test to provide what the unit test needs then it should be
> >>>>>> doable. After all you can build standalone python apps with the QGIS api.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Alex
> >>>>>>
> >>>>>> On 03/02/2011 10:13 PM, Stefan Keller wrote:
> >>>>>>>
> >>>>>>> Hi Alex,
> >>>>>>>
> >>>>>>> Many thanks for your reply. I'll try this.
> >>>>>>>
> >>>>>>> But: So you say implicitly that its impossible to run unit tests
> >>>>>>> (PyUnit) on QGIS plugins?
> >>>>>>> I still hope there is a way to make PyGt4 module interface accessible
> >>>>>>> outside QGIS. At runtime its there, so it should be somehow possible
> >>>>>>> either by "Forced builtin libs" or by extracting the information by
> >>>>>>> hand with dummy stubs...?
> >>>>>>>
> >>>>>>> Yours, S.
> >>>>>>>
> >>>>>>> 2011/3/3 Alex Mandel <tech_dev at wildintellect.com>:
> >>>>>>>>
> >>>>>>>> On 03/02/2011 03:57 PM, Stefan Keller wrote:
> >>>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> I'm new to writing Python plugins for QGIS and I like to debug and
> >>>>>>>>> unit test theses outside QGIS runtime but don't see any instructions
> >>>>>>>>> on how to do this.
> >>>>>>>>>
> >>>>>>>>> Of course, the ultimate test of any plugin is to run it inside QGIS.
> >>>>>>>>> But I'd like do that outside QGIS for example in Eclipse IDE (+ PyDev)
> >>>>>>>>> in order to get code-completion, to debug it and to write unit tests
> >>>>>>>>> on it.
> >>>>>>>>>
> >>>>>>>>> The problem is that one gets "ImportError: No module named ..." with
> >>>>>>>>> mostly PyQt4/qgis modules which (of course) cannot be imported.
> >>>>>>>>>
> >>>>>>>>> This post mentions "Forced builtin libs" and reports a similar
> >>>>>>>>> problem:
> >>>>>>>>> http://www.osgeo.org/pipermail/qgis-developer/2010-February/009182.html
> >>>>>>>>> ... and this PyDev documentation explains "Forced builtin libs" and
> >>>>>>>>> "Predefined completions":
> >>>>>>>>> http://pydev.org/manual_101_interpreter.html#id1
> >>>>>>>>>
> >>>>>>>>> I can't imagine that I'm the first trying to debug and unit test
> >>>>>>>>> Python plugins for QGIS.
> >>>>>>>>>
> >>>>>>>>> Can anybody give me a hand on this?
> >>>>>>>>>
> >>>>>>>>> Yours, S.
> >>>>>>>>
> >>>>>>>> I've been meaning to write this up for some time. The only way I've
> >>>>>>>> found to Debug a QGIS plugin is using a debugger hook in python. While
> >>>>>>>> I
> >>>>>>>> have only tested this method with SPE's Winpdb (No Win does not stand
> >>>>>>>> for MS Windows - SPE is python cross platform) debugger it
> >>>>>>>> theoretically
> >>>>>>>> should be possible with other tools if they support the same kind of
> >>>>>>>> debugging method.
> >>>>>>>>
> >>>>>>>> Basically I insert something like:
> >>>>>>>> import rpdb2; rpdb2.start_embedded_debugger("testing")
> >>>>>>>>
> >>>>>>>> just below the main part of my plugin:
> >>>>>>>> class MetaEditPlugin:
> >>>>>>>>    def __init__(self,iface):
> >>>>>>>>        #Only turn on to use winpdb debugger
> >>>>>>>>        import rpdb2; rpdb2.start_embedded_debugger("testing")
> >>>>>>>>
> >>>>>>>> Then the procedure is to start QGIS, Launch the plugin, switch to SPE's
> >>>>>>>> winpdb, then do File attach, put in the same password "testing".
> >>>>>>>> It will then show you the running apps you can connect to, once you
> >>>>>>>> select and connect the plugin will continue to run.
> >>>>>>>> At this point you can set breakpoints and catch errors. If/When it
> >>>>>>>> stops
> >>>>>>>> at such a point you can interact with the app in a way similar to
> >>>>>>>> Python's pdb sending eval and exec snippets, and inpecting all your
> >>>>>>>> variables in the list.
> >>>>>>>>
> >>>>>>>> Make sure to comment out that line for production use otherwise your
> >>>>>>>> plugin will always stall and wait for the debug to connect.
> >>>>>>>>
> >>>>>>>> Basically I think this falls under the term "Remote Debugging" if you
> >>>>>>>> want to try and find ways to do it with other apps.
> >>>>>>>>
> >>>>>>>> I'm not sure there is any other practical method as most plugins rely
> >>>>>>>> on
> >>>>>>>> the QGIS built in stuff to be in an active session, so I wouldn't waste
> >>>>>>>> too much time trying to do it without QGIS running at all.
> >>>>>>>>
> >>>>>>>> Enjoy,
> >>>>>>>> Alex
> >>
> >>
> >> _______________________________________________
> >> Qgis-developer mailing list
> >> Qgis-developer at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/qgis-developer
> >>
> > _______________________________________________
> > Qgis-developer mailing list
> > Qgis-developer at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/qgis-developer
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
Url : http://lists.osgeo.org/pipermail/qgis-developer/attachments/20110309/cb680f6f/attachment.bin