[Qgis-developer] How to debug and unit test Python plugins outside QGIS (e.g. in Ecplise IDE + PyDev)

Mon Mar 7 17:46:44 EST 2011

Hi Stefan, All,
coming from webdesign I use Aptana (eclipse derivation) with pydev with
success. Eclipse itself always managed to drive me crazy for some
reasons... I tried as well spe and erich and didn't like the feeling.
Ggeany and gedit with many plugins are very light but Aptana/pydev is
what suits my needs best.
ciao
Marco

On 03/06/2011 06:08 PM, Stefan Keller wrote:
> Hi Alex
> 
> You wrote:
>> I tried Eclipse with Pydev years back and felt that it was a dead end
>> for Python programming (also a bit of a resource hog). Partly because
>> all the useful stuff wasn't in the open source version, and partly
>> because Eclipse was a huge memory hole for something as simple as python.
> 
> => Which useful stuff so you mean?
> 
> Eclipse and PyDev are open source and my Eclipse starts in about 5
> seconds on my laptop (Win XP with an Intel Centrino and uses 200MB
> memory i.e. less than Firefox).
> 
>> Eric, SPE, and Spyder all seem like better options - 2 of the 3 are
>> written with PyQt also and I've gotten at least 2 of them to work with
>> OSGeo4W installs (Requires launching via a batch file).
> 
> The requirements of my "modern SW developer environment" are syntax
> highlighting, autocompletion, unit testing und debugging. Eclipse
> seems to deliver that. We also tested Eric4, SPE, PythonWin (Windows
> only), IDLE and PyPE where Eric and SPE seem to be most promising.
> Spyder, JEdit etc. are to me rather text programming editors, not
> IDEs.
> 
> Eclipse and probably all other IDEs/Editors can set PATHs to external
> libraries.
> => With which software did you get problems?
> 
> @all => What Python IDEs/Editors are other QGIS developers using?
> 
> As Martin said, it must be possible to get external compiling,
> debugging and unit testing with QGIS to work since there QGIS
> libraries can als be used for developing custom applications based on
> QGIS API. I'll report on that.
> 
> Yours, S.
> 
> 
> 2011/3/5 Alex Mandel <tech_dev at wildintellect.com>:
>> I tried Eclipse with Pydev years back and felt that it was a dead end
>> for Python programming (Also a bit of a resource hog). Partly because
>> all the useful stuff wasn't in the open source version, and partly
>> because Eclipse was a huge memory hole for something as simple as python.
>>
>> Eric, SPE, and Spyder all seem like better options - 2 of the 3 are
>> written with PyQt also and I've gotten at least 2 of them to work with
>> OSGeo4W installs (Requires launching via a batch file).
>>
>> I'm not so sure this would be a QGIS SOC project, however if it's just a
>> task in say creating unit tests that would make sense.
>>
>> Thanks,
>> Alex
>>
>> On 03/03/2011 12:26 AM, Stefan Keller wrote:
>>> I still hope this should be an issue to be solved within reasonable
>>> time given Eclipse as IDE.
>>> However, yes, it could be that this turns out to be a larger endeavour.
>>> I can imagine that one solution could be that Python stubs have to be
>>> generated out of PyGt4/QGIS libs?
>>>
>>> Yours, S.
>>>
>>> 2011/3/3 Andreas Neumann <a.neumann at carto.net>:
>>>> Couldn't this be a GSoC idea?
>>>>
>>>> Creating a developer-friendly Development environment for Python Developers?
>>>> With a nice debugging solution, code completion, possibility to do unit
>>>> tests, integration with the plugin repository, etc.?
>>>>
>>>> I would guess a lot of devs would gain productivity through such improved
>>>> developer tools and a student could learn quite a bit in putting these
>>>> together.
>>>>
>>>> Andreas
>>>>
>>>> On Thu, 3 Mar 2011 08:38:36 +0100, Stefan Keller wrote:
>>>>>
>>>>> Many thanks also for the hint to standalone apps.!
>>>>> I still hope there is a solution to unit test QGIS plugins.
>>>>> This would make QGIS programmers even more happy :->
>>>>>
>>>>> Yours, S.
>>>>>
>>>>>
>>>>> 2011/3/3 Alex Mandel <tech_dev at wildintellect.com>:
>>>>>>
>>>>>> It might be possible, I'm not super familiar with units tests yet.
>>>>>> The big issue I see is that most plugins rely on the main QGIS app
>>>>>> providing data to the plugin or interacting with the plugin. If you can
>>>>>> design the test to provide what the unit test needs then it should be
>>>>>> doable. After all you can build standalone python apps with the QGIS api.
>>>>>>
>>>>>> Thanks,
>>>>>> Alex
>>>>>>
>>>>>> On 03/02/2011 10:13 PM, Stefan Keller wrote:
>>>>>>>
>>>>>>> Hi Alex,
>>>>>>>
>>>>>>> Many thanks for your reply. I'll try this.
>>>>>>>
>>>>>>> But: So you say implicitly that its impossible to run unit tests
>>>>>>> (PyUnit) on QGIS plugins?
>>>>>>> I still hope there is a way to make PyGt4 module interface accessible
>>>>>>> outside QGIS. At runtime its there, so it should be somehow possible
>>>>>>> either by "Forced builtin libs" or by extracting the information by
>>>>>>> hand with dummy stubs...?
>>>>>>>
>>>>>>> Yours, S.
>>>>>>>
>>>>>>> 2011/3/3 Alex Mandel <tech_dev at wildintellect.com>:
>>>>>>>>
>>>>>>>> On 03/02/2011 03:57 PM, Stefan Keller wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm new to writing Python plugins for QGIS and I like to debug and
>>>>>>>>> unit test theses outside QGIS runtime but don't see any instructions
>>>>>>>>> on how to do this.
>>>>>>>>>
>>>>>>>>> Of course, the ultimate test of any plugin is to run it inside QGIS.
>>>>>>>>> But I'd like do that outside QGIS for example in Eclipse IDE (+ PyDev)
>>>>>>>>> in order to get code-completion, to debug it and to write unit tests
>>>>>>>>> on it.
>>>>>>>>>
>>>>>>>>> The problem is that one gets "ImportError: No module named ..." with
>>>>>>>>> mostly PyQt4/qgis modules which (of course) cannot be imported.
>>>>>>>>>
>>>>>>>>> This post mentions "Forced builtin libs" and reports a similar
>>>>>>>>> problem:
>>>>>>>>> http://www.osgeo.org/pipermail/qgis-developer/2010-February/009182.html
>>>>>>>>> ... and this PyDev documentation explains "Forced builtin libs" and
>>>>>>>>> "Predefined completions":
>>>>>>>>> http://pydev.org/manual_101_interpreter.html#id1
>>>>>>>>>
>>>>>>>>> I can't imagine that I'm the first trying to debug and unit test
>>>>>>>>> Python plugins for QGIS.
>>>>>>>>>
>>>>>>>>> Can anybody give me a hand on this?
>>>>>>>>>
>>>>>>>>> Yours, S.
>>>>>>>>
>>>>>>>> I've been meaning to write this up for some time. The only way I've
>>>>>>>> found to Debug a QGIS plugin is using a debugger hook in python. While
>>>>>>>> I
>>>>>>>> have only tested this method with SPE's Winpdb (No Win does not stand
>>>>>>>> for MS Windows - SPE is python cross platform) debugger it
>>>>>>>> theoretically
>>>>>>>> should be possible with other tools if they support the same kind of
>>>>>>>> debugging method.
>>>>>>>>
>>>>>>>> Basically I insert something like:
>>>>>>>> import rpdb2; rpdb2.start_embedded_debugger("testing")
>>>>>>>>
>>>>>>>> just below the main part of my plugin:
>>>>>>>> class MetaEditPlugin:
>>>>>>>>    def __init__(self,iface):
>>>>>>>>        #Only turn on to use winpdb debugger
>>>>>>>>        import rpdb2; rpdb2.start_embedded_debugger("testing")
>>>>>>>>
>>>>>>>> Then the procedure is to start QGIS, Launch the plugin, switch to SPE's
>>>>>>>> winpdb, then do File attach, put in the same password "testing".
>>>>>>>> It will then show you the running apps you can connect to, once you
>>>>>>>> select and connect the plugin will continue to run.
>>>>>>>> At this point you can set breakpoints and catch errors. If/When it
>>>>>>>> stops
>>>>>>>> at such a point you can interact with the app in a way similar to
>>>>>>>> Python's pdb sending eval and exec snippets, and inpecting all your
>>>>>>>> variables in the list.
>>>>>>>>
>>>>>>>> Make sure to comment out that line for production use otherwise your
>>>>>>>> plugin will always stall and wait for the debug to connect.
>>>>>>>>
>>>>>>>> Basically I think this falls under the term "Remote Debugging" if you
>>>>>>>> want to try and find ways to do it with other apps.
>>>>>>>>
>>>>>>>> I'm not sure there is any other practical method as most plugins rely
>>>>>>>> on
>>>>>>>> the QGIS built in stuff to be in an active session, so I wouldn't waste
>>>>>>>> too much time trying to do it without QGIS running at all.
>>>>>>>>
>>>>>>>> Enjoy,
>>>>>>>> Alex
>>
>>
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer

-- 
Marco Bernasocchi
www.bernawebdesign.ch
skype: mbernasocchi