[Qgis-developer] How to debug and unit test Python plugins outside QGIS (e.g. in Ecplise IDE + PyDev)

Sun Mar 6 12:08:16 EST 2011

Hi Alex

You wrote:
> I tried Eclipse with Pydev years back and felt that it was a dead end
> for Python programming (also a bit of a resource hog). Partly because
> all the useful stuff wasn't in the open source version, and partly
> because Eclipse was a huge memory hole for something as simple as python.

=> Which useful stuff so you mean?

Eclipse and PyDev are open source and my Eclipse starts in about 5
seconds on my laptop (Win XP with an Intel Centrino and uses 200MB
memory i.e. less than Firefox).

> Eric, SPE, and Spyder all seem like better options - 2 of the 3 are
> written with PyQt also and I've gotten at least 2 of them to work with
> OSGeo4W installs (Requires launching via a batch file).

The requirements of my "modern SW developer environment" are syntax
highlighting, autocompletion, unit testing und debugging. Eclipse
seems to deliver that. We also tested Eric4, SPE, PythonWin (Windows
only), IDLE and PyPE where Eric and SPE seem to be most promising.
Spyder, JEdit etc. are to me rather text programming editors, not
IDEs.

Eclipse and probably all other IDEs/Editors can set PATHs to external
libraries.
=> With which software did you get problems?

@all => What Python IDEs/Editors are other QGIS developers using?

As Martin said, it must be possible to get external compiling,
debugging and unit testing with QGIS to work since there QGIS
libraries can als be used for developing custom applications based on
QGIS API. I'll report on that.

Yours, S.

2011/3/5 Alex Mandel <tech_dev at wildintellect.com>:
> I tried Eclipse with Pydev years back and felt that it was a dead end
> for Python programming (Also a bit of a resource hog). Partly because
> all the useful stuff wasn't in the open source version, and partly
> because Eclipse was a huge memory hole for something as simple as python.
>
> Eric, SPE, and Spyder all seem like better options - 2 of the 3 are
> written with PyQt also and I've gotten at least 2 of them to work with
> OSGeo4W installs (Requires launching via a batch file).
>
> I'm not so sure this would be a QGIS SOC project, however if it's just a
> task in say creating unit tests that would make sense.
>
> Thanks,
> Alex
>
> On 03/03/2011 12:26 AM, Stefan Keller wrote:
>> I still hope this should be an issue to be solved within reasonable
>> time given Eclipse as IDE.
>> However, yes, it could be that this turns out to be a larger endeavour.
>> I can imagine that one solution could be that Python stubs have to be
>> generated out of PyGt4/QGIS libs?
>>
>> Yours, S.
>>
>> 2011/3/3 Andreas Neumann <a.neumann at carto.net>:
>>> Couldn't this be a GSoC idea?
>>>
>>> Creating a developer-friendly Development environment for Python Developers?
>>> With a nice debugging solution, code completion, possibility to do unit
>>> tests, integration with the plugin repository, etc.?
>>>
>>> I would guess a lot of devs would gain productivity through such improved
>>> developer tools and a student could learn quite a bit in putting these
>>> together.
>>>
>>> Andreas
>>>
>>> On Thu, 3 Mar 2011 08:38:36 +0100, Stefan Keller wrote:
>>>>
>>>> Many thanks also for the hint to standalone apps.!
>>>> I still hope there is a solution to unit test QGIS plugins.
>>>> This would make QGIS programmers even more happy :->
>>>>
>>>> Yours, S.
>>>>
>>>>
>>>> 2011/3/3 Alex Mandel <tech_dev at wildintellect.com>:
>>>>>
>>>>> It might be possible, I'm not super familiar with units tests yet.
>>>>> The big issue I see is that most plugins rely on the main QGIS app
>>>>> providing data to the plugin or interacting with the plugin. If you can
>>>>> design the test to provide what the unit test needs then it should be
>>>>> doable. After all you can build standalone python apps with the QGIS api.
>>>>>
>>>>> Thanks,
>>>>> Alex
>>>>>
>>>>> On 03/02/2011 10:13 PM, Stefan Keller wrote:
>>>>>>
>>>>>> Hi Alex,
>>>>>>
>>>>>> Many thanks for your reply. I'll try this.
>>>>>>
>>>>>> But: So you say implicitly that its impossible to run unit tests
>>>>>> (PyUnit) on QGIS plugins?
>>>>>> I still hope there is a way to make PyGt4 module interface accessible
>>>>>> outside QGIS. At runtime its there, so it should be somehow possible
>>>>>> either by "Forced builtin libs" or by extracting the information by
>>>>>> hand with dummy stubs...?
>>>>>>
>>>>>> Yours, S.
>>>>>>
>>>>>> 2011/3/3 Alex Mandel <tech_dev at wildintellect.com>:
>>>>>>>
>>>>>>> On 03/02/2011 03:57 PM, Stefan Keller wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I'm new to writing Python plugins for QGIS and I like to debug and
>>>>>>>> unit test theses outside QGIS runtime but don't see any instructions
>>>>>>>> on how to do this.
>>>>>>>>
>>>>>>>> Of course, the ultimate test of any plugin is to run it inside QGIS.
>>>>>>>> But I'd like do that outside QGIS for example in Eclipse IDE (+ PyDev)
>>>>>>>> in order to get code-completion, to debug it and to write unit tests
>>>>>>>> on it.
>>>>>>>>
>>>>>>>> The problem is that one gets "ImportError: No module named ..." with
>>>>>>>> mostly PyQt4/qgis modules which (of course) cannot be imported.
>>>>>>>>
>>>>>>>> This post mentions "Forced builtin libs" and reports a similar
>>>>>>>> problem:
>>>>>>>> http://www.osgeo.org/pipermail/qgis-developer/2010-February/009182.html
>>>>>>>> ... and this PyDev documentation explains "Forced builtin libs" and
>>>>>>>> "Predefined completions":
>>>>>>>> http://pydev.org/manual_101_interpreter.html#id1
>>>>>>>>
>>>>>>>> I can't imagine that I'm the first trying to debug and unit test
>>>>>>>> Python plugins for QGIS.
>>>>>>>>
>>>>>>>> Can anybody give me a hand on this?
>>>>>>>>
>>>>>>>> Yours, S.
>>>>>>>
>>>>>>> I've been meaning to write this up for some time. The only way I've
>>>>>>> found to Debug a QGIS plugin is using a debugger hook in python. While
>>>>>>> I
>>>>>>> have only tested this method with SPE's Winpdb (No Win does not stand
>>>>>>> for MS Windows - SPE is python cross platform) debugger it
>>>>>>> theoretically
>>>>>>> should be possible with other tools if they support the same kind of
>>>>>>> debugging method.
>>>>>>>
>>>>>>> Basically I insert something like:
>>>>>>> import rpdb2; rpdb2.start_embedded_debugger("testing")
>>>>>>>
>>>>>>> just below the main part of my plugin:
>>>>>>> class MetaEditPlugin:
>>>>>>>    def __init__(self,iface):
>>>>>>>        #Only turn on to use winpdb debugger
>>>>>>>        import rpdb2; rpdb2.start_embedded_debugger("testing")
>>>>>>>
>>>>>>> Then the procedure is to start QGIS, Launch the plugin, switch to SPE's
>>>>>>> winpdb, then do File attach, put in the same password "testing".
>>>>>>> It will then show you the running apps you can connect to, once you
>>>>>>> select and connect the plugin will continue to run.
>>>>>>> At this point you can set breakpoints and catch errors. If/When it
>>>>>>> stops
>>>>>>> at such a point you can interact with the app in a way similar to
>>>>>>> Python's pdb sending eval and exec snippets, and inpecting all your
>>>>>>> variables in the list.
>>>>>>>
>>>>>>> Make sure to comment out that line for production use otherwise your
>>>>>>> plugin will always stall and wait for the debug to connect.
>>>>>>>
>>>>>>> Basically I think this falls under the term "Remote Debugging" if you
>>>>>>> want to try and find ways to do it with other apps.
>>>>>>>
>>>>>>> I'm not sure there is any other practical method as most plugins rely
>>>>>>> on
>>>>>>> the QGIS built in stuff to be in an active session, so I wouldn't waste
>>>>>>> too much time trying to do it without QGIS running at all.
>>>>>>>
>>>>>>> Enjoy,
>>>>>>> Alex
>
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>