[Qgis-developer] Ramping up for 0.8 - a QGIS test framework

[Tim Sutton]

Hi

> I have no idea however how a test suite should work.
> The most difficult bugs are those which appeare 'randomly' after
> minutes/hours of user interaction with lots of data in various
> formats loaded. Can we simulate this?

Im not sure - I did have the idea of also creating a 'test-data' dir
which contains small datasets that are know to trigger specific
issues. The thought was to then try to programmatically load each
layer and perhaps for example test that the canvas image rendered for
these layers is not a uniform colour (indicating nothing was
rendered). Combinations of layers could probably be tested in this way
too. What is less obvious to me is how we could test for mis-rendering
(visual artifacts) and so on. Most likely we can also produce a visual
report with side by side pairs of images. The image on the left is the
'expected' rendered output and on the right is what the unit tester
has rendered. This way you can quickly visually scan the report and
see any anomalies. One of our openModeller contributers used a similar
technique when composing the unit test for om.


My thought is that even if unit tests can cover only 80% of problem
scenarios, it will still be a major advancement in quality control for
QGIS. I dont have all the answers yet, but I think if we make a start
and get into a 'testing culture' we can make improvements along the
way....

8<-------- snip---------------

Regards

--
Tim Sutton

Visit http://qgis.org for a great Open Source GIS
Skype: timlinux
MSN: tim_bdworld at msn.com
Yahoo: tim_bdworld at yahoo.com
Jabber: timlinux
Irc: timlinux on #qgis at freenode.net