[Qgis-developer] Testing and releases

[Andrew Chapman]

I think that there are (at least) two different types of testing that are
valuable for a project like QGIS and both play important roles. Different
people use different names, but here's my version...
1) White box: This is very much in the hands of the developer and is aimed
at proving the functionality of their code - stop the bug before it gets out
of your own hands. This type of testing knows how the code should work and
typically, among other things, explores the boundary conditions.
2) Black box: This has no special knowledge of how the internals works and
tests as a dumb user... that can be relied on to provide the same test
coverage every time. One obvious use of this is for regressions testing of
the finished build, but I would suggest that with minor enhancements it
could be used to help ease some of the user documentation tasks.
QGIS supports multiple languages and operating systems, is released
regularly but, consequently, it is hard to produce user manuals and
tutorials with the most appropriate screen shots, etc. 
If a "black box" type of test harness were developed that could be "trained"
(e.g. recording keystrokes and mouse commands with a way to edit them) plus
the ability to capture images of screen objects (windows, dialogs, buttons,
icons) and save them to named files, this would present further
opportunities. The same test (script?) could be run on builds for different
operating systems, languages and versions. A standard set of build specific
images would be generated and, assuming standard file names and locations,
could automatically be imported into the various user documents. There would
still need to be some manual text editing, but the labour intensive
screenshot processing would be removed.
This must be a common requirement across projects, has anyone come across
any tools that already exist?
Andrew Chapman