[GRASS-dev] Test suite for GRASS - proposal, discussion welcome

[Glynn Clements]

Soeren Gebbert wrote:

> i have updated the wiki and added a simple Python example of a
> hypothetical r.series test. Please have a look at:
> http://grass.osgeo.org/wiki/Test_Suite#Test_framework
> 
> The sample Python code shows in principle how a test case would look
> like and what kind of methods should be provided by the framework.

This isn't a particularly good example. For most modules, you won't be
able to synthesise the reference data with r.mapcalc; you'll need to
provide it.

But my main concern is that few people will bother to learn a complex
test framework. If you want people to write tests, they need to be
little more than sample commands. Anything else (e.g. directives which
indicate how to compare the output) needs to be capable of being
summarised in a dozen lines. E.g.:

	# Test the average method
	r.series input=testmap1,testmap2 output=resmap_av method=average
	# Test the sum method
	r.series input=testmap1,testmap2 output=resmap_sum method=sum
	# Test the max method
	r.series input=testmap1,testmap2 output=resmap_max method=max

The implementation of the framework which executes test scripts can be
as complex as you like, but the interface (i.e. the syntax of test
scripts) needs to be trivial.

Boilerplate should be minimised; a region should already be set, test
maps should be available, parameters should have default values. Don't
force the user to specify anything that the framework ought to be able
to do itself.

E.g. in the above example, if a command creates a raster map named
resmap_av and a file named resmap_av.ref exists, that should be
sufficient for the framework to deduce that it should compare the map
to the reference data using default parameters.

-- 
Glynn Clements <glynn at gclements.plus.com>