[GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Glynn Clements
glynn at gclements.plus.com
Mon May 23 10:04:45 EDT 2011
Anne Ghisla wrote:
> > I'd suggest adding a "test" target to the various *.make files, so you
> > can do e.g.:
> >
> > make -C display test
> > make -C display/d.rast test
>
> Good idea. Sadly I don't know makefile system well, so most of my
> questions are basic. Your suggestion is to explicitely run tests on a
> given module. Would it also be possible to run tests in one run,
> launching 'make' in the code root folder?
Yes. In Dir.make (used for directories) the "test" target would be
recursive, i.e. it would run "make -C $dir test" for each
subdirectory, the same way that the "clean" target is handled. This
target would literally be just:
test: test-recursive
Dir.make already has a pattern rule for "%-recursive", which makes the
stem (whatever the "%" matches) for each subdirectory.
> > > Tests are in each module's folder (written in Python)
> >
> > Python might be overkill for module tests. The build system absolutely
> > requires a Bourne shell (for make), so portability isn't an issue.
> >
> > The per-module test script shouldn't need to analyse the output; it
> > can just be a sequence of commands which generate textual output (e.g.
> > a test script for a raster module might use r.out.ascii to generate
> > this; a test script for a display module would output to a PPM file
> > which can just be "cat"d). The analysis would consist of the output
> > being compared against a reference file.
>
> It saves us writing lots of functions that would compare datatypes; but
> OTOH it would add dependencies of other GRASS modules. Then, it will be
> necessary (or at least better) to test first the modules without
> dependencies. We would need to check if a test includes more than one
> GRASS module... or just let all tests run and fail, and then understand
> which modules are actually broken, and causing other (correct) modules'
> test to fail.
I don't think that there's any way around this. Few GRASS modules are
useful in isolation. On the positive side, modules which are essential
for tests (e.g. g.region, *.out.ascii) etc are unlikely to be changed
regularly.
As I see it, this isn't really any different to the situation where
someone commits a change which causes a library to fail to compile,
and every module which uses that library also fails to compile.
> We chose Python, as Sören already wrote tests as shell scripts and found
> them too cumbersome and difficult to mantain.
But is this because you're trying to perform analysis of the results
within the test scripts?
--
Glynn Clements <glynn at gclements.plus.com>
More information about the grass-dev
mailing list