[GRASS-dev] Write a test to proof that the commit works

[Vaclav Petras]

Hi all,

as you know, I'm working on a GSoC project implementing a testing framework
for GRASS [1]. I'm at the beginning but I want to tell you that you can
start to write test right now as I recently did for g.list [2]. If you use
this as a template, you will ensure that it will be compatible with the
framework I'm working on.

Some projects require to have a test (test case or whatever) for each newly
added feature so that it is clear that the feature works and it will be
clear when it breaks (see r606618 for example). In other words, the
change/commit/patch is not accepted if it is not accompanied by a test (if
a test does not exists already). In the combination with tests running
after each commit, this is much more powerfull then reading commits or `svn
blame` because it marks the commits which should be reverted if tests fail.

I don't think that we need to set up rules like this but I would like to
encourage you to do it, to write the tests for your changes, especially for
the commits which are changing core functionality and the test is not hard
to write (the things which need to be tested are text or few numbers, not
maps, for example [4]).

For example to test the functionality of the parser, the simplest solution
I see is to test some module which is using the parser's feature by running
module with `run_command` and testing the return code using `assertEquals`.
There is few other possibilities but anything will do.

Please note that I'm not be able to write the actual tests for the whole
GRASS, not even for significant part of it. The topic of GSoC is a
framework which should provide good tools, environment settings, reports
and so on. The good tools may simplify test writing and will make it more
precise but nothing will save us from writing the tests for our work, our
changes and the functionality we are relying on.

Feel free to ask me questions here or privately about writing tests. I will
try to answer as soon as possible and reflect you comments in the
documentation and the code itself.

Sorry for such a long email but I would like to motivate you to contribute
a really long list of long tests.

Vaclav


[1] http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS
[2]
http://trac.osgeo.org/grass/browser/grass/trunk/general/g.list/test_g_list.py?rev=60619
(currently broken I guess :-)
[3] http://trac.osgeo.org/grass/changeset/60618
[4] http://trac.osgeo.org/grass/changeset/60713

PS: I have some parts of the framework already in the sandbox [5] but it is
unstable and for writing tests you can do the same with standard Python
unittest library [6].

[5] http://trac.osgeo.org/grass/browser/sandbox/wenzeslaus/gunittest
[6] https://docs.python.org/2/library/unittest.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20140605/4b98d1f4/attachment-0001.html>