[SoC] Weekly report 3: Testing framework for GRASS GIS

[Vaclav Petras]

1. What did you get done this week?

The plan was to test unittest and doctest methods for evaluation and
comparison of textual and numerical outputs (results) on GRASS use cases.
The simple cases are solved by unittest. Besides the code I put to sandbox
I wrote some tests of some functionality related to g.list [1], g.mlist and
g.mremove.

I discovered several functions in grass.script.core which can be used to
solve more complex outputs of modules. So, I copied them, generalized them
and wrote a tests for them (more work is needed). These functions can be
used later to write tests but also in GRASS itself.

Regarding the issue with testing using different languages (translations).
With the tests I wrote [2] is seems that the best solution will be to use
regular expressions or just Python `in` operator to test if the message
contains the required information. In other words, test should be
independent on language environment.

I also explored the the possibility of using code coverage to evaluate the
tests. It seems that with Python coverage package [4], we can get code
coverage (reports) for Python library without much burden at least for
basic cases (no modules involved, no ctypes).

2. What do you plan on doing next week?

The plan is to implement some way or ways of comparison of raster and
vector maps. It would be good to implement comparison of other things too
and to finish comparison of textual and numerical outputs from this week
but the topic is so complex that I will focus just on raster maps first.

3. Are you blocked on anything?

The fact that I also have to create the some tests to test the framework on
some real world example slows me down but on the other hand, it is
necessary and even very useful because I'm also testing things, such as the
functions which will be used for creating the tests. And I discovered some
problems right away.

Vaclav


[1]
http://trac.osgeo.org/grass/browser/grass/trunk/general/g.list/test_g_list.py?rev=60619
[2] http://trac.osgeo.org/grass/attachment/ticket/2228/test_g_mremove.py
[3] http://trac.osgeo.org/grass/changeset/60730
[4] https://pypi.python.org/pypi/coverage

http://trac.osgeo.org/grass/browser/sandbox/wenzeslaus/gunittest?rev=60733
http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS#Week03
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/soc/attachments/20140606/1a009cba/attachment.html>