[SoC] Weekly report 8: Testing framework for GRASS GIS

[Vaclav Petras]

1. What did you get done this week?

I extended the API for calling modules to allow convenient syntax, improved
the naming of function and classes, added documentation, removed
unnecessary code and introduced several other improvements based on the
feedback of my mentor who is writing the tests for the code he maintains. I
also added few more assert functions for evaluating the results.
Additionally, I rewrote some existing tests to use the new testing
framework and started to write test for r.slope.aspect module. Finally, I
moved gunittest package from sandbox to trunk and changed the import paths
and documentation accordingly.

The documentation is now available in
dist.../docs/html/libpython/gunittest_testing.html after running `make
sphinxdoc`.

The tests can be run for the whole GRASS using

 python -m grass.gunittest.main gisdbasepath nc_smp_location_name nc

The command can be executed in any directory and it will find all tests in
all subdirectories. The HTML report is stored in testreport/index.html.

There is currently 17 successfully running tests files and 3 failing test
files (each test file contains multiple tests).

I did not advanced in testing of location switching. There is not enough
tests in different locations. Anyway, main testing script switches to
arbitrary location and executes each test file in unique mapset and working
directory.

I postponed the integration with build system. The integration basically
mean that `make test` would run the main test script in the current
directory in some selected location. (Demolocation looks like a good choice
in this case but this will limit us to tests which are not using any data.)

2. What do you plan on doing next week?

The original plan for next week (week 9) is to implement missing
comparison/assert methods. Currently, it is possible to compare kev-value
pairs with different methods and precisions which covers most of the GRASS
modules giving metadata and statistics about maps. Additional assert
methods enables to compare raster maps in this way. For example, the
function assertRastersNoDifference() can subtracts two raster maps and then
checks their difference in a given precision. What is missing is comparison
of vector and 3D raster maps and also temporal datasets.

3. Are you blocked on anything?

I did not solved the issue of distribution of tests for different
platforms. On Unix-like systems (+- Mac OS X), I suppose that all people
wanting to test will be able to compile GRASS and download all locations
they are interested in. There should be some rule in makefiles to build the
C test modules, so that testing framework does not have to deal with that.
However, doing this on MS Windows is not likely. Perhaps some special
package with GRASS source code including testsuite directories, C test
modules' binaries and locations would be the way to go.

Vaclav

Src:
http://trac.osgeo.org/grass/browser/grass/trunk/lib/python/gunittest?rev=61238
Wiki:
http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS#Week08
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/soc/attachments/20140711/1d696c90/attachment.html>