[GRASS-dev] Final report: Testing framework for GRASS GIS (GSoC)

[Vaclav Petras]

It was not required by GSoC but I wanted to do some final report to
summarize what was done and what could be done in the future. So, here is a
shorter version of the report, the full is on Trac.

Ways to run tests:
 * each file separately in the current location and mapset you are in
  * good for testing the code you are just writing
  * good for writing the test
 * running all tests in subdirectories
  * generates a report
  * switches to specified location and mapset
  * generates detailed report
 * using higher level script to run tests from outside GRASS session

How to write tests:
 * using GRASS Python package `gunittest`
  * works in the same way as `unittest`
  * adds GRASS-specific functionality
  * allows generation of HTML reports
  * enables fine control over execution of GRASS modules
  * this is the preferred way, other ways such Shell scripts are discouraged

When writing tests, `gunittest` package allows in a convenient way to:
 * compare map statistics or info to a given reference
 * compare raster and 3D raster maps with each other
 * compare vector maps using different ways
 * compare numbers and texts

Things which should be solved in the short term:
 * tests should specify the location they can run in
 * encapsulate removal of maps and other files
 * test tests on more platforms

Long-term plans:
 * integrate tools such as `valgrind`
 * provide coverage
 * introduce static test of source code
 * time execution of modules

It is probably good to mention that documentation with examples is in
Python API Sphinx documentation. You can get it by `make sphinxdoc`.

And there is probably no need to mention that although there is a lot of
tests which can serve you as examples there is not enough tests to cover
all GRASS functionality, so please contribute.

Thanks to the test, we have already discovered several bugs soon after they
were introduced (the day of after the commit). This is very encouraging to
me and I hope also to others.

If you have some feedback or problems with writing tests, please share.

Vaclav

http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS#Finalreport
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20141004/ab36d662/attachment.html>