[GRASS-dev] [GRASS GIS] #2105: Missing guidelines for testing

[GRASS GIS]

#2105: Missing guidelines for testing
----------------------------+-----------------------------------------------
 Reporter:  wenzeslaus      |       Owner:  grass-dev@…              
     Type:  task            |      Status:  new                      
 Priority:  normal          |   Milestone:  7.0.0                    
Component:  Docs            |     Version:  svn-trunk                
 Keywords:  testing, tests  |    Platform:  Unspecified              
      Cpu:  Unspecified     |  
----------------------------+-----------------------------------------------

Comment(by huhabla):

 Replying to [ticket:2105 wenzeslaus]:
 > I miss any kind of guide how to create test. This applies for the main
 source code and the addons, C/C++ and Python, modules and code itself.

 Have a look at: http://grasswiki.osgeo.org/wiki/Test_Suite

 It is still an idea and guideline, but there are several tests scripts in
 GRASS that implement the guideline concept for GRASS modules.

 >
 > There are tests of temporal modules but it is not currently described
 anywhere how they are organized.

 The temporal module tests follow the test ideas described in:
 http://grasswiki.osgeo.org/wiki/Test_Suite

 > There is also a `testsuite` directory in trunk but it is not maintained.
 >
 > I've also noticed some tests running during compilation but I was never
 able to find out what it is.

 I am not sure what tests you mean? There are three C-libraries that
 implement unit tests for several library functions: gpde, gmath and
 raster3d libraries. These tests are not compiled by default. They must be
 compiled by hand and then executed in a GRASS location.

 > Some Python functions can be tested using `doctest` and Python
 developers usually know how to use it. However, it is not clear how to
 ensure calling from make.

 There is no test mechanism in the make system yet and no guideline how to
 implement Python doctests.

 >
 > There is a page about the test framework we wish to have:
 >  * http://grasswiki.osgeo.org/wiki/Test_Suite
 > But since it does not exists the page creates just confusion.

 This page describes in very detail how the testsuite can be implemented
 for modules and libraries. Python doctests and how to implement automated
 testing in the make system are not covered. It includes a guideline howto
 implement tests for modules based on shell scripts. Hence it is unclear to
 me how this leads to confusion?

 However, i am all for a complete new test suite design. I would suggest to
 implement all module and Python library tests in Python, so that these
 tests can be executed independently from the OS or the used command line
 interpreter, with or without the make system. The next big issue is how to
 automatically test the GUI?

 I would like to mentor this as a GSoC project in 2014. I have some
 experience with test suites ... .[1]
 And i have the experience that nobody cares to implement module or library
 tests ... .

 Best regards
 Soeren

 [1] https://code.google.com/p/grass6-test-suite/

 > There is also a mailing list about testing:
 >  * http://lists.osgeo.org/mailman/listinfo/grass-qa
 > But it is not active.
 >
 > ''We probably also miss the testing infrastructure itself but that's
 another bigger ticket. But some things exists already, so we can use
 them.''
 >
 > ''Maybe, we need Tests component in Trac?''

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/2105#comment:1>
GRASS GIS <http://grass.osgeo.org>