[GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

On 06/10/16 01:39, Nikos Alexandris wrote:
> The question, the way I understand things, is not if and at what extent
> we need tests. GRASS-GIS needs them.
> However, no matter the effort, tests will almost never be complete.-
> That's what I was told by computer scientists, and what I can confirm
> with my minimal experiences by doing, for at least two years now, in a
> dozen of mini-projects.

This is true, in my experience, but Markus' remarks (AFAIU) go beyond 
the question of completeness. One big issue we have with some modules is 
how to define the 'correct' result. Unless we use a tautological 
reasoning to say that the current state of the module gives the correct 
results (thus reducing the signification of the tests to regression 
tests), there are a series of modules for which no third-party 
validation data exists.

For example: what is a "correct" segmentation by i.segment ? Or what is 
the "correct" output of some of r.watershed's results ?

A second issue is that some bugs only appear with large amounts of data 
(see #3084 for example), but I don't think that it is feasible to test 
all our modules on multi-GB datasets. And sometimes it is not the size 
of the data as such, but very specific combinations of data.

This second issue obviously should not stop us from writing tests for 
less sizeable data. The first issue is a bit more difficult to solve.

> Yes, testing costs a lot of time. But, if comparing it with the time
> spent afterwards in debugging bad code, therein lies the wish to have
> spent the "less" time right in the beginning.

I agree...to a point. However, don't forget that tests are code just 
like any other code and thus increasing the lines of code by writing all 
these tests also increases the lines of code that need to be maintained. 
So, a big yes for testing, but let's be careful and efficient about it 
and not pretend we can solve everything with testing. And let's be 
especially wary of the danger cited often in the literature about 
testing: while thinking about tests is important, don't forget to think 
about the actual code you write ;-)

Moritz