[GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Markus Metz]

On Thu, Oct 6, 2016 at 12:09 PM, Moritz Lennert
<mlennert at club.worldonline.be> wrote:
> On 06/10/16 01:39, Nikos Alexandris wrote:
>>
>> The question, the way I understand things, is not if and at what extent
>> we need tests. GRASS-GIS needs them.
>> However, no matter the effort, tests will almost never be complete.-
>> That's what I was told by computer scientists, and what I can confirm
>> with my minimal experiences by doing, for at least two years now, in a
>> dozen of mini-projects.
>
>
> This is true, in my experience, but Markus' remarks (AFAIU) go beyond the
> question of completeness. One big issue we have with some modules is how to
> define the 'correct' result. Unless we use a tautological reasoning to say
> that the current state of the module gives the correct results (thus
> reducing the signification of the tests to regression tests), there are a
> series of modules for which no third-party validation data exists.
>
> For example: what is a "correct" segmentation by i.segment ? Or what is the
> "correct" output of some of r.watershed's results ?

Very good examples because there is no "correct" output of i.segment
or r.watershed because there is no agreement about what is correct.
Different implementations of image segmentation or 2D hydrological
modelling make different assumptions. Most of these assumptions are
supported by peer-reviewed literature. That means there is no
generally accepted "correct" result, and all we can do is to ensure
that the output of GRASS modules (obviously we assume that the output
matches the expectations of the author of the module) should not
change.

Markus M

>
> A second issue is that some bugs only appear with large amounts of data (see
> #3084 for example), but I don't think that it is feasible to test all our
> modules on multi-GB datasets. And sometimes it is not the size of the data
> as such, but very specific combinations of data.
>
> This second issue obviously should not stop us from writing tests for less
> sizeable data. The first issue is a bit more difficult to solve.
>
>> Yes, testing costs a lot of time. But, if comparing it with the time
>> spent afterwards in debugging bad code, therein lies the wish to have
>> spent the "less" time right in the beginning.
>
>
> I agree...to a point. However, don't forget that tests are code just like
> any other code and thus increasing the lines of code by writing all these
> tests also increases the lines of code that need to be maintained. So, a big
> yes for testing, but let's be careful and efficient about it and not pretend
> we can solve everything with testing. And let's be especially wary of the
> danger cited often in the literature about testing: while thinking about
> tests is important, don't forget to think about the actual code you write
> ;-)
>
> Moritz
>
> _______________________________________________
> grass-dev mailing list
> grass-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev