[GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Anna Petrášová]

On Thu, Oct 6, 2016 at 4:08 PM, Markus Metz
<markus.metz.giswork at gmail.com> wrote:
> On Thu, Oct 6, 2016 at 12:09 PM, Moritz Lennert
> <mlennert at club.worldonline.be> wrote:
>> On 06/10/16 01:39, Nikos Alexandris wrote:
>>>
>>> The question, the way I understand things, is not if and at what extent
>>> we need tests. GRASS-GIS needs them.
>>> However, no matter the effort, tests will almost never be complete.-
>>> That's what I was told by computer scientists, and what I can confirm
>>> with my minimal experiences by doing, for at least two years now, in a
>>> dozen of mini-projects.
>>
>>
>> This is true, in my experience, but Markus' remarks (AFAIU) go beyond the
>> question of completeness. One big issue we have with some modules is how to
>> define the 'correct' result. Unless we use a tautological reasoning to say
>> that the current state of the module gives the correct results (thus
>> reducing the signification of the tests to regression tests), there are a
>> series of modules for which no third-party validation data exists.
>>
>> For example: what is a "correct" segmentation by i.segment ? Or what is the
>> "correct" output of some of r.watershed's results ?
>
> Very good examples because there is no "correct" output of i.segment
> or r.watershed because there is no agreement about what is correct.
> Different implementations of image segmentation or 2D hydrological
> modelling make different assumptions. Most of these assumptions are
> supported by peer-reviewed literature. That means there is no
> generally accepted "correct" result, and all we can do is to ensure
> that the output of GRASS modules (obviously we assume that the output
> matches the expectations of the author of the module) should not
> change.
>

There is correct result in the sense "as author intended". So you can
use pen and paper and solve very small example. That doesn't mean it
will cover all options, but better than nothing. This is obviously
hard to do if you are not the original author of the algorithm, and
that's the reason we should insist on tests for new modules.

Anna


> Markus M
>
>>
>> A second issue is that some bugs only appear with large amounts of data (see
>> #3084 for example), but I don't think that it is feasible to test all our
>> modules on multi-GB datasets. And sometimes it is not the size of the data
>> as such, but very specific combinations of data.
>>
>> This second issue obviously should not stop us from writing tests for less
>> sizeable data. The first issue is a bit more difficult to solve.
>>
>>> Yes, testing costs a lot of time. But, if comparing it with the time
>>> spent afterwards in debugging bad code, therein lies the wish to have
>>> spent the "less" time right in the beginning.
>>
>>
>> I agree...to a point. However, don't forget that tests are code just like
>> any other code and thus increasing the lines of code by writing all these
>> tests also increases the lines of code that need to be maintained. So, a big
>> yes for testing, but let's be careful and efficient about it and not pretend
>> we can solve everything with testing. And let's be especially wary of the
>> danger cited often in the literature about testing: while thinking about
>> tests is important, don't forget to think about the actual code you write
>> ;-)
>>
>> Moritz
>>
>> _______________________________________________
>> grass-dev mailing list
>> grass-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
> _______________________________________________
> grass-dev mailing list
> grass-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev