[GRASS-dev] Re: testing in 6.5 before backporting to 6.4

[Michael Barton]

The lack of manpower makes it even more important that all developers test any changes before committing--including backporting. I agree with Glynn that moving all dev work to 7 would simplify things. But it is also important that all developers only commit code that passes a WFM (works for me) validation test. This will catch errors and typos that are easy to fix by the coder at the time of submitting but which are very difficult to track down by someone else later. Only code that the developer can be sure works on his/her local machine should be committed. This kind of test is simple enough to do. If doing a WFM validation on code changes seems to take too much time, not doing it eats up much, much more of time in the rest of the dev and user community. Even with a WFM test, there will always be the possibility that a code change can cause unforeseen problems in other parts of the code base or may not work on another platform. These are problematic enough to identify and debug without having to also deal with errors that should have been caught by the developer with a minimum of checking. We simply don't have the manpower for the hours needed to debug simple errors that should have been caught at the time of committing. This should apply to the dev branch as well as the release branch for a couple of reasons. First, many people use the dev branch on a regular basis. Making a change that breaks a vital function by not testing before committing, impacts the work of many people. Yes there is a risk to using a dev version, but we also should follow practices that attempt to limit as much as possible introducing easily avoidable bugs into a program that is used by so many people globally. Secondly, if all code has at least passed a WFM test, then bugs that are encountered in a dev branch are easier to track down. 

Along the same lines, I don't think that developers should be committing 'unfinished' code that knowingly does not work but is linked to an interface element. When a user encounters a button or pull-down that does nothing, does something incorrectly, or crashes a module it is very frustrating. The user does not know if it is a bug, a platform issue, or a problem with their own system. This starts a chain of emails and tests in the user and dev community to try and solve the issue. When this is done intentionally, it needlessly eats into the very limited developer resources. Sometimes there are good reasons why it is necessary to commit unfinished code; in such cases it should be well-commented as such (so that others can help complete it if necessary or at least avoid duplication) and there should be no associated interface element that could confuse and frustrate users. It is not enough to put in the manual that x function does not yet work. Just don't add the button for x function to the interface in the first place. Again, this is quite easy to do for GRASS development.

Michael
______________________________
C. Michael Barton 
Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University
Tempe, AZ  85287-2402
USA

voice: 	480-965-6262 (SHESC), 480-727-9746 (CSDC)
fax:          480-965-7671(SHESC), 480-727-0709 (CSDC)
www: 	http://csdc.asu.edu, http://shesc.asu.edu
		http://www.public.asu.edu/~cmbarton

On Jun 15, 2011, at 9:00 AM, grass-dev-request at lists.osgeo.org wrote:

> Date: Wed, 15 Jun 2011 08:51:57 +0200
> From: Markus Metz <markus.metz.giswork at googlemail.com>
> Subject: Re: [GRASS-dev] Re: testing in 6.5 before backporting to 6.4
> To: Martin Landa <landa.martin at gmail.com>
> Cc: Helena Mitasova <hmitaso at ncsu.edu>, grass-dev
>        <grass-dev at lists.osgeo.org>
> Message-ID: <BANLkTin8zPoX8MiM6fGdeG0guvUOEdd5_Q at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> On Tue, Jun 14, 2011 at 5:35 PM, Martin Landa <landa.martin at gmail.com> wrote:
>> Hi,
>> 
>> 2011/6/14 Markus Metz <markus.metz.giswork at googlemail.com>:
>>> A number of changes to 6.5 and 7.0 went in untested, which is IMHO not
>>> a good idea, but being development branches, maybe sometimes
>>> excusable. Any changes to the release branch should however be tested
>>> before submitted. For most changes, the kind of test is pretty much
>>> straightforward, and, if omitted and the change introduces a bug,
>>> places a burden on users and other developers to figure out what went
>>> wrong and why, wasting other people's time and leaving a not so good
>>> impression of a releasebranch.
>> 
>> I agree, the real problem is the manpower which we have, or better to
>> say we don't have. Taking special care about three branches - trunk
>> (development), devbr6 (test-bed) and relbr64 (only tested commits) is
>> too much for us. With current manpower and our requirements, we could
>> end up with frozen relbr64 for years. Other issue is G7, when it could
>> be released? One year, two years? At the end the situation forces us
>> to keep relbr64 alive, not only bugfixes. One example is progressive
>> wxGUI. In the case we would just backport bugfixes the wxGUI would
>> remain three more years without improvements (counting 6.4.0 and
>> 7.0.0). Almost impossible to maintain (two different code bases).
>> After that most of the users start to use 7.0 (since 6.5 shouldn't be
>> far away from 6.4 branch). Your point is very good, but unfortunately
>> the number of people actively contributing to the project (GRASS) is
>> so low that it is going to be unrealistic. Nice to be a solid rock,
>> also think about releasing policy. To have old-solid release is nice,
>> at the end the most of user starts to use dev versions. Probably that
>> could be a reason, why such solid stable releases haven't so many
>> reported bugs ;-)
>> 
>> 
> I'm not against new features, I have myself introduced a number of new
> features from 6.4.0 to 6.4.2. What I wanted to say is that I fully
> agree with Helena and Hamish about the need for testing, that we may
> need to think about a way to highly encourage testing before
> backporting to 6.4, and that testing before submitting changes is IMHO
> generally a good idea.
> 
> If you say that taking special care about three different branches is
> too much for us, that means that changes really need to be tested for
> 7 as as well before submitting. As I said, in most cases testing can
> be quickly done by the person introducing the changes, because that
> person should know best what to test for.
> 
> Markus M