[GRASS5] The status of 5.0

[Glynn Clements]

Roger Miller wrote:

> > Actually, I suspect that's likely to be a catch-22 situation. In order
> > to become truly fixed and completely stable, 5.0 needs a lot more
> > testing than it will get while it remains "in development". To get an
> > adequate level of testing, it will need to be put into real-world use;
> > which probably means declaring it to be "released".
> 
> I doubt that the general public wants to be your beta-testers.

They would doubtless prefer that all of the bugs had been found first. 
But if they've been using a computer for more than a week, they will
also have realised that this just doesn't happen.

If users absolutely insisted that software was 100% bug free, the
software industry would be as dead as the proverbial dodo, and the
industry "leader", Microsoft, would probably have been the first to
go.

> It sounds to me like what we actually need is a systematic means of
> testing the software.

Systematic methods exist, e.g. formal methods, "coverage" testing. 
Unfortunately, the effort (and cost) involved makes them impractical
for application-level software.

If you're developing control software for an 8-bit CPU with 8Kb of
memory for a safety-critical application, coverage testing is viable. 
For something as complex as GRASS (XFree86, the Linux kernel, gcc
etc), they're out of the question (unless someone has literally
billions of dollars to spare).

For more complex systems, you're stuck with an ad hoc approach. 
Developers perform the most obvious tests, but a lot of problems will
only be found when it's put into the hands of large numbers of users. 
The more users, the more likely that one of them will use exactly the
combination of data and options which triggers the bug.

> Besides, I doubt that just calling 5.0 "stable" is going to net very many
> more users then it has right now.  To the extent that new users do try it 
> we're likely to lose new users, because the software *isn't* stable.

Obviously, if there are known bugs, they should be fixed. Once you've
eliminated the known bugs, you're left with the unknown bugs, and
there are usually a lot more of those. And they aren't that likely to
be found if you only have a small number of users testing 400 modules.

> The fact that you would even suggest that several minor releases would
> immediately follow the initial release just proves that it isn't stable.

I didn't use the word "immediately". But bug fix releases will follow
the release (unless with either never actually release, or never fix
the bugs which will be found after release).

-- 
Glynn Clements <glynn.clements at virgin.net>