[Qgis-developer] Release schedule discussion - again

[Nyall Dawson]

On 13 October 2015 at 02:49, Jürgen E. <jef at norbit.de> wrote:

> That bugs pop up shortly after the release is probably unavoidable unless it
> gets intensive testing.   Otherwise the people testing (although they probably
> just want to use) a new release will easily outnumber the people testing and
> fixing it before the release and hence might find serious things right after
> the release - or even before we have all the packages ready.

Ahh... if only we had someone who could check every commit with
hundreds of tests immediately after it's committed. And that person
should be unpaid and work 24 hours a day. Also they should check every
PR before it gets merged. Oh wait, we've already got someone, and his
name is Travis ;)

(sarcasm aside) I realise that we need a combination of real life user
testing alongside the CI unit testing, so your point is still very
valid. But that said, my honest thoughts are that the BEST way to
ensure stable, regression-free releases is to extend the test suite.
We're on the right track, and the test suite has exploded since
Matthias' great work implementing the Travis testing, but we've still
got room to improve.

My plea to sponsors:

If you're funding QGIS development work and the contract doesn't
mention something like "we will totally soak this work in unit tests"
then rip up the contract and run. (Or ask them nicely to revise the
contract to include this ;) ) If you're funding work and it's not
being accompanied by unit tests then you aren't getting what you paid
for, and you'll be forever forced to test your funded features in
every new release manually for regressions...


Nyall