[Qgis-developer] Rethinking the testing and release procedure of QGIS

[Marco Hugentobler]

Hi all

Being in favor of unit tests, we cannot hope to solve with them all the 
stability problems, as some mails here suggest. It is just one small piece. In 
fact, thinking about bugfixes in the past, e.g. for the 1.7 release, most of 
them occured as a complex combination of several factors and user interaction, 
very hard to detect with unit tests.

Another point that needs to be considered is that the unit testing code needs 
to be maintained. Adding a test for every little change, even bugfix, would 
create a huge amount of testing code and sample datasets.

What about alternatively creating a unit test base that is limited in size but 
well maintained, e.g. a good coverage of the core classes? 

Regards,
Marco



 

Am Donnerstag, 7. Juli 2011, 10.35:54 schrieb Tim Sutton:
> Hi Andreas and others
> 
> 
> I wrote a bit of an essay on this a week or two back:
> http://linfiniti.com/2011/06/some-thoughts-on-the-future-of-qgis/
> 
> Believe me we have tried just about every suggestion including having
> a policy of unit tests for new features and bug fixes. The truth is
> that until we have undirected funding sufficient for the project to
> employ some of us developers to work full time on QGIS, things will
> probably never improve - developers need to earn a living which means
> they do work on QGIS based on client requirements or do other work and
> work on QGIS in their spare time. Most clients don't mandate tests be
> written to accompany the work they commission. With so many things
> that need working on, testing etc just doesnt attract the attention it
> should. So the idea of creating some kind of fund that is aimed at
> having a group of developers working full time on stabilising and
> professionalising QGIS is where we need to head for. I have already
> been having some behind the scenes discussions with potential funding
> sources on this, but so far nothing has materialised. I believe in the
> future it will come to pass that we can do this. From your side you
> can help too - when you contract developers to create new features,
> mandate unit tests to be part of the deliverables. If every
> organisation who was paying to add functionality to QGIS did this we
> would already be in a lot better  shape!
> 
> Regards
> 
> Tim
> 
> On Sat, Jul 2, 2011 at 10:51 AM, Sandro Santilli <strk at keybit.net> wrote:
> > On Sat, Jul 02, 2011 at 09:32:14AM +0200, Paolo Cavallini wrote:
> >> Il 30/06/2011 21:55, Andreas Neumann ha scritto:
> >> > Maybe the establishment of a testing infrastructure, as proposed by
> >> > Strk and others, would also help to maintain/raise the quality.
> >> 
> >> Agreed: AFAICT this is a major task: any idea of how much work would be
> >> necessary for reasonable results? An order of magnitude would be enough
> >> for now.
> > 
> > I belive a 30k figure might get things going far enough that developers
> > could have a policy of no functional change and no bugfix w/out a test
> > accompanying would be allowed.
> > 
> > --strk;
> > 
> >  ()   Free GIS & Flash consultant/developer
> >  /\   http://strk.keybit.net/services.html
> > _______________________________________________
> > Qgis-developer mailing list
> > Qgis-developer at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/qgis-developer


-- 
Dr. Marco Hugentobler
Sourcepole - Linux & Open Source Solutions
Churerstr. 22, CH-8808 Pfäffikon SZ, Switzerland
marco.hugentobler at sourcepole.ch http://www.sourcepole.ch
Technical Advisor QGIS Project Steering Committee