[Qgis-developer] Ramping up for 0.8 - a QGIS test framework

[Radim Blazek]

Hi.

I realy appreciate any work towards stability. Stability is the most
important thing we should work on I think. QGIS has already enough
features but frequently crashing application is realy useless.

I have no idea however how a test suite should work.
The most difficult bugs are those which appeare 'randomly' after
minutes/hours of user interaction with lots of data in various
formats loaded. Can we simulate this?

Regarding GRASS I have finished adding new features and I'll also
concentrate on bugfixing.

Radim


On 3/29/06, Tim Sutton <tim at linfiniti.com> wrote:
> Greetings all
>
> First a quick personal note: Im about to finish my current contract
> and will have a 2 month period before starting my next job. In this
> period (among other things) I am hoping to spend a good amount of time
> on QGIS getting it towards a release ready state. Gary and I have
> finished moving off all aspects of the sf project management onto a
> private server. This time consuming process accounts for the decided
> lack of svn traffic on out part :-( We hope the new project service
> will provide us with a good mix of flexibility, reliability, speed of
> access and control.
>
> I would really like to address the perception of QGIS being 'buggy'
> and 'unstable' through a more rigourous approach to testing and
> managing bug reports. I believe there are three things we can do to
> address this:
>
> 1) Develop a comprehensive unit testing framework
> 2) Setup automated nightly test builds for validating that head builds
> and passes our test suite, along with html reports etc.
> 3) Setup a system whereby when someone builds QGIS successfully, the
> option is offered of posting some information such as OS, distro,
> build time, cpu, memory, SVN revision, unit test report etc can be
> automatically posted to our server so that we can have a birds eye
> view on the status of which environments are 'habitable' for QGIS at
> any given time
>
> (Item 3 above is a 'nice to have' for the future)
>
> I propose that for the unit tests we should use the test framework
> within qt4. My suggestion is that we create a test stub for every
> class and method in the current src/ tree, replicating the same
> heirachy under qgis/test/. In addition under /qgis/test/algorithms/ we
> can create tests that cut across many classes. Lastly I propose that
> we create qgis/test/gui for creating a suite of gui automated tests.
> The test stubs would be populated over time as we write more and more
> tests.
>
> Ideally these tests should be run as a part of the build process, so
> that running 'make' will fail unless all the tests pass too.
>
> The idea would then be to create a test case for every bug (now seems
> an especially opportune time since our bug queue has been shortened in
> the move over to trac). This will ensure that the all-to-familiar
> problem we have of fixed things being rebroken in subsequent commits.
>
> I am volunteering to create the initial test framework structure (and
> Ill welcome any help offered). Radim has some excellent ideas for
> publicising 0.8 when its released and it would be really nice if we
> could feel confident when it goes 'out the door' that the code base is
> robust and well tested.
>
> I know there are many ways to skin this cat, so any suggestions /
> ideas / concerns / discussion on the matter will welcome. Ill start a
> wiki page to document the ideas above more formally when ./ if I get
> general approval for carrying out this work....
>
> Best regards to all
>
>
> --
> Tim Sutton
>
> Visit http://qgis.org for a great Open Source GIS
> Skype: timlinux
> MSN: tim_bdworld at msn.com
> Yahoo: tim_bdworld at yahoo.com
> Jabber: timlinux
> Irc: timlinux on #qgis at freenode.net
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.qgis.org
> http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-developer
>