[Qgis-developer] Failing tests

[Tim Sutton]

Hi

> On 02 Jun 2015, at 02:24, Matthias Kuhn <matthias at opengis.ch> wrote:
> 
> Hi,
> 
> For more than a week already we have failing tests on master
> https://travis-ci.org/qgis/QGIS/builds
> 
> This basically renders the whole test suite a lot less usable.
> 
> Some examples/thoughts:
> 
> * Every pull request fails tests [1] I often look at the test results
> to check if it's even worth looking at a pull request. Handling pull
> requests is an issue on its own as there is no direct benefit for the
> reviewer most of the time. Don't make the reviewer's time even harder!
> 
> * We don't see when a commit trashes our compiler. Today we had a
> commit which broke the compiler. The author was not informed by mail.
> 
> * We don't see which commit broke a certain test. E.g. At the moment
> the PyQgsAppStartup test is failing. And to find out which commit broke
> it one has to dig deep because failing of this test is masked by the
> already failing tests.
> 
> * It discourages people to write tests if they can be broken easily.
> 
> The tests started to fail with the merge of some big work, just previous
> to feature freeze. I can understand that people want to get commits into
> master before feature freeze. But quality should not suffer from this.
> And if there's something broken, please fix it fast.
> 
> Finally Nyall was starting to fix things - crucial GIS functionality
> like splitting features - although I don't think he received anything in
> return, thank you very much, Nyall. The one thing left to do is to add
> doxygen API documentation - and to fix the newly broken PyQgsAppStartup
> test. I really hope we will finally see the green light again.
> 
> I am a hippy. I don't like rules and policies. I am not in favor of a
> very strong policy concerning the tests. Common sense should be valued
> more than rules. Policies like "every new feature has to come with a
> test" are no good solution as they encourage people to write silly
> tests. And I am much too lazy to write a QEP for a policy to revert
> commits which break the tests.
> 
> So... How do we fix it?
> If tomorrow everybody on this list writes some API documentation we
> should have fixed that test by noon.
> And then I'll promise I'll fix the failing PyQgsAppStartup test to get
> the green light going again.
> 
> Thank you very much
> Matthias
> 
> [1] https://github.com/qgis/QGIS/pulls ;
> https://github.com/qgis/QGIS/pulls?q=is%3Apr+is%3Aclosed
> [2]
> https://github.com/qgis/QGIS/commit/ded11b32562c100aeaae95bf20e21bcfc38d0777

I can only wholeheartedly second Matthias’ message above - we should never push to master if the tests don’t pass, and if for some reason they break we should not push anything more to master before the break is fixed.

Regards

Tim


> 
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer

—





Tim Sutton

Visit http://kartoza.com <http://kartoza.com/> to find out about open source:

* Desktop GIS programming services
* Geospatial web development
* GIS Training
* Consulting Services

Skype: timlinux Irc: timlinux on #qgis at freenode.net
Tim is a member of the QGIS Project Steering Committee

Kartoza is a merger between Linfiniti and Afrispatial

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20150602/89de913e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: KartozaLogo160x66.png
Type: image/png
Size: 9324 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20150602/89de913e/attachment-0001.png>