[QGIS-Developer] Flaky tests and Travis

[Denis Rouzaud]

I am also in favor of disabling them but maybe keep them running and have
them as expected failure?
If ones write some code related to the corresponding tests , they might
still provide some valuable info.
Is this possible?

If not I'd still be in favor of disabling them.

Cheers
Denis

On Mon, 11 Mar 2019, 22:47 Nyall Dawson, <nyall.dawson at gmail.com> wrote:

> Hi all,
>
> For a long time now we've been plagued by intermittently failing tests
> on Travis, which are making the whole QGIS development experience
> quite painful.
>
> I propose that we take an absolute hard line approach from now and
> disable all tests which are causing false positive failures. I've
> started here: https://github.com/qgis/QGIS/pull/9483
>
> This is obviously not ideal, as the failures may be revealing real
> bugs (and in the case of the two disabled above I believe they are
> symptoms of the same underlying bug), but I think now we've passed the
> point where leaving these tests enabled causes more damage then
> skipping them.
>
> Ideally someone would investigate these and fix either the tests or
> the underlying bugs... but it hasn't happened in 6+ months, so I don't
> expect that to happen shortly**. I did spend some time around a month
> ago to see if the fix for these two was trivial, but could not find it
> quickly.
>
> Is anyone opposed to a hard-line "disable if flaky" stance?
>
> Nyall
>
> ** For full disclosure: next round of QGIS grants I plan on filing for
> a grant to investigate all tests disabled on Travis in depth and
> either fix underlying bugs or make the tests more stable. But that's
> grant dependant, and not a short term solution.
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

-- 

Denis Rouzaud
denis at opengis.ch  <denis at opengis.ch>
+41 76 370 21 22
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20190311/439f9953/attachment.html>