<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 3/12/19 12:28 AM, Nyall Dawson
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAB28Asj-SD+d2vFU4G+v0Cx29YS_pV8EPKiVpfr3FSV_eiGgKQ@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">On Tue, 12 Mar 2019 at 07:59, Denis Rouzaud <a class="moz-txt-link-rfc2396E" href="mailto:denis.rouzaud@gmail.com"><denis.rouzaud@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">
I am also in favor of disabling them but maybe keep them running and have them as expected failure?
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
That sounds preferable -- but I'm not (personally) sure if it's
possible. You know the CI setup better than I do, can you see a way to
do this?</pre>
</blockquote>
<p>I'm not sure this works.<br>
</p>
<p>Expected failure "expects" the tests to fail and will error if it
succeeds, since we are dealing with flaky tests here, that will
still lead to unstable behavior.<br>
</p>
<blockquote type="cite"
cite="mid:CAB28Asj-SD+d2vFU4G+v0Cx29YS_pV8EPKiVpfr3FSV_eiGgKQ@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">If not I'd still be in favor of disabling them.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
I honestly believe the harm in leaving these enabled (short term) is
outweighing the risk of disabling them. We are very likely pushing new
contributors away from our project with the current difficulties just
getting a PR to green.</pre>
</blockquote>
<p>I agree with the proposal, these tests do more harm than good
currently.</p>
<p>Matthias<br>
</p>
<blockquote type="cite"
cite="mid:CAB28Asj-SD+d2vFU4G+v0Cx29YS_pV8EPKiVpfr3FSV_eiGgKQ@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">
Nyall
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">
Cheers
Denis
On Mon, 11 Mar 2019, 22:47 Nyall Dawson, <a class="moz-txt-link-rfc2396E" href="mailto:nyall.dawson@gmail.com"><nyall.dawson@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">
Hi all,
For a long time now we've been plagued by intermittently failing tests
on Travis, which are making the whole QGIS development experience
quite painful.
I propose that we take an absolute hard line approach from now and
disable all tests which are causing false positive failures. I've
started here: <a class="moz-txt-link-freetext" href="https://github.com/qgis/QGIS/pull/9483">https://github.com/qgis/QGIS/pull/9483</a>
This is obviously not ideal, as the failures may be revealing real
bugs (and in the case of the two disabled above I believe they are
symptoms of the same underlying bug), but I think now we've passed the
point where leaving these tests enabled causes more damage then
skipping them.
Ideally someone would investigate these and fix either the tests or
the underlying bugs... but it hasn't happened in 6+ months, so I don't
expect that to happen shortly**. I did spend some time around a month
ago to see if the fix for these two was trivial, but could not find it
quickly.
Is anyone opposed to a hard-line "disable if flaky" stance?
Nyall
** For full disclosure: next round of QGIS grants I plan on filing for
a grant to investigate all tests disabled on Travis in depth and
either fix underlying bugs or make the tests more stable. But that's
grant dependant, and not a short term solution.
_______________________________________________
QGIS-Developer mailing list
<a class="moz-txt-link-abbreviated" href="mailto:QGIS-Developer@lists.osgeo.org">QGIS-Developer@lists.osgeo.org</a>
List info: <a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-developer">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a>
Unsubscribe: <a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-developer">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a>
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
--
Denis Rouzaud
<a class="moz-txt-link-abbreviated" href="mailto:denis@opengis.ch">denis@opengis.ch</a>
+41 76 370 21 22
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">_______________________________________________
QGIS-Developer mailing list
<a class="moz-txt-link-abbreviated" href="mailto:QGIS-Developer@lists.osgeo.org">QGIS-Developer@lists.osgeo.org</a>
List info: <a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-developer">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a>
Unsubscribe: <a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-developer">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a>
</pre>
</blockquote>
<div class="moz-signature">-- <br>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div class="moz-signature">
<title></title>
<div class="moz-signature"> <span style="text-align: left;
color: #000000; font-family: 'Verdana', sans-serif;
font-size: 10pt">Matthias Kuhn</span><br>
<a href="mailto:matthias@opengis.ch" target="_blank"> <span
style="text-align: left; color: #000000; font-family:
'Verdana', sans-serif; font-size: 8pt">matthias@opengis.ch</span>
</a><br>
<span style="text-align: left; color: #000000; font-family:
'Verdana', sans-serif; font-size: 8pt"><a
href="tel:+41764356763">+41 (0)76 435 67 63</a></span><br>
<div> <a href="http://www.opengis.ch"> <img
moz-do-not-send="false"
src="cid:part3.28063C6D.3C5193F1@opengis.ch"
alt="OPENGIS.ch Logo" width="200" height="80"></a> </div>
</div>
</div>
</div>
</body>
</html>