<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On 11/10/19 11:42 AM, Sandro Santilli
wrote:<br>
</div>
<blockquote type="cite" cite="mid:20191110104241.GA5492@liz">
<pre class="moz-quote-pre" wrap="">On Sun, Nov 10, 2019 at 09:20:11AM +0100, Alessandro Pasotti wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Just one thing I would recommend investing a bit more time and money: to be
able to run a Travis-like test suite locally
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
+1
I can't remember the last time I had a successful `make check`, bust
must be decades ago... Improving the ability for devs to run testsuite
locally is a huge asset. I think it goes along the lines of:
- Only run tests that can be run on the machine, given the available deps
... and WARN at build/test time about tests being skipped,
to hint developer about improving the test coverage
- Ensure running the tests doesn't mess up with your worktree
See <a class="moz-txt-link-freetext" href="https://github.com/qgis/QGIS/pull/31980">https://github.com/qgis/QGIS/pull/31980</a>
- Make it easy for devs to run single, specific tests, to help
when changing a single specific spot of the codebase</pre>
</blockquote>
<p>See
<a class="moz-txt-link-freetext" href="https://docs.qgis.org/testing/en/docs/developers_guide/unittesting.html#run-your-tests">https://docs.qgis.org/testing/en/docs/developers_guide/unittesting.html#run-your-tests</a>
, it says<br>
</p>
<p>> If a test fails, you can use the ctest command to examine
more closely why it
failed. Use the <code class="docutils literal"><span class="pre">-R</span></code>
option to specify a regex for which tests you want to run
and <code class="docutils literal"><span class="pre">-V</span></code>
to get verbose output:</p>
<div class="highlight-bash">
<div class="highlight">
<pre>> <span></span>$ ctest -R appl -V
</pre>
</div>
</div>
<p>Is there more required?</p>
<blockquote type="cite" cite="mid:20191110104241.GA5492@liz">
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">So, IMHO long life to PRs and Travis for the time being but we need to give
to the developers the tools to run locally all checks that Travis runs
remotely so that they are reasonably sure ("reasonably" because Travis
random unrelated failures are always around the corner) that the build will
pass and they will be able to debug a Travis failure locally without
waiting for hours.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
Now I realize you're talking about travis-like, that's also something
good to have, but as it would likely take much more upfront bandwidth
(docker image download?) and space (docker image) and time, it is also
good to support more lightweight testing.
--strk;
</pre>
</blockquote>
<blockquote type="cite" cite="mid:20191110104241.GA5492@liz">
</blockquote>
</body>
</html>