<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Joanthan,<br>
<br>
Valid points you are raising.<br>
What's currently broken with tests is not, that developers don' care
but rather, that developers are not aware of failing tests due to
the lack of visibility. Also it's not possible currently to tell
which commit is responsible for a failing test.<br>
Another thing is, that there is no reference platform yet. A test
may work on windows but fail on linux. Then somebody tries to "fix"
it and breaks it on the other platform. This method will introduce a
"reference" platform that will have to work. If further reference
platforms will be added later is open, but something to keep in
mind.<br>
These things will be fixed. And I don't assume that somebody will
disagree if a commit that breaks a test will be reverted as well as
it will make it easier to review pull requests because a "red" pull
request will not even have to be considered. But certainly, an
official statement of the PSC concerning this will be highly
appreciated. There is also a recent RFC/QEP from Nyall that deals
with requirements for unit tests. [1]<br>
<br>
I hope I could address your concerns and if not, just ask again :-)<br>
<br>
Kind regards,<br>
Matthias<br>
<br>
[1]
<a class="moz-txt-link-freetext" href="https://github.com/nyalldawson/QGIS-Enhancement-Proposals/blob/728f5ef8cfc6dd4c7ee830213339842db6891ee1/QEP-5-Unit-tests.rst">https://github.com/nyalldawson/QGIS-Enhancement-Proposals/blob/728f5ef8cfc6dd4c7ee830213339842db6891ee1/QEP-5-Unit-tests.rst</a><br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 10/10/2014 02:10 PM, Jonathan Moules
wrote:<br>
</div>
<blockquote
cite="mid:E6100A0537F18549AC8FCCB5D47FFE355F00D1@EXMBX1.hrw-uk.local"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:Consolas;
color:black;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Arial","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">Hi
Matthias,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">It’s
good to see that some initiative is being taken on this, but
if this goes ahead, is there a requirement from the PSC that
devs actually do fix things that cause the tests to fail or
not have their commit accepted? Because otherwise I can see
this devolving into the situation that currently exists
(devs don’t bother with unit tests).<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">Further
thought – if this goes ahead it may be worth building on it
and requiring that unit tests be created for all new
functionality as part of acceptance of that functionality.
Are there such plans? (that’s more something for the dev
list anyway).<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">Certainly
a good start though!<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">Jonathan<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"
lang="EN-US">From:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"
lang="EN-US"> <a class="moz-txt-link-abbreviated" href="mailto:qgis-user-bounces@lists.osgeo.org">qgis-user-bounces@lists.osgeo.org</a>
[<a class="moz-txt-link-freetext" href="mailto:qgis-user-bounces@lists.osgeo.org">mailto:qgis-user-bounces@lists.osgeo.org</a>] <b>On Behalf
Of </b>Matthias Kuhn<br>
<b>Sent:</b> Friday, October 10, 2014 12:58 PM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:qgis-user@lists.osgeo.org">qgis-user@lists.osgeo.org</a><br>
<b>Subject:</b> [Qgis-user] Crowdfunding Project:
Automated tests for QGIS<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Dear QGIS
users,<br>
<br>
I started a crowd-funding campaign for automated unit tests in
QGIS. Please consider to help funding this project in order to
improve the software and code quality in the long run. More
details can be found on the project page and in the text
below.<br>
<br>
<a moz-do-not-send="true"
href="http://blog.vitu.ch/10102014-1046/crowdfunding-initiative-automated-testing">http://blog.vitu.ch/10102014-1046/crowdfunding-initiative-automated-testing</a><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">I plan to improve the
quality of QGIS sustainably by introducing the automatic
execution of test procedures which will inform responsible
developers automatically in case of failure. Your support
for this project is highly welcome and you can contribute to
make QGIS more stable in the long run. To implement the
necessary changes as well as repairing currently broken
tests a minimal amount of CHF 5000.- will be required. The
following paragraphs will describe the project in more
detail. </span><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">----------</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">In the last years,
QGIS matured from a hobby project to a serious alternative
with a broad scope of possibilities and a great potential.</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">With the constant
addition of new features new possibilities have been created
and still are created. Every new feature means changes to
the source code of the software and is a potential source of
errors.</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">There are
possibilities to automatically detect errors. This is done
by executing a series of operations with known result. If a
operation does not return the desired (known) result
something went wrong and it needs to be evaulated why.</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">QGIS has currently
over 100 different tests. However, these are not run
regularly and nobody realizes when tests fail. This led to
the situation, that at the moment approximately 2/3 of the
tests do not report success and nobody knows why and since
when.</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">I intend to take the
following measures to ensure that in the future tests no
longer can silently fail:</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">* tests will be run
on all source code changes in master (main development
branch)</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">* tests will be run
on all pull requests (changes being evaluated for
integration in master)</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">* if tests fail the
responsible developers will be notified by mail</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">* with symbols, the
current state of the tests will be indicated<br>
<br>
</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">With this system it
is possible to react instantaneously when tests fail and to
revert the responsible changes or to not even integrate
these.<br>
<br>
</span><o:p></o:p></p>
<p class="MsoPlainText"><span lang="EN-US">To make sure that
this system works and is accepted, it is important to start
with a working suite of tests. How much time it takes to fix
all tests is difficult to say. The plan is therefore to fix
as many tests with the available funds and to disable the
rest of the tests in the first time. Therefore additional
funds that are provided on top of the required minimum CHF
5000.- will be very welcome for additional working tests.<br>
<br>
</span><o:p></o:p></p>
<p class="MsoPlainText">It would be great if this crowd-funding
effort could help to get this project to fly.<br>
<br>
<o:p></o:p></p>
<p class="MsoPlainText">Kind regards,<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Matthias Kuhn<br>
<br>
<o:p></o:p></p>
<p style="text-align:center" align="center"><span
style="background:white">This message has been scanned for
viruses by
</span><a moz-do-not-send="true"
href="http://www.mailcontrol.com/"><span
style="background:white">MailControl</span></a><span
style="background:white">, a service from BlackSpider
Technology</span><o:p></o:p></p>
<p><span style="font-size:10.0pt">Click <a
moz-do-not-send="true"
href="https://www.mailcontrol.com/sr/h0opETQ15z7GX2PQPOmvUgny0%216zNmYYcr+CLPtUcwBrcg72XoIMJrYkKSbFUrl7COZlUrWwm5I94h7CChK4GQ==">here</a>
to report this email as spam.<o:p></o:p></span></p>
<p style="text-align:center" align="center"><o:p> </o:p></p>
</div>
<br>
<hr>
<p style="font-family:arial;font-color:grey;font-size:8pt"><b>HR
Wallingford and its subsidiaries</b> uses faxes and emails for
confidential and legally privileged business communications.
They do not of themselves create legal commitments. Disclosure
to parties other than addressees requires our specific consent.
We are not liable for unauthorised disclosures nor reliance upon
them.
<br>
If you have received this message in error please advise us
immediately and destroy all copies of it.
<br>
<br>
HR Wallingford Limited<br>
Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom<br>
Registered in England No. 02562099<br>
</p>
<hr>
</blockquote>
<br>
</body>
</html>