[Qgis-psc] Comments from Strk on QGIS-Testsuite

[Matthias Kuhn]

Hi

On 02/20/2016 06:40 PM, Tim Sutton wrote:
> Hi
>
>> On 20 Feb 2016, at 16:38, Matthias Kuhn <matthias at opengis.ch
>> <mailto:matthias at opengis.ch>> wrote:
>>
>> Thanks for this Answer Nyall, my thoughts reading this mail have been
>> very similar.
>>
>> Our test suite has improved a lot and its stability is getting better
>> and better. I am sure that it will help a lot to smooth the transition
>> to QGIS 3, given that refactoring is very likely to introduce small bugs
>> which in turn can often be spotted by a good unit test suite.
>>
>> That does not mean that our test infrastructure is perfect. There is
>> certainly a lot to improve. The most important part IMO is increasing
>> the test coverage and getting CI on Windows (and maybe other platforms)
>> or with different dependency versions. But a perfect test infrastructure
>> will be a neverending quest for a holy grail anyway :-)
>>
>> I'll appreciate a broader discussion on what should be financially
>> supported by the community and the PSC (thanks for starting this
>> discussion anyway!). And then - if testing is an area to invest -
>> evaluate which parts of the testing suite needs more attention.
>
> Having the CI stuff going has been a huge leap forward and with my QA
> PSC member hat on, I would say it is definitely something to invest
> more time and effort into.

Happy to have your backup. We could try to bring back the "test friday" :-)

> As I mentioned before though I would just like us to be more strategic
> about our investments (from the PSC side anyway) and ensure that we
> put more focus on other places that urgently need support (like
> getting decent API docs in place to name a random example). Thanks for
> all the great stuff you have done with the test suite Matthias,
> looking forward to getting more stuff done on the testing front in the
> future.
Thanks for the credits, but I have to admit that recently other people
have been much more active than me on this front!

> I also think we need to start thinking about GUI testing soon - GUI
> testing will put a lot of the underlying API’s to task and also ensure
> that fronted behaviour is consistent from release to release (or at
> least that it is consciously changed instead of unconsciously).

What kind of UI tests would you see as important? Are there any recent
issues which you think could have been avoided by such a unit test?

I think we need to be careful here.
For once, UI issues there are normally spotted quickly in contrast to
algorithms/calculations which result in wrong outcome. E.g. the recent
strange behavior of the "control output order" checkbox in the rendering
properties.
Also I think there are quite a couple of parts of our UI that could be
improved. Adding a test it will consolidate the status quo. E.g. I think
that the keyboard modifiers of the select tools would really need to be
improved. If we had a test it might keep developers from making a pull
request which proposes such an improvement because of the feedback of a
failing test.

I'll be happy to hear your comments, I'm sure you have experience with
this from other projects :-)

Matthias

-- 
Matthias Kuhn
OPENGIS.ch - https://www.opengis.ch
Spatial • (Q)GIS • PostGIS • Open Source

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20160221/6e0f445b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20160221/6e0f445b/attachment.sig>