[Qgis-developer] Release schedule discussion - again

[Hugo Mercier]

Hi,

On 13/10/2015 13:02, Régis Haubourg wrote:
> Sandro Santilli-2 wrote
>>
>>> My plea to sponsors:
>>>
>>> If you're funding QGIS development work and the contract doesn't
>>> mention something like "we will totally soak this work in unit tests"
>>> then rip up the contract and run. (Or ask them nicely to revise the
>>> contract to include this ;) ) If you're funding work and it's not
>>> being accompanied by unit tests then you aren't getting what you paid
>>> for, and you'll be forever forced to test your funded features in
>>> every new release manually for regressions...
>>
>> +FF
>>
>> --strk;
> 
> I'm not a sponsor since there is now legal way for me to sponsor you by now.
> QGIS project still does not gather all financial ressources that are
> possible there. Maybe a discussion on paying Open Source licence for
> Enterprise users who wish to contribute could be a workaround?
> 
> However, I have included Unit tests in all the new features or plugins I
> funded. Yesterday, my plugin broke twice because of an API change. Should we
> also push plugin unit tests to Travis? 

Actually, knowing the impact on a list of plugins of an API break would
be a very useful information : an information on where to put the focus
regarding code reviews and what part of code has to be changed for
plugin developers.
I don't know if a Python script able to introspect most of the API usage
of a plugin is easily doable ... just an idea

> 
> Unit tests will never be sufficient for such large and GUI tool like QGIS,
> relying on so much different libraries. We need to reinforce that, but not
> misconsidering how much user test are critical is a mistake.

I agree.
And especially automatic UI testing. I am not sure if we could one day
have some automatic tools to help testing UI, in the meantime human
tests are needed.

Some ideas to improve quality :

- information on who is doing what to the current code is not always
clear among developers (well at least for me ...). Trying to follow
github notifications, mailing lists, QEP, irc and git logs is too time
consuming. So, in order to avoid possible big changes from being merged
without the other devs being informed, could we choose only one channel
for changes that may affect others ? Something like a "purely
informative" PR on github ? With an implicit rule like "If nobody reacts
within a week, it will be merged"

- (re)state when QEP are needed

- try to include documentation when developing a new feature. If unit
tests are not present, this is a minimum to know what the feature is
supposed to do. Some features are not covered by unit tests neither by
documentation, so it is sometimes very hard (and time consuming) to
ensure non-regression ...