[OpenLayers-Dev] Automated Testing

[Paul Spencer]

On 19-Dec-07, at 3:20 PM, Eric Lemoine wrote:

> On Dec 19, 2007 3:34 PM, Christopher Schmidt  
> <crschmidt at metacarta.com> wrote:
>> On Wed, Dec 19, 2007 at 06:45:41AM +0100, Eric Lemoine wrote:
>>> On Dec 18, 2007 8:44 PM, Paul Spencer <pspencer at dmsolutions.ca>  
>>> wrote:
>>>> Tim,
>>>>
>>>> I think that the non-functional modifications should exclude any
>>>> significant change to the test framework (run-tests, auto-tests) -
>>>> i.e. those would require a ticket and review, since we are  
>>>> relying on
>>>> the test framework to tell when stuff is broken.  This would  
>>>> include
>>>> the changes I made yesterday, which in hindsight I should have
>>>> proposed and got some sort of review on.  It would not include the
>>>> changes I made to the actual tests to accommodate Safari's CSS  
>>>> quirks
>>>> though.
>>>>
>>>> snip ...
>>>
>>> Hello
>>>
>>> I agree with Paul that changes to the test framework should require
>>> tickets. Likewise, I think changes to the build framework (python
>>> files) should require tickets.
>>
>> Really? Why?
>>
>> I agree that all these things should be tested before commit, but I'm
>> not sure I understand why they need tickets. Especially with the  
>> build
>> system, which I've never seen anyone other than Erik or I modify  
>> (since
>> Phil/Schuyler wrote it originally Oh-So-Many-Years-Ago).
>>
>> snip ...
>
> My thinking was the following: the test framework is a core thing, if
> someone breaks it then we can end up with the situation where tests
> can no longer be run. The build system is something that users rely
> on.

My thinking was that if someone changes the framework, it could  
potentially cause tests to pass that would otherwise fail.  Then  
problems could be masked.  I'm not concerned that the framework might  
break, this is easily identifiable and fixable, at least by reverting  
to an older version.

Cheers

Paul


+-----------------------------------------------------------------+
|Paul Spencer                          pspencer at dmsolutions.ca    |
+-----------------------------------------------------------------+
|Chief Technology Officer                                         |
|DM Solutions Group Inc                http://www.dmsolutions.ca/ |
+-----------------------------------------------------------------+