[OpenLayers-Dev] Automated Testing

[Cameron Shorter]

Chris,
Well done. The automated testing should reduce the cost of providing 
quality significantly.

In answer to some of Tim's concerns re testing before committing: This 
could be partially addressed by offering automated tests for sandboxes too.
If I have a complex fix, I could develop it in the sandbox, commit it, 
trigger automated testing, then merge back into the trunk.

Christopher Schmidt wrote:
> On Mon, Dec 17, 2007 at 05:26:06PM -0700, Tim Schaub wrote:
>   
>> I like the idea (and if we were making a policy I would suggest) that a 
>> patch creator does three things:
>>
>> 1) make a ticket (more documentation than commit messages)
>>     
>
> I think that this is true, with the exception of things like:
>
> http://trac.openlayers.org/changeset/5439
>
> Which I don't really think need a ticket.   
>
>   
>> 2) run at least the tests you know are relevant in at least one browser
>> 3) report what you did on the ticket
>>
>> Are you suggesting less?
>>     
>
> Nope, but I think that we're still liable to break trunk that way, and
> I'd like to make it so that breaking trunk is found by a machine instead
> of by a human.
>
> Specifically, I expect that as we support a wider range of browsers with
> our tests (Opera is now as well supported as Firefox), we will find
> things that break other browsers that we don't think to test... and it
> lets us actually build a sane grid of what we support... and it lets us
> ensure that we don't have to run around like chickens with our heads cut
> off repairing Opera at release time.
>
> Right now, I think the requirement for 2) is ardurous because it's not
> 'in at least one browser' -- it's "in at least FF + IE before it hits
> trunk", and I think that's unrealistic long term as we add more
> developers to the project. 
>
> Regards,
>