[OpenLayers-Dev] manual acceptance tests

[Tim Schaub]

Hey-

Christopher Schmidt wrote:
> On Fri, Nov 09, 2007 at 08:49:53PM +0100, Eric Lemoine wrote:
>> On Nov 9, 2007 5:50 PM, Tim Schaub <tschaub at openplans.org> wrote:
>>> Hey-
>>>
>>> Our automated tests are a mix of acceptance and unit tests.  These are
>>> nice (because they are automated), but we need more (manual) acceptance
>>> tests to fully test things like the renderers.
>>>
>>> Instead of scattering manual acceptance tests in the examples directory,
>>> I'd like to have an "acceptance" directory in our "tests" directory (or
>>> "manual" if that makes more sense to folks).  This would make it clear
>>> for developers what acceptance tests needed to be run manually, and
>>> would make it less confusing to users who poke around the examples
>>> directory and stumble into the weird examples that are actually tests
>>> (speaking for myself here).
>>>
>>> Anyone opposed?  Or got a better idea?
>>>
>>> Barring feedback, I think I'll make such a change on Monday.  (This will
>>> only involve adding a directory to the tests directory and starting to
>>> compile regular manual acceptance tests there.)
>> Tim,
>>
>> As you see it, would these acceptance tests need to be run before
>> commit? Would they be part of the commit process?
> 
> I see them only as things that need to be run before a release, not
> before each commit. (And with that in mind, I'm in favor.)
> 

Yeah, I don't want to impose any more process on ourselves.  Just a bit 
more organization.  I'd say it's good practice to run a few acceptance 
tests when you know your modifications might affect things that your 
unit tests don't explicitly treat.  But I agree that we'd only add it as 
another step in the release process - then leave it up to developers to 
decide when else to run them.

Tim

> Regards,