[OpenLayers-Dev] Automated Testing
tschaub at openplans.org
Mon Dec 17 15:29:50 EST 2007
Christopher Schmidt wrote:
> Semi-Automated testing is now here.
This is great Chris. I do have some thoughts that I'm hoping I can get
your feedback on (with regard to rapid development).
I think this process is pretty rapid:
1) I make a change in a working copy.
2) I run tests locally.
3) Since I know what code I changed, if a test fails, it generally
doesn't take that long for me to find the problem.
This is the slowest and most tedious process I can imagine:
1) Someone makes a change on their working copy.
2) They commit.
3) I update, make a change, run tests and get confused by failing tests.
4) I spend time trying to figure out who did what.
5) bla bla bla
I like the idea of automated testing. But I don't know what I'd do if I
got an email that a bunch of tests were failing. Probably just get
frustrated that someone committed a change without running tests.
Basically, I think the most rapid development course is one where the
person making a change confirms that tests pass - and if they don't,
that person fixes the issue.
Again, I appreciate the effort you are putting in to this. I just want
to make sure that this doesn't take the place of committers testing
their own changes.
> To throw yourself into the automated testing pool, simply go to:
> in IE or FF. (Safari doesn't work at all, and Opera has a number of
> failures which I haven't yet debugged.)
> This page will automatically reload every 20 minutes and run again.
> (Okay, this is perhaps a bit much, given our current commit rate, but I
> haven't yet got it to the point where it can check if the code has
> changed and run it again... hopefully I'll get there at some point.)
> You can leave your browser open doing this, and it should act as an
> 'automatic' testing instance.
> The results are posted to http://openlayers.org/test/results.cgi , with
> failures getting a seperate HTML page with the total test text, and an
> email being sent to the new autotest list:
> I'm going to work on setting up an IE machine at work to run the tests
> -- I haven't actually tested with IE yet -- which would let us get
> reports if we commit anything which causes the tests to fail in IE, and
> hoepfully get one set up the same in FF.
> Next on my list is getting a couple of minor tests running against
> single file builds -- the primary thing with those being just to test
> that they compiled correctly, rather than that they're providing the
> full support of the library. That way, when I commit something that
> breaks the build, I can take responsibility for it and fix it.
> And then, of course, there's the need to fix the tests in Opera and
> Safari, so we can get testing going on those as well.
> I'm working hard to figure out how we can make OpenLayers more resilient
> against a slightly less weighty commit process, so that we can open up
> development more. For the past several months, the code has not changed
> much, and I think a big reason is just the fact that we don't have the
> infrastructure in place to make rapid development tenable. I'm hopeful
> we can fix this through the use of technology, and accelerate the pace
> of OpenLayers development in the process.
> If any of this seems like a wrong path to anyone, please feel free to
> let me know, and I can back all this out -- there's really not much to
> it, but having 'broken the build' several times this weekend, I felt
> like I needed to make up for it, and this is my first attempt.
More information about the Dev