[Incubator] autobuild/autotest for OSGeo projects

[Frank Warmerdam]

Sean Gillies wrote:
> Frank,
> 
> As you might guess, because I've been nagging MapServer about this for a 
> long time, I like this proposal. It seems that here we finally start to 
> diverge from the ASF incubator model, which makes no such technical 
> requirements of any project. Question is, when we start to make 
> technical requirements, where do we draw the line? 

Sean,

Good question.  I would note I specifically used the term "highly desirable"
for the build and smoke test so that it would not be a show stopper if it
wasn't done.  But I think that the incubation committee might consider
holding back a project that refused to do several of our highly desirable
things without some good reasoning.

I do think that if we can get broad support for something in this committee
representing eight projects, then it is likely not an onerous requirement
for future projects.

I do think we might want to mandate some other things.  For instance, a
publically accessable bug reporting system could be required.  But I
also don't think we need to micromanage things. For the most part
projects will do the various "good practices" because they make sense for
the project, without us needing to push it down their throat.  Automatic
builds are an interesting example of an obvious step that GDAL and
MapServer never made part of the project, though Howard had already been
doing his automatic builds for a while.  So, in this case the pressure of
expecting a higher degree of professionalism from a foundation project
may be useful as encouragement.

I was particularly pleased to get you (Sean) involved in the committee
because I think you will be a voice holding out for greater professionalism
and quality.  Perhaps a countervailing force to the rest of us trying to
lower the bar for our own convenience. :-)

 > Build tests are good,
> but we really need functional tests to guard against problems like we 
> have had with the MapServer 4.8 release.

I had been writing "smoke test".  I'm not actually sure what the definition
of a smoke test is.   I had assumed it was at least a minimal functional
test showing that something in the project works.

It is certainly my intention that where available the smoke test should be
any unit tests or functional tests that make sense to run in an automatic
context.  In the case of MapServer I hope this would be the python unit
tests, my msautotest suite, and C unit tests (if they get launched).

However, I am hesitant to try and declare what level of automatic testing
ought to be provided by projects.  My hope is that once they have to have
something, and those tests are getting run regularly, then the project
will be inclined to keep adding to them.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF, http://osgeo.org