MapServer Development Process?

[Frank Warmerdam]

On 6/23/05, Sean Gillies <sgillies at frii.com> wrote:
> Hi all,
> 
> After listening to van Gulik's presentation, does anybody feel that the
> MapServer project might benefit from a harder look at its development
> process?

Sean, 

The talk has made me think about code traceability, and for some time
I have felt we needed a slightly more formal way of dealing with potentially
contentious code/functionality changes. 

> I like the voting idea, and I think we should adopt it. I also think
> that votes should be weighted by the number of tests that you have
> written and committed to CVS ;) That was a joke. A voting system will
> require better communication. The Apache projects demand that all code
> decisions take place in the open, and we should adopt this as well.
> This means that decisions will not be made between Howard and I on IRC,
> or behind closed doors at DM Solutions.

Some of the offices at DMSG don't even have doors!  And those that
do are seldom closed.  

> In the open means on the
> mapserver-dev list. We do a fair job of this, but it could be better.
> Writing this in stone and holding each other to it could help to head
> off future problems.

I think that this should apply to architectural, API or user changes with
any significant scope but that folks should still be free to do local tweaks
to the areas of code they "own" without needing a discussion. For instance,
if Assefa wants to add support for a new Filter operator, or I add support for
greyscale+alpha in the raster renderer, those don't have adverse effects
on users, and don't affect widely used interfaces in the code.  But changes
with broader effects should be (in my opinion) explained and justified on
mapserver-dev.  And we might well use the same voting approach as 
Apache.  Essentially this would mean that if no one had an issue or wanted
to get involved the person could assume permission to proceed in a couple
of days.   Well, they could proceed in the meantime, but not commit. 

> It's also long past time to have some tests for MapServer. I propose
> that we start with the new input plugins: no plugins committed without
> tests. These should be tests that can be run under valgrind, and should
> also be automate-able.

Well, we do have a significant though by no means comprehensive set
of tests now between your unit tests and my autotest suite.  I would like
to see their use become more a part of our culture though.   In many
ways I think autotests for new input plugins (stuff like SDE, Oracle, etc)
are the hardest case because they will inherently be very dependent on
specific server circumstances.  A particular database will need to be
setup, etc.  

> My last point is not so much about process as about direction and
> design. I cracked Frank up after the conference by asking what he
> thought about how successful Apache became following a complete
> re-write. I was only half joking. MapServer, as a prominent user said
> to me, has no architecture. Do we put off creating an architecture
> until it becomes practically impossible to add new useful features? It
> seems to me that GRASS got in trouble this way. Do we really want to
> wait until MapServer is widely perceived as clunky and fragile before
> making a change?

Well, we do have an architecture, so I think you will need to be more
specific about what sorts of specifications you would like to see.  
Is it that you want things documented better?  Perhaps you want 
specified APIs for mapserver?  Currently I think our "architecture" 
centers around a common "shapeObj" definition, and a layer rendering
approach.   

We haven't traditionally tried to maintain a stable external API except as 
represented in MapScript.  I think we are moving to have the mapscript
API relfected in concrete C functions rather than having alot of 
functionality embedded in the swig wrappers. 

We also have an architecture for output driver handling, input datasources
and so forth.  However, the "specification" hasn't been all that clear at 
times and we haven't generally made them plugable which I think would
be desirable. 

So, I guess my conclusion is, point out what aspects of the architecture
you want improved.  

I would add that I am still not keen on a big rewrite, but I am interested in
continued improvement on a sub-system by sub-system basis.  I like to
think of my effort with outputFormatObj's as bring clarity and a cleaner
architecture to supporting multiple output formats (for instance). 

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent