Issue #1664: Unit Test support for mapserver
Howard Butler
hobu at IASTATE.EDU
Fri Feb 17 09:15:58 EST 2006
A few thoughts...
1. The developer culture of MapServer is fairly conservative in my
experience. Changing lots of code has two risks/costs... familiarity
and breakage. Unit testing only addresses one of these.
2. There are already nightly builds. I maintain them on Windows
(the pickiest of our supported platforms) at
<http://hobu.stat.iastate.edu/mapserver/build_output/>. I run the
Python test suite nightly also.
3. There is already an initiative to more fully develop a C API for
MapServer that MapScript etc can take advantage of. The unittesting
initiative must be in alignment with this.
4. The RFC is not specific to what it is exactly we are *changing*
about MapServer other than to have unittests.
5. A dependency on cunit for regular users is a deal breaker.
6. The RFC is not in the same style and format as the rest of the
RFCs that have been written.
Howard
At 11:05 AM +0100 2/17/06, Umberto Nicoletti wrote:
>I have published a first public draft of rfc-12: Unit tests
>
>http://mapserver.gis.umn.edu/development/rfc/ms-rfc-12
>
>Please share your thoughts.
>
>Regards,
>Umberto
>
>
>On 2/16/06, Umberto Nicoletti <umberto.nicoletti at gmail.com> wrote:
>> I am trying to add an rfc, could someone enable rfc editing on the
>> plone site for my account (unicoletti) or explain how to write one?
>>
>> Regards,
>> Umberto
>>
>> On 2/15/06, Umberto Nicoletti <umberto.nicoletti at gmail.com> wrote:
>> > Frank,
>> > first of all thanks for your comments.
>> >
>> > On 2/14/06, Frank Warmerdam <warmerdam at pobox.com> wrote:
>> > > Umberto Nicoletti wrote:
>> > > > 'Course I am, as a matter of facts your tests are functional tests (a
>> > > > rather different beast from unit tests), while Sean's are unit tests,
>> > > > but for python Mapscript.
>> > >
>> > > Umberto,
>> > >
>> > > OK, cool.
>> > >
>> > > > The unit tests I am introducing can go deeper and test the intimate of
>> > > > mapserver's code. Also unit tests can test a very limited and well
>> > > > defined subset of code, making memory leaks/memory
>> > > > misbehaviours/threading issue easier to spot. For instance I have
>> > > > written unit tests for the connection pooling which is not covered by
>> > > > Sean's tests because connection pooling functions are not exposed by
>> > > > mapscript and neither by yours.
>> > >
>> > > I think running valgrind on shp2img or mapserv sessions can
>> > > identify most memory leaks. You are quite right that none of the
>> > > existing tests address pooling, or threading issues and it may be
>> > > challenging to do this at any level other than the C API (as you
>> > > suggest).
>> > >
>> > > > IMHO it is not the case I have to argue in favour of unit tests, but
>> > > > quite the contrary, as the beneficial effects of unit testing seem
>> > > > pretty obvious to me (and to anybody who's ever used them).
>> > >
>> > > I'm not arguing that there is no benefit to unit tests, and I have
>> > > used them. But if C level unit tests are to reach anything near their
>> > > full value, it is necessary to get others involved in extending them
>> > > and running them.
>> >
>> > Agreed.
>> >
>> > IMO Unit testing is not an opt-in, but a programming approach that
>> > must be used by all developers, that is a requirement.
>> > This is a clear cut case (to quote Tarantino's Jackie Brown) in my
>> > mind: either unit tests get into the repository AND EVERYBODY adopts
>> > them or they don't get into the repository at all.
>> >
>> > >
>> > > > The reasons are:
>> > > > - better code
>> > > > - safer modifications
>> > > > - more confidence
>> > > > - regression testing
>> > > > - automation of repetitive tasks
>> > > > - integration in the build process
>> > >
>> >
>> > I forgot to add the documentation value that unit tests add to
>>existing code.
>> >
>> > > I don't see that the above items require another testing
>> > > mechanism with the possible exception of things that cannot
>> > > practically be unittested via the python mapscript tests, or
> > > > functionally tested via msautotest.
>> > >
>> >
>> > Unit tests do not replace functional testing, rather functional tests
>> > complete unit tests.
>> >
>> > > > See for instance bug 1629: if we had unit tests we could have
>> > > > committed the modifications to 4.8.1 already without having to recall
>> > > > why in the first we made those changes in the first place.
>> > >
>> > > I have briefly reviewed Bug 1629 and I cannot see why a python
>> > > mapscript or msautotest test item would have been inadequate
>> > > to handle it. Of course, there isn't any such test for PostGIS
>> > > support in mapserver. But we don't need a new approach, we just
>> > > need to more fully fill out the existing approach.
>> > >
>> >
>> > Of course you can do it with mapscript or msautotest, but that
>> > requires a postgis setup which not everybody has available.
>> > With C unit tests you can just test the query composer function and
>> > verify that the string it produces is the one you are expecting,
>> > dropping entirely the need for a full fledged build environment.
>> > Of course this kind of testing requires refactoring of existing code
>> > to make it more easily testable.
>> > For an example see the two methods I have added to the
>>connection pooling apis:
>> >
>> > int msConnPoolGetConnectionCount();
>> > int msConnPoolGetConnectionMax();
>> >
>> > Best regards,
>> > Umberto
>> >
>> > > >> Currently it seems we have roughly as many test suite
>>mechanisms as we
>> > > >> have developers interested in using and supporting them.
>> > > >>
>> > > >> I imagine you ought to write an RFC proposal on your
>>unittest approach
>> > > >> before it is integrated into MapServer.
>> > > >>
>> > > >
>> > > > If that is necessary I'll do it, but I'd better code than write.
>> > >
>> > > I'm sure we would all rather code than write RFCs, but we decided to
>> > > require RFCs for a certain class of changes in MapServer to ensure
>> > > we have coherent progress, avoid bloat, and fine tune approaches to
>> > > new features before they we are committed to them.
>> > >
>> > > You can certainly maintain a test harnass on your own, but if you
>> > > want it in the tree I think it deserves an RFC. As per ms-rfc-1,
>> > > Steve is the final arbiter of what does or does not need an RFC.
>> > >
>> > > Please don't consider my feedback as saying I don't want C unit
>> > > tests. If there is a good case for why we need them instead of
>> > > python unit tests, and if you are willing to do the work to establish
>> > > them then I'm keen on it.
>> > >
>> > > Best regards,
>> > > --
>> > >
>>---------------------------------------+--------------------------------------
>> > > I set the clouds in motion - turn up | Frank Warmerdam,
>>warmerdam at pobox.com
>> > > light and sound - activate the windows | http://pobox.com/~warmerdam
>> > > and watch the world go round - Rush | Geospatial Programmer for Rent
>> > >
>> > >
>> >
>>
More information about the mapserver-dev
mailing list