Automatic coverage testing of mapserv?

[Steve Lime]

There is a project server in the sense that it hosts CVS (and nothing
else) so tests could be integrated into the nightly build process there.
Or are you thinking of something else?

Steve

>>> Sean Gillies <sgillies at FRII.COM> 01/27/05 5:56 PM >>>
On Jan 27, 2005, at 4:23 PM, Petter Reinholdtsen wrote:

> A while ago, I came across the following article in Linux Journal.
>
> <URL:http://www.linuxjournal.com/article.php?sid=6758>
>
>   Maybe you've always wondered what the gcov utility that comes with
>   GCC is used for, or maybe your new project at work has a regulatory
>   or customer requirement that your delivered software be tested to a
>   certain percentage of coverage, and you are looking for how to
>   accomplish that task. In this article, I introduce the general ideas
>   of coverage measurement and of performance profiling, along with the
>   standard GNU tools (gcov and gprof) used in these two techniques.
>
> To do useful coverage testing, one need automated regression tests.  I
> notice some tests in the tests/ directory, but I believe more are
> needed.
>
> I just did a simple test, running mapserv on two simple map files, and
> ggcov reported ca. 17% of the source code lines used.  Did anyone work
> on creating automatic regression tests for MapServer?  Any ideas on
> how to do it?
>
> I started by creating a shell script calling mapserver with
> QUERY_STRING set to a usable value.  This would work for simple tests.
> I guess the simple tests will consist of data files, a map file, a
> list of arguments to use when calling mapserv, and the expected
> output.  Some of the data files could probably be reused across tests.
> I'm not sure if this is a good approach.  Any comments?
>
> My goal is to create tests that can run automatically, and which can
> be used both to detect bugs in mapserver, but also to exercise as much
> of the code as possible.  When the tests are working, I plan to run
> the tests within valgrind and check any problems reported by valgrind.
> The problems reported should be checked out and fixed.  I hope I get
> this far. :)
>
>

Go Petter!  You da man :)

I am primarily a mapscript, rather than mapserv, user.  I've been
writing mapscript unit tests for new features since version 4.0 and
running them under valgrind, but these also span just fraction
(unmeasured fraction) of the MapServer code and miss a lot of
mapserv-specific code.

I welcome regression tests whole heartedly!  Will you consider using
the testing data that we are already distributing under mapserver/tests
to make the tests as portable as possible?

Next we need is a project server where the tests can be run nightly.
Hint, hint ...

cheers,
Sean

--
Sean Gillies
sgillies at frii dot com
http://users.frii.com/sgillies