Automatic coverage testing of mapserv?

[Petter Reinholdtsen]

A while ago, I came across the following article in Linux Journal.

<URL:http://www.linuxjournal.com/article.php?sid=6758>

  Maybe you've always wondered what the gcov utility that comes with
  GCC is used for, or maybe your new project at work has a regulatory
  or customer requirement that your delivered software be tested to a
  certain percentage of coverage, and you are looking for how to
  accomplish that task. In this article, I introduce the general ideas
  of coverage measurement and of performance profiling, along with the
  standard GNU tools (gcov and gprof) used in these two techniques.

To do useful coverage testing, one need automated regression tests.  I
notice some tests in the tests/ directory, but I believe more are
needed.

I just did a simple test, running mapserv on two simple map files, and
ggcov reported ca. 17% of the source code lines used.  Did anyone work
on creating automatic regression tests for MapServer?  Any ideas on
how to do it?

I started by creating a shell script calling mapserver with
QUERY_STRING set to a usable value.  This would work for simple tests.
I guess the simple tests will consist of data files, a map file, a
list of arguments to use when calling mapserv, and the expected
output.  Some of the data files could probably be reused across tests.
I'm not sure if this is a good approach.  Any comments?

My goal is to create tests that can run automatically, and which can
be used both to detect bugs in mapserver, but also to exercise as much
of the code as possible.  When the tests are working, I plan to run
the tests within valgrind and check any problems reported by valgrind.
The problems reported should be checked out and fixed.  I hope I get
this far. :)