[mapserver-dev] c unit tests in MapServer

[Paul Ramsey]

We ended up with belt *and* suspenders in PostGIS, and it works OK. We
have an online regression test, which actually runs the data base and
slams SQL through it, which tests for correctness in the large. But we
also have a cunit hardness that tests the underlying library, things
like computational geometry, format parsers, etc, in isolation from
the database backend. Once a code base gets this far along, and is
this mature, adding coverage is perhaps "too little, too late", but
once you have coverage it's hard to imagine working without it.

P.

On Thu, Dec 4, 2014 at 12:34 PM, Lime, Steve D (MNIT)
<Steve.Lime at state.mn.us> wrote:
> First, thanks so much for your willingness to report/research/fix bugs –
> it’s invaluable.
>
> I don’t know that we’d need to make the full leap to a C Unit testing
> framework to make use of them on a ticket by ticket basis. If you wanted to
> roll a simple C program that tests just the function in question I’d
> certainly want to use it. It makes sense in this case. So often though bugs
> seem to involve a series of functions performing a complex operation where a
> more complete configuration (e.g. data+mapfile) and  request is necessary to
> trigger the issue.
>
>
>
> Steve
>
>
>
> From: mapserver-dev-bounces at lists.osgeo.org
> [mailto:mapserver-dev-bounces at lists.osgeo.org] On Behalf Of geographika
> Sent: Wednesday, December 03, 2014 8:56 AM
> To: mapserver-dev at lists.osgeo.org
> Subject: [mapserver-dev] c unit tests in MapServer
>
>
>
> Hi list,
>
>
> With almost 20 years of development, the MapServer codebase can be
> overwhelming at first. For most open source projects I've found the easiest
> way into helping with a project is:
>
>
>
> 1.       Reporting bugs
>
> 2.       Finding the problematic code
>
> 3.       Submitting a patch and unit test.
>
>
>
> In 2006 there was a Request For Comments (#12) by Umberto Nicoletti about
> adding C Unit tests:
>
>
>
> http://www.mapserver.org/development/rfc/ms-rfc-12.html
>
> Also see GitHub issue #1664:
> https://github.com/mapserver/mapserver/issues/1664
>
> Obviously a full test suite of existing code isn't possible, however if the
> testing framework and skeleton folder/file structure was set up it may help
> more concrete contributions from developers outside of the core MapServer
> team.
>
>
>
> There have been a few bugs I've submitted where I would feel comfortable
> adding a unit test to an existing file to show the error, but creating full
> msautotest examples seems very daunting. As a concrete example while testing
> the latest dev build I noticed the msParseTime function in the maptime.c
> module threw an error. Unit tests for this function seem like the perfect
> solution for these types of bugs, and to avoid breaking changes in the
> future.
>
>
>
> https://github.com/mapserver/mapserver/commit/0cc47e3027a75cf18c5310694877999c6b245371
>
>
>
> C Unit tests have come up a few times in the past e.g. Paul Ramsey's post
> about PostGIS driver testing:
> http://osgeo-org.1560.x6.nabble.com/cunit-RFC-12-td4261340.html
>
>
>
> Frank Warmerdam's response (in 2008) was -
>
>
>
> My preference remains to test specific functions as part of a larger
>
> operations since we already have such technology in place (ie. msautotest)
>
> or to expose the functionality through mapscript and use mapscript tests
>
> like Sean's python unit tests.
>
>
>
> I do not feel that introducing another testing methodology into the
>
> project would be wise.
>
>
>
> The Python unit tests are again useful, but require testing against full
> builds, a difficult task for those not familiar with make files and
> gathering dependencies.
>
>  Testing database drivers as I understand it would require a full test
> database setup with msautotest, whereas for many bugs it is clear which
> function needs modifying and a unit test could be added quickly to highlight
> and help fix the bug.
>
>
>
> I've only experience in Python and .NET unit testing so perhaps C testing is
> more trouble than it is worth.
> I guess it is up to the core devs whether it has any benefit to themselves,
> or the project as a whole, whether this issue should be reconsidered.
>
>
>
> Regards,
>
> Seth
>
>
>
> --
> web: http://geographika.co.uk
> twitter: @geographika
>
>
>
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev