[mapserver-dev] c unit tests in MapServer

Thu Dec 11 09:02:28 PST 2014

No, it isn't really an entry point, just a quality point. "Hey, when
you add a feature, add a test." and because debugging the unit tests
is way easier than debugging the whole database backend (particularly
for things like memory handling) it has gotten a lot of use.

P.

On Wed, Dec 10, 2014 at 3:27 AM, geographika <geographika at gmail.com> wrote:
> Hi Paul,
>
> Did you get more participation from devs new to PostGIS with the unit tests?
> Or have these only been created by the core dev team? It was the easy route
> in for me on Python and JavaScript projects, but C testing doesn't seem as
> simple.
>
> Steve - I'm handy enough with Python so will try to create a test case using
> mapscript, although having trouble building a dev version of MapServer even
> with the Windows SDKs.
>
> Regards,
>
> Seth
>
>
> On 08/12/2014 20:07, Paul Ramsey wrote:
>>
>> Once the tester is set up, it's pretty easy, IMO. The PostGIS setup is
>> basically the cu_tester application with all the cunit-specific and
>> project-specific code (like our hooking into our error handlers), then
>> all the test suites, which are very simple.
>>
>>
>> https://github.com/postgis/postgis/blob/svn-trunk/liblwgeom/cunit/cu_tester.c
>>
>> Most of the code there is actually just stuff to allow users to choose
>> a particular suite or test to run, rather than running all 1000+
>> (useful when debugging a particular test)
>>
>> Everything else is just the tests themselves, which are pretty simple.
>>
>>
>> https://github.com/postgis/postgis/blob/svn-trunk/liblwgeom/cunit/cu_in_wkt.c
>>
>> Since much of the logic in Mapserver is in libmapserver now, cunit
>> could be a pretty good fit for testing that piece, in the same way
>> that it tests just the liblwgeom piece of PostGIS.
>>
>> P.
>>
>>
>> On Mon, Dec 8, 2014 at 8:07 AM, Lime, Steve D (MNIT)
>> <Steve.Lime at state.mn.us> wrote:
>>>
>>> How bad was it to get started? A quick peek at the CUnit simple example
>>> didn't seem that simple.
>>>
>>> -----Original Message-----
>>> From: Paul Ramsey [mailto:pramsey at cleverelephant.ca]
>>> Sent: Saturday, December 06, 2014 5:33 PM
>>> To: Lime, Steve D (MNIT)
>>> Cc: geographika at gmail.com; mapserver-dev at lists.osgeo.org
>>> Subject: Re: [mapserver-dev] c unit tests in MapServer
>>>
>>> We ended up with belt *and* suspenders in PostGIS, and it works OK. We
>>> have an online regression test, which actually runs the data base and slams
>>> SQL through it, which tests for correctness in the large. But we also have a
>>> cunit hardness that tests the underlying library, things like computational
>>> geometry, format parsers, etc, in isolation from the database backend. Once
>>> a code base gets this far along, and is this mature, adding coverage is
>>> perhaps "too little, too late", but once you have coverage it's hard to
>>> imagine working without it.
>>>
>>> P.
>>>
>>> On Thu, Dec 4, 2014 at 12:34 PM, Lime, Steve D (MNIT)
>>> <Steve.Lime at state.mn.us> wrote:
>>>>
>>>> First, thanks so much for your willingness to report/research/fix bugs
>>>> – it’s invaluable.
>>>>
>>>> I don’t know that we’d need to make the full leap to a C Unit testing
>>>> framework to make use of them on a ticket by ticket basis. If you
>>>> wanted to roll a simple C program that tests just the function in
>>>> question I’d certainly want to use it. It makes sense in this case. So
>>>> often though bugs seem to involve a series of functions performing a
>>>> complex operation where a more complete configuration (e.g.
>>>> data+mapfile) and  request is necessary to trigger the issue.
>>>>
>>>>
>>>>
>>>> Steve
>>>>
>>>>
>>>>
>>>> From: mapserver-dev-bounces at lists.osgeo.org
>>>> [mailto:mapserver-dev-bounces at lists.osgeo.org] On Behalf Of
>>>> geographika
>>>> Sent: Wednesday, December 03, 2014 8:56 AM
>>>> To: mapserver-dev at lists.osgeo.org
>>>> Subject: [mapserver-dev] c unit tests in MapServer
>>>>
>>>>
>>>>
>>>> Hi list,
>>>>
>>>>
>>>> With almost 20 years of development, the MapServer codebase can be
>>>> overwhelming at first. For most open source projects I've found the
>>>> easiest way into helping with a project is:
>>>>
>>>>
>>>>
>>>> 1.       Reporting bugs
>>>>
>>>> 2.       Finding the problematic code
>>>>
>>>> 3.       Submitting a patch and unit test.
>>>>
>>>>
>>>>
>>>> In 2006 there was a Request For Comments (#12) by Umberto Nicoletti
>>>> about adding C Unit tests:
>>>>
>>>>
>>>>
>>>> http://www.mapserver.org/development/rfc/ms-rfc-12.html
>>>>
>>>> Also see GitHub issue #1664:
>>>> https://github.com/mapserver/mapserver/issues/1664
>>>>
>>>> Obviously a full test suite of existing code isn't possible, however
>>>> if the testing framework and skeleton folder/file structure was set up
>>>> it may help more concrete contributions from developers outside of the
>>>> core MapServer team.
>>>>
>>>>
>>>>
>>>> There have been a few bugs I've submitted where I would feel
>>>> comfortable adding a unit test to an existing file to show the error,
>>>> but creating full msautotest examples seems very daunting. As a
>>>> concrete example while testing the latest dev build I noticed the
>>>> msParseTime function in the maptime.c module threw an error. Unit
>>>> tests for this function seem like the perfect solution for these types
>>>> of bugs, and to avoid breaking changes in the future.
>>>>
>>>>
>>>>
>>>> https://github.com/mapserver/mapserver/commit/0cc47e3027a75cf18c531069
>>>> 4877999c6b245371
>>>>
>>>>
>>>>
>>>> C Unit tests have come up a few times in the past e.g. Paul Ramsey's
>>>> post about PostGIS driver testing:
>>>> http://osgeo-org.1560.x6.nabble.com/cunit-RFC-12-td4261340.html
>>>>
>>>>
>>>>
>>>> Frank Warmerdam's response (in 2008) was -
>>>>
>>>>
>>>>
>>>> My preference remains to test specific functions as part of a larger
>>>>
>>>> operations since we already have such technology in place (ie.
>>>> msautotest)
>>>>
>>>> or to expose the functionality through mapscript and use mapscript
>>>> tests
>>>>
>>>> like Sean's python unit tests.
>>>>
>>>>
>>>>
>>>> I do not feel that introducing another testing methodology into the
>>>>
>>>> project would be wise.
>>>>
>>>>
>>>>
>>>> The Python unit tests are again useful, but require testing against
>>>> full builds, a difficult task for those not familiar with make files
>>>> and gathering dependencies.
>>>>
>>>>   Testing database drivers as I understand it would require a full test
>>>> database setup with msautotest, whereas for many bugs it is clear
>>>> which function needs modifying and a unit test could be added quickly
>>>> to highlight and help fix the bug.
>>>>
>>>>
>>>>
>>>> I've only experience in Python and .NET unit testing so perhaps C
>>>> testing is more trouble than it is worth.
>>>> I guess it is up to the core devs whether it has any benefit to
>>>> themselves, or the project as a whole, whether this issue should be
>>>> reconsidered.
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Seth
>>>>
>>>>
>>>>
>>>> --
>>>> web: http://geographika.co.uk
>>>> twitter: @geographika
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mapserver-dev mailing list
>>>> mapserver-dev at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
>