unit testing (Re: [Gdal-dev] Re: GDAL 1.3.1 Alpha 2 Released)

[Sean Gillies]

On Oct 2, 2005, at 4:46 PM, Howard Butler wrote:

> It is a happy coincidence that the gdalautotest suite has worked well 
> to allow us to exercise the next generation bindings, but it is my 
> understanding that they were developed to test GDAL itself, and not 
> necessarily the script bindings.
>
> The testing suite as it currently stands can require many different 
> drivers that are not enabled by default on all builds and contain 
> significantly sized data to test the capabilities of certain drivers. 
> I wouldn't want to bloat the distribution of GDAL with all of the test 
> overhead.
>
> My third point is that the next gen bindings development has been 
> disruptive enough without turning over the apple cart of testing at 
> this time.  I vote (there's only one vote that counts, btw ;) that we 
> hold the testing suite still until the dust settles on next gen 
> development and then maybe take a look at it.
>
> Another thing to add is that an even worse scenario is to have *two* 
> test suites like MapServer has.  MapServer has the MapScript unit 
> tests (which have good coverage of methods and calls) and the 
> MapServer test suite which has coverage of the mechanicals of 
> MapServer.  It has made it difficult to divine in an automated way 
> what might be going on, and the fact that there are two different test 
> harnesses with two different invocations unnecessarily adds to the 
> developer overhead IMO.
>
> Howard

Howard,

Unless you're willing to write tests for the underlying C++ classes, 
and then use SWIG to make bindings that are as faithful as possible to 
the C++ API, separate suites for the different language bindings are 
unavoidable. It's probably unvoidable anyway since people want the  
Python/Perl/Ruby classes to behave a little bit differently than the 
C++ classes, and even a little differently from each other.

With MapServer, we have a serious testability gap between the very 
intricate and complex mapserv program, and the OO bindings generated by 
SWIG. The mapserv program does not use the mapscript classes, and 
therefore can not be tested with the mapscript unit tests. MapServer's 
testing situation is sort of sucky for you, but I think two different 
suites is the best we can do.

Sean

--
Sean Gillies
sgillies at frii dot com
http://zcologia.com