[Gdal-dev] Re: unit testing (Re: Re: GDAL 1.3.1 Alpha 2 Released)

[Charlie Savage]

I agree.  I think the only way to reasonably test the SWIG bindings is 
to call each API method from each language.  Doing this for the Ruby 
bindings (by porting the Python tests) has revealed lots of issues where 
the SWIG generated code works for Python but doesn't work for other 
languages (and even cases where its seems to work for Python but in fact 
really doesn't when put under stress due to memory management issues). 
For example, the current OSR swig bindings in CVS don't work for Ruby, 
Java or CSharp but do work for Python because SWIG handles constants and 
#defines differently for Python than the other languages.  The good news 
though is that each time we've found something like this we've been able 
to modify the swig interface files so the main core SWIG interface files 
work for all the languages.

So in my opinion, the autotest suite has been absolutely invaluable, 
because its a whole lot easier to port a bunch of tests than it is to 
develop them from scratch.  I think it would be great if it could be 
ported to every language - as a sort of "official" stamp of approval for 
that language working with GDAL.  That would then make it possible to 
test changes to the SWIG interface files, press a button, and make sure 
you didn't break something in the Java bindings, or the CSharp bindings, 
or the Python bindings.

However, I would view these tests as tests for each specific target 
language and not GDAL.  Instead, I agree with Howard that there should 
be one "official" test suite for that, which is the Python test suite.

Charlie



Sean Gillies wrote:
> On Oct 2, 2005, at 4:46 PM, Howard Butler wrote:
> 
>> It is a happy coincidence that the gdalautotest suite has worked well 
>> to allow us to exercise the next generation bindings, but it is my 
>> understanding that they were developed to test GDAL itself, and not 
>> necessarily the script bindings.
>>
>> The testing suite as it currently stands can require many different 
>> drivers that are not enabled by default on all builds and contain 
>> significantly sized data to test the capabilities of certain drivers. 
>> I wouldn't want to bloat the distribution of GDAL with all of the test 
>> overhead.
>>
>> My third point is that the next gen bindings development has been 
>> disruptive enough without turning over the apple cart of testing at 
>> this time.  I vote (there's only one vote that counts, btw ;) that we 
>> hold the testing suite still until the dust settles on next gen 
>> development and then maybe take a look at it.
>>
>> Another thing to add is that an even worse scenario is to have *two* 
>> test suites like MapServer has.  MapServer has the MapScript unit 
>> tests (which have good coverage of methods and calls) and the 
>> MapServer test suite which has coverage of the mechanicals of 
>> MapServer.  It has made it difficult to divine in an automated way 
>> what might be going on, and the fact that there are two different test 
>> harnesses with two different invocations unnecessarily adds to the 
>> developer overhead IMO.
>>
>> Howard
> 
> Howard,
> 
> Unless you're willing to write tests for the underlying C++ classes, and 
> then use SWIG to make bindings that are as faithful as possible to the 
> C++ API, separate suites for the different language bindings are 
> unavoidable. It's probably unvoidable anyway since people want the  
> Python/Perl/Ruby classes to behave a little bit differently than the C++ 
> classes, and even a little differently from each other.
> 
> With MapServer, we have a serious testability gap between the very 
> intricate and complex mapserv program, and the OO bindings generated by 
> SWIG. The mapserv program does not use the mapscript classes, and 
> therefore can not be tested with the mapscript unit tests. MapServer's 
> testing situation is sort of sucky for you, but I think two different 
> suites is the best we can do.
> 
> Sean
> 
> -- 
> Sean Gillies
> sgillies at frii dot com
> http://zcologia.com