[GRASS5] Automated test-system for basic functionalities.

[Sören Gebbert]

Hi Radim,

On Monday 09 January 2006 18:26, Radim Blazek wrote:
> On 1/9/06, Sören Gebbert <soerengebbert at gmx.de> wrote:
> > Interesting, Markus and i just had the same thoughts and developed a small prototype of a testframework
> > which tests some GRASS modules of success/failure and try to guess the failure.
> > www-pool.math.tu-berlin.de/~soeren/grass/modules/GRASS_Testsuite.tar.gz
> 
> It only tests if a module runs not the results, right?
Yes. Thats one main task of the framework. It should start the user specified grass programms and/or testscripts,
and catch/process the exitstatus (Signals) of the programms/scripts . Other tasks of the framework are, to parse the output (stdout and stderr)
for known errors (Usage, Fatal Error, Segfault and so on) and create a readable summary and a logfile. 
It should also be able to create reports (html ...?).
> 
> > > Currently there is no clear idea, but thoughts about a shell-script
> > > with configuration file have been made.
> > I think bash shell scripts may be the best choice for this kind of tests, the prototype i developed
> > is completely written in bash. It is a *nix standart and we are only able to test the success or failure of the modules.
> 
> We must get it running on Windows (native).
Then Windows should bring a bash within ;). Seriously, the framework prototype runs only within grass and uses the GRASS shell
variables and the shell output of some programms. And if grass-shellsript's (r.mremove) are running in grass/windows
then the framework should run too. 
But if we decide to create a testfamework that works outside grass, creating locations and stuff, we may use another language (Perl?). 
Any suggestions are welcome.
> 
> > > The dataset is also a problem, since it must be small enough to
> > > distribute it to testers. Spearfish is probably too big...
> > I think, the test should be work with every dataset, not only with a provided one.
> 
> But how can you check if module's output is correct?
The framework should not be that smart (i think thats impossible to implement, every test may produce different output ) to check if every output is
correct. But it should provide the functionality so the dev/user can build testfunctions to check the output (r.info -r). The framework 
should handle this funktions and should integrate them into the reports. 

The framework do not test anything, it just provide a functionality to easy create and manage test's.
To have a testsuite we have to implement the framework and the testscripts! (i made some small examples)
> 
> Radim
> 
Regards
Soeren