[GRASS5] Automated test-system for basic functionalities.

[Roger Bivand]

On Tue, 10 Jan 2006, Radim Blazek wrote:

> On 1/9/06, Sören Gebbert <soerengebbert at gmx.de> wrote:
> > > It only tests if a module runs not the results, right?
> > Yes. Thats one main task of the framework. It should start the user specified grass programms and/or testscripts,
> > and catch/process the exitstatus (Signals) of the programms/scripts . Other tasks of the framework are, to parse the output (stdout and stderr)
> > for known errors (Usage, Fatal Error, Segfault and so on) and create a readable summary and a logfile.
> 
> > > We must get it running on Windows (native).
> > Then Windows should bring a bash within ;). Seriously, the framework prototype runs only within grass and uses the GRASS shell
> > variables and the shell output of some programms. And if grass-shellsript's (r.mremove) are running in grass/windows
> > then the framework should run too.
> > But if we decide to create a testfamework that works outside grass, creating locations and stuff, we may use another language (Perl?).
> 
> Say that MSYS can be installed for testing. Anyway, I would
> like to separate test description and the script running it.
> 
> > > But how can you check if module's output is correct?
> > The framework should not be that smart (i think thats impossible to implement, every test may produce different output ) to check if every output is
> > correct. But it should provide the functionality so the dev/user can build testfunctions to check the output (r.info -r). The framework
> > should handle this funktions and should integrate them into the reports.
> 
> Without output data verification it is almos useless in my opinion.
> I have 2 examples in my mind I recently saw:
> - new bug introduced in v.to.rast (last line segment not rasterized)
> - bug in v.in.ascii on Windows (missing header file -> wrong output)
> 
> That is why I suggested a small mapset with etalon output data
> which can be used for verification of test output.

I agree, we need to compare the target output data with the generated. 
Given that they are most often binary, we can't use diff (R test uses diff 
on plain text files), but could we use checksums or something like a file 
digest to show same/different?

Roger

> 
> Radim
> 
> _______________________________________________
> grass5 mailing list
> grass5 at grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass5
> 

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no