[GRASS5] Automated test-system for basic functionalities.

[Radim Blazek]

On 1/9/06, Sören Gebbert <soerengebbert at gmx.de> wrote:
> > It only tests if a module runs not the results, right?
> Yes. Thats one main task of the framework. It should start the user specified grass programms and/or testscripts,
> and catch/process the exitstatus (Signals) of the programms/scripts . Other tasks of the framework are, to parse the output (stdout and stderr)
> for known errors (Usage, Fatal Error, Segfault and so on) and create a readable summary and a logfile.

> > We must get it running on Windows (native).
> Then Windows should bring a bash within ;). Seriously, the framework prototype runs only within grass and uses the GRASS shell
> variables and the shell output of some programms. And if grass-shellsript's (r.mremove) are running in grass/windows
> then the framework should run too.
> But if we decide to create a testfamework that works outside grass, creating locations and stuff, we may use another language (Perl?).

Say that MSYS can be installed for testing. Anyway, I would
like to separate test description and the script running it.

> > But how can you check if module's output is correct?
> The framework should not be that smart (i think thats impossible to implement, every test may produce different output ) to check if every output is
> correct. But it should provide the functionality so the dev/user can build testfunctions to check the output (r.info -r). The framework
> should handle this funktions and should integrate them into the reports.

Without output data verification it is almos useless in my opinion.
I have 2 examples in my mind I recently saw:
- new bug introduced in v.to.rast (last line segment not rasterized)
- bug in v.in.ascii on Windows (missing header file -> wrong output)

That is why I suggested a small mapset with etalon output data
which can be used for verification of test output.

Radim