[GRASS-dev] Re: GRASS Test suite

[Soeren Gebbert]

Hello Anne,

2011/7/4 Anne Ghisla <a.ghisla at gmail.com>:
> Le lundi 04 juillet 2011 à 10:52 +0200, Soeren Gebbert a écrit :
>> Hi all,
>>
>> 2011/7/4 Martin Landa <landa.martin at gmail.com>:
>> > Hi,
>> >
>> > 2011/7/4 Anne Ghisla <a.ghisla at gmail.com>:
>> >>> I'm just looking at grass7/raster/r.colors/ and the test files
>> >>> really dominate the module dir.
>> >>
>> >> +1 from me on having a testsuite/ or tests/ subdir for each module.
>> >
>> > also +1 from me.
>>
>> The directory layout has been discussed here:
>> http://osgeo-org.1803224.n2.nabble.com/Test-suite-for-GRASS-proposal-discussion-welcome-td6391218.html
>>
>> I would suggest to either use reference_data/ subdir for each module,
>> so that the test scripts may stay in the module directory, not in a
>> subdirectory, or a testsuite/ subdir including all data?
>>
>> I personally prefer a reference data subdir, to have a clear
>> separation of test case scripts and reference data. Otherwise the
>> reference data will dominate again and finding the test scripts is not
>> that obvious.
>
> In the case of r.colors, there are a lot of .ref files and one .sh
> script, that is hidden among them when the folders contents are ordered
> alphabetically. I haven't checked if it will be the case of other
> modules.

This is the case for most modules with tests applied:
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.out.ascii
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.cross.rast
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.retile
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.stats
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.to.rast
http://trac.osgeo.org/grass/browser/grass/trunk/raster/r.to.rast3elev

>
> Personally I prefer to keep all test data and scripts together in the
> tests/ subfolder.
> But if the majority find it uncomfortable as the script file is less
> accessible, let's keep it in module folder and put data in the subfolder
> (after correcting the paths in script file).
> Opinions?

The paths used in the test scripts should not be affected by the
location of the reference files. Only the framework and the test
designer need to know that. The user/dev who implements the tests must
assure that the reference files are located in the reference
directory. The framework than take care that while testing generated
files are compared with the reference files located in the reference
directory.

Hmmmm, ... i think the test scripts should always write any ascii file
output in the current working directory, so the framework can execute
the tests in a temporary directory which can be removed for clean up
after reference data validation. I will add this to the wiki.

Lets keep for practical reason the test scrip in the module directory
and add a "test_reference_data" or "test_refs" directory to the module
directory keeping the reference data.

Best regards
Sören