[GRASS5] GRASS test suite update

[Sören Gebbert]

Hi,

On Monday 01 May 2006 13:23, Hamish wrote:
> Hi,
> 
> > i have slightly modified the html output of the test suite:
> > * valgrind memory check information added to the status column
> > * new known error messages added
> > 
> > I have added 4 additonal tests to the user defined location, to test
> > the error handling of the test suite (flag -u). I have generated two
> > html outputs with valgrind memory checks and error handling tests:
> > 
> > AMD machine:
> > http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html/AMD64-2Ghz/Summary.html
> > 
> > P4 machine:
> > http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html/P4-2.8Ghz/Summary.html
> > 
> > The test suite with the modification is available here:
> > http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/GRASS_Testsuite-0.1.6.0.tar.gz
> > 
> > Have fun,
> > best regards
> > Soeren
> > 
> > btw:
> > there are lots of memory errors detected in some modules!
> 
> many modules don't bother free small bits of memory which will be freed
> anyway when the module exits. memory leaks in the libraries or in loops
> are probably bad though.

I'm counting only erros, but no leaks, so a valgrind message like:

==2006-04-23 02:07:03.483 32348== LEAK SUMMARY:
==2006-04-23 02:07:03.483 32348==    definitely lost: 36 bytes in 1 blocks.
==2006-04-23 02:07:03.483 32348==      possibly lost: 0 bytes in 0 blocks.
==2006-04-23 02:07:03.483 32348==    still reachable: 224 bytes in 17 blocks.
==2006-04-23 02:07:03.483 32348==         suppressed: 0 bytes in 0 blocks.
==2006-04-23 02:07:03.483 32348== Reachable blocks (those to which a pointer was found) are not shown.
==2006-04-23 02:07:03.483 32348== To see them, rerun with: --show-reachable=yes

will not be listed as an memory error, only entries like:

 ERROR SUMMARY: 95 errors from 1 contexts (suppressed: 36 from 4)

with ERROR SUMMARY > 0 will be counted as an memory error.

> 
> Is there a way to link "memory-errors-detected" to the valgrind output
> for a quick browse? Maybe figure out how to sort bad memory errors from
> just purposefully orphaned memory and promote bad errors to FAIL
> section?

There are already links (in the htmlfile) to the valgrind logfiles, which lists all the memory errors and leaks.
This link is called "memlog->" and is located on the left side in the htmlfile for every tested module. 

> 
> How about a count of and link to a module's compiler warnings? (redirect
> stderr to a file during make)

Hmmm, this can only be made while the build process of grass, which is 
not linked to the test suite. If compiler warnings occure, they should be written into
a logfile and copied to the grass install directory. In this case the test suite will be 
able to process this file (like the BUILD file in the etc directory of GRASS) 

> 
> How about a link to the actual "validation-failure" or "module-failure"
> error?

I hope i understood this in the right way.
The link on the left side of the htmlfile called "log->" points directly to the error in the logfile (there is a anker defined).

Links to valgrind log-files and the test suite log-file are made for every module, 
even if no error occured. The links are based on test scripts PID's for every test and module.

Example, a logfile and memory error link for the first "r.to.rast3 error detection test (unit)" test:
http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html/AMD64-2Ghz/Logfile.html#32396.0
http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html/AMD64-2Ghz//r.mapcalc-MemCheckLog32396.0.txt

Best regards
Soeren

> 
> 
> 
> thanks,
> Hamish
> 
>