Re: [Denemo-devel] Testsuite, step 3

[Richard Shann]

On Tue, 2013-12-17 at 14:50 +0100, Éloi Rivard wrote:
> Hi,
> 
> I have some good news, I added a test named "coverage" that reads
> Defaults.commands and tests every command it finds in it (1012 tests
> for the moment). For each command the tests checks if
> coverage-data/COMMAND.scm exists, and execute it if it exists. Else it
> simply launch denemo with '-a "(d-COMMAND)(d-Quit)".
> 
> 
> This is still a very weak test, since there is no argument passed to
> the commands, and since that results are not checked. But at least it
> can check for some segfaults. This test double our test coverage (we
> were around 10% now, we are at 20%). The coverage test is very long.
> 
> 
> To keep things clean, a test should be created every time a command is
> created (in tests/coverage-data/COMMAND.scm). Tests should be as
> simple as possible, and provoke failure when the result obtained is
> not the one expected.
> 
> 
> More generally, each implemented feature and each fixed bug should
> have its own test.
> 
> 
> 
> Next step is to also test builtin functions declared in
> scheme-identifiers.c and find a way to run GUI tests, and add some
> unit tests for often used functions.

This is excellent stuff. A couple of points - one is there is a
deliberate scheme error generated in the score-checking command (a bad
idea of course). And, I have had Denemo terminate recently as a result
of scheme errors, which it never did with the old handler. I haven't
been able to figure out the conditions when this happens, but, of
course, it makes it very difficult to find this out.

Richard