|
| From: | Éloi Rivard |
| Subject: | Re: [Denemo-devel] Master is badly broken |
| Date: | Mon, 16 Jun 2014 15:14:06 +0200 |
On Sun, 2014-06-15 at 12:53 +0200, Éloi Rivard wrote:We seem to be so close, ready to write at least one or two tests, but I
> Do you think the testsuite is tough enough to merge the "refactor"
> branch now ?
can't get make -C tests check to work (see below).
Also, you said
is this actually happening?, and if so does it execute the command on
> > I create a new test that will try to execute every not
> builtin scheme
> > command
blank file (very weak) rather than a file with music in it - that would
be a small change that would make the test far stronger...
I started trying to create a test for each new command, but I don't have
a method for creating tests that work.
To run the tests I have a separate user with user name "denemo-user". I
don't see any log file at the end of the tests or details of what
failed, I'll paste here what I see - it ends with 1 of 2 tests fail.
This is the result of:
su denemo-user
git pull
make clean
make distclean
./autogen.sh
make && make -C tests check
[...]
> Denemo - MESSAGE : Opening file /home/denemo-user/denemo-git/tests/tmp/scm/SanityCheck.denemo
> OK
> PASS: integration
> /unit/run-and-quit: Denemo - MESSAGE : Loading preference file: /home/denemo-user/.denemo-1.1.7/denemorc
> Denemo - MESSAGE : Denemo version 1.1.7
> Denemo - MESSAGE : Loaded keymap /home/denemo-user/denemo-git/actions/Default.commands
> OK
> /unit/scheme-log: Denemo - WARNING : This is warning
> Denemo - CRITICAL: This is critical
> Denemo - MESSAGE : Loading preference file: /home/denemo-user/.denemo-1.1.7/denemorc
> Denemo - MESSAGE : Denemo version 1.1.7
> Denemo - MESSAGE : Loaded keymap /home/denemo-user/denemo-git/actions/Default.commands
> Denemo - INFO : This is info
> Denemo - MESSAGE : This is message
> OK
> /unit/scheme-log-error: Denemo - MESSAGE : Loading preference file: /home/denemo-user/.denemo-1.1.7/denemorc
> Denemo - MESSAGE : Denemo version 1.1.7
> Denemo - MESSAGE : Loaded keymap /home/denemo-user/denemo-git/actions/Default.commands
> Denemo - ERROR : This error is fatal
> OK
> /unit/thumbnailer: Running scheme: (d-CreateThumbnail #f "/home/denemo-user/denemo-git/tests/tmp/thumbnail.png")(d-Exit) /home/denemo-user/denemo-git/tests/fixtures/denemo/blank.denemo
> Denemo - MESSAGE : Loading preference file: /home/denemo-user/.denemo-1.1.7/denemorc
> Denemo - MESSAGE : Denemo version 1.1.7
> Denemo - MESSAGE : Loaded keymap /home/denemo-user/denemo-git/actions/Default.commands
> Success 1
> Denemo - CRITICAL: Thumbnail 128x128 file /tmp/Denemo2g5UiI/denemothumb.png gave an error: Failed to open file '/tmp/Denemo2g5UiI/denemothumb.png': No such file or directory
> Denemo - CRITICAL: Thumbnail 256x256 file /tmp/Denemo2g5UiI/denemothumb.png gave an error: Failed to open file '/tmp/Denemo2g5UiI/denemothumb.png': No such file or directory
> Denemo - MESSAGE : Opening file /home/denemo-user/denemo-git/tests/fixtures/denemo/blank.denemo
> Denemo - INFO : Creating thumbnail /home/denemo-user/denemo-git/tests/tmp/thumbnail.png
> **
> ERROR:unit.c:121:test_thumbnailer: assertion failed: (g_file_test(thumbnail, G_FILE_TEST_EXISTS))
> /bin/bash: line 5: 1553 Aborted G_TEST_SRCDIR="/home/denemo-user/denemo-git/tests" G_TEST_BUILDDIR="/home/denemo-user/denemo-git/tests" G_DEBUG=gc-friendly MALLOC_CHECK_=2 MALLOC_PERTURB_=$((${RANDOM:-256} % 256)) ${dir}$tst
> FAIL: unit
> =====================================
> 1 of 2 tests failed
> Please report to address@hidden
> =====================================
> make[2]: *** [check-TESTS] Error 1
> make[2]: Leaving directory `/home/denemo-user/denemo-git/tests'
> make[1]: *** [check-am] Error 2
> make[1]: Leaving directory `/home/denemo-user/denemo-git/tests'
> make: *** [check] Error 2
> make: Leaving directory `/home/denemo-user/denemo-git/tests'
> address@hidden:~/denemo-git$
>
> >
> >
> > The test parses the action directory to find the scheme
> commands.
> >
> > For each command, it checks if an associated test file
> exists, lets
> > say tests/fixtures/scheme/THECOMMAND.scm. If so it executes
> it.
>
> Is that to say it executes the script starting with a blank
> score? Does
> it then save the score after the script has executed and test
> against
> THECOMMAND.denemo ?
> this would sound like a good framework for testing.
>
>
> > If not it just executes "(d-THECOMMAND)(d-quit)".
> >
> > This would be a weak test in that case,
>
> It could be made quite a bit stronger by making the
> environment in which
> (d-THECOMMAND) is executed a more typical environment, by
> installing a
> couple of staffs and some chords, leaving the cursor on a
> chord. Many
> more commands do useful things when the cursor is on something
> and when
> other staffs are present than do something useful in a
> completely empty
> score.
>
> (d-AddAfter)
> (d-A)
> (d-CursorUp)
> (d-CursorUp)
> (d-AddNoteToChord)
> (d-MoveCursorLeft)
> (d-THECOMMAND)
> (d-Save "filename= ....")
> (d-Quit)
>
> would generate a distinctive output file for many commands (it
> creates
> two staffs, populates one and then executes THECOMMAND in that
> situation).
>
> > but it could at least check that the command does not
> provoke a
> > segfault.
>
> >
> >
> > Then the test could be a bit tougher. For example, we could
> decide
> > that if a scheme command return FALSE, it makes the test
> fail.
>
> I'm not sure that Denemo commands return anything useful. But
> detecting
> scheme exceptions would be good - we could exit in the trap
> handler if
> noninteractive was set. We might have to fix one or two
> commands that
> don't expect to be executed in the given environment - well,
> we could
> just write a test for them.
> >
> > What do you think ?
>
>
> I think this is excellent - it will require a rule to generate
> the set
> of expected output files, rather than diff them (for the
> initial
> creation of the expected output), and one to accept an altered
> set of
> files (copying them to the expected ones) would be a
> time-saver too.
> After a change of version in the Denemo file format all the
> expected
> output files would change - you would make just this change,
> check a few
> examples and then run the rule to overwrite all the old
> versions with
> the new ones.
>
> Richard
>
>
>
> >
> >
> > 2014-04-15 21:10 GMT+02:00 Richard Shann
> <address@hidden>:
> > On Tue, 2014-04-15 at 19:13 +0100, Richard Shann
> wrote:
> > > As you remarked, it will be good to generate a
> new .scm
> > script each
> > > time
> > > a new command is made
> >
> > This script could assume that a variable,
> > Denemo-output-filename say,
> > was defined which it should use via
> >
> > (d-Save Denemo-output-filename)
> > (d-Quit)
> >
> > at the end of the test. (I think I missed the
> (d-Quit) out of
> > the
> > current script ...)
Richard
| [Prev in Thread] | Current Thread | [Next in Thread] |