octave-maintainers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Marking bugs as fixed in the test suite


From: John W. Eaton
Subject: Marking bugs as fixed in the test suite
Date: Fri, 7 Jul 2017 14:27:39 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0

About a month ago, I made the following change so that tests for reported bugs could be tagged as fixed in the test suite by prefixing the bug number with "*".

  http://hg.savannah.gnu.org/hgweb/octave/rev/b6144e6dda9e

With this change, any failing tests that are marked this way are shown in the test suite summary as regressions. Previously they were just noted as "expected failures" or "reported bugs" so it was easy to miss that a change actually reintroduced some bug that had been fixed.

I also added a script and Makefile target ("update-bug-status") that attempts to automatically tag tests with bug numbers as fixed:

  http://hg.savannah.gnu.org/hgweb/octave/rev/0703d31e1ec0

Around the same time, I added bug numbers for as many xtests as I could. There are just a few left that don't have bug numbers. Any help with adding bug bug numbers for those would be much appreciated. If bug reports don't already exist, they should be created. I would have done it, but I don't really understand the purpose of the remaining xtests. Maybe someone else could write better reports and I could.

Finally, there are now 4 tests that are always failing and that are marked as regressions. This is happening because the bug report is closed as fixed, but there are multiple parts to the report:

  http://octave.org/testfailure/?50893

The tests that fail are all items that have been marked as "won't fix". So, what should we do? Open a separate bug report that won't be closed as fixed? Add another marker for the tests to indicate "won't fix"?

jwe




reply via email to

[Prev in Thread] Current Thread [Next in Thread]