[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Monotone-devel] RFC: Unit test improvements
From: |
Zack Weinberg |
Subject: |
[Monotone-devel] RFC: Unit test improvements |
Date: |
Tue, 22 Aug 2006 18:09:55 -0700 |
I've been back from my trip for a few days but I'm still pretty wrung
out and not up to doing much in the way of what you might call
_complicated_, so instead today I had a look at improving the way we
do unit tests.
There's three mostly-orthogonal improvements that I found:
* A facility for automatically registering test functions, inspired by
the handling of commands. Instead of having to hand-write
"add_*_tests" functions you just do
UNIT_TEST(groupname, testname)
{
...
}
Group names substitute for the old grouping by registration function.
Groups are run in alphabetical order; within a group the order isn't
nailed down but seems (with GCC) to go in order of definition in the
file.
* Using boost::program_options (which was already being linked with
the unit_tests binary) I improved the argument handling. You can now
ask for a list of supported tests or test groups, you can specify a
particular test within a group to be run, and you get sensible error
messages for command line arguments that don't correspond to tests or
options. (Note: the test framework library claims to support a few
options but they don't work - I'm not sure whether this is because I
need to tell bpo to ignore them, or because the (nearly unreadable)
test framework documentation is just wrong.)
* And last but not least: Now that there are options, it is sensible
to turn debug-log-to-stderr off and require an option (--debug, just
like in monotone proper) to turn it back on. Thus, the noise level of
make check is *dramatically* reduced. I added fake tests to report
progress, it looks like this:
$ ./unit_tests
Running 96 test cases...
charset...
crypto...
cset...
diff_patch...
globish...
key...
netcmd...
packet...
paths...
pipe...
refiner...
restrictions...
mtn: warning: restriction includes unknown path 'foo'
mtn: warning: restriction excludes unknown path 'bar'
revision...
roster...
mtn: performing random action 0
mtn: performing random action 100
mtn: performing random action 200
mtn: performing random action 300
mtn: performing random action 400
mtn: performing random action 500
mtn: performing random action 600
mtn: performing random action 700
mtn: performing random action 800
mtn: performing random action 900
mtn: performing random action 1,000
mtn: performing random action 1,100
mtn: performing random action 1,200
mtn: performing random action 1,300
mtn: performing random action 1,400
mtn: performing random action 1,500
mtn: performing random action 1,600
mtn: performing random action 1,700
mtn: performing random action 1,800
mtn: performing random action 1,900
roster_merge...
simplestring_xform...
string_queue...
transform...
uri...
xdelta...
*** No errors detected
(I also hacked around the test-framework bug that makes the 'Running N
test cases...' message show up after everything else. Arguably, we
should now eliminate the few messages that are still being printed,
due to use of P() instead of L(), but I can also argue that they
should be kept because the roster tests are so very slow.)
Thoughts?
zw
unit-test-improvements.diff
Description: Text Data
- [Monotone-devel] RFC: Unit test improvements,
Zack Weinberg <=