[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Chicken-users] Unit testing
From: |
Kon Lovett |
Subject: |
Re: [Chicken-users] Unit testing |
Date: |
Sat, 18 Nov 2006 21:24:09 -0800 |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Nov 18, 2006, at 8:27 PM, Alejandro Forero Cuervo wrote:
<snip>
However, you want to separate this as two sets of tests. If the
"cons" crashed (in the real world this "initialization" code will
probably be far more sofisticated than just a cons), we wouldn't want
the whole set of tests to fail, just those that depend on it. So the
above should probably be rewritten as:
(test-group
(define x (cons 1 2))
(test-assert (pair? x))
(test-equal? (car x) 1)
(test-equal? (cdr x) 2))
(test-group
(define y (list 1 2 3))
(test-assert (pair? y))
(test-equal? (length y) 3))
That means that the only top-level forms should be macros from the
test API (and perhaps the (use ...); if that fails it can be assumed
that all tests will fail).
I realize I didn't properly explain what I mean by this in my previous
mail. I'm sorry.
Gotcha. Setup & Teardown. "test-infrastructure" does this in a
similar manner, lexically scoped variables + a "side-effect" &
"destructor" form. "check" has explicit mechanisms & "testeez" is
lexically scoped; neither is "safe". (Current release of test-
infrastructure is not completely protected from exceptions in setup &
teardown but will be.)
I actually run my tests compiled by default as it is a more
stringent environment.
Well, running them both compiled and then interpreted has got to be
more stringent. ;-)
Umm, in practice I've found that compiled is enough when I am waiting
for the results. For an automated system no, not enough. (In my
environment an egg build always tries to perform any "X/*-test.scm"
found after a successful build of "X".)
Changing manner in which test results are captured to a "write"
rather than a "cons" would be the big change.
Exactly, this is probably how I'd implement the test software. :-)
I'd define a protocol that allows a test to be executed by a certain
process and its result to be passed back to the parent process. The
individual test forms would just fork, run the test and communicate
back the results (or, if they take too long, the parent would just
kill their process and record failure). test-group in my examples
above would also fork and use the same protocol (though this time it
would be able to report the results of multiple tests back to the
parent). And then you've got the parent that compiles the files
defining tests and also forks a separate process that sets its UID to
a "nobody" user and executes them.
Yeah, an automated system might get away with have this overhead -
each "assertion" executed in its' own OS process. I still think that
a package/group/case/whatever assertion (called "expectations" in
"test-infrastructure") collection is a fine enough grain for an OS
process.
- ---
I sense you a looking for a something that allows disparate
individuals to add arbitrary tests (in the "TestUnit" sense, please
ask if that isn't clear) to any Chicken Extension. Great idea. The
devil is in the details.
I am not advocating "test-infrastructure" as the basis. It just
already does a lot of what people want (even Felix uses it sometimes
due to the HTML report - which I do not like but at least it exists).
Also, do not forget that a lot Scheme code uses "testeez" &
"SchemeUnit" already. (The 2 SRFI testing frameworks I don't think
have much penetration. And "test-infrastructure" has none - except
for me!) I wonder if some sort of "test result" coercion might allow
all these different unit test frameworks to somehow slot in to a
larger scale acceptance test framework.
I think the community can come up w/ something that meets the purpose
of standalone testing (see the "test-it" form in "test-
infrastructure"), per programmer unit testing, & acceptance testing
(your concerns). (Integration test should fold into this but is
application oriented.)
- ---
History:
Well over a year ago I had a back & forth conversation w/ Felix about
Scheme unit testing. At the time the "test-infrastructure" seemed the
most elaborate & "Scheme-ish" of the frameworks. So I began extending
& bullet-proofing it. I never liked the form in which testing
concepts are expressed by it but it does express a lot of testing
concepts, and a lot more now.
The original generated a "s***-load" of code, the current version
keeps the original's signatures but w/ much lower overhead.
Alejo.
http://azul.freaks-unidos.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iEYEARECAAYFAkVf6nkACgkQJJNoeGe+5O6UnwCaAgrMatPCyH/OlJCmwv6lD/bz
Y1sAnRBuPGNuSQV2+bEJ5sF/7p7B+Ryy
=TqWQ
-----END PGP SIGNATURE-----