[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Testing approach (was Re: [Gnu-arch-users] more on the merge-fest)
From: |
Misha Dorman |
Subject: |
Testing approach (was Re: [Gnu-arch-users] more on the merge-fest) |
Date: |
Wed, 26 Nov 2003 10:27:16 +0000 |
zander wrote: [some words suggesting the use of XP-style test-first methods,
including...]
The point being that its easier to define correct behavior from outside
then
it is by writing the actual code that causes that behavior.
Mark Thomas wrote:
The "best" way is for two different people to write the code and the test
...
In the absence of this level of redundancy, writing the code and the test
suite in either order will be "good enough for government work."
Samuel A. Falvo II wrote:
I disagree with this, to an extent. I think the customer of the code is
responsible for writing what are called 'acceptance tests,' but such an
author is not going to know how specific data structures work inside the
program. The application coder must implement a finer-grain of tests
than the acceptance tests.
Mark Thomas wrote:
[test-then-code] encourages "coding to the test suite" where you just write
code so that it passes
Samuel A. Falvo II wrote:
No. The code will work perfectly, as required by the test suite. A
case not considered by the unit tests is, by definition, not part of
the specification of the software.
The most important thing is that the code and the test be derived
independently from a specification (whether written down explicitly or not).
All these methods (code then test, test then code, code & test
independently) attempt to achieve this.
Obviously having a separate coder and tester (typically, both a developers,
and they test each others work; we are not talking about dedicated test
teams here, as Samuel seemed to be) provides the greatest independence,
though it probably isn't particularly suitable to open source. Open source
development is predicated on user/devs "scratching an itch" and how many
peoples itch is "write some test cases for someone elses itch"? (And if it
isn't scratching an itch, it won't be done as well, because the motivation
isn't there.)
Code-then-test runs the risk that the test is either omitted (as Mark said)
or simply written to match the code (set up the inputs, run the code, see
what it produces, after a brief sanity check say "thats the expected" and
set up the test expected values to match, rather than deriving the expected
values from first principles). Obviously an individual developer (Tom, for
example) may have the self-discipline to "do it properly" but in general the
temptation to be lazy is strong (especially in commercial contexts, where
time pressure is stronger, I suspect).
Test-then-code is an attempt to avoid this problem/temptation, as well as a
recognition that (as zander said) in many cases (though not all) writing a
test is a good way to codify the required functionality. Of course, the risk
is then (as Mark noted) that the code is written from the test, which (and
this is the point that Samuel may have missed) can only be a _partial_
codification of the specification (how many test cases do you need to
_exhaustively_ test even something as simple as A + B?). Test-first
advocates would say that this risk is lower than the risk of omitted or
"cheated" tests in code-first approaches.
Note that in theory the end results of code-then-test, test-then-code and
independent-code-and-test could be identical code and identical tests. In
practice, the nature of the approach influences how both code and test are
written and structured, just as individual programmers have their own style
(c.f. "piss on it to make it smell like Tom")
To echo Tom's comments about "art": there is no "best" approach -- it
depends on the development team, process, and discipline (both internal and
external, natural and imposed).
I would suggest that people at least _try_ test-then-code, to see if it
helps them write good tests.
Then you can make an informed decision, based on the project, the team, the
quality requirements etc.
--
Misha Dorman
_________________________________________________________________
Express yourself with cool emoticons - download MSN Messenger today!
http://www.msn.co.uk/messenger
- Testing approach (was Re: [Gnu-arch-users] more on the merge-fest),
Misha Dorman <=