Re: emacs test suite?

[Robert Anderson]

--- Original Message ---
From: Thien-Thi Nguyen <address@hidden>
To: "Robert Anderson" <address@hidden>
CC: address@hidden
Subject: Re: emacs test suite?

>"Robert Anderson" <address@hidden> writes:
>
>   When I help identify and fix bugs, I like to be able to provide a
>   "push-button" test case that demonstrates the problem, and
>   prevents regression once the problem has been fixed.
>
>this approach is also useful for new code.
>
>   When I run "make check" for GNU emacs, it tells me there are no
>   tests for emacs yet.
>
>   Have there been any initiatives to develop a testing system for
>   emacs?
>
>that's what the ftp site is for, no?  (half-smiley.)
>
>i'm in favor of such a system.  for current-column munging (in
the presence of
>variable-width fonts), i have a local testing setup but it is
not generalized.
>this could be because testing interactive programs typically
requires a driver
>program external to the one under test, and that starts stepping
into debugger
>territory (where programmers' disparate debugging styles is
nonregularizable).
>
>however, having said that, perhaps someone could build a system
on top of
>screen(1), for text mode; and something analogous, for X.
>
>thi

[sorry for the screwed up quoting, my mailer is horrible.]

It depends on what you are trying to test, I suppose.  I think
there is a broad class of "textual transformation" tests that
would be trivial to implement as batch jobs called from a simple
script harness.

In fact, I've made a very simple sketch of how such a thing would
work, which verifies the cc-mode filling bug in CVS HEAD (and
also demonstrates that it works "as expected" for 21.1.1 and 20.7.1.)

There is a driver script which runs GNU find on a subtree looking
for programs which have the name, by convention, "run-test". 
Each command "run-test" accepts an argument "--description" which
is spits out a one-line description to be displayed while the
test is running, as well as an option to specify a full path to
the emacs binary to test.  Each program "run-test" returns 0 for
pass and non-zero for fail.

I suspect from perusing the mailing list that this sort of thing
might be valuable, if people would use it.

Bob