[XForms] automated regression tests for XForms

[alessandro basili]

Hi everyone,

some times ago I started to wonder how can a GUI toolkit like XForms
integrate a framework for testing purposes which will allow developers
to apply changes and regularly perform regression tests minimizing the
chance the new feature or bug-fix will show an 'unwanted' behavior on
some other part of the code.

When I normally try to use or fix part of the code I've found myself
building test cases, with simple interfaces and some amount of
interaction (click here, scroll there...). What if then for every object
we cannot build a very simple test case and 'memorize' the sequence of
events in order to reuse it?

I've searched around and found Xnee (http://xnee.wordpress.com/) which
is exactly this, an event recorder/replay for X11 based systems. I have
no experience on how it works but it seems a pretty nice software.

What we can conceive is to go through the list of objects and build test
programs, which will then be used by a regular user who may have his/her
sequence of actions recorded. The test report (as a form of events) can
then be replayed regularly for every build, guaranteeing that the new
fix did not break that part.

As users submit test cases as well as test reports, they can be reused
for next build, without the user needing to get the new version, link
against it and then test his/her application with some random clicks to
see if 'everything' is still fine.

Possibly the test cases should be structured in a framework such that
low level functionality is tested first, while higher level code can be
tested later. And I also believe that good test programs can be also
used as template for newcomers, fostering good programming style with
the package.

By no means this effort should relax the amount of testing, but I guess
it will help developers/maintainers with a more systematic feedback on
their builds, increasing the quality and the amount of work people spend
on repetitive testing.

I wish to hear comments from you and hopefully start a discussion on a
first trial.

Cheers,

Al

-- 
PGP Fingerprint DCBE 430E E93D 808B 45E6 7B8A F68D A276 565C 4450
PGP Public Key available at keys.gnugp.net