[Koha-devel] interest in improving automated tests with a testing database

[Andrew Moore]

Hi Koha developers -

I have caught some of you on IRC and such, but for the rest of you,
I'm a new developer at LibLime working on Koha.

I'm interested in improving the automated test suite a little bit, and
I'm hoping to get some feedback from any interested developers and
users. There are some database specific tests in the t/db_dependent
directory, but I think they can be improved quite a bit. I'll lay out
what I'm picturing, and please let me know if you picture a different
plan, or have any ways to improve this one.

I'd like to have a 'make' target similar to 'make test' that would
build a nearly empty, skeleton database that we could run some
automated tests against. Since this would not be the same database
that live data is held in, we wouldn't have to worry about deleting or
overwriting any useful data. The testing could insert example data and
data known to trigger past bugs to ensure that they don't recur. I
don't imagine making this a standard part of 'make test', at least at
first, since I don't imagine that all end users want to have a second
database built just to test against. I think it could be immensely
useful for us developers, though, right away.

One useful way I've seen this done in the past is trough the use of
the Test::Class perl module. It's similar to the xUnit framework (or
jUnit or nUnit) in that it lets you run a bit of code before and after
each test to set up the environment (the test fixture). That way, you
can ensure that the database looks clean between when different pieces
of the test suite runs so that one test script doesn't interfere with
another. It also lets your test scripts inherit from each other so
that similar test scripts can share the same code to set up their
fixtures.

I have a small working prototype of this, but before I get much
further, I want to ask if you have any suggestions or advice for this.
Please let me know if this sounds like something that would be useful
to you, and if there's anything I need to make sure I consider.

Thanks,
-Andy