Re: [avr-libc-dev] testsuite

[Theodore A. Roth]

On Mon, 9 Sep 2002, Joerg Wunsch wrote:

:) As Theodore A. Roth wrote:
:)
:) > :) For the first version, something that just /compiles/ is certainly a
:) > :) step forward.
:)
:) > That makes some sense. Just to clarify, you're talking about having a test
:) > app file which comiles and links to the library object files to create a
:) > ready to run rom image, right?
:)
:) Perhaps not even a ``ready to run'' image, i. e. it's not necessary
:) that the created image is doing something useful.  The idea that comes
:) to mind is FreeBSDs kernel config file named ``LINT'': it's a configuration
:) that tries to pull every possible feature into a single Unix kernel, just
:) to see whether everything will still compile.  Developers doing a larger
:) redesign of some kernel-internal interface are then required to first
:) successfully compile that `LINT' kernel before they're allowed to CVS
:) commit their changes.

That sounds doable.

:)
:) Likewise, i could imagine that we create test suites that try to
:) access every possible IO subsystem of the respective device, and
:) while it's certainly impossible to create an image that tries to
:) link each and every libc function (in particular for the smaller
:) devices), at least the entire test suite should call every documented
:) function (split across the tests for more than one device) so if
:) someone changes the calling sequence etc., breakage will be noticed.
:) Also, for the larger devices, an attempt should be made to really
:) create a large ROM image (e. g. using some larger tables etc.) so
:) e. g. breakage resulting from using a "rcall" where a library
:) function should use "XCALL" (thus causing unreachable library
:) functions on large devices) might be detected.  Just picking this
:) example since it occurred to me a couple of days before, when working
:) on the dtostre() function.

Agreed. The whole point of a test suite is to try to exercise every bit of
code in a reproducible manner which relieves the developer from having to
having to manually exercise that code which is prone to forgetfulness. ;-)

Of course, the really hard part is making sure that the test suite does
indeed exercise every bit of code.

Ted Roth