Re: [Axiom-developer] Re: [fricas-devel] Re: regression tests

[root]

Martin,

The test facility is based on the regress function which is a lisp
function loaded into the axiom interpreter.

Regression test files contain the expected results. The format is

)spool foo.output                               (0)
--S n of m                                      (1)
2+2                                             (2)
--R                                             (3)
--R  (1) 4                                      
--R                 Type: PositiveInteger
--E n                                           (4)

(...repeat blocks as desired...)

)spool                                          (5)


The file consists of blocks of the above form where:
  (0) is the output file which will contain the output of the regression test
      Note that the input regression test file is a standard axiom 
      input file. all of the markup used for regression testing are
      in axiom comment lines starting with --. thus you can run the
      input files without checking the results.
  (1) is a marker that a test is starting. n of m is a comment
  (2) is the expression to evaluate
  (3) there are result lines that are compared char-by-char with 
      the actual result. these are easily created by clipping a
      prior version and prefixing each line with --R
        In addition, some lines contain gensym symbols which 
      change from run to run. they can be prefixed with --I (ignore)
  (4) is a marker ending the test
  (5) at the end we close the file.

To create a regression test file you simply 
  (a) create an input file,
  (b) run it, 
  (c) comment out the actual results by prefixing them with --R.
  (d) delimit each result with --S and --E
  (e) add it to the REGRES stanza in the src/input/Makefile.pamphlet
  (f) store the foo.input.pamphlet file in src/input

The process for running regression testing has 2 steps which are 
run automatically by the Makefile but can also be done by hand.
  (a) )read foo
      which causes foo.output to be created containing the new
      results followed by the commented expected results and
      delimited by the --S and --E markers.
  (b) )lisp (regress "foo.output")
      which checks the new results against the expected results
      and creates "foo.regress" which contains pass/fail markers

Tim