new style of tests

[John W. Eaton]

On 27-Apr-2000, Paul Kienzle <address@hidden> wrote:

| I'm including the patch here.  Note that the stuff at the end
| (lines starting %! ) is test code which can be run using ftest("kron").
| Here is the current version of ftest in compact form:
| 
| ## usage: ftest(name)
| ## Execute all lines starting with %! in the function file.  To see what
| ## was executed, enter "type __ftest__"
| function ftest(name)
|   if nargin != 1, usage("ftest('function')"); end
|   file=file_in_path(LOADPATH,name);
|   if isempty(file), file=file_in_path(LOADPATH,[name ".m"]); endif
|   if isempty(file), file=file_in_path(LOADPATH,[name ".cc"]); endif
|   if isempty(file), error(["'" name "' does not exist in path"]); endif
|   test = system(["cat " file " | grep '^%!' | cut -b 3-"]);
|   if isempty(test) disp(["No tests available for " file]);
|   else 
|     test=["function __ftest__;" test "\nendfunction; __ftest__;"];
|     eval(test,"disp(__error_text__)");
|   endif
| endfunction

Unless there is some really good reason for putting this in a
dynamically linked fucntion (I think you posted one earlier), I think
it is better to have it in an M-file.

You should use `file_in_loadpath (file)' instead of
`file_in_path (LOADPATH, file)', since LOADPATH may have magic
leading, trailing, or doubled colons that need to be expanded to the
default loadpath, and file_in_path doesn't know to do that but
file_in_loadpath does.

The cat/grep/cut pipeline could be simplified to just

  body = system (sprintf ("sed -n 's/^%%!//p' %s", file));

and I think I would write

  if (isempty (body))
    printf ("no tests available for %s\n", file);
  else
    eval (sprintf ("function __ftest__ ()\n%s\nendfunction", body));
    try
      __ftest__ ();
    catch
      error (__error_text__);
    end_try_catch
  endif

One problem I see is that once an error occurs, the remaining tests
are skipped.  That is probably OK when you are running tests
interactively, but for automated tests, it would be nice to keep going
with the next test.

Also, for automated tests for all functions, it would be good to have
the test function/script (which should know the name of the function
it is currently testing) print something like

  kron............

before running the tests, then printed `ok' or `fail' on the same line
after the tests are completed (similar to the way the perl tests
report success/failure). 

I think it would also be best to only print `ok' or `fail' while the
tests are running.  Detailed information about any failures could be
saved and printed in a summary table after all the tests have been
run.

jwe