Re: Tests fail due to argument list too long

[Bob Friesenhahn]

On Fri, 31 Jul 2009, Ralf Wildenhues wrote:
> > AUTOMAKE_OPTIONS = 1.11 subdir-objects parallel-tests color-tests
> > dist-bzip2 dist-lzma foreign
>
> Does that error go away if you remove the parallel-tests option?

I don't know.  Little changes like that to make a simple test would 
likely consume six hours time.  The system is remote, rather slow, 
and I use CVS commit/checkout to communicate changes to it.  It is not 
important that this issue be fixed right away, but it has implications 
for the future.

> You should be able to work around it by splitting tests in pieces,
> and running things piecewise only for now; assuming the *_TESTS
> variables are all nonempty:
>
>  TESTS = $(WAND_TESTS) $(UTILITIES_TESTS) ...
>
>  check-wand:
>        $(MAKE) $(AM_MAKEFLAGS) check-TESTS \
>                TESTS='$(WAND_TESTS)' TEST_SUITE_LOG=test-suite-wand.log
>
>  check-utilities:
>        ...
>
> I agree that it's a bit of a kludge (and I haven't tested it yet).

Are you saying that the user would then need to explicitly invoke 
'check-wand' or is it possible to create a 'check' dependency so that 
all of these would be implicitly invoked in turn?  It looks like there 
would be a bunch of mini-test reports and so the user is likely to 
miss most of the reports since they would have scrolled on by.  I only 
expect the user to see whatever residue is left on his terminal when 
the tests have completed.  There is also the problem that a single 
failed test early on would prevent all the rest of the test groups 
from completing and if the error is intentionally ignored, then all 
errors will be ignored.

It is likely that the number of tests will quadruple over time since 
currently the tests are the bare minimum (only 785 tests) for my 
software.  I don't mind explicitly splitting internally if there is 
still a way for the user to do the standard 'make check' and 'make 
distcheck' (with reliable ultimate failure on test fail) and see a 
total report for all of the tests at the end.

Bob
--
Bob Friesenhahn
address@hidden, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/