Re: [PATCH 00/17] De-recursion for the 'tests' subdirectory.

[Jim Meyering]

Stefano Lattarini wrote:
...
>> I still haven't reviewed your changes, but in using them I did notice
>> a missing dependency:
>>
>> I ran this command to run the single test:
>>
>>     $ make tests/rm/v-slash.log
>>     PASS: tests/rm/v-slash.log
>>
>> then modified remove.c and recompiled src/rm and went to run the test again:
>>
>>     $ make tests/rm/v-slash.log
>>     make: `tests/rm/v-slash.log' is up to date.
>>
>> Oops.  It didn't run.  The individual test scripts (the .log witness files,
>> actually) do not depend on the programs they test.
>>
> The above usage is not actually supported Automake interface (for one thing,
> it won't honour $(check_PROGRAMS), nor $(BUILT_SOURCES)).  To run a subset
> of tests, you should do this:
>
>     $ make check TESTS="tests/rm/v-slash"
>
> And to run them lazily (i.e., only if one of their prerequisites has
> changed), you should do:
>
>     $ make check TESTS="tests/rm/v-slash" RECHECK_LOGS=
>
> And if you want to re-run a test without overwriting the pre-existing
> test-suite.log, here is what you should do:
>
>     $ make check TESTS="tests/rm/v-slash" TEST_SUITE_LOGS=partial.log
>
> All of this should be explained in the Automake manual, subsection
> 15.2.3, "Parallel Test Harness":
>
>     
> <http://www.gnu.org/software/automake/manual/automake.html#Parallel-Test-Harness>

Thanks for the explanation, but running the commands you suggest generates
over 200 hundreds of lines of output, all but one or two of which are
irrelevant to me.  Running the technically-unsupported command produces
precisely the one or two lines that I wanted to see.  Which do you think
I will be using most of the time? ;-)

>> From 45bf1ab13682f8842b2b9ed81bb9069f6e5b01b3 Mon Sep 17 00:00:00 2001
>> From: Jim Meyering <address@hidden>
>> Date: Tue, 4 Sep 2012 12:12:19 +0200
>> Subject: [PATCH] tests: depend on the programs we're testing
>>
>> * tests/local.mk ($(TEST_LOGS)): Depend on $(PROGRAMS),
>>
> Beware that $(PROGRAMS) is not a documented Automake interface either.

If it changes, I'm sure I'll adapt ;-)
Unless you can propose an alternative or choose to document it.

>> so that tests are rerun when any program is rebuilt.
>>
> Apart from my nits above, I believe this is a good idea, because it will
> cause the documented interface for "lazy re-runs":
>
>     $ make check TESTS="tests/rm/v-slash" RECHECK_LOGS=
>
> to DTRT.
>
> Your idiom won't be completely precise though, since $(TEST_LOGS) can be
> overridden at make runtime by the user (either directly, or as a consequence
> of his overriding of $(TESTS)); but being 100% correct would require you to
> duplicate some of messy logic Automake applies to $(TESTS), to apply it to
> $(all_tests) instead:
>
>     all_test_logs1 = $(all_tests:=.log)
>     all_test_logs2 = $(all_test_logs1:.sh.log=.log)
>     all_test_logs3 = $(all_test_logs2:.pl.log=.log)
>     all_test_logs = $(all_test_logs3:.xpl.log=.log)
>     $(all_test_logs): $(PROGRAMS)
>
> but that's not worth the hassle IMHO.

Good to know, just in case.