bug#13771: make functions cannot be used in RHS of "TESTS ="

[Stefano Lattarini]

Reference:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13771#8>

Resurrecting an old thread to correct some wrong information
(given by myself, sorry about that).

On 03/05/2013 04:20 PM, Stefano Lattarini wrote:
>
> Hi Daiki, sorry for the delay.
> 
> On 02/20/2013 10:27 AM, Daiki Ueno wrote:
>> Hi,
>>
>> I noticed that some modules are using make functions to assign values to
>> TESTS.  For example, I found the following in my GNOME jhbuild checkout:
>>
>> TESTS = $(filter-out fake-tp-backend individual-zeitgeist,$(noinst_PROGRAMS))
>> TESTS = $(sort $(tests))
>>
>> These cause warnings like:
>>
>> Makefile.am:2: warning: sort $(tests: non-POSIX variable name
>> Makefile.am:2: (probably a GNU make extension)
>>
>> and also generate the following rule in Makefile.in, which seems
>> apparently wrong:
>>
>> $(sort.log: $(sort
>>         @p='$(sort'; \
>>         b='$(sort'; \
>>         $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \
>>         --log-file $$b.log --trs-file $$b.trs \
>>         $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) 
>> $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \
>>         "$$tst" $(AM_TESTS_FD_REDIRECT)
>>
>> Shall it be fixed in Makefile.am or Automake?
>>
> This is actually a long-standing (and probably not easily lifted) Automake
> limitation, due to the need to perform $(EXEEXT) appending to entries in
> the $(PROGRAMS) primary (and consequently in the $(TESTS) variable too,
> since that can contain programs).
>
Actually, this automake-time processing is required also to be able
to correctly handle extension-less tests; so ...

>  I won't object to a patch that tries to
> mitigate the issue, but given how old this limitation is, I won't attempt
> it myself anytime soon.
> 
> In the meantime, I see two solutions:
> 
>    * Don't use make functions in the TESTS definition; this might not be
>      trivial in general situations, but seems pretty easy in your setup
>      above -- instead of having:
> 
>          noinst_PROGRAMS = foo bar qux zap
>          TESTS = $(filter-out foo bar,$(noinst_PROGRAMS))
> 
>      use:
> 
>          test_progs = foo bar
>          noinst_PROGRAMS = $(test_progs) qux zap
>          TESTS = $(test_progs)
> 
>     And no need to $(sort) the TESTS -- since you are using the parallel
>     driver, they should be able to run in any order anyway.
> 
>   * If you only care for the package to build for UNIX (and not for Cygwin
>     or MinGW), you can do away with the $(EXEEXT) rewrite machinery by
>     using the 'no-exeext' Automake option.
> 
... this proposed work-around won't work :-(

Note that this has come up again recently in automake bug#14561.

There we gave a a possible and quite generic workaround: wrap any
GNU make specific stuff that you want to use in $(TESTS) into a
configure substitution, which will be left alone by automake; e.g.:

  # In configure.ac:
  AC_SUBST([auto_find_tests], ['$(sort $(wildcard *.test))'])

  # In Makefile.am:
  TESTS = @auto_find_tests@

But alas, in order to have this to work correctly, one should only
use tests whose suffix is listed in TESTS_EXTENSIONS (if that
variable is not explicitly defined, it's default value is ".test").
That's because, missing such extension, automake cannot generate
generic suffix rules to create the '.trs' and '.log' files the
testsuite harness relies upon.

So even this workaround wouldn't work in the particular use case
reported by Daiki :-(  Sorry.

Best regards,
  Stefano