Re: [GNU Autoconf 2.71] testsuite: 254 255 ... catastrophic failure

[Zack Weinberg]

On Wed, Nov 29, 2023, at 9:32 AM, Dennis Clarke via Bug reports for autoconf 
wrote:
...
> ## ------------- ##
> ## Test results. ##
> ## ------------- ##
>
> ERROR: 555 tests were run,
> 129 failed (4 expected failures).
> 44 tests were skipped.

Yes, this is an abnormal number of failures.

> Every single failure is at some line number inside some file that ends
> with the chars ".at"

This, however, is expected.  A failure report like

254: base.at:1069  configure with closed standard fds

means that test #254, which is named "configure with closed standard fds",
has failed, and you can find the source code for the test in
${srcdir}/tests/base.at, beginning at line 1069.  (The specific operation
that failed will be some ways below, probably.)

Autoconf's test suite uses the "Autotest" test harness. Like configure.ac
scripts, Autotest tests are shell scripts with M4 macros layered on top.
The macros are documented here:

https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/html_node/Using-Autotest.html

Consult local.at for the definition of any macros you see that aren't
documented in this manual.

> This is a small Devuan Linux x86_64 machine :

That isn't exotic enough, by Autoconf's standards, that this number of
failures would be expected.

...
> I can reboot and use the kernel from
> the Devuan/Debian folks but I do not think the kernel has anything to
> do with this mess.

No, if this were a kernel-level problem I would expect there to be either
*many more* failures, or just one or two.

> --- ./at_config_vars-state-env-expected 2023-11-29 09:14:04.189405540 -0500
> +++ ./at_config_vars-state-env.after    2023-11-29 09:14:04.189405540 -0500
> @@ -47,7 +47,7 @@
>   PWD=/root/autoconf-2.71/tests/testsuite.dir/254
>   SHELL=/bin/bash
>   SHELLOPTS=braceexpand:hashall:interactive-comments:posix
> -SHLVL=2
> +SHLVL=3
>   TAR_OPTIONS='--owner=0 --group=0 --numeric-owner'
>   TERM=xterm-256color
>   UID=0

Changes in the value of SHLVL are supposed to be ignored.  It's a little hard
to tell from your logfile but I think this is the proximate cause of all your
failures.  To confirm that, I'd appreciate it if you could send us a tarball of
${builddir}/tests/testsuite.dir, where ${builddir} is the directory containing
the *build output* from your attempt to build Autoconf (this may or may not be
the same as the directory containing Autoconf's source code).  I might not have
the pathname quite right; the directory I want is the one that contains a bunch 
of
numbered subdirectories, one for each failing test.

It would also be really helpful if you could send us the output of this little 
script:

for cmd in bash sed diff cmp grep find cut
do printf '\n: %s\n' "$cmd"
$cmd --version 2>&1 | sed -e '/PCRE/p; /Free Software Foundation/,$d'
done

zw