Re: stack-trace: Use libasan as an alternative to libbacktrace

[Bruno Haible]

Two days ago, I did:
> 2024-07-17  Bruno Haible  <bruno@clisp.org>
> 
>       stack-trace: Use libasan as an alternative to libbacktrace.
>       * m4/stack-trace.m4 (gl_STACK_TRACE_EARLY): As a second choice, use
>       libasan.

It turns out that this does not work well:

1) When '-lasan' is added to LIBS, many tests fail, e.g.

==103167==ASan runtime does not come first in initial library list; you should 
either link runtime to your application or manually preload it with LD_PRELOAD.
FAIL test-acos (exit status: 1)

So, '-lasan' needs to be added to CFLAGS and CXXFLAGS, not LIBS, so that it
occurs _before_ other libraries on the link command line. Recall that the
Automake rule for linking a program is essentially
  $(CC) $(AM_CFLAGS) $(CFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@
and both $(AM_LDFLAGS) and $(LDFLAGS) can contain other -l options.

2) With '-lasan' added to CFLAGS, still 29 tests fail. That's because
apparently the address sanitizer gets activated, with default options.

So I set
  $ export ASAN_OPTIONS="detect_leaks=0 abort_on_error=1 
allocator_may_return_null=1"

3) Still, 5 tests fail:

FAIL: test-dprintf-posix2.sh
FAIL: test-fprintf-posix3.sh
FAIL: test-free
FAIL: test-sigsegv-catch-stackoverflow1
FAIL: test-sigsegv-catch-stackoverflow2

These are "normal" failures from the address sanitizer. But still, it's
undesired, since the user has not asked for a sanitizer in the first place.

That's too much complexity (too many unwanted side effects), when the
more reliable workaround is to just install libbacktrace. I'm therefore
reverting the change.


2024-07-20  Bruno Haible  <bruno@clisp.org>

        stack-trace: Don't use libasan by default.
        * m4/stack-trace.m4 (gl_STACK_TRACE_EARLY): Remove libasan as second
        choice.

diff --git a/m4/stack-trace.m4 b/m4/stack-trace.m4
index faf86eefe9..43f6ef2abf 100644
--- a/m4/stack-trace.m4
+++ b/m4/stack-trace.m4
@@ -1,5 +1,5 @@
 # stack-trace.m4
-# serial 2
+# serial 3
 dnl Copyright (C) 2024 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -22,7 +22,6 @@ AC_DEFUN([gl_STACK_TRACE_EARLY]
   AC_MSG_RESULT([$enable_debug])
 
   AC_REQUIRE([gl_STDLIB_H_DEFAULTS])
-  AC_REQUIRE([AC_CANONICAL_HOST])
   if test $enable_debug = yes; then
     dnl The first choice is libbacktrace by Ian Lance Taylor.
     dnl Maintained at https://github.com/ianlancetaylor/libbacktrace,
@@ -49,71 +48,25 @@ AC_DEFUN([gl_STACK_TRACE_EARLY]
       CAN_PRINT_STACK_TRACE=1
       LIBS="$LIBS -lbacktrace"
     else
-      dnl The second choice is GCC's libasan, installed as part of GCC.
-      dnl It produces source file names and line numbers, if the binary
-      dnl is compiled with debug information.
-      AC_CACHE_CHECK([for libasan], [gl_cv_lib_asan], [
-        gl_saved_LIBS="$LIBS"
-        LIBS="$gl_saved_LIBS -lasan"
-        dnl We need AC_RUN_IFELSE here, not merely AC_LINK_IFELSE, because on
-        dnl NetBSD libasan exists but every program that links to it 
immediately
-        dnl prints "This sanitizer is not compatible with enabled ASLR" to
-        dnl standard error and exits with code 0, without even invoking main().
-        AC_RUN_IFELSE(
-          [AC_LANG_PROGRAM(
-             [[#include <stdio.h>
-               extern
-               #ifdef __cplusplus
-               "C"
-               #endif
-               void __sanitizer_print_stack_trace (void);
-             ]],
-             [[remove ("conftest.c");
-               __sanitizer_print_stack_trace ();
-             ]])],
-          [if test -f conftest.c; then
-             dnl main() was not reached. NetBSD!
-             gl_cv_lib_asan=no
-           else
-             gl_cv_lib_asan=yes
-           fi
-          ],
-          [gl_cv_lib_asan=no],
-          [case "$host_os" in
-             netbsd*) gl_cv_lib_asan="guessing no" ;;
-             *)       gl_cv_lib_asan="guessing yes" ;;
-           esac
-          ])
-        LIBS="$gl_saved_LIBS"
-      ])
-      case "$gl_cv_lib_asan" in
-        *yes)
-          AC_DEFINE([HAVE_LIBASAN], [1],
-            [Define if you have the libasan library.])
+      dnl The second choice is libexecinfo.
+      dnl It does not produce source file names and line numbers, only 
addresses
+      dnl (which are mostly useless due to ASLR) and _sometimes_ function 
names.
+      AC_REQUIRE([AC_CANONICAL_HOST])
+      case "$host_os" in
+        *-gnu* | gnu* | darwin* | freebsd* | dragonfly* | netbsd* | openbsd* | 
solaris*)
+          dnl execinfo might be implemented on this platform.
           CAN_PRINT_STACK_TRACE=1
-          LIBS="$LIBS -lasan"
-          ;;
-        *)
-          dnl The third choice is libexecinfo.
-          dnl It does not produce source file names and line numbers, only 
addresses
-          dnl (which are mostly useless due to ASLR) and _sometimes_ function 
names.
+          dnl On *BSD system, link all programs with -lexecinfo. Cf. 
m4/execinfo.m4.
+          case "$host_os" in
+            freebsd* | dragonfly* | netbsd* | openbsd*)
+              LIBS="$LIBS -lexecinfo"
+              ;;
+          esac
+          dnl Link all programs in such a way that the stack trace includes the
+          dnl function names. '-rdynamic' is equivalent to 
'-Wl,-export-dynamic'.
           case "$host_os" in
-            *-gnu* | gnu* | darwin* | freebsd* | dragonfly* | netbsd* | 
openbsd* | solaris*)
-              dnl execinfo might be implemented on this platform.
-              CAN_PRINT_STACK_TRACE=1
-              dnl On *BSD system, link all programs with -lexecinfo. Cf. 
m4/execinfo.m4.
-              case "$host_os" in
-                freebsd* | dragonfly* | netbsd* | openbsd*)
-                  LIBS="$LIBS -lexecinfo"
-                  ;;
-              esac
-              dnl Link all programs in such a way that the stack trace 
includes the
-              dnl function names. '-rdynamic' is equivalent to 
'-Wl,-export-dynamic'.
-              case "$host_os" in
-                *-gnu* | gnu* | openbsd*)
-                  LDFLAGS="$LDFLAGS -rdynamic"
-                  ;;
-              esac
+            *-gnu* | gnu* | openbsd*)
+              LDFLAGS="$LDFLAGS -rdynamic"
               ;;
           esac
           ;;