Re: checking against signed integer overflow

[Bruno Haible]

Hi Paul,

According to the Gnulib documentation, our recommended way to check
against signed integer overflow is to use the INT_*_WRAPV or INT_*_OVERFLOW
macros.

However, in the discussion about idx_t you mentioned that the benefit of
using signed integer types is that one can have the overflow checks be done
at runtime, through compiler support. (At least that's what I understood.)

Now, let me ask the question "What compiler options would a distro have to
provide, in order to globally enable signed integer overflow checks?"
(like -D_FORTIFY_SOURCE=2, which many distro vendors now use).

Through some experiments (below), I conclude that the appropriate options
would be '-fsanitize=signed-integer-overflow 
-fsanitize-undefined-trap-on-error'.

Reasons:
  * As a distro vendor, you wouldn't be satisfied with a sanitizer that
    emits a message and continues the program nevertheless. As a defense
    against an attacker who wants exploit a vulnerability, just emitting
    a message has zero value.
  * A sanitizer that emits a message and stops the program would be the
    appropriate answer, but with GCC it would mean that all programs are
    linked against libstdc++ (including the coreutils) and thus have large
    startup times, or the distro is built with clang instead of GCC.
  * The use of the '-ftrapv' option would be an appropriate answer as
    well, but with GCC it would mean that all code is compiled with -O0,
    or the distro is built with clang instead of GCC.
  * The only appropriate answer that is left is
    '-fsanitize=signed-integer-overflow -fsanitize-undefined-trap-on-error'.

Could we document this in the Gnulib documentation?

Would it make sense to tell the GCC people that
  - the '-fsanitize=signed-integer-overflow 
-fno-sanitize-recover=signed-integer-overflow'
    options are practically useless when they force a dependency towards 
libstdc++,
  - the 'ftrapv' option is useless when it does not work in combination with
    '-O2' ?

Bruno

================================ foo.c ===========================
#include <stdio.h>

unsigned int ua = 0xA0000000;
unsigned int ub = 0x70000000;
unsigned int uc;
int a = 0x50000000;
int b = 0x60000000;
int c;

int main ()
{
  uc = ua + ub;
  c = a + b;
  c -= a;
  printf ("Still there.\n");
  return 0;
}
==================================================================

Results of various GCC and clang options:

1)
   gcc -fsanitize=signed-integer-overflow   (GCC >= 4.9) (clang >= 3.9.1 at 
least)
   or, more generally
   gcc -fsanitize=undefined                 (GCC >= 4.9) (clang >= 3.9.1 at 
least)

To get messages and continue:
$ gcc -Wall -fsanitize=signed-integer-overflow foo.c
  [The program is linked against libubsan, which depends on libstdc++.]
$ clang -Wall -fsanitize=signed-integer-overflow foo.c

To get messages and terminate the program:
$ gcc -Wall -fsanitize=signed-integer-overflow 
-fno-sanitize-recover=signed-integer-overflow foo.c
  [The program is linked against libubsan, which depends on libstdc++.]
$ clang -Wall -fsanitize=signed-integer-overflow 
-fno-sanitize-recover=signed-integer-overflow foo.c

To get a trap (for debugging):
$ gcc -g -Wall -fsanitize=signed-integer-overflow 
-fsanitize-undefined-trap-on-error foo.c
$ clang -g -Wall -fsanitize=signed-integer-overflow 
-fsanitize-trap=signed-integer-overflow foo.c

   https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html
   https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

2)
   gcc -ftrapv                              (GCC >= 4.2 at least) (clang >= 
3.9.1 at least)

To get an abort (for debugging):
$ gcc -g -Wall -ftrapv -O0 foo.c
To get a trap (for debugging):
$ clang -g -Wall -ftrapv foo.c