[avr-gcc-list] More results from the testsuite with avrtest

[Paulo Marques]

Hi, all

I've been running the testsuite from SVN with avrtest to track down the 
latest bugs. I've finally track down every failure in execute.exp, and 
I'm preparing to move on to the full testsuite (I hope).

So, here's a list of all the problems so far:

FAIL: gcc.c-torture/execute/20010122-1.c execution,  -O0
FAIL: gcc.c-torture/execute/20010122-1.c execution,  -O1
FAIL: gcc.c-torture/execute/20010122-1.c execution,  -O2
FAIL: gcc.c-torture/execute/20010122-1.c execution,  -O3 -g
FAIL: gcc.c-torture/execute/20010122-1.c execution,  -Os

There is something fishy going on with __builtin_return_address. This 
test in particular only fails when it is compiled with "-Wl,-u,vfprintf 
-lprintf_flt". This probably is just an indirect effect of some code 
reordering due to the increased image size, or something. Still needs 
more investigation.


FAIL: gcc.c-torture/execute/built-in-setjmp.c execution,  -O2
FAIL: gcc.c-torture/execute/built-in-setjmp.c execution,  -O3 
-fomit-frame-pointer
FAIL: gcc.c-torture/execute/built-in-setjmp.c execution,  -O3 
-fomit-frame-pointer -funroll-loops
FAIL: gcc.c-torture/execute/built-in-setjmp.c execution,  -O3 
-fomit-frame-pointer -funroll-all-loops -finline-functions
FAIL: gcc.c-torture/execute/built-in-setjmp.c execution,  -O3 -g
FAIL: gcc.c-torture/execute/built-in-setjmp.c execution,  -Os

This is an actual bug, that only appears with O2, O3 or Os. The 
__builtin_setjmp stores Y+1 in the setjmp buffer. With -O0 the first 
instruction after the jmp does a "sbiw r28, 1" that restores the 
original value, but for some reason, with higher optimization levels, 
this instruction is optimized away, leaving R28 pointing to the wrong 
address by one.

The __builtin_setjmp code:

 if (__builtin_setjmp (buf))
16a:    ce 01           movw    r24, r28
16c:    01 96           adiw    r24, 0x01       ; 1
 Notice the increment here, before storing R28

16e:    90 93 07 01     sts     0x0107, r25
172:    80 93 06 01     sts     0x0106, r24
176:    8f ee           ldi     r24, 0xEF       ; 239
178:    90 e0           ldi     r25, 0x00       ; 0
17a:    90 93 09 01     sts     0x0109, r25
17e:    80 93 08 01     sts     0x0108, r24
182:    ed b7           in      r30, 0x3d       ; 61
184:    fe b7           in      r31, 0x3e       ; 62
186:    f0 93 0b 01     sts     0x010B, r31
18a:    e0 93 0a 01     sts     0x010A, r30

The __builtin_longjmp code:

 __builtin_longjmp (buf, 1);
10c:    e0 91 08 01     lds     r30, 0x0108
110:    f0 91 09 01     lds     r31, 0x0109
114:    c0 91 06 01     lds     r28, 0x0106
118:    d0 91 07 01     lds     r29, 0x0107
11c:    80 91 0a 01     lds     r24, 0x010A
120:    90 91 0b 01     lds     r25, 0x010B

no decrement, R28 is used "as is".


FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -O0
FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -O1
FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -O2
FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -O3 
-fomit-frame-pointer
FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -O3 
-fomit-frame-pointer -funroll-loops
FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -O3 
-fomit-frame-pointer -funroll-all-loops -finline-functions
FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -O3 -g
FAIL: gcc.c-torture/execute/builtin-bitops-1.c compilation,  -Os

This test fails compilation with these errors:

builtin-bitops-1.c:(.text+0x672): undefined reference to `__ffshi2'
builtin-bitops-1.c:(.text+0x6b4): undefined reference to `__popcounthi2'
builtin-bitops-1.c:(.text+0x6ee): undefined reference to `__parityhi2'
builtin-bitops-1.c:(.text+0x183e): undefined reference to `__clzhi2'
builtin-bitops-1.c:(.text+0x1872): undefined reference to `__ctzhi2'

Maybe these should be defined in avr-libc? Is there something wrong 
with my setup?

I'm using gcc version: version 4.3.0 20080118 (experimental), binutils 
2.18 and avr-libc-1.6.1.


FAIL: gcc.c-torture/execute/ffs-1.c compilation,  -O0
FAIL: gcc.c-torture/execute/ffs-1.c compilation,  -O1
FAIL: gcc.c-torture/execute/ffs-1.c compilation,  -O2
FAIL: gcc.c-torture/execute/ffs-1.c compilation,  -O3 -fomit-frame-pointer
FAIL: gcc.c-torture/execute/ffs-1.c compilation,  -O3 -g
FAIL: gcc.c-torture/execute/ffs-1.c compilation,  -Os

This one just needs:

ffs-1.c:(.text+0x8): undefined reference to `__ffshi2'


FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -O0
FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -O1
FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -O2
FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -O3 -fomit-frame-pointer
FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -O3 
-fomit-frame-pointer -funroll-loops
FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -O3 
-fomit-frame-pointer -funroll-all-loops -finline-functions
FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -O3 -g
FAIL: gcc.c-torture/execute/ffs-2.c compilation,  -Os

And again:

ffs-2.c:(.text+0x8): undefined reference to `__ffshi2'
ffs-2.c:(.text+0x20): undefined reference to `__ffshi2'
ffs-2.c:(.text+0x38): undefined reference to `__ffshi2'
ffs-2.c:(.text+0x50): undefined reference to `__ffshi2'


FAIL: gcc.c-torture/execute/float-floor.c execution,  -O0
FAIL: gcc.c-torture/execute/float-floor.c execution,  -O1
FAIL: gcc.c-torture/execute/float-floor.c execution,  -O2
FAIL: gcc.c-torture/execute/float-floor.c execution,  -O3 -fomit-frame-pointer
FAIL: gcc.c-torture/execute/float-floor.c execution,  -O3 -g
FAIL: gcc.c-torture/execute/float-floor.c execution,  -Os

This test seems to only work if we have a 8 byte "double" type. So, 
this should probably be unsupported for avr.


FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -O0
FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -O1
FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -O2
FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -O3 -fomit-frame-pointer
FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -O3 
-fomit-frame-pointer -funroll-loops
FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -O3 
-fomit-frame-pointer -funroll-all-loops -finline-functions
FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -O3 -g
FAIL: gcc.c-torture/execute/multi-ix.c compilation,  -Os

This fails compilation with:

testsuite/multi-ix.c:38: error: total size of local objects too large

This code allocates large objects on the stack. To avoid having a stack 
overflow, it calculates the maximum size it gives to the type used for 
the vars on the stack using this macro:

#define CHUNK ((STACK_SIZE-40*sizeof(int)-256*sizeof(void *))/40/sizeof(int))

since our tests run with STACK_SIZE=400, this gives

((400-40*2-256*2) / 40 / 2) = -2

If I change the STACK_SIZE to 1000, the test runs successfully. Since 
the atmega128 has 4Kb of RAM, maybe we should increase our requirements 
a little.


FAIL: gcc.c-torture/execute/pr17377.c execution,  -O0
FAIL: gcc.c-torture/execute/pr17377.c execution,  -O1
FAIL: gcc.c-torture/execute/pr17377.c execution,  -O2
FAIL: gcc.c-torture/execute/pr17377.c execution,  -O3 -fomit-frame-pointer
FAIL: gcc.c-torture/execute/pr17377.c execution,  -O3 -g
FAIL: gcc.c-torture/execute/pr17377.c execution,  -Os

This also uses __builtin_return_address. Still tracking down this one.


FAIL: gcc.c-torture/execute/pr22493-1.c execution,  -O1
FAIL: gcc.c-torture/execute/pr22493-1.c execution,  -O2
FAIL: gcc.c-torture/execute/pr22493-1.c execution,  -Os

This is an actual bug. This function:

void f(int i)
{
 if (i>0)
   abort();
 i = -i;
 if (i<0)
   return;
 abort ();
}

is compiled to:

void f(int i)
{
 abort ();
}

because "if (i <= 0)", then "(-i >= 0)" must be true, right?

Unfortunately this is wrong for the corner case where "i = INT_MIN", 
because "-i" is also INT_MIN.


FAIL: gcc.c-torture/execute/pr27364.c execution,  -O1
FAIL: gcc.c-torture/execute/pr27364.c execution,  -O2
FAIL: gcc.c-torture/execute/pr27364.c execution,  -Os

This test assumes 32 bit integers.


And that's all, at least for now ;)

Attached is the latest avrtest I used to run these tests. It has some 
small improvements since last version:
- fixed running code from unnitialized flash, reported by Volkmar Dierkes
- fixed cycle counting for SBIW, ADIW and MOVW, reported by Dmitry K.
- rearranged acessor functions so that the log output for 16bit 
accesses now looks better
- changed the Makefile to always compile 2 versions of the executable: 
avrtest and avrtest_log. This way is not so easy to use the "logging" 
version for an actual testsuite run.

On an unrelated note, I tested running the testsuite with a dummy 
version of avrtest that did nothing except print "EXIT" and return from 
main, to measure the impact of the actual simulation time on the 
testsuite. The dummy test took 686 seconds to run, whereas the real 
test took 720 seconds. This means that the actual execution of code is 
less than 5% of the total time. So I'm not going to try to optimize 
avrtest more, because even if I do an amazing work and reduce the 
execution time to half, that will only improve 2.5% of the total time 
needed to run the testsuite :P


Now I'm trying to report all this through gcc's bugzilla, but I'm 
having some problems with the "component" selection. Possible choices:

- "target" if this is specific to avr, but how do I know that? Do I 
need to run the tests in other archs?

- "c", "middle-end", for the C frontend and middle-end, but where does 
the error come from? That would involve tracking down the error, which 
should be done after reporting it, no?

- "regression" has "regression tester failures" in the help text. Well, 
these are all regression tester failures. Maybe I should use this one 
for everything?

- "testsuite" for problems in the testsuite framework itself

Suggestions?

--
Paulo Marques


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

avrtest.tar.gz
Description: GNU Zip compressed data