Re: [avr-gcc-list] Success running the testsuite with avrtest

[Paulo Marques]

Quoting "Dmitry K." <address@hidden>:
> [...]
> Alas, the first (add.c) is still unworked.

Ok, a new version is ready (attached) with this problem fixed (and a 
few others). In this version there is a "//#define LOG_DUMP	1" line 
right at the top of the avrtest.c file. Uncommenting this line makes 
avrtest dump a lot of debug information about what it is doing. It is a 
good feature to have to track down a particular problem, but it is not 
recommend to be used to run the testsuite (I don't think it works at 
all).

I've been trying to run the testsuite, and fixing problems in avrtest 
as they appear. I've already changed the exit(1) to be treated as an 
error, so that the results are more accurate (but as Andrew predicted, 
it didn't change a thing).

Right now I'm down to:

# of expected passes            11174
# of unexpected failures        66
# of unexpected successes       1
# of unresolved testcases       15
# of unsupported tests          664

Although there are 66 tests that FAIL, only 51 are execution failures, 
and 15 are compilation failures. These 51 failures correspond to just 
10 programs that fail to execute, with different optimization options 
(-O0, -O1, etc.):

execute/20010122-1.c
execute/20060102-1.c
execute/20061101-1.c
execute/float-floor.c
execute/pr17377.c
execute/pr22493-1.c
execute/pr23135.c
execute/pr28651.c
execute/pr29797-1.c
execute/simd-1.c

The tests that fail with "UNRESOLVED" are tests that fail to compile. 
If I understood correctly, the compilation test fails and then the 
execution test remains UNRESOLVED.

I'll do a short summary of problems I've already found here, so that I 
don't lose them (but I'll fill proper bug reports later):

tests that fail from "undefined reference to `__clzhi2'", "undefined 
reference to `__ctzhi2'", "undefined reference to `__popcounthi2'" or 
"undefined reference to `__parityhi2'":

- gcc.c-torture/execute/20061101-1.c
- gcc.c-torture/execute/builtin-bitops-1.c:

Tests that fail because assume 32 bit integers:

- gcc.c-torture/execute/pr31448.c

Tests that fail from "undefined reference to `bzero'":

- gcc.c-torture/execute/multi-ix.c:

Note: right now I'm still just running execute.exp, because the full 
batch of tests starts with a lot of "compilation" tests that don't help 
me debug avrtest at all. Once this test gets more stable, I'll start 
running full tests, to see the global result.

I'll keep you posted,

--
Paulo Marques


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

avrtest.tar.gz
Description: GNU Zip compressed data