[avr-gcc-list] Success running the testsuite with avrtest

[Paulo Marques]

Hi, all

I've finally got my setup working and was able to run the testsuite.

The initial summary I got for avr-gcc 4.2.2:

# of expected passes            30876
# of unexpected failures        1455
# of unexpected successes       1
# of expected failures          86
# of unresolved testcases       176
# of untested testcases         39
# of unsupported tests          848

I couldn't measure the total runtime because the "tree-prof" part of 
the test tried to open a remote shell to host "atmega128-sim" and 
locked up, so there might still be something funny going on.

Anyway, most of the tests run acceptably fast. Just to get an idea, 
after doing some more cleanups and small fixes, running just 
"execute.exp" took 10m14s on my P4 3GHz, resulting in:

# of expected passes            10833
# of unexpected failures        403
# of unexpected successes       1
# of unresolved testcases       15
# of untested testcases         4
# of unsupported tests          664

which gives an average of 19.4 tests per second.

I modified avrtest to fit the test harness and modified exit.c so that 
they worked well together. This way we don't need run-avr any more.

Attached is everything that's needed to run the testsuite. Short instructions:
- unpack the tar file somewhere
- run "make" to build avrtest and copy the avrtest executable to a 
directory in your path
- copy .dejagnurc to your $HOME and change the path inside it appropriatelly
- copy dejagnuboards to your $HOME
- edit $HOME/dejagnuboards/atmega128-sim.exp and change 
"/home/pmarques" to your $HOME
- on the root dir of the gcc build directory type 'make -k check 
RUNTESTFLAGS="--target_board=atmega128-sim"'

If you want to see the output of all the tests, even when they "PASS", 
add "--all" to the RUNTESTFLAGS.

Note that I'm still testing both avrtest and avr-gcc. So, some of the 
"FAIL" tests might be an "avrtest FAIL" and not so much a avr-gcc FAIL.

The next step is to start chasing down some of the FAIL tests and try 
to find out if they are being caused by avrtest, and fix the bugs in 
this case. I already did that initially when most of the tests were 
failing, to find it was a very silly bug in the way conditional branch 
offsets were being calculated :P

Time to go to sleep, now...

--
Paulo Marques


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

avrtest.tar.gz
Description: GNU Zip compressed data