Re: [avr-gcc-list] binutils/.../testsuite/avr

[Theodore A. Roth]

On Sat, 4 Dec 2004, Klaus Rudolph wrote:

> Hi all,
>
> >On the other hand, binutils and gcc just spit out a file. As such, there
> >should be no need for a simulator for the most part. It's just a matter
> >of compiling or assembling a chunk of code and then comparing the output
> >to some expected result. The output used for the compare could either be
> >the raw intermediate asm file for a C code fragment or possibly a
> >disassemle of the object file.
> >
> >
> I think that is ok for assembler but never for C/C++. If you change
> something in gcc maybe a better optimizer
> the code is valid but the comparisson against a older file is not ok. So
> a simulation is needed
> to check that the code *do* the same and not *look* the same. :-)

In that case, you have changed the expected output, so you have to
change the compare expression. Besides, if you change gcc to make a
better optimization, the old output file is no longer of interest since
gcc will no longer generate it. Remember that the compare expression is
going to be tied to the command line options that you feed to gcc and to
the version of gcc that you are running. Also remember that we are not
comparing the previous output to the new output, we are comparing the
current output with an expected expression of that output. What the
testsuite does more than anything is insure that if you change the
output for input X, it doesn't change the output for input Y if X and Y
are not related in any way.

If you were to use a simulator and only look at the simulator's results
instead of the code that gcc generated, you don't really know that your
change to gcc had any effect. Suppose the change to gcc was in the wrong
place so your test case didn't even exercise that new code. You've got a
false positive test result since gcc didn't change the output file, but
the sim still gives the same output.

Also, the sim really doesn't do the same thing if you change it's input.
The end result of the sequence of instructions may be the same, but the
sequence of instructions is different. So in the end, it's the sequence
of instructions that we really care about.

If you are changing gcc to alter the asm it spits out, you better darn
well know what instructions it should spit out and that gives you the
basis to write the expected output compare expression. If you can't
write that expected output compare expression, I fail to see how using a
simulator will prove that gcc did indeed spit out the correct sequence
of instructions.

---
Ted Roth
PGP Key ID: 0x18F846E9
Jabber ID: address@hidden