Re: AVR Benchmark Test Suite [was: RE: [avr-gcc-list] GCC-AVR Register optimisations]

[John Regehr]

I'll just plug Avrora again:

  http://compilers.cs.ucla.edu/avrora/

It runs on many platforms (written all in Java), is quite fast, and is 
well designed.  Best of all it is easy to extend, you just add "monitors" 
that can be configured to receive a wide variety of callbacks about 
program events such as memory operations, I/O operations, execution of 
different kinds of instructions, interrupts, etc.

> focus
> on AVR-specific code, and GCC-specific AVR code at that.

Definitely.  If people want to test avr-gcc against other compilers, or 
compare AVR to other architectures, that's a separate exercise.

MiBench is an aging but useful collection of embedded C codes:

  http://www.eecs.umich.edu/mibench/

> John, I would welcome publicly available code from TinyOS, but I would
> need to be already compiled with nesc, so that way we just have straight
> C that we can feed into avr-gcc.

Sure, this is easy.  It'll target ATmega128 only, howver.

Re. floating point I believe that the "papabench" codes do a lot of this:

http://www.irit.fr/recherches/ARCHI/MARCH/rubrique.php3?id_rubrique=97

This is code extracted from the Paparazzi UAV project, which uses an 
ATmega for onboard flight control.

> There needs to be some consensus on what we measure, how we measure it,
> what output files we want generated, and hopefully some way to
> automatically generate composite results. I'm certainly open to anything
> in this area.

Code size and static RAM consumption are obvious.  Some sort of throughput 
metric is useful.  For interrupt-driven codes, my group often uses 
processor duty cycle as a measure of efficiency.  This is the % of time 
that the CPU is not in a sleep mode.  Dyanmic stack memory consumption is 
good, though this is not a very consistent metric for interrupt-driven 
codes since in a short simulation run the worst-case stack usage is 
unlikely to be encountered.  Perhaps adding up the stack memory usage of 
main + all interrupts would be better.

John Regehr