Re: test result code was Re: [Simulavr-devel] avrtest and simulavr vs simulavrxx

[Joel Sherrill]

Klaus Rudolph wrote:
> Joel Sherrill wrote:
>   
> > Klaus Rudolph wrote:
> >     
> > > > This is WAY overkill if all you want is the exit status.  This
> > > > is why avrtest has magic ports for exit and abort.  You write
> > > > the exit code to the exit magic port and poof exit(value) is
> > > > called.  This is the way all the simulators in gdb work.
> > > >
> > > >     
> > > >         
> > > ???
> > > gdb can read the memory direct from simulator. So there is no need for 
> > > an elf file nor an special exit port.
> > > ???
> > >
> > > Can you explain what is an exit port please. I think I have a complete 
> > > misunderstanding of this topic.
> > >
> > >   
> > >       
> > All of the simulators in gdb have a special syscall call mechanism
> > where you can access the host.  The intent is to have enough
> > support to make the simulators easy to run and use with the
> > GNU tools test suites -- in particular gcc.
> >     
> I think I have not understand that.
>
> Every avr application can do an exit() which results in going to _exit 
> to the avr library. The simulation can have a breakpoint there and gdb 
> can read out return values from the normal register set. This also is 
> possible from exit at main. What we have to implement to do it in an 
> other way?
>
>   
> > Just as simulavrxx already has magic ports for debug IO,
> > the GDB simulators also have a way for the application running
> > on the simulator to force an exit() with a specific return value.
> >     
>
> If my avr code have an exit() it will do that?!
> Or is intended that I have to write a special register to get that 
> behavior? Why?
>
>
>
>   
> > The key is that GCC doesn't care or know about specific CPU
> > models -- only the CPU core (e.g. multilib variant).  The peripherals
> > are irrelevant as are interrupts.  So having "synthetic CPU models"
> > which correspond to a CPU core and simulator specific peripherals
> > which ease testing is perfect.  For example, (if my AVR knowledge
> > is right) from GCC's perspective, these are the same:
> >
> > 64KB/100-pin version: ATmega640 
> > <http://www.atmel.com/dyn/products/product_card.asp?PN=ATmega640>
> > 128KB/100-pin version: ATmega1280 
> > <http://www.atmel.com/dyn/products/product_card.asp?PN=ATmega1280>
> > 256KB/64-pin version: ATmega2561 
> > <http://www.atmel.com/dyn/products/product_card.asp?PN=ATmega2561>
> > 256KB/100-pin version: ATmega2560 
> > <http://www.atmel.com/dyn/products/product_card.asp?PN=ATmega2560>
> >
> >
> > The key features you need in a simulator for gcc testing are:
> >
> > + console output mapped to stdout
> >     
> we have this
>   
> > + time limit on execution
> >     
> checked in a few hours ago
>   
> > + enough memory to run tests
> >     
> means adding testcode to some kind of additional virtual memory?
>   
No.  It generally means that you have a synthetic target with
a reasonable amount of memory.  On many embedded CPUs, this
turns out to be maximum amount of memory supported by the
architecture.  GCC doesn't care about whether the CPU model
has 64, 128 or 256K.  What's the architectural limit for that
CPU core?  Configure a synthetic simulator to support that
and you can verify every test that is able to run on your
CPU core variant. 

You have to think in terms of multilib variants and CPU cores
not CPU models.  Most of what you are testing is NOT concerned
about a particular model -- just the core. 


> > + easy way to get exit status from simulator.
> >     
> read registers
>   
You are failing to see that the simulator can do a lot
for you that a real target can't.

Thinking out load.  If the termination method is
exit and you hit exit, why don't we just print the
values in the status code registers and be done with it?
A few lines of C++ code will eliminate the need to
parse an object dump and probably cut a big chunk off the
GCC test time.

Some of the gdb simulators actually can be told
the high and low address of the stack.  If the pointer
gets outside that range then the simulator detects the
error and traps.


> > The last isn't even needed by the DejaGNU tests since the
> > framework has a way to print the status on exit().
> >     
> Which way?
>   
If you have anyway to shutdown the simulator voluntarily
from the application, then the DejaGNU scripts can convert
all calls to exit into a call to test_exit() and that will print the
status to the standard out.  The normal DejaGNU framework
handles all this.
> > So there is tool testing and application testing.  For
> > application testing, the fidelity of the peripherals is
> > important.  But irrelevant for tools testing -- you want
> > that to be simple.
> >
> > avrtest has three magic ports added by someone for
> > tools testing -- out char, exit, and abort.  For avrtest,
> > the code in question is easy to read:
> >
> > ===========================
> > static void data_write_byte_raw(int address, int value)
> > {
> >    // add code here to handle special events
> >    switch (address) {
> >        case STDIO_PORT:
> >            putchar(value);
> >            return;
> >        case EXIT_PORT:
> >            leave(value ? EXIT_STATUS_ABORTED : EXIT_STATUS_EXIT, "exit 
> > function called");
> >            break;
> >        case ABORT_PORT:
> >            leave(EXIT_STATUS_ABORTED, "abort function called");
> >            break;
> >    }
> >
> >    // default action, just store the value
> >    cpu_data[address] = value;
> > }
> >
> > ===========================
> >     
>
> OK, STDIO_PORT is available from command line. -W
>
> abort and exit function called could be monitored simply by having the 
> breakpoints there. -T or the new :-) -B option.
> If there is a need to write additional data to somewhere, you can do 
> that with some more -W registers if needed.
>   
I have added the extra register functionality from avrtest
with register arguments. 
> Maybe my misunderstanding continuous :-)
>
> If you have an example test for me and the list of results you expect?!
> Maybe this will help me to get on this topic.
>   
All I can do is point you to the GCC test suite.  80K tests
run by DejaGNU.  They are setup based upon what I have
described above.  They generally run quickly and exit
with a status code.  You have to arrange for your
simulator target to return the status code to the
framework and there are at least two ways (exit status
and the print method I described above).
>   
> > I have implemented the functional equivalent of the above
> >     
>
> Especially for the stdio port this is a second implementation. As 
> mentioned, -W is your friend :-)
>
>
>   
I didn't add a new option.  I did use that.  IMO for avrtest,
it is sufficient to say:

for avrtest compatability,  invoke like this: ....

I would be happy to rename the executable and provide wrapper
scripts for higher command line compatibility.  Functional
equivalence - not command line.

--
Joel Sherrill, Ph.D.             Director of Research & Development
address@hidden        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
  Support Available             (256) 722-9985