Re: [Qemu-devel] QEMU ARM SMP: IPI delivery delayed until next main loop event // how to improve IPI latency?

[Peter Maydell]

On 16 June 2015 at 12:53, Peter Maydell <address@hidden> wrote:
> In particular I think the
> 'do cpu_exit if one CPU triggers an interrupt on another'
> approach is probably good, but I need to investigate why
> it isn't working on your test programs without that extra
> 'level &&' condition first...

I've figured out what's happening here, and it's an accidental
artefact of our GIC implementation. What happens is:

 * cpu 0 does an IPI, which turns into "raise IRQ line on cpu 1"
 * arm_cpu_set_irq logic causes us to cpu_exit() cpu 0
 * cpu 1 does then run; however pretty early on it does a read
   on the GIC to acknowledge the interrupt
 * this causes the function gic_update() to run, which recalculates
   the current state and sets CPU interrupt lines accordingly;
   among other things this results in an unnecessary but harmless
   call to arm_cpu_set_irq(CPU #0, irq, 0)
 * without the "level && " clause in the conditional, that causes
   us to cpu_exit() cpu 1
 * we then start running cpu 0 again, which is pointless, and
   since there's no further irq traffic we don't yield til 0
   reaches the end of its timeslice

So basically without the level check we do make 0 yield to 1
as it should, but we then spuriously yield back to 0 again
pretty much immediately.

Next up: see if it gives us a perf improvement on Linux guests...

-- PMM