Re: [Qemu-devel] QEMU ARM SMP: IPI delivery delayed until next main loop event // how to improve IPI latency?

[Peter Maydell]

On 15 June 2015 at 16:07, Alex Züpke <address@hidden> wrote:
> Am 15.06.2015 um 17:04 schrieb Peter Maydell:
>> On 12 June 2015 at 17:38, Alex Züpke <address@hidden> wrote:
>>> Hi,
>>>
>>> I'm benchmarking some IPI (== inter-processor-interrupt) synchronization 
>>> stuff of my custom kernel on QEMU ARM (qemu-system-arm -M vexpress-a15 -smp 
>>> 2) and ran into the following problem: pending IPIs are delayed until the 
>>> QEMU main loop receives an event (for example the timer interrupt expires 
>>> or I press a key on the console).
>>>
>>> The following timing diagram tries to show this:
>>>
>>>   CPU #0                       CPU #1
>>>   ======                       ======
>>>   ... other stuff ...          WFI (wait for interrupt, like x86 "HLT")
>>>   send SGI in MPCore
>>>   polls for completeness
>>>                  <time passes ...>
>>>   polls ...
>>>                  <... and passes ...>
>>>   still polls ...
>>>                  <... and passes ...>
>>>   still polls ...
>>>                  <... and passes ...>
>>>
>>>
>>>                  <timer interrupt expires>
>>>                  <now QEMU switches to CPU #1>
>>>                                receives IPI
>>>                                signals completeness
>>>                                WFI
>>>                  <QEMU switches to CPU #0>
>>>   polling done
>>>   process timer interrupt
>>>   ...
>>
>> Does your polling loop have a YIELD insn in it? We (and hardware)
>> can use that as a hint that you're busy-looping and we should
>> try doing something else. (QEMU doesn't implement that for A32/T32
>> yet, but we should; we already do on A64.)
>
> Yes, I should be yielding here, but SEV isn't implemented.
> Probably the notification should be done there as well.

YIELD isn't related to SEV -- it's just a generic "hey, I'm
polling" hint. We NOP SEV, and make WFE be a "yield this CPU's
timeslice" event", which is architecturally valid, and sufficient
for this situation anyway.

-- PMM