Re: [Qemu-devel] Re: [RFT][PATCH 07/15] qemu_irq: Add IRQ handlers with delivery feedback

[Blue Swirl]

On Thu, May 27, 2010 at 6:31 PM, Jan Kiszka <address@hidden> wrote:
> Blue Swirl wrote:
>> On Wed, May 26, 2010 at 11:26 PM, Paul Brook <address@hidden> wrote:
>>>> At the other extreme, would it be possible to make the educated guests
>>>> aware of the virtualization also in clock aspect: virtio-clock?
>>> The guest doesn't even need to be aware of virtualization. It just needs to 
>>> be
>>> able to accommodate the lack of guaranteed realtime behavior.
>>>
>>> The fundamental problem here is that some guest operating systems assume 
>>> that
>>> the hardware provides certain realtime guarantees with respect to execution 
>>> of
>>> interrupt handlers.  In particular they assume that the CPU will always be
>>> able to complete execution of the timer IRQ handler before the periodic 
>>> timer
>>> triggers again.  In most virtualized environments you have absolutely no
>>> guarantee of realtime response.
>>>
>>> With Linux guests this was solved a long time ago by the introduction of
>>> tickless kernels.  These separate the timekeeping from wakeup events, so it
>>> doesn't matter if several wakeup triggers end up getting merged (either at 
>>> the
>>> hardware level or via top/bottom half guest IRQ handlers).
>>>
>>>
>>> It's worth mentioning that this problem also occurs on real hardware,
>>> typically due to lame hardware/drivers which end up masking interrupts or
>>> otherwise stall the CPU for for long periods of time.
>>>
>>>
>>> The PIT hack attempts to workaround broken guests by adding artificial 
>>> latency
>>> to the timer event, ensuring that the guest "sees" them all.  Unfortunately
>>> guests vary on when it is safe for them to see the next timer event, and
>>> trying to observe this behavior involves potentially harmful heuristics and
>>> collusion between unrelated devices (e.g. interrupt controller and timer).
>>>
>>> In some cases we don't even do that, and just reschedule the event some
>>> arbitrarily small amount of time later. This assumes the guest to do useful
>>> work in that time. In a single threaded environment this is probably true -
>>> qemu got enough CPU to inject the first interrupt, so will probably manage 
>>> to
>>> execute some guest code before the end of its timeslice. In an environment
>>> where interrupt processing/delivery and execution of the guest code happen 
>>> in
>>> different threads this becomes increasingly likely to fail.
>>
>> So any voodoo around timer events is doomed to fail in some cases.
>> What's the amount of hacks what we want then? Is there any generic
>
> The aim of this patch is to reduce the amount of existing and upcoming
> hacks. It may still require some refinements, but I think we haven't
> found any smarter approach yet that fits existing use cases.

I don't feel we have tried other possibilities hard enough.

>> solution, like slowing down the guest system to the point where we can
>> guarantee the interrupt rate vs. CPU execution speed?
>
> That's generally a non-option in virtualized production environments.
> Specifically if the guest system lost interrupts due to host
> overcommitment, you do not want it slow down even further.

I meant that the guest time could be scaled down, for example 2s in
wall clock time would be presented to the guest as 1s. Then the amount
of CPU cycles between timer interrupts would increase and hopefully
the guest can keep up. If the guest sleeps, time base could be
accelerated to catch up with wall clock and then set back to 1:1 rate.

Slowing down could be triggered by measuring the guest load (for
example, by checking for presence of halt instructions), if it's close
to 1, time would be slowed down. If the guest starts to issue halt
instructions because it's more idle, we can increase speed.

If this approach worked, even APIC could be made ignorant about
coalescing voodoo so it should be a major cleanup.