Re: [Qemu-devel] invtsc + migration + TSC scaling

[Radim Krčmář]

2016-10-18 15:09-0200, Marcelo Tosatti:
> On Tue, Oct 18, 2016 at 03:41:03PM +0200, Paolo Bonzini wrote:
>> On 18/10/2016 01:58, Marcelo Tosatti wrote:
>> > > We should also blacklist the TSC deadline timer when invtsc is not
>> > > available.
>> >
>> > Actually, a nicer fix would be to check the different 
>> > frequencies and scale the deadline relative to the difference. 
>> 
>> You cannot know what exactly the guest was thinking when it set the TSC
>> deadline.  Perhaps it wanted an interrupt when the TSC was exactly
>> 9876543210.

Yep, the spec just says that the timer fires when TSC >= deadline.

> You can't expect the correlation between TSC value and timer interrupt
> execution to be precise, because of the delay between HW timer
> expiration and interrupt execution.

Yes, this is valid.

> So you have to live with the fact that the TSC deadline timer can be
> late (which is the same thing as with your paravirt solution, in case 
> of migration to host with faster TSC freq) (which to me renders the
> argument above invalid).
> 
> Solution for old guests and new guests:
> Just save how far ahead in the future the TSC deadline timer is supposed
> to expire on the source (in ns), in the destination save all registers 
> (but disable TSC deadline timer injection), arm a timer in QEMU 
> for expiration time, reenable TSC deadline timer reinjection.

The interrupt can also fire early after migrating to a TSC with lower
frequency, which violates the definition of TSC deadline timer when an
OS could even RDTSC a value lower than the deadline in the interrupt
handler.  (An OS doesn't need to expect that.)

We already have vcpu->arch.virtual_tsc_khz that ought to remain constant
for a lifetime of the guest and KVM uses it to convert TSC delta into
correct nanoseconds.

The main problem is that QEMU changes virtual_tsc_khz when migrating
without hardware scaling, so KVM is forced to get nanoseconds wrong ...

If QEMU doesn't want to keep the TSC frequency constant, then it would
be better if it didn't expose TSC in CPUID -- guest would just use
kvmclock without being tempted by direct TSC accesses.