Re: [Qemu-devel] [Query] Live Migration between machines with different processor ids

[Andrew Jones]

On Wed, Sep 05, 2018 at 11:46:11AM +0000, Jaggi, Manish wrote:
> (a) Changes in KVM:
> 
> - Introducing a specific error code (KVM_EINVARIANT) in case of invariant 
> writes.
> This should not change anything to API SET_ONE_REG KVM API.
> Not sure which is the best place to put the define…
> I have added in include/uapi/linux/kvm_para.h.
> 
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 22fbbdbece3c..c8a4fbe8a8bb 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -1111,7 +1111,7 @@ static int __set_id_reg(const struct sys_reg_desc *rd, 
> void __user *uaddr,
>  
>         /* This is what we mean by invariant: you can't change it. */
>         if (val != read_id_reg(rd, raz))
> -               return -EINVAL;
> +               return -KVM_EINVARIANT;
>  
>         return 0;
>  }
> diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
> index 6c0ce49931e5..3a49a321d0df 100644
> --- a/include/uapi/linux/kvm_para.h
> +++ b/include/uapi/linux/kvm_para.h
> @@ -17,6 +17,7 @@
>  #define KVM_E2BIG              E2BIG
>  #define KVM_EPERM              EPERM
>  #define KVM_EOPNOTSUPP         95
> +#define KVM_EINVARIANT          96
>  
>  #define KVM_HC_VAPIC_POLL_IRQ          1
>  #define KVM_HC_MMU_OP                  2
> 
> (b) Changes in Qemu code
> 
> 1. Handling of new error code, which would update
> guest state with hosts invariant reg values.
> 
> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> index 65f867d569..0cf14323a2 100644
> --- a/target/arm/kvm.c
> +++ b/target/arm/kvm.c
> @@ -452,7 +452,15 @@ bool write_list_to_kvmstate(ARMCPU *cpu, int level)
>              abort();
>          }
>          ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
> -        if (ret) {
> +        if (ignore_invariant && (ret == -KVM_EINVARIANT)) {
> +            /* Update Guest invariant to match with migrated host regs*/
> +            ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
> +            if (ret)
> +                ok = false;
> +            else
> +                cpu->cpreg_values[i] = r.addr;
> +        }
> +        else if (ret) {
>              /* We might fail for "unknown register" and also for
>               * "you tried to set a register which is constant with
>               * a different value from what it actually contains".
> 
> 2. ignore_invariant is the flag I was referring to which is what you also
> mentioned opt-in. This can be supplied as a command line parameter to qemu on 
> Machine B.

The same libvirt folk also suggest that a QMP command be provided that
allows the selection of this ignore-invariant mode (in addition to or
instead of the command line parameter.) Otherwise a guest that has
already started without the parameter will not be able to migrate to
a "close enough" host - even if it's decided later that it would be OK
to do so.

> 
> PS: I will add code to put warning logs as suggested by Dave.

Yeah, I like that idea too.

This approach looks good to me. Let's see what maintainers say when they
see the patch submission.

Thanks,
drew