Re: [Qemu-devel] block ais migration for machines <= 2.9

[Christian Borntraeger]

On 09/20/2017 06:04 PM, Dr. David Alan Gilbert wrote:
> * Christian Borntraeger (address@hidden) wrote:
>> Something like the following seems to do the tricks.
>> Needs proper patch description, review, full test with different kernel 
>> versions.....
> 
> Without knowing anything about 'ais' - will this break migration from
> 2.10 -> 2.10+this fix?

The problem only happens if the source system has kernel >= 4.12 thats why it
somehow missed my testing during the 2.10 freeze.

Now: "2.10" -> "2.10+this fix" will works fine for the 2.10 machine, but it 
does 
fail for the combination of qemu-2.10 with an older machine type like 2.8 and 
2.9.

qemu-system-s390x: Failed to load s390-flic/ais:tmp
qemu-system-s390x: error while loading state for instance 0x0 of device 
's390-flic'
qemu-system-s390x: load of migration failed: Function not implemented

We do run into an explicit check from hw/intc/s390_flic_kvm.c (see the return 
ENOSYS)

e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 443) static int 
kvm_flic_ais_post_load(void *opaque, int version_id)
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 444) {
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 445)     
KVMS390FLICStateMigTmp *tmp = opaque;
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 446)     KVMS390FLICState 
*flic = tmp->parent;
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 447)     struct 
kvm_s390_ais_all ais = {
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 448)         .simm = 
tmp->simm,
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 449)         .nimm = 
tmp->nimm,
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 450)     };
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 451)     struct 
kvm_device_attr attr = {
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 452)         .group = 
KVM_DEV_FLIC_AISM_ALL,
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 453)         .addr = 
(uint64_t)&ais,
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 454)     };
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 455) 
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 456)     /* This can 
happen when the user mis-configures its guests in an
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 457)      * incompatible 
fashion or without a CPU model. For example using
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 458)      * qemu with 
-cpu host (which is not migration safe) and do a
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 459)      * migration 
from a host that has AIS to a host that has no AIS.
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 460)      * In that case 
the target system will reject the migration here.
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 461)      */
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 462)     if 
(!ais_needed(flic)) {
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 463)         return 
-ENOSYS;
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 464)     }
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 465) 
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 466)     return 
ioctl(flic->fd, KVM_SET_DEVICE_ATTR, &attr) ? -errno : 0;
e7be8d49 (Yi Min Zhao       2017-05-16 18:58:44 +0800 467) }

So I think if we fix this we should definitely add this to 2.10.1 
to minimize the affected qemu versions. Not fixing it will break
migrations from 4.12/2.10 to older versions. 
 
> 
> Dave
> 
>> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
>> index 1c7af39..2ff32ba 100644
>> --- a/hw/s390x/s390-virtio-ccw.c
>> +++ b/hw/s390x/s390-virtio-ccw.c
>> @@ -212,6 +212,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void 
>> *data)
>>      s390mc->cpu_model_allowed = true;
>>      s390mc->css_migration_enabled = true;
>>      s390mc->gs_allowed = true;
>> +    s390mc->ais_allowed = true;
>>      mc->init = ccw_init;
>>      mc->reset = s390_machine_reset;
>>      mc->hot_add_cpu = s390_hot_add_cpu;
>> @@ -305,6 +306,11 @@ bool gs_allowed(void)
>>      return false;
>>  }
>>  
>> +bool ais_allowed(void)
>> +{
>> +    return get_machine_class()->ais_allowed;
>> +}
>> +
>>  static char *machine_get_loadparm(Object *obj, Error **errp)
>>  {
>>      S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
>> @@ -533,6 +539,7 @@ static void ccw_machine_2_9_class_options(MachineClass 
>> *mc)
>>      S390CcwMachineClass *s390mc = S390_MACHINE_CLASS(mc);
>>  
>>      s390mc->gs_allowed = false;
>> +    s390mc->ais_allowed = false;
>>      ccw_machine_2_10_class_options(mc);
>>      SET_MACHINE_COMPAT(mc, CCW_COMPAT_2_9);
>>      s390mc->css_migration_enabled = false;
>> diff --git a/include/hw/s390x/s390-virtio-ccw.h 
>> b/include/hw/s390x/s390-virtio-ccw.h
>> index 41a9d28..bba8660 100644
>> --- a/include/hw/s390x/s390-virtio-ccw.h
>> +++ b/include/hw/s390x/s390-virtio-ccw.h
>> @@ -41,6 +41,7 @@ typedef struct S390CcwMachineClass {
>>      bool cpu_model_allowed;
>>      bool css_migration_enabled;
>>      bool gs_allowed;
>> +    bool ais_allowed;
>>  } S390CcwMachineClass;
>>  
>>  /* runtime-instrumentation allowed by the machine */
>> @@ -49,6 +50,8 @@ bool ri_allowed(void);
>>  bool cpu_model_allowed(void);
>>  /* guarded-storage allowed by the machine */
>>  bool gs_allowed(void);
>> +/* ais allowed by the machine */
>> +bool ais_allowed(void);
>>  
>>  /**
>>   * Returns true if (vmstate based) migration of the channel subsystem
>> diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
>> index c4c5791..531d474 100644
>> --- a/target/s390x/kvm.c
>> +++ b/target/s390x/kvm.c
>> @@ -309,7 +309,9 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>>      }
>>  
>>      /* Try to enable AIS facility */
>> -    kvm_vm_enable_cap(s, KVM_CAP_S390_AIS, 0);
>> +    if (ais_allowed()) {
>> +       kvm_vm_enable_cap(s, KVM_CAP_S390_AIS, 0);
>> +    }
>>  
>>      qemu_mutex_init(&qemu_sigp_mutex);
>>
>>
> --
> Dr. David Alan Gilbert / address@hidden / Manchester, UK
>