[Qemu-devel] [RFC 0/4] target-arm: KVM to TCG migration

[Alvise Rigo]

QEMU at the moment doesn't allow the migration between KVM and TCG, in
both directions. This limitation is due first of all to the different set
of coprocessor registers supported by KVM and TCG, but also in the way
the coprocessors values are copied from the incoming data to the local QEMU
structures.
This set of patches is focused on the KVM to TCG migration using the
CortexA15 processor in both sides and tries to enhance the migration of
cp registers in order to prevent QEMU from failing migration in some
particular situations.
To illustrate the changes proposed with this patch series, we will call
"generator" a register that generates other registers in the TCG
coprocessor hash table, such a register can have one or more wildcarded
fields in its definition, or can have an CP_ARM_STATE_BOTH state (in
this case the generator is the AARCH64 view of the register). With the
term "child" a register generated by a generator will be indicated.
It turns out that KVM tries to migrate some child registers (that in TCG
are marked as CP_ARM_NO_MIGRATE) leading to a fault during the copy of
cp registers.

These patch series modify the coprocessor migration with the following steps:
- during the copy of the incoming cp registers inside cpu_post_load a list
  is kept containing the registers that haven't matched any of the registers
  inside the cpreg_list.
- only if the previous step has generated a not empty list, we try to
  handle the copy of these registers "by hand", verifying for every
  register in the list if it is:
    - already present in the hash table
    - flagged with CP_ARM_NO_MIGRATE
In this case if the register has a generator which is migratable we raw
write its value and then we remove it from the list, otherwise we simply
remove it from the list (because still a not migratable register for TCG).
When we raw_write the value of the register in the hash table we take
care of marking the generator register as already "updated" (modifying the
flag skip_cpreglist to true) because it has the same value of its children.
This will prevent the method write_list_to_cpustate from overwriting the value
of a child register with the default value of its parent: this is what
happens when KVM migrates a child register and not its generator.

Finally, it is also necessary to not update those registers with
constant values, because KVM and TCG in some cases don't agree with their
default values; this is done setting the skip_cpreglist flag to true for every
ARM_CP_CONST registers.
This is the case of constant registers like ID_PFR0 (the Jazelle bit for
instance is set in TCG and not set in KVM) and AUXCR (KVM read its
value directly form the processor while in TCG is not set); here a much
cleaner solution is needed.

All this "handling" is confined inside the method 
handle_cpreg_kvm2tcg_migration.
This method will be called only if is detected that we are migrating from
KVM to TCG (a new flag inside ARMCPU was added to address this purpose).
If we are performing a normal migration, be it TCG<->TCG or KVM<->KVM, nothing
will change and the usual copy of cp registers and aborting criteria will hold.

The other way of migration is not yet covered by this patch set. KVM lacks of
several coprocessor registers that in TCG are instead supported; solving this
open issue will be the main challenge of the TCG to KVM migration.

This set of patches has been tested by booting a guest running the
Linux 3.13 kernel in a host using KVM. Two types of hosts have been used:
ARM FastModels and the ARM Chromebook. Then, in the middle of the kernel
boot, the guest was migrated to a x86 host running TCG.
The machine model used is vexpress-a15 with cortex-a15 as CPU.

Alvise Rigo (4):
  Fix issue affecting get_int32_le() in vmstate.c
  Added flag in ARMCPU to track last execution mode
  Add l2ctlr cp register to CPUARMState
  Relevant changes to enable KVM to TCG migration

 target-arm/cpu-qom.h |   3 +
 target-arm/cpu.c     |  22 +++++--
 target-arm/cpu.h     |  43 +++++++++++++
 target-arm/helper.c  | 167 +++++++++++++++++++++++++++++++++++++++++++++++++--
 target-arm/machine.c |  46 ++++++++++----
 vmstate.c            |   8 +--
 6 files changed, 263 insertions(+), 26 deletions(-)

-- 
1.8.3.2