Re: [Qemu-devel] [PATCH v2 1/1] Introduce "xen-load-devices-state"

[Wen Congyang]

On 03/23/2016 04:56 PM, Dr. David Alan Gilbert wrote:
> * Changlong Xie (address@hidden) wrote:
>> On 03/22/2016 08:22 PM, Dr. David Alan Gilbert wrote:
>>> * Changlong Xie (address@hidden) wrote:
>>>> From: Wen Congyang <address@hidden>
>>>>
>>>> Introduce a "xen-load-devices-state" QAPI command that can be used to
>>>> load the state of all devices, but not the RAM or the block devices of
>>>> the VM.
>>>>
>>>> We only have hmp commands savevm/loadvm, and qmp commands
>>>> xen-save-devices-state.
>>>
>>> Can you explain on Xen how the RAM actually gets loaded?
>>
>> Xen use xc(xen toolstack) to do RAM restore/save
>>
>>>
>>>>
>>>> We use this new command for COLO:
>>>> 1. suspend both primary vm and secondary vm
>>>> 2. sync the state
>>>> 3. resume both primary vm and secondary vm
>>>>
>>>> In such case, we need to update all devices' state in any time.
>>>>
>>>> Signed-off-by: Wen Congyang <address@hidden>
>>>> Signed-off-by: Changlong Xie <address@hidden>
>>>> ---
>>>>  migration/savevm.c | 36 ++++++++++++++++++++++++++++++++++++
>>>>  qapi-schema.json   | 14 ++++++++++++++
>>>>  qmp-commands.hx    | 27 +++++++++++++++++++++++++++
>>>>  3 files changed, 77 insertions(+)
>>>>
>>>> diff --git a/migration/savevm.c b/migration/savevm.c
>>>> index 96e7db5..aaead12 100644
>>>> --- a/migration/savevm.c
>>>> +++ b/migration/savevm.c
>>>> @@ -50,6 +50,7 @@
>>>>  #include "qemu/iov.h"
>>>>  #include "block/snapshot.h"
>>>>  #include "block/qapi.h"
>>>> +#include "hw/xen/xen.h"
>>>>
>>>>
>>>>  #ifndef ETH_P_RARP
>>>> @@ -1768,6 +1769,12 @@ qemu_loadvm_section_start_full(QEMUFile *f, 
>>>> MigrationIncomingState *mis)
>>>>          return -EINVAL;
>>>>      }
>>>>
>>>> +    /* Validate if it is a device's state */
>>>> +    if (xen_enabled() && se->is_ram) {
>>>> +        error_report("loadvm: %s RAM loading not allowed on Xen", idstr);
>>>> +        return -EINVAL;
>>>> +    }
>>>> +
>>>>      /* Add entry */
>>>>      le = g_malloc0(sizeof(*le));
>>>>
>>>> @@ -2077,6 +2084,35 @@ void qmp_xen_save_devices_state(const char 
>>>> *filename, Error **errp)
>>>>      }
>>>>  }
>>>>
>>>> +void qmp_xen_load_devices_state(const char *filename, Error **errp)
>>>> +{
>>>> +    QEMUFile *f;
>>>> +    int saved_vm_running;
>>>> +    int ret;
>>>> +
>>>> +    saved_vm_running = runstate_is_running();
>>>> +    vm_stop(RUN_STATE_RESTORE_VM);
>>>> +
>>>> +    f = qemu_fopen(filename, "rb");
>>>> +    if (!f) {
>>>> +        error_setg_file_open(errp, errno, filename);
>>>> +        goto out;
>>>> +    }
>>>> +
>>>> +    migration_incoming_state_new(f);
>>>> +    ret = qemu_loadvm_state(f);
>>>> +    qemu_fclose(f);
>>>> +    migration_incoming_state_destroy();
>>>> +    if (ret < 0) {
>>>> +        error_setg(errp, QERR_IO_ERROR);
>>>> +    }
>>>> +
>>>> +out:
>>>> +    if (saved_vm_running) {
>>>> +        vm_start();
>>>> +    }
>>>
>>> Does it ever happen that you had it running immediately
>>> before you did this command? Somehow you'd have to have loaded the RAM
>>
>> No, we suspend vm before running this command in xen to make sure the
>> condition that you discribed never happen.
> 
> OK, so then if the VM is suspended, what does the:
> 
>     if (saved_vm_running) {
>         vm_start();
>     }
> 
> at the end of your routine do?

+    saved_vm_running = runstate_is_running();
+    vm_stop(RUN_STATE_RESTORE_VM);

It is copied from the other codes in qemu.
I think we should return failure if the vm is running:
if (runstate_is_running()) {
    error_setg(xxx);
    return;
}

Thanks
Wen Congyang

> 
> Dave
> 
> 
>>
>> Thanks
>>      -Xie
>>
>>> at just the right point, and I don't see how that would happen if the guest
>>> was running.
>>>
>>> Dave
>>>
>>>> +}
>>>> +
>>>>  int load_vmstate(const char *name)
>>>>  {
>>>>      BlockDriverState *bs, *bs_vm_state;
>>>> diff --git a/qapi-schema.json b/qapi-schema.json
>>>> index 362c9d8..8cca59d 100644
>>>> --- a/qapi-schema.json
>>>> +++ b/qapi-schema.json
>>>> @@ -4122,3 +4122,17 @@
>>>>  ##
>>>>  { 'enum': 'ReplayMode',
>>>>    'data': [ 'none', 'record', 'play' ] }
>>>> +
>>>> +##
>>>> +# @xen-load-devices-state:
>>>> +#
>>>> +# Load the state of all devices from file. The RAM and the block devices
>>>> +# of the VM are not loaded by this command.
>>>> +#
>>>> +# @filename: the file to load the state of the devices from as binary
>>>> +# data. See xen-save-devices-state.txt for a description of the binary
>>>> +# format.
>>>> +#
>>>> +# Since: 2.7
>>>> +##
>>>> +{ 'command': 'xen-load-devices-state', 'data': {'filename': 'str'} }
>>>> diff --git a/qmp-commands.hx b/qmp-commands.hx
>>>> index b629673..4925702 100644
>>>> --- a/qmp-commands.hx
>>>> +++ b/qmp-commands.hx
>>>> @@ -587,6 +587,33 @@ Example:
>>>>  EQMP
>>>>
>>>>      {
>>>> +        .name       = "xen-load-devices-state",
>>>> +        .args_type  = "filename:F",
>>>> +        .mhandler.cmd_new = qmp_marshal_xen_load_devices_state,
>>>> +    },
>>>> +
>>>> +SQMP
>>>> +xen-load-devices-state
>>>> +----------------------
>>>> +
>>>> +Load the state of all devices from file. The RAM and the block devices
>>>> +of the VM are not loaded by this command.
>>>> +
>>>> +Arguments:
>>>> +
>>>> +- "filename": the file to load the state of the devices from as binary
>>>> +data. See xen-save-devices-state.txt for a description of the binary
>>>> +format.
>>>> +
>>>> +Example:
>>>> +
>>>> +-> { "execute": "xen-load-devices-state",
>>>> +     "arguments": { "filename": "/tmp/resume" } }
>>>> +<- { "return": {} }
>>>> +
>>>> +EQMP
>>>> +
>>>> +    {
>>>>          .name       = "xen-set-global-dirty-log",
>>>>          .args_type  = "enable:b",
>>>>          .mhandler.cmd_new = qmp_marshal_xen_set_global_dirty_log,
>>>> --
>>>> 1.9.3
>>>>
>>>>
>>>>
>>> --
>>> Dr. David Alan Gilbert / address@hidden / Manchester, UK
>>>
>>>
>>> .
>>>
>>
>>
> --
> Dr. David Alan Gilbert / address@hidden / Manchester, UK
> 
> 
> .
>