Re: [Qemu-devel] [PATCH v2 1/1] Introduce "xen-load-devices-state"

[Dr. David Alan Gilbert]

* Changlong Xie (address@hidden) wrote:
> On 03/23/2016 05:41 PM, Dr. David Alan Gilbert wrote:
> >* Wen Congyang (address@hidden) wrote:
> >>On 03/23/2016 04:56 PM, Dr. David Alan Gilbert wrote:
> >>>* Changlong Xie (address@hidden) wrote:
> >>>>On 03/22/2016 08:22 PM, Dr. David Alan Gilbert wrote:
> >>>>>* Changlong Xie (address@hidden) wrote:
> >>>>>>From: Wen Congyang <address@hidden>
> >>>>>>
> >>>>>>Introduce a "xen-load-devices-state" QAPI command that can be used to
> >>>>>>load the state of all devices, but not the RAM or the block devices of
> >>>>>>the VM.
> >>>>>>
> >>>>>>We only have hmp commands savevm/loadvm, and qmp commands
> >>>>>>xen-save-devices-state.
> >>>>>
> >>>>>Can you explain on Xen how the RAM actually gets loaded?
> >>>>
> >>>>Xen use xc(xen toolstack) to do RAM restore/save
> >>>>
> >>>>>
> >>>>>>
> >>>>>>We use this new command for COLO:
> >>>>>>1. suspend both primary vm and secondary vm
> >>>>>>2. sync the state
> >>>>>>3. resume both primary vm and secondary vm
> >>>>>>
> >>>>>>In such case, we need to update all devices' state in any time.
> >>>>>>
> >>>>>>Signed-off-by: Wen Congyang <address@hidden>
> >>>>>>Signed-off-by: Changlong Xie <address@hidden>
> >>>>>>---
> >>>>>>  migration/savevm.c | 36 ++++++++++++++++++++++++++++++++++++
> >>>>>>  qapi-schema.json   | 14 ++++++++++++++
> >>>>>>  qmp-commands.hx    | 27 +++++++++++++++++++++++++++
> >>>>>>  3 files changed, 77 insertions(+)
> >>>>>>
> >>>>>>diff --git a/migration/savevm.c b/migration/savevm.c
> >>>>>>index 96e7db5..aaead12 100644
> >>>>>>--- a/migration/savevm.c
> >>>>>>+++ b/migration/savevm.c
> >>>>>>@@ -50,6 +50,7 @@
> >>>>>>  #include "qemu/iov.h"
> >>>>>>  #include "block/snapshot.h"
> >>>>>>  #include "block/qapi.h"
> >>>>>>+#include "hw/xen/xen.h"
> >>>>>>
> >>>>>>
> >>>>>>  #ifndef ETH_P_RARP
> >>>>>>@@ -1768,6 +1769,12 @@ qemu_loadvm_section_start_full(QEMUFile *f, 
> >>>>>>MigrationIncomingState *mis)
> >>>>>>          return -EINVAL;
> >>>>>>      }
> >>>>>>
> >>>>>>+    /* Validate if it is a device's state */
> >>>>>>+    if (xen_enabled() && se->is_ram) {
> >>>>>>+        error_report("loadvm: %s RAM loading not allowed on Xen", 
> >>>>>>idstr);
> >>>>>>+        return -EINVAL;
> >>>>>>+    }
> >>>>>>+
> >>>>>>      /* Add entry */
> >>>>>>      le = g_malloc0(sizeof(*le));
> >>>>>>
> >>>>>>@@ -2077,6 +2084,35 @@ void qmp_xen_save_devices_state(const char 
> >>>>>>*filename, Error **errp)
> >>>>>>      }
> >>>>>>  }
> >>>>>>
> >>>>>>+void qmp_xen_load_devices_state(const char *filename, Error **errp)
> >>>>>>+{
> >>>>>>+    QEMUFile *f;
> >>>>>>+    int saved_vm_running;
> >>>>>>+    int ret;
> >>>>>>+
> >>>>>>+    saved_vm_running = runstate_is_running();
> >>>>>>+    vm_stop(RUN_STATE_RESTORE_VM);
> >>>>>>+
> >>>>>>+    f = qemu_fopen(filename, "rb");
> >>>>>>+    if (!f) {
> >>>>>>+        error_setg_file_open(errp, errno, filename);
> >>>>>>+        goto out;
> >>>>>>+    }
> >>>>>>+
> >>>>>>+    migration_incoming_state_new(f);
> >>>>>>+    ret = qemu_loadvm_state(f);
> >>>>>>+    qemu_fclose(f);
> >>>>>>+    migration_incoming_state_destroy();
> >>>>>>+    if (ret < 0) {
> >>>>>>+        error_setg(errp, QERR_IO_ERROR);
> >>>>>>+    }
> >>>>>>+
> >>>>>>+out:
> >>>>>>+    if (saved_vm_running) {
> >>>>>>+        vm_start();
> >>>>>>+    }
> >>>>>
> >>>>>Does it ever happen that you had it running immediately
> >>>>>before you did this command? Somehow you'd have to have loaded the RAM
> >>>>
> >>>>No, we suspend vm before running this command in xen to make sure the
> >>>>condition that you discribed never happen.
> >>>
> >>>OK, so then if the VM is suspended, what does the:
> >>>
> >>>     if (saved_vm_running) {
> >>>         vm_start();
> >>>     }
> >>>
> >>>at the end of your routine do?
> >>
> >>+    saved_vm_running = runstate_is_running();
> >>+    vm_stop(RUN_STATE_RESTORE_VM);
> >>
> >>It is copied from the other codes in qemu.
> >>I think we should return failure if the vm is running:
> >>if (runstate_is_running()) {
> >>     error_setg(xxx);
> >>     return;
> >>}
> >
> >OK, yes; and add a comment to explain.
> 
> +    if (runstate_is_running()) {
> +        error_setg(errp, "Cannot update device state while vm is running");
> +        return;
> +    }

No, a *comment*; e.g.


    /* Guest must be paused before loading the device state; the RAM state will
     * already have been loaded using xc.
     */
    if (runstate_is_running()) {
        error_setg(errp, "Cannot update device state while vm is running");
        return;
    }

Dave

> Will fix in next version.
> 
> Thanks
>       -Xie
> >
> >Dave
> >
> >>
> >>Thanks
> >>Wen Congyang
> >>
> >>>
> >>>Dave
> >>>
> >>>
> >>>>
> >>>>Thanks
> >>>>  -Xie
> >>>>
> >>>>>at just the right point, and I don't see how that would happen if the 
> >>>>>guest
> >>>>>was running.
> >>>>>
> >>>>>Dave
> >>>>>
> >>>>>>+}
> >>>>>>+
> >>>>>>  int load_vmstate(const char *name)
> >>>>>>  {
> >>>>>>      BlockDriverState *bs, *bs_vm_state;
> >>>>>>diff --git a/qapi-schema.json b/qapi-schema.json
> >>>>>>index 362c9d8..8cca59d 100644
> >>>>>>--- a/qapi-schema.json
> >>>>>>+++ b/qapi-schema.json
> >>>>>>@@ -4122,3 +4122,17 @@
> >>>>>>  ##
> >>>>>>  { 'enum': 'ReplayMode',
> >>>>>>    'data': [ 'none', 'record', 'play' ] }
> >>>>>>+
> >>>>>>+##
> >>>>>>+# @xen-load-devices-state:
> >>>>>>+#
> >>>>>>+# Load the state of all devices from file. The RAM and the block 
> >>>>>>devices
> >>>>>>+# of the VM are not loaded by this command.
> >>>>>>+#
> >>>>>>+# @filename: the file to load the state of the devices from as binary
> >>>>>>+# data. See xen-save-devices-state.txt for a description of the binary
> >>>>>>+# format.
> >>>>>>+#
> >>>>>>+# Since: 2.7
> >>>>>>+##
> >>>>>>+{ 'command': 'xen-load-devices-state', 'data': {'filename': 'str'} }
> >>>>>>diff --git a/qmp-commands.hx b/qmp-commands.hx
> >>>>>>index b629673..4925702 100644
> >>>>>>--- a/qmp-commands.hx
> >>>>>>+++ b/qmp-commands.hx
> >>>>>>@@ -587,6 +587,33 @@ Example:
> >>>>>>  EQMP
> >>>>>>
> >>>>>>      {
> >>>>>>+        .name       = "xen-load-devices-state",
> >>>>>>+        .args_type  = "filename:F",
> >>>>>>+        .mhandler.cmd_new = qmp_marshal_xen_load_devices_state,
> >>>>>>+    },
> >>>>>>+
> >>>>>>+SQMP
> >>>>>>+xen-load-devices-state
> >>>>>>+----------------------
> >>>>>>+
> >>>>>>+Load the state of all devices from file. The RAM and the block devices
> >>>>>>+of the VM are not loaded by this command.
> >>>>>>+
> >>>>>>+Arguments:
> >>>>>>+
> >>>>>>+- "filename": the file to load the state of the devices from as binary
> >>>>>>+data. See xen-save-devices-state.txt for a description of the binary
> >>>>>>+format.
> >>>>>>+
> >>>>>>+Example:
> >>>>>>+
> >>>>>>+-> { "execute": "xen-load-devices-state",
> >>>>>>+     "arguments": { "filename": "/tmp/resume" } }
> >>>>>>+<- { "return": {} }
> >>>>>>+
> >>>>>>+EQMP
> >>>>>>+
> >>>>>>+    {
> >>>>>>          .name       = "xen-set-global-dirty-log",
> >>>>>>          .args_type  = "enable:b",
> >>>>>>          .mhandler.cmd_new = qmp_marshal_xen_set_global_dirty_log,
> >>>>>>--
> >>>>>>1.9.3
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>--
> >>>>>Dr. David Alan Gilbert / address@hidden / Manchester, UK
> >>>>>
> >>>>>
> >>>>>.
> >>>>>
> >>>>
> >>>>
> >>>--
> >>>Dr. David Alan Gilbert / address@hidden / Manchester, UK
> >>>
> >>>
> >>>.
> >>>
> >>
> >>
> >>
> >--
> >Dr. David Alan Gilbert / address@hidden / Manchester, UK
> >
> >
> >.
> >
> 
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK