For now, we still didn't support live memory snapshot, we have discussed
a scheme which based on userfaultfd long time ago.
You can find the discussion by the follow link:
https://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg01779.html
The scheme is based on userfaultfd's write-protect capability.
The userfaultfd write protection feature is available here:
http://www.spinics.net/lists/linux-mm/msg97422.html
The process of this live memory scheme is like bellow:
1. Pause VM
2. Enable write-protect fault notification by using userfaultfd to
mark VM's memory to write-protect (readonly).
3. Save VM's static state (here is device state) to snapshot file
4. Resume VM, VM is going to run.
5. Snapshot thread begins to save VM's live state (here is RAM) into
snapshot file.
6. During this time, all the actions of writing VM's memory will be blocked
by kernel, and kernel will wakeup the fault treating thread in qemu to
process this write-protect fault. The fault treating thread will deliver this
page's address to snapshot thread.
7. snapshot thread gets this address, save this page into snasphot file,
and then remove the write-protect by using userfaultfd API, after that,
the actions of writing will be recovered.
8. Repeat step 5~7 until all VM's memory is saved to snapshot file
Compared with the feature of 'migrate VM's state to file',
the main difference for live memory snapshot is it has little time delay for
catching VM's state. It just captures the VM's state while got users snapshot
command, just like take a photo of VM's state.
For now, we only support tcg accelerator, since userfaultfd is not supporting
tracking write faults for KVM.
Usage:
1. Take a snapshot
#x86_64-softmmu/qemu-system-x86_64 -machine pc-i440fx-2.5,accel=tcg,usb=off
-drive
file=/mnt/windows/win7_install.qcow2.bak,if=none,id=drive-ide0-0-1,format=qcow2,cache=none
-device ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -vnc :7 -m
8192 -smp 1 -netdev tap,id=bn0 -device virtio-net-pci,id=net-pci0,netdev=bn0
--monitor stdio
Issue snapshot command:
(qemu)migrate -d file:/home/Snapshot
2. Revert to the snapshot
#x86_64-softmmu/qemu-system-x86_64 -machine pc-i440fx-2.5,accel=tcg,usb=off
-drive
file=/mnt/windows/win7_install.qcow2.bak,if=none,id=drive-ide0-0-1,format=qcow2,cache=none
-device ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -vnc :7 -m
8192 -smp 1 -netdev tap,id=bn0 -device virtio-net-pci,id=net-pci0,netdev=bn0
--monitor stdio -incoming file:/home/Snapshot
NOTE:
The userfaultfd write protection feature does not support THP for now,
Before taking snapshot, please disable THP by:
echo never > /sys/kernel/mm/transparent_hugepage/enabled
TODO:
- Reduce the influence for VM while taking snapshot
zhanghailiang (13):
postcopy/migration: Split fault related state into struct
UserfaultState
migration: Allow the migrate command to work on file: urls
migration: Allow -incoming to work on file: urls
migration: Create a snapshot thread to realize saving memory snapshot
migration: implement initialization work for snapshot
QEMUSizedBuffer: Introduce two help functions for qsb
savevm: Split qemu_savevm_state_complete_precopy() into two helper
functions
snapshot: Save VM's device state into snapshot file
migration/postcopy-ram: fix some helper functions to support
userfaultfd write-protect
snapshot: Enable the write-protect notification capability for VM's
RAM
snapshot/migration: Save VM's RAM into snapshot file
migration/ram: Fix some helper functions' parameter to use
PageSearchStatus
snapshot: Remove page's write-protect and copy the content during
setup stage
include/migration/migration.h | 41 +++++--
include/migration/postcopy-ram.h | 9 +-
include/migration/qemu-file.h | 3 +-
include/qemu/typedefs.h | 1 +
include/sysemu/sysemu.h | 3 +
linux-headers/linux/userfaultfd.h | 21 +++-
migration/fd.c | 51 ++++++++-
migration/migration.c | 101 ++++++++++++++++-
migration/postcopy-ram.c | 229 ++++++++++++++++++++++++++++----------
migration/qemu-file-buf.c | 61 ++++++++++
migration/ram.c | 104 ++++++++++++-----
migration/savevm.c | 90 ++++++++++++---
trace-events | 1 +
13 files changed, 587 insertions(+), 128 deletions(-)
--
1.8.3.1