Re: [Qemu-devel] [RFC PATCH 0/4] ARM/ARM64 fixes for live memory snapshot based on userfaultfd

[Dr. David Alan Gilbert]

* Christian Pinto (address@hidden) wrote:
> This patch series introduces a set of fixes to the previous work proposed by
> Hailiang Zhang to enable in QEMU live memory snapshot based
> on userfaultfd. See discussion here:
> http://www.mail-archive.com/address@hidden/msg393118.html

Thanks for posting this,

> These patches apply on top of: 
> https://github.com/coloft/qemu/tree/snapshot-v2
> that is the latest version of Hailiang's work, and rely on the latest work on
> userfaultfd available on Andrea Arcangeli's Linux kernel tree:
> https://git.kernel.org/cgit/linux/kernel/git/andrea/aa.git/log/?h=userfault
> 
> The original work was mainly tested on x86 tcg machines and was not working
> ARM/ARM64 tcg.
> The fixes presented in this series enable the live memory snapshot
> to work for ARM64 tcg guests running on top of an ARM64 host.
> 
> The main problems encountered were:
>     - QEMU uses for ARM a memory page size of 1KB. Even though this size is 
> not
>       supported by the Linux kernel, is is kept for backward compatibility
>       with older ARM CPU MMUs. Initial work was write-unprotecting pages with
>       a granularity not always aligned with host page size, causing 
> userfaultfd
>       to fail.

Yes, Power similarly has a 4kb size for the target page size even though
the host kernel is normally a large page size.

>     - The VM execution was resumed right before the status of the migration
>       was switched from MIGRATION_STATUS_SETUP to MIGRATION_STATUS_ACTIVE.
>       This was causing again the VM to trigger a "Bus error", due to wrong
>       status of some memory pages.
>     - When unprotecting a memory page the flag
>       UFFDIO_WRITEPROTECT_MODE_DONTWAKE was used. This way, after a page is
>       copied into snapshot file, the virtual machine execution is not resumed.
> 
> 
> To test the patches on an ARM64 host, boot an ARM64 tcg machine:
> 
> qemu-system-aarch64 -machine virt,accel=tcg -cpu cortex-a57\
>         -m 256 -kernel Image \
>         -initrd rootfs.cpio.gz \
>         -append "earlyprintk rw console=ttyAMA0" \
>         -net nic -net user \
>         -nographic -serial pty -monitor stdio
> 
> start migration from QEMU monitor:
> 
>     (qemu) migrate file:/root/test_snapshot
> 
> 
> resume VM form snapshot:
> 
> qemu-system-aarch64 -machine virt,accel=tcg -cpu cortex-a57\
>         -m 256 -kernel Image \
>         -initrd rootfs.cpio.gz \
>         -append "earlyprintk rw console=ttyAMA0" \
>         -net nic -net user \
>         -nographic -serial stdio -monitor pty \
>         -incoming file:/root/test_snapshot

Nice, what's your use case and how are you dealing with storage?

Dave

> Christian Pinto (4):
>   migration/postcopy-ram: check pagefault flags in userfaultfd thread
>   migration/ram: Fix for ARM/ARM64 page size
>   migration: snapshot thread
>   migration/postcopy-ram: ram_set_pages_wp fix
> 
>  migration/migration.c    |  9 +++++----
>  migration/postcopy-ram.c | 25 ++++++++-----------------
>  migration/ram.c          | 18 ++++++++++++++----
>  3 files changed, 27 insertions(+), 25 deletions(-)
> 
> -- 
> 2.11.0
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK