Re: [Qemu-devel] [RFC 00/13] Live memory snapshot based on userfaultfd

[Andrea Arcangeli]

Hello everyone,

I've an aa.git tree uptodate on the master & userfault branch (master
includes other pending VM stuff, userfault branch only contains
userfault enhancements):

https://git.kernel.org/cgit/linux/kernel/git/andrea/aa.git/log/?h=userfault

I didn't have time to test KVM live memory snapshot on it yet as I'm
still working to improve it. Did anybody test it? However I'd be happy
to take any bugreports and quickly solve anything that isn't working
right with the shadow MMU.

I got positive report already for another usage of the uffd WP support:

https://medium.com/@MartinCracauer/generational-garbage-collection-write-barriers-write-protection-and-userfaultfd-2-8b0e796b8f7f

The last few things I'm working on to finish the WP support are:

1) pte_swp_mkuffd_wp equivalent of pte_swp_mksoft_dirty to mark in a
   vma->vm_flags with VM_UFFD_WP set, which swap entries were
   generated while the pte was wrprotected.

2) to avoid all false positives the equivalent of pte_mksoft_dirty is
   needed too... and that requires spare software bits on the pte
   which are available on x86. I considered also taking over the
   soft_dirty bit but then you couldn't do checkpoint restore of a
   JIT/to-native compiler that uses uffd WP support so it wasn't
   ideal. Perhaps it would be ok as an incremental patch to make the
   two options mutually exclusive to defer the arch changes that
   pte_mkuffd_wp would require for later.

3) prevent UFFDIO_ZEROPAGE if registering WP|MISSING or trigger a
   cow in userfaultfd_writeprotect.

4) WP selftest

In theory things should work ok already if the userland code is
tolerant against false positives through swap and after fork() and
KSM. For an usage like snapshotting false positives shouldn't be an
issue (it'll just run slower if you swap in the worst case), and point
3) above also isn't an issue because it's going to register into uffd
with WP only.

The current status includes:

1) WP support for anon (with false positives.. work in progress)

2) MISSING support for tmpfs and hugetlbfs

3) non cooperative support

Thanks,
Andrea