Re: [Qemu-devel] [PATCH 0/2][RFC] postcopy migration: Linux char device for postcopy

[Andrea Arcangeli]

On Thu, Dec 29, 2011 at 06:01:45PM +0200, Avi Kivity wrote:
> On 12/29/2011 06:00 PM, Avi Kivity wrote:
> > The NFS client has exactly the same issue, if you mount it with the intr
> > option.  In fact you could use the NFS client as a trivial umem/cuse
> > prototype.
> 
> Actually, NFS can return SIGBUS, it doesn't care about restarting daemons.

During KVMForum I suggested to a few people that it could be done
entirely in userland with PROT_NONE. So the problem is if we do it in
userland with the current functionality you'll run out of VMAs and
slowdown performance too much.

But all you need is the ability to map single pages in the address
space. The only special requirement is that a new vma must not be
created during the map operation. It'd be very similar to
remap_file_pages for MAP_SHARED, it also was created to avoid having
to create new vmas on a large MAP_SHARED mapping and no other reason
at all. In our case we deal with a large MAP_ANONYMOUS mapping and we
must alter the pte without creating new vmas but the problem is very
similar to remap_file_pages.

Qemu in the dst node can do:

        mmap(MAP_ANONYMOUS....)
        fault_area_prepare(start, end, signalnr)

prepare_fault_area will map the range with the magic pte.

Then when the signalnr fires, you do:

     send(givemepageX)
     recv(&tmpaddr_aligned, PAGE_SIZE,...);
     fault_area_map(final_dest_aligned, tmpaddr_aligned, size)

map_fault_area will check the pgprot of the two vmas mapping
final_dest_aligned and tmpaddr_aligned have the same vma->vm_pgprot
and various other vma bits, and if all ok, it'll just copy the pte
from tmpaddr_aligned, to final_dest_aligned and it'll update the
page->index. It can fail if the page is shared to avoid dealing with
the non-linearity of the page mapped in multiple vmas.

You basically need a bypass to avoid altering the pgprot of the vma,
and enter into the pte a "magic" thing that fires signal handlers
if accessed, without having to create new vmas. gup/gup_fast and stuff
should just always fallback into handle_mm_fault when encountering such a
thing, so returning failure as if gup_fast was run on a address beyond
the end of the i_size in the MAP_SHARED case.

THP already works on /dev/zero mmaps as long as it's a MAP_PRIVATE,
KSM should work too but I doubt anybody tested it on MAP_PRIVATE of
/dev/zero.

The device driver provides an advantage in being self contained but I
doubt it's simpler. I suppose after migration is complete you'll still
switch the vma back to regular anonymous vma so leading to the same
result?

The patch 2/2 is small and self contained so it's quite attractive, I
didn't see patch 1/2, was it posted?

Thanks,
Andrea