Re: [Qemu-devel] [PATCH 15/46] Rework loadvm path for subloops

[Dr. David Alan Gilbert]

* Paolo Bonzini (address@hidden) wrote:
> Il 07/07/2014 16:35, Dr. David Alan Gilbert ha scritto:
> >* Paolo Bonzini (address@hidden) wrote:
> >>Il 04/07/2014 19:41, Dr. David Alan Gilbert (git) ha scritto:
> >>>From: "Dr. David Alan Gilbert" <address@hidden>
> >>>
> >>>Postcopy needs to have two migration streams loading concurrently;
> >>>one from memory (with the device state) and the other from the fd
> >>>with the memory transactions.
> >>
> >>Can you explain this?
> >>
> >>I would have though the order is
> >>
> >>    precopy RAM and everything
> >>    prepare postcopy RAM ("sent && dirty" bitmap)
> >>    finish precopy non-RAM
> >>    finish devices
> >>    postcopy RAM
> >>
> >>Why do you need to have all the packaging stuff and a separate memory-based
> >>migration stream for devices?  I'm sure I'm missing something. :)
> >
> >The thing you're missing is the details of 'finish devices'.
> >The device emulation may access guest memory as part of loading it's
> >state, so you can't successfully complete 'finish devices' without
> >having the 'postcopy RAM' available to provide pages.
> 
> I see.  Can you document the flow (preferrably as a reply to this email
> _and_ in docs/ when you send v2 of the code :))?

Yep, will do; I need to go through and check through it before I write the
detail reply.

Dave

> 
> From my cursory read of the code it is something like this on the source:
> 
>     finish precopy non-RAM
>     start RAM postcopy
>     for each device
>         pack up data
>         send it to destination
> 
> and on the destination:
> 
>     while source sends packet
>         pick up packet atomically
>         pass the packet to device loader
>             (while the loader works, userfaultfd does background magic)
> 
> But something is missing still, either some kind of ack is needed between
> device data sends or userfaultfd needs to be able to process device data
> packets.
> 
> Paolo
> 
> >Thus you need to be able to start up 'postcopy RAM' before 'finish devices'
> >has completed, and you can't do that if 'finish devices' is still stuffing
> >data down the fd.
> >
> >Now, if hypothetically you had:
> >  1) A migration format that let you separate out device state so that you
> >could load all the state of the device off the fd without calling the device
> >IO code.
> >  2) All devices were good and didn't touch guest memory while loading their
> >state.
> >
> >then you could avoid this complexity.  However, if you look at how Stefan's
> >BER code tried to do 1 (which I don't do in my way of doing it), it was by
> >using the same trick of stuffing the device data into a dummy memory file
> >to find out the size of the data.   And I'm not convinced (2) will happen
> >this century.
> >
> >>Paolo
> >
> >Dave
> >--
> >Dr. David Alan Gilbert / address@hidden / Manchester, UK
> >
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK