Re: [RFC PATCH 6/7] migration/multifd: Move payload storage out of the channel parameters

[Peter Xu]

On Tue, Jul 16, 2024 at 10:10:25PM +0200, Maciej S. Szmigiero wrote:
> > > > > The comment I removed is slightly misleading to me too, because right 
> > > > > now
> > > > > active_slot contains the data hasn't yet been delivered to multifd, so
> > > > > we're "putting it back to free list" not because of it's free, but 
> > > > > because
> > > > > we know it won't get used until the multifd send thread consumes it
> > > > > (because before that the thread will be busy, and we won't use the 
> > > > > buffer
> > > > > if so in upcoming send()s).
> > > > > 
> > > > > And then when I'm looking at this again, I think maybe it's a slight
> > > > > overkill, and maybe we can still keep the "opaque data" managed by 
> > > > > multifd.
> > > > > One reason might be that I don't expect the "opaque data" payload keep
> > > > > growing at all: it should really be either RAM or device state as I
> > > > > commented elsewhere in a relevant thread, after all it's a thread 
> > > > > model
> > > > > only for migration purpose to move vmstates..
> > > > 
> > > > Some amount of flexibility needs to be baked in. For instance, what
> > > > about the handshake procedure? Don't we want to use multifd threads to
> > > > put some information on the wire for that as well?
> > > 
> > > Is this an orthogonal question?
> > 
> > I don't think so. You say the payload data should be either RAM or
> > device state. I'm asking what other types of data do we want the multifd
> > channel to transmit and suggesting we need to allow room for the
> > addition of that, whatever it is. One thing that comes to mind that is
> > neither RAM or device state is some form of handshake or capabilities
> > negotiation.
> 
> The RFC version of my multifd device state transfer patch set introduced
> a new migration channel header (by Avihai) for clean and extensible
> migration channel handshaking but people didn't like so it was removed in v1.

Hmm, I'm not sure this is relevant to the context of discussion here, but I
confess I didn't notice the per-channel header thing in the previous RFC
series.  Link is here:

https://lore.kernel.org/r/636cec92eb801f13ba893de79d4872f5d8342097.1713269378.git.maciej.szmigiero@oracle.com

Maciej, if you want, you can split that out of the seriess. So far it looks
like a good thing with/without how VFIO tackles it.

Thanks,

-- 
Peter Xu