Re: [PATCH v4 3/4] migration: zero-copy flush only at the end of bitmap scanning

[Peter Xu]

On Mon, Jun 20, 2022 at 11:23:53AM +0200, Juan Quintela wrote:
> Once discussed this, what I asked in the past is that you are having too
> much dirty memory on zero_copy.  When you have a Multiterabyte guest, in
> a single round you have a "potentially" dirty memory on each channel of:
> 
>    total_amount_memory / number of channels.
> 
> In a Multiterabyte guest, this is going to be more that probably in the
> dozens of gigabytes.  As far as I know there is no card/driver that will
> benefit for so many pages in zero_copy, and kernel will move to
> synchronous copy at some point.  (In older threads, daniel showed how to
> test for this case).

I was wondering whether the kernel needs to cache a lot of messages for
zero copy if we don't flush it for a long time, as recvmsg(MSG_ERRQUEUE)
seems to be fetching one message from the kernel one at a time.  And,
whether that queue has a limit in length or something.

Does it mean that when the kernel could have cached enough of these
messages then it'll fallback to the no-zero-copy mode?  And probably that's
the way how kernel protects itself from using too much buffer for the error
msgs?

This reminded me - Leo, have you considered adding the patch altogether to
detect the "fallback to non-zero-copy" condition?  Because when with it and
when the fallback happens at some point (e.g. when the guest memory is
larger than some value) we'll know.

Thanks,

-- 
Peter Xu