Re: [PATCH v5 11/13] migration/multifd: Add migration option set packet size.

[Peter Xu]

On Wed, Jul 17, 2024 at 11:59:50AM -0300, Fabiano Rosas wrote:
> Yichen Wang <yichen.wang@bytedance.com> writes:
> 
> > From: Hao Xiang <hao.xiang@linux.dev>
> >
> > During live migration, if the latency between sender and receiver is
> > high and bandwidth is also high (a long and fat pipe), using a bigger
> > packet size can help reduce migration total time. The current multifd
> > packet size is 128 * 4kb. In addition, Intel DSA offloading performs
> > better with a large batch task.
> 
> Last time we measured, mapped-ram also performed slightly better with a
> larger packet size:
> 
>         2 MiB   1 MiB   512 KiB  256 KiB  128 KiB
> AVG(10) 50814   50396     48732    46423    34574
> DEV       736     552       619      473     1430

I wonder whether we could make the new parameter to be pages-per-packet,
rather than in the form of packet-size, just to make our lifes easier for a
possibly static offset[] buffer in the future for the MultiFDPages_t.

With that, we throttle it with MAX_N_PAGES, we can have MultiFDPages_t
statically allocated always with the max buffer. After all, it won't
consume a lot of memory anyway; for MAX_N_PAGES=1K pages it's 8KB per
channel.

-- 
Peter Xu