[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v3 00/24] Multifd 🔀 device state transfer support with VFIO c
|
From: |
Yanghang Liu |
|
Subject: |
Re: [PATCH v3 00/24] Multifd 🔀 device state transfer support with VFIO consumer |
|
Date: |
Thu, 19 Dec 2024 21:00:51 +0800 |
Sorry for the inconvenience. Let me try to re-send my data via my Gmail client.
Test environment:
Host : Dell 7625
CPU : EPYC-Genoa
VM config : 4 vCPU, 8G memory
Network Device: MT2910
Test report:
+------------------+---------------+----------------+
| multifd=0 | outgoing migration |
+------------------+---------------+----------------+
| VF(s) number | 1 | 4 |
| Time elapsed | 10194 ms | 10650 ms |
| Memory processed | 903.911 MiB | 783.698 MiB |
| Memory bandwidth | 108.722 MiB/s | 101.978 MiB/s |
| Iteration | 4 | 6 |
| Normal data | 881.297 MiB | 747.613 MiB |
| Total downtime | 358ms | 518ms |
| Setup time | 52ms | 450ms |
+------------------+---------------+----------------+
+------------------+---------------+----------------+
| multifd=0 | incoming migration |
+------------------+---------------+----------------+
| VF(s) number | 1 | 4 |
| Time elapsed | 10161 ms | 10569 ms |
| Memory processed | 903.881 MiB | 785.400 MiB |
| Memory bandwidth | 107.952 MiB/s | 100.512 MiB/s |
| Iteration | 4 | 7 |
| Normal data | 881.262 MiB | 749.297 MiB |
| Total downtime | 315ms | 513ms |
| Setup time | 47ms | 414ms |
+------------------+---------------+----------------+
+------------------+---------------+---------------+
| multifd=1 | outgoing migration |
+------------------+---------------+---------------+
| VF(s) number | 1 | 1 |
| Channel | 4 | 5 |
| Time elapsed | 10962 ms | 10071 ms |
| Memory processed | 908.968 MiB | 908.424 MiB |
| Memory bandwidth | 108.378 MiB/s | 110.109 MiB/s |
| Iteration | 4 | 4 |
| Normal data | 882.852 MiB | 882.566 MiB |
| Total downtime | 318ms | 255ms |
| Setup time | 54ms | 43ms |
+------------------+---------------+---------------+
+------------------+---------------+----------------+
| multifd=1 | incoming migration |
+------------------+---------------+----------------+
| VF(s) number | 1 | 1 |
| Channel | 4 | 5 |
| Time elapsed | 10064ms | 10072 ms |
| Memory processed | 909.786 MiB | 923.746 MiB |
| Memory bandwidth | 109.997 MiB/s | 111.308 MiB/s |
| Iteration | 4 | 4 |
| Normal data | 883.664 MiB | 897.848 MiB |
| Total downtime | 313ms | 328ms |
| Setup time | 46ms | 47ms |
+------------------+---------------+----------------+
+------------------+---------------+----------------+
| multifd=1 | outgoing migration |
+------------------+---------------+----------------+
| VF(s) number | 4 | 4 |
| Channel | 8 | 16 |
| Time elapsed | 10805 ms | 10943 ms |
| Memory processed | 786.334 MiB | 784.926 MiB |
| Memory bandwidth | 109.062 MiB/s | 108.610 MiB/s |
| Iteration | 5 | 7 |
| Normal data | 746.758 MiB | 744.938 MiB |
| Total downtime | 344 ms | 335ms |
| Setup time | 445 ms | 463ms |
+------------------+---------------+----------------+
+------------------+---------------+------------------+
| multifd=1 | incoming migration |
+------------------+---------------+------------------+
| VF(s) number | 4 | 4 |
| Channel | 8 | 16 |
| Time elapsed | 10126 ms | 9941 ms |
| Memory processed | 791.308 MiB | 779.560 MiB |
| Memory bandwidth | 108.876 MiB/s | 110.170 MiB/s |
| Iteration | 7 | 5 |
| Normal data | 751.672 MiB | 739.680 MiB |
| Total downtime | 304 ms | 309ms |
| Setup time | 442 ms | 446ms |
+------------------+---------------+------------------+
Best Regards,
Yanghang Liu
On Thu, Dec 19, 2024 at 4:53 PM Cédric Le Goater <clg@redhat.com> wrote:
>
> Hello Yanghang
>
> On 12/19/24 08:55, Yanghang Liu wrote:
> > FYI. The following data comes from the first ping-pong mlx VF
> > migration after rebooting the host.
> >
> >
> > 1. Test for multifd=0:
> >
> > 1.1 Outgoing migration:
> > VF number: 1 VF 4 VF
> > Time elapsed: 10194 ms 10650 ms
> > Memory processed: 903.911 MiB 783.698 MiB
> > Memory bandwidth: 108.722 MiB/s 101.978 MiB/s
> > Iteration: 4 6
> > Normal data: 881.297 MiB 747.613 MiB
> > Total downtime 358ms 518ms
> > Setup time 52ms 450ms
> >
> > 1.2 In coming migration:
> > VF number: 1 VF 4 VF
> > Time elapsed: 10161 ms 10569 ms
> > Memory processed: 903.881 MiB 785.400 MiB
> > Memory bandwidth: 107.952 MiB/s 100.512 MiB/s
> > Iteration: 4 7
> > Normal data: 881.262 MiB 749.297 MiB
> > Total downtime 315ms 513ms
> > Setup time 47ms 414ms
> >
> >
> > 2. Test for multifd=1:
> >
> > 2.1 Outgoing migration:
> > VF number 1 VF 1 VF
> > Channel number 4 5
> > Time elapsed: 10962 ms 10071 ms
> > Memory processed: 908.968 MiB 908.424 MiB
> > Memory bandwidth: 108.378 MiB/s 110.109 MiB/s
> > Iteration: 4
> > 4
> > Normal data: 882.852 MiB 882.566 MiB
> > Total downtime 318ms 255ms
> > Setup time 54ms 43ms
> >
> >
> > VF number 4 VFs 4 VFs
> > Channel number 8 16
> > Time elapsed: 10805 ms 10943 ms
> > Setup time 445 ms 463ms
> > Memory processed: 786.334 MiB 784.926 MiB
> > Memory bandwidth 109.062 MiB/s 108.610 MiB/s
> > Iteration: 5 7
> > Normal data: 746.758 MiB 744.938 MiB
> > Total downtime 344 ms 335ms
> >
> >
> > 2.2 Incoming migration:
> > VF number 1 VF 1 VF
> > Channel number 4 5
> > Time elapsed: 10064ms 10072 ms
> > Memory processed: 909.786 MiB 923.746 MiB
> > Memory bandwidth: 109.997 MiB/s 111.308 MiB/s
> > Iteration: 4 4
> > Normal data: 883.664 MiB 897.848 MiB
> > Total downtime 313ms 328ms
> > Setup time 46ms 47ms
> >
> > VF number 4 VFs 4 VFs
> > Channel number 8 16
> > Time elapsed: 10126 ms 9941 ms
> > Memory processed: 791.308 MiB 779.560 MiB
> > Memory bandwidth: 108.876 MiB/s 110.170 MiB/s
> > Iteration: 7 5
> > Normal data: 751.672 MiB 739.680 MiB
> > Total downtime 304 ms 309ms
> > Setup time 442 ms 446ms
> >
>
> This is difficult to read. Could you please resend with a fixed
> indentation ?
>
> We would need more information on the host and vm config too.
>
> Thanks,
>
> C.
>