[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling
From: |
Gonglei (Arei) |
Subject: |
RE: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling |
Date: |
Tue, 28 May 2024 09:06:04 +0000 |
Hi Peter,
> -----Original Message-----
> From: Peter Xu [mailto:peterx@redhat.com]
> Sent: Wednesday, May 22, 2024 6:15 AM
> To: Yu Zhang <yu.zhang@ionos.com>
> Cc: Michael Galaxy <mgalaxy@akamai.com>; Jinpu Wang
> <jinpu.wang@ionos.com>; Elmar Gerdes <elmar.gerdes@ionos.com>;
> zhengchuan <zhengchuan@huawei.com>; Gonglei (Arei)
> <arei.gonglei@huawei.com>; Daniel P. Berrangé <berrange@redhat.com>;
> Markus Armbruster <armbru@redhat.com>; Zhijian Li (Fujitsu)
> <lizhijian@fujitsu.com>; qemu-devel@nongnu.org; Yuval Shaia
> <yuval.shaia.ml@gmail.com>; Kevin Wolf <kwolf@redhat.com>; Prasanna
> Kumar Kalever <prasanna.kalever@redhat.com>; Cornelia Huck
> <cohuck@redhat.com>; Michael Roth <michael.roth@amd.com>; Prasanna
> Kumar Kalever <prasanna4324@gmail.com>; Paolo Bonzini
> <pbonzini@redhat.com>; qemu-block@nongnu.org; devel@lists.libvirt.org;
> Hanna Reitz <hreitz@redhat.com>; Michael S. Tsirkin <mst@redhat.com>;
> Thomas Huth <thuth@redhat.com>; Eric Blake <eblake@redhat.com>; Song
> Gao <gaosong@loongson.cn>; Marc-André Lureau
> <marcandre.lureau@redhat.com>; Alex Bennée <alex.bennee@linaro.org>;
> Wainer dos Santos Moschetta <wainersm@redhat.com>; Beraldo Leal
> <bleal@redhat.com>; Pannengyuan <pannengyuan@huawei.com>;
> Xiexiangyou <xiexiangyou@huawei.com>; Fabiano Rosas <farosas@suse.de>
> Subject: Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling
>
> On Fri, May 17, 2024 at 03:01:59PM +0200, Yu Zhang wrote:
> > Hello Michael and Peter,
>
> Hi,
>
> >
> > Exactly, not so compelling, as I did it first only on servers widely
> > used for production in our data center. The network adapters are
> >
> > Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720
> > 2-port Gigabit Ethernet PCIe
>
> Hmm... I definitely thinks Jinpu's Mellanox ConnectX-6 looks more reasonable.
>
> https://lore.kernel.org/qemu-devel/CAMGffEn-DKpMZ4tA71MJYdyemg0Zda15
> wVAqk81vXtKzx-LfJQ@mail.gmail.com/
>
> Appreciate a lot for everyone helping on the testings.
>
> > InfiniBand controller: Mellanox Technologies MT27800 Family
> > [ConnectX-5]
> >
> > which doesn't meet our purpose. I can choose RDMA or TCP for VM
> > migration. RDMA traffic is through InfiniBand and TCP through Ethernet
> > on these two hosts. One is standby while the other is active.
> >
> > Now I'll try on a server with more recent Ethernet and InfiniBand
> > network adapters. One of them has:
> > BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller (rev 01)
> >
> > The comparison between RDMA and TCP on the same NIC could make more
> sense.
>
> It looks to me NICs are powerful now, but again as I mentioned I don't think
> it's
> a reason we need to deprecate rdma, especially if QEMU's rdma migration has
> the chance to be refactored using rsocket.
>
> Is there anyone who started looking into that direction? Would it make sense
> we start some PoC now?
>
My team has finished the PoC refactoring which works well.
Progress:
1. Implement io/channel-rdma.c,
2. Add unit test tests/unit/test-io-channel-rdma.c and verifying it is
successful,
3. Remove the original code from migration/rdma.c,
4. Rewrite the rdma_start_outgoing_migration and rdma_start_incoming_migration
logic,
5. Remove all rdma_xxx functions from migration/ram.c. (to prevent RDMA live
migration from polluting the core logic of live migration),
6. The soft-RoCE implemented by software is used to test the RDMA live
migration. It's successful.
We will be submit the patchset later.
Regards,
-Gonglei
> Thanks,
>
> --
> Peter Xu
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, (continued)
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Peter Xu, 2024/05/06
- RE: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Gonglei (Arei), 2024/05/06
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Peter Xu, 2024/05/07
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Zheng Chuan, 2024/05/09
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Peter Xu, 2024/05/09
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Jinpu Wang, 2024/05/13
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Yu Zhang, 2024/05/14
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Michael Galaxy, 2024/05/16
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Yu Zhang, 2024/05/17
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Peter Xu, 2024/05/21
- RE: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling,
Gonglei (Arei) <=
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Jinpu Wang, 2024/05/28
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Peter Xu, 2024/05/28
- RE: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Gonglei (Arei), 2024/05/28
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Jinpu Wang, 2024/05/29
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Greg Sword, 2024/05/29
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Jinpu Wang, 2024/05/29
- RE: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Gonglei (Arei), 2024/05/29
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Jinpu Wang, 2024/05/29
- RE: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Gonglei (Arei), 2024/05/29
- Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling, Jinpu Wang, 2024/05/29