[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v5 09/10] migration: poll the cm event while wai
From: |
858585 jemmy |
Subject: |
Re: [Qemu-devel] [PATCH v5 09/10] migration: poll the cm event while wait RDMA work request completion |
Date: |
Thu, 14 Jun 2018 10:42:14 +0800 |
On Wed, Jun 13, 2018 at 10:24 PM, Dr. David Alan Gilbert
<address@hidden> wrote:
> * Lidong Chen (address@hidden) wrote:
>> If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function
>> maybe loop forever. so we should also poll the cm event fd, and when
>> receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL,
>> we consider some error happened.
>>
>> Signed-off-by: Lidong Chen <address@hidden>
>
> Was there a reply which explained/pointed to docs for cm_event?
https://linux.die.net/man/3/rdma_get_cm_event
> Or a Review-by from one of the Infiniband people would be fine.
yes, I should add Gal Shachaf <address@hidden>,Aviad Yehezkel
<address@hidden>
we are working together on RDMA live migration.
Thanks.
>
> Dave
>
>> ---
>> migration/rdma.c | 33 ++++++++++++++++++++++++++++++---
>> 1 file changed, 30 insertions(+), 3 deletions(-)
>>
>> diff --git a/migration/rdma.c b/migration/rdma.c
>> index f12e8d5..bb6989e 100644
>> --- a/migration/rdma.c
>> +++ b/migration/rdma.c
>> @@ -1489,6 +1489,9 @@ static uint64_t qemu_rdma_poll(RDMAContext *rdma,
>> uint64_t *wr_id_out,
>> */
>> static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)
>> {
>> + struct rdma_cm_event *cm_event;
>> + int ret = -1;
>> +
>> /*
>> * Coroutine doesn't start until migration_fd_process_incoming()
>> * so don't yield unless we know we're running inside of a coroutine.
>> @@ -1505,13 +1508,37 @@ static int qemu_rdma_wait_comp_channel(RDMAContext
>> *rdma)
>> * without hanging forever.
>> */
>> while (!rdma->error_state && !rdma->received_error) {
>> - GPollFD pfds[1];
>> + GPollFD pfds[2];
>> pfds[0].fd = rdma->comp_channel->fd;
>> pfds[0].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
>> + pfds[0].revents = 0;
>> +
>> + pfds[1].fd = rdma->channel->fd;
>> + pfds[1].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
>> + pfds[1].revents = 0;
>> +
>> /* 0.1s timeout, should be fine for a 'cancel' */
>> - switch (qemu_poll_ns(pfds, 1, 100 * 1000 * 1000)) {
>> + switch (qemu_poll_ns(pfds, 2, 100 * 1000 * 1000)) {
>> + case 2:
>> case 1: /* fd active */
>> - return 0;
>> + if (pfds[0].revents) {
>> + return 0;
>> + }
>> +
>> + if (pfds[1].revents) {
>> + ret = rdma_get_cm_event(rdma->channel, &cm_event);
>> + if (!ret) {
>> + rdma_ack_cm_event(cm_event);
>> + }
>> +
>> + error_report("receive cm event while wait comp channel,"
>> + "cm event is %d", cm_event->event);
>> + if (cm_event->event == RDMA_CM_EVENT_DISCONNECTED ||
>> + cm_event->event == RDMA_CM_EVENT_DEVICE_REMOVAL) {
>> + return -EPIPE;
>> + }
>> + }
>> + break;
>>
>> case 0: /* Timeout, go around again */
>> break;
>> --
>> 1.8.3.1
>>
> --
> Dr. David Alan Gilbert / address@hidden / Manchester, UK
- [Qemu-devel] [PATCH v5 03/10] migration: avoid concurrent invoke channel_close by different threads, (continued)
- [Qemu-devel] [PATCH v5 03/10] migration: avoid concurrent invoke channel_close by different threads, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 05/10] migration: Stop rdma yielding during incoming postcopy, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 04/10] migration: implement bi-directional RDMA QIOChannel, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 07/10] migration: invoke qio_channel_yield only when qemu_in_coroutine(), Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 06/10] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 08/10] migration: create a dedicated thread to release rdma resource, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 09/10] migration: poll the cm event while wait RDMA work request completion, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 10/10] migration: implement the shutdown for RDMA QIOChannel, Lidong Chen, 2018/06/05