[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH v5 08/10] migration: create a dedicated thread to re
From: |
Lidong Chen |
Subject: |
[Qemu-devel] [PATCH v5 08/10] migration: create a dedicated thread to release rdma resource |
Date: |
Tue, 5 Jun 2018 23:28:07 +0800 |
ibv_dereg_mr wait for a long time for big memory size virtual server.
The test result is:
10GB 326ms
20GB 699ms
30GB 1021ms
40GB 1387ms
50GB 1712ms
60GB 2034ms
70GB 2457ms
80GB 2807ms
90GB 3107ms
100GB 3474ms
110GB 3735ms
120GB 4064ms
130GB 4567ms
140GB 4886ms
this will cause the guest os hang for a while when migration finished.
So create a dedicated thread to release rdma resource.
Signed-off-by: Lidong Chen <address@hidden>
---
migration/rdma.c | 43 +++++++++++++++++++++++++++----------------
1 file changed, 27 insertions(+), 16 deletions(-)
diff --git a/migration/rdma.c b/migration/rdma.c
index dfa4f77..f12e8d5 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2979,35 +2979,46 @@ static void
qio_channel_rdma_set_aio_fd_handler(QIOChannel *ioc,
}
}
-static int qio_channel_rdma_close(QIOChannel *ioc,
- Error **errp)
+static void *qio_channel_rdma_close_thread(void *arg)
{
- QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
- RDMAContext *rdmain, *rdmaout;
- trace_qemu_rdma_close();
+ RDMAContext **rdma = arg;
+ RDMAContext *rdmain = rdma[0];
+ RDMAContext *rdmaout = rdma[1];
- rdmain = rioc->rdmain;
- if (rdmain) {
- atomic_rcu_set(&rioc->rdmain, NULL);
- }
-
- rdmaout = rioc->rdmaout;
- if (rdmaout) {
- atomic_rcu_set(&rioc->rdmaout, NULL);
- }
+ rcu_register_thread();
synchronize_rcu();
-
if (rdmain) {
qemu_rdma_cleanup(rdmain);
}
-
if (rdmaout) {
qemu_rdma_cleanup(rdmaout);
}
g_free(rdmain);
g_free(rdmaout);
+ g_free(rdma);
+
+ rcu_unregister_thread();
+ return NULL;
+}
+
+static int qio_channel_rdma_close(QIOChannel *ioc,
+ Error **errp)
+{
+ QemuThread t;
+ QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
+ RDMAContext **rdma = g_new0(RDMAContext*, 2);
+
+ trace_qemu_rdma_close();
+ if (rioc->rdmain || rioc->rdmaout) {
+ rdma[0] = rioc->rdmain;
+ rdma[1] = rioc->rdmaout;
+ qemu_thread_create(&t, "rdma cleanup", qio_channel_rdma_close_thread,
+ rdma, QEMU_THREAD_DETACHED);
+ atomic_rcu_set(&rioc->rdmain, NULL);
+ atomic_rcu_set(&rioc->rdmaout, NULL);
+ }
return 0;
}
--
1.8.3.1
- [Qemu-devel] [PATCH v5 02/10] migration: create a dedicated connection for rdma return path, (continued)
- [Qemu-devel] [PATCH v5 02/10] migration: create a dedicated connection for rdma return path, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 03/10] migration: avoid concurrent invoke channel_close by different threads, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 05/10] migration: Stop rdma yielding during incoming postcopy, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 04/10] migration: implement bi-directional RDMA QIOChannel, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 07/10] migration: invoke qio_channel_yield only when qemu_in_coroutine(), Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 06/10] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 08/10] migration: create a dedicated thread to release rdma resource,
Lidong Chen <=
- [Qemu-devel] [PATCH v5 09/10] migration: poll the cm event while wait RDMA work request completion, Lidong Chen, 2018/06/05
- [Qemu-devel] [PATCH v5 10/10] migration: implement the shutdown for RDMA QIOChannel, Lidong Chen, 2018/06/05