[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v4 06/32] migration: allow dst vm pause on postc
From: |
Dr. David Alan Gilbert |
Subject: |
Re: [Qemu-devel] [PATCH v4 06/32] migration: allow dst vm pause on postcopy |
Date: |
Thu, 30 Nov 2017 11:17:53 +0000 |
User-agent: |
Mutt/1.9.1 (2017-09-22) |
* Peter Xu (address@hidden) wrote:
> When there is IO error on the incoming channel (e.g., network down),
> instead of bailing out immediately, we allow the dst vm to switch to the
> new POSTCOPY_PAUSE state. Currently it is still simple - it waits the
> new semaphore, until someone poke it for another attempt.
>
> Signed-off-by: Peter Xu <address@hidden>
As noted last time we need one of the later patches (8/32) to stop a
race; but OK
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
> ---
> migration/migration.c | 1 +
> migration/migration.h | 3 +++
> migration/savevm.c | 60
> ++++++++++++++++++++++++++++++++++++++++++++++++--
> migration/trace-events | 2 ++
> 4 files changed, 64 insertions(+), 2 deletions(-)
>
> diff --git a/migration/migration.c b/migration/migration.c
> index 46e7ca36a4..b166e19785 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -150,6 +150,7 @@ MigrationIncomingState
> *migration_incoming_get_current(void)
> memset(&mis_current, 0, sizeof(MigrationIncomingState));
> qemu_mutex_init(&mis_current.rp_mutex);
> qemu_event_init(&mis_current.main_thread_load_event, false);
> + qemu_sem_init(&mis_current.postcopy_pause_sem_dst, 0);
> once = true;
> }
> return &mis_current;
> diff --git a/migration/migration.h b/migration/migration.h
> index 36aaa13f50..55894ecb79 100644
> --- a/migration/migration.h
> +++ b/migration/migration.h
> @@ -61,6 +61,9 @@ struct MigrationIncomingState {
> /* The coroutine we should enter (back) after failover */
> Coroutine *migration_incoming_co;
> QemuSemaphore colo_incoming_sem;
> +
> + /* notify PAUSED postcopy incoming migrations to try to continue */
> + QemuSemaphore postcopy_pause_sem_dst;
> };
>
> MigrationIncomingState *migration_incoming_get_current(void);
> diff --git a/migration/savevm.c b/migration/savevm.c
> index 1da0255cd7..93e308ebf0 100644
> --- a/migration/savevm.c
> +++ b/migration/savevm.c
> @@ -1529,8 +1529,8 @@ static int
> loadvm_postcopy_ram_handle_discard(MigrationIncomingState *mis,
> */
> static void *postcopy_ram_listen_thread(void *opaque)
> {
> - QEMUFile *f = opaque;
> MigrationIncomingState *mis = migration_incoming_get_current();
> + QEMUFile *f = mis->from_src_file;
> int load_res;
>
> migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE,
> @@ -1544,6 +1544,14 @@ static void *postcopy_ram_listen_thread(void *opaque)
> */
> qemu_file_set_blocking(f, true);
> load_res = qemu_loadvm_state_main(f, mis);
> +
> + /*
> + * This is tricky, but, mis->from_src_file can change after it
> + * returns, when postcopy recovery happened. In the future, we may
> + * want a wrapper for the QEMUFile handle.
> + */
> + f = mis->from_src_file;
> +
> /* And non-blocking again so we don't block in any cleanup */
> qemu_file_set_blocking(f, false);
>
> @@ -1626,7 +1634,7 @@ static int
> loadvm_postcopy_handle_listen(MigrationIncomingState *mis)
> /* Start up the listening thread and wait for it to signal ready */
> qemu_sem_init(&mis->listen_thread_sem, 0);
> qemu_thread_create(&mis->listen_thread, "postcopy/listen",
> - postcopy_ram_listen_thread, mis->from_src_file,
> + postcopy_ram_listen_thread, NULL,
> QEMU_THREAD_DETACHED);
> qemu_sem_wait(&mis->listen_thread_sem);
> qemu_sem_destroy(&mis->listen_thread_sem);
> @@ -2011,11 +2019,44 @@ void qemu_loadvm_state_cleanup(void)
> }
> }
>
> +/* Return true if we should continue the migration, or false. */
> +static bool postcopy_pause_incoming(MigrationIncomingState *mis)
> +{
> + trace_postcopy_pause_incoming();
> +
> + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
> + MIGRATION_STATUS_POSTCOPY_PAUSED);
> +
> + assert(mis->from_src_file);
> + qemu_file_shutdown(mis->from_src_file);
> + qemu_fclose(mis->from_src_file);
> + mis->from_src_file = NULL;
> +
> + assert(mis->to_src_file);
> + qemu_file_shutdown(mis->to_src_file);
> + qemu_mutex_lock(&mis->rp_mutex);
> + qemu_fclose(mis->to_src_file);
> + mis->to_src_file = NULL;
> + qemu_mutex_unlock(&mis->rp_mutex);
> +
> + error_report("Detected IO failure for postcopy. "
> + "Migration paused.");
> +
> + while (mis->state == MIGRATION_STATUS_POSTCOPY_PAUSED) {
> + qemu_sem_wait(&mis->postcopy_pause_sem_dst);
> + }
> +
> + trace_postcopy_pause_incoming_continued();
> +
> + return true;
> +}
> +
> static int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis)
> {
> uint8_t section_type;
> int ret = 0;
>
> +retry:
> while (true) {
> section_type = qemu_get_byte(f);
>
> @@ -2060,6 +2101,21 @@ static int qemu_loadvm_state_main(QEMUFile *f,
> MigrationIncomingState *mis)
> out:
> if (ret < 0) {
> qemu_file_set_error(f, ret);
> +
> + /*
> + * Detect whether it is:
> + *
> + * 1. postcopy running
> + * 2. network failure (-EIO)
> + *
> + * If so, we try to wait for a recovery.
> + */
> + if (mis->state == MIGRATION_STATUS_POSTCOPY_ACTIVE &&
> + ret == -EIO && postcopy_pause_incoming(mis)) {
> + /* Reset f to point to the newly created channel */
> + f = mis->from_src_file;
> + goto retry;
> + }
> }
> return ret;
> }
> diff --git a/migration/trace-events b/migration/trace-events
> index da1c63a933..bed1646cd6 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -100,6 +100,8 @@ open_return_path_on_source(void) ""
> open_return_path_on_source_continue(void) ""
> postcopy_start(void) ""
> postcopy_pause_continued(void) ""
> +postcopy_pause_incoming(void) ""
> +postcopy_pause_incoming_continued(void) ""
> postcopy_start_set_run(void) ""
> source_return_path_thread_bad_end(void) ""
> source_return_path_thread_end(void) ""
> --
> 2.13.6
>
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK
- [Qemu-devel] [PATCH v4 00/32] Migration: postcopy failure recovery, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 02/32] migration: reuse mis->userfault_quit_fd, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 03/32] migration: provide postcopy_fault_thread_notify(), Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 01/32] migration: better error handling with QEMUFile, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 04/32] migration: new postcopy-pause state, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 06/32] migration: allow dst vm pause on postcopy, Peter Xu, 2017/11/08
- Re: [Qemu-devel] [PATCH v4 06/32] migration: allow dst vm pause on postcopy,
Dr. David Alan Gilbert <=
- [Qemu-devel] [PATCH v4 08/32] migration: allow send_rq to fail, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 09/32] migration: allow fault thread to pause, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 10/32] qmp: hmp: add migrate "resume" option, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 11/32] migration: pass MigrationState to migrate_init(), Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 15/32] migration: new cmd MIG_CMD_RECV_BITMAP, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 17/32] migration: new cmd MIG_CMD_POSTCOPY_RESUME, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 18/32] migration: new message MIG_RP_MSG_RESUME_ACK, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 19/32] migration: introduce SaveVMHandlers.resume_prepare, Peter Xu, 2017/11/08
- [Qemu-devel] [PATCH v4 20/32] migration: synchronize dirty bitmap for resume, Peter Xu, 2017/11/08