[Qemu-devel] [PATCH v2 00/42] Migration cleanups and latency improvements

[Paolo Bonzini]

This series does many of the improvements that the migration thread
promised.  First, it simplifies substantially the logic of the migration
thread, which makes it easier to do more changes later (patches 1-14).

Second, It makes a large amount of code run outside the big QEMU
lock, so that blocking writes will not starve other threads seeking
to grab the big QEMU mutex---and this in turn lets us remove buffering
(patches 15-24).

Third, it completes the cleanup of rate limiting, which lets us removes
some duplication between incoming and outgoing migration and use QEMUFile
in a much simpler way (patches 25-42).

While my main purpose here is to move the code in the "right" direction,
the saner locking and removed copies give a nice performance bump.
On trivial workloads (idle guests), having little or no iothread lock
contention already improves performance by ~5%.  In real-world scenarios,
however, QEMU can now achieve 2.5-3 Gbps on a 10G link, compared to
1.5-1.8 before these patches.  Jitter is also improved nicely, though
the previous work on the migration thread already had some effect there.

Much to my surprise, block migration is still working nicely, with no
contortions and only a minimal amount of code running under the iothread
lock.  You probably won't here me talking anymore about deprecating it.
Let it rot^Wrest in peace.

Thanks Juan and Orit for the quick review.

v1->v2:
        rebased; made changes suggested by Juan and Orit; integrated
        Kazuya Saito's migrate_set_state tracepoint; added
        WIFEXITED/WEXITSTATUS defines for Windows; minor checkpatch
        complaints.

Juan Quintela (1):
  Rename buffered_ to migration_

Kazuya Saito (1):
  migration: add migrate_set_state tracepoint

Paolo Bonzini (40):
  migration: simplify while loop
  migration: always use vm_stop_force_state
  migration: move more error handling to migrate_fd_cleanup
  migration: push qemu_savevm_state_cancel out of qemu_savevm_state_*
  block-migration: remove useless calls to blk_mig_cleanup
  qemu-file: pass errno from qemu_fflush via f->last_error
  migration: use qemu_file_set_error to pass error codes back to
    qemu_savevm_state
  qemu-file: temporarily expose qemu_file_set_error and qemu_fflush
  migration: flush all data to fd when buffered_flush is called
  migration: use qemu_file_set_error
  migration: simplify error handling
  migration: do not nest flushing of device data
  migration: prepare to access s->state outside critical sections
  migration: cleanup migration (including thread) in the iothread
  block-migration: remove variables that are never read
  block-migration: small preparatory changes for locking
  block-migration: document usage of state across threads
  block-migration: add lock
  migration: reorder SaveVMHandlers members
  migration: run pending/iterate callbacks out of big lock
  migration: run setup callbacks out of big lock
  migration: yay, buffering is gone
  qemu-file: make qemu_fflush and qemu_file_set_error private again
  migration: eliminate last_round
  migration: detect error before sleeping
  migration: remove useless qemu_file_get_error check
  migration: use qemu_file_rate_limit consistently
  migration: merge qemu_popen_cmd with qemu_popen
  qemu-file: fsync a writable stdio QEMUFile
  qemu-file: check exit status when closing a pipe QEMUFile
  qemu-file: add writable socket QEMUFile
  qemu-file: simplify and export qemu_ftell
  migration: use QEMUFile for migration channel lifetime
  migration: use QEMUFile for writing outgoing migration data
  migration: use qemu_ftell to compute bandwidth
  migration: small changes around rate-limiting
  migration: move rate limiting to QEMUFile
  migration: move contents of migration_close to migrate_fd_cleanup
  migration: eliminate s->migration_file
  migration: inline migrate_fd_close

 arch_init.c                   |  14 +-
 block-migration.c             | 167 ++++++++++++++------
 docs/migration.txt            |  20 +--
 include/migration/migration.h |  12 +-
 include/migration/qemu-file.h |  21 +--
 include/migration/vmstate.h   |  21 ++-
 include/qemu/atomic.h         |   1 +
 include/qemu/osdep.h          |   7 +
 include/sysemu/sysemu.h       |   6 +-
 migration-exec.c              |  39 +----
 migration-fd.c                |  47 +-----
 migration-tcp.c               |  33 +---
 migration-unix.c              |  33 +---
 migration.c                   | 357 ++++++++++--------------------------------
 savevm.c                      | 214 ++++++++++++++-----------
 trace-events                  |   3 +
 16 files changed, 385 insertions(+), 610 deletions(-)

-- 
1.8.1.2