[Qemu-devel] [PATCH 00/41] Migration cleanups and latency improvements

[Paolo Bonzini]

This series does many of the improvements that the migration thread
promised.  It removes buffering, lets a large amount of code run outside
the big QEMU lock, and removes some duplication between incoming and
outgoing migration.

Patches 1 to 7 are simple cleanups.

Patches 8 to 14 simplify the lifecycle of the migration thread and
the migration QEMUFile.

Patches 15 to 18 add fine-grained locking to the block migration
data structures, so that patches 19 to 21 can move RAM/block live
migration out of the big QEMU lock.  At this point blocking writes
will not starve other threads seeking to grab the big QEMU mutex:
patches 22 to 24 removes the buffering and cleanup the code.

Patches 25 to 28 are more cleanups.

Patches 29 to 33 improve QEMUFile so that patches 34 and 35 can
use QEMUFile to write out data, instead of MigrationState.
Patches 36 to 41 then can remove the useless QEMUFile wrapper
that remains.

Please review and test!  You can find these patches at
git://github.com/bonzini/qemu.git, branch migration-thread-20130115.

Juan Quintela (1):
  Rename buffered_ to migration_

Paolo Bonzini (40):
  migration: simplify while loop
  migration: always use vm_stop_force_state
  migration: move more error handling to migrate_fd_cleanup
  migration: push qemu_savevm_state_cancel out of qemu_savevm_state_*
  block-migration: remove useless calls to blk_mig_cleanup
  qemu-file: pass errno from qemu_fflush via f->last_error
  migration: use qemu_file_set_error to pass error codes back to
    qemu_savevm_state
  qemu-file: temporarily expose qemu_file_set_error and qemu_fflush
  migration: flush all data to fd when buffered_flush is called
  migration: use qemu_file_set_error
  migration: simplify error handling
  migration: do not nest flushing of device data
  migration: prepare to access s->state outside critical sections
  migration: cleanup migration (including thread) in the iothread
  block-migration: remove variables that are never read
  block-migration: small preparatory changes for locking
  block-migration: document usage of state across threads
  block-migration: add lock
  migration: reorder SaveVMHandlers members
  migration: run pending/iterate callbacks out of big lock
  migration: run setup callbacks out of big lock
  migration: yay, buffering is gone
  qemu-file: make qemu_fflush and qemu_file_set_error private again
  migration: eliminate last_round
  migration: detect error before sleeping
  migration: remove useless qemu_file_get_error check
  migration: use qemu_file_rate_limit consistently
  migration: merge qemu_popen_cmd with qemu_popen
  qemu-file: fsync a writable stdio QEMUFile
  qemu-file: check exit status when closing a pipe QEMUFile
  qemu-file: add writable socket QEMUFile
  qemu-file: simplify and export qemu_ftell
  migration: use QEMUFile for migration channel lifetime
  migration: use QEMUFile for writing outgoing migration data
  migration: use qemu_ftell to compute bandwidth
  migration: small changes around rate-limiting
  migration: move rate limiting to QEMUFile
  migration: move contents of migration_close to migrate_fd_cleanup
  migration: eliminate s->migration_file
  migration: inline migrate_fd_close

 arch_init.c                   |   14 ++-
 block-migration.c             |  167 +++++++++++++++------
 docs/migration.txt            |   20 +---
 include/migration/migration.h |   12 +--
 include/migration/qemu-file.h |   21 +--
 include/migration/vmstate.h   |   21 ++-
 include/qemu/atomic.h         |    1 +
 include/sysemu/sysemu.h       |    6 +-
 migration-exec.c              |   39 +-----
 migration-fd.c                |   47 +------
 migration-tcp.c               |   33 +----
 migration-unix.c              |   33 +----
 migration.c                   |  345 ++++++++---------------------------------
 savevm.c                      |  214 +++++++++++++++-----------
 util/osdep.c                  |    6 +-
 15 files changed, 367 insertions(+), 612 deletions(-)