[Qemu-devel] [PATCH v8 00/54] Postcopy implementation

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <address@hidden>

  This is the 8th cut of my version of postcopy.

The userfaultfd linux kernel code is now in the upstream kernel
tree, and so 4.3-rc3 can be used without modification.

This qemu series can be found at:
https://github.com/orbitfp7/qemu.git
on the wp3-postcopy-v8 tag


Testing status:
  * Tested heavily on x86
  * Smoke tested on aarch64 (so it does work on different page sizes)
  * Power is unhappy for me (but gets further than the htab problem
    v7 used to have) (I get a kvm run failed)

Note that patches:
   3 Init page size in qtest
   10 Use RAMBlock rather than Memory Region
   14,15 (splitting ram find and save block)
   36 Split out end of migration code

   have also been posted seperately during the last month and
can be taken separately from this series.

This work has been partially funded by the EU Orbit project:
  see http://www.orbitproject.eu/about/

v8
  Huge page changes
     The precopy phase is now allowed to keep transparent-huge-pages
     enabled, although these maybe split in the discard phase.  A change
     to the discard process now causes discards for unsent (as well as
     redirtied) pages; the combination of these changes means that the
     destination starts running with many of the precopy pages as huge
     pages, resulting in a significant performance benefit.  This change
     adds one more state ('discard') which is entered by the destination
     on reception of the 1st discard command.

  Add global_state_store for postcopy
  Moved postcopy_state back out of MigrationIncomingState
     During the end of the main migration thread reading state the
     postcopy state is read to see who should cleanup; in failure
     conditions there was a race between the state being read and the
     freeing of the MIS

  Stop calling the iterate method for non-postcopiable devices during postcopy
     Helps fix Power (thanks for Bharata for helping debug this)

  Review comment fixes
    rework of migration command parsing
    rework of postcopy_chunk_hostpages
    rework of discard code & protocol into start/length rather than start/end
    rename qemu_get_buffer_less_copy -> qemu_get_buffer_in_place
    split 'Postcopy end in migration thread' patch into two
    split of MIG_RP_MSG_REQ_PAGES into subtype with name
    Added comments documenting postcopy_state's use
    lots of other minor fixups

  Notes:
    I kept the mlock support (users are saying they wanted migration/postcopy
      with mlock)

    I'm keeping the x-  for now, until the libvirt interface gets finalised.

    There are two checkpatch errors, that I don't think are right to change:
     a) a 'typedef enum' it wants split - that's the way we do all our enums
        and would force a dummy name for the enum.
     b) A complaint about postcopy_ram_discard_version = 0 that's a global
        static; I could get rid of it by making my version 1, but it doesn't
        seem right to omit the '= 0 ' for a version constant.

Dave

Dr. David Alan Gilbert (54):
  Add postcopy documentation
  Provide runtime Target page information
  Init page sizes in qtest
  Move configuration section writing
  qemu_ram_block_from_host
  Rename mis->file to from_src_file
  Add qemu_get_buffer_in_place to avoid copies some of the time
  Add wrapper for setting blocking status on a QEMUFile
  Add QEMU_MADV_NOHUGEPAGE
  migration/ram.c: Use RAMBlock rather than MemoryRegion
  ram_debug_dump_bitmap: Dump a migration bitmap as text
  migrate_init: Call from savevm
  Move dirty page search state into separate structure
  ram_find_and_save_block: Split out the finding
  Rename save_live_complete to save_live_complete_precopy
  Return path: Open a return path on QEMUFile for sockets
  Return path: socket_writev_buffer: Block even on non-blocking fd's
  Migration commands
  Return path: Control commands
  Return path: Send responses from destination to source
  Return path: Source handling of return path
  Rework loadvm path for subloops
  Add migration-capability boolean for postcopy-ram.
  Add wrappers and handlers for sending/receiving the postcopy-ram
    migration messages.
  MIG_CMD_PACKAGED: Send a packaged chunk of migration stream
  Modify save_live_pending for postcopy
  postcopy: OS support test
  migrate_start_postcopy: Command to trigger transition to postcopy
  MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state
  Avoid sending vmdescription during postcopy
  Add qemu_savevm_state_complete_postcopy
  Postcopy: Maintain sentmap and calculate discard
  postcopy: Incoming initialisation
  postcopy: ram_enable_notify to switch on userfault
  Postcopy: Postcopy startup in migration thread
  Split out end of migration code from migration_thread
  Postcopy: End of iteration
  Page request:  Add MIG_RP_MSG_REQ_PAGES reverse command
  Page request: Process incoming page request
  Page request: Consume pages off the post-copy queue
  postcopy_ram.c: place_page and helpers
  Postcopy: Use helpers to map pages during migration
  Don't sync dirty bitmaps in postcopy
  Don't iterate on precopy-only devices during postcopy
  Host page!=target page: Cleanup bitmaps
  postcopy: Check order of received target pages
  Round up RAMBlock sizes to host page sizes
  Postcopy; Handle userfault requests
  Start up a postcopy/listener thread ready for incoming page data
  postcopy: Wire up loadvm_postcopy_handle_ commands
  Postcopy: Mark nohugepage before discard
  End of migration for postcopy
  Disable mlock around incoming postcopy
  Inhibit ballooning during postcopy

 balloon.c                        |  11 +
 docs/migration.txt               | 191 ++++++++
 exec.c                           |  72 ++-
 hmp-commands.hx                  |  15 +
 hmp.c                            |   7 +
 hmp.h                            |   1 +
 hw/ppc/spapr.c                   |   2 +-
 hw/virtio/virtio-balloon.c       |   4 +-
 include/exec/cpu-common.h        |   3 +
 include/exec/ram_addr.h          |   2 -
 include/migration/migration.h    | 128 ++++-
 include/migration/postcopy-ram.h |  99 ++++
 include/migration/qemu-file.h    |  10 +
 include/migration/vmstate.h      |   8 +-
 include/qemu/osdep.h             |   9 +
 include/qemu/typedefs.h          |   3 +
 include/sysemu/balloon.h         |   2 +
 include/sysemu/sysemu.h          |  46 +-
 migration/Makefile.objs          |   2 +-
 migration/block.c                |   9 +-
 migration/migration.c            | 753 +++++++++++++++++++++++++++--
 migration/postcopy-ram.c         | 763 ++++++++++++++++++++++++++++++
 migration/qemu-file-unix.c       | 111 ++++-
 migration/qemu-file.c            |  74 +++
 migration/ram.c                  | 995 +++++++++++++++++++++++++++++++++++----
 migration/savevm.c               | 825 ++++++++++++++++++++++++++++----
 qapi-schema.json                 |  18 +-
 qmp-commands.hx                  |  19 +
 qtest.c                          |   1 +
 trace-events                     |  81 +++-
 30 files changed, 3996 insertions(+), 268 deletions(-)
 create mode 100644 include/migration/postcopy-ram.h
 create mode 100644 migration/postcopy-ram.c

-- 
2.5.0