qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-devel] [PATCH v9 00/56] Postcopy implementation


From: Dr. David Alan Gilbert (git)
Subject: [Qemu-devel] [PATCH v9 00/56] Postcopy implementation
Date: Thu, 5 Nov 2015 18:10:27 +0000

From: "Dr. David Alan Gilbert" <address@hidden>

  This is the 9th cut of my version of postcopy.

The userfaultfd linux kernel code is now in the upstream kernel
tree, and so 4.3 can be used without modification.

This qemu series can be found at:
https://github.com/orbitfp7/qemu.git
on the wp3-postcopy-v9 tag

Testing status:
  * Tested heavily on x86
  * Smoke tested on aarch64 (so it does work on different page sizes)

This work has been partially funded by the EU Orbit project:
  see http://www.orbitproject.eu/about/

v9
  Almost all of the changes are changes from review comments, and most are
  minor.

  The following are new patches, mostly split out from earlier patches
  (The exception being adding userfaultfd.h header back in - which was in v5
   but we took out, but is needed again due to 1842bdfd)

    04/56 Move page_size_init earlier
    08/56 qemu_ram_block_by_name
    12/56 Factor out host_from_stream_offset call and check
    15/56 Add Linux userfaultfd.h header back
    21/56 migration_is_setup_or_active
    30/56 migration_completion: Take current state
    34/56 Maintain unsentmap

  The previous patches 03,10,13,36/54 went in upstream already.

  Fix for assert using hotplug (Thanks Bharata for spotting that)
  Fix for migrate_cancel a second time
  Rework for migration_bitmap_rcu after Denis's deadlock fix
  Rework ram_load into a separate postcopy loop
  The 'sentmap' is now an 'unsentmap' - this saves a complement step at the end
     The unsentmap creation is now split into a separate patch
     The unsentmap is now stored in the RCU structure (although it can't
       really resize during the migrate)
  Fix for block migration
     still not a suggested combination.
  Minor:
     trace_savevm_send_open_return_path added
     split migration_is_active function into separate patch and made
        migration_is_setup_or_active
     move file reads into migrate_handle_advise and migrate_handle_packaged
     use of MIN in send-packaged-chunk
     migration_thread_started -> migration_thread_running
     updated qemu_get_buffer_in_place with Juan's version (and size_t'ified it)
     split the host_from_stream_offset change into a separate patch
       'ram_load: Factor out host_from_stream_offset call and check'
     split out ram_find_block_by_id into a separate patch and now
       called qemu_ram_block_by_name
     postcopy_discard_send_range etc now take start/length rather than start/end
       (also added another trace)
     split ram_save_host_page into ram_save_host_page/ram_save_target_page
     split host page cleanup function into a core that handles both passes

Dave

Dr. David Alan Gilbert (56):
  Add postcopy documentation
  Provide runtime Target page information
  Move configuration section writing
  Move page_size_init earlier
  qemu_ram_block_from_host
  qemu_ram_block_by_name
  Rename mis->file to from_src_file
  Add qemu_get_buffer_in_place to avoid copies some of the time
  Add wrapper for setting blocking status on a QEMUFile
  Add QEMU_MADV_NOHUGEPAGE
  ram_debug_dump_bitmap: Dump a migration bitmap as text
  ram_load: Factor out host_from_stream_offset call and check
  migrate_init: Call from savevm
  Rename save_live_complete to save_live_complete_precopy
  Add Linux userfaultfd.h header
  Return path: Open a return path on QEMUFile for sockets
  Return path: socket_writev_buffer: Block even on non-blocking fd's
  Migration commands
  Return path: Control commands
  Return path: Send responses from destination to source
  migration_is_setup_or_active
  Return path: Source handling of return path
  Rework loadvm path for subloops
  Add migration-capability boolean for postcopy-ram.
  Add wrappers and handlers for sending/receiving the postcopy-ram
    migration messages.
  MIG_CMD_PACKAGED: Send a packaged chunk of migration stream
  Modify save_live_pending for postcopy
  postcopy: OS support test
  migrate_start_postcopy: Command to trigger transition to postcopy
  migration_completion: Take current state
  MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state
  Avoid sending vmdescription during postcopy
  Add qemu_savevm_state_complete_postcopy
  Postcopy: Maintain unsentmap
  Postcopy: Calculate discard
  postcopy: Incoming initialisation
  postcopy: ram_enable_notify to switch on userfault
  Postcopy: Postcopy startup in migration thread
  Postcopy: End of iteration
  Page request:  Add MIG_RP_MSG_REQ_PAGES reverse command
  Page request: Process incoming page request
  Page request: Consume pages off the post-copy queue
  postcopy_ram.c: place_page and helpers
  Postcopy: Use helpers to map pages during migration
  postcopy: Check order of received target pages
  Don't sync dirty bitmaps in postcopy
  Don't iterate on precopy-only devices during postcopy
  Host page!=target page: Cleanup bitmaps
  Round up RAMBlock sizes to host page sizes
  Postcopy; Handle userfault requests
  Start up a postcopy/listener thread ready for incoming page data
  postcopy: Wire up loadvm_postcopy_handle_ commands
  Postcopy: Mark nohugepage before discard
  End of migration for postcopy
  Disable mlock around incoming postcopy
  Inhibit ballooning during postcopy

 balloon.c                         |  11 +
 docs/migration.txt                | 191 ++++++++
 exec.c                            |  92 +++-
 hmp-commands.hx                   |  15 +
 hmp.c                             |   7 +
 hmp.h                             |   1 +
 hw/ppc/spapr.c                    |   2 +-
 hw/virtio/virtio-balloon.c        |   4 +-
 include/exec/cpu-common.h         |   4 +
 include/exec/exec-all.h           |   1 -
 include/exec/ram_addr.h           |   2 -
 include/migration/migration.h     | 121 ++++-
 include/migration/postcopy-ram.h  |  99 ++++
 include/migration/qemu-file.h     |  10 +
 include/migration/vmstate.h       |   8 +-
 include/qemu-common.h             |   1 +
 include/qemu/osdep.h              |   9 +
 include/qemu/typedefs.h           |   3 +
 include/sysemu/balloon.h          |   2 +
 include/sysemu/sysemu.h           |  46 +-
 kvm-all.c                         |   1 -
 linux-headers/linux/userfaultfd.h | 167 +++++++
 migration/Makefile.objs           |   2 +-
 migration/block.c                 |   9 +-
 migration/migration.c             | 723 +++++++++++++++++++++++++--
 migration/postcopy-ram.c          | 760 +++++++++++++++++++++++++++++
 migration/qemu-file-unix.c        | 111 ++++-
 migration/qemu-file.c             |  64 +++
 migration/ram.c                   | 997 +++++++++++++++++++++++++++++++++++---
 migration/savevm.c                | 823 +++++++++++++++++++++++++++----
 qapi-schema.json                  |  18 +-
 qmp-commands.hx                   |  19 +
 qtest.c                           |   1 -
 trace-events                      |  84 +++-
 vl.c                              |   1 +
 35 files changed, 4159 insertions(+), 250 deletions(-)
 create mode 100644 include/migration/postcopy-ram.h
 create mode 100644 linux-headers/linux/userfaultfd.h
 create mode 100644 migration/postcopy-ram.c

-- 
2.5.0




reply via email to

[Prev in Thread] Current Thread [Next in Thread]