[PATCH v2 00/29] migration: File based migration with multifd and fixed-ram

[Fabiano Rosas]

Hi,

Here's the migration to file using fixed offsets for each RAM
page. We're calling it fixed-ram migration.

There's 3 big pieces in this series:

1) Fixed-ram: The single-threaded (no multifd) implementation of the
fixed-ram migration. This adds:
  - infrastructure for preadv/pwritev;
  - a bitmap to keep track of which pages are written to the file;
  - a migration header containing fixed-ram-specific information;
  - a capability to enable the feature;
  - the /x86/migration/multifd/file/fixed-ram/* tests.

2) Multifd support: Changes to multifd to support fixed-ram. The main
point here is that we don't need the synchronous parts of multifd,
only the page-queuing, multi-threading and IO is of interest. This
adds:
  - the use_packets() option to skip sending packets when doing fixed-ram;

  - the concept of pages to the receiving side. We want to collect the
    pages from the file using multifd as well.

3) Auto-pause capability: A new capability to allow QEMU to pause the
VM when the type of migration selected would already result in a
paused VM at the end of migration (snapshots, file migration). This is
intended to allow QEMU to implement optimizations for the migration
types that don't need the live migration infrastructure.

The feature is opt-in for new migration code in QEMU, but opt-out for
QEMU users. I.e. new migration code must declare that it supports
auto-pause, but the management layer needs to turn the capability off
to force a live migration.

Thanks

v1:
https://lore.kernel.org/r/20230330180336.2791-1-farosas@suse.de

Fabiano Rosas (21):
  tests/qtest: Move QTestMigrationState to libqtest
  tests/qtest: Allow waiting for migration events
  migration: Return the saved state from global_state_store
  migration: Introduce global_state_store_once
  migration: Add auto-pause capability
  migration: Run "file:" migration with a stopped VM
  tests/qtest: File migration auto-pause tests
  migration: fixed-ram: Add URI compatibility check
  migration/ram: Introduce 'fixed-ram' migration capability
  migration/multifd: Allow multifd without packets
  migration/multifd: Add outgoing QIOChannelFile support
  migration/multifd: Add incoming QIOChannelFile support
  migration/multifd: Add pages to the receiving side
  io: Add a pwritev/preadv version that takes a discontiguous iovec
  migration/ram: Add a wrapper for fixed-ram shadow bitmap
  migration/ram: Ignore multifd flush when doing fixed-ram migration
  migration/multifd: Support outgoing fixed-ram stream format
  migration/multifd: Support incoming fixed-ram stream format
  tests/qtest: Add a multifd + fixed-ram migration test
  migration: Add direct-io parameter
  tests/qtest: Add a test for migration with direct-io and multifd

Nikolay Borisov (7):
  io: add and implement QIO_CHANNEL_FEATURE_SEEKABLE for channel file
  io: Add generic pwritev/preadv interface
  io: implement io_pwritev/preadv for QIOChannelFile
  migration/qemu-file: add utility methods for working with seekable
    channels
  migration/ram: Add support for 'fixed-ram' outgoing migration
  migration/ram: Add support for 'fixed-ram' migration restore
  tests/qtest: migration-test: Add tests for fixed-ram file-based
    migration

Steve Sistare (1):
  tests/qtest: migration events

 docs/devel/migration.rst            |  14 ++
 include/exec/ramblock.h             |   8 +
 include/io/channel.h                | 133 ++++++++++++
 include/migration/global_state.h    |   3 +-
 include/migration/qemu-file-types.h |   2 +
 include/qemu/bitops.h               |  13 ++
 include/qemu/osdep.h                |   2 +
 io/channel-file.c                   |  60 ++++++
 io/channel.c                        | 140 +++++++++++++
 migration/file.c                    | 114 ++++++++--
 migration/file.h                    |  10 +-
 migration/global_state.c            |  20 +-
 migration/migration-hmp-cmds.c      |  10 +
 migration/migration.c               |  55 ++++-
 migration/multifd.c                 | 313 +++++++++++++++++++++++-----
 migration/multifd.h                 |  14 +-
 migration/options.c                 |  79 +++++++
 migration/options.h                 |   5 +
 migration/qemu-file.c               |  80 +++++++
 migration/qemu-file.h               |   4 +
 migration/ram.c                     | 203 +++++++++++++++++-
 migration/ram.h                     |   1 +
 migration/savevm.c                  |   1 +
 qapi/migration.json                 |  26 ++-
 tests/qtest/libqtest.c              |  14 ++
 tests/qtest/libqtest.h              |  25 +++
 tests/qtest/migration-helpers.c     | 108 +++++++---
 tests/qtest/migration-helpers.h     |  10 +-
 tests/qtest/migration-test.c        | 219 ++++++++++++++-----
 util/osdep.c                        |   9 +
 30 files changed, 1520 insertions(+), 175 deletions(-)

-- 
2.35.3