[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH v7 00/13] Dirty bitmaps migration
From: |
Vladimir Sementsov-Ogievskiy |
Subject: |
[Qemu-devel] [PATCH v7 00/13] Dirty bitmaps migration |
Date: |
Wed, 11 Nov 2015 18:16:30 +0300 |
These patches provide dirty bitmap migration feature. Only named dirty
bitmaps are to be migrated. Migration may be enabled using migration
capabilities.
v7:
0001: main functions are mostly rewritten to be safer. Granularity of
serialization is now explicit thing. Only the last chunk of
bitmap data may be not aligned.
Reviewed-by deleted.
0002: changed accordingly to 0001 chnages
Reviewed-by deleted.
0004: use ctz64() instead of ffsll(), accordingly to checkpatch
Reviewed-by left. Hope, you don't mind.
0006: since 2.4 -> since 2.6
Reviewed-by's left (Eric Blake's one added).
0007: s/int/size_t for strlen result, add comment
Reviewed-by left. Hope, you don't mind.
0008:
Rename some static functions for better style:
qemu_get_flags -> qemu_get_bitmap_flags
qemu_put_flags -> qemu_put_bitmap_flags
blk_mig_reset_dirty_cursor -> dirty_bitmap_mig_reset_dirty_cursor
Logic, related to 0001 changes:
- bdrv_dirty_bitmap_data_size calls with one count parameter are
changed by bdrv_dirty_bitmap_serialization_size calls with
two parameters, specifying the range
- change dirty_bitmap_save_pending for not using
bdrv_dirty_bitmap_data_size
Rebase on master:
- rename _cancel to _cleanup and .cancel to .cleanup
- change dirty_bitmap_save_pending (postcopy parameters)
- rename .save_live_complete to .save_live_complete_precopy
Reviewed-by deleted.
also: new function hbitmap_serialization_granularity made public,
and may be used for selecting chunk size (it gives the minimum) for
migration, storing, loding. But it is not used, because, I think,
checking that CHUNK_SIZE from migration/block-dirty-bitmap.c is
greater than sizeof(long).
0011:
# @md5: md5 checksum of the last bitmap level (since 2.4)
to
# @md5: md5 checksum (as a hexadecimal string) of the last bitmap level
# (since 2.6)
Reviewed-by left. Hope, my new comment is ok and you don't mind.
0012: new patch
v6:
- rebase on master
- function qemu_get_string() deleted, used existed function
qemu_get_counted_string()
- function qemu_put_string() renamed appropriately to
qemu_put_counted_string()
(and now it is new separate patch)
=====================================================
0004: Reviewed-by: John Snow <address@hidden>
0006: (since 2.3) -> (since 2.4)
0007: new patch, mentioned above
0008: printf format fixed, DEBUG definition set back to 0, in
dirty_bitmap_live_iterate_is_active conditions swapped.
qemu_{get,put}_string removed, used ..counted.. verisions
0011: (since 2.3) -> (since 2.4)
0012: assertIsNotNone -> assertNotEqual for compitibility with
old Python
Reviewed-by: John Snow <address@hidden>
v5:
- rebase on master
- drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class
- remove rfc, as incremental backup series by John Snow are in
upstream
v4 significant changes:
0001: tiny bugfix: out[i] -> out[i-start], same for 'in'
0007: set chunk size to 1kb, disable live iteration for
migrating data < 1mb size.
tests: only one with md5 sum is here. used function event_wait
by John Snow. (I hope, you don't mind me just adding this
function with your 'Signed-off-by')
rfc: This patch set is based on v13 of
"block: incremental backup series" by John Snow, which are
not pushed yet.
v3:
based on v13 of "block: incremental backup series" by John Snow.
changes from v2:
removed patch for adding dirty parameter (migration capablities used
instead).
0001: printf's dropped, qapi used
0002: part0 -> zeroes
0003: part0 -> zeroes
0005: dirty_dirty -> meta
add comments about meta bitmap
0006: the format is changed, nodes used instead of devices.
other patches are new.
rfc: there are two tests. They are the same but using different
interfaces: md5 checksum of the bitmap last layer in query-block or
separate query-block-dirty-bitmap with dirty bitmap regions.
The second form is more appropriate for debugging, the first is more
appropriate for simple regression control. Which should go to
upstream?
v2:
1. bug-fixes, that are already in upstream, and renaming of function
bdrv_reset_dirty_bitmap (which is already in Snow's series) are
dropped
2. bitmap store/restore: the concept renamed to serialization, added
function hbitmap_deserialize_part0, to not transfer zero blocks
3. migration dirty parameter: added description comment
4. Other patches are new.
v2.rfc:
Actually, in this version of the series I'm trying not use
migration/block.c at all. Instead a separate migration unit is added
in the new file migration/dirty-bitmap.c. Now bitmaps are migrated
like blocks in block migration, they have their "dirty-dirty" bitmaps,
for tracking set/unset changes during migration.
The advantages are:
- no complications of migration/block.c
- separate dirty-dirty bitmaps provide handling of "unset's"
- more effective meta-data/data ratio - no tiny bitmap-blocks.
v1:
These patches provide dirty bitmap migration feature. Only named dirty
bitmaps are to be migrated. Migration is made as a part of block
migration in block-migration.c.
Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate
command. If "blk" and "inc" parameters are false when "dirty" is true
block migration is actually skipped: no allocatoions, no bdrv_read's,
no bdrv_write's, only bitmaps are migrated.
Vladimir Sementsov-Ogievskiy (13):
hbitmap: serialization
block: BdrvDirtyBitmap serialization interface
block: tiny refactoring: minimize hbitmap_(set/reset) usage
block: add meta bitmaps
block: add bdrv_next_dirty_bitmap()
qapi: add dirty-bitmaps migration capability
migration/qemu-file: add qemu_put_counted_string()
migration: add migration/block-dirty-bitmap.c
iotests: maintain several vms in test
iotests: add add_incoming_migration to VM class
qapi: add md5 checksum of last dirty bitmap level to query-block
iotests: add default node-name
iotests: add dirty bitmap migration test
block.c | 93 +++++-
include/block/block.h | 22 ++
include/migration/block.h | 1 +
include/migration/migration.h | 1 +
include/migration/qemu-file.h | 2 +
include/qemu/hbitmap.h | 81 +++++
migration/Makefile.objs | 2 +-
migration/block-dirty-bitmap.c | 708 +++++++++++++++++++++++++++++++++++++++++
migration/migration.c | 9 +
migration/qemu-file.c | 13 +
qapi-schema.json | 4 +-
qapi/block-core.json | 5 +-
tests/qemu-iotests/117 | 84 +++++
tests/qemu-iotests/117.out | 5 +
tests/qemu-iotests/group | 1 +
tests/qemu-iotests/iotests.py | 17 +-
util/hbitmap.c | 122 +++++++
vl.c | 1 +
18 files changed, 1163 insertions(+), 8 deletions(-)
create mode 100644 migration/block-dirty-bitmap.c
create mode 100755 tests/qemu-iotests/117
create mode 100644 tests/qemu-iotests/117.out
--
2.1.4
- [Qemu-devel] [PATCH v7 00/13] Dirty bitmaps migration,
Vladimir Sementsov-Ogievskiy <=
- [Qemu-devel] [PATCH 03/13] block: tiny refactoring: minimize hbitmap_(set/reset) usage, Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 12/13] iotests: add default node-name, Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 05/13] block: add bdrv_next_dirty_bitmap(), Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 10/13] iotests: add add_incoming_migration to VM class, Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 07/13] migration/qemu-file: add qemu_put_counted_string(), Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 11/13] qapi: add md5 checksum of last dirty bitmap level to query-block, Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 06/13] qapi: add dirty-bitmaps migration capability, Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 09/13] iotests: maintain several vms in test, Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 01/13] hbitmap: serialization, Vladimir Sementsov-Ogievskiy, 2015/11/11
- [Qemu-devel] [PATCH 13/13] iotests: add dirty bitmap migration test, Vladimir Sementsov-Ogievskiy, 2015/11/11