[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH v9 00/21] qcow2: persistent dirty bitmaps
From: |
Vladimir Sementsov-Ogievskiy |
Subject: |
[Qemu-devel] [PATCH v9 00/21] qcow2: persistent dirty bitmaps |
Date: |
Tue, 22 Nov 2016 20:26:05 +0300 |
Hi all!
There is a new update of qcow2-bitmap series - v9.
web:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v9
git: https://src.openvz.org/scm/~vsementsov/qemu.git (tag qcow2-bitmap-v9)
v9:
rebase on master!
01-04,06,07: John's r-b
03: rewording in comment ("The concurrent resetting ..."), John's r-b
07: reword error messages
commit message: dirty bitmap -> bitmap
09: fix uninitialized ret in bitmap_list_store() (fix compilation warning)
18: fix compilation of tests/test-hbitmap
also, change constants names for qcow2: DIRTY_BITMAP -> BITMAP (as it is not
dirty bitmaps extension, but just bitmaps extension), QCOW -> QCOW2 (bitmaps
are only for qcow2)
v8:
Patches 01-06 are mostly unchanged, except for spelling and wording.
02 - add Eric's r-b
03,04 - add Max's r-b
07-09 is refactored "bitmap-read" part of the feature. Main changes:
- introduce bitmap list - normal list, which represents bitmap directory.
Function bitmap_list_load loads bitmap directory, checks everything and
creates normal list.
- introduce .bdrv_load_autoloading_dirty_bitmaps : instead of loading bitmaps
in qcow2_open, lets load them only in bdrv_open, to avoid reloading in
bdrv_snapshot_goto
- do not delete bitmaps after read, but mark them IN_USE
10 has contextual changes and rewording of comment. I've added Max's r-b, hope
it's ok.
11: add error_report("Persistent bitmaps are lost") in case of failed bitmap
store
12: add Max's r-b
13 is refactored "bitmap-store" part of the feature. see 07-09 description
- for now I just free old clusters and allocate new. This will be improved
with a separate patch.
patch about "delete bitmaps on truncate" is removed. This case will be handled
later.
IN_USE, autoclear, check-constraints things are merged into other patches.
14-15 are mew patches, to early check possibility of creating persistent bitmap
with
specified name and granularity in specified BDS
16-17: spelling, rewording, indenting, tiny code simplifying, check can_store
(by 14-15),
s/if (autoload && !persistent)/if (has_autoload && !persistent)/,
rebase (deleted qmp-commands.hx)
18: alternative to md5 in query-block:
- separated qmp command -x-debug-block-dirty-bitmap-sha256
- as adviced by Daniel P. Berrange in my parallel thread:
- sha256 instead of md5
- use qcrypto_hash_* instead of GChecksum
- fix bug =) (size was wrong in hbitmap_md5)
19: s/3999/3fff/, use x-debug-block-dirty-bitmap-sha256
20: new patch to rename and publish inc_refcounts
21: some fixes and refactoring mostyly by Max's comments.
Max, Eric, great tanks for your review!
I hope, I've covered most of your comments by this update.
TODO:
- handle reopening image RO->RW and incoming migration, set IN_USE for existing
loaded bitmaps
in these cases.
- reuse old, already allocated data clusters for bitmaps storing
- truncate bitmaps in the image on truncate
v7:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v7
based on block-next (https://github.com/XanClic/qemu/commits/block-next)
- a lot of refactoring and reordering of patches.
- dead code removed (bdrv_dirty_bitmap_load, etc.)
- do not maintain extra data for now
- do not store dirty bitmap directory in memory
(as we use it seldom, we can just reread if needed)
By Kevin's review:
01 - commit message changed: fix->improvement (as it was not a bug)
03 - r-b
04 - r-b
05 - add 21 patch to fix spec, also, removed all (I hope) mentions of
"Bitmap Header", switch to one unified name for it - "Bitmap
Directory Entry", to avoid misunderstanding with Qcow2 header.
(also, add patch 22, to fix it in spec)
v6.06 - improve for_each_dir_entry loop, reorder patches, other small fixes
v6.07 ~> v7.09 - dead code removed, I've moved to one function
.bdrv_store_persistent_bitmaps and have wrapper and callback in one
patch (with also some other staff. If it not ok, I can split them)
v6.08 - about keeping bitmap directory instead of bitmap list: no I don't keep
it at all.
v6.16 - old bdrv_ bitmap-storing related functions are removed. The new one is
bdrv_store_persistent_bitmaps.
v6:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v6
based on block-next (https://github.com/XanClic/qemu/commits/block-next)
There are a lot of changes, reorderings and additions in comparement with v5.
One principal thing: now bitmaps are removed from image after loading instead
of marking them in_use. It is simpler and we do not need to store superfluous
data.
Also, we are no more interested in command line interface to dirty bitmaps.
So it is dropped. If someone needs it I can add it later.
Vladimir Sementsov-Ogievskiy (21):
specs/qcow2: fix bitmap granularity qemu-specific note
specs/qcow2: do not use wording 'bitmap header'
hbitmap: improve dirty iter
tests: add hbitmap iter test
block: fix bdrv_dirty_bitmap_granularity signature
block/dirty-bitmap: add deserialize_ones func
qcow2: add bitmaps extension
block: introduce auto-loading bitmaps
qcow2: add .bdrv_load_autoloading_dirty_bitmaps
block/dirty-bitmap: add autoload field to BdrvDirtyBitmap
block: introduce persistent dirty bitmaps
block/dirty-bitmap: add bdrv_dirty_bitmap_next()
qcow2: add .bdrv_store_persistent_dirty_bitmaps()
block: add bdrv_can_store_dirty_bitmap
qcow2: add .bdrv_can_store_dirty_bitmap
qmp: add persistent flag to block-dirty-bitmap-add
qmp: add autoload parameter to block-dirty-bitmap-add
qmp: add x-debug-block-dirty-bitmap-sha256
iotests: test qcow2 persistent dirty bitmap
qcow2-refcount: rename inc_refcounts() and make it public
qcow2-bitmap: refcounts
block.c | 67 +++
block/Makefile.objs | 2 +-
block/dirty-bitmap.c | 63 ++-
block/qcow2-bitmap.c | 1230 ++++++++++++++++++++++++++++++++++++++++++
block/qcow2-refcount.c | 59 +-
block/qcow2.c | 111 +++-
block/qcow2.h | 39 ++
blockdev.c | 64 ++-
docs/qmp-commands.txt | 7 +
docs/specs/qcow2.txt | 8 +-
include/block/block.h | 5 +
include/block/block_int.h | 7 +
include/block/dirty-bitmap.h | 18 +-
include/qemu/hbitmap.h | 49 +-
qapi/block-core.json | 37 +-
tests/Makefile.include | 2 +-
tests/qemu-iotests/165 | 89 +++
tests/qemu-iotests/165.out | 5 +
tests/qemu-iotests/group | 1 +
tests/test-hbitmap.c | 19 +
util/hbitmap.c | 51 +-
21 files changed, 1872 insertions(+), 61 deletions(-)
create mode 100644 block/qcow2-bitmap.c
create mode 100755 tests/qemu-iotests/165
create mode 100644 tests/qemu-iotests/165.out
--
1.8.3.1
- [Qemu-devel] [PATCH v9 00/21] qcow2: persistent dirty bitmaps,
Vladimir Sementsov-Ogievskiy <=
- [Qemu-devel] [PATCH 04/21] tests: add hbitmap iter test, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 05/21] block: fix bdrv_dirty_bitmap_granularity signature, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 01/21] specs/qcow2: fix bitmap granularity qemu-specific note, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 07/21] qcow2: add bitmaps extension, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 02/21] specs/qcow2: do not use wording 'bitmap header', Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 03/21] hbitmap: improve dirty iter, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 16/21] qmp: add persistent flag to block-dirty-bitmap-add, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 14/21] block: add bdrv_can_store_dirty_bitmap, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 10/21] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap, Vladimir Sementsov-Ogievskiy, 2016/11/22
- [Qemu-devel] [PATCH 12/21] block/dirty-bitmap: add bdrv_dirty_bitmap_next(), Vladimir Sementsov-Ogievskiy, 2016/11/22