As has been requested, this series adds new overlap check functions to
the qcow2 code. My local branch is called "qcow2-improved-overlap-v1",
but I am not so sure whether it is actually an improvement; that is left
for you to decide, dear reviewers.
See patch 1 for an explanation of why this series exists and what it
does. Patch 1 is basically the core of this series, the rest just
employs the functions introduced there.
In a later patch, we may want to change the meaning of the "constant"
overlap checking option to mean the same as "cached", which is
everything except for inactive L2 tables. This series does make
checking for overlaps with inactive L2 tables at runtime just as cheap
as everything else (constant time plus caching), but using these checks
means qemu has to read all the snapshot L1 tables when opening a qcow2
file. This does not take long, of course, but it does result in a bit of
overhead so I did not want to enable it by default.
I think just enabling all overlap checks by default after this series
should be fine and useful, though.
For benchmarks, please see my cover letter for v2:
http://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg03430.html
I'll just quote the (fixed) tl;dr here:
* CPU usage at runtime decreased by 15000 to 27500 percent on
overlap-check-heavy tasks
* No additional performance problems at loading time (in theory has the
same runtime complexity as a single overlap check right now; in
practice I could not find any problems)
* Decent RAM usage (170 kB for a 1 TB image with 64 kB clusters; 170 MB
for a 1 PB image etc. pp.)
v3:
- Patch 1:
- s/few memory/little memory/ [Eric]
- s/2014/2015/ [Eric]
- s/of a or a/of or a/ [Eric]
- Use nb_clusters_minus_one instead of nb_clusters, because
nb_clusters == 0 is invalid [Eric]
- Drop cached_windows_index [Stefan]
- Reset current_nb_clusters to 1 when a new range starts in the
algorithm which rebuilds the fragment list from the bitmap
representation
- Patch 3: Added a TODO comment that the user should be able to override
the default cache size limit
- Patch 8: As the patch "qcow2: Buffer L1 table in snapshot refcount
update" has been dropped, this patch was changed to no longer rely on
it
- Patch 11: Replaced the size limit for a snapshot's L1 table by
QCOW_MAX_L1_SIZE (was: INT_MAX / sizeof(uint64_t)) [Eric]
git-backport-diff against v2:
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream
patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences,
respectively
001/12:[0026] [FC] 'qcow2: Add new overlap check functions'
002/12:[----] [--] 'qcow2: Pull up overlap check option evaluation'
003/12:[0001] [FC] 'qcow2: Create metadata list'
004/12:[----] [--] 'qcow2/overlaps: Protect image header'
005/12:[----] [--] 'qcow2/overlaps: Protect refcount table'
006/12:[----] [--] 'qcow2/overlaps: Protect refcount blocks'
007/12:[----] [--] 'qcow2/overlaps: Protect active L1 table'
008/12:[0002] [FC] 'qcow2/overlaps: Protect active L2 tables'
009/12:[----] [--] 'qcow2/overlaps: Protect snapshot table'
010/12:[----] [--] 'qcow2/overlaps: Protect inactive L1 tables'
011/12:[0002] [FC] 'qcow2/overlaps: Protect inactive L2 tables'
012/12:[----] [--] 'qcow2: Use new metadata overlap check function'
Max Reitz (12):
qcow2: Add new overlap check functions
qcow2: Pull up overlap check option evaluation
qcow2: Create metadata list
qcow2/overlaps: Protect image header
qcow2/overlaps: Protect refcount table
qcow2/overlaps: Protect refcount blocks
qcow2/overlaps: Protect active L1 table
qcow2/overlaps: Protect active L2 tables
qcow2/overlaps: Protect snapshot table
qcow2/overlaps: Protect inactive L1 tables
qcow2/overlaps: Protect inactive L2 tables
qcow2: Use new metadata overlap check function
block/Makefile.objs | 3 +-
block/qcow2-cluster.c | 13 ++
block/qcow2-overlap.c | 396
+++++++++++++++++++++++++++++++++++++++++++++++++
block/qcow2-refcount.c | 202 ++++++++++---------------
block/qcow2-snapshot.c | 105 ++++++++++++-
block/qcow2.c | 131 ++++++++++------
block/qcow2.h | 13 ++
7 files changed, 691 insertions(+), 172 deletions(-)
create mode 100644 block/qcow2-overlap.c