On 02/22/2017 03:45 AM, Peter Lieven wrote:
Am 21.02.2017 um 22:13 schrieb John Snow:
On 02/21/2017 07:43 AM, Peter Lieven wrote:
Hi,
is there anyone ever thought about implementing something like VMware
CBT in Qemu?
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1020128
Thanks,
Peter
A bit outdated now, but:
http://wiki.qemu-project.org/Features/IncrementalBackup
and also a summary I wrote not too far back (PDF):
https://drive.google.com/file/d/0B3CFr1TuHydWalVJaEdPaE5PbFE
and I'm sure the Virtuozzo developers could chime in on this subject,
but basically we do have something similar in the works, as eblake says.
Hi John, Hi Erik,
thanks for your feedback. Are you both the ones working primary on this topic?
If there is anything to review or help needed, please let me know.
I've been working on incremental backups; Fam and I now co-maintain
block/dirty-bitmap.c.
Vladimir Sementsov-Ogievskiy has been working on bitmap persistence and
migration from Virtuozzo; as well as the NBD specification amendment to
allow us to fleece images with dirty bitmaps.
(Check the wiki and the whitepaper I linked!)
Eric has been guiding the review process for the NBD side of things.
My 2 cents:
I thing I had in mind if there is no image fleecing available, but fetching the
dirty bitmap
from external would be a feauture to put a write lock on a block device.
Write lock means, drain all pending writes and queue all further writes until
unlock (as if they
were throttled to zero). This could help fetch consistent backups from storage
device (thinking of iSCSI SAN) without
the help of the hypervisor to actually transfer data (no load in the frontend
network or the host). What would further
be needed is a write generation for each block, not just only a dirty bitmap.
In this case something like this via QMP (and external software) should work:
---8<---
gen = write generation of last backup (or 0 for full backup)
do {
nextgen = fetch current write generation (via QMP)
As Eric said, there's a lot of hostility to using QMP as a metadata
transmission protocol.
dirtymap = send all block whose write generation is greater than 'gen'
(via QMP)
dirtycnt = 0
foreach block in dirtymap {
copy to backup via external software
dirtycnt++
}
gen = nextgen
} while (dirtycnt < X) <--- to achieve this a thorttling or similar
might be needed
fsfreeze (optional)
write lock (via QMP)
backupgen = fetch current write generation (via QMP)
dirtymap = send all block whose write generation is greater than 'gen' (via QMP)
foreach block in dirtymap {
copy to backup via external software
}
unlock (via QMP)
fsthaw (optional)
--->8---
As far as I understand CBT in VMware is not just only a dirty bitmap, but also
a write generation tracking for blocks (size 64kb or whatever)
I think at the moment I'm worried about getting the basic features out
the door, but I'm not opposed to adding fancier features if there's
justification or demand for them.