|
From: | Denis V. Lunev |
Subject: | Re: [Qemu-devel] [PATCH v2 0/6] external backup api |
Date: | Wed, 10 Feb 2016 11:04:32 +0300 |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 |
On 02/09/2016 10:25 PM, Denis V. Lunev wrote:
On 02/09/2016 09:12 PM, John Snow wrote:On 02/09/2016 11:58 AM, Denis V. Lunev wrote:On 02/09/2016 07:49 PM, John Snow wrote:On 02/09/2016 09:37 AM, Denis V. Lunev wrote:On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:On 02/03/2016 11:14 AM, Fam Zheng wrote:On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:Hi all.These series which aims to add external backup api. This is neededto allow backup software use our dirty bitmaps. Vmware and Parallels Cloud Server have this feature.What is the advantage of this appraoch over "drive-backup sync=incremental ..."?This will allow third-party vendors to backup QEMU VMs into their own formats or to the cloud etc.Backup software can implement NBD to receive the incremental blocks from QEMU. Use drive-backup sync=incremental and the backup appliance as the NBD target.It's more complicated to add this QMP command flow to backup softwarethan to implement NBD. Stefanit can, but this is a matter of problem due to the nature of how this software is implemented. Usually it is written in a semi-standard way and it uses "plugins" to actually collect the data, i.e. the code is written in standard interface/real implementation pattern and interfaces are basically the same. With this standard approach backup software is working as an active side of the process, i.e. it performs operations and controls the flow. This means that "non-standard" QEMU technology will be pain here. DenAm I to understand that for e.g. VMWare the backup appliance is literally reading the disk image from storage directly while the VM is running? I'd be a bit surprised if that were true.I think that backup software is asking alive VM about the data.My biggest concern here is that there is not a safe way, today, to readfrom a disk image atomically while the VM is running. I think that'd take a lot of work to do and you'll not find a lot of support in implementing it.Of course, while the VM is paused/off is a different story, but for nowI still feel like NBD is the right answer for getting block data from QEMU. What am I missing? --jsIn general, in Parallels Server the backup was created using the following approach: - create external snapshot. In this case the base image (backing store in QEMU terminology) will be READ-ONLY and could be safely read by any entity - backup that read-only disk image (any way) - merge snapshotsI see.In this process backup software is active while PCS is passive.PCS?Parallels Cloud Server. Sorry for abbreviation :(With QEMU the approach looks the same to me: - start a backup - ask QEMU to give a data to be backuped (using NBD server in QEMU with Fam's image fleecing) - finish backup Important bit here is that dirty bitmap should be provided by QEMU by request. This dirty bitmap will be read-only at that moment, current active dirty bitmap should be replaced with new at backup start operation. DenI don't have any problems providing the bitmap data through an external API, but the part I want to be 100% clear on before I ACK it is the API portion where we allow an external client to split or merge bitmaps externally -- that's functionality you don't need if you query the data from QEMU itself.That is fine :)
OOPS. This could be mistake! The client should not perform operations on the bitmap manually, but it should instruct QEMU to do this job for him. Bitmap child should be created by request of backup software and its existence should also be controlled by the backup software. This could be either hidden or not, but the state should be controlled. Lets again discuss the sequence: 1. call guest-fsfreeze-freeze to make consistent backup. This means that the guest ensures that all its journals are committed and there are no pending writes. Filesystem is in a good state. 2. start backup. At this moment original dirty bitmap child should be created to track new writes. There are no writes from the guest thanks to step (1). Thus we could avoid to move QEMU to VM_PAUSED state to perform the operation. This requires to 2.1 start image fleecing 2.2 create bitmap child (successor) 3. call guest-fsfreeze-thaw to unfreeze the guest 4. get dirty bitmap (parent). It is read-only and could be safely extracted 5. collect data from QEMU using NBD server inside QEMU (image fleecing) 6a. finish backup successfully. This implies the following ops: 6a.1 remove fleecing image 6a.2 drop old dirty (parent) bitmap (abdicate) 6b. finish backup unsuccessfully: 6b.1 remove fleecing image 6b.2(a) merge child to parent (reclaim) or alternatively 6b.2(b) drop both child and parent dirty bitmaps and stop tracking to facilitate new full backup next round Constraints: - step (4) is slow. We do not want to do it before step (3) - paired operations aka (6.1)/(6.2) and (2.1)/(2.2) could be done either by the single QMP command or by two different commands, we do not care much on this There are also troubles on guest snapshot operations. For now we drop dirty bitmaps in Parallels Server on switch-to-snapshot operations as it is unclear how to properly make dirty bitmap for the case. Den
[Prev in Thread] | Current Thread | [Next in Thread] |