Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Daniel P. Berrange]

On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> >On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
> >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> >>>>On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> >>>>>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >>>>>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>>>>>>Hi all.
> >>>>>>>
> >>>>>>>These series which aims to add external backup api. This is needed to 
> >>>>>>>allow
> >>>>>>>backup software use our dirty bitmaps.
> >>>>>>>
> >>>>>>>Vmware and Parallels Cloud Server have this feature.
> >>>>>>What is the advantage of this appraoch over "drive-backup 
> >>>>>>sync=incremental
> >>>>>>..."?
> >>>>>This will allow third-party vendors to backup QEMU VMs into
> >>>>>their own formats or to the cloud etc.
> >>>>As an example, there is a third-party backup format called VMA from
> >>>>Proxmox.  A few years ago I posted a proof-of-concept external backup
> >>>>tool in Python:
> >>>>
> >>>>https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> >>>>
> >>>>It takes a full backup using drive-backup NBD (plus RAM/device state)
> >>>>but the same can be done with incremental backups.
> >>>>
> >>>>Does this NBD approach meet your requirements?
> >>>>
> >>>>Stefan
> >>>for us we should somehow provide implementation of
> >>>calls posted by Vladimir. They are available in Parallels Server
> >>>version 6 and should be available in the next QEMU based
> >>>release using "Parallels SDK to libvirt" convertor. The problem
> >>>for us is that this old approach is used in the other side
> >>>of the product - in containers implementation while this
> >>>SDK is a universal access tool to both things.
> >>Point taken.  I think many other backup applications will expect a
> >>similar API, so it's pragmatic to provide something compatible.
> >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> >concerns about the QMP monitor:
> >
> >Previously I described incremental backup in "push" mode (already
> >supported today with drive-backup).  QEMU connects to a remote NBD
> >server and writes out the contents of all dirty blocks:
> >
> >   QEMU ---Write dirty blocks--> Backup appliance (server)
> >
> >This doesn't lend itself well to existing backup applications that
> >expect to iterate the dirty bitmap manually.
> >
> >Let's add a "pull" mode where the connection of the NBD connection is
> >reversed.  The backup application connects to QEMU's NBD server (image
> >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> >Status command for querying block provisioning information.  Now the
> >backup application can use Get LBA Status to fetch the dirty block
> >information from QEMU.
> >
> >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
> >
> >The dirty block information goes over the same NBD connection used to
> >read the contents of the dirty blocks.  The QMP monitor is not used to
> >transfer dirty block information.
> >
> >It may be necessary to extend the nbd-server-add command so that a
> >bitmap name can be passed.  This bitmap will be used to answer Get LBA
> >Status queries instead of using on bdrv_co_get_block_status().  This
> >would be necessary if image fleecing (point in time snapshot) is used.
> >
> >Stefan
> 
> There are no such commands in nbd spec here:
> 
> https://github.com/yoe/nbd/blob/master/doc/proto.md
> 
> 
> So, I'm not sure, that adding something qemu-specific to this external
> protocol will be simple or even true way. Is Qemu already extending original
> nbd?

No, we don't do any QEMU specific extensions. The point of the approach
Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
concept, it is relevant to any NBD server implementation.

For example, consider you were using a regular NBD server to export a
sparse LVM volume. This proposed extension would be relevant to such
a use case. As such this proposed extension is something that is likely
to be acceptable for the generic NBD specification.

> What about exporting bitmap as separate nbd entity? Just implement block
> driver, which will read from bitmap? If consider only read access and
> disabled (or frozen) bitmaps it should be simple enough.

IMHO that's not a desirable idea, since that *is* creating a QEMU specific
solution to the problem.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|