Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Mon, 02/29 10:42, Paolo Bonzini wrote:
> 
> 
> On 29/02/2016 09:54, Paolo Bonzini wrote:
> > 
> > 
> > On 29/02/2016 09:14, Markus Armbruster wrote:
> >> I completely agree with you that Get LBA Status cannot just reflect the
> >> top layer.  But that's not what I meant to propose.  Let me try to
> >> explain myself more clearly.
> >>
> >> Consider a QCOW2 image D (for delta) with a backing file B (for base).
> >> If you open it normally, you see "D over B".  Get LBA Status should
> >> certainly claim the "deallocated" state only for blocks that are
> >> allocated neither in D nor B.
> >>
> >> However, you can also open D *without* its backing file.  Then you see
> >> "D over nothing".  Here, get LBA Status should claim "deallocated" state
> >> for anything not allocated in D.
> > 
> > Ok, this makes more sense.
> > 
> > The question then is whether to implement this NBD server inside QEMU,
> > or outside it as a separate process to which QEMU "pushes" blocks as in
> > the existing backup job.  I would prefer the latter, so that it is
> > possible to implement various APIs (get block status, but also VMware or
> > Parallels or whatever).
> > 
> > Basically the same points made in
> > https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
> > still apply.
> 
> Talked a bit to Fam now and I noted Denis's observation that QEMU would
> still use the backup block job, plus the NBD server as in Fam's
> fleecing.  Then the NBD server is already the push->pull adapter.  It's
> a bit clearer now.
> 
> Opening D without backing file still feels a bit weird, because the NBD
> server would provide wrong data for clean blocks.  I would think that a
> "stupid" backup software could always ignore the get LBA status command
> and get a full backup.  Is this a requirement or not, and if not, why?

Can we return -EIO instead of zero for unallocated/clean sectors?

> 
> I don't have any particular opinion against an NBD get LBA status
> command that returns deallocated/allocated _and_ clean/dirty.  But
> reusing one as the other feels like the kind of hack that seems clever
> and that you regret down the road.

Yes, I second this, my preference is also leaning towards separate states for
allocation and dirtiness in "Get LBA State" command.

> 
> I'm not sure whether the clean/dirty status is data plane or control
> plane either.  I don't think the terms have a well-defined meaning in
> terms of storage.  In the networking world, stuff like routing protocols
> (OSPF, BGP, etc.) is control plane, and based on this analogy dirty
> bitmaps seem like control plane to me.
> 
> So I wouldn't rule out QMP-based export of the dirty bitmap---either
> directly or optimized through a separate socket as in
> http://article.gmane.org/gmane.comp.emulators.qemu/397083.  Dirty
> bitmaps should also compress well, so perhaps gzip+base64 over JSON
> might work as well.  I'm not saying we certainly won't regret it, but it
> seems "less different".  Don't really know how to put it better...

One more side point here: a backup archive is useless until it has fully
completed, so in this particular use case, push or pull model doesn't matter in
sending the dirty bitmap.  But while we are at it, probably just be future
proof and do a pull iterface to allow more potential use cases? For example,
during data inspecting (a fleecing client is checking whether a particular file
in the guest file system has changed), only certain ranges in the dirty bitmap
is interesting.

Fam