Re: [Qemu-devel] Incremental backup call

[Kashyap Chamarthy]

On Tue, Feb 16, 2016 at 02:02:58PM +0000, Stefan Hajnoczi wrote:
> Hi,
> There are several ongoing efforts to implement incremental
> backup-related features.
> 
> Let's have a voice/video conference to get everyone on the same page,
> avoid duplicated work, and get patches merged faster.
> 
> Agenda:
> 
>  * External incremental backup API.  Summarize requirements common to
> third-party backup appliances and agree on suitable API direction.
> 
>  * Persistent dirty bitmaps.  Agree on how qcow2 and/or qbm will hold
> bitmaps in various use cases.
> 
>  * Anything else?

I was in a listen-only mode on the call, here's some quick minutes
(read: haphazard scribbling) so please excuse grammar
thinkos/typos/missed points.  Feel free to correct/adjust.

-----------------------------------------------------------------------
- [denis-lunev] Migration support for dirty bitmaps?
- [stefanha] One discussion that might need to reset things is Fam's
  QEMU QBM (Qemu Bit Map)
- [stefanha] Incremental backups _should_ be possible with RAW
- [jsnow] Apart from just from having incremental backups for RAW
  devices, any reason why qcow2 _cannot_ be the bitmap provider?
- [kwolf] What are the actual requirements?
- [stefanha] Vladimir's series that modified qcow2 - the idea was that
  you don't _have_ to use qcow2 format, but you could use it as a bitmap
  container for other images.
- [kwolf] Something about VM state internal snapshots: where the VM
  state is saved into qcow2 file:
   - Don't store bitmaps in a qcow2 file
   - Stack the qcow2 on the specific image that the bitmap is for
   - [stefanha] What happens when there's guest I/O (?) 
- [kwolf] The question is: if it wouldn't be simple to extend qcow2 - to
  forward all writes to the backing file
- If we _don't_ want to use qcow2 to store this:
- [kwolf] Concern is consistency: doing one thing for qcow2; and other
  thing for other formats?
- [eblake] This is not the first time we're adding something on top of
  raw images
   - [eblake] bitmap on top of RAW - this concept has been floating on
     the list for several years
      - https://lists.nongnu.org/archive/html/qemu-devel/2013-08/msg03682.html
      - [kwolf] What is the use case?
         - Backing files with raw images
         - If you have a raw image file, you can use backing files with
           it; if at some point in time
- [eblake] Mentioned LUKS encryption of danpb (in comparision with QBM
  work?)
   - A seperate driver (general purpose) 'luks' format driver which can
     be layered over any other existing block driver.
   - Embedded LUKS inside qcow2
      - it's encrypted as _part_ of the qcow2 file
   - A general purpose 'luks' format driver which can be layered over
     any other existing block driver.
   - To differentiate intentionally encrypted as part of qcow2 vs. the
     guest data
   - [kwolf] One important difference: Dan didn't introduce a new file
     _format_ for qcow2.
  - [eblake] In a sense, LUKS _is_ a new file format - but indeed it's
    interoperating with existing format

- [stefanha] Is the least controversial part: getting the Qcow2 support
  in that Vladimir is working on?
- [kwolf] Concern: is the relationship with QBM -- I don't want to end
  up with duplicated things at the end
   - Once we accept this API, we can't change it
- [denis-lunev] A lot of resources/time has been invested in this.
  Specification about bitmaps.  Version-17 for this spec is too much.
- [kwolf] If you go forward with qcow2 path, then, we're not going to do
  the QBM approach.
- [fam] Qcow2 extension has made its progress
   - We should go ahead and make the extension in qcow2 and see how it
     goes
   - People will start complaining about missing features in raw (?)
   - Concern: in certain protocols like Ceph, people would tend to use
     Raw images, without any tools on top
- [stefanha] If we do what Kevin told: qcow2 will have a node - where
  the writes will be forwarded to backing file, what's holding up there
  (?)
   - You have a qcow2 file that doesn't have much data, except metadata
     and including bitmap
- [jsnow] We should move forward with qcow2 persistence approach.
   - We can re-discuss the merits of duplication again
- [stefanha] Question for Eric: In terms of committing an API, client
  applications, external plugins for backup/ related libvirt work?
   - [eblake] From libvirt's perspective, the biggest thing we need for
     2.6 is having QMP 'blockdev-add' for everything:
      - We want to get Ceph, NBD and Gluster all of those under QAPI, to
        be able to programatically address them from libvirt
         - We currently sometimes fallback to HMP
      - [kwolf] I don't think it's possible for 2.6 yet.  So far, we
        have NBD support on the list
      - [eblake] Going to look at 'blockdev-add' to see what's missing
        and what needs to be added
- [eblake] Ability to track persistent node names in libvirt XML
    - We're at the point now that every node creation can have a name.
    - libvirt does have some work to do take advantage of node names,
      remembering what names it has assigned
-----------------------------------------------------------------------


-- 
/kashyap