Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Daniel P. Berrange]

On Wed, Aug 30, 2017 at 02:36:11PM +0100, Stefan Hajnoczi wrote:
> On Tue, Aug 22, 2017 at 03:07:04PM -0400, John Snow wrote:
> > (3) Add either a new flag that turns qcow2's backing file into a full
> > R/W backing file, or add a new extension to qcow2 entirely (bypassing
> > the traditional backing file mechanism to avoid confusion for older
> > tooling) that adds a new read-write backing file field.
> > 
> > This RW backing file field will be used for all reads AND writes; the
> > qcow2 in question becomes a metadata container on top of the BDS chain.
> > We can re-use Vladimir's bitmap persistence extension to save bitmaps in
> > a qcow2 shell.
> > 
> > The qcow2 becomes effectively a metadata cache for a new (essentially)
> > filter node that handles features such as bitmaps. This could also be
> > used to provide allocation map data for RAW files and other goodies down
> > the road.
> > 
> > Hopefully this achieves our desire to not create new formats AND our
> > desire to concentrate features (and debugging, testing, etc) into qcow2,
> > while allowing users to "have bitmaps with raw files."
> > 
> > Of course, in this scenario, users now have two files: a qcow2 wrapper
> > and the actual raw file in question; but regardless of how we were going
> > to solve this, a raw file necessitates an external file of some sort,
> > else we give up the idea that it was a raw file.
> 
> There is some complexity here for management tools:
> 
> If the underlying image is resized, who resizes the qcow2 and how do
> they know to do it?
> 
> If QEMU's resize/truncate command it used, does first try to resize the
> underlying image and then resize the qcow2?  This is probably the sanest
> approach.
> 
> If the underlying image is moved to a new location, does the qcow2 file
> need to be modified and who does that?
> 
> Management tools need to figure out how to represent manage this extra
> qcow2 file.  The easiest solution is to punt it to the user and treat it
> as part of a backing file chain.  If the management tool wants to
> automatically manage the qcow2 so the user just specifies the underlying
> image and enables the persistent bitmap checkbox, then it becomes more
> complicated.

Indeed, I don't think it is practical to have libvirt / QEMU automagically
create a qcow2 overlay on disk. Something has to decide where this would
be stored. You might say just put it alongside the raw file, but it might
not be a local file at all, it could be a NBD, or RBD raw "file". So do
we create  local qcow2 file, or store a qcow2 file inside another RBD
volume to hold the persistent bitmap. This kind of decision needs to be
made by the mgmt app since only it knows about its storage mgmt model.
At this point you might as well just let the mgmt app take care of it
all and not try to do anything magical with qcow2 overlays in libvirt/QEMU

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|