Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Max Reitz]

On 2017-08-29 03:18, John Snow wrote:
> 
> 
> On 08/25/2017 09:44 AM, Max Reitz wrote:
>> On 2017-08-25 02:55, John Snow wrote:

[...]

>>> In (1), we avoid saving or specifying the relationship between these two
>>> data stores in any way. This is certainly easy to do, and will save us
>>> some headache on the CLI. As a downside, we now have random orphaned
>>> files that aren't very interesting or useful on their own. The
>>> likelihood for desync between metadata and data increases. The use of
>>> management software is all but necessitated.
>>>
>>> In (2) We have to now specify, with a dizzying long list of
>>> possibilities, the location of the implementation data. qcow2 only has a
>>> filename for backing files presently, but this is likely inadequate.
>>> What if the data store isn't a locally kept file? What if it's a socket,
>>> or a stream, or literally anything else?
>>
>> I don't see the difference.  In (1), your data image gets a "bitmap" or
>> "metadata" child.  In (2), your qcow2 image gets the usual "backing"
>> child, or maybe call it "passthrough" or whatever, if you want to make
>> the difference more explicit than just passing an option to the qcow2
>> image to pass writes to its backing file.
>>
> 
> The difference in the relationship in-memory is actually kind of
> uninteresting.
> 
> The difference as I see it is primarily how we specify the relationship
> between the qcow2 and the implementation storage; in (1) It's defined
> on-disk, and in (2) It's defined via CLI only, so
> 
> (1) Incurs a cost of having to define the link syntax (possibly causing
> a rather qemu-specific syntax), and
> (2) Avoids that cost, but leaves the data on-disk unrelated, which you hate.

What I meant was that in the end both have some parent-child
relationship in the BDS graph, so you can freely define both on the CLI.
So for (1), it usually is defined on-disk, yes, but you can also define
it on the command line -- just like it works for backing files.

>>> We'd have to develop a new syntax for specifying these resources that
>>> can be stored in a qcow2 file,
>>
>> It's called the json-pseudo-protocol and was developed exactly for this.
>>
> 
> That's what I was hinting at for "or otherwise co-opt an existing
> syntax" but I was unaware that it was intended for "exactly" this.
> 
> Do we actually use it in any on-disk format, currently? qcow2 only lets
> you specify simple filenames in the qcow2 metadata, right?

I don't think it's specified.  And I'm pretty sure that people would
agree that e.g. HTTP URLs are fine to specify in the backing link field.
 So I think that non-simple filenames are fine, too, but the question of
course is where to draw the line.

I'd argue that every time your start to allow filenames to have
protocols, you end up with some that not all programs can interpret;
like the json: protocol.

>>>                                or otherwise co-opt an existing syntax
>>> in-use by QEMU. This syntax would likely be useful only to QEMU, which
>>> would steer the qcow2 format in a direction not too useful by other
>>> emulators, and qcow2 is an open format, so we may want to avoid this.
>>
>> Storing a file name in the backing link field that cannot be interpreted
>> by other programs is in my opinion still very much better than not
>> storing any information whatsoever, because in the former case other
>> programs can at least say "sorry, I have no idea what this means" (or
>> maybe they can indeed interpret it, who knows), whereas in the latter
>> they may not even know that the qcow2 image is incomplete.
>>
> 
> I don't disagree personally, but I seem to recall that Kevin was adamant
> that the qcow2 bitmap extension should remain useful and semantically
> meaningful to third parties, so I try to keep that in mind. Maybe I
> should let him chime in instead of try to "concern troll" my own
> suggestions into the ground.

I seem to recall that I was adamant, too. :-)

All I'm saying is if you allow qcow2 to have writable backing files, it
doesn't make it any less meaningful to 3rd party programs than the
backing link field currently is.

>> Also note that we are making an effort to be able to generate plain file
>> names (such as URLs which should be usable by other programs) whenever
>> possible.
>>
> 
> Noted. Do we have a useful discriminator anywhere that allows us to
> easily check if a filename/locator/URI/whatever is in an accepted
> format, or if we still have QEMU-specific garbage?

Are NBD URLs qemu-specific? :-)

I think protocol URLs are just always icky.  If it's json: (when
bs->exact_filename is empty), you can be sure it's qemu-specific --
otherwise, it may be.

> We could always just disallow QEMU-specific protocol-talk from getting
> written and allow this only for configurations that QEMU understands to
> be universal...

I'd say it's too late for that, and also that json: was introduced
precisely to be able to get full BDS graph configuration into a backing
filename field.

[...]

>> OTOH, there is one issue I have with the R/W backing approach: Every
>> request to the raw file has to go through the qcow2 layer.  And since
>> you probably want to use raw because of the speed, this is not so nice.
>>
> 
> We probably could fix that by changing the relationship so that it isn't
> *really* a backing file, but that maybe creates other problems.

Yes.  This will be reflected in the BDS graph relationship, and it's
just weird that sometimes a qcow2 backing file would be loaded as its
parent.

Max

signature.asc
Description: OpenPGP digital signature