qemu-block
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-block] [Qemu-devel] [PAT​CH 03/17] spec: add qcow2-dirty-bitma


From: Max Reitz
Subject: Re: [Qemu-block] [Qemu-devel] [PAT​CH 03/17] spec: add qcow2-dirty-bitmaps specification
Date: Fri, 9 Oct 2015 20:14:01 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0

On 08.10.2015 22:56, Denis V. Lunev wrote:
> On 10/08/2015 11:28 PM, John Snow wrote:

[...]

>> That's about all of the thoughts I have on the matter currently.
>> Does anybody else have strong feelings on where we should go from here?
>>
>> (A) Argue with Max and push for qcow2-as-container
>> (B) Use qcow2 for self-reference bitmaps only, use an external format
>> for formats that do not support .bitmap_load or .bitmap_store
>> (C) Forget about the qcow2 extension entirely, use only the new external
>> format
>> (D) Something else?
>>
>> My vote is for (B), and if I can find a bit of consensus on that, we can
>> draft an internal-use specification for the file, but I am very wary of
>> how we will manage de-sync or if we will be able manage it at all.
>> ("Your fault for touching this file when QEMU was not running.")
>>
>> Thoughts?
>>
>> --js
> the better way is (A) if possible at all but we can follow (B)
> if (A) is not possible at all.

Well, arguing and pushing is always possible, the question is whether
it's more effort and whether it'll succeed.

Kevin has not participated in this discussion yet, but I seem to
remember that he wasn't happy about the specific wording “using qcow2 as
tar” either. Him being the qcow2 maintainer means that he'll have the
final say.

> At least we know what to do. Frankly speaking the only sad
> really necessary to support format is raw image which does
> not have obvious container to keep the bitmap.

I'm not opposed to only support qcow2, you and John are. :-)

> Here are some arguments which could be valuable or may
> be not valuable to Max.

Any honest argument is valuable.

> We have to have a bitmap inside QCOW2 file for a reasons
> listed above by John. They are really valuable. For the time
> being we were able to keep a lot of binary data inside the
> image and VM management was really quite simple. We have
> just to copy the image from one host to another. It seems
> important to me to keep this feature rolling. Thus the bitmap
> will stay inside QCOW2 image.

As long as the bitmap directly relates to the qcow2 file, i.e. its
entries describe the dirtiness of clusters in the qcow2 file, all good.

> Actually all later things are a matter of external API. Would
> we allow to create image without data or not. End-users
> will try to fake us with all their brains to save dirty bitmap
> if bitmap based backup will become useful and if they will
> not use QCOW2.

Well, the first argument would be “use qcow2”. That was the topic of
Kevin's and my talk at KVM Forum: If you need features only qcow2
provides, you should just use qcow2.

There is not a significant performance difference between preallocated
qcow2 and raw images. Thus, if you want raw performance and need dirty
bitmaps, you can just create a preallocation=metadata qcow2 image and
that's it.

Other than that: What do you mean by “fake us with all their brains”? It
just wouldn't be possible if we didn't allow foreign bitmaps in qcow2
files and didn't have an external container format for non-qcow2 files.

Maybe they can try to press us with all their might, but first I'd still
consider the “use preallocated qcow2 images” to be sufficient, and
second, well, if they press hard enough, that'd be enough to devise and
external container format.

> Are we stopping the train at full speed using a sheet of paper
> or not, preventing to create such files or not is a real question.
> Any other extra different external format will costs us a LOT of
> efforts

Will they really? The greatest facilitation I can see one gains by
putting it all into qcow2 is that you can rely on features already
present in the block layer.

The implementation would be ugly, but other than that I fail to see a
problem right now. It'd be ugly because there's no infrastructure we can
use to put a general tar driver in; so we'd have to fuse the tar driver
with the bitmap storage driver. But even that doesn't seem too bad to me.

>         and I do not see a volunteer who will perform this job
> and this is an unfortunate side of things.

I'd probably take up the challenge if I didn't have close to a hundred
patches in flight on qemu-devel, and if I weren't working part time.

I think it would be fine to implement the feature for qcow2 only for
now. Later, we can do one of three things:

(1) Add the “proprietary” external container format for non-qcow2
    images.

(2) Add foreign bitmaps to qcow2 as envisioned in this series.

(3) Make qcow2 that proprietary external container format. That means,
    these files would technically be qcow2 files, but we would modify
    them in a way that they cannot be opened by programs other than qemu
    (e.g. make the version number -1). These files would not contain an
    L1 table, but only bitmap data for foreign images.

Way (3) would be a compromise I'd still oppose but not as strongly as
way (2).

Way (2) is abuse without acknowledging it. Way (3) at least shows that
we know that what we're doing isn't right and that no program other than
qemu can work with these files, even though qcow2 is supposed to be an
open format.

> From my side I am really uncomfortable to drop the work
> performed by Vladimir for a lot of reasons

There is no need to drop it. For qcow2 it's fine. The question remains
whether it is for other formats, and I don't think it is as it is.

>                                            and one of
> them is time frame. We have already spent around 9 months
> of work to get here. I am feeling like a farther :)

I can relate, a lot of the patches I have on qemu-devel I have worked on
for close to a year. But still, if I don't like it I won't say I do just
because it has been a lot of work.

> Max, do you have the force with you to drive creation of this
> new format stuff?

Yes, I do. I'm so opposed to dumping binary blobs into regular qcow2
files that if need be I'd put all other stuff I have going on on hold
and implement it myself.

That is, unless e.g. Kevin is completely fine with it. He's the
maintainer, he has to decide.

> Anyway, we all have written several really lengthy letters.
> May be it would be wise to discuss things verbally somehow?

I don't know. We have written lengthy letters because I didn't notice
the initial discussion, I suppose. That's why I felt I had to make my
point very clear why I'm opposing at a point when a consensus between
some people had already been reached and patches existed.

If you feel like a verbal discussion (IRC? Phone?) might be better than
just emails, I'm in. But I feel it will mostly be me saying “I don't
want it” and you saying “But it's been so long and it's the simplest way
to implement it”.

To me, it looks like we acknowledge each other's arguments, but we weigh
them differently. I think a qcow2 file which we pretend is a standard
qcow2 file but of no use to anyone but qemu is not a qcow2 file, and
that this therefore is nothing we can reasonably add to the
specification. I don't think having the code for it and having worked on
it for a long time is a very strong argument.*

You on the other hand do consider the effort and time already spent a
very strong argument, but don't consider keeping qcow2 open as strong as me.

I don't know whether a verbal discussion will help us with that.
Ideally, we want to find a compromise with which both of us are happy,
of course. Unideally, Kevin will just pick one because he's the maintainer.


* While I do accept some blame for not noticing the discussion, I will
not accept “You did not notice, the code is here, so it's your problem
now.” ;-)

git blame would have told you that I do some work on qcow2, so inviting
me to the discussion early might have been a good idea. You probably did
the right thing by inviting Kevin, because he's done the most work, he's
the maintainer, and he's done most of the qcow2 v3 design, as far as I
know. But the thing is that he gets so much mail that he often doesn't
notice some discussions until late.

Anyway, that wouldn't have been a problem if you had put “qcow2” in the
subject of the cover letter. I personally skip through qemu-devel and
qemu-block based on the subject of the cover letter, and if it doesn't
seem directly relevant to me, I'll mark it read and that's it.

So it's not my fault alone that I did not participate in discussions
leading up to this point, and this is why while I do have compassion for
patches which have been worked on for a long time (because I can
relate), I am not really willing to “go weak” on these patches.

Please don't take this as a “not my fault, you're to blame!”, though. :-)

I'm just saying that it's very unfortunate that we're only discussing
this now, and that it's not really anyone's fault.

(And I'm saying this so you know why I don't feel like “We've worked on
it for so long” is a strong point.)

Max

Attachment: signature.asc
Description: OpenPGP digital signature


reply via email to

[Prev in Thread] Current Thread [Next in Thread]