[Qemu-devel] Re: libvirt vs. in-qemu management

[Avi Kivity]

On 04/06/2010 12:11 AM, Alexander Graf wrote:
> Howdy,
>
> I've been thinking a bit further on the whole issue around libvirt and why the 
> situation as is isn't satisfying. I came to the following points that currently 
> hurt building ease of use for KVM:
>
> 1) Brand
>
> This is one of the major issues we have ourselves when it comes to appliances. 
> We can ship appliances built for VMware. We can ship appliances built for Xen. 
> But we can't ship appliances built for KVM, because there is no single 
> management app we could target.

There are already at least three management apps for kvm:  virt-manager, 
proxmox, and RHEV-M (my personal favorite).  If we define our own format 
then we need those management apps to understand it.  That means we 
either include only simple features, or we wait until the management 
apps catch up to all the features we provide.  Otherwise those 
appliances aren't universal.

An additional problem is that our format will exclude metadata that the 
management app may want to add.

> That destroys the KVM brand IMHO.
>    

That's because  kvm is infrastructure instead of a complete stack.  I 
agree it's a problem but I see no way around it.

> 2) Machine description
>
> If we build an appliance, we also create a configuration file that describes 
> the VM. We can create .vmx files, we can create xen config files. We can not 
> create KVM config files. There are none. We could create shell scripts, but 
> would that help?
>    

It's not enough for qemu to be able to read the configuration file.  The 
management app needs to read it as well, to understand how much memory 
and cpu the guest needs (so it can schedule it on the cluster), what 
kind of network connectivity it needs (how many interfaces, what 
networks those interfaces connect to, does it need firewall ports 
open).  An appliance configuration is more than a vm configuration, and 
again, the management app needs to be able to understand all of it.

> 3) Configuration conversion
>
> Party due to qemu not having a configuration format, partly due to libvirt's 
> ambivalent approach, there is always conversion in configuration formats 
> involved. I think this is the main reason for the feature lag. If there wasn't 
> a conversion step, there wouldn't be lag. You could just hand edit the config 
> file and be good.
>    

There will always be a lag, since management apps (at least the 
non-trivial ones) want to display the configuration in a GUI, allow 
users to edit it, and want to understand it.  It's not just conversion, 
it's plumbing down the whole stack.

> Point 2 needs to be solved anyways. We need a machine config format for qemu. 
> For general -M description as well as for specific VM description. The command 
> line options just become too complicated and too hard to reproduce and save. 
> Just think of live migration with hot-plugged devices. Or safe savevm + loadvm. 
> The current logic ends there.
>    

I don't think the management apps will want to use it.  They will need 
to parse it (currently they only need to write it, which is simpler).  
Things like 'query all smp guests with >4GB memory' become complicated 
instead just a database query.

For managed guests, I think we want to get rid of the command line at 
all.  Start the guest with just a case and cold-plug the motherboard, 
processors, memory, cards.  Migration starts with a replay of these 
(including any hotplugged cards added while the guest is running).  
Hotplugs during migration are relayed to the other side over the wire.

> I can imagine 1) going away if we would set libvirt + virt-manager as _the_ 
> front-end and have everyone focus on it. I suppose it would also help to 
> rebrand it by then, but I'm not 100% sure about that. Either way, there would 
> have to be a definite statement that libvirt is the solution to use. And 
> _everyone_ would have to agree on that. Sounds like a hard task. And by then we 
> still don't really have a branded product stack.
>
> Point 3 is the really tough one. It's the very basis of libvirt. And it's plain 
> wrong IMHO. I hate XML. I hate duplicated efforts. The current conversion 
> involves both. Every option added to qemu needs to be added to libvirt.

Not just libvirt, virt-manager as well.  And that is typically more 
difficult technically (though probably takes a lot less time).

> In XML. Bleks.
>    

Yeah.

> Reading on IRC I seem to not be the only person thinking that, just the first 
> one mentioning this aloud I suppose. But that whole XML mess really hurts us 
> too. Nobody wants to edit XML files. Nobody wants to have two separate syntaxes 
> to describe the same thing. It complicates everything without a clear benefit. 
> And it puts me in a position where I can't help people because I don't know the 
> XML format. That should never happen.
>    


> Sure, for libvirt it makes sense to be hypervisor-agnostic. For qemu it 
> doesn't. We want to be _the_ hypervisor. Setting our default front-end to 
> something that is agnostic weakens our point. And it slows down development. 
> And it hurts integration. And thus usability, thus adoption. It hurts us.
>    

It doesn't make sense for libvirt to be hypervisor agnostic.  If it is, 
people who want to use one hypervisor's advanced features are forced to 
work around it.  Anthony wants multiple monitors for this, but that's a 
bad workaround.  libvirt is placing developers using it in an impossible 
situation - the developers want to use kvm-specific features and libvirt 
is in the way.

> That's what I've concluded so far on the whole situation as is. I find it sad 
> to be the one speaking it out, but IMHO going with libvirt as default 
> management front-end is a dead end. It will hurt us more than it will help us.
>    

Well, I did suggest (and then withdraw) qemud.  The problem is that to 
get something working we'd duplicate all the work that's gone into 
libvirt - storage pools, svirt, network setup, etc.

> That said I don't think it'd be bad to cooperate or encourage people to use 
> libvirt. In fact I believe the opposite - it's great if you want to be 
> agnostic. It just isn't when you're not. And we should differentiate there.
>    

Much of the problem is the hypervisor compatibility thing, yes.


--
Do not meddle in the internals of kernels, for they are subtle and quick to 
panic.