Re: [Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Alon Levy]

On Wed, Jan 12, 2011 at 07:59:40PM +0100, Hans de Goede wrote:
> Hi,
> 
> On 01/12/2011 06:40 PM, Anthony Liguori wrote:
> >On 01/12/2011 10:12 AM, Gerd Hoffmann wrote:
> >>
> >>Hi folks,
> >>
> >>Looks like the spicevmc patch kicked the guest qagent discussion, so lets 
> >>start with this, although it isn't related much to the agent issue
> >>itself ...
> >>
> >>
> >>The spicevmc chardev just pipes data from a chardev user within qemu
> >>to libspice and adds a "type" tag to it so libspice knows now to wind
> >>up the other end. There are several types:
> >>
> >>(1) vdagent, the spice guest agent. Will discuss this in detail
> >>below.
> >>
> >>(2) smartcard, this basically pipes the smartcard protocol over spice.
> >>Patches for smartcard support are on the list and should be almost
> >>ready for merge now. If you want connect a remote smart card reader
> >>to your guest you can use a tcp chardev, which will build a data
> >>pipeline like this:
> >>
> >>ccid-passthrough <-> tcp chardev <-> tcp protocol <->
> >>vcsclient <-> libcacard
> >>
> >>Or you can use the spicevmc chardev to use spice as transport:
> >>
> >>ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
> >>spice client <-> libcacard
> >>
> >>If someone comes up with a vnc extention one could create something
> >>simliar for vnc tunneling:
> >>
> >>ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
> >>gtk-vnc widget <-> libcacard
> >>
> >>(3) usb forwarding. Hans is busy with this. No working code yet.
> >>Will probably work pretty simliar to smartcard.
> >>
> >>(4) termial forwarding. Just an idea right now. Nowdays that the spice
> >>client side moves to gtk it would be easy to embed a termial widget,
> >>therby allowing easy access to the serial console using something
> >>like this:
> >>
> >>-chardev spicevmc,id=console,type=terminal
> >>-device isa-serial,index=0,chardev=console
> >>
> >>So even if you put the guest agent discussion completely aside the
> >>spicevmc chardev clearly has its uses.
> >>
> >>
> >>Ok, now for the spice vdagent. Alon posted the link to the specs in
> >>the spice wiki already, here it is again:
> >>http://spice-space.org/page/Whiteboard/AgentProtocol
> >>
> >>The header file with the protocol specification comes with the
> >>spice-protocol package, here is a direct link:
> >>http://cgit.freedesktop.org/spice/spice-protocol/tree/spic/vd_agent.h
> >>
> >>Linux agent code is here:
> >>http://cgit.freedesktop.org/~jwrdegoede/vdagent-linux/
> >>
> >>The protocol can be multiplexed via VDIChunkHeader->port. spice
> >>actually does that as the mouse messages (VDAgentMouseState) are
> >>generated by the spice server itself. Everything else is just piped
> >>from the guest to the spice client (and back). The protocol also has
> >>capabilities (VDAgentAnnounceCapabilities).
> >>
> >>There isn't much spice-specific stuff in there. The clipboard bits
> >>for example should work unmodified with vnc, one would just have to
> >>hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
> >>client support of course).
> >
> >VNC already supports copy/paste as part of the protocol so can the agent 
> >protocol be terminated in QEMU such that the server can make use of the 
> >standard protocol extensions?
> >
> 
> That depends on if the VNC copy/paste support was done right. With right
> I mean that it should consist of the following messages:
> 1) A clipboard grab (send by guest -> client if ctrl+c pressed inside
>    the guest, other direction if ctrl-c is pressed in any app on the
>    client machine.) This should include a list of supported types the
>    app now owning the clipboard can offer the clipboard in. For example
>    text/plain-utf-8, text/html
> 2) A clipboard release send when the clipboard owning app exits
> 3) A clipboard request, send by one side when it wants to get the
>    clipboard data, prereq: the other side owns the clipboard, the
>    request is for a type in the list of types advertised when grabbing
> 4) clipboard data, response to clipboard request, could also be a nack
>    when a release / request message cross each other.
> 
> This is basically how both windows and X11 work wrt copy and paste at
> the low level, and by far the sanest way to support guest <-> client
> os copy paste (event driven, no polling or some such involved).
> 

Reading RealVNC's (authorative afaik) protocol 3.8 there are only two
messages, ClientCutText which supports latin (ISO 8859-1) only, and
ServerCutText, which is the same.

There are a few message extensions that I haven't looked at (defined
in the spec), maybe they include more copy paste functionality? Also,
we could of course propose an extension.

So I think we could easily work with this with the spice agent, just
have to generate some messages in qemu.

> I agree with the notion of having 2 agents in the guest, 1 system
> wide and 1 per session. Assuming we get a standardized protocol for
> the per session agent and we can somehow fit vnc's existing copy and
> paste support into the above model, it should be able to use that.
> 
> While on the 2 agent topic. As I've just completed written the
> spice agent for Linux I've some experience on this. Currently the
> spice agent actually already consists of 2 processes an always
> running daemon and a per X-session agent, there are 2 reasons for this:
> 1) Thinks like the paravirt mouse are best done at the system level
> 2) A multiplexer is needed between the client -> session agent channel,
> and the actual session agents, as multiple session agents could be
> running (think switch user functionality), and the messages should be
> forwarded to the session agent in the active (focussed / showing)
> session. spice-vdagentd uses console kit to determine the active
> session and monitor session changes.
> 
> This does not really impact any protocol designs, but still it is
> something to keep in mind.
> 
> >>
> >>I think the only spice-specific bit in the protocol is the display
> >>enumeration in the VDAgentMonitorsConfig and VDAgentMouseState
> >>messages. The numbers there simply reference the spice display
> >>channel number of the display in question, which just doesn't exist
> >>outside spice. Of course one can just ignore that for now as there is
> >>no multihead support outside spice anyway ...
> >>
> >>
> >>Also related: paravirtual mouse. I'd suggest to go for something new,
> >>based on virtio-serial, doing just the mouse and nothing else.
> >
> >I'd agree. I think we want something that actually terminates in the kernel 
> >for Linux guests since then we can expose it as an evdev device. No special 
> >X driver would be needed.
> 
> A paravirt mouse would need some sort of guest os support, yes. But
> not necessarily in the kernel. currently spice-vdagentd uses uinput
> to create an evdev device and send events from userspace. And the spice
> agent under windows does the same (although there the events are
> generated by the per user session agent process).
> 
> I'm not saying this should not be in the kernel, just that it does
> not have to be in the kernel. It might even make sense to try and
> use such a ipc mechanism for this, that in one guest os it could be
> a kernel driver and in the other a userspace solution, but I'm not
> sure how feasible that it.
> 
> >Is this something that makes sense for Spice in the future?
> 
> Yes, as long as guest support is kept in mind. Looking at it from
> that pov. I think I might favor a userspace solution, as we likely
> need an agent anyways, and adding / doing drivers for some guest
> os-s is a pain.
> 
> >
> >>
> >>The VDAgentMouseState messages have one problem: They send the pointer
> >>position as-is, which introduces a dependency on the screen size.
> 
> Yeah, if we could get rid of that, that would be great. We could even
> introduce a new mouse message type to the existing spice vdagent protocol
> and use capabilities to switch between the 2.
> 
> >>Spice deals with it, and although that is a little bit ugly it isn't a
> >>big issue either as spice knows how big the screens are.
> 
> Well the client knows, but a guest os system level agent does not
> necessarily know. With the current linux spice agent, the system level
> daemon depends on the per session agent to notify it of resolution changes
> (and if there is no per session agent active, the mouse gets dropped
>  into server mode). So I'm in favor of fixing this even within the
> current spice specific vdagent protocol
> 
> >> When using
> >>this outside spice context this becomes nasty though.
> >>
> >>Also I think it is a bad idea to mix guest agent stuff with the
> >>paravirtual mouse. The mouse events need to get routed to the
> >>X-Server, whereas the agent stuff is handled by some other daemon.
> >>Also the multiplexing the spice server has to do to inject the mouse
> >>messages supports that view IMHO.
> >>
> >>Related note: It is probably a good idea to also separate stuff which
> >>is handled best by a system-wide guest daemon (such as reboot
> >>requests, package installs, whatever, vdagent doesn't do that kind of
> >>management btw) and stuff which needs to be handled by a per-user
> >>daemon which has access to the user's X-Session (clipboard, ...).
> >
> >Definitely. VMware uses two separate daemons--one that runs in a system 
> >context and one that runs per user X session.
> >
> 
> +1
> 
> >>
> >>Ok, how to go forward now? Here is what I think we should do:
> >>
> >>(1) Merge spicevmc chardev. Needed anyway for compatibility and
> >>for the other use cases outlined above.
> >>(2) Design a paravirtual mouse device. Make sure it fullfills the
> >>spice requirements, which are basically:
> >>* support multiple displays.
> >>* don't eat tons of cpu like usb-tablet does.
> >>(3) spice can use the new pvmouse now and depricate sending mouse
> >>events via vdagent protocol.
> >>(4) In parallel review the vdagent protocol, then either extend or
> >>redesign it for other use cases.
> >
> >I'm okay with this as long as there is a commitment from the Spice team to 
> >convert on a common framework.
> 
> I (*) am in favor of moving to a common framework. As long as
> compatibility is maintained with the old agent, but that all lives inside
> spice server, so should not be an issue.
> 
> *) Speaking for myself not the team (not that I know of anyone in the
> team who is against).

+1

> 
> >I'd like to have a little more discussion about agent design first to make 
> >sure that we're on the same page.
> >
> >For instance, Spice makes use of a 1-off protocol whereas something like 
> >virt-agent uses an established RPC protocol (XML-RPC). I'm not tied to using 
> >any particular protocol, but I think it's very important to use a 
> >standardized, well specified protocol.
> 
> I'm not sure what something like XML-RPC buys us here, other then
> dragging in a lot of extra dependencies. I'm not saying that I'm against
> using xmlrpc but I'm not sold on it either.

Regarding wire protocol this would of course be very wasteful. If it's used
between the guest and host except for the translation overhead, which is minimal
I think (except for a copy-paste of a large amount of data - I'm not familiar
enough with XML-RPC but I guess it has some way to pass binary buffers 
unmodified?),
I don't have a particular objection, other then hoping we consider alternatives 
(there
are a few).

Regarding one-off protocols in general, I agree it creates code duplication
and should be avoided. In that respect I prefer mechanisems that produce
implementations from a declarative description. But I would prefer a binary
protocol, possibly a convertion of the existing spice protocol to declarative
form (like Alex did to the core spice protocols, see spice.proto and 
spice1.proto,
http://cgit.freedesktop.org/spice/spice/tree/spice.proto
Of course there are many implementations of this outside of spice, like protocol
buffers). I also don't think a one-off is necessarily a bad thing.

> 
> Note while on the subject of design, I think that having some sort of
> capabilities negotiation so that we can provide compatibility between
> different versions is important.
> 
> Regards,
> 
> Hans
>