[Qemu-devel] [RFC][PATCH v4 00/18] virtagent: host/guest RPC communication agent

[Michael Roth]

This set of patches is meant to be applied on top of the recently submitted 
Virtproxy v3 patchset. It can also be obtained at:

git://repo.or.cz/qemu/mdroth.git virtproxy_v3

OVERVIEW:

There are a wide range of use cases motivating the need for a guest agent of 
some sort to extend the functionality/usability/control offered by QEMU. Some 
examples include graceful guest shutdown/reboot and notifications thereof, 
copy/paste syncing between host/guest, guest statistics gathering, file access, 
etc.

Ideally these would all be served by a single, easilly extensible agent that 
can be deployed in a wide range of guests. Virtagent is an XMLRPC server, 
integrated into QEMU and the Virtproxy guest daemon, aimed at providing this 
type of functionality.

NOTE: The guest agent can potentially be implemented independently of virtproxy 
depending on the feedback, we simply make use of it to provide an abstraction 
from the actual transport layer (ISA vs. Virtio serial) and use it's 
multiplexing capabilities to avoid having to dedicate 2 isa/virtio serial ports 
to the virtagent service. Please evaluate these patches as being seperate from 
virtproxy.

CHANGES IN V4:

 - Added guest agent capabilities negotiation
 - Added RPC/monitor command to invoke guest shutdown/reboot/powerdown
 - Added RPC/monitor command to the guest agent
 - Added guest startup notification ("hello")
 - Added syslog()'ing of guest agent RPCs
 - Various cleanups

CHANGES IN V3:

 - Integrated virtagent server into virtproxy chardev. Usage examples below.
 - Consolidated RPC server/client setup into a pair of init routines
 - Fixed buffer overflow in agent_viewfile() and various memory leaks

CHANGES IN V2:

 - All RPC communication is now done using asynchronous/non-blocking read/write 
handlers
 - Previously fork()'d RPC server loop is now integrated into qemu-vp/virtproxy 
i/o loop
 - Cleanups/suggestions from previous RFC

DESIGN:

There are actually 2 RPC servers:

1) a server in the guest integrated into qemu-vp, the Virtproxy guest daemon, 
which handles RPC requests from QEMU
2) a server in the host, integrated into the virtproxy chardev, to handle RPC 
requests sent by the guest agent (mainly for handling asynchronous events 
reported by the agent).

At the Virtagent level, communication is done via standard RPCs (HTTP between 
host and guest). Virtproxy transparently handles transport over a network or 
isa/virtio serial channel, allowing the agent to be deployed on older guests 
which may not support virtio-serial.

EXAMPLE USAGE:

 - Configure guest agent to talk to host via virtio-serial
    # start guest with virtio-serial/virtproxy/virtagent. for example 
(RHEL6rc1):
    qemu \
    -chardev virtproxy,id=test0,virtagent=on \
    -device virtio-serial \
    -device virtserialport,chardev=test0,name=virtagent0 \
    -monitor stdio
    ...
    # in the guest:
    ./qemu-vp -c virtserial-open:/dev/virtio-ports/virtagent0:- -g
    ...
    # monitor commands
    (qemu) agent_viewdmesg
    [139311.710326] wlan0: deauthenticating from 00:30:bd:f7:12:d5 by local 
choice (reason=3)
    [139323.469857] wlan0: deauthenticating from 00:21:29:cd:41:ee by local 
choice (reason=3)
    ...
    [257683.375646] wlan0: authenticated
    [257683.375684] wlan0: associate with AP 00:30:bd:f7:12:d5 (try 1)
    [257683.377932] wlan0: RX AssocResp from 00:30:bd:f7:12:d5 (capab=0x411 
status=0 aid=4)
    [257683.377940] wlan0: associated
    
    (qemu) agent_viewfile /proc/meminfo
    MemTotal:        3985488 kB
    MemFree:          400524 kB
    Buffers:          220556 kB
    Cached:          2073160 kB
    SwapCached:            0 kB
    ...
    Hugepagesize:       2048 kB
    DirectMap4k:        8896 kB
    DirectMap2M:     4110336 kB
    (qemu) agent_shutdown powerdown
    (qemu)

KNOWN ISSUES/PLANS:
 - the client socket that qemu connects to send RPCs is a hardcoded filepath. 
This is unacceptable as the socket is channel/process specific and things will 
break when multiple guests are started.
 - proper channel negotiation is needed to avoid hung monitors and such when a 
guest reboots or the guest agent is stopped for whatever reason. additionally, 
a timeout may need to be imposed on the amount of time the http read handler 
can block the monitor.

[RFC][PATCH v4 01/18] virtagent: add common rpc transport defs
[RFC][PATCH v4 02/18] virtagent: base definitions for host/guest RPC server
[RFC][PATCH v4 03/18] virtagent: qemu-vp, integrate virtagent server
[RFC][PATCH v4 04/18] virtagent: base RPC client definitions
[RFC][PATCH v4 05/18] virtagent: add getfile RPC
[RFC][PATCH v4 06/18] virtagent: add agent_viewfile command
[RFC][PATCH v4 07/18] virtagent: add getdmesg RPC
[RFC][PATCH v4 08/18] virtagent: add agent_viewdmesg command
[RFC][PATCH v4 09/18] virtagent: add va_shutdown RPC
[RFC][PATCH v4 10/18] virtagent: add agent_shutdown monitor command
[RFC][PATCH v4 11/18] virtagent: add va_ping RPC
[RFC][PATCH v4 12/18] virtagent: add agent_ping monitor command
[RFC][PATCH v4 13/18] virtagent: add agent_capabilities monitor function
[RFC][PATCH v4 14/18] virtagent: add client capabilities init function
[RFC][PATCH v4 15/18] virtagent: add va_hello RPC function
[RFC][PATCH v4 16/18] virtagent: add va_send_hello() client function
[RFC][PATCH v4 17/18] virtagent: qemu-vp, va_send_hello() on startup
[RFC][PATCH v4 18/18] virtagent: Makefile/configure changes to build virtagent 
bits

 Makefile           |    2 +-
 Makefile.target    |    2 +-
 configure          |   25 ++
 hmp-commands.hx    |   80 ++++++
 monitor.c          |    1 +
 qemu-char.c        |   26 ++
 qemu-config.c      |    3 +
 qemu-vp.c          |   96 ++++++-
 qerror.c           |    4 +
 qerror.h           |    3 +
 qmp-commands.hx    |  164 +++++++++++
 roms/seabios       |    2 +-
 virtagent-common.c |  440 +++++++++++++++++++++++++++++
 virtagent-common.h |   73 +++++
 virtagent-daemon.c |  404 +++++++++++++++++++++++++++
 virtagent-daemon.h |   23 ++
 virtagent.c        |  785 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 virtagent.h        |   43 +++
 18 files changed, 2164 insertions(+), 12 deletions(-)