[Qemu-devel] Re: paravirtual mouse/tablet, v5

[Peter Hutterer]

sorry, late again. conference last week.

On Thu, Jan 27, 2011 at 02:11:35PM +0100, Gerd Hoffmann wrote:
> Next revision the pvmouse protocol.  It is quite different now, I've
> decided to move to a model with one message per updated value,
> simliar to the linux input layer.  There isn't a "mouse move"
> message any more.  A mouse move event will be three messages now:
> one to update X, one to update Y and a third sync message to mark
> the end of the message block.  That should be *alot* easier to
> extend in the future.

I like this approach, it's a lot more flexible and quite close to what the
linux kernel already does.
Having said that, I'm thinking why not do exactly what the linux kernel
does?

* the kernel has a stable API that has a number of users and is well
understood, plus 
* it allows for code re-use on the consumer side
* you'd need to do protocol conversion anyway. having the same API means at
least one platform (i.e. linux) can skip the conversion.
* MT is being worked on by a number of people and two protocols that cater
for current hardware (with and without HW tracking features). trying to
duplicate this means we'll likely re-do the same mistakes.

fwiw, this protocol is already quite similar
    MSG_SYNC is SYN_REPORT
    MSG_POINT is SYN_MT_REPORT
    AXIS x is the same as EV_ABS + ABS_X

there may be some optimization for networks of course if you want to change
the message format but at least the principle seems sound enough.

> Header file is attached.  Comments are welcome.
> 
> thanks,
>   Gerd

> #ifndef __QEMU_PVMOUSE__
> #define __QEMU_PVMOUSE__ 1
> 
> /*
>  * qemu patavirtual mouse/tablet interface
>  *
>  * quick overview
>  * ==============
>  *
>  * device initialization
>  * ---------------------
>  *
>  *  (1) host sends INIT with supported feature bits
>  *  (2) guests sends ACK to ack the features it is
>  *      able to handle (and willing to use).

what's the effect of a guest not able to handle certain features? The host
filters them?

>  *  (3) host sends a INFO message for each button and
>  *      each axis supported.  More INFO messages might
>  *      follow for features added to the protocol later
>  *      on if enabled by the INIT+ACK handshake.
>  *      A SYNC message marks the end of the device
>  *      information messages.
>  *
>  * input event reporting
>  * ---------------------
>  *
>  *  host sends one or more BTN_DOWN, BTN_UP and AXIS messages,
>  *  followed by a SYNC.  A button press would be just BTN_DOWN+SYNC.
>  *  A mouse move would be two AXIS messages (one for x, one for y)
>  *  followed by SYNC.

fwiw, kernel doesn't have BTN_DOWN events but rather BTN_FOO 0 or 1 events
(for released and pressed, respectively). Same for keys, which also allows
keys to have a value of 2 for autorepeat.

it provides more symmetry as well because currently you have
    AXIS <axis code> <axis value>
    BTN_DOWN <button code>

so axis and button_down are asymmetrical in format.
alternatively, you could use
    AXIS <axis code> <axis value>
    BTN <button code> <button value>

thus making the message format identical.

>  *
>  * multitouch events
>  * -----------------
>  *
>  *  Each reported touch point starts with a POINT message, followed by
>  *  multiple AXIS messages (at least x+y, most likely also pressure).
>  *  The whole group is followed by SYNC.  A report for two fingers
>  *  would look like this:
>  *
>  *   POINT
>  *     AXIS x=23
>  *     AXIS y=42
>  *     AXIS pressure=3
>  *   POINT
>  *     AXIS x=78
>  *     AXIS y=56
>  *     AXIS pressure=4
>  *   SYNC
>  *
>  *  In case the device supports ID tracking the POINT message will
>  *  carry the ID.  Updates need only be sent for points which did
>  *  change.  IDs are added by using them the first time.  IDs are
>  *  invalidated when the finger is lifted (aka pressure=0).

how about devices that support hovering, where the pressure is 0 but the
touchpoint is still clearly in proximity and active?

>  *  In case the device doesn't support ID tracking each report must
>  *  include all current touch points (in unspecified order).
>  *
>  */
> 
> #include <inttypes.h>
> 
> /*
>  * our virtio-serial channel name(s)
>  */
> #define QEMU_PVMOUSE_FORMAT "org.qemu.pvmouse.%d"
> 
> enum qemu_pvmouse_msg_type {
>     /* feature flag negotiation */
>     QEMU_PVMOUSE_MSG_INIT,
>     QEMU_PVMOUSE_MSG_ACK,
>     /* device information */
>     QEMU_PVMOUSE_MSG_AXIS_INFO,
>     QEMU_PVMOUSE_MSG_BUTTON_INFO,
>     /* device events */
>     QEMU_PVMOUSE_MSG_BTN_DOWN,
>     QEMU_PVMOUSE_MSG_BTN_UP,
>     QEMU_PVMOUSE_MSG_AXIS,
>     /* message grouping */
>     QEMU_PVMOUSE_MSG_POINT,
>     QEMU_PVMOUSE_MSG_SYNC,
> };
> 
> typedef enum qemu_pvmouse_features {
>     QEMU_PVMOUSE_FEATURE_MULTITOUCH,
> };

with or without tracking? I can't see any flag that tells us that ahead of
time but for consumers it's quite important.

how many simultaneous touchpoint does the device support?

again, quick overview of the kernel here:
MT protocol A is essentially just positions + MT_SYN (POINT) events.
MT protocol B for tracked uses TRACKING_ID as well as MT_SLOT. the slots say
how many simultaneous trackpoints can go on. tracking ID is >= 0 for any
given touchpoint and -1 to cancel a touchpoint in the current slot.

So without kernel-side filtering, you'd get

ABS_MT_SLOT             0
ABS_MT_TRACKING_ID      23
ABS_MT_POSITION_X       100 
ABS_MT_SLOT             1
ABS_MT_TRACKING_ID      28
ABS_MT_POSITION_X       23
ABS_MT_POSITION_Y       156
SYN_REPORT
-----------------------------
ABS_MT_SLOT             0
ABS_MT_TRACKING_ID      -1
ABS_MT_SLOT             1
ABS_MT_TRACKING_ID      28
ABS_MT_POSITION_X       30
ABS_MT_POSITION_Y       162
SYN_REPORT
-----------------------------
ABS_MT_SLOT             0
ABS_MT_TRACKING_ID      29
ABS_MT_POSITION_X       600
ABS_MT_POSITION_Y       876
ABS_MT_SLOT             1
ABS_MT_TRACKING_ID      28
ABS_MT_POSITION_X       30
ABS_MT_POSITION_Y       162
SYN_REPORT

so you have three events, first two touchpoints, then first touchpoint is
lifted, new touchpoint is created.

> /*
>  * QEMU_PVMOUSE_MSG_INIT, host -> guest
>  * First message.  Sent before any other event.
>  */
> typedef struct qemu_pvmouse_init {
>     uint32_t features;        /* qemu_pvmouse_features */
> } qemu_pvmouse_init;
> 
> /*
>  * QEMU_PVMOUSE_MSG_ACK, guest -> host
>  * Sent after pvmouse_init.  Host will not send
>  * additional messages until this is received.
>  */
> typedef struct qemu_pvmouse_ack {
>     uint32_t features; /* qemu_pvtable_features */
> };
> 
> 
> enum qemu_pvmouse_axis_type {
>     /* absolute */
>     QEMU_PVMOUSE_AXIS_POS_X = 1,
>     QEMU_PVMOUSE_AXIS_POS_Y,
>     QEMU_PVMOUSE_AXIS_PRESSURE,
> 
>     /* relative */
>     QEMU_PVMOUSE_AXIS_REL_X = 257,
>     QEMU_PVMOUSE_AXIS_REL_Y,
>     QEMU_PVMOUSE_AXIS_WHEEL_VERT,
>     QEMU_PVMOUSE_AXIS_WHEEL_HORIZ,
> };
> 
> enum qemu_pvmouse_axis_flags {
>     QEMU_PVMOUSE_AXIS_PEN,
>     QEMU_PVMOUSE_AXIS_TOUCH,
>     QEMU_PVMOUSE_AXIS_MULTITOUCH,
>     QEMU_PVMOUSE_AXIS_POINT_ID,
> };
> #define QEMU_PVMOUSE_AXIS_MASK_PEN         (1 << QEMU_PVMOUSE_AXIS_PEN)
> #define QEMU_PVMOUSE_AXIS_MASK_TOUCH       (1 << QEMU_PVMOUSE_AXIS_TOUCH)
> #define QEMU_PVMOUSE_AXIS_MASK_MULTITOUCH  (1 << QEMU_PVMOUSE_AXIS_MULTITOUCH)
> #define QEMU_PVMOUSE_AXIS_MASK_POINT_ID    (1 << QEMU_PVMOUSE_AXIS_ID)

fwiw, the kernel already defines 14 tools, not that far off until you run
out of tools on 32 bits. why not make the initial feature negotiation a
stream of events instead of a mask?

Cheers,
  Peter
> 
> /*
>  * QEMU_PVMOUSE_MSG_AXIS_INFO, host -> guest
>  * Send axis informations.
>  */
> typedef struct qemu_pvmouse_axis_info {
>     uint32_t axis_id;
>     uint32_t type;
>     uint32_t flags;
>     int32_t  min;
>     int32_t  max;
>     char     label[];
> };
> 
> /*
>  * QEMU_PVMOUSE_MSG_AXIS, host -> guest
>  * Send mouse/pen/finger/wheel move events.
>  */
> typedef struct qemu_pvmouse_axis_event {
>     uint32_t axis_id;
>     int32_t  value;
> };
> 
> /*
>  * QEMU_PVMOUSE_MSG_BUTTON_INFO, host -> guest
>  * Send button informations.
>  */
> typedef struct qemu_pvmouse_button_info {
>     uint32_t button_id;
>     char     label[];
> };
> 
> /*
>  * QEMU_PVMOUSE_MSG_BTN_{DOWN,UP}, host -> guest
>  * Send button press+release events.
>  */
> typedef struct qemu_pvmouse_btn_event {
>     uint32_t button_id;
> };
> 
> /*
>  * QEMU_PVMOUSE_MSG_POINT, host -> guest
>  * marks the start of a point group for multitouch devices.
>  */
> typedef struct qemu_pvmouse_point {
>     uint32_t id;
> };
> 
> /*
>  * QEMU_PVMOUSE_MSG_SYNC, host -> guest
>  * Marks the end of a message group which belongs together
>  * and carries the time stamp for all those events.
>  *
>  * The timestamp is specified in nanoseconds.  Timebase is undefined.
>  * This is supposed to be used to figure how much time passed between
>  * two events, to decide whenever two mouse clicks should be
>  * interpreted as double click or not and simliar stuff.
>  */
> typedef struct qemu_pvmouse_sync {
>     uint64_t timestamp;
> };
> 
> 
> typedef struct qemu_pvmouse_header {
>     uint32_t size;            /* whole message size */
>     uint32_t type;            /* qemu_pvmouse_type */
> } qemu_pvmouse_header;
> 
> typedef union qemu_pvmouse_payload {
>     qemu_pvmouse_init     init;
>     qemu_pvmouse_ack      ack;
>     /* ... */
> };
> 
> typedef struct qemu_pvmouse_message {
>     qemu_pvmouse_header  hdr;
>     qemu_pvmouse_payload data;
> } qemu_pvmouse_message;
> 
> #endif /* __QEMU_PVMOUSE__ */