[Bug-xnee] [bug #28238] XI2 extension support

[Rami Ylimaki]

Follow-up Comment #2, bug #28238 (project xnee):

Below is a more detailed description of what I'd like CNEE to be able to do.
Please clarify if I have misunderstood something.



1. Current situation: recording and playback of core input events

Let's assume that I have a simple drawing application and I'll use CNEE to
test it. Here are the steps that I'd take:

a. start recording pointer events: "cnee --record --mouse > output.txt"
b. draw to the application window
c. stop recording by terminating cnee

As a result the output.txt contains lines like:

0,6,228,181,0,0,4294967295,156381343
0,4,0,0,1,0,0,156381344
0,6,228,180,0,0,4294967295,156381361
...
0,5,0,0,1,0,0,156382915

CNEE has decoded the information from MotionNotify (coordinates are there),
ButtonPress and ButtonRelease events and saved them to output.txt.

Sometime after recording and saving output.txt, I'd playback these events to
see that the drawing application is still producing desired results:

a. execute "cnee --replay -f output.txt" 
b. examine the drawing application canvas
c. a familiar figure appears on the screen



2. Feature request: recording and playback of extension input events

If the drawing application is modernized to support extension events, I can't
test the application anymore.

a. start recording: "cnee --record --devera 2-80 > output.txt
b. draw
c. stop recording

I can see the extension events now, but they aren't decoded anywhere. In
output.txt, I'll see the following kind of lines:
...
0,73,0,0,0,0,0,156935358
...

There's no information about coordinates or pressure levels on those lines,
so I won't be able to replicate the drawing using extension events.

My understanding is that the following modifications should be done to CNEE
to make this work:
a. Check if X server supports XI2
b. If so, decode DeviceMotionNotify, DeviceButtonPress and
DeviceButtonRelease in xnee_record_handle_event_printer just like
MotionNotify, ButtonPress and ButtonRelease are decoded.
c. When replaying output.txt, the core motion events are faked in
xnee_fake_motion_event with XTestFakeMotionNotify. You'd have to call
XTestFakeDeviceMotionNotify for the extension motion events. Similarly two
different versions of the faking functions can be found for the button presses
and releases.



3. Limiting recording to specified devices

After implementing the above changes, CNEE would be able to record and replay
all input events. The core X protocol is aware of only one pointer and one
keyboard. However, XI2 allows the user to have multiple devices. Therefore the
decoding of extension events would have to include information about the
device id that originated the events so that the events can be played back for
correct devices.

By default CNEE could record from all devices if --mouse option is given. It
would be nice if the user could limit recording to some specific device. In
this case the user would use --mouse="my touchpad" or --mouse="my bluetooth
mouse" to limit recording to a specified device.

Also it would be nice if the user could change the input device on playback.
The user would first create a test case with --mouse="my bluetooth mouse".
Later the configuration on his computer has changed but he still wants to
replay the test case. In this case the --replay method would need to have some
kind of option to tell that events originating from "my bluetooth mouse"
should be played back from "my usb mouse".

Before implementing this kind of functionality, it'd would be easy to just
assume everything originates from "Virtual core pointer" and play the events
back for that device also. That would probably work correctly on most use
cases because usually the pointer tests are done for a single pointer. The
important thing would be to allow recording and playback of extension events
for the XI2 default "Virtual core pointer" and worry about specific devices
later.



4. Keyboards

I'm talking about pointers above because that's what I'm currently mostly
interested in. However, everything that has been said above applies to
--keyboard option and KeyPress/KeyRelease DeviceKeyPress/DeviceKeyRelease.



5. Event ranges

As I have understood it, CNEE just records whatever event IDs the X server is
sending. However, to identify extension events, X server uses a base event
number for the extension and the actual event numbers are relative to the
base. When recording extension input events, CNEE can't just save an event
number like 74 to mean DeviceButtonRelease, because the event base may be a
different number on different systems. Maybe CNEE should just print names for
the input events when they are recorded or be a little more generic and print
"XI2+1" instead of "74" for the DeviceButtonRelease.

This kind of functionality is needed if the user wants to record a test case
on one computer and then play it back later on a different computer. The
current legacy functionality doesn't have this problem, because the core input
event IDs are pretty much hard coded to known values.


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?28238>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/