X Window System event scripting

Event scripting lets software generate keypresses, mouse movements and clicks, and thus control and test other software.

We could use it for activity testing, documentation generation, and perhaps demos.

Software

• xmacro

http://download.sarine.nl/xmacro/Description.html

http://download.sarine.nl/xmacro/xmacro-0.4.5.tar.gz

Example use:

http://ikester.blogspot.com/2007/01/im-huge-fan-of-autohotkey.html

Simple.

• xautomation

http://hoopajoo.net/projects/xautomation.html

http://hoopajoo.net/static/projects/xautomation-0.98.tar.gz

Example use:

http://syn.theti.ca/articles/2006/10/25/tuxpaint-gui-testing-with-xautomation

http://syn.theti.ca/articles/2006/12/17/xautomation-visually-grepping-for-gui-elements

Being actively developed. Some ability to scrape the screen to find things in non-predictable locations (visgrep).

• xnee

http://www.sandklef.com/xnee/

ftp://ftp.gnu.org/gnu/xnee/xnee-3.02.tar.gz

Example use:

http://www.linuxjournal.com/article/6660

Maybe more complex to use?

None of them have Fedora rpms (Ubuntu has all three) (as of 2007-11-12). xmacro compiled fairly easily on an xo. All on fc6.

Possible uses

Run what, where?

• Run on xo. Eg, for demos.
• Run on host, pointing into xo's X. Eg, for hardware testing.
• Run on host, pointing into remote X (ie, Xephyr), which a real or emulated xo is using as its DISPLAY.

Testing stories:

• One emulator per language, running remote X in Xephyr, being driven to generate screen shots, and do basic regression testing of activities.

Examples

Some examples below use olpc_xo_qemu, which allows more easily controlling both QEMU-emulated and remote real XO's (on linux). It provides the laptops/NAME/xo ACTION commands used below. But the ".../xo do some-command" simply runs some-command, having set up DISPLAY and XAUTHORITY environment variables to point to a laptop. Which is something you could do by yourself too, without using xo-qemu.

xmacro on an XO

You can run xmacrorec2/xmacroplay directly on a real XO. It lets you record, and later play again, mouse and keyboard actions.

Getting started:

• Obtain xmacro... Regrettably, an rpm doesn't yet exist. Nor anyone providing a binary.  :( If you have a real or emulated laptop equipped with gcc and a few other things, it compiles fairly easily.

We should at least have binaries easily available, no? A todo item for someone. MitchellNCharity 19:25, 31 January 2008 (EST)

• Change the XTest line in /etc/X11/xorg.conf from "Disable" to "Enable". Then restart X.
• Run it. No xo-qemu cruft is needed.

$ xmacrorec2 -o log
...
$ cat log | xmacroplay

Recording user actions, and replaying them

$ laptops/demo/xo do xmacrorec2 -o log
$ cat log | laptops/demo/xo do xmacroplay

xmacrorec2 records mouse movement, keys, and the time delays between them. It can also make screenshots. You can then later play back the recorded events with xmacroplay.

Note that xmacroplay blindly replays the mouse, keyboard, and screenshot actions. It is not at all aware of what is happening with the programs being poked at. It would replay the same actions, unaltered, even if sugar were dead and nothing was happening on the screen.

So recording and playback is most useful when the same actions will reliably result in the desired behavior. It lets you repeat the same user actions multiple times, without having to recreate them by hand. As long as the underlying software hasn't changed too much between recording and playback (eg, moving things you clicked on, so on replay, the mouse clicks miss). And as long as the software's behavior is itself deterministic (ie, when the user does exactly the same thing, the software too, does exactly the same thing).

The log files are simple, can can be easily edited.

For testing, it can be used for regression testing. To test your belief that everything continues to behave unchanged.

Should describe details of use, to help folks get started. Including using it to take screenshots. MitchellNCharity 18:23, 31 January 2008 (EST)

Interactively driving multiple XO's

By interacting with a single XO, you can simultaneously control several others. This is just a real-time variation on recording and playing back an event log.

You log and interact with one xo:

$ unbuffer laptops/one/xo do xmacrorec2 > log

And elsewhere (you will have to kill them yourself when you are done):

$ tail -f log | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/two/xo do xmacroplay

and

$ tail -f log | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/three/xo do xmacroplay

etc.

Everything you do to laptop one, will also be done to laptops two and three as well. It's rather neat to watch.

This could be used, for instance, with several laptops, configured in different languages, to run a demo or test, and get a full multilingual set of screenshots.

When you only need a single slave laptop, the commands can be simplified to:

unbuffer laptops/one/xo do xmacrorec2 | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/two/xo do xmacroplay

unbuffer is from the expect rpm/deb.

You can interact with the slave laptops normally, without interfering with playback, so you have some ability correct for "they didn't all do quite the same thing" differences. This, plus the ability to adjust your actions to the full set of responses, rather than only to one, is what makes this simultaneous multiplexed approach useful.

See also

• http://linuxbrit.co.uk/scrot/ Screen capture tool.
• http://homepage3.nifty.com/tsato/xvkbd/ Virtual keyboard. Not sure how relevant this is. But might be combined with http://projects.o-hand.com/xoo/ to create a "virtual xo"? :)