X Window System event scripting

From OLPC
Revision as of 23:23, 31 January 2008 by MitchellNCharity (talk | contribs) (→‎Recording user actions: tweak section name)
Jump to navigation Jump to search

Event scripting lets software generate keypresses, mouse movements and clicks, and thus control and test other software.

We could use it for activity testing, documentation generation, and perhaps demos.

Software

  • xmacro
http://download.sarine.nl/xmacro/Description.html
http://download.sarine.nl/xmacro/xmacro-0.4.5.tar.gz
Example use:
http://ikester.blogspot.com/2007/01/im-huge-fan-of-autohotkey.html
Simple.
  • xautomation
http://hoopajoo.net/projects/xautomation.html
http://hoopajoo.net/static/projects/xautomation-0.98.tar.gz
Example use:
http://syn.theti.ca/articles/2006/10/25/tuxpaint-gui-testing-with-xautomation
http://syn.theti.ca/articles/2006/12/17/xautomation-visually-grepping-for-gui-elements
Being actively developed. Some ability to scrape the screen to find things in non-predictable locations (visgrep).
  • xnee
http://www.sandklef.com/xnee/
ftp://ftp.gnu.org/gnu/xnee/xnee-3.02.tar.gz
Example use:
http://www.linuxjournal.com/article/6660
Maybe more complex to use?

None of them have Fedora rpms (Ubuntu has all three) (as of 2007-11-12). xmacro compiled fairly easily on an xo. All on fc6.

Possible uses

Run what, where?

  • Run on xo. Eg, for demos.
  • Run on host, pointing into xo's X. Eg, for hardware testing.
  • Run on host, pointing into remote X (ie, Xephyr), which a real or emulated xo is using as its DISPLAY.

Testing stories:

  • One emulator per language, running remote X in Xephyr, being driven to generate screen shots, and do basic regression testing of activities.

Examples

Some examples below use olpc_xo_qemu, which allows more easily controlling both QEMU-emulated and real XO's (on linux). It provides the laptops/NAME/xo ACTION commands used below. But the ".../xo do some-command" simply runs some-command, having set up DISPLAY and XAUTHORITY environment variables to point to a laptop. Which is something you could do by yourself too, without using xo-qemu.

Recording, and replaying, user actions

$ laptops/demo/xo do xmacrorec2 -o log
$ cat log | laptops/demo/xo do xmacroplay

xmacrorec2 records mouse movement, keys, and the time delays between them. It can also make screenshots. You can then later play back the recorded events with xmacroplay.

Note that xmacroplay blindly replays the mouse, keyboard, and screenshot actions. It is not at all aware of what is happening with the programs being poked at. It would replay the same actions, unaltered, even if sugar were dead and nothing was happening on the screen.

So recording and playback is most useful when the same actions will reliably result in the desired behavior. It lets you repeat the same user actions multiple times, without having to recreate them by hand. As long as the underlying software hasn't changed too much between recording and playback (eg, moving things you clicked on, so on replay, the mouse clicks miss). And as long as the software's behavior is itself deterministic (ie, when the user does exactly the same thing, the software too, does exactly the same thing).

For testing, it can be used for regression testing. To test your belief that everything continues to behave unchanged.

Should describe details of use, to help folks get started. Including using it to take screenshots. MitchellNCharity 18:23, 31 January 2008 (EST)

Interactively driving multiple XO's

By interacting with a single XO, you can simultaneously control several others. This is just a real-time variation on recording and playing back an event log.

You log and interact with one xo:

$ unbuffer laptops/one/xo do xmacrorec2 > log

And elsewhere (you will have to kill them yourself when you are done):

$ tail -f log | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/two/xo do xmacroplay

and

$ tail -f log | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/three/xo do xmacroplay

etc.

Everything you do to laptop one, will also be done to laptops two and three as well. It's rather neat to watch.

This could be used, for instance, with several laptops, configured in different languages, to run a demo or test, and get a full multilingual set of screenshots.

When you only need a single slave laptop, the commands can be simplified to:

unbuffer laptops/one/xo do xmacrorec2 | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/two/xo do xmacroplay

unbuffer is from the expect rpm/deb.

You can interact with the slave laptops normally, without interfering with playback, so you have some ability correct for "they didn't all do quite the same thing" differences. This, plus the ability to adjust your actions to the full set of responses, rather than only to one, is what makes this simultaneous multiplexed approach useful.

See also