X Window System event scripting
Event scripting lets software generate keypresses, mouse movements and clicks, and thus control and test other software.
We could use it for activity testing, documentation generation, and perhaps demos.
Contents
Software
- xmacro
- http://download.sarine.nl/xmacro/Description.html
- http://download.sarine.nl/xmacro/xmacro-0.4.5.tar.gz
- Example use:
- Simple.
- xautomation
- http://hoopajoo.net/projects/xautomation.html
- http://hoopajoo.net/static/projects/xautomation-0.98.tar.gz
- Example use:
- Being actively developed. Some ability to scrape the screen to find things in non-predictable locations (visgrep).
- xnee
- http://www.sandklef.com/xnee/
- ftp://ftp.gnu.org/gnu/xnee/xnee-3.02.tar.gz
- Example use:
- Maybe more complex to use?
None of them have Fedora rpms (Ubuntu has all three) (as of 2007-11-12). xmacro compiled fairly easily on an xo. All on fc6.
Possible uses
Run what, where?
- Run on xo. Eg, for demos.
- Run on host, pointing into xo's X. Eg, for hardware testing.
- Run on host, pointing into remote X (ie, Xephyr), which a real or emulated xo is using as its DISPLAY.
Testing stories:
- One emulator per language, running remote X in Xephyr, being driven to generate screen shots, and do basic regression testing of activities.
Examples
Some examples below use olpc_xo_qemu, which allows more easily controlling both QEMU-emulated and real XO's (on linux). It provides the laptops/NAME/xo ACTION commands used below. But the ".../xo do some-command" simply runs some-command, having set up DISPLAY and XAUTHORITY environment variables to point to a laptop. Which is something you could do by yourself too, without using xo-qemu.
Recording user actions, and replaying them
$ laptops/demo/xo do xmacrorec2 -o log $ cat log | laptops/demo/xo do xmacroplay
xmacrorec2 records mouse movement, keys, and the time delays between them. It can also make screenshots. You can then later play back the recorded events with xmacroplay.
Note that xmacroplay blindly replays the mouse, keyboard, and screenshot actions. It is not at all aware of what is happening with the programs being poked at. It would replay the same actions, unaltered, even if sugar were dead and nothing was happening on the screen.
So recording and playback is most useful when the same actions will reliably result in the desired behavior. It lets you repeat the same user actions multiple times, without having to recreate them by hand. As long as the underlying software hasn't changed too much between recording and playback (eg, moving things you clicked on, so on replay, the mouse clicks miss). And as long as the software's behavior is itself deterministic (ie, when the user does exactly the same thing, the software too, does exactly the same thing).
For testing, it can be used for regression testing. To test your belief that everything continues to behave unchanged.
- Should describe details of use, to help folks get started. Including using it to take screenshots. MitchellNCharity 18:23, 31 January 2008 (EST)
Interactively driving multiple XO's
By interacting with a single XO, you can simultaneously control several others. This is just a real-time variation on recording and playing back an event log.
You log and interact with one xo:
$ unbuffer laptops/one/xo do xmacrorec2 > log
And elsewhere (you will have to kill them yourself when you are done):
$ tail -f log | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/two/xo do xmacroplay
and
$ tail -f log | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/three/xo do xmacroplay
etc.
Everything you do to laptop one, will also be done to laptops two and three as well. It's rather neat to watch.
This could be used, for instance, with several laptops, configured in different languages, to run a demo or test, and get a full multilingual set of screenshots.
When you only need a single slave laptop, the commands can be simplified to:
unbuffer laptops/one/xo do xmacrorec2 | grep --line-buffered -v 'Delay ' |unbuffer -p laptops/two/xo do xmacroplay
unbuffer is from the expect rpm/deb.
You can interact with the slave laptops normally, without interfering with playback, so you have some ability correct for "they didn't all do quite the same thing" differences. This, plus the ability to adjust your actions to the full set of responses, rather than only to one, is what makes this simultaneous multiplexed approach useful.
See also
- http://linuxbrit.co.uk/scrot/ Screen capture tool.
- http://homepage3.nifty.com/tsato/xvkbd/ Virtual keyboard. Not sure how relevant this is. But might be combined with http://projects.o-hand.com/xoo/ to create a "virtual xo"? :)
