Pen Tablet UI

Pen Tablet UI Patrick Dubroy

This describes the user interaction of the PenTablet, which was worked on by Patrick Dubroy from Feb - June 2008. A summary of the problem and some of the prototypes in the project video.

1-to-1 Case

This describes the case when there is a 1-to-1 mapping between the PenTablet and the on-screen area that the tablet maps to. For example, an activity might allow the child to practice their penmanship. When the child writes on the tablet, the marks will shown inside a GTK widget which has the same aspect ratio, and is approximately the same size, as the physical PenTablet area.

The user interaction for this case should be fairly obvious.

For more information about this case, see Pen_Tablet_Support/GTK_Widget.

Unconstrained drawing

This describes the case where the area that the user wishes to draw on is bigger and/or does not have the same aspect ratio as the physical PenTablet. The Paint activity is a good example of this case. It's not entirely clear how the user interface should support this kind of drawing. The following explores some of the considerations and presents a few different ideas.

In April 2008, an informal user study was performed to evaluate some of these ideas. For more information, see here.

Prototype

I built a prototype to explore some of these ideas. The source is in

 git://dev.laptop.org/git/activities/tabletui 

or on gitweb. Alternatively, you can just grab the bundle: TabletUI-1.xo.

NB: because the driver support for the Pen Tablet is still in flux, this activity forks an evtest process to read input from the tablet. It assumes that /dev/input/event5 is the tablet, which may not be the case on every XO. This hack will not be necessary in the future. So in order for the activity to work, you will either need to run it as root, or make the evtest executable setuid. This can be done from a terminal window using the following commands:

 su -
 cd <activity_dir>
 chown root:olpc evtest
 chmod 4775 evtest

In addition, it can sometimes leave zombie processes lying around, which can be cured like so:

 killall evtest

If anyone has a system where the tablet driver works properly (which I've heard is the case in the latest joyride builds), I welcome a patch to get this activity working with that.

Basic Assumptions

Relative vs. Absolute

It is assumed that the PenTablet will always operate in absolute mode, like a conventional graphics tablet. If the tablet were to operate in relative mode (like the touchpad), it would not be possible to use a stylus to write on the tablet as if writing on a piece of paper.

No control-display gain

A naive way of handling the tablet would be to just scale the Y-axis to match the screen. In other words, one unit of movement on the Y-axis would correspond to 3 units of movement on the screen, whereas one unit of movement in the X-axis would correspond to just 1 unit on the screen. This is not desirable because it would be nearly impossible to write or draw on the tablet with any amount of accuracy.

Mapping

The general idea is that at any given time, when drawing with the stylus on the PenTablet, there will a rectangular area on the screen that indicates how the tablet is currently mapped to the on-screen drawing area. This rectangle will have the same aspect ratio as the physical tablet area, and will be approximately the same size.

The main question is how the user will be able to adjust this mapping to be able to draw on an arbitrary on-screearea. A secondary question is how the user will be able to precisely indicate where drawing will begin.

Adjusting the Mapping

Here are some options for how the user would adjust the tablet-to-screen mapping to be able to draw on an arbitrary on-screen area.

Option 1: Fixed Mapping (Moving the canvas)

In this method, the mapping rectangle would be fixed in the center of the screen. By holding down one of the grab keys, the user would be able to use the stylus to move the canvas. In this way, they would be able to draw on any part of the canvas.

Variation: instead of using the stylus to move the canvas, the user could use the touchpad.

This is the interaction currently proposed for the Draw activity.

Option 2: Movable Mapping

This would be similar to "Fixed Mapping", except that when the grab button is held down, it will cause the mapping rectangle to move around the screen, rather than moving the canvas underneath the rectangle.

Option 3: Dynamic mapping

This is a summary of the idea proposed by Bert Freudenberg and Jim Gettys (as gathered from http://lists.laptop.org/pipermail/devel/2007-March/004302.html).

Using the touchpad, the user would move the cursor to the point where he/she wishes drawing to begin. Then, the user puts the stylus down somewhere on the tablet and begins drawing. The mapping rectangle will then be placed based on where the first tablet event was detected. In some cases, the mapping rectangle might not be fully visible on-screen.

This option would allow for precise drawing, such as drawing a line which connects two dots.

Precise Drawing

In Option 3 above, the user can precisely indicate ever single line that they draw. However, in options 1 and 2, this is not the case. The ALPS PenTablet doesn't support a "hover state" like a conventional graphics tablet. This means that when the user begins drawing on the tablet, he/she will only know approximately (and not *exactly*) where the line will appear on the screen.

There are a few options for how we might make it easier to do precise drawing in the case of Option 1 and Option 2 above.

Option A

When a down event is detected on the tablet, no mark is made yet. An icon appears to show the user where drawing will begin when they move the stylus. This allows the user to take a trial-and-error approach to more precisely specify where drawing will begin. Once the user is satisfied with where drawing will begin, they can drag the stylus across the tablet to actually begin drawing.

This method has been proposed for the Draw activity.

This approach suffers from a couple problems:

• what if the user wants to draw a small dot? They would have to move the pen slightly in a small scribble motion.
• it is difficult to put the stylus down in such a way that only a single "down" event is triggered. More often, what happens is a "down" event followed by a very small move event. We could ignore small movements in this case, but this would be difficult to get "just right" without having an adverse effect on normal drawing

Option B

For precise drawing tasks, the user could enter a "hover mode" by pressing down the Alt key before putting the stylus down on the tablet. As long as the user is holding down the Alt key, and movement on the tablet will NOT cause any marks to be made on the canvas. Instead, an icon will appear to indicate where drawing will begin once the Alt key is released. Once the user is happy with the positioning, he/she can release the Alt key to begin drawing from that point.

Option C

For the sake of simplicity, do nothing. Maybe it doesn't matter if drawing can not be absolutely precise.

Option D

(Suggested by Albert Cahalan)

Upon transition to pen tablet mode, change the pointer to inform the user and start a timer. Drawing doesn't begin until the timer expires. This allows the user to fine-tune location before drawing begins, and allows the user to cancel any accidental usage of the pen tablet before any damage is done.