Pen Tablet UI/User Study

This page describes a user study that was performed on April 8 and 10, 2008 by Patrick Dubroy in order to evaluate prototypes for a Pen Tablet UI.

Method

Participants

Five participants were selected for the study. No participant was familiar with the XO hardware or the Sugar UI, nor did they have any experience with a conventional graphics tablet. [More details to come]

The number of participants was chosen based on Jakob Neilsen's rationale (Why You Only Need to Test With 5 Users) that this will find approximately 85% of the usability problems in a design. Given the early design stage, this seems like an appropriate approach.

Study method

Partipants were asked to perform a several drawing tasks using the Think Aloud protocol. Before beginning the tasks, participants were given an opportunity to become familiar with the OLPC and get a feel for the use of the tablet. This warm-up period was also used to get the participants comfortable with using think-aloud.

During the drawing tasks, video of the screen and tablet area was captured along with audio. Afterwards, I reviewed the videos and made notes of any interesting results, behaviours, or comments.

Results

Detailed notes from each participant:

• C
• S
• M
• A
• MO

Fixed Mode

• two users did not understand it at all
• everybody said that they like Movable Mode better

Movable Mode

• some users tried to use the touchpad to move the mapping rect, instead of using the stylus
• due to the pressure required with the stylus, it doesn't really seem that there would be any reason to use the stylus rather than the touchpad. Although, we could easily allow both to move the mapping

Alt-hover

• 2 users (S, M) weren't initially pressing hard enough to see the crosshairs. It seemed like because there was no line being drawn, they thought they didn't have to press as hard
• 4 users (C, S, A, MO)thought that once the crosshairs were in the right place, they could lift the stylus off the tablet, like in Dynamic mode. It's unclear whether this was due to having learned Dynamic mode first

Dynamic Mode

• if you accidentally stop drawing a line -- e.g. by releasing a bit of the pressure, or by going off the edge of the tablet -- things go totally wrong, and a new line is started at the cursor.
• an ever worse version of this (because it's less obvious what's happening) is if you don't press hard enough while drawing a line. You end up getting a bunch of marks being made right under the cursor, with the mapping rectangle sliding in the opposite direction
• one user (A) commented that it would be hard to draw a dotted line or something similar
• one user (M) didn't realize she had to move the cursor for every new line that she wanted to draw, and another (A) sometimes forgot to do it (it didn't seem to come naturally, for lines that are close together)
• 2 users (S, A) would set the cursor, but then being drawing in a way that wasn't completely consistent with how they had placed the cursor
• users didn't generally feel that the rectangle was useful to display
• 2 users (C, M) thought they had to click the button to set the start of the line in DYNAMIC mode. But, that didn't actually cause any problems, because it would still work just as they intended

Problems with the prototype

• in Dynamic mode, the lines did not seem to start *exactly* where the cursor was
• in Dynamic mode, the cursor often obscured the line that was being drawn
• the mapping rectangle should be *exactly* the same aspect ratio as the physical tablet, and maybe have the same grey marks as the tablet to make it obvious exactly how it maps
• dragging with the stylus seemed to be very slow -- it should maybe be accelerated, and support for dragging with the mouse should be added

Problems with the study methodology

• having 3 figures to draw in each rectangle encouraged people to think of the screen as being divided into thirds, which wasn't necessarily representative of a normal drawing task

• since the exact placement of the objects isn't that important, maybe go with a different task next time

• using Dynamic mode before seeing the Alt-hover may have had a negative learning effect on their use of Alt-hover. Should consider doing the Alt-hover task before teaching them Dynamic mode

Problems with the hardware

• people consistently had some trouble with pushing hard enough on the tablet. It shouldn't be too much of a problem to learn to push hard enough, but this might be difficult for children.
• one user (A) commented that because the tablet surface was higher than the table where she was resting her hand, that she had to hold the stylus different than she would a regular pen
• it's difficult to get any events on the absolute edges of the tablet area
• it's not obvious why you can't use your finger on the whole tablet area, not just the middle
• there is an area about 0.5 cm around the edge of the tablet area that is not actually pressure sensitive. The physical barrier should be at the same place that the sensors stop.

Notes

• software should allow people to move the shapes they draw
• If the physical tablet had crosshairs built in, you could use a movable mapping mode that wasn't contrained in the x-axis, then they could just use that for precise drawing. You would move the on-screen crosshairs to where you want to start drawing, and then put the stylus down on the corresponding physical crosshairs

Conclusions

Methodology

There were a few issues with my study methodology, as noted above. Recommendations for a future study:

• Test Alt-hover mode before introducing Dynamic mode. There may have been a learning effect from Dynamic -> Alt-hover. The effect would probably not be present in the opposite direction.
• Test precision modes separately from mapping modes (as I learned from the first test subject, and corrected)
• Modify the drawings to be a bit more representative of realistic tasks, and do not use 3 equally spaced graphics

Future Design Prototypes

• Movable mode seems to be equal or superior to Fixed mode
• All participants found Dynamic mode to be quite useful and intuitive, however it also has some serious usability issues, as noted above. We should consider a hybrid mode that combines Movable (for the majority of drawing) with the ability to enter Dynamic mode when high precision is required
• Putting crosshairs on the physical tablet and using a Movable mode that also allows movement in the X axis might make a separate precision mode unnecessary. However, this would require modifying the XO (although only to use a permanent marker to make a small mark on the tablet area, which seems reasonable)