OLPC Rochester, NY/Usability testing class project/10 heuristics

Applying Ten Usability Heuristics* to the OLPC (One Laptop per Child) “XO” Laptop

* from Jakob Nielsen, Ten Usability Heuristics with 0-5 point scoring.

The following are the comments of one undergraduate and two graduate students in a Usability Testing course in Rochester Institute of Technology's masters program in Human–Computer Interaction. It is based on about one week of personal and small group interaction in March 2008 with G1G1 XOs (that included the early software build #656) and with limited review of the project wiki. Many of the observations have been noticed by others, and subsequent development of the Sugar user interface addresses many of the noted shortcomings. Annotations have been inserted as indented text to provide a more complete review.

Please note that the OLPC laptop is named “XO” and the Linux-based graphical interface designed for it is called “SUGAR”.

Visibility of system status

2.5/5.0

The SUGAR interface targets elementary school-aged children and is considerably graphical, displaying only icons devoid of labels unless highlighted by roll-over with the cursor. While this simplistic design lends to the main interface’s intended ease of use, it also results in a lack of feedback of the system’s status to the user.

When the laptop is turned on, the user is informed by a short chime and the XO logo. The logo is a very simple representation of a child with two crossed lines for the arms and legs (“X”) and a circle for the head (“O”). The start-up progress is indicated by a single dot below the logo that multiplies in a clockwise pattern as the “arms” of the logo swivel (appearing now as an arrow or carat) to point at each newly appearing dot. This visual animation resembles a clock and, while the metaphor may be lost on those in the target group unfamiliar with the clock-face, it is surmised that the user could expect the completion of the boot-up sequence when the swiveling arms return to their starting position (completing both the XO logo as it first appeared and the circle of newly formed dots).

The circular theme is continued when the start-up completes, placing the XO logo in the center of a segmented circle (known as the Activity ring). As noted earlier there is absolutely no text on this “home” screen unless the cursor is rolled over an icon to reveal a pop-up label with information and/or options for it. Clicking on one of the application icons along the bottom of the home screen starts it, and the icon appears in another segment of the ring that surrounds the XO logo. While the application loads, its icon in the ring pulsates to indicate that it is running before the entire display is replaced with the newly opened main application screen. As more programs are loaded and added to the ring however, the limited resources and processing power of the XO unit leads to long loading times (as long as a minute or more for some programs) that may leave the user wondering if an error has occurred. Similarly, open wireless or mesh networks are represented on this and other screens as small filled circles and their connection progress is indicated only by the entire circle flashing. Long delays in either situation, with little more than a pulsating or flashing icon, can leave the user confused about system status. Unfortunately, these are the most effective feedback indicators observed in SUGAR. The rest of the interface does not communicate system status as well.

There are four major screens/views that are quickly and easily accessible by four icons in the upper left border of the screen or by identically mapped and labeled buttons on the keyboard. They are “neighborhood”, which shows all available mesh & WiFi networks along with shared activities and other users; “group”, which shows the XO user’s “friends”; “home”, which shows the segmented ring indicating all running programs/activities, other available applications to run, and battery & network information; “activity”, which shows the main screen of the last running activity or a “journal” of past activities. Not one of these views explicitly states or describes what it is or what is happening while the user is on it. With time and experience the user may learn and remember the purpose of each screen but, until that time, the child must experiment with roll-overs and vague icon symbolism. For example, “friends” are represented on the “group” screen as identical XO logos but are grayed out when they are not presently connected, and the currently active/connected network is differentiated from many other virtually identical icons on the neighborhood view only by a thin white outline.

Although the various programs available for SUGAR have unique visual interfaces, many of them also provide poor feedback of program status or progress to the user. While collaboration and sharing of activities is a core concept of SUGAR and the OLPC Project, when a user explicitly shares an activity with another user or group, there is no notification to the others. Instead, “invited” users must navigate to the neighborhood screen where they may see an icon for the shared activity. Additionally, the original user who initiated the sharing waits without knowing if anyone else is aware they were invited to collaborate. In a test trial involving a shared writing activity, one user repeatedly attempted to communicate with the other after they had left the collaboration because there was no indication of the other participant’s departure. The lack of feedback is present throughout the main SUGAR interface as well as the various applications that run on it.

There are status indicators located on different surfaces of the unit (visible when opened or closed) that illuminate to indicate activity. The power icons remain lit as long as the unit is powered on, while the battery icons illuminate green when charged (or charging) and red when it nears depletion. However, when the unit’s battery power nears full depletion there is no other warning to the user so that he/she may save any progress before it shuts down. The wireless signal indicators blink when there is connectivity or wireless activity. There are small LEDs above the camera and microphone that illuminate to indicate when they are active and these are the only status indicators that are not visible while the lid is closed.

These issues could be addressed by better feedback indicators, such as a larger or slightly changed icon for connected networks or a simple textual message that communicates that the system is working. Arrows, flashing indicators, or pop-up notifications could be used for invitations or shared activities. Progress bars and meters could also be used, not only to indicate that a program is loading, but to also estimate how long it may take.

This discussion thread by the Sugar designers reveals many design changes that should resolve a lot of the shortcomings noted in this section.

Match between system and the real world

3.0/5.0

The XO units we had in our possession were “American” units, but we have assumed that the icons in the SUGAR interface are the same for every version of the XO.

The XO logo that is branded on the cover of the laptop is used within SUGAR to represent the user at the center of the system and activities, as well as other XO users. The concept of the child users being represented by the “XO” initially appeared to be an easily understood, universal symbol. However, the rather abstract representation could resemble letters, glyphs, symbols, and warning signs in other cultures. These alternate meanings could conflict with the intended use in the SUGAR interface.

The icons representing the four main SUGAR views: “neighborhood” being a ring of dots; “group” being three dots in a triangle formation; “home” being a single, central dot; “activity” being a square, are also rather abstract representations that may have little or no initial meaning to those in other cultures. In fact, the concept of “home” being the starting point as used in this (and other) GUIs may be a difficult concept to grasp. In some cultures a child may share his or her home with a large group of people causing the distinction between “home” and “group” or “neighborhood” to be confusing.

In addition, the various icons used to represent the different applications are all questionable in the same manner. While many of us are familiar with the “chat” bubble as it is shown in comics and cartoons, it may be a completely novel concept and image to those in foreign countries. This may also be true for many of the other icons: the calculator, the drum, the musical notes, the battery, and the hourglass – all may be objects that have never been seen by those in developing countries. Some of these icons even left us wondering what programs they started. For example, none of us could predict that the “shooting star” loaded the Etoys program while the “eye” started the camera.

Many of the icon representations are “Western” concepts that may have little or alternate meanings in other cultures. We realize the difficulty inherent in attempting to create a “universal” symbol to represent many, if not all, of these programs – especially if users from other cultures have never been exposed to such concepts before. After some instruction and explanation as to which symbols represent which programs, the users may remember them without ever fully understanding the meaning behind them. It may be more important at this time that the icons differ enough from one another to be easily distinguishable so that they are readily recognizable once the user learns what they stand for.

User control and freedom

2.0/5.0

When programs are started, there is no clear, consistent way to exit them. While some applications immediately display an “X” in the upper right hand corner when opened, it is only visible in others if the “activity” heading (similar to a file menu) is selected along the upper portion of the program screen. If the user does not select the “activity” menu to see the “X” (exit) button/icon then the only way to “exit” the current program is to press the button for one of the four main screens (neighborhood, group, etc) on the keyboard or on the border of main display (sometimes only visible if the cursor is pushed to one of the four corners of the screen). However, since switching views does not end the previous program, it is left running and uses some of the already limited processing power of the XO (slowing down the system significantly).

Within programs, the undo functions must be “found” by selecting the correct “file menu” heading. In the writing application, for example, one must choose the “edit” heading in order to see the “undo/redo” icons (arrows curved from left-right-left and vice versa). While this is the same for the paint program, other applications do not have these functions at all. After some repeated experience with programs containing similar menu set-ups and options, these functions may be found more easily if they exist.

There is a key on the keyboard marked “X” that does not seem to have any functionality at this time. With future updates, it is possible that pressing this button at any time will quit or exit any current action or activity.

Consistency and standards

3.5/5.0

As stated previously, some applications can be exited by pressing or selecting the “X” icon in the upper right hand portion of the screen. In some programs, however, this “X” does not appear unless the “activity” menu heading along the top is selected. This inconsistency makes it difficult to exit or quit an application.

The use of the “X” icon to represent “exit/quit”, however, is consistent in that it serves the same function on various other views in SUGAR. On the “home” screen, placing the cursor over any of the current running application icons reveals a pop-up label menu that contains a “stop” option preceded with the “X” symbol. On the “neighborhood” screen, however, the “disconnect” option appears only as text without an “X”.

Throughout the different views in SUGAR, users are represented by the XO logo and networks by a circle. The extension of these icons does not permeate through to the individual applications at this time. For example, the chat program does not represent participating users with their custom colored XO logos.

The different applications do consistently display menu and file options in a row along the top (if they have them at all) as well as various icons along the very top that reveal their function when the cursor is rolled over them. These menu options and icons vary depending on the application being used but are generally persistent across applications.

Error prevention

3.0/5.0

As stated earlier, when the unit’s power is fully depleted, there is no warning to prompt the user to save any open work or current progress before the system completely shuts down.

When activities are started and the user chooses to exit the program, there is no confirmation dialog to do so even when current work or progress is in session. The application will immediately exit without saving or prompting to save.

In either of these instances, all of a child’s hard work can disappear with the simple press of the exit button or loss of power.

We were not able to perform any actions that led to error messages. SUGAR may be designed in such a way that it only allows users access and options that do not result in errors. In that respect it succeeds in limiting errors.

Recognition rather than recall

2.5/5.0

The “familiar” “X” button or option to exit an application is not always visible in the upper right hand corner of the screen. At times the user must select the “activity” menu option in order for this to appear.

Access to the four main screens of the XO interface are always immediately available on the upper left of the keyboard or can be made visible on any screen by pushing the cursor into any of the four corners of the screen. These two options allow the user to quickly access any of SUGAR’s main interface views.

While the icons representing applications may be confusing or hold little meaning in another culture, rolling the cursor over them reveals a pop-up label that will indicate what the program is and what options it has to select from. Without this rollover action, however, there are virtually no text indicators on these screens. Since the text labels are not visible without hovering the cursor over an icon, the user is forced to remember what the various symbols mean and how to interpret their various states (e.g., flashing network circles).

Flexibility and efficiency of use

1.0/5.0

We were not able to identify any accelerators in the XO system. Perhaps the four hard keys that are mapped to those on the screen to quickly access the main areas of the interface could be used more by experienced users in the future – but these are accessible at any point in the user experience. There does not seem to be any customizable aspect of the interface that could lead to more efficient or flexible use by more experienced users.

Aesthetic and minimalist design

2.5/5.0

The interface design of SUGAR is very minimal and simple. On the four main SUGAR screens text does not exist unless an icon is rolled over with the cursor. Instructions that exist and appear when an application opens are very short and concise (e.g. opening “chat” briefly displays the message “Off-line. Share, or invite someone”). However, the presence of such messages is very rare. Many of the programs that run on the XO do not have dialog boxes that appear or instructions that can be found. The programs simply load with little to no guidance or instruction.

We discovered that, for many programs, there was no indication of how to use them at all. The only option available was to simply attempt to use them by experimenting and trying different options and functions. In this respect, SUGAR’s interface was overly simple.

Helping users recognize, diagnose, and recover from errors

2.0/5.0

During the use of the systems in our possession we did not see any error messages. We intended to force an error by attempting to connect to a secured (encrypted) wireless network with an incorrect key but still only managed to have the key entry dialog re-appear.

The only error experienced more than once occurred during the shut-down sequence when multiple lines of white text on a black screen appeared briefly before shutting down. The error text lines were in a very small font, cryptic, un-descriptive, and did not allow any opportunity for addressing the problem or methods of prevention in the future.

Help and documentation

0.5/5.0

There was no help or documentation anywhere in the XO system. Virtually every program began with no instruction at all and no help option to get the user started (Etoys did have a tutorial and demo option). There are some programs, such as TurtleArt and TamTamJam, which we still do not know how to use.

The XO’s were designed with the goal of having children work collaboratively as well as independently – however, it would be difficult for children with little to no prior computer experience (or concept of computers and programs) to learn how to use a system with no help and/or documentation.