OLPC Rochester, NY/Usability testing class project

Top page for class project.

Testing project abstract

The OLPC (One Laptop per Child) Project provides school-aged children in developing countries with specially-designed “learning”machines (the “XO” laptop). These machines are intended to allow students access to new tools as well as a window to the rest of the world. Although the XO is in mass production and has been widely distributed, the open-source software at its core is under continuous development. Considering the target user-group of technologically inexperienced children, we will examine the ease of use of the XO's “SUGAR” graphical user interface for elementary school children by focusing on the built-in Write activity (application/program). The OLPC’s ideal of collaboration will be studied by observing how the children interact and perform in this shared activity which offers multiple simultaneous cursors for synchronous editing. This is a relatively unstudied social computing capability.

Testing will be executed on site at the Wheatland-Chili Central School District elementary school on multiple dates. An introduction to our team and our intentions will be carried out by the school superintendent, class teacher, or other school faculty/staff member. The tests will then be conducted in two stages, using different participants in each stage. The initial test format will be almost exclusively observational—allowing the students to freely interact with the XO laptops and taking note of their various experiences. The second phase of testing will involve a new set of students, studied in pairs as they interact on separate XO’s using the Write activity.

Following a short period of familiarizing each student with the basic functions of the XO laptop, we will wirelessly network both units so that the Write application is shared between them. Each student will then be provided with a clear, large-print document containing approximately 6 to 8 lines of simple text (we intend to use verses of a Dr. Seuss book). Different lines of text will be one of two colors (e.g., RED vs BLUE) and each student will be assigned one of the two colors. Their task will be to re-create the provided verse using the shared Write activity. Each child will be responsible only for typing in text displayed in their assigned color. We will observe the manner in which they perform the task (i.e., waiting and taking turns or working simultaneously). We may also alter portions of the text, such as adding highlights or bold letters, to measure their success in using some of the features in Write.

Both stages of testing will involve video recording (including voice) of the sessions. One to three cameras will document how the students interact and work with each other and the XO units. We will attempt to restrict and focus all visual recording to the students’ physical interactions with the XO (on the hands or over-the-shoulder at the screen activity). The video footage will then be reviewed and edited or altered in such a way that any instances of a participant’s face shall be censored or removed to protect their privacy.

We expect the observational test phase to last approximately 15 minutes per “free play” session. The writing task phase is expected to last approximately 5 – 20 minutes, or less if the task is fully completed. In the event that a pair of students encounters difficulty completing the task, we will conclude the testing after a cap of 25 minutes. At the conclusion of each session, the students will return to their normal instruction in the classroom.

We are considering the distribution of individually bagged pretzels or another approved snack as a reward for participation. After all testing is completed, we are also considering providing the class with a collection of color printouts containing the various verses used in the test session. Each page of verse will include an area where the participating students can write their names in the color they were assigned. All of the printouts could be arranged and combined to reveal a larger portion of the work it was selected from (Dr. Seuss) and the participating children can share with and show others what they accomplished by working together using the XO laptops.

16 March 2008 Meeting

NOTE: The contents of this page are not set in stone, and are subject to change! This page is a draft in active flux ... Please leave suggestions on the talk page.

Draft notes: Image:Meeting1Summary.doc Meeting #1 Description OLPC
Attending: Deidra Robinson, Man Truong, Jacob Barber, Frederick Grose

Goals and Expectations:

Since the OLPC laptops are relatively new, we realize that there is a broad spectrum of areas we could focus on. For purposes of our usability tests we will focus on a piece of software or software related issues. The client realizes that there have been some complaints about the laptop, but leaves it up to us to find the problem area/s. The client would like us to discover the area where we can contribute the most to developers in the time allowed for our testing. In this meeting we did not settle on one particular application or idea; however, we did narrow down possible topics to existing software on the laptops.

I think we are leaning toward testing one of the 'collaboration' facilities after reviewing what's available and how to procede theoretically and practically. --FGrose 21:37, 16 March 2008 (EDT)

It is expected that we will share and track our testing and findings on the OLPC wiki website. Also the client would like to take a very active role throughout this process. There are some plans already in store for the laptops that the client understands we are not aware of and we have come to terms on the limitations of our studies.

Types of users that you will want to represent with participants:

The OLPC laptops are sent to underdeveloped countries for children, teachers, and parents to use as a learning tool. The main audience is children between the ages of 6 and 12, with an even more targeted focus on 6 – 10 year olds. This targeted focus is because the laptops are usually given to the younger population and the students keep them for 4-5 years, so by the time they reach the older ages they will be already fairly familiar with the laptops. Since for many of these children these laptops are this may be the first computer they have ever seen nonetheless use we will aim at finding users with very little computer use. Ideally we would find children who have never saw a computer.

Type of test you are thinking about:

We are looking into types of software testing in addition to doing traditional observation and surveying.

Approximate number of participants needed:

Resources limit the maximum number of participants; to get reliable data we will need, at a minimum, 10 participants.

Our final numbers will also be influenced by what we intend to test and how many units we will have available for testing. For example, testing the collaboration between two users and the XO's "paint" program if we only have access to two units.

Access to the system to be tested:

Frederick Grose (the client) informed us that there are about 6 laptops on campus and is working to get us access to some/all of those machines. Stephen Jacobs, an R.I.T. professor (http://www.it.rit.edu/~sxj/), owns one to three "XO" units and Fred obtained permission to loan one of them to Man Truong on 17 March 2008. The XO was handed off to Man on the evening of 18 March 2008 17:30 (EDT). Since Man had a class from 18:00 to 21:50 and works from 08:00 to 17:00 there was limited time available to examine the unit before the next team meeting. This particular XO will be more thoroughly reviewed by all team members during class on 19 March 2008.

The team also anticipates access to multiple XO units during the OLPC Workshop at R.I.T. on 26 March 2008. Our first experience with the XO's collaborative interface to other physically present XO units may be that evening.

In the meantime, two team members will attempt to installed the SUGAR interface/OS on their own machines with access to software packages and installation instructions from the OLPC wiki. All software is open source so we should have unlimited access. We hope to join these emulations in collaboration with any actual XO units in our possession.

Anticipated special challenges for study:

At this point our biggest foreseen challenge will be finding children with little to no computer experience. Any participants' prior experience with Windows or Mac (or other) operating systems and physical laptop devices will influence their experiences with the XO unit during testing. These effects conflict with the intended user group of children with virtually no exposure to computers and operating systems. This variance will be addressed during the actual study.

NOTE: Heuristic evaluation is not possible this week due to no access to the XO unit.