OLPC research/Evaluations review 2009-2010

Evaluations of OLPC projects: A review of 2009-2010 reports

This review was presented at the Innovation in Evaluation workshop on April 4th and 5th, 2011 at OLPC Cambridge, MA.

We conducted a review of the evaluations to projects that implement the One Laptop per Child’s 1:1 educational computing model across the region. For each evaluation report we describe the indicators measured and tools used for data collection. The indicators are divided into two groups: social impacts and impacts related to learning. Some reports also measure aspects related to program implementation. We also include a summary of the key findings from each report. Finally we summarize the most common indicators measured throughout the eight reports as well as most common tools used for data collection. The second part of the document includes two overview reports, followed by a summary of the main conclusions obtained from the evaluations analyzed.

Educational Monitoring and Evaluation of Plan Ceibal

Report date: 2009 Country: Uruguay Author: Plan Ceibal Evaluation Department http://wiki.laptop.org/go/OLPC_research#Official_overviews.2C_December_2009

Plan Ceibal was launched in 2008 with the vision of bridging the digital divide as a factor of inequity in the country, through universal access to computers and Internet. In 2009, the distribution of 395,000 XO was completed to children in public primary schools in the country.

Indicators related to Social Impacts	Program Implementation	Indicators related to Learning Impacts	Tools
Locations for internet Access Frequency of use of XO by children	Operating status of XO Teachers’ opinion on training Evaluation of Plan Ceibal Problems at school level	Activities preferred by children Activities used for homework Time needed to learn to use the XO (children) Uses of XO Motivation to use the XO in class Teachers: Instructional planning	Self-administered surveys for children, teachers, principals and parents Computer based text production by children Individual and group interviews

Key Findings:

• The internet sites have changed from cyber cafes (65% in 2006 to 25% in 2008 in the country side), to school (32% in 2006 to 67% in 2008).
• The teachers believe that most children take less than two weeks to learn to handle the XO, and most learned by individual exploration.
• Over 50% of teachers report they include the XO in instructional planning at least once a week, and up to 21% almost daily.
• 92% of children reported using the XO for homework.
• Browse activity is the mostly preferred activity of children of all ages, a preference that increases with grade. The same pattern occurs with Etoys, and Record activities.
• Activities such as Write, Paint, and Memorize tend to be less preferred by children as they advance in grades.
• About 90% of teachers have used Browse and Write activities mostly for teaching purposes. Browse is mostly used in higher grades and Write activity in lower grades. Half of teachers at schools without connectivity opt to use the writing activity, and secondly -1 in 5 - the browser to search for content on the XO digital library. Finally, 1 in 10 used to chat and recording of photos and videos as alternatives for use in school.
• 77% of children said they like to work more with the XO in the classroom.
• Access to the XO not only closes the gap in access to computers and the Internet but also its use levels the students who don’t have a computer at home with the ones who do.

OLPC pre-pilot Evaluation Report (Haiti)

Report Date: 2009 Country: Haiti Author: Inter-American Development Bank http://wiki.laptop.org/go/OLPC_research#OLPC_Haiti_pre-pilot_evaluation_report_.28IDB.29

Haiti received more than 10,000 XOs part of the initiative Give One Get One program; the pre-pilot program was conducted during May and July 2008, implemented during a summer camp with 116 students.

Indicators related to Learning Impacts	Tools
Uses of the XO, activities, and frequency of use Teachers: Teacher’s knowledge of the XO Characteristics of teaching environment: Level of student attention and focus Teacher discipline system Level of participation and encouragement of the teacher Relationship between teachers and students Student attitudes and behaviors	Classroom observations of teachers and students Tracking the use of the XO through the journal Interviews to students, teachers, administrators Case Studies

Key Findings:

• The distribution of XOs was 1:2, 1 laptop shared by a pair of students. Only a limited group of 4th graders took the XO home throughout the duration of the camp.
• The attention span of students decreased because of the following factors: when they shared the XO between students, short duration of battery power, low commitment / motivation from the teacher.
• 4 of 17 activities available on the XO (Write, Write, Browse and Painting) represented 88% of the use of the XO from camp participants.
• The tendency to use certain activities were strongly related to the instruction provided by the teacher, the use of these activities showed a steady increase, while the exploratory use of other activities decreased.
• 40% of students explored to reach a point of frustration or confusion, and were shy or reluctant to seek help from the teacher
• Students and teachers use the XO to practice or improve reading and writing in Haitian Creole and French, 56% of teaching staff noticed an improvement in the students in this aspect as well as the positive effects of having content in these languages for improvement in reading comprehension.
• 40% of students believe the XO could facilitate learning and provide opportunities for the future.
• Teachers think that the XO could boost school attendance, interest and independent thinking, thus contributing to a better understanding and learning