Research: learning and achievement outcomes

These two sets of notes need to be merged

1. Speakers
   1. Arlene Bailey, monitoring progress
   2. Sameer Verma
   3. Stacey Kurtzman
      1. Johannesburg South Africa
      2. partnering schools in collaborative learning environments
      3. Collecting information on health and safety issues
2. Stories
   1. Jamaica project: 150 XOs deployed
   2. Teachers are changing their methods away from rote learning techniques towards encouraging children to learn in informal spaces
   3. Mentors at some schools get together after the lessons to share experiences and update their personal journals. These groups have provided a way of tracking progress across classes and cross-fertilising the ways of teaching with the XO that can be seen to be having the biggest impact
   4. In one area there was an increase in test scores by 21% over 2.5 months where the XO was used to enhance the learning experience, in this case for the topics of maths, art & problem solving
   5. Some areas have no books in the home or in the school until the children go to secondary schools; putting free ebooks on the XS and XOs at schools has no benefit as there is no tradition of book reading in the culture. In such cases you either need to focus on the activities that would be used naturally by that culture or have a specific initiative that expounds the value of reading books.
   6. Building lesson plans based on activities in sugar enables those lesson plans to be shared and distributed with the activity
3. Looking at qualitative assessment of
   1. Areas
      1. Technology adoptions
         1. Relative advantages
         2. Complexity
         3. Compatibility with the work environment
      2. Educational assessment
      3. Social outreach
         1. Research has shown that technology can bridge social gaps as kids with XOs or with interest in particular activities on XOs get to know each other through this common connection
   2. Approach
      1. Want to develop Sugar software to pull out journal data and aggregate it to determine how the story told by the students and teachers correlates with the journal activity
      2. Determining the correlation between qualitative assessment by teachers and quantitative assessment through aggregated journal analytics
4. Challenges
   1. How to measure the progress of non formal learning outside the classroom
   2. Difficult to get data on community involvement
      1. Mentors and teachers can share and journal their lessons learned after class
   3. There are researchers wanting to look at the difference OLPC makes and there are people out there doing deployments; how can we join the two together so the researchers can get hold of the real data from the field to assess the impact of the new approach?
   4. Teachers may not be confident in using the XO and may not feel able to share that with the mentor. The mentor can look at the journal on the teacher’s XO and quickly see whether there is a correlation between what they are saying in terms of their familiarity with the activities and their actual use of them; this can provide the mentor with valuable information about where the teacher may need more help
5. Ideas
   1. It would be useful to get a visualisation of the journals across an entire population within a class or within a school.
   2. Developers could visit the schools to see how the XOs are actually being used; this could provide valuable insights into content or activity development that could aid use of the XO for that culture or curriculum
   3. Analytics to visualise which deployments are using which activities
   4. There is an inventario activity that augments journal data with additional information on child sex, age, etc.

---

Notes from Jessica Curtis jess.ganas@gmail.com

http://forum.laptop.org/chag/join#olpc-553

Sameer Verma Arlene at U of West Indies, ICT in communities, monitoring and evaluation Stacey Kertsman: , eduweavers (http://eduweavers.org). Partners schools and learning environments. School has a potential to be a site, emerging markets, developing world. U of California, UC Berkeleley. And U of South Africa, especially around health initiatives.

Jamaica: UWI (http://uwimona.edu.jm) + SFSU (http://www.sfsu.edu). 115 laptops. Ages 5-6, other school grade 4 (age 8). Looking at tech adoption, education assessment, and social outreach. See if can do qualitative assessment of these things. Grounded in Rogers approach to diffusion of innovation and adoption.

Journals of each school. Want to get quantitative data from journal including time stamps, (quantitative). See if there is any correlation between qualitative side and what the journals say. Correlation is all we can establish, not causality.

Baseline data, interviews with children and parents, re: what experience is, expectations. Educational assessment: both schools have initial tests, standard tests, not measuring creativity. Social outreach, some of these communities are on the poorer side. Researchers have shown that technologies can bridge the gap. A friend who has an XO, there might be bonds forming.

Sameer did his sabbatical at U of West Indies. That’s how he connected with the project there.

Stacey. Creating school-to-school partnerships, teachers can create curriculum together. Father professor at UC Berkeley. His questions: Health, food, nutritional security. U of witwadesrand. Distributing information to children, how does it shift what children request and consume, HIV and STDs. Developed a curriculum with self-reporting of kids. Players in conception: SA Center for Epidemic Modeling and Analysis, UC Berkely with data and Center for emerging and neglected diseases. Eduweavers, SA university. Mangusi education center, the local center where the initiative would occur. It’s not an American research project: also SA and local (center for education – the most effective practices to collect data). A collaborative initative.

Morgan Ames - at Stanford, and in Paraguay 130 interviews, qualitative work. Some work on Journal, though journal is not a good data source, in Paraguay kids don’t use their journals. Working on assessment, requested by Paraguay educational team (a cognitive reasoning test of children).

Harriet Vidyasagar - Goa, 105 children, work backed up on Journal. Have documented a baseline, what they what to learn, and need someone to work with their data.

Paraguay: planning and evolution. Have shifted plans.

OLPC Barbara Barry, director of learning for the OLPC foundation, focusing on the middle east. “You (Paraguay) evaluate, we learn.” Motivation goes up a lot, but how will that last over 5 years? Use it, psyched, learn with friends and school. Teachers are actually changing their methods. XO dropped into place to give people an excuse to change their methods. Laptop encourages learning in informal spaces, but not measured or evaluated.

Sameer: in Jamaica project, can see what activity happens in school and what will happen after school Laptops must go home with the kids. Can also interview kids and parents.

Tim Falconer: What is it, why should I care, can you prove it? Wafeplace: journal studied with scripts. Request people write on paper or whatever, after each class/activity. Waveplace can find out the issues, and it makes the project the kids/teachers’s.

Stacey: school-to-school project. School in San Rafael is 1-1 Mac program 6th-8th grade, SA has laptops. Journalling is implemented in different ways in both locations. 4 teachers and 8 students coming to San Rafael; Get buy-in.

?? Great documentation: Romania survey??? Great TED talk about evaluation aid projects.: people’s time frame: progress can take decades, not just for the year or so that the person is involved in an aid project. Some kids can watch for years, then suddenly doit. Survey Monkey. Use of community, “when I’m rady it will be there.” Pride, etc. Hard to measure. In Film, what are the best things you’ve learned in the program, q 6 months. Getting over fear of computers, film skill, pride and personal growth. Their exam is to write back to us what they’ve learned.

Mike Dawson from Afghanistan. Did a baseline survey, increased results about 21%, before-after without a control group. If you oook at all the research about 1-1 computer. You change the impact as much as you give creativity, math etc. tools and time of contact. Are you going to deliver better than a conventional education, XO is very expensive, vs free conventional education. No use comparing to baseline. Looking for improvements in logical problem solving ability, tests of creativity, e.g. how many different approaches a personcan generate tosolving a problem, then determine what is responsible for the change. Compare to those with XOs, those without, and a group of comparable increased resources. Result: only we know now, 21% increase of curriculum tests results in subjects we used XO to teach, which were math, problem-solving. Time-frame 2.5 months.,

?? Nadine M. For small problems, how can we show results so small projects can grow into large projects?

How can we put together researchers with deployments that want to be measured? Concept of open data; how much will be open for other people to use?

JOIN RESEARCH LIST at lists.laptop.org. SUGAR LABS: Developers want to know what the teachers want: no feedback loop from the teachers. Developers work on what they think is important, no infor from teachers.

Give the teacher an XO so they can journal. So we can see their journal. Teachers don’t want to admit they don’t know. Borrow the teacher’s XO for a few minutes and check out their journal, support those who are having more trouble. Does the teacher have an XO to take home?

Tim: “My Twee” twitter stuff. Visualization of usage?

Paraguay: disconnect between teachers and developers. Have developers go to a deployment and experience it. Want to go to Jamaica?

Open data: a call back routine, calls home when machines get online, “I’m here, this is the build I’m running.” How can we exand that function?

Huge safety concern: don’t steal any children. Tabitha, don’t just turn up at schools.

Maps to see which deployments are using which actities the most? Size of application usage? That way we can see who’s actually an expert in a given application/activity.

Morgan in Paraguay. System in Paraguay that tracks all the breaks, etc. Inventaria. Who is using the XO, which gender, age, etc. etc.

Uruguay: Univwersity. Flor… Education of U students, outready to community, and research. A lot of research has been done but not very systemative. Ned more diagnostic tools. Arlene: do students train users? A course in school, trained for first month. Goal is a multidisciplinary approach.

Tim: talk withwalter bender and Sugar: Make it possible to add notes to journal, make it part of linear record of use. Can put hooks into Jornal to get this data? Talk to a developer. “write to Journal anytime.” A chat log where teacher can come into journal at any time.. MIT market team, MBA students. Get the developers down to the learners to sit with them. Devellpers come from…

Barbara: Is OLPCD working in this community or country? OK. But are computers used in education? Can we gather data on how kids are doing in the moment, can adjust their learning before they get labeled as behind, moderate or excellent? Real-time feedback loop can work toward personalization.

Brewster Kahle, Internet Archive. 1 million books are available to kids on the XO, 200 languages. NO FEEDBACK YET RECEIVED. Do the back end. But don’t know how to do the outreach. If you donate a book, we’ll digitize it. 1,000/day. Lending library for in-copywrite books. Tabitha: hard to find downloadable ebooks: have to do it one at a time. Archive.org. Does anyone read the books? In those cultures, people are not traditional book readers. If there is no feedback, the thing has to exist for a while, people hasve to try it, feedback is after some time. PLEASE DON’T STOP.

A repository for wish-lists from teachers?

NGOs want feedback on how their money is being used. Will champion be burdened with this aw well?  

Invite more schoolteachers who do the daily work. This project is very technology-heavy.

Lesson-plans: do them around sugar, then we can send the lesson plan with the activity.