Metrics, Feedback and Evaluation

Welcome to the pages on Metrics, Feedback and Evaluation

Intro

Wouldn't it be great if there would be objective data to be measured in olpc deployments so it would become unequivocally clear that the investment in an XO-XS deployment pays itself back within 1 or 2 years?

And that organizations specialized in evaluations and auditing would confirm olpc kids excell their peers without XO-XS laptop-x-tablets? Then of course the things they measure, how they evaluate how useful an XO-XS deployment in a kids and families life is, should be clear and universally accepted. Well it turns out this isn't the case. There are evaluation methodologies on how to measure things like school performance, e.g. http://www.oecd.org/pisa/ = PISA = "Programme for International Student Assessment of the OECD". But the only thing that was revealed when applying this or similar evaluation techniques for education, is the consensus these evaluation techniques don't apply / make visible what one laptopschools are provoking in a kids and families life - as the kids take the laptops home.

An Open Source project like the OLPC thrives on ideas, competence, energy and competences that society can be made free, open source and - hardware, open standards. Let's develop the latter here a bit: Open Standards for Evaluation to be measure the impact of olpc deployments, the metrics - what parameters to keep an eye on, how to compare one educational project to that of the olpc educational project, how to obtain the numbers - the feedback, how to evaluate an olpc deployment, statistics? How useful is it? Is it worth the money and efforts? Imagine olpc deployments just destract time/energy/money/ressources to groups that need every coin of money ... this could really be a life or death situation. Horrible.

We need to have some order in these fields and topics, because different groups will be working on more specific aspects of the project. There are already many idea pages in this Wiki about these topics, by the way.

Imagine

Imagine you'd be in front of the Minister of Education, the Minister of Finance, the Prime Minister and the President - of course - , some people of the IMF, a team from a Development Bank and a team from a foundation.

Fortunately you're equipped with a brilliant 10 minute presentation on the OLPC XO-XS educational project. Everybody is hooked by the nice pictures, spot on short video's.

A short Q&A - Question and Answer session of course out of politeness. First buying signals come in as someone asks how much money you need and how this olpc xo-xs pays itself back. The audience has discovered your paper printed documentation holds a paper stating "Mandate", a paper stating "Order form", :) . The financially minded people are silently applauding your practical sense. Some are already clicking their pens, ready to sign the order forms.

Your laptop is already hooked-up to the beamer, anyhow, you start-up the spreadsheet to show how you calculated how much you need for the laptops, portable pv panels, additional equipment, training, electricity, you name it, it's all foreseen in the spreadsheet.

You explain there's a couple of phases the project has to go through - go slow and one step at a time kind of common sense approach:

And that you don't even ask for an indecent giant amount of money - say 1 billion € to educate the whole country to the next level - , but just in a first phase a mere couple of 100.000's of euro's to get through the first phase, to sensibilize and inform their respective cabinets/specialists, the press and society, and get all the required data relevant to the country, provided by the specialists of the country who are of course in the Cabinets of the Minister of Education, the Minister of Finance, the Prime Minister and the President. This will result in these people immediately making the connexion: ok - I'm going to be re-elected, all I have to do is follow that olpc person geek here in front of us. Brilliant brilliant, all the problems are being solved for me.

The ministers glance at the programme and notice that in 10 minutes, you've organized a press conference for them. How much happier can a minister get: pictures of the minister next to a brilliant person like you and some smiling innocent looking kids holding an olpc ... and this broadcasted all over during this evenings news and in the newspapers tomorrow. But hey, that's how you've been trained by the olpc community. Life is great, stocks of arm producing companies drop.

So we have a calculator to calculate what budget is needed for your olpc xo-xs deployment, but - here's where the happy news ends: we don't have a calculator yet that shows how an olpc xo-xs project pays itself back to the country and in how much time. Enter the pages "Welcome to the pages on Metrics, Feedback and Evaluation".

Things to get done

We need several things to get to this:

1. Stand on the shoulders of giants: groups like the UN / UNESCO / OECD / Ministries of Education themselves have a lot of data on where the educational landscape and level education of their population is at, and compares to other nations. So the data these teams gather, we don't have to gather anymore. But it would be useful to give a quick call to the ministry of education and get in touch with the team that's responsible for collecting that data and have a look at what data and parameters they follow, and what the results are.
2. Looking back over our shoulders: we need more data from the olpc xo-xs deployments.
3. We need a database tool in which we can feed-in all that data
4. The feeding in of the data is going to be easy, cause we're all about connected laptops, so data can be fed-in directly from the terrain every so many months, some data can even stream-in automatically
5. We need to know what questions, parameters, etc. to go after and for.
6. We need an Economic Model and Calculator capable of calculating what are the Additional Tax Revenues from an OLPC XO-XS deployment
7. For those looking for funds or implementing new / future olpc xo-xs deployments, we need a Template Metrics, Feedback and Evaluation-page.
8. Note: now I have to attend my 6 year old. I expect you to have expanded, worked out and done all necessary by the time I get back. As usual I don't give you any resources, people, budget nor links where to go to. And not only me, but the whole entire world, all Ministers of Education, and ICT reporters of the whole world are watching you. --SvenAERTS 13:23, 3 August 2013 (UTC)

Projects that have been evaluated

Click the category "Evaluation". Normally every deployment that has been evaluated, shows up in the category page.