Drill and test software

From OLPC
Revision as of 05:49, 12 August 2007 by Ricardo (talk | contribs) (Improved wording of supermemo response.)
Jump to navigation Jump to search

Software to drill and test is often considered anathema under the constructivist paradigm which guides this wiki. It fundamentally encourages a focus on rote knowledge, which constructivists rightly point out is subsidiary to real-world skills and attitudes. However, that does not mean that rote knowledge does not exist, or that it is not sometimes important and even interesting. I expect that most people reading this have at some point in their lives studied for a test using flash cards - do you think that that is fundamentally a waste of time?

Testing software

Moreover, if this laptop will be used in hundreds of thousands of classrooms, many existing teachers WILL try to use it for drills and tests, for good reasons or bad. OLPC and friends, constructivist as we may be, should consider what should best happen in this case.

Possible results:

  • Makeshift testing solutions are difficult for teachers to create and use, and easy for students to cheat on. Teachers either:
  • Use the broken tools, so that students cheat (on the one hand learning dishonesty; on the other, perhaps improving their learning process of the material by engaging with it through cheating, or perhaps just wasting their time cutting-and-pasting)
  • Give up and accept the constructivist doctrine.
  • Or give standard paper-based tests, banning the laptops from the classroom and wasting their time grading.
  • An "official" OLPC testing app exists which allows for restrictions, but also provides infrastructure for "open-book" and/or "group" testing, as well as constructivist counsel on how to keep rote tests to an appropriate minimum. Teachers either:
  • Ignore the constructivist prosyletism and just teach as they would anyway, but at least have some of the grunt-work of grading lifted from their shoulders.
  • Adapt their teaching methods to be more constructivist.

Drill software

A well-designed memorization drill software would be able to create an adaptive, individualized learning schedule; offer support and advice for mnemonic tricks; offer individual or socially-based motivation (eg. quiz show format, etc.) and otherwise improve on old-fasioned flash cards.

What makes this constructivist? If the laptop is always a tool, not a taskmaster. This means that the primary motivation for learning will ALWAYS be from outside this activity - "I want to study X because it will help me do Y, ok computer, help me study." How do you design this outlook into the software? Not sure, here's some ideas: keep the connection to Y salient, even if just with the scores - instead of just "86%" it should respond "86%, you're ready to do intermediate Y-ing" or something; somehow simulate a real-world Y-ing task; ask "how did you do the last time you tried to Y" and track that along with in-program scores... ???

Supermemo

Here is the theory behind it.

Supermemo is basically a drill-and-test program, using question-answer pairs for learning foreign-language vocabulary (such as English/French – Q: House/A: Maison), grammar points or any other body of knowledge, such as historical facts, mathematical terminology, whole topic-paragraphs, key-facts, etc. The material can include text, hyperlinks, multi-media diagrams/photos/videos/etc.

Its main feature is that it uses feedback from the learner to schedule the learning. Users are shown a question, think what the answer is, reveal the answer, then mark themselves on how well they knew the answer (from 1 = I knew it without hesitation, 2 = I new it with some hesitation, down to 5 = I’ve never seen it before). It keeps your level of information-retention in a feedback-loop to optimize the re-test interval (1 day/week/month/3 months/etc), achieve the maximum amount of learning/revision in the minimum time and keep your knowledge fresh. Children almost always succeed in learning the material.

The optimum use of time is when people are only re-tested on a question-answer pair just before it was about to fade from their memory. Too-frequent revision wastes time and waiting too long means forgetting an item completely and learning it again as if they had never seen it before.

Children can use off-the-shelf sets of questions and answers, sets produced by the teacher, enter their own questions one at a time, import/export a set to/from a text-file and share sets via the internet . Children start with a small set of questions. As facts become well-learned and scheduled less often, more questions can be added to the set to keep the learning/practice time at 20 minutes per day or whatever. The system avoids last-minute test-revision. Teachers and children can use reports to monitor progress.

Drill-and-test may sound like the exact opposite of constructionism, but the same mechanism could be used to schedule repetitions of each child’s constructionist-activities until they are well-understood and then keep them fresh in their minds.

There are MS Windows, Palm Pilot and operating-system-independent Online versions of Supermemo.

It isn’t available for Linux at present, due to some porting problems. This is the status of previous attempts to produce a Linux version, and a list of similar programs.

If anyone can solve the problems of porting it to Linux/Sugar or find/write a program with a similar feedback technique, it could be quite useful, for both scheduling activities and learning vocabulary or key-facts.

If software isn’t available, the method could be used with pencil-and-paper/text-file records of activities.

If you want to try it, some older versions of Supermemo are free, such as the MS Windows 98 version.

--Ricardo 04:08, 9 August 2007 (EDT)

Note that this is non-open-source software, and as such is not likely to ever make it into an OLPC distribution. Three things could happen:

  • The author could license it using an open source license, and ask for voluntary donations to replace the income it currently brings them
  • The author could create a version for OLPC *without* an open source license. This would make it usable on the OLPC, but it would still never be distributed with the OLPC.
  • Somebody could make (or find and adapt?) a similar piece of open-source software for OLPC. It would be useful to be able to read the SuperMemo format (undocumented?) and to use the algorithm which obviously has years of work and study behind it.

I think the first two are unlikely, and that the latter is worthwhile.

Disclaimer: I myself am the author of an unmaintained closed-source shareware of a similar nature for PalmOS. I used an algorithm I invented from first principles and did not study results. Thus I have both great respect for the sophistication of the SuperMemo algorithm, and an understanding that, however refined, it (and any competing algorithm) is biased by the mathematical models it uses.

Homunq 05:41, 11 August 2007 (EDT)

Since it's a commercial program, I take your point that a similar open-source program would be required.

The file formats for importing/exporting questions are covered on the Supermemo File page. In the text format, please note that a space is required between Q: or A: and the question/answer.

The questions and answers don't have to be like plain-text flash-cards...

Q: House

A: Maison

They can include HTML web-pages, diagrams, photos, videos, mind-maps, mnemonics to assist learning, hypertext links to further explanation/dictionaries/encyclopedias, etc. Children can start with a set of questions from a Teacher or other educator and edit the answers with personal notes, pictures, etc.

I've used it sucessfully to get up to speed in a new language in 3 months, half an hour a day, so a similar open-source program could be quite effective. The scheduling algorithm to keep learned material fresh in the user's mind doesn't have to be identical.

It isn't itself constructionist, but it could have a place in supporting constructionist language-learning activities. It could be used for vocabulary-learning or to keep fresh a lot of vocabulary learnt via constructionist activities.

In language-learning games, a supermemo-like formula and feedback score 1-5 could be used to determine the interval before each word of vocabulary re-appears.

Also, the teacher or child could use the method to schedule constructionist activities on any subject. They would use the child's rating of an activity (1 = I found it easy, down to 5 = I need lots more practice) plus the scheduling formula/table to decide when to perform the same/similar activity again (after 1 day, 2 days, 1 week, 1 month, 3 months, etc).

I see the straightforward drill-and-test way of using it as something each child can use if they want. I imagine it will suit some children's learning-style and not others. I found it very effective.

--Ricardo 01:49, 12 August 2007 (EDT)

See also

Educational toolkit

The ready-to-test idea