Drill and test software

Software to drill and test is often considered anathema under the constructivist paradigm which guides this wiki. It fundamentally encourages a focus on rote knowledge, which constructivists rightly point out is subsidiary to real-world skills and attitudes. However, that does not mean that rote knowledge does not exist, or that it is not sometimes important and even interesting. I expect that most people reading this have at some point in their lives studied for a test using flash cards - do you think that that is fundamentally a waste of time?

The 'Drill' in 'Drill-and-Test' is another name for Spaced Repetition. The 'Wikipedia: Spaced Repetition' article says...

"Spaced repetition is a learning technique in which increasing intervals of time are used between subsequent reviews. Alternative names include expanding rehearsal, graduated intervals, repetition spacing, repetition scheduling, spaced retrieval and expanded retrieval. Spaced repetition is not simple rote repetition of phrases. It emphasizes active recall rather than recognition, and uses recall methods of memory such as questioning, free recall, free association, intellective reflection, or prompting using other mnemonic triggers or cues.

Humans differ little in the way their memory works. People with large differences in IQ will often exhibit similar spaced repetition patterns. For example, when we learn a foreign word, most people require a repetition within 1-10 days to ensure 95% retention. The second repetition may already come later. For example, between 2-25 days. After a few repetitions, well structured and meaningful learning material can be repeated with intervals lasting many years."

Testing software for classroom evaluation (traditional exams)

If this laptop will be used in hundreds of thousands of classrooms, many existing teachers WILL try to use it for drills and tests, for good reasons or bad. OLPC and friends, constructivist as we may be, should consider what should best happen in this case.

Possible results:

• Makeshift testing solutions are difficult for teachers to create and use, and easy for students to cheat on. Teachers either:

• Use the broken tools, so that students cheat (on the one hand learning dishonesty; on the other, perhaps improving their learning process of the material by engaging with it through cheating, or perhaps just wasting their time cutting-and-pasting)
• Give up and accept the constructivist doctrine.
• Or give standard paper-based tests, banning the laptops from the classroom and wasting their time grading.

• An "official" OLPC testing app exists which allows for restrictions, but also provides infrastructure for "open-book" and/or "group" testing, as well as constructivist counsel on how to keep rote tests to an appropriate minimum. Teachers either:

• Ignore the constructivist prosyletism and just teach as they would anyway, but at least have some of the grunt-work of grading lifted from their shoulders.
• Adapt their teaching methods to be more constructivist.

Drill software

A well-designed memorization drill software would be able to create an adaptive, individualized learning schedule; offer support and advice for mnemonic tricks; offer individual or socially-based motivation (eg. quiz show format, etc.) and otherwise improve on old-fasioned flash cards.

What makes this constructivist? If the laptop is always a tool, not a taskmaster. This means that the primary motivation for learning will ALWAYS be from outside this activity - "I want to study X because it will help me do Y, ok computer, help me study." How do you design this outlook into the software? Not sure, here's some ideas: keep the connection to Y salient, even if just with the scores - instead of just "86%" it should respond "86%, you're ready to do intermediate Y-ing" or something; somehow simulate a real-world Y-ing task; ask "how did you do the last time you tried to Y" and track that along with in-program scores... ???

One method might be to do a short test to found out a child's strong and weak areas of topic Y, then for the weaker sub-topics of Y (say Y.1, Y.4, Y.7, etc), suggest what activities a child might like to do next. For example, "Very good. How about trying <name of Y.4> next?". For variety, Y.4, etc. is picked at random from the weak sub-topics list. This mechanism could be included in other activity-programs as a "What shall I do next?" button, which children are free to use if they want, perhaps when the teacher is busy. Over time, they would get stronger in all aspects of topic Y. --Ricardo 04:11, 21 August 2007 (EDT)

Software for scheduling constructionist activities or teaching vocabulary, key-facts, etc.

Teachers sometimes find it hard to supervise mixed-ability groups, since each child learns at a different rate. There is a commercial program called Supermemo (an abbreviation of Super-Memorization), which demonstrates a Spaced repetition technique that might be used to schedule the learning-activities for each child or teach vocabulary, key-facts, etc.

Although it's commercial, open-source alternatives exist: Anki and Mnemosyne. These could be used, for example, to remind children to review and update mind-maps to stop knowledge fading from memory, where the knowledge was acquired originally through constructionist activities. They are also good for teaching (and keeping fresh) foreign language vocabulary.

These programs are basically drill-and-test, using question-answer pairs for learning foreign-language vocabulary (such as English/French – Q: House/A: Maison), grammar points or any other body of knowledge, such as historical facts, mathematical terminology, whole topic-paragraphs, key-facts, etc. The material can include text, hyperlinks, multi-media diagrams/photos/videos/etc.

However, they each incorporate algorithms for spaced-repetition, taking feedback from the learner in order to schedule the review of individual flashcards. Users are shown a question, think what the answer is, reveal the answer, then mark themselves on how well they knew the answer (eg in SupermemoÑ from 1 = I knew it without hesitation, 2 = I new it with some hesitation, down to 5 = I’ve never seen it before). It keeps your level of information-retention in a feedback-loop to optimize the re-test interval (1 day/week/month/3 months/etc), achieve the maximum amount of learning/revision in the minimum time and keep your knowledge fresh. Children almost always succeed in learning the material.

The optimum use of time is when people are only re-tested on a question-answer pair just before it was about to fade from their memory. Too-frequent revision wastes time and waiting too long means forgetting an item completely and learning it again as if they had never seen it before.

Children can use off-the-shelf sets of questions and answers, sets produced by the teacher, enter their own questions one at a time, import/export a set to/from a text-file and share sets via the internet . Children start with a small set of questions. As facts become well-learned and scheduled less often, more questions can be added to the set to keep the learning/practice time at 20 minutes per day or whatever. The system avoids last-minute test-revision. Teachers and children can use reports to monitor progress.

Drill-and-test may sound like the exact opposite of constructionism, but the same mechanism could be used to schedule repetitions of each child’s constructionist-activities or reviews of mind-maps/notes until they are well-understood and then keep them fresh in their minds.

There are MS Windows, Palm Pilot and operating-system-independent Online versions of Supermemo.

It isn’t available for Linux at present, due to some porting problems. This is the status of previous attempts to produce a Linux version, and a list of similar programs.

If anyone can find/write a program with a similar spaced-repetition feedback technique, it could be quite useful, for both scheduling activities and learning vocabulary or key-facts.

If software isn’t available, the method could be used with pencil-and-paper/text-file records of activities.

If you want to try it, some older versions of Supermemo are free, such as the MS Windows 98 version.

--Ricardo 03:31, 13 August 2007 (EDT)

Note that this is non-open-source software, and as such is not likely to ever make it into an OLPC distribution. Three things could happen:

• The author could license it using an open source license, and ask for voluntary donations to replace the income it currently brings them
• The author could create a version for OLPC *without* an open source license. This would make it usable on the OLPC, but it would still never be distributed with the OLPC.
• Somebody could make (or find and adapt?) a similar piece of open-source software for OLPC. It would be useful to be able to read the SuperMemo format (undocumented?) and to use the algorithm which obviously has years of work and study behind it.

I think the first two are unlikely, and that the latter is worthwhile.

Disclaimer: I myself am the author of an unmaintained closed-source shareware of a similar nature for PalmOS. I used an algorithm I invented from first principles and did not study results. Thus I have both great respect for the sophistication of the SuperMemo algorithm, and an understanding that, however refined, it (and any competing algorithm) is biased by the mathematical models it uses.

Homunq 05:41, 11 August 2007 (EDT)

Since it's a commercial program, I take your point that a similar open-source program would be required.

The formula for re-scheduling items doesn't have to be identical to Supermemo. The formula also doesn't need to be perfect since the revision-interval for each item is made longer or shorter in a feedback-loop, according to the self-assessment score 1-5, keeping information-retention from falling below a certain figure, such as a score of 2 ("I knew it with some hesitation").

The file formats for importing/exporting questions are covered on the Supermemo File page. In the text format, please note that a space is required between Q: or A: and the question/answer.

The questions and answers don't have to be like plain-text flash-cards...

Q: House

A: Maison

They can include HTML web-pages, diagrams, photos, videos, mind-maps, mnemonics to assist learning, hypertext links to further explanation/dictionaries/encyclopedias, etc. Children can start with a set of questions from a Teacher or other educator and edit the answers with personal notes, pictures, etc.

I've used it sucessfully to get up to speed in a new language in 3 months, half an hour a day, so a similar open-source program could be quite effective. The scheduling algorithm to keep learned material fresh in the user's mind doesn't have to be identical.

It isn't itself constructionist, but it could have a place in supporting constructionist language-learning activities. It could be used for vocabulary-learning or to keep fresh a lot of vocabulary learnt via constructionist activities.

In language-learning game software for the OLPC, a supermemo-like formula and feedback score 1-5 could be used to determine the interval before each word of vocabulary re-appears.

I see the straightforward drill-and-test way of using it as something each child can use if they want. I imagine it will suit some children's learning-style and not others. I found it very effective.

The teacher or child could also use the method (pencil-and-paper or software) to schedule constructionist activities on any subject. They would use the child's rating of an activity (1 = I found it easy, down to 5 = I need lots more practice) plus the scheduling formula/table to decide when to perform the same/similar activity again (after 1 day, 2 days, 1 week, 1 month, 3 months, etc).

--Ricardo 03:31, 13 August 2007 (EDT)

I have a GPL/Linux program called Gradint http://people.pwf.cam.ac.uk/ssb22/gradint/ I don't know if that's useful as a starting point. Gradint itself is audio only (it's meant for learning foreign languages) and has no real-time feedback (it pre-calculates the most likely spaced-interval pattern, and it usually works although if you get stuck you can do things like edit the progress file in a text editor). Its associated program Charlearn (which works in HTML) has feedback but it's not real-time and doesn't seem to work so well. Of course this stuff is nowhere near XO-ready, but welcome to read the source and see what you think (I personally have learned 5,000+ words of Chinese with Gradint, and I know people who've learned several hundred words of other languages, so I know it can be effective; however I also know other people who didn't get on at all well with it, so it's not a universal panacea).

--ssb22 9 September 2007

What you search for might be this wonderful piece of Software: Mnemosyne. It meets your License issues (GPL), is written in Python, but uses QT. Not sure if this is the right GUI toolkit. Handling is very easy, questions/answers could be text, images or soundfiles. Also LaTex could be integrated. Software is ready for import of SuperMemo format as well.

Mnemosyne - it is available for Linux, Windows and Mac. Definitely a good application to add to the OLPC. --DouglasReay 21:15, 15 May 2008 (BST)

Web-page implementation of Drill and Test activities

Drill and Test activities would be quite easy to implement as a small local mini-website, uing HTML pages with forms and Javascript. Set of questions can be read from comma-delimited .csv text files (produced easily using a spreadsheet program) or some other source. This way of implementing it makes it accessible to the large pool of web-developers without much experience of other programming languages. An example website would be very easy to share and adapt.

--Ricardo 04:21, 21 August 2007 (EDT)

What about Moodle? ffm 19:58, 3 December 2007 (EST)

System for children to create their own quizzes

Drill and Test is really just a specialized quiz for a particular purpose. To make things a little more contructionist and creative, software or a web-based system should be found for children to create their own quizzes on any subject, either to use themselves for practice or for other children to use. The same tools could then be used by educators to set up quizzes/drill-and-test question-sets for children. There are many open-source quiz programs.

--Ricardo 04:31, 21 August 2007 (EDT)

A modest example of drill and Test

See ConstellationsFlashCards. This program will consider how often the user gets a particular constellation's name correct and will present that constellation less frequently the better the user has demonstrated his ability to recognize it. I think that feature is important: why drill repeatedly something the student has demonstrated he has successfully learned? In the program, the knowledge of the user's successes is "aged" so that over time even the familiar constellation would occasionally appear. This may be something like an implementation of the Leitner system, but with more boxes. -- Davewa 00:59, 1 October 2010 (UTC)

See also

Educational toolkit

The ready-to-test idea

Educational Content - language learning

Application Program: Test of ability with arithmetic

Wikipedia: Supermemo

Wikipedia: Flashcards

Sourceforge: Granule

Wikipedia: Spaced Repetition (including Software for Spaced Repetition - there are 9 OSS programs in this domain. However, none are in Python, OLPC's preferred language.)

valodas - free cross-platform language learning software