Nell/Handwriting

From OLPC
< Nell
Revision as of 01:30, 7 March 2012 by CScott (talk | contribs) (Basic handwriting recognition info.)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

One interesting interface to Nell is the "Tom Riddle's Diary" UI -- where the child handwrites sentences on the page, which are recognized and treated as inputs to a conversation with Nell. Nell responds by writing on the page herself.

This reinforces the literacy aspect of the device: learning to read and write is fundamental to communication with Nell. On the other hand, it means that young children may initially not be able to interact in deep ways with Nell until they master basic literacy.

Links related to handwriting interfaces:

  • On-Line Handwriting Recognition Using Hidden Markov Models: a decent thesis from 1997 describing a "baseline" handwriting recognition system, with lots of details of choices made for features, etc.
  • Newton handwriting recognition: the Newton (and "Inkwell" in OS X 10.2) used a neural network for recognition, rather than a Hidden Markov Model. This 1998 paper describes the architecture in some detail; Chapter 6 in "Pattern Classification and Scene Analysis" gives useful background for Neural Nets.
    • Hidden Markov Models are preferred today because they offer a uniform composable model which can integrate the segment identification and word- and sentence-letter bigram/trigram probabilities into the recognizer. The Newton's architecture used the neural net only for individual character recognition, and then had ad-hoc search routines running on top of that to try out segment boundaries and guess words.
    • But on the other hand, copying the Newton approach is likely to yield a very performant implementation on modern hardware.