Accessibility Line Based Interface

Note: if you are interested in this, I am at tim<dot>thelion<at>gmail.com

This is my proposal for how to make the Sugar interface accessible to users with motor, visual, and audio impairments. We make the interface line based like in emacs. Here's how EMACS does things and how I plan to accomplish the same thing but more easily, more stably, in sugar.

EMACS

• Hooks to send the current lines text to a screen reader, it also has faces which the screen reader can read with differing voices if it's an auditory reader.
• Commands for going to the next anchor, this allows for very easy, quick, keyboard navigation.
• The ability to change the size/polarity/colorization of the current line. This is much better than normal magnification.
• Easy hooks for cursor movement and keypresses so auditory notifications while typing of what keys have been pressed are possible.

Sugar Line Based system.

• SA: Abbreviation for Sugar Activity.
• line: An arbitrary rectangle of the screen which contains a list of anchors, text areas, and labels. A line for instance is the bar at the top of this web browser, it is a rectangle on the screen, and in this rectangle are 4 buttons and one text area; <button "Back"/><button "Forward"/><button "Refresh"><textarea "Editing Accessibility Line Based Interface - OLPC"/><button "Bookmark"/>.
  • Scrolling v Navigating: SAs must have a set of hot keys for navigating between lines distinct those used for scrolling a pages content or moving the cursor. The reason for this is simple, in the Browser SA I need to navigate upwards 10 visual lines on the screen to get to the address bar, but I'd need to scroll at least 100! Also, navigating right one anchor is much different from scrolling right one charicter, say I'm editing the address bar, scrolling right moves me one char to the right, navigating right moves me to the "Bookmark" button.

For now, I will suggest that the navigation keys be the arrow keys prefixed with control, would we want a control lock to help with navigation?

• LC: For Line Control a program which sits in an override redirect(floating above of it's own accord outside of the control of the window manager) window above the current line of the current SA but not grabbing focus from the SA.

LC will provide screen reader, magnification, and navigational support to the end user.

• • Configuration: More about configuration at bottom of document.
    • Magnification: LC will float above the SA and potentially magnify one line. If it is acting as a magnifier, it will need to tell the SA to set it's width a certain ratio smaller than the screen. LC needs a magnification ratio that can be set by the user and will be read by the SA before startup. For successful magnification of an entire line by x2, the SA will have to start up with a width of 1/2 it's normal size so that the LC will fit on the screen after having magnified the line.
    • Modes: LC will present the line to the end user either though TTS, a brail screen reader, or magnification, these modes must be set by the end user. NOTE: it is possible LC will have two maginfication modes, 'live image' and 'text'. 'live image' would be like a normal magnifying glass. 'text' would display text in a large font size or with opposite polarity over the SA's display which would be identical to the text spoken by LC's TTS or the text sent by LC to the screen reader.
  • SA-LC Communication: LC will be in what I will refer to as an IM conversation with that current program.
    • Example conversation between Write Activity and LC(statements follow ":", actions follow "!").

writeSA: <line x=300 y=200 w=0 h=0><label "Starting Write..."/></line>

LC! In TTS mode LC would say "Starting Write..."

LC! In screen reader mode LC(LC-SR) would print "Starting Write..."

LC! In Magnification text mode LC(LC-MT) would display it's window with the text at the users desired font size, color, and polarity in the center of the screen.

LC! In magnification 'live image' mode LC(LC-MLI) would do nothing as the rectangle which is the current line is of size 0.

writeSA: <line x=0 y=30 w=640 h=12><textarea ""/></line>

LC-TTS! Say "Blank text area."

LC-SR! Print "Blank text area"

LC-MT! Do nothing.

LC-MLI! Magnify that rectangle by specified ratio.

User! presses 'h'

writeSA: <action><keypress 'h'/></action>

LC-TTS! Say 'achee'

LC-SR! Change text to "h"

LC-MT! Open window with text "h" of the desired size ect in it.

LC-MLI! Do nothing, updating should be live and the 'h' should have already appeared.

User! continues to type out 'hello'

User! presses Control-<up arrow>(as you remember, this navigates up by one line)

WriteSA: <line x=0 y=30 w=640 h=20><button "Activity"/> <button "Edit"/> <button "Text"/> <button "Image"/> <button "Table"/> <button "Format/> <button "View"/></line>

LC-TTS! Say in voice-focused-anchor "Activity" then list the other tabs in voice-unfocused-anchor.

LC-SR! I'm not sure what the screen reader should do... How does the screen reader inform the user that the text has changed? I've never used one and am unsure of how interactivity should work on one.

LC-MT! Show the LC window with those anchors displayed in the large font size.

LC-MLI! Move the LC window to hover over those tabs and magnify them.

I think the intelligent reader should understand what I'm getting at by now.

SA-LC communication language standard

xml based, here are the proposed element types

• line
  • options: x y w h
  • contents:

The contents of line should be of class object which has the option focused which defaults to false.

• • • button
    • label
    • textarea
      • options: cursor-position
• action
  • contents:
    • keypress
    • cursor-move
    • cursor-move-select
    • anchor-change

Options in LC-TTS

• word-or-beep: Do we beep with different tones to denote anchors, text areas and the like, or do we use words?
• voice-focused-anchor
• voice-unfocused-anchor
• voice-text-area
• voice-selected-text
• voice-label voice for normal text
• beep-*

Options in LC-SR

Don't know...

Options in LC-MT

• font size, font, font style...
• polarity
• Colors for different elements as listed previously for LC-TTS

Optoins in LC-MLI

• magnification ratio
• polarity
• slice or squish: do we make a warped or squished edge at the top and bottom of the bar? Or do we cut of the lines above an bellow?