Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
Text Input to Handheld
Devices for People with
Physical Disabilities
Brad A. Myers and Jacob O. Wobbrock
Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213-3891
bam@cs.cmu.edu
http://www.pebbles.hcii.cmu.edu/assistive/
Motivation


Provide text entry for people with physical
disabilities
For handheld devices like Palm Pilots


Also called Personal Digital Assistants (PDAs)
Also use Palm Pilots as interface to desktop
computers
Brad Myers
Carnegie Mellon
2
Part of the Pebbles Project

Investigate use of handheld devices like
Palm Pilots and PocketPCs, and
Smartphones at the same time as PCs
and other devices
PDAs for
Entry of
Both
Bytes and
L ocations from
External
Sources.
http://www.pebbles.hcii.cmu.edu/
Brad Myers
Carnegie Mellon
3
First Steps



Support people with Muscular Dystrophy
Low strength, limited motion, but high
accuracy
Tiny keyboard on handheld screens are good
Brad Myers
Carnegie Mellon
4
Use Our Remote Commander


Palm Pilot application to control PC using the Palm
Extend tiny keyboard with other keys




Word completion
Other adaptations: no auto-repeat, no auto-off
Mouse control on same screen
Wireless or wired
Brad Myers
Carnegie Mellon
5
But what about other disabilities?




Cerebral Palsy, Parkinson’s, Spinal Injury,
and many others
Loose accuracy as well as strength
Tiny buttons are exactly wrong
Still can’t do Graffiti, etc.
Brad Myers
Carnegie Mellon
6
New Design: EdgeWrite™




Added custom template with a square hole to
input area
Gain stability from edges
Travel along edges and into corners
Designed alphabet using
gestures inside of hole,
along edges and into
corners
Brad Myers
Carnegie Mellon
7
Alphabet Properties

Recognizes based on sequence of corners



Users can wiggle during diagonals
Not speed dependent
Easy to customize and adapt letter forms


Letters “feel” normal



Use multiple strokes along edge
Presentation (only) uses arcs
No capitalization mode


Even user-driven
Capitals end in upper left
No special mode for numbers

I on left, 1 on right
Brad Myers
Carnegie Mellon
8
Actual Alphabet from User Studies

Wanted high guessability






Correlates with initial usability
Designed a technique to let users guess the
gestures
Told users the rules
Then resolve conflicts if same shape used twice
Left multiple forms for each letter
Result: guessability improved from 51% to 80%
Brad Myers
Carnegie Mellon
9
Current Alphabet
Brad Myers
Carnegie Mellon
10
EdgeWrite on Palm

Novice able-bodied users



Same speed with Graffiti and EdgeWrite
But EdgeWrite 18% more accurate
Handicapped users



Too few for statistical significance, but dramatic
E.g., accuracy: 31% with Graffiti vs. 94% EdgeWrite
For example:
 “The dog is going fast” vs:
“The g i gbsiangu% fast”
Brad Myers
Carnegie Mellon
11
EdgeWrite on Game Controllers


Some joysticks use a square hole
Current techniques are very
awkward:


On-screen keyboard
“Date-stamp” cycling letters at each spot
7
6
5
EdgeWrite
4
Date Stamp
3
Selection
Keyboard
2
1
0
Brad Myers
WPM
Carnegie Mellon
12
EdgeWrite on Wheelchair



Look at people who use power
wheelchairs
Custom hardware and software
to interface to commercial joystick
Informally evaluated with 7 disabled powerwheelchair users

6 with Cerebral Palsy, 1 with Multiple Sclerosis
Brad Myers
Carnegie Mellon
13
Touchpad EdgeWrite

Adaptation that uses elevated edges around
a Synaptics touchpad

May be easier for people with motor impairments
Brad Myers
Carnegie Mellon
14
Results


Touchpad worked better than wheelchair
joystick
Still pretty slow
Brad Myers
Carnegie Mellon
15
EdgeWrite on 4 Buttons



Only need four corners
Implementation for any 4 keys
Four keys on the Logitech steering wheel


Not necessarily for text entry while driving
Text entry especially for navigation system
Brad Myers
Carnegie Mellon
16
EdgeWrite with Trackball & Pointing Stick



Newest version
Many disabled users prefer trackball
for desktop pointing
EdgeWrite for desktop

Optimized for trackball and pointing stick





Also works with regular mouse
Highly parameterized
Input window can be transparent when in use
Has help, etc.
Up to 20 wpm
Brad Myers
Carnegie Mellon
17
Initial Reaction to Trackball Version

One user in long-term trial


Spinal injury, quadriplegic, with some mobility of one arm
Given up on-screen keyboard in favor of EdgeWrite


Still uses speech entry when possible
With practice, EdgeWrite is faster
“With an on-screen keyboard, there is just too much visual scanning
and concentration. In EdgeWrite, if you know the letter, you just
bang it out by feel, and you can keep your eyes on your document.”
Brad Myers
Carnegie Mellon
18
Summary and Future Work


EdgeWrite works across a wide range
of devices
With the same alphabet



Learn once
Use most comfortable device
Future work




Continue to refine
Add word completion
Further tests and deployment
New devices (phone?)
Brad Myers
Carnegie Mellon
19
Acknowledgements


Funded by grants from NSF, General Motors
and The NEC Foundation of America
Equipment grants from:



Synaptics, Inc.
A.T. Sciences
Microsoft
http://www.edgewrite.com/
http://www.pebbles.hcii.cmu.edu/
Brad Myers
Carnegie Mellon
20