tion. It was assumed that each student might possess
unique learning problems; thus, flexibility was
another watchword.
Rewards were built into the degree of success the
user had on his or her level of difficulty. Short games
or visual displays provided the maximum incentives
for the student's success.
Another item that was given considerable attention
was the security of the program. A difficult situation
could arise if a student tried to break out of the program, thus erasing his poor performance. Limitations and checks on input statements, on error commands, and on masking of keys were used to eliminate many of these areas of concern.
SELECTED READINGS
B. Barnes, "Specific Dyslexia Legislation," J. Learn. Disabil. 60
(Jan 1968).
L. M. Dunn (ed.), Exceptional Children in the Schools: Special
Education in Transition (2nd Ed.), Holt, Rinehart and Wilson,
Inc. (1973).
E. E. Ekwall, Diagnosis and Remediation of the Disabled Reader,
Allyn and Bacon, Inc. (1976).
R. R. Farrald and R. G. Schambert, A Diagnostic and Prescriptive
Technique (2nd Ed.), Adapt Press, Inc. (1973).
N. B. Hannaman, Up Mainstream with a Paddle: A Handbookfor
Mainstreaming Special Students, North East I.S.D. Dept. of Student Resources, San Antonio, Tex., p. 19.
G. Kaluger and C. 1. Kolson, Reading and Learning Disabilities
(2nd Ed.), Charles E. Merrill Publishing Co. (1978).
G. E. Mason and 1. S. Blachard, Computer Applications in
Reading, International Reading Association, Inc. (1979).
D. L. Shepherd, Comprehensive High School Reading Methods
(2nd Ed.), Charles E. Merrill Publishing Co. (1978).
D. Turner, "School Violence," Texas Outlook, 4-7 (Jun 1978).
MOVEMENT
THE EYETRACKER COMMUNICATION SYSTEM
Mark B. Friedman, Gary Kiliany, Mark Dzmura,
and Drew Anderson
Rehabilitation Institute of Pittsburgh
Pittsburgh, Pa.
system consists of three main modules, each of which
is independently useful. The video-based sensor module determines the direction of eye gaze under most
indoor lighting conditions. The dynamic display
module incorporates a hierarchical data base that
allows quick access to a large amount of data. The
accessible data include not only letters and words but
also full phrases and sentences. The output module
allows the data to be output in a sex- and age-identifiable voice or printed on paper.
Introduction
Mark B. Friedman (standing), Mark Dzmura
(leff), and Gary Kiliany.
Our Eye Tracker Communication System allows a
physically handicapped person to use eye movements
to control high-quality speech output efficiently. A
teaching system has been in use for over six months.
We are currently developing a portable, personal
communication system whose main features are ruggedness, convenience, safety, and portability. The
250
The EyeTracker Communication System has been
designed to aid severely physically impaired children.
It is currently in classroom use as a teaching aid. We
are developing it to be used as a portable, personal
communication system.
The system features safety, ruggedness, portability, and convenience. There is no physical contact
with the user. The device is all solid state, powered by
low-voltage direct current, and there are no moving
parts or critical alignments. User control of the
device is rapid and nonfatiguing. Printed and highquality vocal outputs are available. The handicapped
user can use the EyeTracker Communication System
without supervision.
Modular Organization
The EyeTracker system consists of three related
modules, all of which are controlled by the same microprocessor. The sensor module allows the Eyefohns Hopkins APL Technical Digest
Tracker to sense remotely which one of up to nine
display areas a user is looking at. The display module, with an integral, hierarchical data base, facilitates the communication function of the EyeTracker.
Finally, the output module incorporates high-quality
speech generation and, optionally, printed copy.
The remote eye direction sensor module, which
costs less then $1500, uses an infrared video camera
and a digital preprocessor. This economy was
achieved by using a conventional, infrared-sensitive,
commercial surveillance camera. For extra safety and
ruggedness, an industrial solid-state automation
camera can be substituted.
A sharp image of the user's eye is maintained by an
ultrasonic-ranging and automatic-focusing system.
Direction of gaze is computed from the relative positions of a corneal reflection and an image of the
pupil.
The display module currently consists of eight
solid-state alphanumeric display regions arranged on
the perimeter of a 3 x 3 matrix. InitiallY, the display
presented to the user consists of up to eight subject
headings. Gazing at one of these areas results in the
presentation of up to eight subject-related words or
phrases. This process of presenting successively more
specific subject choices continues until the final level
of specification is achieved. In practice, a specific
word, such as a name or place, is inserted into a
general phrase, such as a greeting or request, and the
resulting sentence is sent to the output module.
The output module produces high-quality, concatenated speech, which can be specific to the age and
sex of the user. In order to achieve age- and sex-identifiable speech quality, compressed speech is encoded
digitally and reproduced using a continuously variable-slope delta-modulation technique. If desirable,
the output can be reproduced on the on-board
printer.
Features
There are two main reasons for using eye-movement-based communication aids over more conventionallimb- or body-movement-based devices. First,
two substantial groups of handicapped children
(severely spastic and paralytic) do not have any other
reliable body movements under voluntary control.
Second, even among children with other reliable
body movements, eye movement control is much
more rapid and less fatiguing. This latter factor, ease
and rapidity of encoding a message, often makes the
difference between whether a person will voluntarily
use a communication aid or will use it only when
obliged to, under supervision.
Weare still selecting and orgamzmg the
vocabulary data base while we finish the hardware
support. We anticipate that most of the "small talk"
of daily living will be encodable with two passes
through the hierarchical data base (in two sets of four
glances), because some of the lowest levels of the
data base include phrases (for instance, greetings or
requests) that lack only a word or phrase to make
Volume 3, Number 3, 1982
Hierarchical
data base
memory
Eyeposition
sensor
Microcomputer
High-quality
speech
generator
Multi-unit
symbolic
display
EyeTracker Communication System, high-level conceptual
diagram.
them specific. For instance, after one pass through
the data base (four glances), a phrase such as "please
give me" or "I would like to go to" is stored in a
sentence buffer. Another pass completes the sentence
with a phrase such as "a drink of water" or "the
toilet.' ,
The hierarchical, tree-structured organization of
the speech data base in conjunction with the dynamic
display provides a very efficient means of accessing
the large number of phrases and sentences available.
The philosophy of frame-oriented menu selection
used in the EyeTracker was developed as PROMIS at
the University of Vermont and as ZOG at the Carnegie-Mellon University. Using this system, five eye
movements, taking less than a minute, enable a user
to choose from among over 4000 phrases; adding one
more level to the hierarchy would expand the possible
choice list to over 32,000 phrases. Such a large data
space allows for a high degree of redundancy, which
greatly facilitates inexperienced users' mastery of the
system.
The delta-encoded speech of the output module
approximates the tonal quality of good telephone
voice transmission. We use high-quality compressed
speech components rather than totally synthesized
speech because it is better accepted by handicapped
children. In pilot studies, we found that normal
nursery- and elementary-school-aged children enjoyed making a computer generate its typical monotonous, synthetic speech (which they characterized as
sounding like Darth Vader of Star Wars)_ In contrast, speech-impaired, handicapped children did not
like to use "machine-sounding" speech and much
preferred to use speech communication that was sexand, roughly, age-specific. They wanted their communicator to sound the way they thought they should
sound.
Currently, in a classroom at the Rehabilitation Institute of Pittsburgh (formerly the Pittsburgh Home
for Crippled Children), our device is in almost daily
use aiding the teaching of a severely movement-impaired child. In response to a teacher's or therapist's
question, the child uses her gaze to reply. The child's
possible responses are limited to four words, which
251
are preselected for that trial by the teacher or
therapist from a large, precomposed menu presented
by the computer. The EyeTracker is used both for
vocabulary building (testing word recognition) and
concept testing.
In addition to allowing the child to use gaze for
selection, another virtue of the EyeTracker Communication System in this application is patience.
Whereas a teacher or aide may easily become impatient with the slow decision time of some handicapped children, the EyeTracker will wait indefinitely for the child's response. It will then clearly say the
word, which calls the teacher's attention to that student, thus enabling the teacher to work with several
disabled children simultaneously.
Hardware
This teaching system is based on an Apple II microcomputer donated to the Rehabilitation Institute
by the Alcoa Foundation. To make a practical personal communication aid, we are building our communication systems using another 6502-based microcomputer, the Rockwell AIM-65 single-board computer with built-in printer. Our goal is to have a rugged, portable, battery-powered communication system about the size of a portable black-and-white TV
set. For ruggedness and compactness, we have replaced the Apple floppy disk drives with solid-state
Magnetic Bubble Memory from National Semiconductor (for which we have written our own operating
system). Although bubble memory is expensive now,
we anticipate large price reductions in the next few
years. Ultimately, any field-modifiable, nonvolatile
memory technology will suffice, for instance, battery
backed-up CMOS RAM or EEPROM. Throughout
our design process, we have written PROM firmware
to replace dedicated interface logic.
,
DDD
DBD
DDD
Current EyeTracker
diagram.
252
implementation
hardware
block
" ... we have tried to maintain the
principle that, wherever possible, we
should use the "intelligence" of the personal computer to minimize the physical
and mental effort that handicapped users
must expend to use our communication
aids. "
Software
The software for the AIM-65-based EyeTracker
prototype is stored in PROM and was written entirely
in 6502 assembly language. Much effort was made to
code the system in a modular fashion; consequently,
a software module is associated with each distinct
hardware subsystem. Each module output calls for a
well-defined set of operations on its associated hardware. This policy has helped make the coding of the
abstract, higher level routines as straightforward as if
they were written in a high-level, compiled language.
Further, as good practice dictates, all software at the
user interface level is "friendly;" that is, an inexperienced user can do nothing to destroy data or otherwise compromise the system, and the system attempts
to present reasonable error messages on fnvalid user
entry or failure of a system component.
Conclusions
The hardware and software modularity of the
EyeTracker Communication System is such that each
of the major modules can be incorporated as an independent entity into other aids for the handicapped.
The EyeTracker sensor module could be adapted for
virtually any control function where it is advantageous to use eye-movement control rather than gross
motor movements. It would be practical to implement a physical switch-based communication system
using our dynamic display and hierarchical data
base. This hierarchical data organization allows for
phrase- and sentence-oriented communication without the need for expert knowledge of the system or
reference to external manuals.
We have built a limited-vocabulary EyeTracker
system based on Apple II that has been in daily use
for over six months in a classroom for young physically handicapped children. During this time, we
have been developing hardware and software to extend the teaching system into a less expensive, portable personal communication system. Throughout our
work, we have tried to maintain the principle that,
wherever possible, we should use the "intelligence"
of the personal computer to minimize the physical
and mental effort that handicapped users must expend to use our communication aids. To the extent
that children voluntarily use the EyeTracker Communication System, we will feel that we are successful in our efforts.
Johns Hopkins APL Technical Digest