American Sign Language Vocabulary: Computer Aided
Instruction for Non-signers
Valerie Henderson-Summet, Kimberly Weaver, Tracy L. Westeyn, and Thad Starner
Georgia Institute of Technology, Atlanta, GA
School of Interactive Computing, College of Computing
{valerie, kimberly.weaver}@gatech.edu, {turtle, thad}@cc.gatech.edu
ABSTRACT
In this paper we present the results of a study designed
to evaluate the computer-based methods of learning American Sign Language (ASL). We describe a method including an initial instruction session along with receptive and
generative language tests which were administered after a
week-long retention interval. We show a strong correlations
(ρ=.62, ρ=.57) between the initial session’s instruction and
the receptive and generative levels of vocabulary signing.
Based on the results of our experiment, we establish a baseline for further exploration of ASL vocabulary acquisition
and identify further paths for language based instruction.
Categories and Subject Descriptors
K.3.1 [Computing Milieux]: Computers and Education—
Computer-assisted instruction
General Terms
Languages, Human factors, Experimentation
Keywords
Sign Language, computer assisted language learning
1. INTRODUCTION
Communication between deaf individuals and hearing individuals can be very difficult. For people who are born deaf,
English is often a second language with the first language
being American Sign Language (ASL).
Ninety percent of deaf children are born to hearing parents
who may not know sign language [3]. Often these children’s
only exposure to language is signing at school, and they are
often not immersed in language in the same way as hearing
children of hearing parents (or deaf children of deaf parents)
who are immersed in language from birth. Without this
immersion, deaf children of hearing parents may miss the
critical period for language acquisition. Strong and Prinz [8]
Copyright is held by the author/owner(s).
ASSETS’08 October 13–15, 2008, Halifax, Nova Scotia, Canada.
ACM 978-1-59593-976-0/08/10.
have shown that there is a strong relationship between early
ASL proficiency and later English literacy for deaf children.
In order to address early sign language acquisition, we
must consider the two sides to the problem. One strategy to
combat this problem is to enhance young children’s signing
skills. The other strategy is to increase their immersion in
ASL by teaching their parents ASL. There are many projects
which address children learning ASL (e.g., [5, 4]). Thus, our
work concentrates on the task of teaching parents basic level
ASL vocabulary.
Knowledge of vocabulary is a crucial first step for many
parents who wish to communicate with their deaf children.
While vocabulary knowledge alone does not constitute fluency or command of ASL, Marchman and Bates have shown
that a knowledge of approximately 150 words is enough to
increase the rate at which new words and grammatical skills
are acquired [6]. Further, research has shown that some level
of ASL exposure for deaf children at home is vastly superior
to no language exposure [7]. This means that even “survival
level” vocabulary is a worthwhile endeavor for families with
deaf children.
2.
METHODOLOGY
We have designed an experiment and Flash-based Web interface to evaluate test–and–train methods of ASL instruction. Participants completed two sessions which occurred
approximately a week apart to learn 80 basic ASL vocabulary signs. The words were chosen by an ASL linguist from
the MacArthur-Bates Communicative Development Inventory (MCDI) [2] as words suitable for communication with
a child from infancy.
Participants were asked to provide demographic information including handedness, as the web application changed
the handedness of the signed video accordingly. Subjects
completed 5 trials with mandatory 5 minutes breaks in between each trial. Each trial consisted of 80 videos of ASL vocabulary words presented in random order. After each video,
the participant was presented with multiple choice answers
consisting of the correct answer, 3 incorrect answers, and
“I don’t know”. After participants selected an answer, they
would be told whether their answer was correct or incorrect,
and the correct answer was indicated. Subjects spent an
average of 51.68 minutes (SD=6.2 minutes) on language instruction (not including breaks) ranging from 44.72 minutes
to 64.58 minutes.
Participants were scheduled for a second session approximately a week later. During session 2, participants were
given receptive and generative language tests. During the
receptive test, participants were shown videos of 40 words
and asked to write down the English translations. During
the generative test, the participant attempted to sign 40
English words. The participants had 10 seconds to provide
each word or sign. The words for both the receptive and
generative tests were randomly selected from the original 80
word vocabulary list. Half the participants performed the
generative test first, and the other half the receptive test.
The generative test was analyzed for correctness using a 3
point scale. A score of 1 indicated the subject either made
no attempt at the sign or an incorrect attempt. A score of
2 indicated that some aspects of the sign were correct. A
score of 3 indicated an entirely correct sign comprehensible
to someone familiar or fluent in ASL
3. RESULTS AND DISCUSSION
During the multiple choice sessions, we collected 100 data
points for each of the 80 words (20 participants x 5 trials).
During session 2, we collected 40 generative data points and
40 receptive data points per participant. Because the 80
words were randomly divided between generative and receptive, some words have more receptive data points then
generative, or vice versa.
As a simple measure of mastery of a sign across subjects,
we summed the results of the multiple choice trials T1 –T5 .
For example, the aggregate data points for the correct choice
of the English word “school” (which appeared 5 times per
subject) were T1 =2, T2 =11, T3 =16, T4 =18, and T5 =19.
Thus, school was correctly recognized 66 out of 100 times.
For the generative test during session 2, we averaged the
number of signs which were correct and intelligible (i.e.,
those rated as a 3). on a per word basis. Collectively, our
participants generated 278 words perfectly and recalled 524
words after less then one hour of computer based instruction.
3.1 Adaptive Instruction
Our participants learned certain signs much more effectively then others. Some signs are highly iconic, such as
book which mimics opening a book with two hands while
other signs such as “grandmother,”“grandfather,”“mother,”
and “father” are highly related in handshape, sign-space, and
motion.
Previous work has shown that adaptive presentation in
second language learning can be highly beneficial for the
learner [1]. In this manner, more time is spent on the signs
which are hard for participants to learn, and less time is
spent on the signs which have already been learned. Instances in our data (such as the sign for “banana” which
was recalled correctly 100% of the time and generated correctly 98% of the time) can make a strong case for adaptive
instruction.
In the future, we hope to continue work on the modeling of
ASL instruction. In particular, we hope to improve upon the
baseline established in this paper and develop a more adaptive sequencing of instruction which would generate higher
retention rates.
4. CONCLUSIONS
We have introduced a computer aided training and testing
algorithm plus software to assist adults with the acquisition
of key vocabulary words in American Sign Language. We
selected an 80 word subset from the MCDI that conveyed
distinct and effective content for use during adult-child interactions. Our study consisted of 20 participants that were
trained with five trials of 80 multiple choice selection tasks
during the first session. After a week hiatus, the participants
were asked to perform 40 generative tasks and 40 receptive
tasks using the vocabulary from the previous session. We
found a strong correlation between the number of times a
word was correctly identified in session 1 (the training) and
the number of correct responses to the receptive tasks of session 2. Likewise we found a strong correlation between the
number of correct responses to the generative tasks of session 2. Thus, performance in the training session may lead
to predictions in performance for both receptive and generative tasks given a particular vocabulary set. In addition,
the results suggest that adaptive presentation of vocabulary
during training may further increase optimal learning and
retention of ASL.
5.
ACKNOWLEDGMENTS
This work was supported by Deptarment of Education
Grant #R324A070196. The authors would like to thank
Don Schoner, Dr. Jay Summet, and Dr. Harley Hamilton.
6.
REFERENCES
[1] R. C. Atkinson. Optimizing the learning of a second
language vocabulary. Journal of Experimental
Psychology, 96:124–129, 1972.
[2] L. Fenson, V. A. Marchman, D. J. Thal, P. S. Dale,
J. S. Reznick, and E. Bates. MacArthur-Bates
Communicative Development Inventories. Brookes
Publishing, 2004.
[3] Gallaudet Research Institute. Regions regional and
national summary report of data from the 1999-2000
annual survey of deaf and hard of hearing children and
youth. Technical report, Gallaudet University,
Washington, D. C., January 2001.
[4] V. L. Hanson and C. A. Padden. Handson: A
multi-media program for bilingual language instruction
of deaf children. In Proc. of Computing Applications to
Assist Persons with Disabilities, pages 5–6, 1992.
[5] V. Henderson, S. Lee, H. Brashear, H. Hamilton,
T. Starner, and S. Hamilton. Development of an
American Sign Language game for deaf children. In
Proc. of Interaction Design and Children, pages 70–79,
June 2005.
[6] V. Marchman and E. Bates. Continuity in lexical and
morphological development: A test of the critical mass
hypothesis. Journal of Child Language, 21:339–366,
1994.
[7] J. Singleton and E. Newport. When learners surpass
their models: The acquisition of American Sign
Language from inconsistent input. Cognitive
Psychology, 49:370–407, 2004.
[8] M. Strong and P. Prinz. A study of the relationship
between American Sign Language and English literacy.
Journal of Deaf Studies and Deaf Education,
2(1):37–46, 1997.