Captions and Subtitles for Second Language Acquisition.
Introduction
Second language videos including second language movies, TV programs and other forms of
animated video clips, have long been used for learning second language because they deliver
authentic target language in culturally-rich and enjoyable materials. Watching second language
videos allows language learners to passively immerse in a virtual target language environment.
However, listening to the flow of authentic foreign utterances in videos may not be an effective
way to improve the viewers’ language skill especially if the discourse and dialogues in the video
are far above the learners’ comprehension level. A central concept in cognitive approaches to
SLA is that learners have the opportunity to acquire features of the second language when the
input is comprehensible (Krashen, 1985). Captions and subtitles are forms of input modification
that serves as input enhancement to benefit second language acquisition (Chapelle, 2003). Using
captions and subtitles for SLA is the focus of this study.
Captions, subtitles, and SLA
Captions are commonly referred to as the text version of the spoken part of a video. They are in
the same language as the medium. The main purpose of captioning is to provide hearingimpaired people access to the spoken text in the video. For second language learners, captions
are in their L2; Subtitles, on the other hand, are translations of the audio into another language.
The main purpose of subtitling is to translate spoken audio into the viewer’s language to help
them understand the spoken part of the audio. For second language learners, subtitles are in
language learners’ L1. Sometimes the term caption and subtitle are used interchangeably even
among educators and language researchers. In many European countries, both captions and
subtitles are called subtitles. In the US, many people refer to subtitles as foreign language
captions. Some researchers use L2 caption or L2 subtitle to refer to subtitles, and some use L1
subtitle and L1 caption to refer to captions. Since caption and subtitle are in different languages,
serve different purposes in language acquisition and function differently in the learning process,
it is important to make the distinction between them. In this paper, we use the caption to refer to
text that is in the same language as the soundtrack in the video, i.e., in language learners’ L2. We
define subtitle as the translation of the spoken audio into the viewers’ native language. i.e., in
language learners’ L1.
The potential for using captions and subtitles to help second language learners to increase their
second language competence been documented in previous multimedia language teaching
research and the more recent CALL research. The affordance of computers and advancement in
computer-assisted language learning technology made captions and subtitles more accessible to
language learners and enabled different ways to display various formats of captions and subtitles
for language learners. How exactly can captions and subtitles help second language learning in
different languages or under different text display settings and how does learner differences
influenced the effectiveness of captions and subtitles for SLA are still active research areas in the
past decade and calls for systematic investigation and relevant data collection especially in the
CALL context. Researchers also hope that more in-depth pedagogical research on the effective
use of captions and subtitles will help the development of language curricula and self-learning
programs (Danan, 2004). The purpose of this paper is to provide a synthesis of research on using
caption and subtitle as a major CALL tool in SLA in the past 12 years.
Method
Collection of relevant studies
The collection of studies in this research synthesis was conducted in two stages. The first stage
began with an electronic search using the EBSCOHost online research database and focused on
top CALL journals. The five publications initially targeted in this synthesis are CALICO Journal,
ReCall, Computer Assisted Language Learning, Language Learning and Technology and System
that published between 2007 and 2019. The searching keywords used in stage one are caption,
captions, captioning, subtitle, subtitles, subtitling. The second search stage continued with the
same publication period but covered more journals in second language acquisition, teaching and
education, and psychology fields to expand the pool of studies that were most relevant to the
research topic. The focus of in the second stage was captions and subtitles used in second
language acquisition, and the search was conducted with Google Scholar search engine with the
following additional keywords: second language learning and second language acquisition.
Inclusion and Exclusion Criteria
Through the initial electronic search, a total of 71 studies were identified to fit our search scope.
To qualify for analysis, each of the studies was reviewed according to the following inclusion
and exclusion criteria:
Inclusion Criteria
1, The study was focused on the use of captions or subtitles for second language
acquisition.
2, Captions or subtitles were used as the major language learning tool in the study.
Exclusion Criteria
1, Captions and subtitles are not directly used for language acquisition.
2, Captions and subtitles are used for language acquisition but only serve in a minor
function in a CALL tool or as one feature of a large learning tool portfolio.
4, Research is not about enhancing language skill but on other non-linguistic aspects of
language learning (e.g., social-cultural issues, learner perception or attitude, language
policy etc.).
5. Literature or book reviews, journal editorials, conference speeches, and courseware or
product reviews.
6. Used with static pictures, song lyrics or gaming settings.
7. Reverse subtitles. i.e., the video soundtrack is in viewers’ L1 but subtitles are in
viewers L2.
Table 1.
Included and Excluded Articles from Target Publications
Journal
CALICO Journal
Language Learning & Technology
ReCall
System
Computer-Assisted Language Learning
Computer and Education
Linguists and Education
British Journal of Educational Technology
Educational Technology and Society
Teaching English with Technology
Journal of Language Teaching and Learning
Included
3
4
5
3
3
2
1
1
1
1
1
Excluded Total
9
3
3
4
10
0
0
0
0
0
0
12
7
8
7
13
2
1
1
1
1
1
jalt CALL Journal
International Journal of Psychology
Plone
International Journal of Call and Teaching
The Modern Language Journal
Language Teaching
Studies in Second Language Acquisition
Social and Behavioral Science
Total
1
1
1
1
1
0
1
4
35
0
0
0
0
3
1
0
3
36
1
1
1
1
3
1
1
7
71
As Table 1 shows, most captions and subtitles studies appear in CALL journals, followed by
psychology/behavior, and teaching/education journals. Only one relevant study was found in
SLA journals (Studies in Second Language Acquisition) that fit our inclusion criteria. In total, I
found 35 studies that fit the research criteria of this study. Table 2 shows the studies of captions
and subtitles for SLA per year in this synthesis.
Table 2
Captions and subtitles for SLA Studies Per Year in this Synthesis
Year
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
Number of
Studies
1
2
1
2
3
2
2
8
6
2
3
4
An overview of the research participants in these studies revealed that most studies regarded
second language listening skill and vocabulary learning as the two major second language
competence that captions and subtitles to be most effective. Most studies dealt with adult
language learners. Only three studies used children as target learner groups in the research. The
lack of children participants in the studies may due to children’ low L1 and L2 reading skill.
Most studies used English as the target language, only three studies used French as target
languages, and no other languages were found. This might occur due to the affordances of
abundant English and French captions that were easily accessible for research.
Results
Listening competence
Subtitles had been used in movies for many decades and they were very popular in Europe.
Caption became available on TV in the 1980s in the US. Ever since Price’s (1983) pioneering
study, captioning has been studied extensively as a tool to improve L2 listeners’ comprehension
of audiovisual input (e.g., Baltova, 1999.)
The bulk research studies on listening comprehension in the caption or subtitle settings
administered comprehension tests to control groups where participants watched videos without
caption, and treated groups where participants watched videos with captions (e.g. Markham,
1989). One of the functions of captions had been characterized as acting as a “mediating device”
(Vanderplank, 1988, p. 280) that helps learners when automated sound-script recognition falls
short. In recent years, more research projects focused on exploring how captions would help
listening competence under different conditions. Winke, Gass, and Sydorenko (2010) examined
the effects of captioning order, target language, and language proficiency on comprehension of
videos. Ghasemboland (2012) test listening comprehension differences between caption and
without caption groups among Iranian EFL students. Hsu et al. (2014) compared listening
performance between two groups of Taiwanese college students under filtered caption condition
and full caption with e-dictionary condition.
Full or keyword captioning
Comparing full caption and keyword captioning or partial subtitling as ways to enhance input
was another traditionally heavily investigated subsector. Markham (1999) proposed that full
caption aid learner’s comprehension by clearly visualizing word boundaries of the speech
stream. Chapelle (2009) , using the example Guillory’s (1998) research on keyword captions of
French video on learner comprehension, interpreted that keyword caption/subtitle as a way of
making particular linguistic forms salient from the cognitive interactionalist angle. In recent
years, various forms of keyword caption and subtitle had been adopted in listening
comprehension research. Mirzaei et al. (2017) filtered slow speed vocabularies to make a
partially synchronized caption and tested how it influenced learners’ listening skill. Hsu (2015)
filtered out different levels of high-frequency words that were familiar to her Taiwanese
participants and tested how their listening comprehension was affected. Perez et al. (2014)
compared differences among no caption, keyword caption and full caption on listening skill.
Aldera and Mohsen (2013) researched how did the availability of annotations for caption affect
the listening ability of Arabic EFL learners. Rooney (2014) analyzed the relationships between
keyword caption ratio with listening comprehension.
Caption or subtitle
Comparing the different effect on listening comprehension between caption and subtitle was
another research focus. Guichon (2008) compared the effects of comprehension in learner groups
under “image with or without sound”, “image with sound and caption”, and “image with sound
and subtitle” conditions. Jones (2008) researched the effects of captions and subtitles on TV
programs on EFL learners’ comprehension. Mitterer and McQueen (2009) compared how
caption and subtitle affect viewers’ speech perception differently. Latifi (2011) compared
listening comprehension ability among Iranian EFL students in “caption”, “subtitle”, “both
caption and subtitle”, and “no caption/subtitle” groups. Hayati (2011) compared groups of
viewers under no caption/subtitle, caption, and subtitle conditions.
Functions in CALL tool
A final subset on the listening competence is the use of captions and subtitles in Help Options
and Advanced Organizers. The caption is commonly used as one of the features in Multimedia
Help Options to enhance listening comprehension. Grgurovic and Hegelheimer (2007)
investigated captions as one of the important parts of CALL Help Options. In their research,
students were watching a video in an academic context. The students would be able to activate
captions and transcripts in the help option when their listening comprehension broke down. Yang
(2014) investigated comprehension using captions and subtitles in Advanced Organizers.
Vocabulary learning
Moving beyond listening comprehension, vocabulary learning is the second major focus of video
captioning and subtitling.
Incidental Vocabulary Learning
Krashen (1989) in his input hypothesis argues that the incidental acquisition of vocabulary takes
place naturally by providing the learner with comprehensible input. Most researchers consider
vocabulary learning in videos was incidental in that learners were not intentionally learning
vocabularies and constantly guess word meanings from context while captions and subtitles
serve as an enhancement for input to help comprehension. Sydorenkon (2011) investigated
students’ recognition and comprehension of vocabulary using caption/no caption comparison and
found differences in recognizing written and aural word forms between the groups. Peters et al.
(2016) compared Dutch EFL learners’ English vocabulary recognition and meaning learning
under caption and subtitles conditions. Perez et al. (2014) examined French second language
learners’ incidental vocabulary learning under “full caption”, “keyword caption”, “highlighted
caption” and “no caption condition” conditions. Peters and Webb (2018) conducted research on
Dutch EFL learners’ incidental vocabulary learning by testing their form recognition and
meaning recall after watching a captioned video.
Gass (1999) considers incidental vocabulary learning to take place “as a by-product of other
cognitive exercises (e.g., reading/listening) involving comprehension” (p.319). Several studies
tested learners’ incidental vocabulary learning by directing learners’ attention to cognition
demanding activities of video watching. Karakas and Saricoban (2011) investigated first grade
Turkish EFL learners’ incidental vocabulary learning after watching captioned subtitled cartoons
and compared their word learning with the no subtitle control group. Mousavi and Gholami
(2014) used flash story with or with caption to test learners’ incidental vocabulary learning of
Iranian EFL learners.
Intentional Vocabulary Learning
Some researchers argue vocabulary learning could be intentional if learners’ attentions were
directed to vocabulary learning. Using Vocabulary Test Announcement as way to direct learners’
attention on vocabulary learning, Perez et al. (2015) assigned Dutch French learners into
intentional or incidental vocabulary learning groups under full caption and keyword caption
groups, and used eye-tracking technology to analyze these different vocabulary conditions. and
subtitles help language learners’ vocabulary learning by either making the word form salient or
more comprehensible. Hsu (2014) compared learners’ vocabulary used in writing production
under caption and no caption learning conditions.
Learner Differences
Recent research found the effectiveness of using captions and subtitles was closely related to
learner differences, especially learners’ proficiency level with relation to the complexity of the
content.
Difficulty level of the content
Although captions and subtitles are found beneficial to second language learning, however, the
type of context seems to affect the correct interpretation of lexical meaning, since it may lead
learners to correctly or incorrectly infer the meaning of words (Webb, 2008). Michael and Webb
(2017) compared how viewers used captions to help listening comprehension under easy or
difficult content.
Proficiency level of the leaners
Learners’ proficiency levels were found to have a great impact on the degree of comprehension
and effectiveness of captions and subtitles in second language acquisition. Lavaur and Bairstow
(2011) found comprehension related to viewers proficiency level and the language used in
captions and subtitles. Chang (2011) compared low and high proficiency level English learners’
listening with or without caption on handheld devices. Teng (2019) investigated primary school
students’ comprehension of captioned videos and found that learner’s proficiency level to be a
major factor. Leveridge and Yang (2013) used Degree of Reliance test to find out learners’
reliance on captions. Yeldham (2018) found that low proficiency learners read caption more
while high proficiency learners used a wider range of cues to help them understand the content.
Other second learning and teaching purposes
Some studies invested captions and subtitles fell outside of the listening and vocabulary skill.
Frumuselu et al. (2015) tested intentional slang and colloquial word learning of college EFL
learners of various L1 backgrounds. Lunin (2015) investigated using captions and translation of
captions as a foreign language teaching method. Hosogoshi (2016) conducted research on
different listening strategies used by Japanese college EFL students in the caption group and the
subtitle group. Visky (2014) investigated using French subtitles as a way to train professional
Romanian interpreters.
Conclusion
The purpose of this synthesis was to examine recent studies on video captions and subtitles for
SLA.
Organized on the base of using of captions and subtitles as modified input that enhances
comprehension (Chapelle, 2003), this synthesis was able to explore different aspect of listening
comprehension enhancement settings of captions and subtitles including different methods to
manipulate caption and subtitle formats, and comparisons of different learning effects on
different listening context and various learner proficiency levels.
This paper identified vocabulary learning as anther investigated focus of captions and subtitles
for SLA. The majority of vocabulary focused studies examined learners’ incidental vocabulary
learning in terms of recognition and comprehension under caption and subtitle conditions.
Studies learner differences appeared to be a trend in recent research on captions and subtitles for
SLA. Although most learner difference research design covered listening and vocabulary
learning thus could be put either under listening or vocabulary learning theme, the focus of these
studies was on how learners different proficiency level affect second language learning with
captions and subtitles. I felt it would be appropriate to categorize these studies under the learner
difference group. Only five studies focused on learning other than listening or vocabulary
competence. One for slang and colloquial, the other three were on how to use captions and
subtitles as an effective way for teaching or to develop teaching strategies. The breakdown of
research focuses on the included studies is shown in Table 3.
Table 3
Breakdown of Research Focuses on the Included Studies
Research Focus
Listening
number of studies
15
Vocabulary
Learner Differences
Other Purposes
8
7
5
The decision to restrict the journal article search in the past 12 years. Since using captions and
subtitles for SLA has been researched for many decades, such a restriction was a clear limitation
of this research as it failed to capture the major trend and research direction in this field. Also,
because captions and subtitles have been widely used in language education and researched in
psychology and cognition field, the focus on CALL journals might miss valuable research
studies published in other education or psychology focused journals.
Despite these limitations, this synthesis has shown the development of research on the
effectiveness of using captions and subtitles for SLA. The focus and discussions of rexent studies
on how different formats of captions and subtitles affect second language learning results and
how learners’ different proficiency level influenced learning with captions and subtitles may
shed some lights on future research directions in this field.