WHAT IS DONALD DUCK SAYING? MULTIMEDIA AND L2 LISTENING
Speaker: Luis Díaz Parra
Paivio´s dual coding Theory (1986)
Human possess two separate channels for processing visual and auditory information.
AUDITORY
VISUAL
Dually coded information that acts as input may have additive effects for recalling information
since it involves a more elaborate semantic encoding within these two separate, but
interconnected channels, and also because it allows for more than one retrieval route.
Mayer´s Cognitive Theory of Multimedia Learning (2001)
It addresses the fact that Multimedia Learning (ML) is a process that requires selecting ,
organizing and integrating verbal and visual information into coherent verbal and pictorial
representations. Furthermore, Mayer claims that “…multimedial messages that are design in the
light of how the human mind works are more likely to lead to meaningful learning than those that
are not”.
Fig 1
Mayer´s theory states that active learning from multimedia instructional messages includes five
co-coordinated processes: (1) selecting relevant words, (2) selecting relevant images, (3)
organizing the selected words into a verbal mental model, (4) organizing the selected images into
a pictorial mental model, and (5) integrating the verbal model and the pictorial model with prior
knowledge into a coherent mental representation. Figure 1 displays the design.
Advantages of Multimedia Environments
 To clarify meanings and clarify language:
Words- Pronunciation- Chunks- Grammar
 Using different modalities ( in combination):
Audio – Text - Visuals
Advantages of visual input
 Within text: more effective than other types of modalities.
Meskill (1996): pictures more effective when reading a text.
 Guichon, N. & McLornan, S. (2008) : Only audio v/s Video : Ss remember more semantic
units when exposed to audio + images
 Conclusion: Using visuals = more favorable recalling effect than other kind of modalities.
Advantages of captions/subtitles
 Determining factor: immediate feedback of what it is being said consecutively shown on
the screen. (hear and read)
 Segmenting of input: Understanding reduced speech “whadjagonnado?” (ellision of
phonemes )
 CONSENSUS: Captions/Subtitles are more beneficial for immediate language
comprehension.
Listening comprehension in Multimodal Environments
There is an integration of different modalities:
 Text : (on-screen text, subtitles, transcripts)
 Visuals : (e.g pictures, slides, video images)
Not only enhances the comprehension of language listening-based activities, but also the
development of learners´ listening skills.
Theoretical and pedagogical implications (When exposing students to multimodal presentations)
 Better recalling of semantic units and overall comprehension:
 Images and subtitles provide enhanced input and therefore more comprehensible input
more beneficial
 Exposure to multimodal input is more beneficial: Redundancy of information
Suggestions for future research
 Consider Learners´ level of proficiency and learning styles.
 Conduct studies using other kinds of video texts.
 Use of multimodal texts in longitudinal studies.
 Students manipulation of input. (they can pause or rewind when watching a video)

Test other aspects of language acquisition: pronunciation, vocabulary acquisition, spelling.
 Use of other kinds of assessments.
CONCLUSIONS
 Multimodality is more beneficial.
 Exposing subjects to one condition alone may cause an intrinsic cognitive load.
 Redundancy of information helps text comprehension.
 Necessity to design multimedia materials that address students´ learning styles and
preferences