CALICO 2004
“CALL: Focusing on the Learner”
Carnegie Mellon University
Bridging Theory and Practice:
Research-Based Listening Tasks for Video Comprehension
Luba Iskold, Ed. D.
Statistical analysis
Greg Cicconetti, Ph.D.
Muhlenberg College
Allentown, PA
Statement of the Problem
•
How do students develop listening skills by using video materials?
•
Do they learn best by mere “exposure to …comprehensible input?”
•
Should we assist students in comprehending a videotext?
•
•
Do listening tasks performed by students while they are watching a
video episode help them concentrate on important information in
that video?
Which tasks leave the learner with higher levels of video
comprehension?
The Purpose of the Study
•
•
•
To investigate if activities performed by learners during the listening
stage enhance comprehension of a video episode.
Two conditions of video viewing: ‘exposure only ‘and using Video
Guides specifically designed for the present investigation.
Thus, the purpose of this empirical study was to examine:
1. the effect of listening tasks performed during video viewing on the
comprehension of second-semester Russian undergraduate college
students of video episodes, as measured by Immediate Recall Protocols
written in English, and
2. the effect of a Video Guide condition on students’ recall, recognition,
and application of the content, vocabulary, structures, and cultural
information from “Начало” video, as measured by immediate and
delayed tests completed in Russian.
Previous Research
Theoretical perspectives on language learning
– Comprehension-based approaches
– Cognitive-theoretical views
– Sociocultural perspectives
Research Related to Listening
–
–
–
–
–
Research on listening and reading comprehension
Factors that affect listening comprehension
Research on listener characteristics
Authentic materials in listening research
Video in listening research
How does one teach listening?
• Constructing tasks for listening
•
comprehension
Assessment of listening
Constructing Tasks for Listening
Richards (1983) suggested manipulation of two variables
as a means to develop listening micro-skills:
the input and the task
INPUT  MICRO-SKILLS  TASKS
His taxonomy includes
33 micro-skills for listening to conversations
18 additional skills for academic listening:
Joiner (1990) recommended listening/viewing guides which
students use while watching the video episode
Assessment of Listening
Comprehension
How does one determine which listening tasks work best
for learners?
•
•
•
Recall protocols, frequently used in L1 research.
In 1986-87, Bernhardt and James applied recall protocols to assess
reading and listening
comprehension in L2.
Glisan (1988) advocated using comprehension checks in the native
language, therefore avoiding the mixture of skills.
Materials designed for this study:
•
•
•
•
•
•
Interactive Viewing Guides
Recall Protocols Scoring Sheets
Immediate and Delayed Quizzes
(recall, recognition, application)
Exit Survey
Participant Information Survey
Baseline Test
The Viewing Guides Constructed for
the Study
•
The purpose:
To facilitate understanding of the video episodes.
To help students focus on specific important aspects of each videotext, NOT
to assess their comprehension of the video
•
Features:
The tasks were explained before students start watching the video. The
reading load of the tasks was kept to a minimum. The items were
presented in the true/false format
•
Sections:
Instructions
Main Ideas: characters, places and events
Details: vocabulary, phrases, cognates, idioms
Cultural similarities/differences
Space for note taking
The Study
Setting
An undergraduate, four-year college with 1, 800 students.
Maintains a foreign language requirement.
Languages: French, German, Hebrew, Italian, Russian, Spanish,
Greek, and Latin.
An interdisciplinary major/minor in Russian Studies.
Elementary Language Level I and II are first and second semester
courses of a target language sequence offered by the college.
Placement is dependent upon experience or a placement test.
Population and Sample
The study took place in the spring of 2004.
Sixteen students enrolled in one section of a fifteen-week,
second semester Elementary Russian II participated in
the study:
12 freshman, 4 sophomores; 7 men, 9 women;
all students took Elementary Russian I;
13 students never took Russian in high school;
3 participants had 2 years of Russian;
11 were planning to major/minor;
5 were fulfilling the College’s FL requirement;
14 students were native speakers of English;
2 were heritage speakers of Polish.
Treatments
An intact class of sixteen Elementary Russian II
Taught by one faculty member participated
The class was divided into two groups
Group homogeneity in composition:
Students were ranked by prior achievement in
Elementary Russian I and then randomly
assigned to Group 1 (G1, n1 = 8) and Group 2
(G2, n2 = 8).
• Baseline Test on listening comprehension.
• Both groups followed the same syllabus:
»
»
»
»
»
received the same instruction
completed the same assignments
took the same quizzes
on the same dates
in the same setting.
Watching the Video
The participants watched four «Начало» video episodes on CD-ROM
at the Language Learning Center, under two different conditions:
•
•
•
•
A control group: ‘exposure only’ to the videotext in the commercially available format.
An experimental group: the same videos in conjunction with the on-line Video Guides
designed for the present investigation.
Group One (G1) served as the control while Group Two (G2) served as the
experimental group for video episodes one and three.
Group One (G1) served as the experimental group, while Group Two (G2) served as
the control for video episodes two and four.
Table 1
Allocation of Groups to Video Formats
Video
Episode 1
Episode 2
Episode 3
Episode 4
Group I
Control
Experimental
Control
Experimental
Group II
Experimental
Control
Experimental
Control
Variables:
Controlled variables:
•
•
Groups (control and experimental)
Treatments, conditions for video viewing.
Dependent variables:
1. the amount of text material (or the number of scored points)
recalled in four Immediate Recall Protocols;
2. the scores in listening comprehension achievement, as measured
by four sets of immediate and delayed quizzes, and
3. participants’ opinions regarding the effectiveness of treatments, as
measured by the Exit Survey.
Materials
Texts
The videotexts employed in the study were taken from
«Начало» (Lubensky et. al)|
– a video-driven package for beginning students of Russian
– integrates the teaching of language and culture
– provides a balance between structure and communicative
activities.
– sixteen 2.5-3.5- video episodes, connected by a story line;
filmed in Moscow
Simulated Authentic discourse
Testing and Scoring Procedures|
Sample Recall Protocol Scoring Sheet Начало # 5: Бизнес по-московски
4 --most important event
1 --least important event
Idea Units/Sentences: 15
Total Points: 48
Points Possible
Idea Units
2
What a nightmare! How terrible!
4
The house is new, but there is no asphalt.
4
Lena cannot find rubber boots. They don’t have her size.
4
All neighbors have a big problem.
4
Viktor understands their problem and is ready to help.
4
Viktor has a business. He has rubber boots of all sizes, but he does not sell them.
3
Every morning he waits for the neighbors in the hallway and hands out rubber boots to them.
3
Viktor’s friend is at the bus stop. He waits for the neighbors and takes the boots.
3
The neighbors pay 500 rubles.
3
They think it’s expensive.
4
Viktor says that 500 is not expensive. For this money they get clean boots and good mood.
2
Viktor says that pretty girls get the boots for free.
2
Sasha thinks, its discrimination.
2
Victor says it’s a joke.
4
Viktor thinks that Lena is a very pretty girl.
Points Scored
Sample Quiz (Immediate)
sections 2 & 3
Findings
Table 2
Summary of Nested Factorial Statistical Analysis of the Immediate Tests
Quiz
Group
N
Recall (1)
P = 0.087
P = 0.370
62
Recognition (2)
P < 0.0005
P = 0.585
62
Application (3)
P = 0.002
P = 0.444
62
Total Score (4)
P = 0.002
P = 0.354
62
Immediate Recall Protocols (5)
P < 0.0005
P = 0.022
58
•
•
The results presented in Table 2 suggest that:
There were no significant differences among scores from Recall (1) that may be
attributed to either episode version, or treatment;
Although the scores from Recognition (2), Application (3) and Total scores do exhibit
differences, the treatment participation of a student (Control or Experimental) does not
appear to affect his/ her scores.
Statistically significant differences were found among the treatment groups
and video episodes on the Immediate Recall Protocol:
The Experimental group consistently performed better on Immediate Recall
Protocols.
The findings strongly suggest that the Video Guides were effective.
Table 3
Mean Scores and Standard Deviations on Immediate Recall Protocols
Video Episode
1
2
3
4
Treatment Group
Mean
Standard Deviation
N
Control
83.67
18.82
7
Experimental
88.69
20.98
8
Control
82.74
21.84
7
Experimental
98.62
2.88
8
Control
78.29
20.51
7
Experimental
96.57
4.86
7
Control
52.75
10.35
7
Experimental
71.98
20.32
7
Table 4
Mean Scores and Standard Deviations on the Exit Survey
Question
Mean
Standard
Deviation
Using Video Guides helped me to stay focused during video viewing.
2.05
0.42
Using Video Guides helped me to remember the video episodes better.
2.00
0.61
In my opinion, using an interactive Video Guide may help me learn more from the
video episodes.
2.00
0.50
I would like to continue using a Viewing Guide while watching «Начало» episodes
next year.
1.82
0.52
I liked the design of the current Video Guide.
2.17
0.80
The Video Guide distracted me from watching the video.
3.82
0.95
Limitations
•
•
•
•
•
•
•
•
The study was conducted at a small liberal arts undergraduate institution.
The nature of the population may present questions about the role of
attitude, interest, and motivation in FL video viewing.
Because all students were in the same class, there was no random selection
of participants.
The sample size was limited to 16 students. Randomizing the sample and
broadening its size might enhance the validity of the study.
In the future, incentives may be incorporated to decrease the number of
missing observations.
The findings are based on a particular kind of videotext, a simulated
authentic discourse based on a fictional narrative, that might not be
applicable to other types of videotext (e.g., news broadcasts or interviews).
The results may not generalize to studies that employ different assessment
instruments.
A longitudinal study over several semesters may be better suited for the
purpose.
Contact Information:
Dr. Luba Iskold
2400 Chew Street
Muhlenberg College,
Languages, Literatures and Cultures,
Allentown, PA 18104
Phone: 484-664-3516
Fax: 484-664-3722
E-mail: iskold@muhlenberg.edu
http://www.muhlenberg.edu/depts/forlang/LLC/iskold_home/index.htm