Reasoning in lexical tasks
during reading: Effects on
recognition memory and L2
development
Ana Martínez-Fernández
Department of Spanish and Portuguese
Georgetown University
TBLT 2009
Introduction
Task demands
Deep / cognitive processing
Long-term recognition
memory?
L2 development
• SLA research on incidental vocabulary learning
• SLA research on task-based learning
• Cognitive psychology research on recognition memory
Literature review I
• The Involvement Load Hypothesis (Laufer & Hulstijn, 2001)
 Tasks inducing higher involvement lead to higher
vocabulary retention
 ‘Need’
 ‘Search’
 ‘Evaluation’
(N)
(S)
(E)
Glosses
[+N, -S, -E]
 Conflicting results
 Hulstijn & Laufer (2001)
 Martínez-Fernández (2008)
Fill-in-the-blanks
[+N, -S, +E]
Literature Review II
• No control for the distance between the targeted words
to fill in the blanks and distracters
• Hulstijn & Laufer (2001)
 No process measures to ensure that tasks led to the
involvement predicted
• Martínez-Fernández (2008)
 Think-aloud protocols
 No control for reactivity
Literature review III
• The Cognition Hypothesis (Robinson, 2001, 2005, 2007)
 Task complexity is determined by cognitive factors
Complex tasks with features that direct
the cognitive effort to linguistic forms
[+/- Reasoning]
Noticing and long-term retention
 This prediction has not been tested in the area of
incidental vocabulary learning and text comprehension
Literature review IV
• No studies investigating the effects of reasoning on memory
 Tulving (1983) distinguished two subsystems of recognition memory
Long-term declarative memory
Episodic memory
Experience of
remembering
Semantic memory
Experience of
knowing
• Studies show that deep processing enhances episodic memory,
which may have a positive effect on learning (Gardiner, 2008)
Research questions
• Does the extent of reasoning involved in a reading
comprehension task plus fill-in task have a differential effect on
recognition memory?
• Does the extent of reasoning involved in a reading
comprehension task plus fill-in task have a differential effect on
vocabulary development?
• Does the extent of reasoning involved in a reading
comprehension task plus fill-in task have a differential effect on
text comprehension?
Research Design
• Independent variables
 Between-subjects: Extent of reasoning
 [+R]
[+R, +TA], [+R, -TA]
 [-R]
[-R, +TA], [-R, -TA]
 Control
 Within-subject:
 Time: Immediate posttest, 1-week delayed posttest
 Type of word: Targeted words, alternate words
• Dependent variables
 Recognition memory
 Vocabulary development: production & recognition
 Text comprehension: local ideas & global ideas
Participants
• 71 English-speaking adults enrolled in college-level
second-year Spanish language courses
• Original pool: N = 147; participants were excluded from
the final sample for any of the following reasons:




They failed to attend all sessions
They did not follow the instructions
They had prior knowledge of one or more targeted words
They had had outside exposure to the targeted words either
between the pretest and the treatment or between the treatment and
the delayed posttest
Experimental Texts and Items
• Experimental text
 Fairy tale in Spanish (“El verdadero valor del anillo”
in Déjame que te cuente, by Jorge Bucay)
 646 words
• Targeted items
 12 unfamiliar nouns (after excluding 4)
 6 targeted words (3 concrete and 3 abstract)
 6 alternate words (3 concrete and 3 abstract)
Experimental Task I
 [+ Reasoning]
 Participants read a text containing 8 blanks. To fill in
each blank they had to choose one of two glossed
words. Only one was correct based on specific
contextual clues.
The man looked into the distance and
(1)
pointed at (1) ______________ with his
- un cortijo:
an estate
hand. “You will find him there,” he said.
When Fernando got there, he saw the wise - una chabola:
a shack
man standing outside, wearing ripped
pants and an old shirt. He looked quite
poor.
Experimental Task II
• [- Reasoning]
 Participants read the same text and filled in the blanks. One of the
two options was clearly incorrect because it had the opposite
lexical feature [+/- concrete] to that selected by the previous
word.
The man looked into the distance and
pointed at (1) ______________ with his (1)
hand. “You will find him there,” he said. - la desidia:
the apathy
When Fernando got there, he saw the
- una chabola:
wise man standing outside, wearing
ripped pants and an old shirt. He looked a shack
quite poor.
 Both tasks were pilot-tested with two native speakers and three
students of Spanish
Treatment
 Random assignment to groups





[+R, +TA]
N = 15
[+R, -TA] N = 16
[-R, +TA] N = 13
[-R, -TA]
N = 14
[Control, +TA]
N = 13
 Oral instructions to think aloud
 Information about subsequent comprehension questions
 Time on task was measured but not controlled
Assessment Tasks and Procedure I
Week 1:
Pretest
Week 2: Treatment +
Immediate Posttest
Week 3:Delayed
Posttest
Multiple-choice
meaning
recognition test
(32 items)
Text comprehension test (12
open questions: 6 global and
6 local)
Memory judgment
task
Oral + written instructions
for the memory judgment
task followed by the task
Vocabulary tests
Vocabulary tests (32 items)
-Meaning production
-Meaning recognition
Post-debriefing
questionnaire
- Production
- Recognition
Assessment Tasks and Procedure II
• Instructions for the memory judgment task:
YES / NO
IF YES: R / K / G
 Definitions and examples
 4 participants were asked to think aloud
 Items: 6 targeted words, 6 alternate words, 6 distractors
• All tasks were timed
• Items were presented one by one with a Power Point
presentation
• Participants provided their answers in an answer sheet
Analysis and Results
• One-way ANOVA on time on task:
 F(4, 71) = 8.698, p = .00
 [+TA] groups spent significantly more time on task than [-TA] groups
• One-way ANOVA on performance on the fill-in task:
 F(3, 58) = 10.854, p = .00
 [-R] groups significantly outperformed [+R] groups
• Index of performance quality on the remember-know task
 Hits minus false alarms
• Reliability of tasks: Cronbach’s alpha
 Text comprehension = .71
 Meaning production = .57
 Meaning recognition = .39
Results: Recognition Memory
• Data were submitted to 5 x 2 x 3 Repeated measures
ANOVA
Items
Targeted
words
Alternate
words
Time
.00*
.01*
Response
.00*
.20
Group
.44
.03*
Time x Group
.66
.94
Response by Time
.30
.14
Response x Group
.24
.60
Time x Response x Group
.02*
.73
Effects
Results: Recognition Memory (Targeted Words)
Response by Time for Targeted Words (TW)
25%
20%
15%
Immediate
1-week-delayed
10%
5%
0%
-5%
Remember
Know
Guess
Results: Recognition Memory (Targeted Words)
Response by Group for TW on the immediate posttest
35%
30%
25%
20%
R
15%
K
10%
G
5%
0%
[+R, +TA] [+R, -TA]
[-R, +TA]
[-R, -TA]
[C, +TA]
Response by Group for TW on the delayed posttest
35%
30%
25%
20%
15%
10%
5%
0%
-5%
-10%
[+R, +TA] [+R, -TA] [-R, +TA] [-R, -TA] [C, +TA]
Results: Recognition Memory (Alternate Words)
Response by Group for AW on the immediate posttest
25%
20%
15%
R
K
G
10%
5%
0%
[+R, +TA] [+R, -TA] [-R, +TA] [-R, -TA]
[C, +TA]
Response by Group for AW on the delayed posttest
25%
20%
15%
10%
5%
0%
-5%
[+R+T A] [+R -TA] [-R +TA]
[-R -T A]
[C +TA]
Results: Vocabulary Development
o
Data on vocabulary were submitted to 5 x 2 x 3 Repeated
Measures ANOVA
Tests
Meaning
production
Meaning
recognition
Time
.00*
.00*
Word type
.00*
.07
Group
.00*
.00*
Time x Group
.00*
.00*
Time x Word type
.01*
.20
Word type x Group
.04*
.14
Time x Word type x Group
.00*
.31
Effects
Results: Meaning Production (TW)
Results: Meaning Production (AW)
+R, +TA
+R, -TA
Results: Meaning Recognition (TW)
Results: Meaning Recognition (AW)
+R, +TA
+R, -TA
Results: Text Comprehension I
• Data on text comprehension were submitted to
5 x 2 Repeated measures ANOVA
Tests
Effects
Text
comprehension
Idea Type
.00*
Group
.00*
Idea Type x Group
.00*
Results: Text Comprehension II
Results: Qualitative Data I
• [+R] conditions
 Considered both options given to fill in the blanks
 Continued reading to find clues in the context, and test their hypotheses
“I think it’s un cortijo…[7 lines below] You know what?
This man is very poor, he would never have an estate, he has
a shack… chabola”
“clumsiness or weakness, both could work [continues reading]
Oh, he breaks everything. He’s clumsy, not weak” ”
Results: Qualitative Data II
• [-R] conditions
 Do not verbalize the alternate words
 Reasoning refers to common knowledge of the world
“he is pointing at a shack, chabola”
“weakness doesn’t make sense, so it’s gonna
be horse, because he’s gonna take something
and the weakness is not something you can take”
• Control condition
 Skipped targeted unfamiliar words
 Commented lack of knowledge
 Contextual guessing
Results: Qualitative Data III
• R: I remember because I remember reading it on a side, and
also because I wasn’t sure if it was pronounced “real” or
“rial” so I remember thinking about that
• K: I think it was… I don’t remember where but I’m pretty sure
that it was in there; Yes, I don’t remember where I saw it, but I
know it was in there
• G: It’s a pure guess, I don’t remember that word, period; Yes, I
am guessing, maybe when Fernando was in the market
Results: Qualitative Data IV
• Some Know experiences may be interpreted as
guessing:
 I feel like that was in there, I almost think that it was one of
the words in the margin but I am not very sure; Yes, but I’m
guessing
• Some Remember experiences may be interpreted as
knowing:
 Yes, it was in the text, I remember reading it, I remember
being unsure of what it meant, but I can’t remember where
in the text it was. So I guess I know it was in the text, but I
don’t remember where
Discussion I
• RQ 1: Effect of [+/-R] on recognition memory
 Overall, the [+R] groups remembered more TW than the
other groups
 Thinking aloud while performing a complex task may help maintain
episodic memories over a period of a week
 There was a significant decrease of R responses in the [-TA] group on
the delayed posttest, and a significant increase of K responses
• Remember-to-know shift (Knowlton & Squire, 1995; Conway et al.,
1997)
 The [+R] groups remembered significantly more AW than the
other groups, and this effect was mantained over a period of a
week
 No reactivity for thinking aloud on immediate or delayed posttests
Discussion II
• RQ 2: Effect of [+/-R] on vocabulary development
 Only the [+R] groups significantly outperformed the control
group, due to their higher performance on AW
 Even though performance on the fill-in task was significantly lower
in the [+R] conditions
 Cognitive complexity did not affect time on task
 Thinking aloud and time on task did not affect performance
on the posttests
 Support for previous SLA studies on reactivity (Leow & MorganShort, 2004; Bowles & Leow, 2005; Bowles, 2008; Medina, 2008;
Yoshida, 2008)
Discussion III
 The [+R] effect was not retained after one week, in contrast to
what the Cognition Hypothesis predicts
 Consistent with incidental vocabulary learning studies (Watanabe,
1997; Bowles, 2004)
 Studies where multiple encounters with the targeted words are
provided show significant differences between task conditions on
delayed posttests (Rott, 2005; Martínez-Fernández, 2008)
 The superiority of [+R] group cannot be explained by the
Involvement Load Hypothesis, since both tasks have the same
involvement load: [+N, -S, +E]
Discussion IV
• The [+R] groups:
 Remembered significantly more words, especially alternate words, than
the [-R] groups and the control group
 Produced and recognized the meaning of significantly more words,
especially alternate words, than the control group
• It may be hypothesized that the meaning of those words that
received remember responses were produced and recognized
to a greater extent than those that received know or guess
responses
Discussion V
• RQ 3: Effect of [+/-R] on text comprehension
 All groups reached high comprehension of global ideas
 All experimental groups reached significantly higher
comprehension of local ideas than the control group
 Learners can perform this type of dual task (i.e., a lexical task
embedded in a reading comprehension task) with different
degrees of cognitive complexity, without negatively affecting
text comprehension
Limitations and future research
• Increase the validity of the remember-know task
• Refine vocabulary learning measures
 Low reliability
• Refine the design of fill-in tasks
 Think-aloud protocols showed that not all blanks in the fill-in task led
to reasoning and evaluating to the same degree in the complex
condition
• Future studies should
 Investigate the interaction between cognitive complexity and amount of
exposure
 Compare the effect of fill-in tasks and glosses in reading
comprehension tasks, controlling for the effect of cognitive complexity
 Investigate the role of recognition memory in L2 development
 Examine the effects of thinking aloud on recognition memory and L2
learning
References
 Bowles, M. A. (2004). L2 glossing: To CALL or not CALL. Hispania, 87(3), 541-552.
 Bowles, M. A. (2008). Task type and reactivity of verbal reports in SLA: A first look at a L2 task
other than reading. Studies in Second Language Acquisition, 30(3), 359-387.
 Bowles, M. A. & Leow, R. P. (2005). Reactivity and type of verbal report in SLA research
methodology. Studies in Second Language Acquisition, 27(3), 415-440.
 Conway, M.A., Gardiner, J. M., Perfect, T.J., Anderson, S.J., & Cohen, G.M. (1997). Changes
in Memory Awareness During Learning: The Acquisition of Knowledge by Psychology
Undergraduates. Journal of Experimental Psychology: General, 126,393-413.
 Gardiner, J.M. (2008). Remembering and Knowing. In Byrne, J. H. (Ed.), Learning and memory: A
comprehensive reference, Vol. 2, pp. 285-305. UK: Academic Press.
 Knowlton, B. J. & Squire, L. R. (1995). Remembering and Knowing: Two Different Expressions
of Declarative Memory. Journal of Experimental Psychology: Learning, Memory, and Cognition,
21(3), 699-710.
 Laufer, B., & Hulstijn, J. H. (2001). Incidental vocabulary acquisition in a second language:
the construct of task-induced involvement. Applied Linguistics, 22(1), 1-26.
 Leow, R. P., & Morgan-Short, K. (2004). To think aloud or not to think aloud: The issue of reactivity
in SLA research methodology. Studies in Second Language Acquisition, 26(1), 35-57.
• Martínez-Fernández, A. (2008). Revisiting the involvement load hypothesis: Awareness, type
of task and type of item. In M. Bowles, R. Foote, S. Perpiñán, & B. Rakesh (eds.), Selected
Proceedings of the 2007 Second Language Research Forum (pp. 210-228). Somerville (MA):
Cascadilla Proceedings Press.
• Medina, A. (2008). Concurrent verbalization, task complexity, and working memory: effects on L2
learning in a computerized task. Unpublished dissertation, Georgetown University, Washington,
DC.
• Robinson, P. (2001). Task complexity, cognitive resources, and syllabus design: A triadic
framework
for examining task influences on SLA. In P. Robinson (Ed.), Cognition and Second Language
Instruction, p. 237-318. Cambridge: Cambridge University Press.
• Robinson, P. (2005). Cognitive complexity and task sequencing: A review of studies in a
Componential Framework for second language task design. International Review of Applied
Linguistics, 25, 45-73.
• Robinson, P. (2007). Criteria for grading and sequencing pedagogic tasks. In M. P. García Mayo
(Ed.), Investigating Tasks in Formal Language Learning (p. 7-27). Clevedon: Multilingual Matters.
• Rott, S. (2005). Processing glosses: A qualitative exploration of how form-meaning connections are
established and strengthened. Reading in a Foreign Language, 17(2), 95-124.
• Tulving, E. (1983). Elements of episodic memory. Oxford: Oxford University Press.
• Watanabe, Y. (1997). Input, intake, and retention: Effects of increased processing on incidental
learning of foreign language vocabulary. Studies in Second Language Acquisition, 19(3), 287-307.
• Yoshida, M. (2008). Think-aloud protocols and type of reading task: The issue of reactivity in
L2 reading research. In M. Bowles, R. Foote, S. Perpiñán, & Bhatt, R. (Eds.) Selected Proceedings
of the 2007 Second Language Research Forum, (pp. 199-209). Somerville, MA: Cascadilla
Proceedings Project.
Thank you!