3rd Biennial International Conference on Task-Based Language Teaching, Lancaster University, Sept 15, 2009
Task-internal and task-external readiness:
A report of the effects of topic familiarity and strategic
planning on task performance by L2 learners of
different proficiency levels
Gavin Bei Xiaoyue
The Chinese University of Hong Kong
gavinbei@gmail.com
Part 1
Research background
Contextualizing
• Task-based instruction research looks at:
• 1. Task characteristics:
• Subjective or objective, structural or non-structural,
familiar or unfamiliar…
• 2. Task conditions:
• Monologic or interactive, Pre/Post task activities,
planning or non-planning…
• 3. Participants:
• Gender, motivation, learning style, proficiency…
Research background 1: topic familiarity
• 1. Comprehension (many)
• facilitative
• e.g., Shimioda, 1993; Barry and Lazarte, 1995; Bügel &
Buunk, 1996; Chen and Donin, 1997; Johnson, 1982; Lee,
1986; and Chang, 2006
• no effect on comprehension
• e.g., Hammadou, 1991; Peretz & Shoham, 1990; and
Carrell (1983)
• 2. Production (few)
• Mostly in L1 research by psychologists
• Higher fluency, but inconsistent in accuracy or complexity
Research Background 2: planning types
•
•
•
•
•
Two macro and four micro types of planning (Ellis, 2005)
1. pre-task
1) rehearsal 2) strategic planning
2. within-task
1) pressured 2) unpressured
• Or simply three micro types (Ellis, in press)
• 1) rehearsal 2) strategic planning 3) within-task planning
Research background 3: strategic planning
• Ample studies (e.g., Skehan, Foster, Ellis,
Crookes, Wigglesworth, etc.) with quite some
consistent results.
• Planning raises: Fluency + Complexity
• (sometimes, but usually not) Accuracy
• Skehan: Trade-off of between Comp. and Accu.
• Robinson: Planning does not lead to Comp, no
trade-off.
• Is proficiency important here?
Research background 4: Proficiency and
Familiarity
•
•
•
•
Hudson (1982):
In Reading: Familiarity > Proficiency.
Schmidt-Rinehart (1994):
In Listening: Familiarity > Proficiency.
•
•
•
•
Carrell (1983):
In Reading: Proficiency > Familiarity (NS:NNS)
Chern (1993):
In Reading: Proficiency > Familiarity.
Research background 5: proficiency
and planning
• Wigglesworth (1997): low proficiency did not
benefit from planning.
• Tavokoli and Skehan (2005): planning drove
high and low learners for better performance.
• Kawauchi (2005): more Flu. and Comp. for
higher learners, more Accu. for the lower. The
advanced gained the least.
• Most other studies did not consider proficiency.
Part 2
The study design
and methodology
1.Participants and proficiency test
• Participants: 80 HK Cantonesespeaking undergraduates volunteers to
participate.
• A C-test as proficiency test to group
participants
------- borrowed from Dornyei and Katona (1992).
--------The validity and reliability are good in the
literature and in the present context. See appendix 1
2. Tasks
• Topic 1: Natural Viruses.
• Topic 2: Computer Viruses.
Medicine
Majors (N=40)
Computer
Majors (N=40)
Topic 1
Topic 2
+ familiar
– familiar
– familiar
+ familiar
3. Independent Variables
• 1. Topic familiarity (within): 2 levels
• familiar VS unfamiliar task
• 2. Planning (between): 2 levels
• non-planning VS 10-min planning
• 3. Proficiency (between): 2 levels
• intermediate VS high
Study design
Planning
Proficiency
(between)
(between)
Topic familiarity (within)
Familiar
Unfamiliar
High
20
20
Intermediate
20
20
High
20
20
Intermediate
20
20
Planners
Nonplanners
Each cell consists of 10 computer majors and 10 medicine majors as
counterbalancing to rule out the topic effect.
4. Dependent Variables
• Fluency: pausing, speech rate, MLR, phonation time, repairs, etc.
• Accuracy: error-free clauses ratio, length of correct clause, and
errors per 100 words.
• Complexity: Clauses per AS unit, AS unit length, and clause
length
• Lexis: lexical diversity, lexical sophistication, and lexical density.
• Formality: F-score, DB-score
• Totally 21 measure were employed. See Appendix 2 for a
detailed description.
• P value: the significance level to tell whether there is an effect.
• Cohen’s D value: the effect size to tell how big the effect is.
5. Statistical procedures
• A repeated measures analysis of variance
(ANOVA) was employed.
Part 3
Results
Breakdown fluency main effects 1
Table 1. p and Cohen’s D value
T. Familiarity
Planning
Proficiency
D
.26
p
.000
D
.58
p
D
Speech rate
p
.000
ns
/
Phonation time
.000
.35
.000
.62
ns
/
Mean length of run
.016
.17
.046
.32
ns
/
No. Mid-clause pauses
.000
.38
.001
.54
ns
/
ns
/
ns
/
.027
.48
No. End-of-clause pauses
Means omitted due to the space limit. The means show that the familiar topics and the
planning time improve fluency. Same directions below unless there is a note.
Breakdown fluency main effects 2
Table 2. p and Cohen’s D value
T. Familiarity
Planning
Proficiency
p
D
p
D
p
D
Mid-clause silence total
.000
.38
.000
.61
ns
/
End-of-clause silence ttl.
.014
.19
.007
.59
ns
/
Av. mid-clause pause
.019
.28
.000
.71
ns
/
ns
/
.004
.64
ns
/
.085
/
ns
/
ns
/
Av. End-of-clause pause
No. filled pauses
Breakdown fluency: interactions
• T. Familiarity has significant interactions with
Planning in:
•
•
•
•
•
1) speech rate
2) phonation time
3) No. Mid-clause pauses
4) Mid-clause silence total (per 100 words)
5) End-of-clause silence total (per 100 words)
All showing one pattern:
planning compensates for the unfamiliar topics.
Measures: speech rate and mid-clause silence total
Breakdown fluency: a summary
1) T. Familiarity affects fluency in a strikingly
similar way as Planning does.
2) Approximately, the effect sizes of T. Familiarity is
half as big as those of Planning.
3) Planning mitigate the difference between familiar
and unfamiliar topics.
4) The effects of Proficiency is marginal, and
probably overridden by T. F. and Planning.
Repair fluency
Table 3. p and Cohen’s D value
T. Familiarity
Planning
Proficiency
p
D
p
D
p
D
ns
/
.000
1.02
ns
/
Reformulations
.088
/
.001
.53
ns
/
Replacements
.077
/
.008
.43
ns
/
Repetitions
.001
.40
.000
.75
ns
/
False starts
Note: planning induced more replacements, though reducing others.
Accuracy
Table 4. p and Cohen’s D value
T. Familiarity
Planning
Proficiency
p
.02
D
.22
p
ns
D
/
p
.000
D
.69
70% accuracy
clause length
ns
/
ns
/
.000
.57
Errors per 100
words
.000
.38
ns
/
.000
.77
Error-free clauses
ratio
Complexity
Table 5. p and Cohen’s D value
T. Familiarity
Planning
Proficiency
p
ns
D
/
p
.018
D
.39
p
.067
D
/
Words per AS unit
ns
/
.000
.81
.000
.52
Words per clause
ns
/
ns
/
ns
/
Clauses per AS
unit
Lexis
Table 6. p and Cohen’s D value
T. Familiarity
Planning
Lexical diversity
p
.018
D
.29
p
ns
D
/
p
ns
D
/
Lexical sophistication
.000
.41
ns
/
ns
/
ns
/
.008
.43
.031
.39
Lexical density
Proficiency
Formality
Table 7. p and Cohen’s D value
T. Familiarity
F-score
DB-score
Planning
Proficiency
p
D
p
D
p
D
.000
.49
.003
.48
ns
/
ns
/
.019
.38
ns
/
Part 4
Some conclusions
Conclusions 1
• 1. Planning is more powerful in driving fluency than
T. Familiarity. It can reduce the differences between
familiar and unfamiliar topics in breakdown fluency.
• 2. Topic familiarity and planning seem to be more
concerned with meaning expression (similar).
• 3. T. familiarity and planning affect different
syntactic areas (different).
• 4. Proficiency affects mostly forms, esp. accuracy,
but not so much meaning expression (fluency and
lexis).
• 5. Higher proficiency does not appear to remove the
trade-off effects. So L2 learners are L2 learners!
Conclusion 2
• 6. Factor analyses of all measures show:
• 1) there’s probably an end-of-clause fluency different
from breakdown and repair fluency.
• (Av. Pause and total silence at the end of a clause, and
phonation time. )
• 2) there’s probably a noun-phrase complexity as
compared to the syntactic complexity.
• (words per clause, F-score, DB-score, and Lambda)
• 6. A broader perspective on planning stems from the similarities
and differences between T. familiarity and strategic planning in
this study, in which I argue that T.F. can be regarded as a kind
of implicit planning (see next page).
A general framework of task-readiness
Macrodimension
 Taskinternal
readiness
(implicit
planning)
Learner
readiness
for
a task
 Taskexternal
readiness
(explicit
planning)
Micro-dimension
Sample studies
 Topic familiarity (prior
domain knowledge)
This study
 Schematic familiarity
(story structure)
Skehan and Foster (1999)
 Task familiarity (task
types)
Bygate (2001)
 Rehearsal ( content
repetition)
Bygate (1996)
 Strategic (pre-task)
planning
Foster and Skehan (1996)
 Within-task (on-line)
planning
Yuan and Ellis (2003)
Thank you!
Q and A
Appendix 1: reliability and validity of C-test
Dornyei and Katona (1992) found that the C-test is reliable
(the internal consistency coefficients are very consistent, .75
and .77 respectively, for university English majors and secondary
students) and valid (C-test is significantly highly correlated with
different other proficiency tests like the General Language
Proficiency and TOEIC). Cronbach’s alpha reached .84 in Daller
and Phelan (2006). Klein-Braley and Raatz (1984), Klein-Braley
(1985), Cohen, Segal and Bar-Siman-Tov (1984), Klei-Braley
(1997), and Grotjahn, 1995 generally supported such a claim on
written tasks. More importantly here, the C-test was reported to
be highly correlated with oral tasks as well in recent studies (e.g.,
r=.64 in Arras, Eckes and Grotjahn, 2002, and also in oral lexical
performance in Daller and Xue, 2007).
More recently Dai (this conference) reported in Chinese
context, Cronbach’s Alpha=.770, Concurrent validity r= .633,
p<0.01 (correlated with CET-4).
In this study, the Cronbach Alpha is=.64 in the pilot study, but
=.74 in the main study.
back
Appendix 2: description of dependent variables
General Category
Fluency
Variable Name
Description
Pausing
The number of pauses and the amount of silence.
In the present study it is operationalized as any
break of 0.4 second or longer.
Repair Fluency
This measure is orthogonal to breakdown fluency
and should be treated separtely. In the present
study it is operationalized as the total number
of repetitions, replacements, false starts and
reformulations.
Speech Rate
A pruned speech rate is investigated here because it
shows the ‘real’ speed of the speaker. It is
operationalized as the total words per minute
after deletion of reformulations, replacements,
false starts, repetitions, pauses and silence total.
Mean Length of Run
The number of words uttered before any
breakdown or repair fluency is encountered.
Phonation time
The ratio of voicing time to the total time of
utterance.
Accuracy
Complexity
Lexis
Error-free Ratio
The ratio of error-free clauses to all clauses.
Errors per 100 Words
The number of errors in every pruned one hundred
words.
Length Accuracy
The length of a clause with 50% of all clauses of the
same length correct is set as the cut-off point
beyond which the participant cannot produce
correct clause at 50% level.
Subordination Ratio
The ratio of subordinate clauses per AS unit.
Words Per AS Unit
The average word number in all AS units.
Words Per Clause
The Average word number in all clauses
Lexical diversity:
the D value
Corrected Type-token ratio, an index of the extent to
which the speaker avoid returning to the same
set of words.
Lexical sophistication: the The extent to which speech contains difficult or rare
Lambda value
words.
Lexical Density
The ratio of content words to the total words.
Formality
F-score
(Noun frequency+adjective freq.+preposition
freq.+article freq.-pron. Freq. –verb freq.adverb freq. – interjection freq. +100) / 2
From F. Heylighen and J. Dewaele (1999).
DB-score
The ‘involved’ style words in Biber, Conrad
and Reppen (1998).
back
References
Barry, S. & Lazarte, A. (1995). Embedded clause effects on recall: Does high prior knowledge of
content domain overcome syntactic complexity in students of Spanish? Modern Language
Journal, 79, 491–504.
Biber, D., Conrad, S. & Reppen, R. (1998). Corpus Linguistics: Investigating Language Structure
and Use. Cambridge: Cambridge University Press.
Bügel, K. & Buunk, B. (1996). Sex differences in foreign language text comprehension: The role of
interests and prior knowledge. Modern Language Journal, 80, 15–31.
Bygate, M. (1996). Effects of task repetition: appraising the developing language of learners. In Jane
Willis & Dave Willis (Eds). Challenge and change in language teaching. Oxford : Heinemann.
Bygate, M. 2001. Effects of task repetition on the structure and control of oral language [A]. In M.
Bygate, P. Skehan & M. Swain (eds). Researching Pedagogical Tasks: Second Language
Learning, Teaching and Testing [C]. Harlow, England: Longman.
Carrell, P. L. (1983). Three components of background knowledge in reading comprehension.
Language Learning, 33, 183-207.
Chang, C. (2006). Effects of topic familiarity and linguistic difficulty on the reading strategies and
mental representations of non-native readers of Chinese. Journal of Language and Learning, 4,
172-198.
Chen, Q. & Donin, J. (1997). Discourse processing of first and second language biology texts: Effects
of language proficiency and domain-specific knowledge. Modern Language Journal, 81, 209–
227.
Chern, C. (1993). Chinese students’ word-solving strategies in reading in English. In T. Huckin, M.
Haynes, & J. Coady (Eds). Second language reading and vocabulary learning. Pp. 67–82.
Westport, CT: Ablex.
Crookes, G. (1989). Planning and interlanguage variation. SSLA, 11, 367-383.
Ellis, R. (2005). Planning and task-based performance: Theory and research. In R. Ellis. (Ed).
Planning and task performance in a second language. Amsterdam: Johan Benjamins.
Ellis, R. (in press). The Differential Effects of Three Types of-Task Planning on the Fluency,
Complexity and Accuracy in L2 Oral Production. Applied Linguistics.
Hudson, T. (1982). The effects of induced schemata on the “short-circuit” in L2 reading: nondecoding factors in L2 reading performance. Language Learning, 32/1, 1-31.
Hammadou, J. (1991). Interrelationships among prior knowledge, inference, and language
proficiency in foreign language reading. The Modern Language Journal,75: 27-39.
Johnson, P. (1982). Effects on reading comprehension of language complexity and cultural
background of text. TESOL Quarterly, 16, 169–181.
Kawauchi, C. (2005). The effects of strategic planning on the oral narratives of learners with low and
high intermediate proficiency. In R. Ellis. (Ed). Planning and task performance in a second
language. Amsterdam: Johan Benjamins.
Lee, J. F. (1986). Background knowledge and L2 reading. Modern Language Journal, 70, 350–354.
Peretz, A., & Shoham, M. (1990). Testing reading comprehension in LSP: Does topic familiarity
affect assessed difficulty and actual performance? Reading in a Foreign Language, 7, 447–455.
Shimoda, T. A. (1993). The effects of interesting examples and topic familiarity on text
comprehension, attention, and reading speed. Journal of Experimental Education, 61, 93-103.
Schmidt-Rinehart, B. C. (1994). The effects of topic familiarity on second language listening
comprehension. The Modern Language Journal, 78, 179-189.
Skehan, P. & Foster, P. (1999). The influence of task structure and processing conditions in narrative
retellings. Language Learning, 49 (1): 93-120.
Tavakoli, P. & Skehan, P. (2005). Strategic planning, task structure, and performance testing. In R.
Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins.
Foster, P. & Skehan, P. (1996). The influence of planning and task type in second language
performance. SSLA, 18, 299-323.
Wigglesworth, G. (1997). An investigation of planning time and proficiency level on oral test
discourse. Language Testing, 14 (1): 85-106.
Yuan, F. & Ellis, R. (2003). The effects of pre-task and on-line planning on fluency, complexity and
accuracy in L2 monologic oral production. Applied Linguistics 24(1): 1–27.