Examinations of Tasks Used to Measure Performance
Tsutomu KOGA (12/15/2005)
1.
Introduction
: performance may differ from task type to task type due to inherent characteristics involved in each task
: if the above is the case, learners’ attitudes/perceptions towards the task would also vary
: performance elicited by only a limited number of tasks is sufficient to make generalizations
2.
Purpose
: to examine whether tasks can measure what we intend to measure
: to examine to what extent performance measured is properly related to another construct
3.
Previous Studies
 Tremblay and Gardner (1995)
: they employed two achievement measures: French essay and French course grade
 five criteria for French essay (grammar, vocabulary, sentence structure, content, and
originality)
 course grade consists of grammar and comprehension test, essay, and presentation
: correlational and causal path: motivation→proficiency: .35 (standardized coefficient)
: no correlation between achievement measures and motivational behaviour (attention, effort, persistence)
 Tremblay, Goldberg, and Gardner (1995)
: a paired associates Hebrew/English vocabulary-learning task
: trait motivation (general) had a direct influence on state motivation (specific)
: state motivation influenced task performance.
 Gardner, Tremblay, Masgoret (1997)
: five tools are used to measure French achievement
(1)
(2)
(3)
(4)
(5)
French achievement test: knowledge of French verbs, adjectives, pronouns, and prepositions
Cloze test: knowledge of grammar, vocabulary, and inferencing skills
Thing category test: vocabulary knowledge and production (e.g., list fruits)
Theme test: essay (grammar, vocabulary, sentence structure, length, use of idiomatic French)
Grades in French: no account for this measure
: : correlational and causal coefficient: motivation→proficiency: .48
1
 Gardner and Tremblay (1998)
: a paired associates learning task
: state motivation functioned as a mediator between trait motivation and achievement
 Koizumi (2002)
: two measurement tools were used: a STEP test and a speaking test
: STEP (MC) consists of vocabulary/grammar (30%), reading (20%), writing (10%), and listening (40%)
: the speaking test is an interview test consisting of a role play and a question and answer type, but only
the former was used as a proficiency measure (four criteria: vocabulary, comprehensibility, complexity,
and interaction)
: correlational and causal coefficient: motivation→proficiency: .49
 Yashima (2002)
: using TOEFL (listening, grammar/vocabulary, and reading), she found the positive relationship
between motivation and L2 proficiency (correlational and causal path: motivation→proficiency: .41)
: closely looking at the correlation matrix, the following correlations are observed
(a) listening and motivational intensity (effort): .18**
(b) listening and desire to learn L2: .12*
(c) grammar/vocabulary and motivational intensity: .32**
(d) grammar/vocabulary and desire to learn L2: .28**
(e) reading and motivational intensity: .24*
(f) reading and desire to learn L2: .29*
: these correlation coefficients, though significant, are not very high
4.
Questions Raised in the Previous Studies
: noticeable that some studies use multiple tasks to analyze motivation and performance, whereas others
use only a limited type of tasks
: all of the studies introduced here provided the consistent results: motivation affects achievement to
some extent
: these results are, on the one hand, understandable, because of the fact that motivation plays a crucial
role in learning, but on the other hand, curious given the nature of tasks employed, which is very
uninspiring
: all of the studies, in common, administered tasks measuring learners’ grammatical competence and
vocabulary knowledge
: the first question is “Are such tasks unmotivating by nature??” (multiple choice, cloze, etc)
: if so, how can we expect a positive relationship (high correlation) between motivation and
performance?
: is demonstrating one’s knowledge a motivated behaviour?
2
: if this is not true, in what kinds of tasks does a level of motivation more properly reflect?
: if some other tasks with different characteristics had been employed, would the results have been
different?
: if so, what kinds of tasks should be used to examine relationships between motivation and performance
5.
Tasks as a Motivating Factor
: satisfying learner needs through need analysis (Lee, 1995)
: implementing various tasks that appeal to each student with different goals
6.
Some Considerations of Performance/Achievement/Proficiency Measures
: On what ground can we decide tasks to measure what we intend to measure? (e.g., TOEFL is often used
to measure learners’ overall proficiency in English; however, is it valid enough since it is widely used
and the validity research and the official rubric suggest it is?)
: What attention should we pay to such measures especially when we compare one performance with
another?
7. References
Gardner, R. C. & Tremblay, P. F. (1998). Specificity of affective variables and the trait/state
conceptualization of motivation in second language acquisition. In R. K. Agnihotri, A. L. Khanna,
& I. Sachdev (Eds.), Social psychological perspectives on second language learning. New Delhi:
Sage Publications.
Gardner, R.C., Tremblay, P.F., & Anne-Marie Masgoret. (1997). Towards a full model of second
language learning: An empirical investigation. Modern Language Journal, 81, 344-362.
Koizumi, R. (2002). The effects of motivation, language anxiety, and test anxiety on English proficiency
of Japanese junior high school students. Japan Language Testing Association Journal, 5, 91-110.
Lee, W. Y. (1995). Authenticity revisited: text authenticity and learner authenticity. ELT Journal, 49,
323-28.
Tremblay, P. F. & Gardner, R. C. (1995). Expanding the motivational construct in language learning.
Modern Language Learning, 79, 505-518.
Tremblay, P. F., Goldberg, M. P., & Gardner, R. C. (1995). Trait and state motivation and the
acquisition of Hebrew vocabulary. Canadian Journal of Behavioural Science, 27, 356-371.
Yashima, T. (2002). Willingness to communicate in a second language: The Japanese EFL context.
Modern Language Journal, 86, 54-66.
3