The CEFR for teaching and
assessing young learners
LTF, 21 Nov 2009
CRELLA, University of Bedfordshire, Luton
Szilvia Papp, Hanan Khalifa, Nick Charge
University of Cambridge ESOL Examinations
Aim of Talk
2
Questions
1. the construct behind YLE
a) linguistic progression
b) cognitive development
c) educational, social, psychological aspects
2.
3.
4.
5.
3
the link between YLE and the CEFR
CEFR proficiency levels reflected in YLE
salient features of YLE performance in CEFR
additional salient features of YLE performance not
in CEFR
Tests for YLLs need to be
supporting but challenging enough to cater for all
cognitive and emotional needs on both ends of the
spectrum
build on and reflect previous experience of language
learning
test language use rather than language knowledge
provide children with
a range of social situations and
sufficient breadth of experience with language
cover a wide enough sample of language
4
(Simon Smith 2009, University of York)
Question 1
What is the construct behind Young Learners of
English tests?
a) What linguistic progression do the YLE tests
reflect?
b) How do the tasks reflect the cognitive
development of YLE candidature?
c) What educational, social, psychological
aspects of young language learners are catered for
in YLE tests?
5
(Papp 2007, 2008a)
Linguistic progression
Limitation to lexical, structural, pragmatic
6
development due to age
Similar route, but different mechanisms, rate and
ultimate attainment among child L2 learners,
depending on age of onset of learning
Communication strategies
Language learning strategies
Basic interpersonal communication skills (BICS)
Cognitive academic language proficiency (CALP)
(Cummins 2001)
Linguistic progression
Differences in language development based on age
can manifest themselves in variations in:
7
accuracy, fluency, complexity
use of formulaic sequences
interactional patterns and interactional moves
accent
content, topic, range of notions that the candidate is able to,
wants to or needs to encode
different goals
the presence of specific grammatical features in the ILG
(Nicholas and Lightbown 2008, Philp, Oliver and Mackey 2008)
8
Cognitive development
Strategies for working memory use
Abstract concept formation
Reasoning abilities
Metalinguistic awareness
Executive control
Mind reading abilities
9
(Meadows 2006)
10
Educational, social, psychological
variables
Amount & quality of input, exposure to, experience
11
with L2
Social, cultural and educational context of interaction
Development of literacy in L1 (different writing
systems)
Theory of mind (use of complex discourse &
narrative skills)
Individual differences
Scaffolding
(Cameron 2003)
Task demands and scaffolding
12
Summary
Age-related differences between young learners which
have an impact on their L2 development:
13
improved cognitive abilities related to growing attentional and
memory capacities
changing social and psychological make-up, such as
fluctuating self-confidence, self-generated and peerinfluenced interests, motivation, related affective schemata
unique experiential characteristics, such as differential
exposure to the target language, differential acceptance and
participation in activities and task types, etc.
(McKay 2006)
Question 2
What is the demonstrable link between YLE test and
task specifications, YLE candidate performance and
the relevant CEFR scales and descriptors in terms of
a) proficiency levels
b) activities, strategies and competences YLE candidates
can be expected to do or have?
14
Link between YLE and CEFR
YLE linked to CEFR by design through test
and task specifications (1992-1997 test
development, 2002-2007 revision):
Flyers at A2 (KET)
Movers at A1
Starters below A1
The link has been demonstrated empirically
and qualitatively:
equivalence and equating studies
formal alignment process in May 2008
next equivalence study in March 2010
15
(Gardiner 2000, Flux 2001, Jones 2001, Jones 2002a,b,c,
Burrow 2004, Barker and Shaw 2007, Papp and Salamoura 2009)
Logits
Flyers
Listening
shields
KET grade
1.65
5
Merit
0.77
4
0.22
3
Pass
2.09
5
Merit
1.32
4
0.55
0.00
-1.32
-1.65
Narrow
Fail
-1.87
Fail
2
-0.11
-0.33
-0.44
-0.77
-2.53
-0.88
-2.75
-1.10
-2.97
-1.21
-3.19
-1.32
1
-3.74
(Gardiner 2000, Flux 2001)
3
Pass
-0.66
-2.31
-3.41
16
KET
grade
0.33
-0.99
-2.09
Flyers
Reading
& Writing
Shields
0.88
-0.33
-0.66
Logits
-1.43
-1.54
Narrow
Fail
Logits
Flyers
Listening
shields
Movers
Listening
shields
3.96
4
5
3.19
3
4
5
4.51
4
4.07
3.19
1.54
2.97
1.32
2
3
2.86
3
2.64
0.88
2.53
0.66
2.42
0.44
2.31
0.33
2.20
0.11
1
2
-0.33
-0.55
-0.77
-0.99
-1.21
2.09
1.98
1.87
1.76
1.65
1.54
1.43
17
5
3.41
1.76
-0.11
5.16
Movers
Reading &
Writing shields
3.74
2.09
1.10
Flyers
Reading &
Writing shields
4.18
2.75
2.31
Logits
(Gardiner 2000, Flux 2001)
1.32
2
4
Activities, strategies and competences
of YLE candidates
The relationship between Starters and Movers and
Breakthrough level investigated by a consultant in
2003.
In this project, the lexical content and topic areas of YLE tests
were mapped against the preliminary Breakthrough
objectives.
Results:
substantial areas of overlap between Starters and Movers &
Breakthrough in terms of lexis and topic areas,
however, YLE topic areas are expressed in more concrete
terms such as School & the Classroom rather than Education.
18
(Burrow 2004)
Verifying YLE alignment to the CEFR
Formal alignment/linking exercise May 2008: CEFR ratings for levels& skills
Skill
YLE Starters
YLE Movers
YLE Flyers
Speaking
1.06
1.85
2.94
Listening
2.17
2.46
3.13
Reading
1.79
2.63
3.10
Average
(all skills)
CEFR level
1.67
2.31
3.06
Pre-A1/A1
A1+
A2
19
(Papp and Salamoura 2009)
Question 3
How do the YLE tests embody and reflect the CEFR?
20
Describing YLE exams using CEFR?
21
trade-off between control of candidate output and
authenticity of tasks, not reflected in CEFR
scaffolding of tasks absent in CEFR
learning to learn, enabling skills very sparse in CEFR
limits to children’s cognition, linguistic progress,
psychological, emotional, social development not
acknowledged in CEFR:
no task should tax children’s cognition by requiring them to
deal with multiple perspectives other than their own
effect of writing on the reader should be taken into account
in marking criteria
(Papp and Salamoura 2009)
Question 4
What are the salient features of YLE candidate
performance that reflect the CEFR in terms of
a) proficiency levels
b) activities, strategies and competences YLE candidates
can be expected to do or have?
22
Salient features of YLE candidates’
performance: listening and speaking
Starters (pre-A1) candidates’ performance:
• Ability to rely purely on a very finite, rehearsed repertoire of phrases, frequently used routines and patterns
limited to the performance of isolated tasks in specific situations, i.e. a list of pedagogic tasks in a primary
school setting.
Movers (A1) candidates’ performance:
• Ability to interact in a simple way, initiate and respond to simple statements in areas of immediate need or on
very familiar topics (ask and answer simple questions about themselves, where they live, people they
know, and things they have, etc.).
Flyers (A2) candidates’ performance:
• Ability to handle social functions (greet people, ask how they are and react to news; handle very short social
exchanges; ask and answer questions about what they do at school and in free time; make and respond to
invitations; discuss what to do, where to go and make arrangements to meet; make and accept offers).
• Ability to perform simple classroom tasks in English.
Strong Flyers (A2+) candidates’ performance:
• Ability to more actively participate in conversation.
• Ability to sustain monologues.
23
(CEFR 2001)
Question 5
Which additional salient features can be identified in
YLE candidate performances that are not currently
covered in the CEFR?
24
Children’s literacy: reading and writing
There is a need to
list children’s L2 reading abilities, activities, strategies and
competences
investigate emerging L2 writing abilities (e.g. copying,
handwriting, spelling and other enabling skills)
list children’s specific L2 writing activities, strategies and
competences
in order to identify salient features of typical/likely candidate
performance at each level of YLE not currently in CEFR
This requires additional work
on sample performances from YLE reading & writing tests
with children in the classroom (ECML)
review of materials used in classrooms across the world
25
Recommendations for future research
For Cambridge ESOL
further explore salient features of YLE performance, esp for
children’s reading and writing typical in & outside classroom
develop Can Do statements for YLE candidates (aged 7–12)
to complement those for 11–14 year-old school learners
taking KET/PET for Schools examinations
further work on link between summative and formative
assessment (assessment for learning)
For other YLL test providers
carry out a similar linking exercise in order to
define and explicitly state construct tested
generate evidence of CEFR relatedness
identify qualities of YLL tests claiming links to the CEFR
raise awareness of good testing practices
26
Recommendations for future research
For all YLL test providers
provide feedback to CoE on linking process for relating
exams such as the YLE tests to the framework of reference
provide calibrated samples to CoE of children’s performance
in tests thus related to the CEFR
answer the question that remains whether it is necessary to
carry out a whole-scale adaptation of current CEFR scales to reflect
27
young learner needs and contexts or
start developing scales from scratch to reflect the nature of second
language learning and use among children?
Selected references
Barker, Fiona and Stuart Shaw (2006) YLE Vertical Linking Project. Validation Project No 171. Cambridge ESOL internal
document.
Barker, Fiona and Stuart Shaw (2007) Linking language assessments for younger learners across proficiency levels (Phase 1).
Research Notes 28: 14-18. UCLES: Cambridge ESOL.
Burrow, Trish (2004) Exploring the relationship between YLE Starters and Movers and Breakthrough level. Research Notes 16:
6-7. UCLES: Cambridge ESOL.
Gardiner, Kerri (2000) KET / Flyers Research Project: Analysis of data. Research and Validation Report No 106/806. Cambridge
ESOL internal document.
Flux, Tracy (2001) KET / YLE Link Project 2001. Research and Validation Report No 246. Cambridge ESOL internal document.
Jones, Neil (2001) Reliability of YLE: Dependability of grades for a high-facility exam. Research and Validation Report No 316.
Cambridge ESOL internal document.
Jones, Neil (2002a) Equating YLE levels: A study based on candidates taking two exams. Research and Validation Report No
318. Cambridge ESOL internal document.
Jones, Neil (2002b) Linking YLE levels into a single framework. Research Notes 10: 14-15. UCLES: Cambridge ESOL.
Jones, Neil (2002c) A Framework for YLE: Proposals for an integrated grade system and implications for calibration and grading.
Research and EFL Validation Report No 439. Cambridge ESOL internal document.
Papp, Szilvia (2007) The Cambridge YLE tests in the light of cognitive, linguistic, educational, social-psychological and cultural
aspects of children’s L2 development and the CEFR, internal Cambridge ESOL Research and Validation report.
Papp, Szilvia (2008a) Factors influencing L2 development and use in the 8–14 age group – towards defining the construct,
Cambridge ESOL internal Research and Validation report.
Papp, Szilvia (2008b) Skills profiles by age in KET and PET, Cambridge ESOL internal Research and Validation report.
Papp, Szilvia (2008c) Coverage of Waystage and Threshold learning objectives within KET and PET and KET and PET for
Schools, Cambridge ESOL internal Research and Validation report.
Papp, Szilvia (2008) Quantitative linking YLE to the CEFR – summary of empirical studies to date, internal Cambridge ESOL
Research and Validation report.
Papp, Szilvia (2009) Development of Can-do statements for KET and PET for Schools, Research Notes 36, 8–12, Cambridge:
Cambridge ESOL.
Papp, Szilvia and Salamoura, Angeliki (2009) An exploratory study linking young learners examinations to the CEFR, Research
Notes 37, 15–22,Cambridge: Cambridge ESOL.
Trim, J L M (2001b) Breakthrough, unpublished manuscript, now available on the English Profile website.
van Ek, J and Trim, J L M (1990a/1998a) Threshold 1990, Cambridge: Cambridge University Press.
28Ek, J and Trim, J L M (1990b/1998b) Waystage 1990, Cambridge: Cambridge University Press.
van