CAMBRIDGE CPE 2013 1
An Analysis of Paired Speaking Tasks in
The Cambridge Certificate of Proficiency in English
Susan Mayer
The Monterey Institute of International Studies
MA TESOL Program
Dr. Kathi Bailey
4 October 2013
2
CAMBRIDGE CPE 2013
I was lucky enough to have a summer internship with a language program in the alpine
paradise of Garmisch, Germany. One of the seminars I observed was hosted by the Bureau for
International Language Coordination (BILC), an organization tasked to work with NATO and
NATO-partner countries to organize English language teaching and testing around a common set
of standards, called the STANNAG 6001 (BILC 2013). My role was to observe the instructors
and learn about this newly-developed seminar “Language Standards and Assessment Seminar”.
Worried that it might be tedious, I was instead pleased to discover the complexity of the task of
language testing, as the two weeks flew by. Thirteen people represented thirteen countries,
spending their time learning strategies to undertake this standardization process when they
returned to their roles as leaders in their country’s English program. Often during this seminar, I
heard the term ‘CPE’ used, with a reverent tone, suggesting it might be the gold standard in the
testing industry. Since my dream job is to work with language learners at the diplomat level, I
realized that the Cambridge Certificate of Proficiency in English (CPE) is the type of test they
are likely to take in order to demonstrate their professional-level English proficiency.
In order to provide context for the reader, I will begin with a brief history of the test, and
move into a description of the individual parts of the test, looking more closely at the paired oral
test format. Finally, I will provide an analysis of the test, using frameworks developed by
Wesche (1983) and Swain (1984).
History of the CPE
As the first of Cambridge University’s tests of English for non-native speakers, the
Certificate of Proficiency in English (CPE) is comprised of a series of papers, each of which tests
discrete skills. In 2013, the CPE passed the 100-year mark of its existence. In the hundred years
CAMBRIDGE CPE 2013 3
since it was first given to three candidates, the test has gone through several major revisions on
the way to the most recent (2013 UCLES) version. On the occasion of the launch of the 2002
revisions, Weir (2002) published a summary of how the CPE has been transformed over time1 ,
connecting the rationale for each new revision to the state of language teaching and testing at the
time.
According to O’Sullivan (2011), the early CPE was based upon the premise of using the
language, surprisingly parallel to its current permutation. However, we do see a link to the
behaviorist/cognitive paradigm (Myles p. 320), dominant at the time of the early CPE tests, in
the required tasks. In fact we see little change in the test suite during the years between 1913 and
1945, when language learning was organized around language as a cognitive process, and
teaching focused on translations (Myles p. 320) and memorization of vocabulary. Early CPE
‘papers’ as the test sections are called (UCLES 2013) contained British literary or cultural
references, allowing test-takers options to choose from, among those presented. This remained
as the format of the CPE until 1975, when it was recognized that a growing number of candidates
came from countries outside the United Kingdom (Weir 2002) and might not have the
sociocultural frame necessary to respond appropriately2.
According to Weir (2002), the 1975 revision to the CPE represented a turning point in the
Cambridge testing family, reflecting a growing trend in language teaching toward ‘language in
use’ (Weir p. 5) which favored communicative language teaching and the use of relevant
materials in both teaching and testing. Further revisions have resulted in the CPE being offered
both on paper at one of the thousands of testing centers worldwide, and online for the first time
1
See APPENDIX A, Changes in Cambridge CPE Over Time (adapted from Weir 2002)
2
See APPENDIX A, Changes in Cambridge CPE Over Time (adapted from Weir 2002)
4
CAMBRIDGE CPE 2013
in March of 2013. Currently the test is administered to approximately 45,000 people in 130
countries (UCLES 2013)3.
Description of the Test
General Overview
According to Stoynoff & Chapelle (2005, ch. 3), the guidelines for a large-scale test
should provide the best source of information about the test, and in fact “test practices are
enhanced when developers and users embrace the same professional standards.” To that end,
much of the information gained in learning about the test comes from the materials provided by
the University of Cambridge Local Examinations Syndicate (UCLES) on their website
[www.cambridgeenglish.org] or written materials available to the public. The website provided
a wealth of information beyond what a test-taker needs to know about this test; the site also has
extensive information about the background of test-writing, statistical information about scoring,
evidence of reliability, and links to hundreds of researched journal articles which touch upon
every aspect of the test. It would be impossible to include links to each relevant article or web
page within the site for this summary, but the reader is invited and encouraged to explore further.
The Cambridge Certificate of Proficiency in English, or CPE, is a criterion-referenced
test which assesses candidates’ English abilities based on all of the four skills; reading, writing,
speaking and listening, using ‘realistic tasks and situations’ (UCLES 2013). It represents
UCLES’ highest level test, rating candidates based on level C2 on the Common European
Framework of Reference (CEFR)4. At this level, the learner is approaching the “linguistic
3
4
Please see APPENDIX for grade statistics for each country/territory (UCLES 2010).
For more information about the Common European Framework, please see the ALTE website.
CAMBRIDGE CPE 2013 5
competence of an educated native speaker, and is able to use the language in a range of culturally
appropriate ways” (UCLES 2013). Success on the CPE is an indicator that the person is capable
of undertaking graduate-level work in English at a university setting. Employers also use the
CPE as a qualifying standard for employment if English use at a near-native level is a
requirement of the position. According to Leung & Lewkowicz (2006), the job of a test is to
‘encourage the display of the target qualities through some kind of performance’ (p. 220). The
information provided to the English learner on the CPE website explicitly states their
commitment to excellent testing overall, through constant evaluation of test materials and
oversight of the marking and grading of each section (UCLES 2013). Information about the cost
of the CPE on the UCLES site is elusive; however O’Sullivan (2011) alludes to the pricing
system in his videotaped lecture, which has a baseline price plus ‘local costs’, which can vary
widely. Without signing up for the test on the UCLES’ website, it is sufficient to say that for
most Western European countries, the cost will be approximately £150 (O’Sullivan 2011). One
might guess that for countries where the economy does not align with Western economic
standards, the local costs (O’Sullivan 2011) are likely to be much lower than their more affluent
counterparts.
Test Information
Reading & Use of English. Candidates will read and demonstrate their understanding of
meaning from a variety of sources, both fiction and non-fiction; such as novels, journals,
newspapers, magazines (UCLES 2013). This test is divided into sections, each of which matches
a reading passage with questions in various formats, including Cloze, modified Cloze, multiple
choice, Gapped text, and matching. Skills assessed in this section include idioms, collocations,
complementation, phrasal verbs, grammar awareness and control, affixation, understanding of
6
CAMBRIDGE CPE 2013
text structure, cohesion, global meanings, opinion vs. fact, attitude and detail. Sample questions
with answer key can be found in APPENDIX B.
Writing. This section assesses the candidates’ skills in writing a variety of texts, such as
articles, letters, and essays. Skills include summarizing, evaluating, synthesizing, and
reformulating ideas in their own words to form a cohesive argument. Responses are generally
between 250 and 500 words (UCLES 2013). See APPENDIX B.
Listening. To demonstrate listening skills, candidates are given four separate listening
tasks and asked to listen for things such as gist, detail, function, attitude, opinion, topic, specific
information, inference, and interpret context or discover the main idea. The listening passages
comes with its own performative task. In the Sample Test for 2013 [See APPENDIX C] we can
see that the candidate will be asked to answer multiple choice questions (parts 1 and 3), fill in the
blanks (part 2), and match (part 4).
The Paired Speaking Task. Unique in the English language proficiency testing world,
the Cambridge family of exams pairs two candidates together for the speaking portion of the
exam. Two candidates are in the room with two examiners. One of the examiner will be seated
at a table opposite the two candidates, acting as the interlocutor, while the other examiner is
seated at a table slightly removed, using a checklist (O’Sullivan) to tabulate scoring. The 16minute test begins with a 2-minute individual interview, in which the candidate is asked to give
information about him/herself, and express an opinion about a topic introduced by the examiner.
Part 2 is a 4-minute task where the examiner asks both candidates to look at a set of photographs
[see APPENDIX D], first answering a question, and then interacting with the other candidate in a
decision-making task (UCLES 2013). In the final section, the interlocutor will present a card
CAMBRIDGE CPE 2013 7
with a question and some ideas. Each candidate will speak for two minutes individually, after
which the interlocutor will ask both candidates a question, opening a discussion among the two
candidates.
Scoring System
Each section of the CPE is scored independently and represents 20% of the total score,
with the Reading & Use of English sections combining to account for 40% of the total. For each
Paper, the grade boundaries for A, B, C, D, and E, are set based upon statistics for the overall
candidature, the performance of the individual candidates, recommendations by the Examiners
and comparison with previous years’ scores (UCLES 1995:14). Grades of A, B, and C are
considered passing grades, and a candidate’s overall score is based upon the aggregate of the test
papers. Once passed, the CPE award does not expire5.
Analysis
The CPE can be analyzed on two levels. First, taking the test as a whole, we must
consider Wesche’s Framework, which asks us to consider four things in regard to a test:
Stimulus, Task, Response, and Scoring Criteria.
Figure 1. Wesche’s Structural Analysis Framework (1983)
Stimulus
Task
Response
Scoring Criteria
CPE Test
Component
Read
texts about
Three short
things in the real world.
Reading
& interesting
Use
of
English
Read general,
Use of
texts
academic, or
English
Three longer
work-related
reading texts
texts.
5
Cloze or C-test
Form a new word
Re-writing
sentences
Multiple Choice
Please see APPENDIX E for breakdown of scoring by country
Time: 1.5 hours
40% of total score
8
CAMBRIDGE CPE 2013
Gapped
Paragraphs
Multiple
Matching
Writing
Two short
texts
Summarize main
idea from texts
Choose task
from possible
options
Write an article,
report or review
Listening
Four parts.
Short extracts
Long
discussion
Listen to workrelated,
university, or onthe-street
discourse
Multiple choice
Matching Tests
Interpret
Language
Time: 1.5 hours
20% of total score
Scoring is based upon
four criteria:
Content – how well the
candidate has done what
was asked of him/her
Communicative
Achievement – is the
writing appropriate for
the task in terms of
register
Organization – do the
ideas flow logically and
orderly
Language – is the
vocabulary and grammar
appropriate to this task
Time: 40 minutes
20% of total score
As seen in Figure 1, each skill is based upon real-world tasks that are engaging and relevant to
the test-taker. According to UCLES 2013, the best preparation strategy for success on the CPE
is to use English in a variety of ways, every day.
Another way of looking at the test as a whole is to view it using Swain’s framework
(1984). Swain argued that test development should follow four principles: Start from
somewhere, Concentrate on content, Bias for best, and Work for Washback. Bailey (1998, p.
154-5) provides context for these frames. Of particular interest and value is the concept of
washback. Bailey tells us about the meaning of washback, which can be either positive or
negative, depending on the effect the test has on teaching methods. UCLES addresses washback
explicitly, acknowledging that test design should align with teaching goals, for positive test
CAMBRIDGE CPE 2013 9
washback, The less preferred outcome, that of negative washback, is seen when teaching is
adjusted in order to ‘teach to the test’. In language learning, this is especially undesirable if we
are seeking to encourage language usage in real-life circumstances. This is again evidenced by
the extensive research on washback reported on the UCLES 2013 website under the heading of
Research Notes.
Figure 2. Swain’s Theoretical Analysis Framework
CPE Test
Component
Reading &
Use of
English
Writing
Listening
Speaking
Start from
Somewhere
Concentrate
on Content
Preparing for
the reading
portion of the
test can
enhance
learners’ reallife reading
skills.
CPE seeks to
create tasks that
the learner
would use in
normal life, at
work, or at
school.
Learners
enhance this by
engaging in
everyday
conversation,
listening to the
radio, watching
tv
CPE seeks to
evaluate coconstructed
meaning
through the use
of interesting
materials.
Read texts about
interesting things
in the real world.
Read general,
academic, or
work-related
texts.
Summarize ideas
from real-world
materials
Write essays on
current, relevant
topics
Listen to workrelated,
university, or onthe-street
discourse, radio
broadcasts.
Learners give
information about
themselves.
Express opinions,
exchange ideas,
making
suggestion and
speculations,
evaluating,
negotiating
outcomes.
Bias for Best
Work for
Washback
CPE designers and
researchers are
engaged in the
ongoing process of
production and
pretesting of
question papers.
Each test must
meet the
internationally
recognized ISO
9001:2008
standard for
quality, supporting
five essential
principles:
Validity,
Reliability, Impact,
Practicality and
Quality. (UCLES
2013).
Firmly connected to
authenticity, the tasks
in each section of the
CPE are extracted from
real-life sources which
encourage positive
learning experiences
and a positive impact
on teaching as well.
(UCLES 2013).
See above
See above
10 CAMBRIDGE CPE 2013
In a high-stakes test such as the CPE and its sister tests, Cambridge re-iterates UCLES’ (2013)
firm belief in remaining current by using relevant, real-life situations and materials at every test
level and for every skill. O’Sullivan (2011) alludes to what he terms ‘the Atlantic divide’
between testing philosophies in Great Britain and the United States. Since relevance has
explicitly remained an ongoing guiding principle in the UCLES products over time6, the shift to
more authentic testing materials by testing companies in the United States is viewed by some as
catching up to what the Cambridge Tests have done all along.
The final point of analysis of the CPE is based upon the unique use of paired speaking
tests. Critics of this strategy point to issues that need to be considered. Macqueen & Harding
(2009 p. 470) argue that by introducing the second candidate into the testing session, the door is
opened to a wide range of factors which ‘may pose threats’ to the test’s validity. Among these
were listed age, social class and personality, as well as the potential effect if the co-candidates
happen to be acquaintances. Macqueen & Harding point us to the test’s own website (UCLES
2013) for further information on this issue.
To that end, we find several revealing studies among Cambridge’s Research Notes. One
is an extensive analysis undertaken by O’Sullivan et. al., (2002), who conclude that in regard to
potential skewing of scores among two interlocutors in the Speaking portion of the CPE, the role
of the examiner is to ensure that each person is judged on their own merit (O’Sullivan et. al.,
2002). UCLES is explicit in providing information about examiner training, allowing public
access to the examiner’s checklist7 which these authors developed to address potential scoring
6
7
See APPENDIX A, Changes in Cambridge CPE Over Time (adapted from Weir 2002)
Please see APPENDIX F – Validating Speaking Tasks
CAMBRIDGE CPE 2013 1
1
inconsistencies.
All of the Cambridge research points to the same conclusion on the point of testing
speaking in pairs: conversation in life is always linguistically co-constructed. None of the
research demonstrated significant difference between groups of people who tested individually
versus those tested in pairs (Nakatsuhara 2006, p. 20) using 24 undergraduate participants at
Essex University, England, to prove or disprove the hypothesis that there is in fact a difference
between the two groups. They constructed careful procedures and compared results using
ANOVA. The data did not reveal statistically significant differences among the groups tested. In
addition, as Macqueen & Harding (2009, p. 471) remind us, speaking in conversation requires de
facto listening skills in the co-construction of the responses elicited during this part of the
examination. To prepare themselves for success in this part of the CPE, candidates must spend
time on both speaking and listening skills, which we may recall is the kind of positive washback
Swain (1984) would have preferred.
Further Study
In order to grasp the depth and breadth of planning that has gone into the CPE over the
past 100 years, one would want to study further the volumes of Research Notes available on the
UCLES website. From the earliest version (1913) of the Certificate of Proficiency in English
test until the present time, there is strong evidence of a continual commitment to creating an
instrument which measures the linguistic competence of the candidate against criterion which are
internationally accepted and codified in the Common European Framework of Reference (CEFR
2011). A comparison of the evolution of testing in the United States, alluded to by O’Sullivan
(2001) in his lecture on the past and future of testing, would also prove useful, in order to gain a
12 CAMBRIDGE CPE 2013
greater understanding of the implications of these various tests in terms of work and educational
opportunities that may be made available, or not, as a result of a score or achievement level from
these tests. Additionally, it might be of value to conduct research to determine if there is validity
to O’Sullivan’s (2011) suggestion that the Atlantic Divide has significance in the matter of
washback upon language teaching and learning. If we are to accept that the Cambridge tests are
carefully constructed to reveal language in use, then it would be hoped that norm-referenced tests
used to make educational and employment decisions would yield similar results.