Language awareness in assessment and teaching: Cambridge bilingual
research and support
Stuart Shaw and Helen Imam, Cambridge International Examinations
A paper presented at the 38th Annual Conference of the International Association for
Educational Assessment, 16-21 September 2012, National Testing Centre of Ministry of
Education and Science of the Republic of Kazakhstan
Abstract
Cambridge International Examinations (CIE) is attempting to build an understanding of
best practice in the area of bilingual education to guide future developments in language
awareness and bilingual assessment which can be shared more widely across the assessment
community. International schools prepare students for CIE assessments in a range of
subjects through the medium of English, some as part of a bilingual programme where
certain subjects are taught through the first language. These schools represent diverse
linguistic and cultural backgrounds. This places an additional responsibility on examiners
and on subject teachers to be ‘language aware’. CIE has conducted a number of reviews of
the bilingual education literature. The key messages emerging from the reviews are that
bilingual education is challenging, complex and changing. CIE is therefore conducting a
number of studies designed to address these three issues: 1) research into language
awareness, leading to the provision of language guidance for CIE question writers, as well
as language guidance to support teachers and learners in the classroom; 2) research into
bilingual education, leading to a publication for schools focusing on effective school
management and classroom practice; 3) Research into developing bilingual assessments,
exploring current thinking in order to consider future challenges and opportunities.
Key words: Bilingual Education, Cambridge, Bilingual Assessment, Language Awareness
Contact details:
Cambridge International Examinations, 1 Hills Road, Cambridge, CB1 2EU, England.
Stuart Shaw. Tel. +44 (0)1223 556089. Email: shaw.s@cie.org.uk
Helen Imam. Tel. +44 (0)1223 553169. Email: imam.h@cie.org.uk
1
Bilingualism and bilingual education
Bilingualism is at least as common as monolingualism. Throughout the voluminous research
literature, academic definitions of the term ‘bilingual’ abound, from the early, limited and
very narrow definitions, ‘native-like control of two languages’ (Bloomfield, 1935, p.56) to
more flexible contemporary descriptions, ‘the presence of two or more languages’ (Dewaele,
Housen & Li, 2003, p.1), which reflect the awareness of the interdisciplinary nature of studies
in bilingualism. Seen simply, the current view of bilingualism is the ability or need to perform
in two or more languages. Therefore, ‘bilingualism’ can also denote several languages, along
with terms such as’ trilingualism’, ‘multilingualism’, ‘polylingualism’ and ‘plurilingualism’.
Bilingual education –‘a simplistic label for a complex phenomenon’ (Baker, 2006, p.213) –
has been widely researched since the mid-1960s, and experts throughout the world have
attempted to define and analyse the complexity and effectiveness of different bilingual
education programmes. Cummins (2009) offers a helpful conceptualisation of bilingual
education: ‘an organized and planned program that uses two (or more) languages of
instruction. The central defining feature of bilingual programs is that the languages are used
to teach subject matter content rather than just the languages themselves.’ (2009, p.161).
Interpretation of the research on bilingualism and bilingual education has been highly
controversial among both academics and policy-makers, and political sensitivities
surrounding the issue have contributed to considerable confusion about what the research is
actually suggesting. However, over the last decade as knowledge of the extent of bilingualism
has grown, discussions of bilingualism have focused on ‘the many kinds and degrees of
bilingualism and bilingual situations’ (Crystal, 2003, p.51), leading to in-depth descriptions of
the varied and disparate circumstances involved in bilingualism, anticipating the call for
understanding the bilingual situation through its purpose and its context (Edwards, 2004).
The educational context
The contexts within which students prepare for Cambridge International Examinations (CIE)
assessments are often linguistically and educationally diverse. Whatever the country, the
common denominator of CIE schools is that students are taking CIE assessments through the
medium of English and therefore being educated through the medium of English. Varying
levels of English amongst both students and teachers can increase the challenge that teachers
face in supporting students with the language they need to master content. Some schools use
bilingual instruction, delivering certain subjects through English as an additional language
and other subjects through the first language, often trying to meet standards in both an
international curriculum and a national curriculum. The opportunity to learn an additional
language through a content subject has led to the practice of content and language integrated
learning (CLIL) programmes. Other schools use monolingual instruction and deliver all
subjects through English, either as a first or as an additional language. Some of these schools
will have monocultural student populations, whilst others will have multicultural populations.
The latter places an additional responsibility on subject teachers to be ‘language aware’ and
sensitive to the needs of students from different linguistic backgrounds.
Such language challenges are increased in a trilingual context such as Kazakhstan, where CIE
is supporting Nazarbayev Intellectual Schools in developing its own trilingual education
programme. Learning content subjects through Russian, Kazakh and English, for example,
provides rich opportunities for language development, but also places additional
responsibilities on teachers to support both content and language learning, as well as to
provide equal access to learning in a multilingual population. On the other hand, it could be
argued that even first language teachers, as part of good pedagogy, should be making the
academic language of their subjects visible and supporting students linguistically.
As a consequence, CIE is keen to understand such multilingual contexts in order to evaluate
the impact of the choice of a bilingual education programme and particularly the role of
assessment within it (see Imam and Shaw, in press). The Education Division (CIE) has
outlined a coherent programme of research designed to address a number of key issues
2
relating to bilingualism and learning, curriculum, pedagogy and assessment. The overall aim
of the research is:
 to help build a pool of expertise and reputation in bilingual education
 to support the work of other parts of CIE involved in bilingual education, and
 to improve service to CIE schools.
The CIE bilingual research programme
The research programme is designed to address a number of specific questions grouped
according to four principal themes:
1. What is the impact of different teaching environments?
 What is the impact of different amounts of time studying in each language?
 What is the impact of the choice of subject taught in the Second Language (L2)?
 What is the impact of the way teachers share the language roles?
 Are there significant differences in the way bilingual education is organised at primary
and secondary level?
2. What impact does bilingual education have on the teaching and learning process?
 Do academic skills and subject knowledge skills transfer across languages?
 Do literacy skills transfer across languages?
 What different approaches are taken to learning to read and write in bilingual education
systems?
 What research has been carried out about bilingual education in early years?
3. What is the impact of bilingual education on learner outcomes?
 What is the impact of bilingual education on achievement at school?
 What is the impact of having a bilingual education background for Higher Education?
4. What are the key assessment issues?
 What methods of assessment are used in bilingual education programmes?
 How is assessment adapted for bilingual education?
Review of the literature
Over the last two years CIE has conducted a number of reviews of the bilingual education
literature (Lewis, 2010; Chu et al. 2011) in order to begin to address some of these questions.
The most salient points to emerge from the reviews are:
 More research is needed into ways of making academic content more accessible and
meaningful to students in bilingual programmes particularly in areas/subjects considered
challenging when learning academic content occurs through the L2
 Bilingual education is a complex phenomenon
 Different models of bilingual education – ‘weak’ (‘subtractive’) and ‘strong’ forms
(‘additive’) – impact differently on learner outcomes and achievement at school
 Traditional models over the last 40 years do not suffice in the twenty-first century
 There has been a move away from traditional models of bilingual education and a focus
more on effective classroom practice
 Urgent need to develop effective bilingual assessment methods that reflect classroom
practices
 Effective evaluation of learning and understanding of emergent bilinguals through a
framework of dynamic bilingualism and performance-based, on-going, multimodal
assessments
 A way to create more equitable assessments for emergent bilinguals is to employ
‘translanguaging’ practices within assessments. Baker (2000, p.104-105) defines
translanguaging as ‘the hearing or reading of a lesson, a passage in a book or a section of
work in one language and the development of the work (i.e. by discussion, writing a
passage, completing a worksheet, conducting an experiment) in the other language’.
3
What are the key messages?
The key messages emerging from the literature reviews are that bilingual education is:
 Challenging, in terms of learning subjects through an additional language
 Complex, with a discernible shift away from the simplicity and variety of typologies of
bilingual education to ‘engaging with optimal classroom dual language practices that
maximise growth and gains for individual children’ (Baker, 2008, p.106)
 Changing, and In need of urgent development, in terms of assessment. Traditional
typologies are in need of expansion in order to capture the linguistic complexity of the
emerging bilingual (and multilingual) education practices of the twenty-first century
(Garcia, 2009a). In other words, there is a move away from effective models towards
effective practices, and a shift from strict separation of languages to bilingual or
multilingual discourse practices.
Research questions
CIE is therefore conducting a number of studies designed to address specific language-related
questions in relation to its own assessments, as well as exploring the potential for bilingual
assessments (formative or summative). For example:
 What level of English, according to the Common European Framework of References for
Languages (CEFR), is useful to access CIE assessments?
 What cognitive and academic language skills are needed to access CIE assessments?
 How can schools be supported to prepare teachers and learners for whom English is a
second language (E2L) for bilingual programmes which include high-stakes, international
assessment? For example, they could be given the tools to develop language awareness
amongst subject teachers as well as coordination between subject and language teachers.
Schools could also be guided on mapping their national against the international
curriculum to streamline teaching and learning.
 Does the level of English impact on standards achieved in CIE non-language
qualifications?
 What does successful attainment of CIE non-language qualifications indicate about
language proficiency?
 What form of new assessments would enable bilingual students to demonstrate their
strengths?
Research into language awareness
To tackle the first message of ‘challenge’ (and the first three research questions above), CIE
has conducted research into the English language levels and skills required to achieve in
typical
 Cambridge International General Certificate of Secondary Education (IGCSE) subjects (a
major, high-stakes qualification for 14 to 16 year-olds) focusing on History, Geography
and Biology and
 International Advanced Level (A Level) Geography and Physics (another high-stakes
qualification for 16 to 19 year-olds).
Data for the IGCSE analysis included syllabuses, question papers, mark schemes and
candidate performances (Shaw, 2012). Findings suggest that each subject necessarily requires
Cognitive Academic Language Proficiency (CALP) and some of the key academic language
skills are identified. An average language proficiency level of B2 on the Common European
Framework of References for Languages (CEFR) is useful to access typical IGCSE subjects,
and CEFR level of C1 could provide an added advantage of linguistic resources to be able to
develop arguments needed for higher grades for Humanities subjects such as History and
Geography. The outcomes have contributed to the question writing process and have
informed the construction of a toolkit for content and language teachers.
Building on the IGCSE research reported here, a current study attempts to assess the impact
of linguistic complexity and language accessibility on candidates taking international A Level
examinations. The research comprises three phases. In phase 1, the marks obtained by each
student for each sub-question on the exam papers for a random sample of at least 200 scripts
4
for A Level Geography have been collected and keyed into data spreadsheets. The data sets
have been used to conduct a number of statistical analyses to describe question functioning
for both whole questions and question parts using traditional and item response analyses. In
phase 2, questions that statistical analyses suggest are performing in ‘unexpected’ ways
(extremes of difficulty; reverse thresholds, a number of overfitting and underfitting items)
have been identified and explored using textual and discourse analytic techniques in order to
determine whether the questions present language problems for international candidates and,
more importantly, why these questions might be problematic. In the final phase of the
research, students studying in their second year of A Level Geography from a range of
linguistic backgrounds will be asked to engage with the input language of questions identified
in phase 1 and to comment on their linguistic complexity. Triangulation of textual analysis
and think-aloud protocols will provide a powerful means to explore complex syntactic and
lexical features that challenge English language learners. Through the ‘voices’ of students,
this work will scrutinize the appropriateness of inferences about English language learners’
content knowledge based on linguistically complex test items.
By analysing assessment input and candidate output, CIE has identified useful target CEFR
English language levels, as well as the type of cognitive academic language proficiency
(CALP) needed for different subjects. For example (IGCSE):
History
Mainly open questions
with
constructed
responses:
describing,
explaining, evaluating.
In-depth source evaluation.
E.g.
How far…
How successful…
Biology
Precision: limited range of
language functions signalled
by different command words
each with a precise meaning.
E.g.
Name…
Describe…
Predict …
Geography
Flexibility: variety of question
types requiring a range of
language skills. Broad range
of command words. E.g.
Calculate…
(prompting
non-linguistic
answers)
Identify …
(prompting short answers)
Explain …
(prompting developed answers)
Practical applications have arisen from this research:

Language guidance has been provided for CIE question writers. For example:
Avoid phrasal verbs

Phrasal verbs use simple words. However, they are colloquial
and potentially difficult to process.
Candidates may have difficulty with expressions like ‘put up
with’. This could be replaced by ‘tolerate’.
Language guidance to support teachers and learners in the classroom has been
commissioned and co-published by CIE (Chadwick, 2012). For example, this encourages
subject teachers to consider the following questions:
5
The guidance also encourages subject teachers to consider the cognitive processes that relate
to the functional language of their lessons (see Shaw, 2012a, pp.12-13):
Language for describing
1. Language of
7. Language for classification
self-reflection
2. Language of
description
3. Language of
process
8. Language for
creating
Language for interpreting
4. Language of
analysis
5. Language of
evaluation
6. Language of
conclusion
Research into bilingual education
To tackle the second message of ‘complexity’, CIE commissioned and co-published a guide
into excellence in bilingual education, capturing the key research on bilingual education as
well as focusing on effective school management and classroom practice (Mehisto, 2012).
This focuses on core considerations applicable to diverse contexts, regardless of the particular
model of bilingual education that a school uses.
Research into developing bilingual assessments
Regarding the third message of ‘change’, CIE is at this stage simply exploring current
thinking in order to consider future challenges and opportunities. Garcia (2009) notes that ‘no
area of bilingual education is in more need of development than that of bilingual assessment.’
(p.378). She concludes that, ‘ … without large-scale bilingual assessment that would take
into consideration the bilingual continuum in which bilingual children operate, as well as the
integrated nature of their language proficiency and content knowledge, bilingual children will
never be able to demonstrate their strength’ (p.378).
6
Bilingual assessment is an issue that needs to be developed and researched in order to
accommodate the bilingual continuum in which bilingual children operate. It is evident that
this ‘most thorny issue’ (García, 2009, p.396) stems from the fact that assessment methods for
bilinguals have developed from the practice of testing monolinguals and most often ignore the
children’s bilingualism by assessing their abilities and knowledge as if they were performing
as two monolinguals. Consequently, the monolingualism of most assessments does not reflect
the bilingual practices of the classroom (García, 2009). There is a pressing requirement,
therefore, to develop effective bilingual assessment methods that reflect classroom practices
of using two (or more) languages for teaching and learning so that bilingual children are given
the opportunity to show their proficiency and competences in more than one language.
García (2009, p. 371-375) has conducted a comprehensive review of ways in which all
assessments, including large-scale standardised assessment, could be done in bilingual ways.
One of her main recommendations includes a translanguaging mode of bilingual assessment.
Translanguaging ‘ … is the act performed by bilinguals of accessing different linguistic
features or various modes of what are described as autonomous languages, in order to
maximize communicative potential’ (Garcia, 2009a, p. 140). Translanguaging reinforces the
interrelationship between the two languages while also reinforcing the languages. According
to Baker (2000), translanguaging has the potential to ‘promote a deeper and fuller
understanding of the subject matter’ (p.104) and to ‘develop skills in the weaker language …’
(p.105) and as a communicative practice, offers a range of communicative and educational
possibilities.
The way forward
The growth of multiple multilingual education programmes at the end of the 20th century has
been in response to the type of complex bilingualism brought about by globalisation (Garcia,
2009a, p.146). There is now a movement requirement to construct bilingual models that
reflect the fluidity of classroom practices that have come about because of the
‘translanguaging’ that is characteristic of bilingual classrooms today. Echoing Garcia’s
concerns, it is crucial then that future bilingual assessment practice ‘can tap the pluriliteracies
of multimodal texts which bilingual children must produce in the twenty-first century’
(Garcia 2009, p.378).
Translanguaging in high-stakes situations may be only appropriate to localities where only
two languages are in use (such as Welsh-English in Wales, or Spanish-English in certain parts
of the USA). There would be considerable pragmatic issues in providing translanguaging
opportunities on an equitable basis in multilingual contexts. Much of the literature focuses on
the experience of classroom practice and formative assessment, and this may be where the
potential lies for CIE schools.
CIE is attempting, therefore, to build an understanding of best practice in the area of bilingual
(and trilingual) education and to guide any future developments in terms of language
awareness and bilingual assessment which can be shared more widely across the assessment
community.
References
Bloomfield, L. (1935). Language. London: Allen and Unwin.
Baker, C. (2000). The Care and Education of Young Bilinguals: An Introduction for
Professionals. Clevedon: Multilingual Matters.
Baker, C. (2006). Foundations of Bilingual Education and Bilingualism. Fourth edition. 2006.
Clevedon: Multilingual Matters.
Baker, C. (2008). Postlude. In Multilingualism and Minority Languages: Achievements and
Challenges in Education, AILA Review 21, Jasone Cenoz & Durk Gorter (eds.), 69-86, 2008.
Chadwick, T. (2012). Language Awareness in Teaching: A Toolkit for Content and Language
Teachers. Cambridge: Cambridge University Press and University of Cambridge International
Examinations.
7
Chu, H. Flores, N. & Woodley, H. H. (2011). Dynamic Assessments for Bilingual Learners: A
report of research and design. Produced for University of Cambridge International
Examinations. Graduate Centre, City University of New York.
Cummins (2009). Bilingual and immersion programs. In Long & Doughty (Eds.) The
Handbook of Second Language Teaching). Oxford: Blackwell.
Crystal, D. (2003). Bilingual. A Dictionary of Linguistics and Phonetics. Oxford: Blackwell
Publishing.
Dewaele, J.M, Housen, A. & Li W, (eds). (2003). Bilingualism: Beyond Basic Principles.
Clevedon: Multilingual Matters.
Edwards, J. (2004). Foundations of bilingualism. In T. K. Bhatia and W. C. Ritchie (Eds.),
The Handbook of Bilingualism (pp. 7-31). Oxford: Blackwell Publishing.
García, O. (2009). Bilingual Education in the 21st Century: A Global Perspective. Chichester:
Wiley-Blackwell.
García, O. (2009a). Education, multilingualism and translanguaging in the 21st century. In
Tove Skutnabb-Kangas, Robert Phillipson, Ajit K. Mohanty and Minati Panda (Eds.), Social
Justice through Multilingual Education, pp. 140-158. Bristol: Multilingual Matters.
Imam, H. C. and Shaw, S. D. (in press). The Cambridge International Examinations Bilingual
Research Agenda. Research Matters.
Lewis, G. W. (2010). Bilingual Education: Research Review. Produced for University of
Cambridge International Examinations. ESRC Centre for Research on Bilingualism in Theory
and Practice, and College of Education and Lifelong Learning. Bangor: University Bangor.
Mehisto, P. (2012). Excellence in Bilingual Education: A Guide for School Principals.
Cambridge: Cambridge University Press and University of Cambridge International
Examinations.
Shaw, S. D. (2012). International assessment through the medium of English: analysing the
language skills required. Research Matters. Issue 13, January, 2012, pp.2-10.
Shaw, S. D. (2012a). Creative Thinking Skills: Cognitive processes that relate to functional
language. In Chadwick, T. (2012).
Acknowledgements
The authors of this article would like to thank Gerry Delaney, Paul Ellis and Cristina Rimini
at Cambridge International Examinations for their research input.
8