Quality of education and educational assessment:
How can they support each other?
What is ‘quality’ (of education)?
Goal 6 of EFA
 Quality vs. Quantity (qualitative
and quantitative research)
 In education: Quality vs. Access
 Relates to goal Nº 6 of EFA
 With > 95% access in LAC
(ANER 2008), this is the
principal focus of UNESCO in
the región
What is education?
Teaching + Learning = Education
Conceptualisations of quality of
education: several models
The Tikly model (2010): policy,
school and home / community
The UNESCO Santiago model
 Relevance: Learning which




prepares for life
Pertinence: Educational offer
relating to distinct groups
Equity: Education that does not
discriminate
Effectiveness: Achieving goals
related to access
Efficiency: Effective use of
resources
The international understanding:
access rates and rankings
 The international understanding on monitoring of
educational progress is that:
 Access is measured by means of access rates (such as
GER, NER, ANER, etc.) – there is a difference between
enrolment and attendance
 Quality is measured by means of (international)
assessments of learning outcomes: almost always
resulting in ‘rankings’
Kenya: attempting to increase
without considering quality
Source : Somerset, A.
The generic CIPP model
 Generally speaking, the
conceptualisation of
(measuring) quality of
education has shifted from
input indicators to
outcome indicators
 The focus nowadays being
on education outcomes,
there is a risk of losing the
view on how we achieve
these results
Rankings: a look at SERCE
 In SERCE, there are three clear
groups of countries:
 Achievement above average
achieved by CUB
 Achievement at average level
achieved by CHI, CRI, MEX and
URU
 Achievement below average by
ECU, GUA, NIC, PAN, PAR and
RDO
 Further, PER and SAL are
between groups 2 and 3
[Analysis: PREAL, Políticas 37]
Access and achievement according
to the SERCE data
¿Donde están los países en resultados y acceso?
Tasa neta de matrícula y resultados en lenguaje en 3° de primaria
Puntaje promedio SERCE
650
CU
600
CH
550
ME
COL
500
NI
450
PAR
BRA
GU
AR UR
PE PAN
EC
RD
400
350
75
80
85
90
Tasa neta de matrícula primaria
95
100
105
Relation between investment and
achievement in SERCE
Gasto Público por alumno en primaria y resultados en lenguaje de los
alumnos de 3° grado
Puntajes promedio SERCE
600
CR
CH
550
ME
COL
UR
500
PE
BRA
AR
PAN
PAR
GU
450
R2 = 0,3301
400
350
5
7
9
11
13
15
GP por alumno como % del PIB per-capita
17
19
21
Statistical ‘artefacts’
 In Chile, is the private
subsidised school really
better than the municipal, or
public, school? (See graph)
 In Lebanon, for example,
sometimes learners are
excluded from the last year of
study, as their results might
impact negatively on the
overall score of a school
Source: presentación Dr Schiefelbein
Rankings: a look at PISA 2009
The ‘tiger mother’ and her retreat
 The year 2010 saw the debate on
the tiger mother in the United
States
 The tiger mother was an idea of
author Amy Chua: the Chinese
mother who makes sure her
children achieve excellent
results in education - typically,
she maintains an ‘iron’ discipline
in her family
 Tragical cases of suicide have led
to doubt around excessive
discipline
PISA 2009 in the region
Reading
Maths
Sciences
 The data from PISA 2009
suggest that the countries of
the region have an average
achievement below OECD
average in all subjects
 Nevertheless, the ‘best’
countries in SERCE are also
the ‘best’ in SERCE (i.e. CHI,
MEX, URU)
Chile
449
421
447
439
Uruguay
426
427
427
427
Mexico
425
419
416
420
T&T
416
414
410
413
Brazil
412
386
405
401
Colombia
413
381
402
399
Argentina
398
388
401
396
Panama
371
360
376
369
Peru
370
365
369
368
An alternative ranking: inequality
as per PISA 2009 (Duncan Index)
Source : El Mercurio, 17 abril 2011, p. D21
What do teachers have to do with
this?
Source : Apuntes – Revista de Educación, p. 24 (TEDS)
More rankings: the globalising
market of tertiary education
 Nowadays, tertiary education
has become a market of
global dimensions
 International /
intercontinental mobility in
2007, according to UIS: 2.8
mlln.
 Principal direction: from Asia
Pacific to Europe and North
America
 Has knowledge become a
Source: UIS GED 2009
consumer good?
Is education not important at the
tertiary level?
Ra
nk
 There exist several rankings:
 Times Higher Education
Supplement;
 Shanghai Jiaotong (ARWU);
 Webometrics, etc.
 The methodology includes
variables related to research,
but usually not related to
education (the exception:
AHELO)
Institution
Country
1
University of Cambridge
United
Kingdom
2
Harvard University
United States
3
Massachusetts Institute of Technology (MIT)
United States
4
Yale University
United States
5
University of Oxford
United
Kingdom
6
Imperial College London
United
Kingdom
7
UCL (University College London)
United
Kingdom
8
University of Chicago
United States
9
University of Pennsylvania
United States
Columbia University
United States
10
Added value
 University rankings usually
favour ‘western’ universities,
but:
 They do not have the same
average student entry-level
 They often have a languageadvantage
 Therefore, the moment of
measurement does not reflect
the ‘added value’ in terms of
education
 The rankings do not reflect
institutional achievement
Université de
Ouagadougou
University of
Oxford
Therefore…
 To what extent do rankings tell
us something about all the
areas that education could
address? (See, for example,
Gardner´s model of multiple
intelligences)
 To what extent do rankings tell
us something about how
systems achieve the level they
ultimately achieve, according to
the rankings? (For example,
inclusively or not, and under
which circumstances)
Source:
representation of a
model by H. Gardner
También a considerar: la definición
de competencias
 A competency (working
definition, Opertti et al.) is:
A competency implies the generation, mobilization
and integration of resources such as knowledge, knowhow, skills, attitudes and values aiming to enable one
to act effectively while facing different changing life
situations. It implies mobilizing capacities (knowledge
and know-how) to read the contexts and act
consequently avoiding stereotype responses.
People
Education
 It is important that education
systems can develop skills that
are not only ‘relevant’ to a
context, but ‘transformational’
Society
How to improve: the remedy is
not always the same
Poor ›
Reasonable G0od › Very Very good ›
Reasonable › Good
good
Excellent
Teachers
and HT
Assessment
Information
systems
Policies and
laws
Standards,
curriculums
Remunerati
on
Armenia: centralised
teacher training
programme
Singapore: increased
flexibility for teachers
What are we assessing when we
are assessing education?
Schools: leadership,
infrastructure
Teachers: ability,
training
Family / socioeconomic context
Learners: inherent
aptitude, motivation
Assessment
System: curriculum,
policies, resources
How can we improve assessment
for learning?
 It has already been said, but should be reiterated: assessment is not
only descriptive, but also normative
 Let us, therefore, reconsider assessment methods:
 Can assessment be group-wise instead of focused on an individual (as
inter-personal skills and teamwork are paradigmatic for the concept of 21st
century skills)?
 Non-standardised assessment: practical tasks, inter-disciplinary
problems, creative achievements
 Relation between knowledge, attitude and behaviour
 Programme evaluation: instances of violence in a school / community
context
Questions for discussion
 Which are the values, skills and knowledge (competencies)
which education should instill for a local and a global society?
 How can we rethink assessment so that it captures and
promotes, to a greater extent, quality education for all?
 Can we think of horizontal assessment, which classifies
according to talent / interest, rather than in ‘vertical’ terms, that
is to say, which classifies are ‘better’ or ‘worse’?
Gracias / Thank you / Obrigado /
Hartelijk dank
Moritz Bilagher
OREALC-UNESCO Santiago