OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
11
Programme for International Student Assessment
Strong Performers and
Successful Reformers in
Education
Francesca Borgonovi
Early Childhood and Schools Division, EDU
OECD
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
22
Foundation skills for the future
Rapid pace of change in most work
environments requires strong foundation skills
even in relatively routine manual occupations
 Strong foundation skills are needed to make
the most of computerisation and digitisation



Literacy to understand descriptions of new
procedures for rapidly changing tasks
Numeracy to be able to understand the basic
mathematics that permeates models used by
computers
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
33
Advanced skills for the future


Expert thinking and ability to solve problems
with no rules-based solutions
Complex communication and the ability of
interacting with humans to acquire information,
explain it and lead others to action.
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
44
Skill use by occupational groups
Problem solving
1.00
Computer use
Teamwork
0.80
0.60
Internet use
Oral communication
0.40
0.20
0.00
-0.20
Basic numeracy
Influence others
-0.40
-0.60
-0.80
-1.00
Advanced numeracy
Plan own time
Write
Plan others time
Read document type texts
Read prose type texts
Fine motor skills
Gross motor skills
Total
Service (low-skill)
Goods
Information (high-skill)
Managers
Knowledge (expert)
Source: PIAAC Field trial
Information (low-skill)
Skill mismatch by occupational groups
HIGH-SKILL MATCH
60%
50%
40%
30%
20%
MISMATCH-SKILL
DEFICIT
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
55
10%
0%
MISMATCH-SKILL
SURPLUS
LOW-SKILL MATCH
Goods
Service (low-skill)
Information (low-skill)
Information (high-skill)
Managers
Knowledge (expert)
Source: PIAAC Field trial
How the demand for skills has changed
Economy-wide measures of routine and non-routine task input (US)
Mean task input as percentiles of the 1960 task distribution
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
66
65
Routine manual
60
Nonroutine manual
55
Routine cognitive
50
Nonroutine analytic
45
Nonroutine interactiv
40
The dilemma of schools:
easiest to teach and
1960
1970
1980 The skills
1990that are2002
test are also the ones that are easiest to
(Levy and Murnane)
digitise, automate and outsource
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
77
Implications for education systems


Technology can change the nature of work
faster than people can change their skills
Education systems need to proactively and
continuously re-think about how to develop the
advanced skills that will be needed in the
unpredictable labour markets and societies of
the future
1. Excluding ISCED 3C short programmes
3. Including some ISCED 3C short programmes
2. Year of reference 2004
3. Year of reference 2003.
Brazil2
Mexico
Portugal
Turkey
Spain
Italy
1
Greece
13
Chile2
90
Korea
Ireland
Poland
Belgium
Iceland
Australia
France
OECD average
EU19 average
Luxembourg
1970s
Netherlands
United Kingdom3
Finland
Hungary
1980s
New Zealand
Slovak Republic
Israel
Slovenia
1990s
Austria3
Russian Federation4
Sweden
Norway
Canada
Denmark
Switzerland
Germany
20
Estonia
80
Czech Republic
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
%
United States
OECD Programme for
International Student Assessment
PISA
88 A world of change in the global talent pool
Approximated by percentage of persons with high school or equivalent qualfications
in the age groups 55-64, 45-55, 45-44 und 25-34 years
1960s
100
1
70
60
50
40
30
27
10
0
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
99
OECD Programme for
International Student Assessment
PISA

Methods…
PISA


PISA in brief
countries in 2001
2003
2000
2009
2006
1998
TargetCoverage
populationof15-year-old
students
world economy
83%
81%
77%
86%
85%
87%
Random samples
– PISA 2009: over half a million students…
….representing 28 million 15-year-olds in 74 countries/economies



2 hour paper and pencil test in reading, mathematics
and science
Item types- constructed response and multiple
choice; real life tasks
Student background questionnaire
– their personal background, their schools
and their engagement with learning and school

School and system background questionnaires
– school policies, practices, resources and institutional
factors that help explain performance differences .
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
10
10
PISA... a (relatively) long story
PISA in 2000, 2003, 2006 and 2009
 PISA 2012 – In the field
 PISA 2015 – Work in Progress



Focus on students’ capacity to extrapolate
from what they know and creatively apply their
knowledge in novel situations
Less emphasis on whether they can reproduce
what they were taught
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
11
11
Strong Performers and Successful
Reformers in Education

Quality of Learning outcomes

Equity of learning opportunities

Factors that make a difference
High performance

Increasing performance over time

OECD Programme for
International Student Assessment
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012

PISA
12
12
Quality of Learning Outcomes
The learning outcomes of poor performing and
top performing students
High performance

Increasing performance over time

OECD Programme for
International Student Assessment
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012

PISA
13
13
Quality of Learning Outcomes
The learning outcomes of poor performing and
top performing students
Shanghai-China
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
14
14
High reading performance
Average performance
of 15-year-olds in
reading
Korea
Finland
Hong Kong-China
540
Singapore
New Zealand
Japan
Australia
Belgium
Poland, Switzerland
United States
Germany, Sweden
France, Ireland
Hungary, United Kingdom
Macao-China
Slovenia
Slovak Republic, Czech Republic
Luxembourg, Israel
Austria
Dubai (UAE)
Canada
520
Netherlands
Norway , Estonia
Iceland
500
Liechtenstein
Chinese Taipei
Denmark
Portugal
Italy
Latvia
Greece
480
Spain
Croatia
Lithuania
Turkey
460
Russian Federation
Chile
Serbia
440
… 17 countries perform below this line
Low reading performance
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
15
15
Performance in reading (2009)
539
495
High performance

Increasing performance over time

OECD Programme for
International Student Assessment
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012

PISA
16
16
Quality of Learning Outcomes
The learning outcomes of poor performing and
top performing students
50
40
30
-10
-20
-30
-40
Peru
Chile
Albania
Indonesia
Latvia
Israel
Poland
Portugal
Liechtenstein
Brazil
Korea
Hungary
Germany
Greece
Hong Kong-China
Switzerland
Mexico
OECD average-26
Belgium
Bulgaria
Italy
Denmark
Norway
Russian Federation
Japan
Romania
United States
Iceland
New Zealand
France
Thailand
Canada
Finland
Spain
Australia
Czech Republic
Sweden
Argentina
Ireland
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
Score point change
OECD Programme for
International Student Assessment
PISA
17
17
Change in reading performance between 2000 and 2009
Reading
performance
improved
20
10
0
Reading
performance
declined
High performance

Increasing performance over time

OECD Programme for
International Student Assessment
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012

PISA
18
18
Quality of Learning Outcomes
The learning outcomes of poor performing and
top performing students
50
30
0
o
o
o
o
o
o
o
o
+
+
+
o
o
+
+
o
o
+
o
o
o
o
+
o
-
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
60
Korea
Finland
Hong Kong-China
Canada
Japan
Australia
New Zealand
Norway
Poland
Denmark
Liechtenstein
Switzerland
Iceland
Ireland
Sweden
Hungary
Latvia
United States
Portugal
Belgium
OECD average-26
Germany
Spain
France
Italy
Greece
Czech Republic
Israel
Russian Federation
Chile
Mexico
Romania
Bulgaria
Thailand
Brazil
Argentina
Indonesia
Albania
Peru
OECD Programme for
International Student Assessment
PISA
19
19
Percentage of students below proficiency Level 2 in
reading between
2000 and 2009
%
2009
2000
90
80
70
+ 2009 higher than 2000
- 2009 lower than 2000
o Not statistically
significant difference
2000
40
2009
20
10
16
14
12
10
+
+
+
o
o
o
o
o
o
+
o
o
o
o
o
o
o
o
o
o
+
+
o
o
o
o
o
o
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
%
New Zealand
Finland
Japan
Korea
Australia
Canada
Hong Kong-China
Belgium
United States
France
Sweden
Iceland
Norway
Switzerland
Germany
Israel
Poland
Ireland
Hungary
Italy
Greece
Czech Republic
Portugal
Denmark
Liechtenstein
Spain
Russian Federation
Latvia
Bulgaria
Brazil
Chile
Argentina
Romania
Peru
Mexico
Thailand
Albania
Indonesia
OECD Programme for
International Student Assessment
PISA
20
20
Percentage of top performers in reading between
2000 and 2009
2009
2000
20
18
2009
+ 2009 higher than 2000
- 2009 lower than 2000
o Not statistically significant
difference
2000
8
6
4
2
0
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
21
21
Equity of learning opportunities




Are students held back because of their socioeconomic condition?
The role of schools: between and within school
differences in performance
The performance of students with an
immigrant background
Gender gap
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
22
22
Equity of learning opportunities




Are students held back because of their socioeconomic condition?
The role of schools: between and within school
differences in performance
The performance of students with an
immigrant background
Gender gap
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
23
23
Measures of the relationship between socio-economic
background and reading performance
Strength of the gradient
(% of variance explained by ESCS)
OECD Programme for
International Student Assessment
PISA
OECD average
Denmark
Finland
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
20
10
Shanghai-China
International Student Assessment
Hong Kong-China
Korea
Macao-China
Singapore
Finland
Japan
Turkey
Canada
Portugal
Chinese Taipei
Poland
New Zealand
Spain
Liechtenstein
Estonia
Netherlands
Italy
Switzerland
Latvia
Australia
OECD average
France
Belgium
Ireland
Iceland
Mexico
United States
Greece
Thailand
Croatia
Tunisia
Norway
Hungary
Sweden
Slovenia
Indonesia
Denmark
Chile
United Kingdom
Israel
Colombia
Germany
Brazil
Czech Republic
Slovak Republic
Luxembourg
Lithuania
Austria
Russian Federation
rinidad and Tobago
Uruguay
Serbia
Jordan
Albania
Argentina
Dubai (UAE)
Romania
Bulgaria
Panama
Montenegro
Kazakhstan
Peru
0
OECD Programme for
PISA
70
60
50
40
30
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
24
24
80
%
Percentage of resilient students among
disadvantaged students
Resilient student: Comes from the bottom
quarter of the socially most disadvantaged
students but performs among the top quarter of
students internationally (after accounting for
social background)
Less than 15% resilient
students among
disadvantaged students
More than 30% resilient
students among
disadvantaged students
Between 15%-30% of resilient
students among
disadvantaged students
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
25
25
Equity of learning opportunities




Are students held back because of their socioeconomic condition?
The role of schools: between and within school
differences in performance
The performance of students with an
immigrant background
Gender gap
Denmark
Private school
Public school in rural area
Public school in urban area
Score
700
Student performance
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
26
26School performance and socio-economic background
493
200
-2
Disadvantage
-1
0
1
PISA Index of socio-economic background
2
Advantage
Finland
Private school
Public school in rural area
Public school in urban area
700
Student performance
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
27
27School performance and socio-economic background
493
200
-3
Disadvantage
-2
-1
0
1
2
PISA Index of socio-economic background
3
Advantage
Germany
Private school
Public school in rural area
Public school in urban area
Score
700
Student performance
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
28
28School performance and socio-economic background
493
200
-2
Disadvantage
-1
0
1
PISA Index of socio-economic background
2
Advantage
Strong Performers and Successful Reformers in
PISA Variance
Education - Copenhagen
60
40
20
0
Argentina
Trinidad and Tobago
Italy
Qatar
Turkey
Bulgaria
Israel
Panama
Germany
Peru
Hungary
Dubai (UAE)
Austria
Belgium
Luxembourg
Netherlands
Japan
Chile
Uruguay
Greece
Brazil
Czech Republic
Slovenia
Romania
Croatia
Serbia
United States
Mexico
Singapore
Jordan
Kyrgyzstan
Colombia
Montenegro
Hong Kong-China
Albania
Tunisia
Slovak Republic
Liechtenstein
Kazakhstan
Macao-China
Ireland
United Kingdom
Chinese Taipei
Korea
Switzerland
Australia
New Zealand
Portugal
Shanghai-China
Azerbaijan
Russian Federation
Canada
Sweden
Lithuania
Indonesia
Spain
Scotland
Poland
Estonia
Latvia
Iceland
Thailand
Denmark
Norway
OECD Programme for
International Student Assessment
140
120
100
80
Francesca Borgonovi
9 May 2012
29
29
Variability in student performance
200
180
160
OECD Programme for
International Student Assessment
Francesca Borgonovi
9 May 2012
Strong Performers and Successful Reformers in
PISA Variance
Education - Copenhagen
30
30
Variability in student performance
between and within schools
Performance differences
between schools
Performance variation of
students in schools
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
31
31
Equity of learning opportunities




Are students held back because of their socioeconomic condition?
The role of schools: between and within school
differences in performance
The performance of students with an
immigrant background
Gender gap
400
350
300
450
Francesca Borgonovi
9 May 2012
500
Finland
Hong Kong-China
Singapore
Canada
New Zealand
Australia
Netherlands
Belgium
Norway
Estonia
Switzerland
United States
Liechtenstein
Sweden
Germany
Ireland
France
Denmark
United Kingdom
Hungary
OECD average
Portugal
Macao-China
Italy
Slovenia
Greece
Spain
Czech Republic
Croatia
Israel
Luxembourg
Austria
Dubai (UAE)
Russian Federation
Serbia
Mexico
Trinidad and Tobago
Brazil
Montenegro
Jordan
Argentina
Kazakhstan
Qatar
Panama
OECD Programme for
International Student Assessment
Strong Performers and Successful Reformers in
performance
Mean reading
PISA
Education - Copenhagen
32
32 Immigrants
and reading
performance
Native
Students
Second-generation
students
First-generation students
600
550
Native students
Second-generation students
First-generation students
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
33
33
Equity of learning opportunities




Are students held back because of their socioeconomic condition?
The role of schools: between and within school
differences in performance
The performance of students with an
immigrant background
Gender gap
Finland
Slovenia
Slovak Republic
Poland
Czech Republic
Norway
Greece
Italy
New Zealand
Sweden
Estonia
Iceland
Turkey
Israel
Austria
France
Germany
Luxembourg
Ireland
Japan
Switzerland
Portugal
Hungary
Australia
Korea
Canada
Spain
Denmark
Belgium
United Kingdom
Mexico
United States
Netherlands
Chile
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
34
34
Gender Gap in Reading
(PISA 2009, girls - boys)
60
50
Girls perform better
40
30
20
10
0
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
10
-50
Belgium
Chile
United Kingdom
United States
Switzerland
Luxembourg
Austria
Spain
Netherlands
France
Denmark
Germany
Italy
Greece
Mexico
Hungary
Canada
Portugal
Turkey
Australia
Japan
Israel
New Zealand
Ireland
Norway
Czech Republic
Poland
Iceland
Korea
Slovak Republic
Finland
Slovenia
Sweden
OECD Programme for
International Student Assessment
PISA
35
35
Gender Gap in Mathematics
(PISA 2009, girls - boys)
Score difference
0
-10
-20
-30
-40
Boys perform better
-60
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
36
36
Factors that make a difference
What factors make a difference?

For the individual student

Reading for enjoyment
– Very strongly related to performance in all countries
– 37% of students in OECD countries on average do not read
for enjoyment

OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
37
37
Diversity of reading materials
– Better readers tend to read a diversity of materials

Knowledge of strategies
– High-performing countries are generally those whose
students know how to summarise information well

And for system and school policies……
High reading performance
2009
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
38
38
High average performance
Large socio-economic disparities
Durchschnittliche
High average performance
Schülerleistungen im
High social equity
Bereich Mathematik
Strong socioeconomic impact on
student performance
Socially equitable
distribution of learning
opportunities
Low and
average performance
Early selection
institutional
differentiation
Large
socio-economic disparities
Low average performance
High degree of stratification
Low degree of stratification
Low reading performance
High social equity
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
39
39
Policy
Policies and practices
R
R
System
E
School
Equity
Learning climate
Discipline

Teacher behaviour

Parental pressure
Teacher-student
relationships
Dealing with heterogeneity

Grade repetition
Prevalence of
tracking
Expulsions
Ability grouping
(all subjects)










Standards /accountability
Nat. examination


OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
40
40
What does it all mean?
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
41
41
Some lessons
from successful
systems
OECD Programme for
International Student Assessment
PISA
Strong Performers and Successful Reformers in
Education - Copenhagen
Francesca Borgonovi
9 May 2012
49
49
Find out more about PISA at…
www.oecd.org/pisa
PISA In Focus
www.oecd.org/pisa/infocus
Email:
Francesca.Borgonovi@oecd.org