PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
11
OECD Programme for International Student Assessment (PISA)
Launch of PISA 2006
London, 4 December 2007
Barbara Ischinger
Director
Directorate for Education, OECD
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
22
PISA
A three-yearly global assessment that…
… examines the performance of 15-year-olds in key
subject areas as well as a wider range of
educational outcomes
• Including students attitudes to learning, their beliefs
about themselves, and their learning strategies
… collects contextual data
from students schools, parents and systems to
identify policy levers
Coverage



Representative samples of between 3,500 and
50,000 15-year-old students drawn in each country
Most federal countries also draw regional samples
PISA covers roughly 90% of the world economy .
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
33
PISA countries in 2001
2003
2000
2009
2006
1998
Coverage of world economy 83%
81%
77%
87%
86%
85%
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
44
How PISA works

A strong international network of expertise
among the participating countries…

From establishing the assessment frameworks…
– The PISA assessments include tasks from more than 40 countries
… developing the instruments…
– Cross-national and cross-cultural validity
… to analysing and interpreting the results
– National, regional and international analyses and reports
– In-depths country peer reviews
… supported by a consortium of
the leading research institutions…

ACER, CITO, ETS, NIER, WESTAT
… co-ordinated through the OECD in collaboration with
other international organisations .
Science in PISA 2006
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
55
PISA defines scientific literacy in terms of an individual’s:

Scientific knowledge and use of that knowledge to…
… identify scientific issues,
… explain scientific phenomena, and
… draw evidence-based conclusions about science-related issues



Understanding of the characteristic features of
science as a form of human knowledge and enquiry
Awareness of how science and technology shape our
material, intellectual and cultural environments
Willingness to engage with science-related issues .
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
66
High science performance
565
Finland
Average performance
of 15-year-olds in
science – extrapolate
545
Hong Kong-China
and apply
Canada
Japan
New Zealand
Australia
525
Netherlands
Liechtenstein
Korea
Slovenia
United Kingdom Germany
Czech Republic Switzerland
Macao-China Austria
Belgium
Ireland
505
Hungary
Sweden
Poland
France Denmark
Iceland Croatia
United States Latvia
Slovak Republic, Spain, Lithuania
Norway 485
Luxembourg
Chinese Taipei
Estonia
Quality
in educational outcomes
Portugal
Russian Federation
Italy
Greece
465
Israel
445
16
… 18 countries perform below this line
Low science performance
6
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
77
Mean science scores – OECD countries
300
350
400
450
500
550
600
Finland
Canada
Japan
New Zealand
Australia
Netherlands
Korea
Germany
United Kingdom
Czech Republic
Switzerland
Austria
Belgium
Ireland
Hungary
Sweden
Poland
Denmark
France
Iceland
United States
Slovak Republic
Spain
Norway
Luxembourg
Italy
Portugal
Greece
Turkey
Mexico
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1c
88
Overall science score
United Kingdom=515
Scientific
competencies
Identifying scientific issues
Explaining phenomena scientifically
Using scientific evidence
OECD Programme for
International Student Assessment
PISA
Science Competencies
for Tomorrow’s World
Comparison of performance on the
different scales in science (UK)
Knowledge about science
Earth and space
Scientific
knowledge
Living systems
Physical systems
-35
-25
-15
-5
5
15
25
35
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13
Gender differences in science
performance (UK)
United Kingdom
-35
OECD average
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
99
-25
-15
-5
5
15
25
35
Overall
Identifying scientific
Explaining
phenomena
issues
Using
scientific
scientifically
Knowledge
evidence about science
Earth and
Living systems
Physical systems
Overall
Identifying scientific issues
Explaining phenomena scientifically
Using scientific evidence
Knowledge about science
Earth and space
Living systems
Physical systems
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Tables 2.1c, 2.2c, 2.3c, 2.4c, 2.7, 2.8, 2.9, 2.10
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
10
10
PISA Proficiency Levels in Science
OECD
Level 6
OECD
UK
1%
3%
Level 5
8%
Level 4
20%
Level 3
Level 2
Level 1
Below
Level 1
Science Level 6
11%
22%
27%
24%
14%
26%
22%
Student can consistently
identify, explain and apply
scientific knowledge and
knowledge about science in a
variety of complex life
situations
Science Level 1
Student have such a limited
scientific knowledge that it can
only be applied to a few, familiar
situations
12%
Below Level 1
5%
5%
Unable to use scientific skills
in ways required by easiest
PISA tasks.
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1a
Top and bottom performers
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
11
11
Large proportion of top performers
Large prop. of poor perf.
OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 2.1a
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
12
12
Investments and outcomes

Since 2000, expenditure per primary and secondary
student increased across OECD countries by 39%
(in real terms) …
… while PISA outcomes generally remained flat…
… but there are notable exceptions…
Poland raised its
reading
by 28 PISA points,
Between
PISA 2000performance
and
2003 Poland delayed the
equivalent
to ¾ of a school year - What happened?
separation of students into
%
different school types
Level 1 theBelow
Level
1
Level 2
beyond
age of
15 years
Level 3
Level 4
90
70
In 2003, performance
Between 2000 and 2003
variation among schools
had fallen from 51% toshowed the second16% of the variation of largest increase in
reading (17 points) and a
student performance
further 11 point increase
since 2003
But did this lead to
genuine improvements of
school performance?
30
10
10
Poland 2003
Poland 2000
30
Most of that increase
resulted from smaller
proportions at the bottom
level (23% in 2000, and
three-quarters in vocational
tracks, 17%in 2003)
Poland 2006
50
OECD 2006
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
13
13
Did this harm
the better performers?
OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 6.1a
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
14
14
Mean reading scores – OECD countries
300
350
400
450
500
550
600
Korea
Finland
Canada
New Zealand
Ireland
Australia
Poland
Sweden
Netherlands
Belgium
Switzerland
Japan
United Kingdom
Germany
Denmark
Austria
France
Iceland
Norway
Czech Republic
Hungary
Luxembourg
Portugal
Italy
Slovak Republic
Spain
Greece
Turkey
Mexico
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 6.1c
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
15
15
Mean mathematics scores – OECD
countries
300
350
400
450
500
550
600
Finland
Korea
Netherlands
Switzerland
Canada
Japan
New Zealand
Belgium
Australia
Denmark
Czech Republic
Iceland
Austria
Germany
Sweden
Ireland
France
United
Poland
Slovak
Hungary
Luxembourg
Norway
Spain
United States
Portugal
Italy
Greece
Turkey
Mexico
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 6.2c
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
16
16
High science performance
565
Finland
Average performance
of 15-year-olds in
science – extrapolate
545
Hong Kong-China
and apply
Canada
Japan
New Zealand
Australia
525
Netherlands
Liechtenstein
Korea
Slovenia
United Kingdom Germany
Czech Republic Switzerland
Macao-China Austria
Belgium
Ireland
505
Hungary
Sweden
Poland
France Denmark
Iceland Croatia
United States Latvia
Slovak Republic, Spain, Lithuania
Norway 485
Luxembourg
Chinese Taipei
Estonia
Attitudes
towards science
Portugal
Russian Federation
Italy
Greece
465
Israel
445
16
… 18 countries perform below this line
Low science performance
6
General value of science
United Kingdom
OECD average
Science is important for helping us to
understand the natural world
Top performer
Portugal
Advances in science and technology
usually improve people's living
conditions
Portugal
Science is valuable to society
Korea
100
Advances in science and technology
usually bring social benefits
50
Korea
25
Advances in science and technology
usually help to improve the economy
75
Portugal
0
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
17
17
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 3.2
Personal value of science
United Kingdom
OECD average
Top performer
I find that science helps me to
understand things around me
Portugal
I will use science in many ways when
I am an adult
Poland
Some concepts in science help me
see how I relate to other people
Mexico
When I leave school there will be
many opportunities for me to use
science
Mexico
Science is very relevant to me
100
75
50
25
Mexico
0
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
18
18
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 3.4
PISA
Student engagement with science
Students show strong support for scientific enquiry
93% said that science was important for understanding the natural world
92% said that advances in science and technology usually improved people’s living conditions
75% said that science helped them to understand things around them
57% said that science was very relevant to them personally
0
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
19
19
10
20
30
40
50
60
70
80
90
100
Students expressed confidence in be able to do scientific tasks,
but more so for some tasks than others
76% said they could explain why earthquakes occurred more frequently in some areas than in others
64% said they could predict how changes to an environment would affect the survival of certain species
51% said they could discuss how new evidence could lead to a change in understanding about the possibility
of life on Mars
0
10
20
30
40
50
60
70
80
90
100
OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 4.1a
High science performance
565
Finland
20
20
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
High average performance
Large socio-economic disparities
Average performance
15-year-olds
Highof
average
performancein
science – extrapolate
High social equity
545
Hong Kong-China
and apply
Canada
Japan
New Zealand
Australia
525
Netherlands
Liechtenstein
Korea
Slovenia
United Kingdom Germany
Czech Republic Switzerland
Macao-China Austria
Belgium
Ireland
Socially equitable
505
Hungary
Sweden
distribution of learning
Poland
opportunities
France Denmark
Iceland Croatia
United States Latvia
Slovak Republic, Spain, Lithuania
Norway 485
Luxembourg
Chinese Taipei
Estonia
Strong socioeconomic impact on
student performance
Equity
in educational opportunities
Portugal
Russian Federation
Italy
Greece
465
Israel
Low average performance
Large socio-economic disparities
Low average performance
445
16
Low science performance
High social equity
6
High science performance
21
21
560
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
High average performance
Large socio-economic disparities
540
Chinese Taipei
New Zealand
Finland
Durchschnittliche
High average performance
Schülerleistungen im
High social equity
Bereich
Mathematik
Hong Kong-China
Estonai
Australia
Netherlands
Slovenia 520
Germany
United Kingdom
Switzerland
Czech Republic
Austria
Belgium
Ireland
Strong socioHungary
Sweden
economic impact on
500
Poland
student performance
Denmark
France
Croatia
Latvia
United States
Slovak Republic
Lithuania
Spain
Luxembourg
Canada
Japan
Korea
Liechtenstein
Macao-China
Socially equitable
distribution of learning
opportunities
Iceland
Norway
480
Portugal
Russian Federation
Italy
Greece
460
Low average performance
LowIsrael
average performance
Large socio-economic disparities
High social equity
440
22
Low science performance
12
2
22
22School performance and socio-economic background
Student performance and students’ socio-economic background within schools
School performance and schools’ socio-economic background
Student performance and students’ socio-economic background
700
Schools proportional
to size
Student performance
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
Finland
500
300
-3
Disadvantage
-2
-1
0
1
PISA Index of socio-economic background
2
3
Advantage
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
24
24
Student performance and migration
Native students
600
First-generation students
Native students
Second-generation
OECDstudents
average = 500
550
500
450
First-generation students
400
350
300
PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.2a.
Second-generation students
PISA
Finland
Iceland
Norway
Sweden
Poland
Spain
Denmark
Ireland
Canada
Australia
New Zealand
United Kingdom
Mexico
Portugal
United States
Luxembourg
Korea
Switzerland
Turkey
Slovak Republic
Greece
60
Italy
Japan
Belgium
Netherlands
80
Hungary
100
Austria
Science Competencies
for Tomorrow’s World
120
Czech Republic
Germany
OECD Programme for
International Student Assessment
25
25
Is it all innate ability?
Variation in student performance
140
Performance variation
across schools
40
20
0
OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 4.1a
PISA
Finland
Iceland
Norway
Sweden
Poland
Spain
Denmark
Ireland
Canada
Australia
New Zealand
United Kingdom
Mexico
Portugal
United States
Luxembourg
Korea
Switzerland
Turkey
Slovak Republic
Greece
Italy
Japan
Science Competencies
for Tomorrow’s World
180
Belgium
Netherlands
Hungary
Austria
Czech Republic
Germany
OECD Programme for
International Student Assessment
26
26
Is it all innate ability?
Variation in student performance
160
140
120
100
80
60
40
20
0
PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.1a.
PISA
Finland
Iceland
Norway
Sweden
Poland
Spain
Denmark
Ireland
Canada
Australia
New Zealand
United Kingdom
Mexico
-60
Portugal
-40
United States
Luxembourg
Korea
Switzerland
Turkey
Slovak Republic
Greece
Italy
Japan
100
Belgium
Netherlands
Hungary
60
Austria
Science Competencies
for Tomorrow’s World
80
Czech Republic
Germany
OECD Programme for
International Student Assessment
27
27
Is it all innate ability?
Variation in student performance
Variation of
performance within
schools
40
20
0
-20
Variation of
performance between
schools
-80
PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.1a.
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
28
28
Money matters - but other things do too
Science
performance
575
Finland
550
Some school
and system factors
Japan
New Zealand Australia
Korea Germany Netherlands
Switzerland
Czech Republic
Austria
United Kingdom
Belgium
Ireland
Sweden
Hungary
Denmark United States
Poland
France
Slovak Republic
Spain
Iceland
Norway
Italy
Greece Portugal
525
500
475
450
Turkey
425
y = 0.0006x + 462
R² = 0.1904
Mexico
400
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
100000
Cumulative expenditure (US$ converted using PPPs)
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
29
29
School autonomy and standards-based
examination on science performance
School autonomy in selecting teachers for hire
70
PISA score
in science
60
50
41
40
30
20
10
Yes
0
No
No
Yes
School autonomy
in selecting teachers for hire
Standards based
external
examinations
PISA
OECD Programme for
International Student Assessment
Science Competencies
for Tomorrow’s World
30
30
Impact of selected student and school factors
on school performance
(after accounting for all other factors in the model)
%
100
90
80
70
60
50
40
30
20
10
0
10
20
30
Approx. one
school year
School principal’s positive
evaluation of quality of
Schools with more
educational materials
competing schools
(gross only)
(gross only)
Schools with greater
autonomy (resources)
(gross and net)
School activities to
One additional
of
promote
sciencehour
learning
self-study
(gross or
andhomework
net)
One additional hour of
(gross and net)
science learning at school
School
results
posted
(gross
and net)
publicly
(grossselective
and net)
Academically
schools (gross and net) but
no system-wide effect
Schools practicing ability
One
additional
outgrouping
(grosshour
and of
net)
of-school
lessons
Each
additional
10% of
School
principal’s
(gross
and net)
public funding
perception
that
lack of
(gross only)
Effect after accounting
qualified teachers hinders
for the socio-economic
instruction
Measured
Gross effect
Net
(gross only) background of students,
schools and countries
OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a