Education for Innovation: the Role of Arts and STEM Education
OECD/France Workshop – Paris, 23-24 May 2011
Effective Teaching for Improving
Students’ Motivation, Curiosity,
and Self-Confidence in Science
Evidence from PISA 2006
Francesco Avvisati
OECD Centre for Educational
Research and Innovation (CERI)
Motivation
Motivation
Outline
• PISA measures of behavioural skills
• The paradox of international comparisons
of interest, curiosity, and self-confidence
– between and within-country evidence
• Teaching activities’ effectiveness on
students’ learning
– within-country evidence
PISA measures of behavioural skills
• PISA Test
– interest in science topics (“embedded”)
• PISA Context Questionnaire
– Enjoyment of science
– Science self-concept
– Science self-efficacy
Part I
THE TEST-SCORE / INTEREST
PARADOX
The Test-Score/Interest Paradox
Interest in Science Topics
620
IDN
MEX
600
BRA
CHL
580
PRT
560
540
GRC
RUS
TUR
ITA
520
500
480
460
FRA
MAC
DEU JPN
HUN
AUT
LUX
SVN
EST
POL
BEL CHE
CZE
KOR
IRL
USA
CAN
NOR
GBR AUS
ISL
NZL
DNK
SWE
NLD
SVK
ISR
HKG
ESP
FIN
440
380
400
420
440
460
480
500
Science Score
520
540
560
580
Youth Suicide Rate (15-24)
The Test-Score/Interest Paradox
A Logical Fallacy
25
KOR
20
HUN
JPN
BEL
FIN
FRA
15
AUT
POL
CZE
CHE
ISL
NOR DNKSWE
USA
10
AUS
DEU
NZL
CAN
IRL
PRT
LUX
NLD
ITA
5
ESP
GBR
MEX
GRC
0
400
420
440
460
480
500
Science Score
520
540
560
580
-0.8
JPN
KOR
0
-0.4
BRA
MEX
0.8
RUS
IDN
SWE
GBR
GRC
NOR
FIN
POL
AUT
CHE
ISL
EST
SVK
TUR
ITA
CHL
USA
SVN
LUX
CAN
ISR
DEU
PRT
0.4
HKG
MAC
NLD
HUN
BEL
IRL
FRA
DNK
NZL
CZE
AUS
ESP
Authentic attitudes or fake postures?
Science Self-Concept
-1.2
• Social desirability bias?
The Test-Score/Interest Paradox
Robustness
• Interest is less sensitive to cultural biases
• Adding confounders:
0
-0.5
-1
0.5
HKG
IDN
MAC
BRA
RUS
0.3
IDN
MAC
HKG
MEX
HUN
PRT
USA
EST
TUR
SVK
CZE
ITA
ESP
ISR
GRC
LUX
AUT
DEU
BEL
NZL
NLD
GBR
IRL
CHE
FRA
AUS
CAN
KOR
SVN
DNK
JPN
SWE
FIN
NOR
ISL
0.6
BRA
RUS
1
GBR
SVN
GRC
CAN
DEU
FIN
IRL
AUS
CHE
LUX
FRA
DNK
SWE
NOR
ISL
JPN
KOR
-0.3
CHL
POL
0
CHL
USA
PRT
ISR
ITA
POL
BEL
NZL
MEX
TUR
NLD
SVK
ESP
CZE
HUN
AUT
EST
The Test-Score/Interest Paradox
Interpretation
within-school correlation of individual interest and scores
-0.6
between-school correlation of average interest and scores
The Test-Score/Interest Paradox
Conclusion
• Some school cultures (teaching cultures?)
have diverging effects on interest and
scores
– High stake testing? (extrinsic motivation)
– Low academic standards?
– Teaching to the test, lack of cognitive
activation?
Japan, Germany, Korea, France –
United States, Chile, Israel, Portugal
Part II
EFFECTIVE TEACHING
Effective Teaching: Motivation
• What teaching activities are associated
with better test-scores?
(subject-based competences)
• What type of teaching develops
motivation, curiosity and self-confidence
the most?
(behavioural competences)
Effective Teaching: Data
student reports on 4 clusters of activities
• Interaction
Collaboration and
participatory
exchanges
• Application
Drawing connections
between school science
and the outside world
• Hands-on
Guided activities
around lab
experiments
• Investigation
Autonomous
student inquiries
Effective Teaching: Data
• example: focus on models & applications
Effective Teaching: Methods
Problems with interpreting correlations causally:
• Pupils differ ex-ante (correlation might be
spurious)
– Control for observable individual and peer
characteristics; control for reading scores.
– Explore robustness across countries
• Measurement error and reverse causality
in reports of teaching activities
– “back-up” reports with peer reports
(instrumental variables estimation)
Effective Teaching: Results
Science Score
0.3
0.15
0
-0.15
4
8
-1
-2
1
-2
-10
application
hands-on
interaction investigation
Effective Teaching: Results
Interest in Science Topics
0.3
20
0.15
3
0
0.15
application
6
-2
-1
-1
hands-on
interaction
investigation
Effective Teaching: Results
Science Enjoyment
0.3
26
0.15
11
1
0
-0.15
-1
application
hands-on
2
-3
interaction investigation
Effective Teaching: Results
Science Self-Efficacy
0.3
0.15
15
4
5
4
0
-0.15
application
hands-on
interaction investigation
Effective Teaching: Interpretation
• How big are effect sizes?
• example: focus on models & applications
How to interpret a unit increase?
(+1 standard deviation)
activity
at index = -.5
at index = +.5
1) Apply ideas to different phenomena
2.8
1.3
2) Relate to outside world
4.8
2.8
3) Explain relevance for student lives
4.0
2.2
4) Technological applications
5.1
3.1
every…
…lessons
based on: “in all lessons” = every single lsn, “in most lsns” = every 2 lsns,
“in some lsns” = every 5 lsns, “never or hardly ever” = every 10 lsns.
Conclusion
• International comparisons do not need to
limit themselves to “cognitive” test scores;
• Patterns diverge between subject-based
skills and science-related attitudes;
• Inquiry based activities: effectiveness
hinges critically on guidance;
• Application activities foster positive
attitudes to learning and self.
francesco.avvisati@oecd.org
THANK YOU