Beyond test scores: the role of
primary schools in improving
multiple child outcomes
Claire Crawford and Anna Vignoles
Institute of Education, University of London
Motivation
• League tables measure school
performance in terms of academic
achievement e.g. test scores
• But Every Child Matters agenda suggests
schools should have a wider remit than
just increasing educational attainment
– e.g. improving health, socio-emotional skills,
etc
Research Questions
• Are schools at the top of the league tables
also the best at improving child welfare?
Or is there a trade-off?
• What methodological approaches are
most appropriate when measuring multiple
outcomes from education?
Previous research
• Small (but positive) impact of schools on
educational attainment once variation in pupil
characteristics is taken into account
– e.g. Goldstein and Sammons 1997; Reynolds et al.
1996; Leckie, 2008; Hanushek 1997
• Limited work on role of schools in promoting
wider outcomes, such as well being
– e.g Gibbons and Silva, 2009; Feinstein & Gutman
2008
• But different disciplines have used different
methods and comparisons of results is difficult
– e.g. educationalists relied on multi level models/
economists tend to use fixed effect models
Outline of today’s talk
• Use simple value-added model of
educational attainment to compare RE and
FE models
– Are assumptions underlying RE/multi level
model satisfied?
– Use results to inform later model development
Outline of today’s talk
• Document development of cognitive and
non-cognitive skills in primary school
– Can we think of these relationships as
causal?
• What are our next steps?
– How to consider joint determination of
cognitive and non-cognitive skills
RE vs. FE
• Start with a model of the following form:
Yist = α + βYist-1 + γXist-1 + δs + εist-1
• Where:
– Y: Key Stage maths points
– X: vector of pupil characteristics (including family composition,
parents’ education, parents’ socio-economic status, etc)
• RE model assumes: E[Xist-1|δs] = 0
– i.e. pupil characteristics are unrelated to the school effects
• If E[Xist-1|δs] ≠ 0, RE model produces biased estimates
– Rely on FE model instead
– Use Hausman test to judge whether condition has been met
• Future work will explore other tests as well
Determinants of KS2 maths score
OLS
FE
RE
KS1 maths score
0.113**
[0.003]
0.114**
[0.003]
0.113**
[0.003]
2nd SES quintile
-0.015
[0.027]
-0.009
[0.028]
-0.015
[0.027]
3rd SES quintile
-0.013
[0.028]
-0.010
[0.029]
-0.004
[0.028]
4th SES quintile
-0.045
[0.029]
-0.030
[0.030]
-0.037
[0.029]
Most deprived SES quintile
-0.127**
[0.032]
-0.070**
[0.033]
-0.104**
[0.032]
Number of schools
• Hausman test rejects RE model
414
Modelling cognitive and noncognitive skills
• Focus on value-added models between ages 7 and 11:
KSist = α + β1KSist-1 + β2SDQist-1 + γXist-1 + δs + εist-1
SDQist = α + β1KSist-1 + β2SDQist-1 + γXist-1 + δs + εist-1
• KS: Key Stage test results
– Standardised average of reading, writing, maths at age 7
– Standardised average of English, maths, science at age 11
• SDQ: Strengths & Difficulties score
– Teacher reports
– 5 questions on 5 topics: pro-social, hyperactivity, emotional
symptoms, conduct problems, peer problems
– Invert pro-social and combine to create total “difficulties” score
– Standardise total “difficulties” score and invert so SDQ measures
“strengths” rather than “difficulties”
Model controls
• X is a vector of pupil characteristics, including:
– Month of birth
– Ethnicity
– Family composition (including twin status, number of
older and younger siblings, whether parents are
married or cohabiting)
– Parents’ education
– Parents’ socio-economic status
– Birthweight and whether child was breastfed
– Whether parents often read to the child at age 3
• δ represent school fixed effects
Model estimation
• Estimate using seemingly unrelated
regression analysis
– Takes account of the fact that the error terms
across the two equations may be correlated
– Can extend to consider more outcomes
• e.g. may additionally want to consider health?
Development of cognitive and noncognitive skills
No school effects
School fixed effects
Std KS2 APS
Std KS1 APS
0.546**
[0.010]
0.542**
[0.010]
Std SDQ score (age 7)
0.097**
[0.009]
0.115**
[0.009]
Std SDQ score (age 11)
Std KS1 APS
0.221**
[0.013]
0.219**
[0.014]
Std SDQ score (age 7)
0.267**
[0.011]
0.304**
[0.013]
Causal effects?
• Instrument KS1 attainment using month of birth to
identify causal effects of KS1 on KS2 and SDQ scores
Std KS2 APS
Std SDQ score
Using std KS1 APS
Std KS1 APS
0.542**
[0.010]
0.219**
[0.014]
Std SDQ score (age 7)
0.115**
[0.009]
0.304**
[0.013]
Using MOB as instrument for std KS1 APS
Std KS1 APS
(instrument using MOB)
0.559**
[0.011]
0.075
[0.083]
Std SDQ score (age 7)
0.109**
[0.010]
0.328**
[0.018]
Summary
• Tested RE model with rich set of controls
– Rejected RE model, so focus on FE models instead
• Investigated development of cognitive and noncognitive skills between ages 7 and 11
– Found strong statistical relationships
• Inclusion of school fixed effects marginally increases the
relationship between SDQ scores at 7 and 11
– Impact of attainment at age 7 on KS2 APS appears
causal; but impact of attainment at age 7 on SDQ
score at age 11 is not
Next steps
• Further robustness checks surrounding
rejection of RE model, using other models
and data
• Use other statistical software to run
seemingly unrelated IV regressions
• Consider first difference models of KS and
SDQ scores
– Instrument change in SDQ score with death of
a parent/close family member