@ProfCoe www.twitter.com/ProfCoe
Evidence-based use of the pupil premium
Robert Coe
Closing the Gap in North Yorkshire, Harrogate, 27 June 2014

 What can research tell us about the likely impacts and costs of different strategies?
 How do we implement these strategies to …
1. Focus on what matters
3. Target areas of need
4. Produce demonstrable benefits
Improving Education: A triumph of hope over experience http://www.cem.org/attachments/publications/ImprovingEducation2013.pdf
2

3

www.educationendowmentfoundation.org.uk/toolkit
Most promising for raising attainment
8
Feedback
Meta-cognitive
May be worth it
Peer tutoring
0
Homework
(Secondary)
∂
Collaborative
Behaviour
Phonics
Small gp tuition
Social
Homework
ICT
Individualised learning
Parental involvement
Summer schools
Mentoring
1-1 tuition
Teaching assistants
(Primary)
Performance
Aspirations
£0 Setting pay
Cost per pupil
After school
£1000
Early Years
Smaller classes
Small effects / high cost

 Choose from the top left
 Go back to school and do it
∂
For every complex problem there is an answer that is clear, simple, and wrong
H.L. Mencken
5

 We have been doing some of these things for a long time, but have generally not seen improvement
 We do not know how to get large groups of
∂ teachers and schools to implement these interventions in ways that are
– faithful,
– effective
– sustainable
6

7

1. Focus on what matters
– Think hard about learning
2. Change classroom practice
3. Target areas of need
– Evaluate teaching quality
4. Produce demonstrable benefits
– Evaluate impact of changes

Think hard about learning

1. Reducing class size is one of the most effective ways to increase learning
[evidence]
2.
Differentiation and ‘personalised learning’ resources maximise learning
[evidence]
∂ persist with hard tasks
[evidence]
4. Technology supports learning by engaging and motivating learners
[evidence]
5. The best way to raise attainment is to enhance motivation and interest
[evidence]
10

www.educationendowmentfoundation.org.uk/toolkit
Most promising for raising attainment
8
Feedback
Meta-cognitive
May be worth it
Peer tutoring
0
Homework
(Secondary)
∂
Collaborative
Behaviour
Phonics
Small gp tuition
Social
Homework
ICT
Individualised learning
Parental involvement
Summer schools
Mentoring
1-1 tuition
Teaching assistants
(Primary)
Performance
Aspirations
£0 Setting pay
Cost per pupil
After school
£1000
Early Years
Smaller classes
Small effects / high cost

 Students are busy: lots of work is done (especially written work)
 Students are engaged, interested, motivated
 Students are getting attention: feedback, explanations
 Classroom is ordered, calm, under control
 Curriculum has been ‘covered’ (ie presented to students in some form)
 (At least some) students have supplied correct answers, even if they
– Have not really understood them
– Could not reproduce them independently
– Will have forgotten it by next week (tomorrow?)
– Already knew how to do this anyway
12

∂

 How many minutes does an average pupil on an average day spend really thinking hard?
 Do you really want pupils to be ‘stuck’ in your lessons?
 If they knew the right answer but didn’t know why, how many pupils would care?
14

Invest in effective CPD

 Teacher quality is what matters
 We need to focus on teacher learning
 Teachers learn just like other people
– Be clear what you want them to learn
– Get good information about where they are at
– Give good feedback
16

How do we get students to learn hard things?
Eg
 Place value
 Persuasive writing
 Music composition
 Balancing chemical equations
• Explain what they should do
• Demonstrate it
• Get them to do it (with
• Provide feedback
• Get them to practise until it is secure
• Assess their skill/ understanding

How do we get teachers to learn hard things?
Eg
 Using formative assessment
 Assertive discipline
 How to teach algebra
• Explain what they should do
∂

 Intense : at least 30 contact hours, preferably 50
 Sustained : over at least two terms
 Content focused : on teachers’ knowledge of
 subject content & how students learn it
∂
Active : opportunities to try it out & discuss
 Supported : external feedback and networks to improve and sustain
 Evidence based : promotes strategies supported by robust evaluation evidence

Evaluate teaching quality

 Good evidence of (potential) benefit from
– Performance feedback
(Coe, 2002)
– Target setting
(Locke & Latham, 2006)
– Accountability
(Coe & Sahlgren, 2014)
 Individual teachers matter most
 Teachers typically stop improving after 3-5 years
 Everyone can improve
 Judging real quality/effectiveness is very hard
– Multidimensional
– Not easily visible
– Confounded
21

 Progress in assessments
– Quality of assessment matters ( cem.org/blog )
– Regular, high quality assessment across curriculum ( InCAS ,
INSIGHT )
 Classroom observation
– Much harder than you think! ( cem.org/blog )
– Multiple observations/ers, trained and QA’d
 Student ratings ∂
– Extremely valuable, if done properly
( http://www.cem.org/latest/student-evaluation-of-teaching-canit-raise-attainment-in-secondary-schools )
 Other
– Parent ratings feedback
– Student work scrutiny
– Colleague perceptions (360)
– Self assessment
– Pedagogical content knowledge
22

 How do you know that it has captured understanding of key concepts?
– vs ‘check-list’ (eg ‘;’=L5, 3 tenses=L7)
 How do you know standards are comparable?
– Is progress good?
 How have you resolved tensions from teacher judgments being used to judge teachers?
– Summative assessment includes teacher feedback
23

1. Two teachers observe the same lesson, one rates it ‘Inadequate’. What is the probability the other will agree?
a) 10% b) 40% c) 60% d) 80%
∂
2.
An observer judges a lesson ‘Outstanding’.
What is the probability that pupils are really making sustained, outstanding progress?
a) 5% b) 30% c) 50% d) 70% www.cem.org/blog
24

Evidence-Based Lesson Observation
 Behaviour and organisation
– Maximise time on task, engagement, rules & consequences
 Classroom climate
∂ expectations, growth mindset
 Learning
– What made students think hard?
– Quality of: exposition, demonstration, scaffolding, feedback, practice, assessment
– What provided evidence of students’ understanding?
– How was this responded to? (Feedback)
25

Evaluate impact of changes

 Draws on knowledge and understanding of research to inform
– Pedagogical practice
∂
– Attempts to implement and embed more effective practices
 Robustly evaluates
– Its ongoing performance on a range of outcomes
– The impact of any changes made
27

∂
 Clear, well defined, replicable intervention
 Good assessment of appropriate outcomes
 Well-matched comparison group

RISE: Research-leads Improving Students’ Education
∂
 With Alex Quigley, John Tomsett, Stuart Kime
 Based around York
 RCT: 20 school leaders trained in research, 20 controls
 Contact: aj.quigley@huntington-ed.org.uk
29

www.cem.org
@ProfCoe
Robert.Coe@cem.dur.ac.uk
1. Think hard about learning
2. Invest in good CPD
3. Evaluate teaching quality
4. Evaluate impact of changes