Beyond the teaching and learning toolkit putting evidence to work
Inspiring Leadership Conference
Birmingham 12th June 2014
Professor Rob Coe and Kevan Collins
Kevan.collins@eefoundation.org.uk
www.educationendowmentfoundation.org.uk
The next 60 minutes…
• The case for an evidence led approach
• Putting evidence to work
 lessons from lesson observation
 lessons from transition studies
• An invitation to join the ‘quiet revolution’
Teaching and Learning Toolkit
• The Toolkit is an accessible, teacher-friendly summary of
educational research. ‘Which?’ for education
• Practice focused: tries to
give schools the
information they need to
make informed decisions
and narrow the gap.
• Based on meta-analyses
conducted by Durham
University.
Using the Toolkit
Use the evidence as a starting
point for discussion.
Dig deeper into what the
evidence actually says
Understand the ‘active
ingredients’ of implementation
The Toolkit is a starting point for
making decisions
Do teachers need to know
about research?
Lessons from Lesson Observation
Knowledge of research
Put these in order of effectiveness:
A. A one-to-one numeracy intervention (two 15-minute
sessions per week, delivered by teaching assistants) for
Year 2-6 pupils who are struggling with numeracy
(outcome: maths)
B. Nine weekly one-hour sessions where Y7 pupils below L4
read and discuss an age-appropriate book, with tools and
resources to encourage reading for pleasure (outcome:
reading)
C. A four-week summer school programme (between Y6 & 7)
for pupils who had been predicted to achieve KS2 below
Level 4b in English, focussed on poetry and writing
(outcome: writing).
D. Y6 & 7 teachers trained to deliver a programme to help low
attaining pupils plan, monitor and evaluate their writing
using memorable experiences, eg trips and visitors
(outcome: writing).
7
Improving Teaching
• Teacher quality is what matters
• We need to focus on teacher learning
• Teachers learn just like other people
– Be clear what you want them to learn
– Get good information about where they
are at
– Give good feedback
8
Sept 2013: ResearchEd
Do We Know a Successful Teacher
When We See One?
• Filmed lessons (or short clips) of effective (value-added)
and ineffective teachers shown to
– School Principals and Vice-Principals
– Teachers
– Public
• Some agreement among raters, but unable to identify
effective teaching
• No difference between education experts and others
• Training in CLASS did help a bit
Strong et al 2011 J Tchr Edn
10
Reliability
1st rater gives
%
Outstanding
Good
Req. Impr.
12%
55%
29%
Inadequate
4%
Overall
Probability that 2nd rater
disagrees
Best case
Worst case
r = 0.7
r = 0.24
51%
78%
31%
43%
46%
64%
62%
90%
39%
55%
Percentages based on simulations
11
Validity
Probability value-added
data disagrees
1st rater gives
%
Best case
r = 0.4
Worst case
r = -0.3
Outstanding
12%
71%
96%
Good
55%
40%
45%
Req. improv.
29%
59%
79%
Inadequate
4%
83%
>99%
51%
63%
Overall
Percentages based on simulations
12
How can something
that feels so right
be so wrong?
cem.org/blog
Obvious – but not true
Why do we believe we can spot good
teaching?
• We absolutely know what we like
– Strong emotional response to particular
behaviours/styles is hard to over-rule
• We focus on observable proxies for learning
– Learning is invisible
• Preferences for particular pedagogies are widely
shared, but evidence/understanding of their
effectiveness is limited
• We assume that if you can do it you can spot it
• We don’t believe observation can miss so much
14
Poor Proxies for Learning
• Students are busy: lots of work is done (especially
written work)
• Students are engaged, interested, motivated
• Students are getting attention: feedback,
explanations
• Classroom is ordered, calm, under control
• Curriculum has been ‘covered’ (ie presented to
students in some form)
• (At least some) students have supplied correct
answers (whether or not they really understood them
or could reproduce them independently)
15
Evidence based advice
Judgements from lesson observation may be used
for low-stakes interpretations (eg to advise on
areas for improvement) if at least two observers
independently observe a total of at least six
lessons, provided those observers have been
trained and quality assured by a rigorous process
(2-3 days training & exam).
High-stakes inference (eg Ofsted grading,
competence) should not be based on lesson
observation alone, no matter how it is done.
What does ‘good teaching’
look like?
Improving Teaching
• Teacher quality is what matters
• We need to focus on teacher learning
• Teachers learn just like other people
– Be clear what you want them to learn
– Get good information about where they
are at
– Give good feedback
18
19
Danielson Framework
http://danielsongroup.org/framework/
1. Planning and preparation
a. Demonstrating Knowledge of
Content and Pedagogy
b. Demonstrating Knowledge of
Students
c. Setting Instructional Outcomes
d. Demonstrating Knowledge of
Resources
e. Designing Coherent Instruction
f. Designing Student
Assessments
2. Classroom environment
a. Creating an Environment of
Respect and Rapport
b. Establishing a Culture for
Learning
c. Managing Classroom
Procedures
d. Managing Student Behavior
e. Organizing Physical Space
3. Instruction
a. Communicating with Students
b. Using Questioning and
Discussion Techniques
c. Engaging Students in Learning
d. Using Assessment in Instruction
e. Demonstrating Flexibility and
Responsiveness
4. Professional
responsibilities
a.
b.
c.
d.
Reflecting on Teaching
Maintaining Accurate Records
Communicating with Families
Participating in the Professional
Community
e. Growing and Developing
Professionally
f. Showing Professionalism
20
Evidence-based standards for
effective teaching?
• Evidence about relationships between teacher skills,
knowledge & behaviours and ‘effectiveness’
• Evidence about what can be changed (and how)
• Based on ‘best’ theories of
– Pupil learning
– Pedagogy & teaching effectiveness
– Behaviour change (individual, institutional, systemic)
• Most important: does focusing on these things lead
to improvement?
21
Improving Teaching
• Teacher quality is what matters
• We need to focus on teacher learning
• Teachers learn just like other people
– Be clear what you want them to learn
– Get good information about where they
are at
– Give good feedback
22
Why monitor teaching quality?
• Strong evidence of (potential) benefit from
– Performance feedback (Coe, 2002)
– Target setting (Locke & Latham, 2006)
– Intelligent accountability (Wiliam 2010)
• Individual teachers matter most
• Everyone can improve
• Teachers stop improving after 3-5 years
• Judging real quality/effectiveness is very hard
– Multidimensional
– Not easily visible
– Confounded
23
Monitoring the quality of teaching
• Progress in assessments
– Quality of assessment matters (cem.org/blog)
– Regular, high quality assessment across curriculum (InCAS,
INSIGHT)
• Classroom observation
– Much harder than you think! (cem.org/blog)
– Multiple observations/ers, trained and QA’d
• Student ratings
– Extremely valuable, if done properly
(http://www.cem.org/latest/student-evaluation-of-teachingcan-it-raise-attainment-in-secondary-schools)
• Other
– Parent ratings feedback
– Student work scrutiny
– Colleague perceptions (360)
– Self assessment
– Pedagogical content knowledge
24
Lessons from transition studies
The level 4 jeopardy…
KS2 Outcomes 2008
Achieving
L4 +
447,733
Tracked GCSE outcomes 2013
Not
achieving
level 4
93,161
,
Not
achieving 5
A*-C (EM),
82,829
Focusing on transition
In 2012 with government support the EEF funded 24
transition studies involving 1:10 secondary schools and
many of their partner primaries and just under 17,000
pupils.
We asked schools to bring the best of their literacy
transition work for evaluation and testing
Funded programmes included commercial products, school
grown solutions, in and out of school activities
7 Lessons from transition studies
1.The educational chances of pupils who begin secondary school
without having achieved Level 4 in literacy are very poor. In 2013,
72,000 children began secondary school without a Level 4 in reading. If
this cohort follows previous trends, approximately 1 in 10 will achieve
5A*-C, including English and Maths at GCSE.
2. Due to the size of the gap, helping struggling readers catch-up with
their peers in Year 6 is very challenging. It is highly unlikely that any
single approach will be sufficient to close it.
3.Supporting struggling readers is likely to require a concerted effort
across the curriculum, and a combination of approaches.
7 Lessons from transition studies
4. When viewed in isolation some approaches seem to be more
promising than others. On average, reading comprehension
approaches appear to be more effective than phonics or oral language
approaches for low attaining older readers. It may be that children who
have not succeeded using phonics previously benefit from approaches
which place a greater emphasis on meaning and context.
5.The effectiveness of any approach to reading catch-up is related to
the pupil’s stage of reading development, so it is important that staff
have the skill and training in diagnostic assessment, as well as in
delivering any particular intervention.
7 Lessons from transition studies
6. Summer schools can improve reading ability but their effectiveness
will be limited by the quality of teaching which takes place. In addition, it
is likely that other approaches delivered in school will be more costeffective.
7. Though some progress is possible, tackling reading catch-up at the
transition is extremely difficult. The findings of this summary underline
the importance of effective early intervention in Key Stage 1 and the
beginning of Key Stage
Conclusions
1. The new focus on evidence will support informed
professional debate – it’s not a panacea
2. Implementation is all and putting evidence to work
requires informed and engaged leadership
3. Enduring question - why is the education sector so
weak at spreading and sharing lessons from disciplined
and informed innovation?
Taking part in EEF research: Kevan.collins@eefoundation.org.uk