The Dynamic Trio
of Effective Teaching Measures:
Classroom observations, student surveys and
achievement gains
Thomas Kane
Harvard Graduate School of Education
Which team would you pick?
Or
 Largest study of instruction ever undertaken
 3,000 teacher volunteers in 7 districts
 Multiple Measures
– >22,500 classroom observations
– Student surveys
– Student achievement gains on state tests & supplemental tests
3
Student Achievement Gains
Student Achievement Gains
Bottom
25%
MATH
Top
25%
7.6 months
of learning
Classroom Observations
Actual scores for
7500 lessons.
6
Classroom Observations
Managing student behavior
Creating Environment
& rapport
Rapport
environment of Respect
respect &
1. UNSATISFACTORY
• Yes/no questions
• Posed in rapid succession
• Few students participate
2. BASIC
Engaging students in learning
Managing classroom procedures
Establishing
Establishingaaculture
cultureof
oflearning
learning
Using assessment in learning
• Some questions ask for
explanation
• Uneven attempts to engage all
students
Actual scores for
4. PROFICIENT
7500 ask
lessons.
• Most questions
for
explanation
• Discussion develops/teacher
steps aside
• All students participate
Using questioning & discussion techniques
4. ADVANCED
Communicating with students
• All questions high quality
• Students initiate questions
• Students engage each other
7
Better Observation Scores, Higher Achievement Gains
Student Achievement Gains
- 0.11
- 0.17
Top
Bottom
Managing
student
behavior
25% + 0.08
25%
Environment of respect & rapport
+ 0.02
Difference
0.19
0.19
- 0.14
Engaging students in learning
+ 0.08
0.22
- 0.07
Managing classroom procedures
+ 0.14
0.21
- 0.13
Establish culture of learning
+ 0.12
0.25
- 0.03
Using assessments
+ 0.06
0.09
- 0.11
Questioning and discussion
+ 0.02
0.13
- 0.07
Communicating with students
- 0.01
0.06
Student Perception Surveys
9
Student Perception Surveys
CARE: My teacher seems to know if something is bothering me.
CONTROL: My classmates behave the way the teacher wants them to.
CLARIFY: My teacher knows when the class understands.
CHALLENGE: In this class, we learn to correct our mistakes.
CAPTIVATE: I like the way we learn in this class.
CONFER: My teacher wants us to share our thoughts.
CONSOLIDATE: The comments I get help me know how to improve.
0
20
40
Bottom 25 %
60
Top 25 %
80
100
10
Better Student Perceptions, Higher Achievement Gains
Student Achievement Gains
Difference
0.38
Control
Top
+ 0.29
25%
+ 0.12
- 0.11
Clarify
+ 0.41
0.52
+ 0.08
Challenge
+ 0.57
0.49
- 0.11
Captivate
+ 0.31
0.42
- 0.14
Confer
+ 0.22
0.36
-0.04
Consolidate
+ 0.42
0.46
- Bottom
0.09
25%
- 0.36
Care
0.48
Student Perceptions = Highest Reliability
Correlation for same teacher on same measure, two different course sections
0
.2
.4
.6
Correlation for Same Teacher, Two Different Course Sections
Observation Score (FFT)
Observation
Scores
Value-Added
Student
Achievement
Gains
Student Survey
Student Survey
Results
Dynamic Trio
Value-Added
Most predicts
student outcomes,
No diagnostic use
Observations
Most diagnostic,
Potentially least
reliable
Student Surveys
Predictive,
Most reliable
13
Relative to What?
Months of Learning (MATH)
Master’s
Experience
Dynamic Trio
14
www.metproject.org