Teacher and
Principal
Effectiveness:
What Do We Know?
Philanthropy Roundtable
Atlanta, GA
September, 2009
1. There are much bigger
differences among our teachers than
we ever knew. And those
differences matter hugely to
students.
2008THE
THEEDUCATION
EDUCATION TRUST
©©2009
TRUST
10 Point Average Difference Between
Top and Bottom Teachers
Source: Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job.
Washington, D.C.: The Brookings Institution.
© 2009 THE EDUCATION TRUST
Comparing the Average Student in the
Classrooms of Bottom Quartile and Top
Quartile Teachers
Percentile Points Gained/Loss
6
4
Bottom Quartile
Top Quartile
2
10
0
-2
-4
-6
Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job.
Washington, D.C.: The Brookings Institution.
© 2009 THE EDUCATION TRUST
Students in Dallas Gain More in Math with
Effective Teachers: One Year Growth From
3rd-4th Grade
© 2009
THE EDUCATION
TRUST
Source: Heather Jordan, Robert Mendro, and Dash Weerasinghe, The Effects of Teachers on Longitudinal Student
Achievement,
1997.
LOW ACHIEVING STUDENTS IN TN GAIN
MORE WITH EFFECTIVE TEACHERS: One
Year Growth
60
53
50
40
30
20
14
10
0
low
high
Sanders and Rivers, Cumulative and Residual Effects of Teachers on Future Academic Achievement, 1998.
© 2009 THE EDUCATION TRUST
2. Impact of effective
teachers swamps almost every
other “intervention,” including
class size reduction.
©
© 2009
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
Cumulative Teacher Effects On Students’
Math Scores in Dallas (Grades 3-5)
Beginning Grade 3
Percentile Rank= 57
Beginning Grade 3
Percentile Rank= 55
© 2009
THE EDUCATION
TRUST
Source: Heather Jordan, Robert Mendro, and Dash Weerasinghe, The Effects of Teachers on Longitudinal Student
Achievement,
1997.
3. Though there are large
differences among our teachers we
pretend that there aren’t.
©
© 2009
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
The Widget Effect
“When it comes to measuring instructional performance,
current policies and systems overlook significant differences
between teachers. There is little or no differentiation of
excellent teaching from good, good from fair, or fair from
poor. This is the Widget Effect: a tendency to treat all
teachers as roughly interchangeable, even when their
teaching is quite variable. Consequently, teachers are not
developed as professionals with individual strengths and
capabilities, and poor performance is rarely identified or
addressed.”
• The New Teacher Project, 2009
© 2009 THE EDUCATION TRUST
Source:
Source:
4. Good teachers are not fairly
distributed.
©
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
© 2009
More Classes in High-Poverty Secondary
Schools Taught by Out-of-Field* Teachers
National
HighPoverty
Lowpoverty
Note: High Poverty school-75% or more of the students are eligible for free/reduced price lunch. Low-poverty school -15% or fewer of the
students are eligible for free/reduced price lunch.
*Teachers with neither certification nor major. Data for secondary-level core academic classes (Math, Science, Social Studies, English) across USA.
Source: Analysis of 2003-2004 Schools and Staffing Survey data by Richard Ingersoll, University of Pennsylvania 2007.© 2009 THE EDUCATION TRUST
Math Classes at High-Poverty and High- Minority Schools
More Likely to be Taught by Out of Field* Teachers
Note: High Poverty school-75% or more of the students are eligible for free/reduced price lunch. Low-poverty school -15% or fewer of the students are
eligible for free/reduced price lunch. High minority school-75% or more of the students are Black, Hispanic, American Indian or Alaskan Native, Asian or
Pacific Islander. Low-minority school -10% or fewer of the students are non-White students.
*Teachers with neither certification nor major. Data for secondary-level core academic classes (Math, Science, Social Studies, English) across USA.
Source: Analysis of 2003-2004 Schools and Staffing Survey data by Richard Ingersoll, University of Pennsylvania 2007.© 2009 THE EDUCATION TRUST
Students at High-Minority Schools More
Likely to Be Taught By Novice* Teachers
Note: High minority school-75% or more of the students are Black, Hispanic, American Indian or Alaskan Native, Asian or Pacific Islander.
Low-minority school -10% or fewer of the students are non-White students.
*Novice teachers are those with three years or fewer experience.
Source: Analysis of 2003-2004 Schools and Staffing Survey data by Richard Ingersoll, University of Pennsylvania 2007.© 2009 THE EDUCATION TRUST
© 2009 THE
EDUCATION
TRUST
1998
by The Education
Trust,
Tennessee: High poverty/high minority schools have fewer
of the “most effective” teachers and more “least effective”
teachers
Note: High Poverty/High minority means at least 75% qualify for FRPL and at least 75% are minority.
Source: Tennessee Department of Education 2007. “Tennessee’s Most Effective Teachers: Are they assigned to the schools that need them
© 2009 THE EDUCATION TRUST
most?” http://tennessee.gov/education/nclb/doc/TeacherEffectiveness2007_03.pdf
Low-Achieving Students are More Likely to be Assigned to
Ineffective Teachers than Effective Teachers
Source: Sitha Babu and Robert Mendro, Teacher Accountability: HLM-Based Teacher Effectiveness Indices in the Investigation of
© 2009 THE EDUCATION TRUST
Teacher Effects on Student Achievement in a State Assessment Program, AERA Annual Meeting, 2003.
4. We don’t know nearly enough
about the characteristics of effective
teachers…and even less about their
practices.
©
© 2009
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
In last few years, volume of
studies has grown rather
dramatically. And we are
beginning to learn some of the
basics.
© 2009 THE EDUCATION TRUST
What are we learning?
• What Seems to
Matter
• Experience, but only
for first year or two;
• Content knowledge,
at least in math and
science;
• Teachers’ own test
performance.
• What Doesn’t Seem
to Matter
• Traditional vs.
Alternate
Certification;
• Masters Degrees;
• Experience beyond
the first several
years.
© 2009 THE EDUCATION TRUST
But there is still a lot more
digging to do, if only to
understand what seem to be
exceptions…
© 2009 THE EDUCATION TRUST
For example, while novices usually
aren’t as good as teachers with at
least 3 years experience….
• Studies in Louisiana show that some programs
produce teachers who are more effective in
year one, than veteran teachers in same
district;
• Recent study in North Carolina found Teach
For America Corps Members who taught in
secondary schools as effective as veteran
teachers in those same schools.
© 2009 THE EDUCATION TRUST
And within any “type” of teacher,
there is a wide range of
effectiveness.
© 2009 THE EDUCATION TRUST
Similar Effectiveness,
Regardless of Certification
Source: Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job.
Washington, D.C.: The Brookings Institution.
© 2009 THE EDUCATION TRUST
Some nuances…
• Traditional a little better with younger
children, especially in reading;
• Alternates a little better with older children,
especially in math;
• Most differences in lower grades wash out by
year 3.
© 2009 THE EDUCATION TRUST
Effectiveness More Important
than Certification
“The difference between the 75th percentile
teacher and the 50th percentile teacher for all
three groups of teachers was roughly five
times as large as the difference between the
average certified teacher and the average
uncertified teacher.”
Three groups = traditionally certified, alternatively certified, and uncertified
Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job.
Washington, D.C.: The Brookings Institution.
© 2009 THE EDUCATION TRUST
Bottom line: Most proxies for
teacher effectiveness—especially
things like Masters Degrees or
average years of experience-don’t turn out to mean very
much.
• That’s why incorporating and using
value-added measures is so
important.
© 2009 THE EDUCATION TRUST
5. Principals are critical—at least in
part because they are the ones who
attract and hold good teachers.
2008THE
THEEDUCATION
EDUCATION TRUST
©©2009
TRUST
But if you think we know
shockingly little about good
teachers…
© 2009 THE EDUCATION TRUST
You will be thoroughly
despondent about how little we
know about effective principals,
other than that they are
RELENTLESS and GOOD TEAM
BUILDERS, and that ed leadership
programs DON’T PRODUCE VERY
GOOD ONES.
© 2009 THE EDUCATION TRUST
6. The bottom line, though, is very clear:
if we are going to improve achievement
and close gaps, we’ve got to act on what
we know, while also digging deeper for
insights and answers.
©
© 2009
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
And we’ve got to provide generous
support for the break-the-mold
teacher and principal selection and
preparation programs.
©
© 2009
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
Why?
© 2009 THE EDUCATION TRUST
A few years ago, we got a wake up
call when the 2000 PISA results were
published.
©
© 2009
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
PISA Performance
U.S.A. Ranks Near Bottom, Has Fallen Since 2000
2000 Rank
2003 Rank
2006 Rank
(out of 26)
(out of 26)
(out of 26)
Mathematics
17th
22nd
22nd
Science
13th
Tied for 17th
19th
Subject
Rankings are for the 26 OECD countries participating in PISA in 2000, 2003, and 2006.
© 2009 THE EDUCATION TRUST
Source: Organization for Economic Cooperation and Development (OECD), PISA 2006 Results, http://www.oecd.org/
PISA 2003 Math
Of 29 OECD Countries, U.S.A. Ranked 24th
U.S.A.
© 2009 THE EDUCATION TRUST
Source: Organization for Economic Cooperation and Development (OECD), PISA 2003 Results, http://www.oecd.org/
PISA 2006 Science
Of 30 OECD Countries, U.S.A. Ranked 21st
U.S.A.
Higher than U.S. average
Not measurably different from U.S. average
Source: NCES, PISA 2006 Results, http://nces.ed.gov/surveys/pisa/
Lower than U.S. average
© 2009 THE EDUCATION TRUST
PISA 2003 Problem-Solving
U.S. Ranks 24th Out of 29 OECD Countries
U.S.A.
© 2009 THE EDUCATION TRUST
Source: Organization for Economic Cooperation and Development (OECD), PISA 2003 Results, http://www.oecd.org/
Only place we rank high?
Inequality.
© 2009 THE EDUCATION TRUST
PISA 2003: Gaps in Performance Of U.S.15
Year-Olds Are Among the Largest of OECD
Countries
Mathematical Literacy
Problem Solving
Rank in Performance
Gaps Between Highest
and Lowest Achieving
Students *
8th
6th
*Of 29 OECD countries, based on scores of students at the 5th and
95th percentiles.
Source: Organization for Economic Cooperation and Development (OECD), PISA 2003 Results, data available at
http://www.oecd.org/
© 2009 THE EDUCATION TRUST
Among OECD Countries, U.S.A. has the 4th Largest Gap
Between High-SES and Low-SES Students
PISA 2006 Science
U.S.A.
Source: OECD, PISA 2006 Results, table 4.8b, http://www.oecd.org/
© 2009 THE EDUCATION TRUST
Fortunately, we are making some
progress—especially at the
elementary and middle school
levels.
©
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
© 2009
4th Grade Reading:
Record Performance with Gap
Narrowing
*Denotes previous assessment format
• NAEP 2008 Trends in
© 2009 THE EDUCATION TRUST
4th Grade Math:
Record Performance with Gap
Narrowing
*Denotes previous assessment format
• NAEP 2008 Trends in
© 2009 THE EDUCATION TRUST
8th Grade Reading: Recent Gap
Narrowing for Blacks, Less for Latinos
*Denotes previous assessment format
• NAEP 2008 Trends in
© 2009 THE EDUCATION TRUST
8th Grade Math:
Progress for All Groups, Some Gap Narrowing
*Denotes previous assessment format
• NAEP 2008 Trends in
© 2009 THE EDUCATION TRUST
And much of that progress is
being led by urban school
districts, including the one right
here in Atlanta.
© 2009 THE EDUCATION TRUST
NAEP Grade 4 Reading – African American
Districts Outperforming Their States
in Movement into Basic (2003-2007)
District
District Movement State Movement
Atlanta
9
6
Los Angeles
7
5
San Diego
6
5
Note: Data refer to the percentage point increase in the percent of students at Basic and Above between 2003 and 2007.
• NAEP Data Explorer, NCES
© 2009 THE EDUCATION TRUST
NAEP Grade 4 Math – African
American
Districts Outperforming Their States
in Movement into Basic (2003-2007)
District
District Movement
State Movement
Boston
16
13
New York City
14
11
San Diego
11
7
Atlanta
10
8
Houston
7
5
Charlotte
2
0
Note: Data refer to the percentage point increase in the percent of students at Basic and Above between 2003 and 2007.
• NAEP Data Explorer, NCES
© 2009 THE EDUCATION TRUST
NAEP Grade 8 Reading – African
American
Districts Outperforming Their States
in Movement into Basic (2003-2007)
District
District Movement
State Movement
Houston
9
5
Boston
7
3
Atlanta
6
2
Cleveland
6
-3
San Diego
2
-1
Charlotte
1
-3
Note: Data refer to the percentage point increase in the percent of students at Basic and Above between 2003 and 2007.
• NAEP Data Explorer, NCES
© 2009 THE EDUCATION TRUST
We can’t go the distance, though,
without strong teachers.
©
© 2009
2009 THE
THE EDUCATION
EDUCATION TRUST
TRUST
“Massive Impact”
“If the effects were to accumulate, having a
top-quartile teacher rather than a bottom
quartile teacher four years in a row would be:
– enough to close the black-white test score gap…; and,
– Have twice the impact of reducing class size from 22 to
16.”
Source: Gordon, R., Kane, T.J., and Staiger, D.O. (2006). Identifying Effective teachers Using Performance on the Job.
Washington, D.C.: The Brookings Institution.
© 2009 THE EDUCATION TRUST
For more information, visit www.edtrust.org
Send a team to our 2009 Conference
WHEN THE GOING GETS TOUGH:
Smart Choices and Bold Action
to Raise Achievement and Close Gaps
November 12-14, 2009, in Arlington, Va.
1250 H Street N.W. Suite 700
Washington, D.C. 20005
202/293-1217