How We Did The Story
“Grading the Teachers”
Molly Bloom / StateImpact Ohio * Patrick O’Donnell / The Cleveland Plain Dealer
@m_bloom * mollypbloom@gmail.com * PaODonnell@plaind.com
The Series
Read
●
●
●
●
●
Overview: Using Data To Evaluate Teachers
Pay vs. Value-Added Performance
Secrets Of Two “Most Effective” Teachers
Value-Added’s Poverty Factor
How is Value-Added Calculated?
Multimedia
●
●
●
Audio: Measuring Performance Through Growth
Audio: Push for Performance Pay
Video: Guide to Ohio’s New Way of Evaluating
Teachers
Data
●
●
●
●
Search Ratings For 4,200 Ohio Teachers
Map: See Teacher Ratings at Your School
Why We Published Teachers’ Ratings
About Our Analysis
Grading the Teachers: Where the
“Most Effective” Teachers Are (MAP)
Search Teacher Ratings
http://www.cleveland.com/datacentral/index.ssf/
2013/06/find_ohio_teacher_ratings_base.htm
l
http://stateimpact.npr.org/ohio/2013/06/16/gradi
ng-the-teachers-search-value-added-gradesfor-4200-ohio-teachers/
What We Found
● Some teachers find new evaluations useful,
some don’t
● Little relationship between value-added
scores and teacher pay
● Clear relationship between value-added
scores and teachers’ ages
● Clear relationship between the % of poor
kids in a school and teachers’ value-added
scores
The department’s top research official, Matt Cohen,
acknowledged that he can’t explain the details of
exactly how Ohio’s value-added model works. He
said that’s not a problem.
“It’s not important for me to be able to be the expert,”
he said. “I rely on the expertise of people who have
been involved in the field.” •
How It Began
We had some questions
about value-added…
1) Is it really independent of student family income?
2) Does it show teachers getting better over time? In
keeping with their pay increases?
3) Does it show any difference between teacher prep
programs?
4) Does a masters degree improve teacher performance,
as shown by value-added?
State’s contention: Performance
may be related to income…
But value-added isn’t.
Teachers who
receive top valueadded rating are
more likely to be in
schools with fewer
poor students
How We Did It
● Talking with teachers
● Interviewing experts
● Working with data
Teachers
Meet Maria Plecnik
Experts
Seriously: Value-Added is complicated
Data
● Know what you want to ask the data
● Get outside advice
● Bulletproof it
● Few people care about the details
This was our post explaining the
analysis. Almost no one read it.
StateImpact Ohio and The Plain Dealer analyzed the relationships between a statistical
measure called value-added and teachers’ pay, and between value-added and teachers’ ages
using data obtained from the Ohio Department of Education.
Value-added looks at how much an individual teacher’s students learn in a given year,
regardless of their level at the start of the year. Value-added is a key part of Ohio’s new teacher
evaluation system, but is not an overall measure of teacher performance.
The analysis included value-added scores and ratings for about 16,000 reading and math
teachers in grades four through eight in more than 450 districts across the state.
StateImpact Ohio and The Plain Dealer consulted with Ohio University researchers Anirudh
Ruhil and Marsha Lewis, who are leading a statewide study of value-added and related
measures, on the analysis.
The pay analysis looked district-by-district at the correlation between teachers’ pay and valueadded scores. It found significant correlations in just three districts across the state. None of
those districts had more than 40 teachers rated.
That analysis did not take into account other factors that might influence student learning, such
as poverty levels, or other factors related to teachers, such as advanced degrees. It also did not
take into account the small numbers of teachers who received value-added scores in some
districts or the fact that some teachers had only a single year of value-added data available.
Ruhil and Lewis cautioned that additional analysis is necessary to more completely describe
the relationship between teacher pay and value-added.
Scale matters
The Naming of the Names
● If you publish, know
why
● Explain to
readers/listeners
● Engage
online/offline