Vardaman 1
MaryAnne Vardaman
Economics 398
29 October 2012
Teacher’s Credentials or Value-added:
The True Determinant of Teacher Quality
Introduction
Many researchers have tried to predict teacher quality based on observable
characteristics such as teacher’s highest education level, teaching experience, and possession of
a teaching certificate. However, research has shown that this method of prediction is not very
accurate. Therefore, researchers have proposed using a “value-added” method, similar to that
used in production of goods, in order to measure the effect that the teacher has on her
students throughout the year. This method can be a more accurate predictor of teacher
quality, but it also has its issues. Therefore, I propose a longitudinal study of public school
children and their teachers in order to compare some of the richer existing value-added models
with “traditional” methods of measuring teacher quality that include some characteristics that
are not always observable, like where the teacher attended college and the teacher’s SAT
scores.
Vardaman 2
It makes sense that high-quality, effective teachers will have the greatest positive
impacts on student achievement, but this has actually been a difficult point to prove. The
difficulties in measuring teacher quality have arisen because not everyone agrees what it is that
qualifies a teacher as “high-quality.” Obviously, there are some broad characteristics that
people can generally agree that quality teachers possess including: knowledge of the subject
they are teaching, possession of a “good education,” inclusion of all students, and an engaging
demeanor. However, there is debate on what constitutes a “good” education for a teacher,
how nurturing a teacher should be to her students, or whether the qualities that constitute a
good teacher change as the child moves to higher grades.
Literature Review
Why does teacher quality matter? As Koedel and Betts demonstrate in their paper,
“variation in teacher quality is an important contributor to student achievement.” 1 In the past,
researchers like Hanushek have summed up previous research that shows teacher education,
experience, and other readily observable measure of teacher qualifications are inaccurate
predictors of teacher quality, and therefore, student achievement. 2 Naturally, researchers have
been searching for an alternative way to measure teacher quality, so they have proposed using
a value-added method. Ideally, this method would highlight the effective, high-quality teachers
by showing the positive impacts they have on their students’ achievement.
1
Koedel, Cory, and Julian Betts. Re-Examining the Role of Teacher Quality in the Educational Production Function.,
2007. EconLit. Web. 5 Oct. 2012.
2
Hanushek, Eric A. The Impact of Differential Expenditures on School Performance, 1989. JSTOR. Web. 29 Oct.
2012.
Vardaman 3
There are some value-added models that are more commonly used than others, but
these all have bias issues, due to the general simplicity of the models. Jesse Rothstein examines
three of the most common models in greater depth, and he finds many potential sources of
bias including: error in sampling, mean reversion (students who do poorly on year are more
likely to do better the following year as they revert back to their mean score), nonrandom
classroom assignment (students have the same peers in their classroom every year and
teachers are assigned certain students deliberately), excessive dependence on sorting by
observables (unobservable variables are virtually ignored in the method specifications), and
omitted variable bias. 3 To help correct for these biases, it seems necessary to create more
complex value-added models that can be tested before implementation.
Unfortunately, as work by Kirabo Jackson has shown, it is not currently possible to
create a “one-size-fits-all” value-added model that can be applied across the board. He finds
that value-added models of teacher performance for elementary school are not necessarily
accurate for high school and that these models are more predictive of performance in certain
subjects, but not in others. 4 These findings are supported by work from Koedel and Betts, who
also demonstrate that value-added models must be limited in their application. 5
3
Rothstein, Jesse. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement."
Quarterly Journal of Economics 125.1 (2010): 175-214. EconLit. Web. 5 Oct. 2012
4
Jackson, C. Kirabo. (2012) "Teacher Quality at the High-School Level: The Importance of Accounting for Tracks"
NBER Working Paper No. 17722. 25 Oct. 2012.
5
Koedel, Cory, and Julian R. Betts. "Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness?
an Extended Analysis of the Rothstein Critique." Education Finance and Policy 6.1 (2011): 18-42. EconLit. Web. 23
Oct. 2012.
Vardaman 4
“Track-level” bias is a relatively recent discovery in the investigation of value-added
models. 6 This bias, discovered by Kirabo Jackson, arises when teachers are routinely assigned
classes that lie on certain tracks or sequences of achievement, like honors Algebra. In these
cases, the students exhibit higher test scores, not because of the teacher, but because of the
class itself and the achievement track on which it is placed. He finds that there is greater
variability within test scores of a certain subject within a certain grade for the school compared
to achievement tracks within the school. Additionally, he finds that, “within the same course
(but not the same track), some teachers consistently teach students who take more/less honors
classes than the average student than other teachers.” 7 To account for this, Jackson creates a
value-added model that controls for track-level achievement from the student perspective and
the teacher perspective.
None of the researchers believe that value-added methods alone should be used to
determine teacher effectiveness. As Hanushek and Rivkin illustrate, because of the great
potential for bias, it is necessary to combine these models with other (subjective) methods of
evaluating teacher effectiveness and teacher compensation. 8 Otherwise, the high-stakes
situations can create incentives for teachers to help students cheat on tests or only teach to
those students who have the best possibility of doing well on the test. Hanushek and Rivkin also
cite random occurrences prior to and during the tests that can introduce measurement bias. 9
In these cases, the presence of something distracting (like construction) during the test can
6
Jackson, C. Kirabo et al
Jackson, C. Kirabo, p.11
8
Hanushek, Eric A., and Steven G. Rivkin. "Generalizations about using Value-Added Measures of Teacher Quality."
American Economic Review 100.2 (2010): 267-71. EconLit. Web. 23 Oct. 2012.
7
9
Hanushek and Rivkin et al
Vardaman 5
result in lower test scores, despite student and teacher preparation. It is impossible to control
for such randomness, so the value-added model must not be weighted too heavily in teacher
evaluations. Also, Kane and Staiger believe that if test scores are aggregated from several
years, this would help reduce the bias these random events introduce into the data. 10 Basically,
teachers would be evaluated on their average “performance,” rather than by their students’
achievements each year.
There is no easy answer on how to measure teacher effectiveness. A teacher’s impact
on a student may not be immediately observable and there are various factors influencing a
student’s achievement besides their teacher. Considering all of the factors and all of the
potentials for bias, it seems impossible to create a value-added model that on its own will
accurately reveal a teacher’s quality. Therefore, it is necessary to make the most complex,
accurate models possible and then hope to explain the rest of the variation by subjective
means.
10
Kane, Thomas J., and Douglas O. Staiger. "The Promise and Pitfalls of using Imprecise School Accountability
Measures." Journal of Economic Perspectives 16.4 (2002): 91-114. EconLit. Web. 24 Oct. 2012.
Vardaman 6
Works Cited
Hanushek, Eric A. The Impact of Differential Expenditures on School Performance, 1989. JSTOR.
Web. 29 Oct. 2012.
Hanushek, Eric A., and Steven G. Rivkin. "Generalizations about using Value-Added Measures of
Teacher Quality." American Economic Review 100.2 (2010): 267-71. EconLit. Web. 23
Oct. 2012.
Jackson, C. Kirabo. (2012) "Teacher Quality at the High-School Level: The Importance of
Accounting for Tracks" NBER Working Paper No. 17722. 25 Oct. 2012.
Kane, Thomas J., and Douglas O. Staiger. "The Promise and Pitfalls of using Imprecise School
Accountability Measures." Journal of Economic Perspectives 16.4 (2002): 91-114.
EconLit. Web. 24 Oct. 2012.
Koedel, Cory, and Julian R. Betts. "Does Student Sorting Invalidate Value-Added Models of
Teacher Effectiveness? an Extended Analysis of the Rothstein Critique." Education
Finance and Policy 6.1 (2011): 18-42. EconLit. Web. 23 Oct. 2012.
Koedel, Cory, and Julian Betts. Re-Examining the Role of Teacher Quality in the Educational
Production Function., 2007. EconLit. Web. 5 Oct. 2012.
Rothstein, Jesse. "Teacher Quality in Educational Production: Tracking, Decay, and Student
Achievement." Quarterly Journal of Economics 125.1 (2010): 175-214. EconLit. Web. 5
Oct. 2012
Vardaman 7