A New Series of Papers
on Teacher Compensation
from
the University of Wisconsin CPRE Group
There is strong consensus around the country that talented and capable teachers will be needed in
all classrooms in order to accomplish the nation’s goals of teaching all students to high
standards, and closing the achievement gap. Although there are many policy and practice issues
that have to be addressed in order for the nation’s education systems to recruit and retain the
quality of individuals that are required, including schools and classrooms in many of the
country’s large urban and poverty impacted districts, the teacher compensation system itself
must be changed. Teacher salary levels will have to be hiked in many places to enable school
systems to compete for the quality of talent required to be successful, and the salary structures
themselves need to be changed in order to pay teachers for the knowledge, skills and
responsibilities to be successful, including bonuses for improved student performance.
With support from the College Board, the Consortium for Policy Research in Education (CPRE)
Group at the University of Wisconsin-Madison is producing a series of papers that addresses the
compensation aspect of the strategic management of human capital in public education:
1. New Teacher Pay Structures: The Compensation Side of the Strategic Management of
Human Capital, by Allan Odden
2. Do Teacher Pay Levels Matter?, by Anthony Milanowski
3. How to Design New Teacher Salary Structures, by Herbert G. Heneman, III and Steve
Kimball
4. How to Pay Teachers for Student Performance Outcomes, by Anthony Milanowski
5. How to Fun d Teacher Compensation Changes, by Allan Odden
6. Exploring a Federal Government Role in Funding Increased Teacher Compensation, by
Andrew Reschovsky
This paper is available in the Resources section of http://www.smhc-cpre.org.
September 2008
The research reported in this paper was supported by a grant from the College Board, New York,
NY (Grant No. 2007-1035) to the Consortium for Policy Research in Education (CPRE) and the
Wisconsin Center for Education Research, School of Education, University of WisconsinMadison. The opinions expressed are those of the authors and do not necessarily reflect the
view of the institutional partners of CPRE, the College Board or the Wisconsin Center for
Education Research.
HOW TO PAY TEACHERS FOR STUDENT PERFORMANCE OUTCOMES
By
Anthony Milanowski
The paper discusses the third key element in a modernized teacher compensation system,
variable pay. Variable pay, also often called incentive pay, or pay for results, is typically based
on some measure of worker or organizational performance over a period of time, and goes up or
down as performance changes. Usually, variable pay is provided as a one-time bonus, and is not
included in an employee’s base pay. Variable pay complements the base pay and base pay
progression elements of a strategic teacher compensation system discussed in the previous papers
in this series by providing an incentive for teachers to act in ways that contribute to school
district success, such as improving student achievement. It can also contribute to attracting and
retaining better performers and to the shaping of district culture by communicating which
outcomes are valued and who the high performers who should be emulated are.
Educational policy makers have identified improved student achievement as the most important
performance outcome of educators’ efforts. Thus it is natural to think of basing variable pay on
student achievement. This is, however, a big change from tradition. Variable pay has not been a
major feature of teacher pay systems, and the culture of education has tended to reward inputs
and effort rather than outcomes and results. Though incentives tied to student achievement were
introduced in the later 1980’s in Dallas, Texas, and in the early 1990’s in Kentucky and
Charlotte-Mecklenburg, North Carolina, it is only in the past few years that these ideas have
entered the mainstream of reform thinking in education. Indicators of this include Congressional
interest in including performance pay in the No Child Left Behind re-authorization, 1 the Federal
government’s Teacher Incentive Fund grants, which are providing support to 32 school districts,
schools, or consortia of schools that will be paying teacher based on student achievement, 2 and
new programs or proposals for basing some part of teachers’ pay on student achievement in
diverse states including Arizona, Florida, Minnesota, Idaho, Kentucky, Tennessee, and Texas.
As the movement to bring performance incentives into K–12 education gathers strength, many
interesting program designs are being implemented by the nation’s states and school districts.
Unfortunately, some appear to have been designed based on simplistic thinking. The experience
of the private sector and the small body of research on pay innovations in education show that
designing a variable pay program that will actually contribute to improved student achievement
is a fairly complex undertaking. To help policy makers and program designers refine their
thinking about paying for student achievement, this paper covers five key design questions:
1. How can paying teachers for student achievement actually contribute to improving student
achievement?
2. Should teachers be rewarded for student achievement on a group or individual basis?
3. How should improvement in student achievement be defined?
4. Should other indicators of performance be included in a variable pay program?
1
5. How large should the bonus payouts of the variable pay program be?
After addressing these questions, the paper concludes with some guidelines for designing a
variable pay program. Note that the paper is not intended to be a complete guide to program
design. A list of useful publications and Web sites covering the gamut of design issues is
included at the end.
1. HOW CAN PAYING TEACHERS FOR STUDENT PERFORMANCE OUTCOMES CONTRIBUTE TO
IMPROVED STUDENT ACHIEVEMENT?
This paper assumes that the fundamental purpose of paying teachers for student performance
outcomes is to improve student achievement. We recognize that some policy makers (and
citizens) believe that paying more money to teachers who produce higher student achievement is
simply the right thing to do. It’s a basic value in much of U.S. society that those who accomplish
more should get more. This is the meritocratic ideal. But if the primary rationale for paying
teachers for improved student achievement is to improve that achievement, program designers
need to think about how extra pay for teachers will, in practice, affect student achievement.
While private sector research has found that pay incentives can be an effective way to improve
individual and organizational performance, 3 research and practice with variable pay have
identified three different plausible “theories of action” that explain how incentives can work:
• Directing worker effort toward performance goals and motivating increased effort
• Sorting workers
• Changing organizational culture and management systems
Motivating Increased Effort
This theory of action is based on the premise that the additional pay promised will motivate
teachers to exert more effort on those activities that they believe will contribute to achieving the
performance outcomes and thus earning the incentive payment. While both behavioral research
and common sense support this proposition, it is important to recognize that successful
motivation requires more than promising a bonus. Figure 1 shows a simplified motivational
model drawn from goal setting and expectancy theories 4 that helps clarify what is required for
incentives to motivate. This model does not pre-judge whether teacher find financial incentives
more or less motivating than other rewards. Rather, it identifies several judgments that teachers
participating in a variable pay program need to make in order for them to be motivated by the
promised incentive to focus effort, increase effort, and/or persist in expending effort. In this
model, the motivational impact of an incentive depends on teachers’ beliefs about how likely it is
that that effort will lead to achieving the goals, how likely it is that achieving the goals will lead
to certain consequences, both positive and negative, and the value placed on these consequences.
First, teachers have to believe that their efforts have a high likelihood of actually succeeding in
meeting the goals. This belief is represented by the arrow between Educator Effort and
Performance Goals (labeled “1” in the figure). If teachers do not believe that their efforts can
lead to goal attainment, they are unlikely to expend much effort toward the goals. One important
2
influence on this belief is the clarity, specificity, and difficulty of the goals themselves. Another
important influence, labeled “competencies,” is teachers’ perceptions of whether they have the
knowledge, skills, and abilities needed to achieve the goals. A third important influence, labeled
“enablers,” represents teachers’ perceptions of whether they have the individual or school
resources needed to meet the goals. Enablers include books and curriculum materials, class size,
facilities, and school or district leadership.
Figure 1: Motivational Model
Competencies
Teacher Effort
- Focus
- Intensity
- Persistence
1
Performance
Goals
(Student
Achievement)
Enablers
2
Consequences
- Pay Bonus
- Recognition
- Feeling of
accomplishment
- Stress, higher
workload
- Less freedom
- Peer reactions
So far, the model suggests three requirements to which designers of incentive programs need to
attend in order to maximize motivational impact. First, performance goals need to be clear and
specific, so teachers understand what they are expected to do to get the bonus. Second, goals
need to be set at a level that is both achievable and worthwhile. This requires, in turn, a serious
review of what is feasible considering the levels of teacher competencies and enablers in place.
To make ambitious goals motivating, program designers need to supplement the incentive
program with efforts to build the competencies and provide the enablers so teachers will believe
they have the necessary skills and resources to meet the goals.
Next, consider the link between performance goals and consequences (labeled “2” in the figure).
For an incentive to motivate, teachers have to believe that if they do expend the effort and meet
the goals, they will actually receive the incentive. Why wouldn’t teachers believe this? Maybe
the better question is why would they, given concerns about districts’ financial capacities, the
short tenure of many big city superintendents, turnover on district governing boards, and the
“reform de jour” syndrome infecting many districts. The implication here is that if incentive
designers want incentives to motivate, they need to find ways to reassure educators that the
money will be there if the goals are met and over the long term. The designers of Denver’s
ProComp incentive program took this to heart when they guaranteed the financing. 5
Lastly, consider the box labeled “consequences.” These include both the positive outcomes
teachers might experience from participating in an incentive program, including the pay bonus,
but also receiving recognition for a job well done and seeing that student achievement has
3
improved. Obviously, for incentives to motivate, educators must value these outcomes. But we
also need to consider potential negative outcomes that may arise from expending the effort
needed to achieve the goals. These include extra workload and job stress, less freedom to teach
favorite material or in desired ways, and the potential negative reactions from peers (the “rate
buster” stigma familiar from some private sector performance pay experiences). It is important to
recognize that educators may differ substantially in the value that they place on these potential
outcomes. It behooves program designers to recognize that not all educators will find incentives
equally motivating, and therefore that they may want to couple financial incentives with nonfinancial rewards like recognition. Another somewhat subtle implication is that for incentives to
motivate, they have to be valuable enough and certain enough to overcome the negative
consequences that can arise from expending the effort. If teachers perceive that the undesirable
consequences (like more stress) are more likely, and more painful than the positive consequences
are pleasant, then the incentives can hardly be expected to motivate. Program designers therefore
need to maximize the value of the incentives, maximize the likelihood of receiving the incentives
if the goals are met, and minimize the probability of the negative consequences.
To summarize, the model’s implications for designing programs to pay for student achievement
are:
1. Rewards should be large enough to motivate teachers to seek them, and sufficient to
compensate for the effort needed to meet them.
2. Teachers must see a strong link between meeting the student achievement goals and receiving
a pay reward. Programs should be structured to minimize doubt that meeting the goals will lead
to the promised payout.
3. Teachers must believe that if they put forth focused, intensive, and persistent effort they will
stand a good chance of meeting the student achievement goals. Thus designers need to attend to
educator competencies, supports and enablers, and the difficulty of the student achievement
goals. Roadblocks to turning effort into goal achievement need to be addressed so that teachers
believe they have a real chance to succeed.
Unless the other elements in the motivational model are addressed, motivational effects will be
weak. Districts can use the motivation model to assess the design and implementation of the
program to ensure that motivational potential is maximized.
Research on private sector performance pay plans has also found that perceptions of fairness and
trust between the managers of the incentive plan and those working under it affect the
motivational impact of rewards. 6 Perceptions of gross unfairness in the process can actually
lower workers’ valuation of the rewards. Fairness and trust also bolster perceptions that
achieving the goals will actually result in receiving the reward. Fairness and trust are also
important for workers to believe that their effort will lead to goal attainment. Those working
under the plan need to believe that the goals are set in such a way that they have a chance to win,
that they are not disadvantaged in relation to others, and that the performance measures fairly
represent the results of their efforts. To influence fairness perceptions, the rationale behind the
performance measures, the level of the improvement goals, and the mechanics of the program
4
need to be explained to educators in detail. Reliability and face validity of student achievement
measures and other measures of school performance are also likely to be crucial.
Incentives as Employee Sorters
This theory of action is based on the idea that rewarding teachers for student performance will
preferentially attract and retain the more effective teachers, and discourage the hiring and
retention of less effective teachers. This “sorting effect” can work both for new hires and current
staff. In the case of new hires, if the incentives and the requirements for earning them are
explained to job applicants, they will evaluate their own abilities and decide whether or not to
pursue to the job depending on part on how likely they think they are to earn the incentive. Job
applicants who believe they can succeed will be more likely to join the organization, and those
who are less sure will be more likely to self-select out of the hiring process. Applied to current
staff, this theory of action holds that the incentive will encourage high performers to stay and
low performer to leave, since they do not receive the same level of compensation as the higher
performers. 7 This effect could be quite powerful, over time, in improving the average
performance of the district’s teachers. Making this theory of action work, however, is likely to
require that incentives be fairly large as a proportion of total pay. Receiving or not receiving a
small incentive is not likely to cause much sorting. Further, a reliable and valid measure of the
performance outcomes, that can be tied to the individual teacher, is needed. Lastly, teachers need
to be made aware of the incentive and the level of performance needed to achieve it, so they can
realistically evaluate their chances of success given their ability and the effort they want to
expend.
Creating a Performance Culture
The third theory of action works through the effects of incentives on district culture and
management systems. It is more complex and indirect that the other two, but may in the long run
be the most plausible. It is based on the notion that paying for student performance outcomes
signals or symbolizes what the district really values. This communication helps to shape a
culture in which results matter, in which achieving results is the “right” thing to do, which in turn
indirectly affects motivation. While most districts are already concerned with improving student
achievement due to Federal and state accountability programs, putting money on the table shows
the district’s commitment to this goal. Incentives also help identify role models. They recognize
and make apparent performance differences among teachers or schools. This sends the message
that performance matters, and points out who should be emulated.
Putting money on the table also ups the pressure for improved management practices. The
literature on private sector group rewards programs suggests that giving workers a financial stake
in the organization motivates them to push for improved production processes, for example,
through employee suggestions. 8 Incentives for results require managers to set and track explicit
goals, decide which goals are really important, and find ways to help front line employees
succeed. A recent OECD review of public sector performance pay concluded that it was just
these influences that produced positive results, rather than direct motivation of effort. 9 Though
the NCLB act has already required districts to focus management efforts on student achievement,
incentives raise the stakes for both district leaders and school staff. Stakes at the school level
5
could encourage teachers to press school leaders and school leaders to press the district office to
provide the supports needed to improve performance, now that money is at stake. Incentive plans
can also force districts to develop more sophisticated performance measurement systems,
because incentives require more reliable and valid measurement. (Several of the Teacher
Incentive Fund grantees have found, for example, that they did not have accurate ways to
determine which students were taught by which teachers. California’s ill-fated school
performance award program brought to light many problems the state had with its testing
program and with tracking students back to schools.) Further, when districts try to maximize the
motivational impact of the incentives by addressing teacher competencies and performance
enablers, these efforts themselves will likely lead to improved student achievement.
Studies of private sector group incentive plans have found some other very concrete ways in
which these incentives influence behavior through culture. One of these is by legitimizing mutual
monitoring. 10 In most organizations, workers generally pay attention only to their own
performance, and it is considered inappropriate to concern oneself with how others are doing.
That is the supervisor’s job. Group incentives provide a motivation and legitimate reason for
workers to monitor each others’ behavior, and to encourage co-workers to do what needs to be
done to earn the incentive for the group. Group incentives could help break down the tacit norms
in most districts that discourage teachers from monitoring each other’s practice and suggesting
ways to improve. Group incentives can also contribute to workers feeling that they are “all in this
together,” making use of the powerful force of group solidarity to enhance focus on group goals,
and can give workers an additional reason to cooperate with one another. Research on group
incentives supports the idea that rewards can encourage workers to share knowledge and
expertise. 11 Individual incentives also affect organizational culture, as does the lack of individual
incentives
This discussion has three broad implications for districts considering paying for student
performance outcomes:
1. Districts need to think about which theory of action they want to apply. As will be shown in
the rest of the paper, the theory of action chosen has implications for the design of the
variable pay system.
2. The teacher sorting and performance culture theories of action are going to take time to work
A district expecting results from sorting or culture change might drop the incentive program
in disappointment over minimal short term effects. Sadly, some districts that have begun
incentive programs appear to expect result in a year or two. Districts need to think long term,
and not expect these incentives to be another quick fix for lagging student achievement.
3. Paying teachers for improved student achievement needs to be part of a multifaceted strategy.
For the motivational theory of action to work, a district needs to be concerned about teacher
competencies and performance enablers, especially in schools and classrooms that are at the
bottom of the performance distribution. This in turn requires hard thought about what
competencies and enablers do in fact help improve student achievement. These competencies
and enablers should be derived from the district’s strategy for improving instruction and
learning conditions. The culture and the management improvement stories are also intimately
6
connected to strategy. For example, if one mechanism is to make student achievement goals
more salient to teachers and schools, those goals should be the same as in the district’s
strategic plan. The goals should be those the district is already working toward with other
programs and strategies. Otherwise the district continues to broadcast a cacophony of
messages, drowning out the message that the student achievement results that are being
rewarded are critical.
2. SHOULD TEACHERS BE REWARDED ON AN INDIVIDUAL OR GROUP BASIS?
Most of the first programs that paid for student performance outcomes in the U.S. were schoolbased performance awards. These programs provided bonuses for all the teachers, administrators,
and often support staff in a school when that school achieved pre-established performance goals,
including improvements in student achievement. A current example of a school-based award is
discussed below for the Charlotte-Mecklenburg Incentive Program. More recently, incentive
programs have been designed to reward individual teachers for the achievement of the students
they as individuals teach. The Dallas Independent School District is an example of a plan that
includes an individual incentive. An intermediate alternative would be to reward teams of
teachers within a school who regularly work together to educate a defined group of students
(e.g., at the grade level or within a subject.) Research on private sector incentive plans has
generally found that individual, small group, and large group incentives can contribute to
improved individual or organizational performance. 12 We do not yet know enough about what
works in public education to recommend one type over another, but it is unlikely that one best
practice applies to all districts.
Charlotte-Mecklenburg School District‘s Program for Paying for Student Achievement 1
The Charlotte-Mecklenburg School District has had a school-based performance award since the
early 1990’s. The original program was based on nine dimensions of school performance defined
by the district, including primary grade readiness, absenteeism, statewide science and social
science test scores, statewide end-of-grade or end-of-course test sores (high school), statewide
writing test scores, pre-algebra and algebra preparedness, dropouts, enrollment in higher level
courses. In 1998, the State of North Carolina introduced a state-wide incentive program, and the
district modified their program to integrate it with the state’s approach. The district regularly
modifies its program based on changing goals and its experiences with incentives. The
description below is of the program for the 2006-07 school year.
The CMS program consists of two parts: the state incentive and the local component. The state
program, called the ABC’s of Public Education, is a school-based award program that provides
an annual cash bonus to staff in schools in two performance categories: Expected Growth
Schools and High Academic Change Schools. Both certified staff (teachers, school
administrators, counselors, psychologists) and support staff (teacher assistants, office staff) are
eligible for the bonus according to the following schedule:
1
More information on the North Carolina ABC plan can be found at http://abcs.ncpublicschools.org/abcs/
and more information on the Charlotte-Mecklenburg plan can be found at:
http://www.cms.k12.nc.us/departments/HR/recruitment.asp.
7
Certified
Other
High Academic Change
$1,500
$500
Expected Growth
$750
$375
An expected growth school is one in which, on the average, tested students achieve target test
scores set based on their prior year scores (typically the two prior years) and an adjustment factor
meant to account for regression to the mean 13 in the test scores. This factor was developed based
on a statistical model of the relationship between current and prior year test scores during a
standard-setting period (e.g., 2001-02, 2002-03, and 2003-04) for all tested students in the state.
A student’s growth is calculated by subtracting target scores from the actual scores obtained each
year and multiplying by the adjustment factor. For each school, these differences are averaged
across all tested subjects and grades to derive a school score. A school achieves expected growth
when the average of the differences is zero, indicating that the school is producing the level of
student achievement that would be expected, based on statewide patterns, given students’ prior
levels of achievement. A high academic change school is one in which 60% or more of the
students achieve the expected growth. Tests are given in reading and mathematics for grades 3-8,
and in English, Biology, Civics and Economics, U.S. History, Physical Science, Physics,
Chemistry, Algebra I and II, and Geometry in high school.
One notable feature of North Carolina system is that the expected growth factor is fixed for
several years. This avoids a situation in which in any year about half the students would be
below expected growth and half above, no matter how much the average achievement grew
statewide, as would happen if a conventional value-added model was used. In essence, the
system requires schools to meet or exceed a standard amount of growth, where the standard is
based on a fixed historical period. The result is that all schools have a chance to earn the bonus,
and improvement in growth over the standard is recognized. The standard is recalculated every
several years, however, to keep raising the bar for student achievement.
The district program in Charlotte-Mecklenburg, called the Local Accountability Bonus, is also a
school-based award that adds an additional amount to the state bonus based on the school’s
achieving one of two levels of performance. Tier 1 (the highest level) requires the school to
achieve either expected growth or high academic change as per the state program, but also to
make Adequate Yearly Progress (AYP) for all student subgroups. Tier 2 requires the school to
achieve either AYP for all subgroups or expected growth or high academic change, but not both,
as in Tier 1. As with the state program, both certified and other school staff are eligible for the
bonus. The bonus amount changes from year to year due to funding available and the number of
schools that qualify. In the past, amounts have been in the ranges shown below.
8
Certified
Other
Tier 1
$650-850
$325-425
Tier 2
$400-600
$200-300
Dallas Independent School District‘s Program for Paying for Student Achievement 2
The Dallas Independent School District has had a school-based performance award since the
mid-1980’s. Recently, the district has refined their reward program as part of the Teacher
Incentive Fund and added a component that rewards individual teachers for the achievement of
their students. These revised incentives are being piloted in the 2007-08 school year.
The school-based award provides bonuses of from $1,250 to 2,000 to professional staff and
$625-1,000 to support staff based on the school’s School Effectiveness Index. This index is
based on a value-added model and measures the amount of gain in achievement students at a
school make compared to other similar students. Test scores used to measure gain include the
Texas state assessments, the Iowa Test of Basic Skills, and other standardized achievement tests
given by the district. The value-added model used by the district includes an extensive set of
controls at the student and school level in order to level the paying field across schools and
ensure that schools receive credit for improving the performance of low achieving students. Each
school in the district is ranked on the School Effectiveness Index (SEI). Staff in schools in the
60th percentile or above on the index receive bonuses according to the following schedule:
SEI Percentile Rank
90-99
80-89
70-79
60-69
Below 60
Professional Staff Award
$2,000
$1,750
$1,500
$1,250
$0
Support Staff award
$1,000
$875
$750
$625
$0
Professional staff include teachers, principals and assistant principals, librarians and media
specialists, academic coordinators, nurses, counselors, and speech therapists. Support staff
include office staff, teaching assistants, parent liaisons, custodians, and cafeteria workers.
The classroom-based award provides bonuses of from $2,000 to 8,000 to full-time teachers for
whom a Classroom Effectiveness Index (CEI) can be calculated (i.e., those teaching tested
subjects which include English/Language Arts, Mathematics, Social studies, Science, World
Languages, and Computer Science). Similar to the SEI, this is based on a value-added model and
compares the gain made by a teacher’s students to the gains made by similar students in other
classrooms. Similar to the school-based award, teachers in the 60th percentile or above on the
index receive bonuses. The bonus schedule is:
2
Taken from Dallas Independent School District, Performance Pay Manual for Teachers and Campus Employees,
2007-08.
9
CEI Percentile Rank
90-99
80-89
70-79
60-69
Below 60
Classroom-Level Award
$8,000
$6,000
$4,000
$2,000
$0
CEI’s are calculated for teachers who teach in more than one subject and bonus amounts are
determined based on proration, so that a teacher who teachers reading and math would receive
50% of the bonus amount corresponding to the reading CEI percentile and 50% corresponding to
the math CEI percentile. Kindergarten, special education, fine art, and physical education
teachers do not have a CEI, but may participate in the school award.
In order for a teacher to be eligible for receiving either award, she or he has to have received a
performance evaluation based on at least two classroom observations. Teachers for who the CEI
can be calculated and who are otherwise eligible can receive both the classroom and a schoolbased award. To participate, teachers had to explicitly opt into the program.
Considerations Based on the Theory of Action
Four important considerations bear on the choice of group or individual incentives: 1) Which of
three theories of action the district intends to apply; 2) The level at which student achievement
goals can be set and measured, and credit for achieving them attributed; 3) The fit with other
strategies for improving student achievement; and, intertwined with the first three, 4) Which
strategy would be more likely to be accepted by teachers.
If the primary theory of action to be applied is the use of incentives to motivate teacher effort,
individual incentives may be the best choice. Motivation theory suggests that individual
incentives will be the most effective way to directly motivate teachers because the relationship
between effort and reward is the most direct. Compared to a group incentive, teachers are more
likely to see how their own effort can earn a reward based on the performance of their own
students. Incentive designers call this direct connection “line of sight.” All else equal group
rewards will not motivate as strongly because the relationship between any individual’s effort
and the reward is not as direct. It also depends on the efforts of all the other teachers in the
group. Any individual teacher has less control over this than her/his own effort applied to her/his
own students. Individual teachers may also doubt whether all of their colleagues have the ability
and commitment to do what it takes to meet the performance goals. The other motivational issue
is the potential for free riders: those are workers who don’t contribute any extra effort because
they believe they will receive the reward when other group members do the work. Free riding is
possible because individual effort and results are not being measured. The motivational impact of
group rewards can be vitiated when educators believe that others in the group will free ride. Our
own research suggests that new teachers may actually prefer individually based rewards due to
line of sight and free rider issues. 14 Research from the private sector also provides some
evidence that very high performers react negatively to group rewards because they believe that
10
though they have made a greater contribution, they receive the same reward as the rest of the
group. 15
If the purpose of the incentive is to encourage teacher sorting, individual incentives are likely to
be more effective Group awards have a less powerful sorting effect because of the potential for
free riders. Further some research has suggested that school incentives can influence teachers to
leave low performing schools. 16 Given that such schools often have higher turnover, districts are
not likely to want to encourage teachers to leave these schools, unless they can be sure it is the
low performers who are leaving. But this requires measuring individual, rather than school,
performance.
If the purpose is to change culture, both individual and group rewards can be appropriate. In
deciding which to use, it is important to think about how well each fits with the existing culture.
Too much cultural dissonance will lead to conflict and turmoil. This in turn can make pay the
issue, rather than student achievement. Group incentives may be a better fit with the culture of
many districts. Much of public education has an egalitarian culture in which distinctions among
teachers that can’t be based on simple, objective factors are taboo. There is some evidence that
group rewards increase the level of cooperation and a sense of community of fate, aspects of
culture districts are likely to want to reinforce. On the other hand, group rewards may not be
effective if other elements of school culture discourage teachers from seeing their work as
interdependent. In many schools, existing work patterns are isolated and resistant to
collaboration. 17 While is often argued that individual rewards will increase competition among
teachers and negate efforts to build a cohesive faculty, the private sector research shows
surprisingly little evidence for these effects.
Measuring Teacher Performance
Individual incentives can be hard to implement because it can be difficult to measure
performance, or to correctly attribute outcomes, at the individual teacher level. One problem is
that in any one year most teachers have only a relatively small number of students for which test
scores are available. This makes it harder to reliably distinguish one teacher’s performance from
another, except at the far ends of the performance distribution. 18 Unreliable performance
estimates waste incentive money by rewarding individuals who may not have actually deserved a
reward, and if educators are aware of the unreliability, it weakens the credibility of the
performance measure. Generally, school-level measures are based on larger samples, and can be
more reliable. (Note, however, that school-based measures for small schools can also have
reliability problems.) A second measurement problem is that many teachers—perhaps one half or
more—do not teach tested subjects. In most states this includes social studies, vocational,
physical education, art, and music teachers, as well as many special education teachers. As some
of the Teacher Incentive Fund sites are finding, this is an equity issue of concern to both those
who teach tested students (because they feel singled out for accountability pressure) and those
who do not (because they do not have the same opportunity to earn the reward). A third problem
is the difficulty many districts have in linking teachers to students. Not only do districts have
problems getting student and teacher information systems to “talk” to each other, but many
districts do not have accurate records of who provided instruction to which student in which
tested subject over a given time period. Student mobility, poor system design, and lax reporting
11
procedures all make it hard to establish the needed link. The problem is especially difficult for
middle and high school teachers.
Several strategies have been used to address the question of how to include teachers who do not
teach tested subjects in individual incentive programs, as well as other members of the teacher
bargaining unit such as psychologists, counselors, or speech pathologists. One strategy has been
to offer both an individual and school-based incentive, then simply base these teachers’ bonuses
on the school-based performance measures. Districts using the Teacher Advancement Program
provide both individual and school rewards. Teachers for whom classroom student achievement
gains can be calculated receive a bonus based partly on classroom and partly on school results.
Other teachers can receive a similar bonus amount based only on school results. Other districts
have developed tests in areas not covered by state testing programs.
The Orange County, Florida district, discussed below, incorporates these teachers into its
individual incentive program by allowing them to link their rewards to specific teachers who
teach tested subjects. The teacher would earn a bonus when the teacher or small group of
teachers to which s/he is linked do. The Denver ProComp plan, by contrast, allows teachers of
subjects the state does not test and student services staff to earn an individual bonus by meeting
performance goals set in collaboration with the school principal. These staff members also
participate in the school-based award program.
The Orange County, Florida Individual Teacher Design
This is an individual teacher incentive based on improvement in the achievement of the specific
students in a teacher’s classes. Currently, teachers in 10 middle and high schools are eligible to
participate. Participation is voluntary. The design is based on an existing professional
development planning process in which teachers work with their principals to set goals for
improvement in proficiency level on subjects covered by the Florida state (FCAT) tests. State
tests are given in writing, (Grades 5, 4, 8, and 10), reading and mathematics (Grades 3-8, 10) and
science (Grades 5, 8, and 11). Teachers are provided with the previous year’s test scores and
proficiency levels of their students at the beginning of the year, and set goals for student
achievement for the current year. Because the FCAT tests are vertically scaled, teachers can set
goals for learning gains based on last year’s scores, and the state is able to calculate learning
gains from year to year, and provide them to teachers and schools.
An example of student achievement goals for a high school teacher would be:
Baseline
Goal
All 21 students in my Algebra IB class are
among the 52% of the lowest quartile who
did not make learning gains last year.
1) At least 60% of the targeted students will score 50% or higher on
the Measurement portion of FCAT math
Of these students, 19 scored below 45% on
the Measurement portion of FCAT math
2) At least 53% of the targeted group will improve their overall FCAT
math score by 10 points.
3) At least 53% of the targeted students will show learning gains on
FCAT
Source: District Web site: https://www.ocps.net/es/hr/PDS/instructional/IPDP/Documents/
12
Teachers who meet their student achievement goals and the two eligibility requirements receive
a bonus of up to $4,000. The different levels of bonus amounts are determined based on the
percent of students for whom the goal is reached. The percentage thresholds and bonus amounts
are:
40% or more: $4,000
30%:
$3,000
20%:
$2,000
10%:
$1,000
In the example above, if 40% or more of the students in the target group (8 students) make the
specified gain the teacher would get $4,000. If only 2 students make the gain, the teacher would
be eligible for $1,000.
In addition to meeting the achievement goal, teachers need to fulfill two other requirements to be
eligible for the incentive bonus:
1. They must receive a rating of satisfactory or above on their annual performance evaluation.
2. They must complete specified content-oriented professional development. Currently, this can
be either a Harvard Graduate School of Education WIDE World Internet-based course
(http://wideworld.gse.harvard.edu/) or a College Board summer institute which offers subjectspecific professional development on the content and resources to teach Advanced Placement
courses.
A distinctive feature of this design is that teachers in non-tested subjects can choose to link
themselves with a teacher of a tested subject who has set student achievement goals. The former
then can receive a bonus if the latter’s goals are met. The district believes this will encourage
teachers in non-tested subjects to support the efforts of their colleagues, as well as providing a
way for these teachers to earn the bonus.
This design was first implemented in the 2007-08 school year as part of the district’s
participation in the U.S. Department’s Teacher Incentive Fund grant program. The incentive
program is administered at the school level and did not require any elaborate data collection
systems by the district, since all the information teachers and principals need to set goals is
provided to schools by the state Department of Education.
Fit with Other Strategies
The fit with other district strategies to improve student achievement should also influence the
choice between individual or group rewards. If a district is trying to improve instruction by
implementing ideas like teacher teaming, curriculum integration, or professional learning
communities, then team or school-based rewards are likely to be the most aligned. For example,
in one of the Teacher Incentive Fund districts, the performance of each elementary grade English
13
language learner is considered the responsibility of 3-5 educators, rather than just the classroom
teacher of record. In this situation, the district is working on team level incentives, recognizing
that an individual incentive would not reinforce the team concept. Individual incentives would
also create a measurement headache when trying to apportion credit for student achievement
among team teachers. On the other hand, individual rewards might fit well with a strategy of
attracting and retaining top instructional talent to work in a more traditional setting. This would
be especially true if a district had the ability to remove or remediate the lowest individual
performers. Individual incentives would also fit with a strategy of developing a culture of
individual teacher excellence. However, individual incentives need to be coupled with feedback,
coaching, and professional development if the goal is to improve individual practice.
Teacher Acceptance
Group incentives are likely to be more accepted by teachers and teacher associations, and are
more in tune with the current culture of most districts. Individual incentives are likely to be
particularly problematic for teachers’ associations, because they can reduce group solidarity and
involve the association in contesting distinctions between teachers made by management.
Teachers in most districts seem uncomfortable with individual rewards, due to both measurement
problems and a tradition of egalitarianism. Individually based awards have been withdrawn in
the Colonial, Pennsylvania district 19 and in Little Rock, Arkansas 20 due in part to teacher
opposition. In contrast, the North Carolina school-based awards has provoked little opposition. 21
Note that in Dallas, the individual reward was implemented following a long history of schoolbased rewards.
An Incremental Strategy
For a district trying to introduce variable pay for the first time, it is probably easier to start with a
school-based reward. School-based rewards may help develop some of the trust needed for
individual incentives to motivate. The measurement problems are also less severe and they are
consistent with much of the current thinking on building better school cultures. Over time, the
district could add team, and individual incentives. Many private sector organizations use a
combination of large group (firm, division, or business unit) small group (team) and individual
incentives. Different forms may be used with different employee groups, or employees may
participate in two or three at once. This has not typically been the case in education, though the
Teacher Advancement Program (as discussed in the prior paper in this series) does use both
individual and school-level rewards. Having both a school and an individual or small team
incentive, while making participation at the team or individual level voluntary, could accomplish
four objectives. It could reinforce a more cooperative school culture, while providing additional
recognition and rewards for individual teachers or teams for which suitable student achievement
measures are available. Being voluntary at the individual or team level, it could be more easily
accepted by teachers and teacher associations. And it would cover all teachers in a school,
avoiding some equity arguments. The Dallas plan, described in Example 2 of this paper, includes
both teacher and school level incentives, and provides for voluntary participation.
14
3. HOW SHOULD IMPROVEMENT IN STUDENT ACHIEVEMENT BE DEFINED?
Four basic approaches have been used to define student achievement as the basis for paying an
incentive: 1) the level of student attainment, defined in terms of test score averages or
percentages of students at proficiency levels; 2) cross-cohort change in the level of attainment
(e.g. comparing the test scores or proficiency levels of 4th graders in 2008 with 4th graders in
2007); 3) change in the level of attainment, comparing the same group or cohort of students over
time (e.g., comparing change in scores between 3rd grade on 2007 and 4th grade in 2008); 4)
value-added models that try to isolate the school or teacher contribution statistically and control
for factors like poverty and ethnic heritage that are outside schools’ and teachers’ control.
Moving from simple attainment to value-added models, measuring student achievement becomes
potentially more fair to schools and students, at the expense of complexity and de-emphasizing
teachers’ and schools’ ultimate goal: that all the students they teach learn to high standards.
Defining Student Achievement by Level of Attainment
Defining school or teacher performance as the average level of attainment is not generally a good
option, because test scores are highly influenced by student socio-economic characteristics, such
as family income, that are outside educators’ control. Nor does defining performance as level of
attainment take into account the level of knowledge students brought with them on entering the
school, grade, or classroom. Thus comparing schools or teachers on the level of test score
attainment of their students without taking these factors into account is not a fair or accurate
comparison of the schools’ or teachers’ performance. And it has a significant motivational
drawback, because educators in low SES or low attainment schools typically have much farther
to go to move their students to score levels easily reached by high SES or high attainment
students taught by other educators. Thus their belief that they can achieve an ambitious test score
goal is likely to be very weak. However, in some instances attainment may be an appropriate
performance indicator in an incentive program. Some districts have schools in which most
students are already at high levels of attainment, so there is little room to improve test scores or
proficiency rates. If the district wants to include these schools in an incentive program, it may
want to include an incentive for maintaining the high level of achievement, plus or minus a
factor that accounts for the measurement error of the tests.
Defining Student Achievement by Cross-Cohort Change in Attainment
Defining school or teacher performance in terms of cross-cohort change in attainment is the basic
approach used for school accountability under the Federal NCLB act. School or teacher
performance is measured by comparing test scores for the same subject and grade level across
yearly cohorts of students. Incentives can be offered for improving on last year’s scores or
proficiency levels by some percent or for reducing the difference between a desired attainment
level (e.g., 100% of students scoring proficient or above) and last year’s level. Additional goals
may be set for improving the average test scores of those in the lower categories, or for
decreasing the percent of students in a “below basic” category, in order to prevent all the
attention being paid to students likely to be close to the proficient cut off. This approach is fairer
because it compares schools or teachers to their own past performance and partially controls for
determinants of achievement such as student socio-economic status (which, especially at the
15
school level, is likely to be similar year to year). It is likely to be more motivating to teachers and
schools with lower performing students, because the goal is to improve from the current level,
not to meet a performance level that could be far beyond what is possible. The problem with this
approach is that because the students in each cohort are different, it is possible that changes in
test score averages or proficiency percentages over time are influenced by differences in the
students tested as much as by school or teacher efforts. Teachers often perceive that certain years
of students were better prepared, easier to work with, or more motivated, and thus may question
the validity of the cross-year comparison.
Defining Student Achievement as Same-Student Change in Attainment
Defining school or teacher performance as the average change in attainment across years for the
same group of students (e.g., test score average or percent proficient in Grade 4 math versus
score average or percent proficient in Grade 5 math) avoids the major drawback of cross-cohort
comparison. But it requires tests that are designed to measure the progression of student learning
from year to year (be vertically linked) and data systems that allow tracking individual students
across grades. If proficiency tests are vertically linked and student tracking systems in place, a
district could set goals for moving students between proficiency levels (e.g., move a certain
percent who were below basic last year to basic this year, or from proficient last year to
advanced this year) or for increases in average test scores. Paying for improvements in
attainment focuses attention on the bottom line: raising students’ proficiency to the levels
defined by state content standards. The drawback to this approach is that it may not be fair to
expect different teachers and schools to improve student achievement by the same amount each
year if there are substantial differences in student socio-economic status or prior learning across
schools and classrooms.
Defining Student Achievement Using Value-Added Methods
The value-added approach attempts to isolate the contribution of the teacher or school to student
achievement by controlling for student (and sometimes classroom or school) characteristics that
influence learning but are not under teachers’ or schools’ control. There are two basic valueadded approaches. The first defines the value-added by a school or teacher as the average “gain”
or change in student test scores, typically over one school year, after controlling for student
characteristics. This version relies on tests that are vertically scaled so that it makes sense to
estimate a gain by subtracting the prior from current year scores. The most sophisticated versions
of this approach estimate multi-year growth curves for students and define teacher or school
value-added as the difference between the students’ actual test scores and the score that would be
predicted given an extension of the growth curve. This approach lends itself well to thinking
about value-added as school or teacher contributions to student learning gains. The second is to
compare the actual test scores students receive in the current year with test scores that a
statistical model predicts they would have received, based on a variety of factors such as prior
year test scores, student background characteristics, and school features. This approach does not
require vertically scaled tests. Using this approach, value-added by a teacher or school is best
thought of as the difference between the actual test scores received by a teacher’s or school’s
students and the average test score of all students in the district (or even state) with a similar
prior year test score and similar background characteristics. Both approaches can become quite
16
complex, as researchers try to isolate school or teacher contributions to student learning from
factors outside schools’ or teachers’ control. The literature on value-added continues to expand,
and many of the finer points are still being debated. Good introductions or commentaries have
been written by Meyer, Braun, Rivkin, and Raudenbush. 22
Because value-added models attempt to remove as many confounding factors as possible so that
the contribution to learning that can be attributed to a school or teacher can be accurately
estimated, it would seem to be the best measure of school or teacher performance to use in an
incentive pay system. However, this approach has two drawbacks: complexity and the relative
nature of performance comparisons based on value-added. Complexity is a problem because
most teachers do not understand how value-added is calculated. Because the process is a black
box, it is easy for teachers who don’t do well to believe that the performance measure may have
been manipulated by administrators or calculated in error. As one Dallas teacher said about what
he contended was an inaccurate value-added score (called a CEI) "Look, I don't know a CEI
from Yankee Doodle, but if you're going to grade my performance on faulty information, I have
a problem with that." 23 Because the calculations are hard to understand, most teachers are also
unlikely to know what they need to do to improve their value-added.
It is important to recognize the relative nature of value-added estimates of school or teacher
performance. Value-added estimates are typically deviations from an average, typically the
average achievement level or change in achievement in the school, district, or state. Value-added
models compare schools or teachers to this average, rather than to an absolute goal for attainment
or improvement in attainment. This is why when incentives are attached to value-added
estimates, the incentive is usually provided to some top percentage of teachers and schools (see
Example 2). From a motivational perspective, this approach has two problems. First, because
value-added is measured relative to other teachers or schools, no matter how hard a teacher or
school tries, their value-added depends on how well other teachers or schools do. So an
important factor in whether teachers or schools receive the incentive or not is outside their
control. Second, since teachers do not know the value-added that other schools or teachers are
going to achieve, it is hard for them to judge whether their efforts will be sufficient to earn the
incentive, and hard to set a target for their own performance. It is also clear that some teachers or
schools have to be at the bottom (below average) no matter how well they are doing in absolute
terms. According to the motivational model presented above, these characteristics will reduce the
motivational impact of the incentive because teachers are likely to be uncertain about how their
efforts can result in meeting the performance goal and about what the level of the goal actually
is.
An incentive system based on value-added that explicitly ranks teachers or schools by valueadded and rewards only some top percent is what economists refer to as a tournament model.
Tournaments are often used in the private sector for sales and executive jobs where competition
is used to sift out and retain only the highest performers. A tournament model is attractive for an
incentive system based on value-added since many value-added models can only reliably
differentiate between the very top and very bottom performing schools or teachers. But in order
to motivate average and low performers, motivation theory implies that these educators need to
believe they can equal or beat the schools or teachers who were identified as the best last time
around. Those serving historically lower performing students, who have not been successful in
17
the past, may have difficulty believing that they can succeed. This will likely lower the
motivational impact of the incentive. A related problem, which we have seen in our own
research, is that if the low value-added teachers do improve instruction substantially, the
variation in value-added across teachers in a school or district declines. It is therefore harder to
distinguish between high and low performers and harder to decide who should be rewarded.
Improvement by those at the bottom actually makes it less likely that any teacher or school will
be rewarded, which is not likely to improve motivation to share best practices or cooperate
toward system-wide improvement.
Choosing the Method
Granting that there is no perfect way to measure school or teacher performance, 24 what method
should a district designing an incentive pay system use? The answer involves consideration of
both the theory of action by which incentives are intended to influence student achievement and
the district’s other performance improvement strategies.
Value-added is clearly the most appropriate way to define performance if the theory of action is
teacher sorting. As mentioned above, sorting relies on incentives that help attract and retain more
productive teachers. Value-added is a more accurate estimate of teacher or school productivity,
and it is best to sort on the most accurate productivity measure. That value-added estimates of
productivity are relative to a group average is not a problem here, even in a tournament model,
because the goal is to retain the best teachers in the group. An individual teacher incentive based
on value-added fits well with a strategy of attracting and retaining top instructional talent. A
comprehensive approach would involve more selective recruiting from known sources of quality
candidates, hiring based on valid predictors of future performance, more aggressive use of the
probationary pre-tenure period to winnow new teachers, and more aggressive efforts to
remediate or outplace the least effective teachers.
Cross-cohort and same-student attainment change measures seem like a better fit with the
motivational theory of action. Attainment change measures are much less opaque to teachers and
make it easier to set specific goals. Specific goals are easier to aim for, and assess progress
toward. Unlike value-added, there is no inherent barrier preventing all schools or teachers from
having an opportunity to succeed. A team or school-level incentive based on attainment change
would be a good fit with a strategy of improving the instruction of the average teacher or school.
Focusing on improving student proficiency levels or average test scores provides a more easily
understood metric to assess changes in instruction, and one that is directly related to the “bottom
line.” And an incentive that does not require one teacher to lose for another to win will be less
likely to inhibit sharing of successful practices and to support the cooperative, school-level
professional development, coaching, and teacher acculturation needed to substantially improve
achievement at most schools. 25 A comprehensive approach would provide teachers and schools
with diagnostic assessments linked to the state tests that help them identify where students need
more help and how well current instruction is moving them toward achievement goals. The
diagnostic tests would provide a baseline from which to measure and compare changes in
achievement brought about by professional development and instructional interventions.
18
Incentives based on value-added or either attainment change approach can work with the culture
change theory of action, though they do so in different ways. Value-added measures send the
message that some schools and teachers are better than others, and that these are the ones that
should be emulated. This helps build a culture that values performance. But the attainment
change approaches have some advantages in communicating and building a performance culture.
One is that change in attainment keeps the focus on the ultimate goal of student proficiency.
Right now there is no easy way to relate specific levels of value-added to specific improvements
in student proficiency. In fact, there is no research yet that tells us whether even the highest
levels of value-added observed in a district would be enough to meet proficiency goals. It is
likely that instruction in even the highest value-added schools and classrooms needs to get better,
but simply rewarding those with the highest value-added does not communicate this. The
message is “you are already the best,” from which it is easy to infer that no change is needed.
These schools or teachers also have little motivation to share their best practices, especially in a
tournament model, since doing so reduces their chance of winning again in the next cycle.
Further, like it or not, the current version of NCLB’s Adequate Yearly Progress defines school
success as improvement in attainment across cohorts. Basing all of the incentive for improving
student achievement on value-added alone sends a mixed message to teachers about what is
important. At best it introduces a degree of goal conflict that is likely to reduce the motivational
impact of both the incentive and NCLB.
When vertically-scaled tests are available that make year-to-year test score or proficiency level
comparisons possible, the same-student change in attainment model may be the best compromise
between complexity and fairness. A district could provide schools or teachers with last year’s
test scores for all incoming students at the beginning of the year, then set goals in terms of an
average score improvement for each teachers’ or schools’ students. The district can set the yearly
goals with reference to long term goals for moving students up to higher proficiency levels, thus
maintaining the link with what is ultimately important. Since each school’s or teacher’s goal is to
make some progress with each student starting from where they were at the end of last year, it is
fair to teachers or schools with many low achieving students. It gives credit for individual
student progress facilitated by the teacher or school across the achievement distribution and
reduces the incentive to focus on “bubble kids” near proficiency cut-offs. This approach could be
viewed as a very simple value-added model. It is being used in the Orange County, Florida
district (see Example 3). While it does not control for all of the student background factors that
affect achievement growth but which are outside of teacher or school control, it is more
transparent than more complex value-added models, fairer than cross-cohort comparisons, and
allows setting specific goals that can be met without reference to how well other teachers or
schools do.
Some Promising Variations
A district might consider using both change in attainment and value-added measures, but at
different levels. One possibility would be to reward improvement in attainment at the school
level and value-added at the teacher level. This would provide incentives for working toward the
ultimate goal while also recognizing the most productive teachers. The teacher-level value-added
incentive would contribute to improving student achievement through the sorting effect. The
school-level improvement in attainment incentive would reinforce schools’ focus on the ultimate
19
goal, keep the pressure on schools and the district office to systematically improve the conditions
for teaching and learning, and activate cultural influences like mutual monitoring, as well as be
more consistent with school goals already set under NCLB. Note that the Charlotte-Mecklenburg
district uses two kinds of goals, though both are defined at the school level (see Example 1). In
this example, there are two tiers of school-based performance awards. Meeting a cross-cohort
improvement in attainment goal (i.e. the school’s NCLB goals) or reaching a target level of
student achievement gain (based on a simplified value-added model) earns staff at a school one
level of the bonus, while meeting both earns a higher level. The program recognizes both highly
productive schools and schools that are moving toward the proficiency standard.
Some of the motivational shortcomings of value-added as a performance measure might be
addressed by rewarding improvements in value-added, either at the school or teacher level. This
would require developing a baseline value-added estimate for each teacher or school, using
perhaps 3-4 years of test score data to get a reliable estimate of value-added. Then, in subsequent
years, the teacher’s or school’s yearly value-added would be compared to this standard. If is
higher (by an amount large enough to be greater than measurement error), the district would pay
a bonus. The baseline would be re-established every few years to keep the incentive on
improving productivity. 26 This approach is somewhat like that used in the North Carolina
school-based award system (described as part of Example 1), which fixes a periodic baseline for
student learning gains. The advantages of this approach are that schools or teachers would have a
specific goal to shoot for, and unlike a tournament model, all have a chance to earn the bonus
without others having to lose to the competition. This should improve the motivational potential.
The disadvantages are that it requires a substantial amount of data for each teacher or school, and
requires re-calibration when the tests used to measure student achievement change substantially,
or when the student population served by the teacher or school changes drastically. To be most
effective, this approach would also require supports for teachers or schools trying to improve
their value-added, including professional development on instructional practices that have been
shown to improve value-added. Note that this brings us back once again to the district’s strategy
for improving student achievement and the desirability of aligning the incentive program with
that strategy.
4. SHOULD OTHER INDICATORS OF PERFORMANCE BE INCLUDED IN A VARIABLE PAY
PROGRAM?
Districts can add other indicators of performance to a variable pay program, no matter which pay
program they have selected. These indicators can support the strategies for improving student
achievement. Additional indicators may be especially appropriate for school-based rewards,
since instructional improvement strategies ranging from building professional learning
communities to curriculum alignment frequently need to be implemented building-wide or
department-wide to succeed. The indicators to consider are those that measure strategy
implementation and other potential drivers of student achievement. For example, consider a
strategy that includes improving teacher skills and instructional coordination by implementing
professional learning communities 27 and aligning curriculum across grades and to state
standards. The goals for which the incentive would be paid could include the degree to which
curriculum alignment and professional learning communities had been implemented in schools.
Principals and teachers in schools showing a target level of implementation or a substantial
20
improvement in implementation would receive some portion of the total potential bonus
payment. Another example would be including an incentive for teacher and/or student
attendance. Since students are not likely to benefit from even the best instruction if they are not
in school, and improvements in teacher skills are likely to be less effective if substitutes must be
extensively used, attendance might be a key factor in the success of a strategy. An incentive
might encourage educators to find ways to improve attendance. The Hypothetical
Multidimensional Incentive Program, discussed next, shows how a program with other strategic
indicators might look.
Hypothetical Multidimensional Incentive Design
This is an example of a school-level group incentive that rewards both improvements in student
proficiency and supports district strategies for improving student achievement by rewarding
strategy implementation. The two strategies the incentive rewards are supporting are: a) a
curriculum that is aligned across grades and with state standards and assessments, and b)
increasing student attendance. Goals are set on five dimensions:
1. Decrease the percent of students below ‘proficient’ on state reading tests.
2. Decrease the percent of students below ‘proficient’ on state mathematics tests.
3. Increase the average scale score of those students in the lowest quartile of reading and
math tests.
4. Improve student attendance.
5. Implement the district’s aligned curriculum model.
The dimensions are given weights and different goals levels are developed. Goals for the first
three dimensions are based changes in attainment. Goals for the other two dimensions are based
on attaining specific percentages. Goals for these two can be set based on historical experience
and district averages. As shown in the table below, total point score is calculated by taking each
dimension, finding the level of goal attainment, multiplying the point value at the bottom of the
appropriate column by the weight, then dividing by 100. For example, if a school decreases the
percentage of students below proficient in reading by 3%, the points for that dimension would be
(80 X 30)/100 = 24. Adding the dimension scores results in a total score, which is then
compared to a payout table that gives the amount of the incentive bonus.
This method of calculating a total point score, then making the bonus amount proportional to the
number of points earned, not only makes it possible to combine different levels of goal
attainment across multiple performance dimensions, but also allows for a graduated series of
goal levels and bonus amounts. This avoids having to decide on one goal attainment level that
would be reasonable for all schools to meet, and one corresponding bonus amount. It also gives
more schools the potential for earning at least some bonus amount. The threshold performance
level is thus set at the lowest level of goal attainment viewed as being of practical value by the
district, and above a level of change that would be likely to be due to measurement error.
21
1. Decrease % below
proficient in reading by:
2. Decrease % below
proficient in math by:
3. Increase scale score
average by:
4. Percent average daily
attendance:
5. Percent of classrooms
aligned:
Weight
Maximum
Target
Threshold
Dimension
Dimensions, Goals*, and Points
2%
3%
4%
5%
6%
7%
8%
25%
2%
3%
4%
5%
6%
7%
8%
25%
1 pt.
2 pts.
3 pts.
4 pts.
5 pts.
6 pts.
7 pts.
20%
93%
94%
95%
96%
97%
98%
99%
15%
75%
80%
85%
90%
93%
95%
100%
15%
Points
70
80
90
100
110
120
130
*Goal values for illustration only. Percent or point values would depend on exiting performance levels and test
scales in an actual program.
A hypothetical payout table to go with the points table above is shown below.
Payout Table
Target Bonus Amount = $1,500
Bonus
Amount
$0
$750
$1000
$1500
$1750
$2000
$2250
Outcome
< 70 points
70-79 points
80-89 points
90-99 points
100-109 points
110-119 points
120-130 points
Note that the target amount is set at a level that most teachers in the district would find
meaningful. The threshold amount is set at 50% of this level and the maximum amount at 150%.
In an actual program, these amounts would be set after ascertaining their potential acceptability
to teachers and the district’s ability to pay.
The downside here is of course that more goals make the program more complex, potentially
reducing teachers’ understanding of how the program works and removing some of the
motivational focus from student achievement outcomes. Some of the loss of focus may be
compensated by the gain in line of sight. Teachers may be more inclined to believe that they can
influence the strategy implementation measures than the student achievement measure because
they have more control over their own behavior. Also, teachers in historically lower performing
22
schools might be more motivated because they may believe that they have at least some chance
to meet the goals set for strategy implementation. If the strategies that are included in the
broadened incentive program are tightly coupled to student achievement, the wider focus will be
less of a problem. The key to making this trade-off work is whether the strategies are actually
likely to be effective. (If districts are unsure of the effectiveness of specific strategies, they may
not want to couple them to the incentive program.) This approach also requires developing
reliable measures of the additional performance dimensions.
5. HOW BIG SHOULD THE INCENTIVE PAYMENT BE?
This is one of the most difficult questions designers of variable pay have to confront. Among the
factors to consider are the theory of action by which the district expects the incentive to work
(i.e., direct motivation, sorting, or culture change), the level of the reward (i.e., school, team,
individual), the difficulty of the performance goals to be rewarded, the degree of trust between
teachers and district leaders, and the district’s ability to pay. Because these vary from district to
district, a definitive recommendation on bonus amounts cannot be given here. One way to think
about the question is to start with establishing a minimum amount that would get teachers’
attention, then adjusting that amount based on the factors mentioned above.
Surprisingly, relatively little research has been conducted on how big an incentive is needed to
begin to motivate workers in any sector. The few results we have define a fairly broad range: 3 to
12% of base pay, with most tending toward the higher figure. 28 Research has not been conclusive
because there are large individual differences among people in the value they place on financial
rewards, and because the motivating power of the reward also depends on the perceived costs of
earning it, which also differs across people and situations. A survey of private sector
performance incentive programs found that the median payout was around 5% of base pay. 29 Our
own research on school-based awards suggests that bonuses less than $1,000 (between 3 and 4%
of base pay at the time) had minimal motivational impact. 30 The amount deducted for taxes
needs to be considered as well, since winners will see less than the nominal amount on their
checks. Bonuses in the $1,500-2,000 range, or 5% of the average teacher salary, may be needed.
Design processes that involve teachers (as we recommend below) provide the opportunity to
verify the meaningfulness to teachers of these amounts.
Figure 2 summarizes how considerations of goal level, use, trust, ability to pay, and goal
difficulty might be used to adjust the bonus above or below the target amount.
23
Figure 2: Decision Considerations for Size of Bonus
Individual basis of goals?
Use to increase direct
motivation?
Use to encourage teacher
sorting?
Difficult goals?
Start with a
meaningful
amount
(minimum of
$1,500)
School basis of goals?
Use to influence culture &
management?
Adjust
Upward
Stay near
meaningful
minimum
Adjust
Downward
Low trust?
Low ability to pay?
The theory of action and level of the incentive (school, team, individual) are primary
considerations. A school-based incentive intended to influence culture rather than directly
motivate effort should be set near the meaningful minimum. School-based rewards intended to
directly motivate increased effort need to use considerably higher amounts. Because they have a
less direct line of sight between individual effort and goal attainment, school awards are likely to
have less direct motivational impact than individual or team-based incentives. A higher bonus is
needed to overcome these limitations. An individual incentive intended to affect teacher sorting
is likely to require the use of a bonus significantly larger that the meaningful minimum.
Similarly, if the purpose of an individual incentive is to motivate increased effort, a bonus above
this minimum will be needed to keep the performance goal in teachers’ minds over the course of
a busy school year. In general, if the aim is to use incentives to motivate effort, larger bonus
amounts are likely to work better than smaller ones. Too small an incentive prompts a negative
answer to the question “Is the extra effort worth the reward?” According to the motivation
model, to be motivating the value of the reward needs to be judged large enough to overcome the
extra work and stress that are perceived to be involved with earning it.
Ability to pay and trust are also important. If ability to pay and/or trust is low, districts should
choose an amount lower than the $1,500-2,000. Ability to pay is not only a matter of having the
available funds to finance the bonus this year, but also the amounts needed to do so in the future.
As the motivation model implies, the certainty with which teachers believe that attaining
performance goals will lead to a payout affects the motivating potential of the bonus. If teachers
believe that bonus amounts are set too high for the district to sustain, a high bonus amount may
not be much more motivating than the minimum needed to get teachers’ attention. (California’s
short lived school awards are a classic example of bonus amounts too big to sustain. 31 ) Where
trust is low between teachers and district leaders (or between teachers and the school board or
state legislature), the likelihood of receiving any amount may seem low to teachers. In low trust
situations, it may be better to start with amounts somewhat below the meaningful minimum, and
sustain them for a few years, in order to build trust in the system.
24
A district may also want to establish multiple performance levels and provide different bonus
amounts for each level attained. Both the Charlotte and Dallas plans use this approach. If
multiple bonus amounts are chosen, the district might want to first establish a target amount,
based on the considerations discussed above. This would be linked to a substantially improved
but attainable target level of performance. Then a smaller amount, say 50% less) would be linked
to a threshold level that represents the smallest significant performance improvement thought
worth rewarding. A maximum amount (say 150% of target) could be linked to achieving
performance substantially above the target performance level. This would represent a “stretch”
goal that only the best performers would be likely to reach. The advantages of this approach
include giving more teachers the chance of receiving at least some incentive payment, serving as
a “consolation prize” for those who just missed the target level, and providing a reason for
working toward bigger improvements in student achievement for those who could easily make
the target level without too much extra effort. Private sector plans sometimes establish a range of
performance levels and bonus amounts extending in small increments from a threshold to a
maximum amount. This minimizes “step” effects in motivation (i.e., the next level is too hard so
why bother to try?). Example 4, the Hypothetical Multidimensional Incentive Design, includes
this concept.
Before leaving the question of the size of the incentive, it should be noted that it is much less
vexing in the private sector, where the value of the various levels of increased productivity can
be at least estimated in dollar terms and compared to the likely bonus payout. Because any
increased productivity or performance contributes to increased revenue, private sector incentives
can pay for themselves, which lessens worker concerns with sustainability. Further, the amounts
to be paid out can be made proportional to the value of the increased performance, making
incentive size decisions simpler. In contrast, a school district cannot directly market any
increased student achievement to raise additional funds to pay the bonus, or even value the
addition easily enough to make bonuses proportional to its value. Since incentives are a cost to
be paid from a fixed budget deriving from tax revenues, school districts have inherently less
flexibility in setting bonus amounts.
6. PROCESS GUIDELINES FOR DESIGNING TEACHER VARIABLE PAY PLANS
This section presents ten practice guidelines for designing plans to pay teachers for student
achievement. They are based on the private sector literature, research done by the CPRE Teacher
Compensation project, and the emerging experience of districts in developing and successfully
rolling out incentive programs. These guidelines are not all-inclusive. Readers are referred to the
resources for design provided at the end of this section for additional information.
Know What the District Wants to Accomplish by Paying for Improved Student Achievement
Paying teachers for improved student achievement is not by itself a sufficient strategy for
improving student achievement. Pay is a critical support, but not a driver of performance
improvement, even in the private sector. Discussing the change process that private sector
organizations were undergoing to become more competitive in the 1980’s and 1990’s, members
of one prominent consulting firm wrote: “…pay itself cannot drive or lead the change process. It
25
cannot define what change should be. It cannot establish values. It cannot replace effective
leadership.” 32
So the first priority is to think through how paying for student achievement will support the
district’s other performance improvement strategies. Is pay meant to be a motivator, an influence
on recruitment or retention, or an agent of culture change and management practice
improvement? A clear articulation of the purpose of the incentive plan, and how it is intended to
work together with other performance improvement initiatives, will help guide all other design
decisions and help ensure that the plan supports, rather than working at cross purposes to, the
district’s other efforts to improve student achievement.
Before plunging into the details of incentive plan design, a district should review its strategies
and plans to improve student achievement. For example, incentive plan designers should analyze
the district strategic plan to identify student achievement goals that are worth rewarding, the
specific strategies the district is using to meet these goals and how performance pay can support
them, and the enablers the district needs to provide to schools and educators to make the
strategies work. As the discussion of motivation pointed out, incentives will not be as effective in
encouraging teachers to focus and persist in their efforts if there are major roadblocks to carrying
out the district’s strategies for improving student achievement. Teachers’ perceptions that the
district is trying to remove these roadblocks, and its success in doing so, will foster the
perception that improving student achievement is possible, increasing motivation to try to do so.
Create the Design Process to Maximize the Chances of Success
Depending on where the district is in its journey, designing a performance incentive program
could take from nine months to two years. The key determinants of the length of the process are
how well the district has coalesced around a strategy to improve performance, and the degree of
trust among the major stakeholders. The two are likely to be related, and the higher the trust and
consensus on the strategy, the faster the plan can be designed. Thus if a plan is to be used in a
high trust environment to support other change efforts already agreed upon, nine months would
be plenty of time to develop a plan. In a low trust environment with little consensus on
performance improvement strategies, a full year or more might be needed for development. This
is not necessarily a problem if the variable pay design process is being used as the vehicle to
develop and gain consensus on a district strategy for improving performance.
To begin the process, the district should create a design team or steering committee that includes
representatives of all the important stakeholder groups likely to be affected by the plan.
Naturally, this group should include teachers and administrators from all three school levels
(elementary, middle, and high schools) and from a mix of more and less successful schools. The
committee should include a manager who can speak for the superintendent, and representatives
of teacher and/or principal associations if present. Including one or two key central office staff,
such as the HR director and assessment/evaluation director to provide expertise on technical
issues is also advisable. A primary committee of 10-15 people should allow enough
representation without making the steering group unwieldy. Subcommittees focusing on specific
parts of the design process, headed by steering committee members and drawing in more
participants, can be used to expand participation and bring needed expertise to particular issues.
26
The books by Odden and Wallace and Odden and Kelley cited in the resources section provide a
good overview of design team structure and process.
A key function of the design team is to articulate a vision for what the performance pay plan is
supposed to accomplish. This is important because as argued above, the primary purpose of
performance pay is to help accomplish strategic goals. An effective plan must be derived from
key educational and organizational goals, and the design must start from them. So some time and
effort need to be spent communicating about and discussing the district’s goals and how
performance pay is supposed to contribute. To start the development of a program, the designers
should develop a purpose statement that explicitly sets forth the goals of the program and the
educational goals it will support.
Another important design team function is communication. Lack of initial acceptance or lack of
support once implemented is often due to lack of understanding of the program’s intent and
design. Since communication with those affected by the program will be a key factor in their
acceptance and participation, the design team should develop an explicit communication plan.
The Denver ProComp plan has been an exemplar of communication, and any design team can
profit from looking at the ProComp Web site, http://www.denverprocomp.org. Communication
should start early in the design phase and continue throughout the life of the program. The guide
to stakeholder engagement and communication on the Center for Educator Compensation
Reform Web site http://www.cecr.ed.gov/guides/stakeholderEngagement.cfm provides a good
overview of communication issues and communication planning.
Review the Affected Educators’ Total Compensation Package
Since incentives for student achievement likely will be a relatively small component of total
compensation, the burden will continue to fall on salary and benefits to attract and retain
sufficient numbers of highly qualified teachers. The salary and benefit package must be
competitive before embarking on a performance pay program. A variable pay plan built atop
noncompetitive base salaries and benefits will have trouble winning teacher acceptance and have
limited motivational impact. It may be more useful to redesign base pay programs and benefits if
the basic problems facing the district include difficulties in attracting and retaining quality
teachers and school administrators. In order to use pay to motivate improvements in student
achievement, the district needs a sufficient number of educators who have the ability to make
improvements. The use of base pay schedules to attract, retain, and motivate the development of
teacher competencies was covered in the prior paper in this series.
Assess the Potential for Financial Sustainability
Before committing to an incentive plan, the district needs to take a hard look at its ability to
obtain stable and adequate funding of the incentive program. Without such funding, the program
will stand a good chance of fading away. Even before it does, its motivational impact will be
diluted because teachers may not believe that if they succeed in improving student achievement,
the money will be there for the promised incentives. If only small amounts can be offered as
incentives (less than $500 per year) the district should consider using the money in other ways.
The funding need not be an infusion of new dollars from outside the system; funding also can
come about internally through a combination of resource reallocation, veteran teacher attrition
27
(to be replaced with less costly novices), and reduced backloading of the single salary
schedule. 33
Be Sure to Include Principals and Other Administrators Along with Teachers
Any effort to implement performance incentives for student achievement should also include
principals, administrators, and superintendents. Excluding them sends the wrong signals to
teachers: that the burden of improving student achievement falls solely on their shoulders, and
that principals and administrators are exempt from the need to contribute. As argued above, a
performance pay plan should be part of and should support an overall performance improvement
strategy, and so should include all of those responsible for implementing the strategy.
Assess Readiness for Variable Pay
Compensation can be a powerful force for change, but also a powerful source of disagreements
and dissention. The current pay system is deeply embedded in the culture of public education. A
major change in incentives—such as payment of individual classroom student achievement—
may cause a considerable degree of conflict and confusion since teachers will likely be uncertain
of how well they will do under the new incentive regime, and uncertain as to just how to achieve
the goals being incentivized. The new incentive system may be seen as a violation of the old
“psychological contract” 34 teachers may perceive they have with the district: stay around,
continue your education, and perform to minimum standards and you will move up the pay
schedule. The resulting stress and fear may cause opposition to the incentive program. Despite
the increasing use of variable pay, many teachers are still suspicious. Even the carefully crafted
and communicated Denver ProComp plan received less than 60 percent vote approval from
teachers, and only about 30 percent of eligible teachers opted in during the first two annual optin periods.
The stronger the incentive for student achievement is, the more likely it is to change the old
organizational culture, but also the more likely it is to provoke conflict and confusion. Designers
of strong incentive systems need to assess the district’s ability to tolerate this. Factors that affect
this ability include the sense of urgency felt by all those involved and the level of trust between
teachers and district leaders. While private sector experience suggests that changing
compensation systems can be effective in supporting culture change, 35 change in supporting
systems that give teachers the tools to understand and earn the reward need to be made
alongside. These include detailed information on how the plan works, the opportunity to develop
the skills needed, and the flexibility to make the changes in practice needed to improve student
achievement. Districts therefore also need to assess their capacity to truly support teachers in
succeeding under the variable pay design chosen.
Another important area of readiness is district measurement and data collection systems. A
variable pay program depends on the systems that measure the results for which incentives are
being offered. The system must provide reliable, valid measurements of student achievement,
and link them to the level (e.g., school, team, classroom) at which the incentive is defined.
Using inaccurate, delayed, or difficult-to-retrieve performance data will undermine perceptions
of the plan's fairness and threaten its survival. As districts working in this area have discovered,
28
building the necessary data systems can be a complex undertaking, especially when there is the
need to link teachers to the schools they teach in, their students, the subjects taught, and test
results. An overview of the information technology considerations involved in designing a
performance incentive system can be found on the Center for Educator Compensation Reform
Web site at http://cecr.ed.gov/guides/itConsiderations.pdf .
At this point, it may be important to recognize that the district is not yet ready to add a variable
pay element to its compensation program. Given the complexity of successfully implementing a
system that will actually help improve student achievement rather than simply following a
reform trend, it would be better not to go forward than to implement a system destined to fail.
This just poisons the well for future compensation changes as well as wasting district resources
that could be better employed toward proven methods of improving student achievement. 36
Plan for an Effective and Lasting Implementation
Our own research and the unfortunate experiences of several districts reported in the media,
suggest that implementing an incentive pay program is a complex task, and that implementation
glitches reduce the credibility of the plan and in turn reduce teacher acceptance. Poor
implementation can also lower motivational impact when glitches raise issues about fairness and
the validity of measurement. Thus after the plan is designed, the design team also needs to
develop an implementation plan. This plan should identify who is responsible for plan elements
such as ensuring performance data is collected and analyzed, and when that will be done. All
deadlines and procedures need to be reviewed for feasibility by the people who have to carry
them out (e.g., the testing and evaluation department, information technology staff running the
employee and student information systems, payroll). Designers need to spend as much effort
planning the implementation as they did designing the plan, to avoid changing timelines and
modifying design elements midstream (and confusing teachers and administrators). For the more
complex plans, one of the best defenses against implementation problems is a pilot or test year
during which only some schools or teachers participate, or during which measurements are made
but no money is yet at stake. It may help to explicitly identify the first year as a pilot period, after
which an evaluation will be done and refinements made.
In the long run, almost all incentive pay systems are at risk of fading away due to changes in
funding and organizational priorities. Compensation plan designers from the private sector
recommend several things that might be done to improve sustainability. First, a "champion" for
the plan can be identified, and made responsible for overseeing its implementation and the
evaluation of its effectiveness. Second, top management should be continually engaged with the
plan, communicating its importance and the district’s commitment to it to the educators affected.
This engagement will be more likely if the plan truly supports the district’s overall strategies for
improving student achievement and rewards attaining its strategic goals. This is another reason
why integration of the plan with these goals and strategies is important.
Third, districts need to consider the alignment of other parts of the human resource management
systems with the incentive plan. Many of the drivers of educator performance improvement
reside within the human resources (HR) domain. 37 If HR programs are aligned to support the
incentive plan, its chances of sustained success will be improved. For example, during
29
recruitment, educators can be informed of the incentive plan and those who do not believe they
can succeed can self-screen out of the selection process. Interviews and other screening tools can
be focused on the competencies that educators need to succeed in improving student
achievement. Professional development programs can be revamped to support the competencies
educators need to carry out the strategies that the district believes will improve student
achievement. These efforts at alignment will help ensure that the district’s educators have the
needed competencies and believe they have what it takes to earn the incentive. This in turn will
maximize the motivational impact of the incentives.
Consider Developing a Comprehensive Compensation Program that Includes More Than
Just Paying for Student Achievement
Since teacher skills are an important influence on instructional quality, and since instruction as
provided is a key driver of student learning, it makes sense to consider pay innovations that
reward teachers for developing the skills needed to improve instruction and for implementing
instruction that is consistent with the district’s strategies for improving student learning. Several
districts have broadened their performance pay programs to reward these things. Douglas County
and Denver, Colorado, have models of combined performance pay plans. Schools using the
National Institute for Excellence in Teaching’s Teacher Advancement Program (e.g., in Chicago,
Minneapolis, and South Carolina) based variable pay on both student achievement and
instructional practice. These programs attempt to use pay in ways that support each other by
rewarding both the ultimate outcome—student achievement—and educator behaviors that
contribute to that outcome. Properly designed and implemented, such programs have the
potential to have a stronger effect than paying for student achievement alone. The trade-off is the
additional complexity and administrative overhead these efforts typically require. Some districts
combine student achievement incentives with incentives to attract high quality teachers to high
poverty or low achieving schools. This is another way to address the instructional capacity issue
so that these schools have a chance to succeed.
30
Resources for Designing Incentives for Student Achievement
Books, Briefs, and Monographs
Azordegan, J., Byrnett, P., Campbell, K., Greenman, J., & Coulter, T. (2005 November).
Diversifying Teacher Compensation. Denver: Education Commission of the States.
Gorning, P., Teske, P., & Jupp, B. (2007). Pay for Performance Teacher Compensation: An
Inside View of Denver’s ProComp Plan. Cambridge, MA: Harvard Education Press.
Heneman, H.G., III, Milanowski, A., & Kimball, S. (2007). Teacher Performance Pay: Synthesis
of Plans, Research, and Guidelines for Practice (CPRE Policy Brief RB-46).
Philadelphia, PA: University of Pennsylvania, Graduate School of Education, Consortium
for Policy Research in Education.
Kelly, C., Odden, A., Milanowski, A.T., , & Heneman, H.G., III. (2000). Motivational Effects of
School-Based Performance Awards (CPRE Policy Brief RB-29). Philadelphia, PA:
University of Pennsylvania, Graduate School of Education, Consortium for Policy
Research in Education.
National Governors Association, Center for Best Practices. (2007). Improving Teaching Through
Pay For Contribution. Washington, DC: Author.
Odden, A., & Kelly, C. (2002). Paying Teachers for What They Know and Do: New and
Smarter Compensation Strategies to Improve Schools (2nd ed.). Thousand Oaks, CA:
Corwin Press.
Odden, A., & Wallace, M. C., Jr. (2007a). How to Create World Class Teacher Compensation.
St. Paul, MN: Freeload Press.
Odden, A., & Wallace, M. C., Jr. (2007b). Rewarding Teacher Excellence: A Teacher
Compensation Handbook for State and Local Policymakers. Madison: University of
Wisconsin, Wisconsin Center for Education Research, Consortium for Policy Research in
Education.
Web Sites
Consortium for Policy Research in Education, Teacher Compensation Project at the University
of Wisconsin-Madison: http://www.wcer.wisc.edu/cpre/ (Contains case studies of design
processes and research on teacher compensation innovations.)
Denver ProComp: http://denverprocomp.org/generalinformation (A good example of the type of
communication that teachers need to understand and accept pay system innovation.)
31
Education Commission of the States, Compensation and Diversified Pay Systems:
http://www.ecs.org/ecsmain.asp?page=/html/issues.asp (Contains an overview of state activities
and review papers on teacher performance pay.)
National Center for Educator Compensation Reform, sponsored by the U.S. Department of
Education, Teacher Incentive Fund: http://cecr.ed.gov/ (Contains capsule descriptions of the
programs supported by the Teacher Incentive Fund, as well as several useful documents
describing the requirements for successful incentive design and implementation.)
National Center on Performance Incentives at Vanderbilt University:
http://www.performanceincentives.org/ (Contains research on incentive plans and a literature
review of the effects of incentives. Includes an in-depth study of the Texas state incentive plan.)
32
Endnotes
1
Klein, A., and Hoff, D.J. (2007). Union leaders assail teacher ideas in NCLB draft. Education
Week, December 19, page1.
2
http://cecr.ed.gov/initiatives/
3
Blinder, A.S. (ed.)(1990). Paying for productivity: A look at the evidence. Washington, D.C.:
Brookings Institution; Bullock, R. J. and Tubbs, M. E. (1991). A case meta-analysis of
gainsharing plans as organizational development interventions. Journal of Applied Behavioral
Science, 26(3): 383-404; Thierry, H. (1987). Payment by results systems: A review of research
1945-1985. Applied Psychology: An International Review, 36,(1), 91-108; Jenkins, G.D.,
Mitra, A., Gupta, N; and Shaw, J.D. (1998).Are financial incentives related to performance? A
meta-analytic review of empirical research. Journal of Applied Psychology, 83(5), 777-787;
Lawler, E.E. III (1998). Gainsharing theory and research: Findings and future directions.
Research in Organizational Change and Development, 2, 323-344. Kruse, D.L. (1993).Profit
Sharing: Do s Ii Make a Difference? Kalamazoo, MI: W.E. Upjohn Institute for Employment
Research. Lazear, E.P. (2000). Performance pay and productivity. American Economic
Review, 90(5), 1346-61; Rynes, S.L., Gerhart, B, and Parks, L. (2005). Performance evaluation
and pay for performance. Annual Review of Psychology, 56:571-600; Honeywell-Johnson,
J.A., and Dickenson, A.M.(1999). Small group incentives: A review of the literature. Journal
of Organizational Behavior Management, 19(2), 89-120.
4
Lawler, E.E. III. (1971). Pay and Organizational Effectiveness: A Psychological View. New
York: McGraw-Hill; Locke, E. A., and Latham, G. P. (1990). A Theory Of Goal Setting and
Task Performance. Englewood Cliffs, NJ: Prentice Hall; Heneman, H.G. III. Assessment of
the motivational reactions of teachers to a school-based performance award program. Journal
of Personnel Evaluation in Education, 12(1) 43-59.
5
Gorning, P., Teske, P., and Jupp. B.(2007). Pay for Performance Teacher Compensation: An
Inside View of Denver’s ProComp Plan. Cambridge, MA: Harvard Education Press.
6
Scarpello, V., & Jones, F. (1996). Why justice matters in compensation decision making.
Journal of Organizational Behavior, 17(3), 285-299. Cooper, C., Dyck, B., & Frohlich, N.
(1992). Improving the effectiveness of gainsharing: The role of fairness and participation.
Administrative Science Quarterly, 37(3), 471-490. Welbourne. T.M. and Gomez-Mejia, L.R.
(1995). Gainsharing: A critical review and a future research agenda. Journal of Management,
21, 559-609. Gross, S.E. and Barcher, J.P. (1993, January/February). The new variable pay
programs: How some succeed, why some don’t. Compensation and Benefits Review: 51-56.
7
Lazear, E. (1996). Performance pay and productivity. NBER Working Paper 5672. Cambridge,
MA: National Bureau of Economic Research. Cadsby, C.B., Song, F., and Tapon, F. (2007).
Sorting and incentive effects of pay for performance: An experimental investigation. Academy
of Management Journal, 50(2),387-405.
8
Graham-Moore, B. and Ross, T.L. (1990). Gainsharing: Plans for Improving Performance.
Washington DC: Bureau of National Affairs; Hanlon, S.C., Meyer, D.G., and Taylor, R.R.
(1994). Consequences of gainsharing: A field experiment revisited. Group and Organization
Management, 19(1), 87-111.
9
Landel, D. (ed.) (2005). Performance-related pay policies for government employees. Paris,
France: Organization for Economic Co-operation and Development;
33
10
Welbourne, T., Balkin, D., & Gomez-Mejia, L. (1995. Gainsharing and mutual monitoring: A
combined agency-organizational justice interpretation. Academy of Management Journal,
38(3), 881-899.
11
Arthur, J,B., and Kim, D.O. (2005). Gainsharing and knowledge sharing; The effects of
labour-management co-operation. International Journal of Human Resource
Management,16(9),1564-1582.
12
See the citations in note 3 above.
13
Regression to the mean occurs because students with very high test scores in any one year are
likely to have somewhat lower scores, compared to the average, in the next, while students
with lower test scores are likely to do relatively better. This is because some of the factors that
contributed to the high or low scores (e.g., luck, being particularly well or poorly rested) are
likely to be absent in the subsequent year.
14
Milanowski, A.T., (2007). Performance Pay System Preferences of Students Preparing to be
Teachers. Education Finance and Policy, 2(2), 111-132.
15
Haines III, V.Y., and Tagger, S. (2006). Antecedents of team reward attitude. Group
Dynamics: Theory, Research, and Practice, 10(3), 194-205.
16
Clotfelter, C., Ladd, H., Vigdor, J., & Diaz, R. (2004, January 1). Do School Accountability
Systems Make It More Difficult for Low-Performing Schools to Attract and Retain HighQuality Teachers?. Journal of Policy Analysis and Management, 23(2), 251-271; Heneman,
H.G. III, and Milanowski, A.T. Employees’ withdrawal responses to their individual base pay
and group bonus pay system. Paper presented at the Southern Academy of Management
Meeting, New Orleans, Louisiana, November, 1998.
17
Joyce, B. (2004). How are professional communities created? Phi Delta Kappan, September.
18
McCafferey, D.F., Lockwood, J.R., Koretz, D.M., and Hamilton, L.S. (2003). Evaluating
Value-Added Models for Teacher Accountability. Santa Monica, CA: RAND Corp.;
19
Langland, C. (2001). Merit pay plan irked teachers. Philadelphia Inquirer. Downloaded
September 6th, 2001 from
www.inq.philly.com/content/inquirer/2..t_page/SSTRIKE06.htm?template=aprint.htm.
20
Winters, M, Greene, J.P., Ritter, G., and Marsh, R. (2008). The Effects of Performance-Pay in
Little Rock, Arkansas on Student Achievement. Paper presented at the National Center for
Performance Incentives conference, February 2008 and available at
www.performanceincentives.org.
21
Vigdor, J. (2008). Teacher Salary Bonuses in North Carolina. Paper presented at the National
Center for Performance Incentives conference, February 2008 and available at
www.performanceincentives.org.
22
Meyer, R. (1996) “Value-Added Indicators of School Performance,” in Hanushek, Eric A. and
Jorgenson, Dale W. (eds.), Improving the Performance of America’s Schools, Washington,
DC: National Academy Press, pp. 197-223; Braun, H.I. (2005) Using Student Progress to
Evaluate Teachers: A Primer on Value-Added Models. Princeton, NJ: Educational Testing
Service. Raudenbush, S.W. (2004). Schooling, Statistics, and Poverty: Can We Measure
School Improvement? Princeton, NJ: Educational Testing Service. Rivkin, S.G. (2007). ValueAdded Analysis and Education Policy. National Center for Analysis of Longitudinal Data in
Education Research Brief 1: Washington, DC: Urban Institute.
23
Fischer, K. (2008). Dallas schools teacher ratings sometimes off mark. The Dallas Morning
News, Saturday, February 23 web edition.
34
24
None of the approaches are free of measurement problems either. See Kane, T.J., and Staiger,
D.O. (2002). The promise and pitfalls of using imprecise school accountability measures.
Journal of Economic Perspectives, 16(4), 91-114; Ladd, H.F., and Walsh, R.P. (2002).
Implementing value-added measures of school effectiveness: getting the incentives right.
Economics of Education Review, 21, 1-17.
25
Chenoweth, K. (2007). It’s Being Done: Academic Success in Unexpected Places. Cambridge,
MA: Harvard Education Press; Odden, A., Archibald, S, Fermanich, M., and Gallagher, H.A.
(2002). A cost framework for professional development. Journal of Education Finance, 28(1),
51-74; Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. (2001, January 1). What
makes professional development effective? results from a national sample of teachers.
American Educational Research Journal, 38(4), 915-945..
26
This approach is analogous to productivity incentives based on beating a standard sometimes
used in the private sector. Examples of such incentives at the group level are described by B.
Graham-Moore and T, L, Ross in Gainsharing: Plans for Improving Performance. (1990).
Washington DC: Bureau of National Affairs; Chapter 2.
27
Hord, S.M. (2004) (ed.) Learning Together, Leading Together: Changing Schools Through
Professional Learning Communities. New York: Teachers College Press.
28
Varadarajan, P. and Futrell, C. (1984). Factors affecting perceptions of smallest meaningful
pay increases. Industrial Relations, 23(2), 278-285;. Worley, C.G., Bowen, D.E., and Lawler,
E.E. III. (1992). On the relationship between objective increases in pay and employees’
subjective reactions. Journal of Organizational Behavior, 13, 559-571; Rambo, W.W., and
Pinto, J.N. (1989). Employee perception of pay increases. Journal of Occupational
Psychology, 62, 135-145.
29
McAdams, J.C., and Hawk,E.J. (1994) Organizational Performance and Rewards: 663
Experiences in Making the Link. Scottsdale, AZ: American Compensation Association..
30
Kelley, C., Heneman, H. III, and Milanowski, A. (2002). School-based performance rewards:
research findings and future directions. Educational Administration Quarterly, 38:3, 372-401;
Milanowski, A.T. (2000). School-based performance award programs and teacher motivation.
Journal of Education Finance, 25:4, 517-544..
31
Sack, J.L. (2002, October 30). Calif. Drops Its Bonuses to Schools. Education Week.
32
Flannery, T.P., Hofrichter, D.A., and Platten, P.E. (1996). People, Performance, and Pay:
Dynamic Compensation for Changing Organizations. NY: Free Press, page 59.
33
Odden, A, and Wallace, M. (2008). How to Create World Class Teacher Compensation.
Freeload Press
34
Kaplan, S. and Henderson, R. (2005). Inertia and incentives: Bridging organizational
economics and organizational theory. Organization Science, 16(5), 509-521.
35
Lawler, E.E. III (2000). Rewarding Excellence: Pay Strategies for the New Economy. San
Francisco: Jossey-Bass, Chapters 10 and 12; Richter, A.S. (1998). Compensation management
and cultural change at IBM: Paying the people in black at Big Blue. Compensation and
Benefits Review, May/June.
36
Unfortunately, the research base that connects educator incentives to improved student
achievement is rather slim. A recent review by Podgursky and Springer (Podgursky, M.J., and
Springer, M.G., 2007. Teacher performance pay: A review. Journal of Policy Analysis and
Management, 26(4), 909-949.) concluded that in most of the relatively few (eight) studies they
35
located, incentives were associated with positive student achievement effects. On the other
hand, Lavy (Lavy, V., 2007. Using performance-based pay to improve the quality of teachers.
The Future of Children, 17(1), 87- 109.) finds that there is little causal evidence that one type
of incentive, school-based performance awards, have been effective in the U.S., and that
studies on individual teacher incentives in the U.S. have had mixed findings. Studies done in
other (primarily third world) countries showed more positive effects, but Lavy cautions that
what this says about potential for success in the U.S. is not known. Three recent studies
available from the National Center on Performance Incentives
(www.performanceincentives.org) found evidence of effects on student achievement. A study
of the Teacher Advancement Program by Matthew Springer and colleagues found positive
effects of that program on elementary school mathematics achievement, but not on middle or
high school achievement. Jacob Vigdor’s study of the North Carolina program found that test
scores, especially in math, have improved since that state introduced its incentive program, but
that achievement gaps have not narrowed. A study of an incentive program in Little Rock,
Arkansas by Marcus Winters and colleagues found that students in three elementary schools
whose teachers were eligible for the incentive made substantially larger test score gains in
reading, mathematics, and language arts, compared to other district schools. While each of
these studies reports some encouraging evidence, none can separate the effect of the
performance incentive from other program or school effects acting at the same time. While it
seems safe to say that paying for student achievement is not likely to lead to reduced student
achievement, there is not yet enough conclusive evidence of a positive incentive effect.
37
Heneman, H.G. III, and Milanowski, A.T. (2004) Alignment of human resource practices and
teacher performance competency. Peabody Journal of Education, 79:4, 108-125.
36