Leveraging Teacher Evaluation for
Improving Teaching and Learning
Laura Goe, Ph.D.
Research Scientist, ETS
Sr. Research and Technical Assistance Expert, Center for Great
Teachers and Leaders
Georgia Title IIa Conference  March 2013  Savannah, Georgia
Laura Goe, Ph.D.
 Former teacher in rural & urban schools
• Special education (7th & 8th grade, Tunica, MS)
• Language arts (7th grade, Memphis, TN)
 Graduate of UC Berkeley’s Policy, Organizations,
Measurement & Evaluation doctoral program
 Research Scientist in the Understanding Teaching Quality
Research Group at ETS
 Senior Research & Technical Assistance Expert for the
federally-funded Center for Great Teachers and Leaders
2
The goal of teacher evaluation
The ultimate goal of all teacher
evaluation should be…
TO IMPROVE
TEACHING AND
LEARNING
3
To be discussed…
1. Using teacher evaluation results to inform
professional growth opportunities
2. Characteristics of high-quality professional
development
3. Assessing impact of professional growth
4. Recruitment, retention, and distribution of
effective teachers
4
Accountability vs. professional growth
 Effective evaluation for accountability
relies on:
• Clearly defined and communicated standards for
performance
• Quality tools for measuring and differentiating
performance
• Quality training on standards and tools
• Evidence to make decisions
5
Accountability vs. Professional Growth
 Effective evaluation for professional
growth relies on:
• Clearly defined and communicated standards for
performance
• Quality tools for measuring and differentiating
performance
• Quality training on standards and tools
• Evidence to make decisions
6
Validity of measures key to PD aligned with
evaluation results
 Consider whether human resources and capacity are sufficient to
ensure fidelity of implementation
• Poor implementation threatens validity of results
 Establish a plan to evaluate measures to determine if they can
effectively differentiate among teacher performance
• Need to identify potential “widget effects” in measures
• If measure is not differentiating among teachers, may be faulty training or
poor implementation, not the measure itself
 Examine correlations among results from measures
 Evaluate processes and data each year and make needed
adjustments
7
Interpreting results for alignment with teacher
professional learning options
 Different approach; not looking at “absolute gains”
 Requires ability to determine and/or link student outcomes
to what likely happened instructionally
 Requires ability to “diagnose” instruction and
recommend/and or provide appropriate professional
growth opportunities
• Individual coaching/feedback on instruction
• Observing “master teachers”
• Group professional development (when several teachers have
similar needs)
8
Five components in an aligned teacher
evaluation-professional growth system
High quality
standards for
instruction
Multiple standardsbased measures of
teacher
effectiveness
Quality training on
standards and
tools
Trained individuals
to interpret results
and make
professional
development
recommendations
High-quality
professional
growth
opportunities for
individuals and
groups of teachers
with similar growth
plans
9
High quality professional growth
opportunities
 Characteristics of High Quality Professional
Development (Archibald, et al, 2011)
• Alignment with school goals, state and district standards and
assessments, and other professional learning activities including
formative teacher evaluation
• Focus on core content and modeling of teaching strategies for the
content
• Inclusion of opportunities for active learning of new teaching
strategies
• Provision of opportunities for collaboration among teachers
• Inclusion of embedded follow-up and continuous feedback
10
Teaching standards
 A set of practices teachers should aspire to
 A teaching tool in teacher preparation programs
 A guiding document with which to align:
• Measurement tools and processes for teacher evaluation, such as
classroom observations, surveys, portfolios/evidence binders,
student outcomes, etc.
• Teacher professional growth opportunities, based on evaluation of
performance on standards
 A tool for coaching and mentoring teachers:
• Teachers analyze and reflect on their strengths and challenges and
discuss with consulting teachers
11
Quality training for teachers and leaders on
teaching standards and evaluation tools
 Increases inter-rater reliability
 Increases the validity of system
• Without valid results, professional development based on
results will be unlikely to improve practice
 Ensures mutual understanding
 Is a form of professional development for those
trained
 Training, certification, and calibration on
instruments may be more important than who the
evaluator is
12
Memphis professional development system:
An aligned system
 Teaching and Learning Academy began April ‘96
• Nationally commended program intended to
• “…provide a collegial place for teachers, teacher leaders
and administrators to meet, study, and discuss application
and implementation of learning…to impact student growth
and development”
• Practitioners propose and develop courses
• Responsive to school/district evaluation results
• Offerings must be aligned with NSDC standards
• ~300+ On-line and in-person courses, many topics
13
Professional Growth Plan (From GTEP
manual)
 “The plan includes specific objectives for
improvement, activities and a time line for meeting
these objectives, criteria for measurement of
progress toward meeting the objectives, a record
of participation in recommended activities, and a
record of performance on specified criteria.”
http://www.pelhamcity.k12.ga.us/docs/GTEP%20manual.pdf
14
Georgia School Keys Professional
Development
 “The increased practice of more professional learning
in individual schools connected to student data and
improvement plans has increased the opportunities for
supporting the improved performance of students and
staff. This practice has lead to more results-driven
professional learning, which is standards-based, jobembedded, and collaborative.”
http://www.doe.k12.ga.us/SchoolImprovement/Documents/FY%202011%20Professional%2
0Learning%20Annual%20Report%204-3-12.pdf
15
Professional development planning
 Tap in-district expertise to provide professional
development
• Could be teachers, administrators, instructional leaders, etc.
(Memphis is a good example)
 Find outside supports to fill gaps in internal capacity
• For example, partner with a local college
 Create mechanism for connecting teachers to PD based
on evaluation results (such as Memphis model)
 Prepare for structural changes in the school week/calendar
16
Effective teachers
 Strong, effective teachers and leaders are the key to
improving student outcomes
 Effective teachers may be in limited supply
 Ways to get effective teachers and leaders:
• Hire only highly effective teachers
• Remove less effective teachers and leaders and replace
them with more effective ones
• Provide guidance and support to help less effective
teachers become highly effective teachers
17
Teacher turnover
Each year, close to 200,000 teachers
leave the profession, with another
200,000 transferring schools.
This teacher turnover costs our public
school system about $5 billion annually
(Alliance for Excellent Education,
2005).
18
A single “bad” year vs. a trend
 Any teacher can have a year where they struggle
 The key is to identify whether this is a “difficult”
year or a “pattern” of poor performance
• Response to a “difficult” year should be mostly supportive
with targeted assistance
• Response to a pattern should be more intensive with
diagnosis of problem areas, improvement plan, time limit
for improvement, etc.
 Teachers want to be successful!
19
Interpreting results and making professional
development recommendations
 Analyzes evaluation data to connect with PD
opportunities at the teacher, school and
district level
• Ideally provides guidance at the time of feedback
 Can be a trained administrator, evaluator,
coach, mentor, etc. (for individual teachers)
 Could be a committee (for determining grade,
subject, school or district-wide needs)
20
Criteria for Measurement of Progress (from
GTEP manual)
 “Specific criteria for measurement of progress on the PDP
should be described. Measurement of progress involves
keeping a record of successful completion of activities and
determining whether or not the teacher's performance has
improved in the targeted areas. ‘Improvement in targeted areas’
means previously marked areas have been judged to show
improvement. The evaluator makes this judgment. For teachers
who have received a required plan for specific needs
development, progress toward completion of the PDP shall be
evaluated during the next evaluation cycle if the individual is
employed in the same school system. http://www.pelhamcity.k12.ga.us/docs/GTEP%20manual.pdf
21
Are those who leave more effective?
 Research suggests that those who leave
(whether the school, the district, or the
profession) tend to be less effective than those
who stay in schools
 The exception is first year teachers who
leave—they tend to be more effective than
those who stay (Hanushek and Rivkin, 2010)
22
Match between teachers & schools
 Investigated the contribution of “match between
teachers and schools” to student achievement
(Jackson, 2010)
 Showed that teacher effectiveness is higher after
a move to a different school
 25% of what is typically considered to be a
teacher effect may actually be a teacher-school
(match) effect
23
“Fit” between person and organization
 “…characteristics that predict individual effectiveness
may not predict organization effectiveness, even if the
assumptions about individual predictors are valid”
(Harris & Rutledge, 2010, pg 923)
 Person-Organization fit matters more when the
organization has a “distinctive culture, the career
ladder is lengthy, the interaction among organization
members is great, and work flexibility is high.” (ibid, pg
924)
24
Impact of incentives on teacher turnover
 Evaluators found that the probability of teacher turnover fell
as the magnitude of the TEEG bonus award increased,
while the probability of teacher turnover increased sharply
among teachers receiving no bonus award, or a relatively
small award (Springer et al., 2009)
• “…a $3,000 award reduced turnover for beginning teachers by
roughly 23 percentage points”
• “…awards of $3,000 reduced turnover rates to less than a
quarter of what the turnover rate was prior to implementation
of a TEEG performance pay plan.”
25
Teacher peer effects
 Using longitudinal elementary school teacher and student data,
we document that students have larger test score gains when
their teachers experience improvements in the observable
characteristics of their colleagues. Using within-school and
within-teacher variation, we further show that a teacher’s
students have larger achievement gains in math and reading
when she has more effective colleagues (based on estimated
value-added from an out-of-sample pre-period). Spillovers are
strongest for less-experienced teachers and persist over time,
and historical peer quality explains away about twenty percent
of the own-teacher effect, results that suggest peer learning.
(Jackson & Bruegmann, 2009)
26
Cohort model to improve recruitment and
retention of effective teachers
• Consider a cohort model of teacher recruitment
• Retain cohorts through ensuring collegial interactions such as
professional learning communities are a priority
• May include a pay incentive for retention at the cohort level (i.e.,
percentage of cohort that stays in school)
• Partner with institutions of higher education to better
prepare teachers for teaching at-risk students
• Create a feedback loop with IHEs
• Field experiences in high-needs schools should be extensive and
high quality
27
Documenting distribution patterns of
effective & less effective teachers
 Many states have the capability to follow teachers’
movements but don’t track them
 Develop a database that allows you to track teachers from
school to school as well as exiting and reentering the
profession
28
Documenting school & district changes in
teacher effectiveness
 With teacher effectiveness data from evaluations, districts
may be better able to document the current distribution of
effective teachers by the percentage of poor and minority
students in the school
 The challenge is to document change in the distribution of
effectiveness over time and relate changes to specific
policies or incentives, such as teacher professional growth
 To do this, you need complete and accurate data about
teachers’ participation in professional growth activities,
mentoring, coaching, participation in professional learning
communities, content-specific coursework, etc.
29
Percentage of Classes Taught by
Effective Teachers by Year
Percentage
Tracking change: An Illustration
100
80
60
40
20
0
School School School School School School School
101
102
103
104
201
202
203
District 1
District 2
Schools Within Districts
30
Questions to ask about your district’s
professional development plan
 Within your current or proposed teacher evaluation system,
do you have a mechanism to ensure that evaluation results
are aligned with and being used to guide professional
development plans?
 Are teachers given the opportunity to reflect on and
discuss evaluation results and develop goals for further
growth with supervisors, coaches, mentors, colleagues?
 Can teachers collaborate with instructional managers to
develop a plan for improvement and/or professional
growth?
31
Questions to ask about your district’s
professional development plan (cont’d.)
 What additional supports and requirements are in place for
teachers when trends in evaluation scores indicate that
they are struggling?
 What opportunities are available for teachers with high
evaluation scores to pursue additional knowledge, growth,
and leadership?
 What follow-up mechanisms are in place to ensure that
teachers are taking maximum advantage of professional
growth opportunities?
 How will change in teacher practice be documented?
32
Next steps
 Ensure that evaluation systems are valid for identifying
effective and less effective teachers
 Focus on improving effectiveness of teachers you already
have with high-quality, sustained professional development
 Develop strategies for supporting and retaining effective
and potentially effective teachers
 (YOUR GREAT IDEAS HERE)
33
Resources/further reading
 Coggshall, J. G., Rasmussen, C., Colton, A., Milton, J., & Jacques, C. (2012).
Generating teaching effectiveness: The role of job-embedded professional learning in
teacher evaluation. Washington, DC: National Comprehensive Center for Teacher
Quality. http://www.tqsource.org/publications/GeneratingTeachingEffectiveness.pdf
 Goe, L., Biggers, K., & Croft, A. (2012). Linking teacher evaluation to professional
development: Focusing on improving teaching and learning. Washington, DC: National
Comprehensive Center for Teacher Quality,
http://www.tqsource.org/publications/LinkingTeacherEval.pdf
 Imazeki, J., & Goe, L. (2009). The distribution of highly-qualified, experienced teachers:
Challenges and opportunities. Washington, DC: National Comprehensive Center for
Teacher Quality. http://www.tqsource.org/publications/August2009Brief.pdf
 Memphis Teacher Development and Support Individualized Growth Resource Book
http://tntp.org/assets/tools/MCS%20TEMAligned%20PD%20Resource%20Guide_TSLT%203.12.pdf
34
References
Alliance for Excellent Education. (2005). Teacher attrition: A costly loss to the nation and to
the states. Washington, DC: Author.
http://www.all4ed.org/files/archive/publications/TeacherAttrition.pdf
Archibald, S., Coggshall, J., Croft, A., & Goe, L. (2011). High-quality professional
development for all teachers: Effectively allocating resources. Washington, DC:
National Comprehensive Center for Teacher Quality.
http://www.tqsource.org/publications/HighQualityProfessionalDevelopment.pdf
Harris, D., & Rutledge, S. A. (2010). Models and predictors of teacher effectiveness: A
comparison of research about teaching and other occupations. Teachers College
Record, 112(3), 914-960.
http://eps.education.wisc.edu/Faculty%20papers/Harris/Manuscripts/IES%20Harris%2
0Rutledge%20Models%20and%20Predictors%20TCR%20Nonblind.doc
Hanushek, E. A., & Rivkin, S. G. (2010). Constrained job matching: Does teacher job search
harm disadvantaged urban schools? Cambridge, MA: National Bureau of Economic
Research. http://www.urban.org/uploadedpdf/1001395-constrained-job-matching.pdf
35
References (cont’d.)
Jackson, C. K. (2010). Match quality, worker productivity, and worker mobility: Direct
evidence from teachers. Cambridge, MA: National Bureau of Economic Research.
http://www.ssc.wisc.edu/~scholz/Seminar/Jackson.pdf
Jackson, C. K., & Bruegmann, E. (2009). Teaching students and teaching each other: The
importance of peer learning for teachers. Cambridge, MA: National Bureau of
Economic Research.
http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1079&context=working
papers
Springer, M., Lewis, J. L., Podgursky, M. J., Ehlert, M. W., Taylor, L. L., Lopez, O. S., &
Peng, A. X. (2009). Governor’s Educator Excellence Grant (GEEG) Program: Year
three evaluation report. Nashville, TN: National Center on Performance Incentives.
http://ritter.tea.state.tx.us/opge/progeval/TeacherIncentive/GEEG_Y3_0809.pdf
36
Questions?
37
Laura Goe, Ph.D.
609-619-1648
lgoe@ets.org
www.lauragoe.com
https://twitter.com/GoeLaura
Center on Great Teachers and Leaders at AIR
1000 Thomas Jefferson Street NW
Washington, DC 20007-3835
General Information: 877-322-8700
www.gtlcenter.org
38