Measuring Teacher Impact on Student
Learning
PEAC Discussion Document| August 20, 2010
To ensure the most accurate and complete understanding of each
teacher’s performance and development needs, an evaluation system
should draw from a wide range of evidence. Potential components of
an evaluation system can include:
Component
Student Learning
Outcomes
Description
Student learning measured in multiple ways:
1. Student growth as indicated by an applied growth
model, where available; and
2. Student mastery of rigorous academic goals and
standards, based upon a variety of summative
assessments.
Instructional
Practice
The extent to which a teacher executes a set of core
competencies, through observations of teacher and student
actions and document reviews.
Professional
Responsibilities
The extent to which a teacher exhibits non-skill and
knowledge based actions and attitudes that reflect a clearly
defined set of professional responsibilities.
Contribution to
School
Community
The extent to which a teacher’s actions build a positive
school culture for students and families.
2
© The New Teacher Project, 2010
Given the current availability of data in the state, Illinois must rely on an
evaluator’s assessment of student learning until new standardized
measures are built. There are two main alternatives to consider.
Option A – Goal Attainment Process
o Teachers measure student growth by setting student academic goals, aligned
to meaningful standards.
o Evaluators confer with teachers to establish each goal’s degree of ambition
and select the appropriate assessments for measuring progress against the
goals.
o Teacher evaluation is based on students’ progress on the established goals, as
determined by an end-of-the-year evaluator review of the pre-determined
assessments and their results.
Option B – Review of Student Progress
o Student learning is measured through the evaluator’s assessment of the extent
to which a teacher’s students have mastered or made growth toward specific
standards, as indicated by performance on multiple assessments.
o Teachers track student performance over time, and evaluators review ongoing
student work and make judgments using a rubric.
3
© The New Teacher Project, 2010
In Option A, teachers work with evaluators to set ambitious goals
surrounding selected student assessments.
At the beginning of the year, teachers, in collaboration with their evaluator, decide
the appropriate number of goals to set for their particular classes, as well as to
ensure that the goals set are “acceptable” (aligned to standards, challenging but
attainable, and measureable). Some sample goals can include:
Some sample goals can include:
5th
Grade
9th
Grade
Art
Sample Goal 1: Increase class average of % mastery across all prioritized math standards from X% to
85%
Sample Goal 2: 80% of students in Subgroup A will show growth on prioritized science and writing
standards through achievement of at or above “meets expectations” on summative “Animal
Encyclopedia” project (>3 out of 4 on rubric)
Sample Goal 1: 90% of final portfolio presentations attain the level of “shows significant improvement”
or “shows outstanding improvement” on rubric at year-end review.
Sample Goal 2: 80% of “Artist Spotlight” booklet submissions show growth AND achieve a rating of at
least “Good” (>40/50) on rubric and are published in our end-of-year famous artist profiles magazine.
Sample Goal 1: 90% of students achieve improved score on 21st Century Skills Rubric AND score at or
11th
above “above expectations” for final research paper (>85/100)
Grade
Sample Goal 2: Median score of at least 80% on end of year summative examination
English
4
© The New Teacher Project, 2010
At the end of the year, teachers are evaluated based on goal
ambition and achievement using the rubric below to assign a
student learning rating for each goal.
Table 3 - Goal Attainment Rubric
5
Teacher has at least one high-quality source of evidence (i.e., one that is rigorous and
reliable) demonstrating that student progress:
Exceeded a very ambitious goal.
Teacher has at least one high-quality source of evidence (i.e., one that is rigorous and
reliable) demonstrating that student progress:
4
-
Met a very ambitious goal; OR
-
Exceeded an adequately ambitious goal.
Teacher has at least one high-quality source of evidence (i.e., one that is rigorous and
reliable) demonstrating that student progress:
3
-
Nearly met a very ambitious goal; OR
-
Met an adequately ambitious goal.
Teacher has at least one high-quality source of evidence (i.e., one that is rigorous and
reliable) demonstrating that students:
-
Nearly met an adequately ambitious goal; OR
-
Met less ambitious goal; OR
2
Exceeded a less ambitious goal.
Evidence of student progress toward goal is missing, incomplete, and/or unreliable;
AND/OR Students did not demonstrate significant academic progress (regardless of
1
ambitiousness of goal); AND/OR Student progress nearly met a less ambitious goal.
5
© The New Teacher Project, 2010
In Option B, evaluators review student progress on an ongoing basis
using multiple sources of information and assess progress relative to
standards.
Beginning of Year
o Teacher and evaluator
choose a set of priority
standards against which
student outcomes will be
measured
o Teacher articulates a
plan including
assessments for tracking
student progress toward
mastery of the chosen
standards
o Evaluator reviews the
assessment instrument(s)
using a rubric and offers
feedback until the
assessment(s) meets
expectations
During Year
o Teacher tracks ongoing
progress
o Evaluator monitors student
progress during classroom
observations and through
ongoing reviews of student
work.
End of Year
o Evaluator assesses the
quality of the assessment
instrument(s) used as
well as the student
learning outcomes to
assign a rating.
o Evaluator flags any tracking
and assessment issues, and
helps the teacher overcome
these issues
6
© The New Teacher Project, 2010
At the end of the year, evaluators use a rubric to evaluate teachers
and assign a rating.
Strongly
disagree
=1
Criteria
1) Were
planned
outcomes
ambitious and
feasible?
2) Were the
assessments
good?
3) Is the data
credible?
Disagree
=2
Agree
=3
Strongly
Agree
=4
The amount and type of content and skills the teacher prioritized and taught during the
year were:
• Aligned to approved grade/course content and end-of-year assessments.
• Differentiated by students’ starting points so that students starting near grade level
could grow at least one year and students starting below grade level could grow more
than one year (e.g., included remedial objectives)
• Allowed for a scope and sequence was appropriately paced and included time for reteaching
• Applied to appropriate and increasing levels of text difficulty (if applicable)
• Discrete, clear, and testable
• Aligned to or more ambitious than IEP goals and ELL goals
The assessments the teacher chose, designed, and used to summatively and conclusively
determine mastery were:
• Aligned to standards, the right level of rigor, and any external end-of-year assessments
• Assessed mastery of aligned content and skills accurately and reliably
• The teacher proctored and scored summative assessments such that results accurately
reflect students’ knowledge and skills
• Each student’s trend over the course of the year made sense and was realistic
• There is consistency between the teacher’s data and separately verifiable evidence of
student mastery (e.g., benchmark assessments)
The evaluator must agree with statements 1 through 3 in order to agree or strongly agree with criteria 4.
4) Have the
students
learned an
ambitious
amount of
material?
The vast majority of the teacher’s students who started near grade level learned at least a
year’s worth of material*
The vast majority of the teacher’s students who started far behind, including
mild/moderate SPED students, learned more than a year’s worth of material*
The teacher’s moderate/severe students learned an ambitious amount of material
including their functional, life, and vocational skills*
* Note: Missing data is counted as no mastery.
7
© The New Teacher Project, 2010
Regardless of the option chosen, the process must align with the overall
evaluation system and should promote discussions between the teacher
and evaluator on student performance and teacher professional growth.
Situational feedback conversations; student data reviews and data team meetings
Evaluation
Instructional Rounds (drop-ins) and/or full-period classroom observations
Development
Aug
Mid-year
check-in
conference,
informed by
all available
data
Beginning
of year
conference
to develop
evaluation
plan
Sep
Oct
Selfassessment
and develop
professional
focus areas
Nov
Dec
Jan
Feb
End of year
summative
evaluation
conference
Mar
Apr
Selfassessment
and revisit
professional
focus areas
May
Jun
Selfassessment
and discuss
next year’s
professional
focus areas
Targeted development activities (i.e., coaching, co-teaching, etc.)
Professional learning community meetings and feedback sessions
Note: Additional evaluation and development activities for non-tenured teachers, developing teachers, and/or teachers in need of improvement (such
as mentoring from coaches, progress check-ins with instructional managers, additional conferences, etc) are proposed in greater frequency but do not
appear in the timeline above.
8
© The New Teacher Project, 2010
PEAC can consider the following discussion questions when
selecting a process for measuring student learning outcomes.
Discussion Questions:
•What are the strengths of each of the options? What concerns or challenges does each
of these options raise?
•What are the implications for school-level capacity between the two options? For the
kinds of support that districts will need to implement the process with fidelity?
•How should this align to the overall process and outcomes of the evaluation system?
•How might these two options differ in practice for an elementary vs. secondary level?
•Is there another way to go?
9
© The New Teacher Project, 2010
Appendix
10
© The New Teacher Project, 2010
Categories of Student Learning Measures
Measure of Student Learning
Rating Tool Used
Current Status in Illinois
Category 1: Student growth on state
standardized tests that are developed
and/or scored by ISBE (e.g., ISAT)
Value-Add or Growth
Measures
•
Category 2: Student performance (as
measured by growth) on standardized
district-wide tests that are developed
and/or scored by either the district or
by an external party but not by ISBE
(e.g. NWEA, AP, PLAN, EXPLORE,
ACT, DIBELS, ACCESS, etc.)
Evaluator’s
Assessment of Student
Learning (EASL)
• Evaluators can take student
performance on district assessments into
account and can begin developing
processes around this.
Category 3: Other, more subjective
measures of student performance that
would likely be developed and/or
scored at the district- or school-level
(e.g., student performance on schoolor teacher-selected assessments)
Evaluator’s
Assessment of Student
Learning (EASL)
• Evaluators can take student
performance on school and classroom
assessments into account and can begin
developing processes around this.
11
Illinois currently assigns teacher
identification numbers and unique
student identifiers, but does not have a
data system capable of matching
individual teacher records to individual
student records needed for growth
measures (NCTQ 2009 Report Card)
© The New Teacher Project, 2010