Educator Evaluation
Workshop:
Gathering Evidence,
Conducting Observations &
Providing Feedback
MSSAA Summer Institute
July 26, 2012
Agenda
The Role of Evidence in the 5-Step Cycle
o Three types of evidence
o Roles & responsibilities
Artifacts of Practice
Observations & Feedback
Tips & Strategies
Resources
2
Intended Outcomes
At the end of this session, participants will be able to:
 Define “evidence of practice” and understand the role of
artifacts, observations, and feedback in the 5-Step Cycle
 Understand the value of frequent, unannounced
observations with targeted feedback
 Identify tools and processes for gathering and organizing
evidence that will make evidence collection and feedback
more doable in their schools.
3
Every educator is an active participant in the
evaluation process
Continuous
Learning
Every educator &
evaluator collects
evidence and
assesses progress
4
Collaboration and Continuous Learning are the focus
Massachusetts Department of Elementary and Secondary Education
“I was evaluated today…”
sound familiar?
Evaluation  Observations
5
Multiple sources of evidence inform
the Performance Rating
Three categories of evidence must be collected
for each educator:
1. Multiple measures of student learning, growth and
achievement
2. Judgments based on observations and artifacts of
professional practice
3. Additional evidence relevant to standards
̶ student/staff feedback (2013-2014)
6
Multiple sources of evidence inform
the performance rating
7
Massachusetts Department of Elementary and Secondary Education
What does this look like?
Products of Practice
related to Standards
Multiple Measures of
Student Learning
Artifacts
• Teacher-developed unit • Student work (quizzes,
homework,
assessments
presentations, etc.)
• Grade level meeting
• Portfolios
notes
• Performance
• Parent/teacher
assessments (including
communication log
arts, vocational, health
• PLC meeting notes
& wellness)
Observations
• Notes/feedback from
short, frequent
observations
(inside/outside
classrooms)
• Notes and feedback from
announced observations
Other Evidence
related to Standards
• Student and staff
feedback (2013/2014
school year)
• Interim assessments
• State or district
assessments
8
Implementation Responsibility
Educator Responsibilities:
o Documenting action steps completed.
o Collecting, organizing and submitting evidence to
demonstrate progress toward professional practice
and student learning goals.
Evaluator Responsibilities:
o Observing practice on a regular basis and providing
targeted feedback on performance
o Making resources and supports available.
o Identification of common artifacts/evidence.
9
Products of Practice:
Artifacts
10
It starts with the Educator Plan…
Student Learning Goal: In order to ensure mathematical literacy in each of the three content areas for 8th
grade geometry (8.G), I will incorporate at least one essay question into each unit assessment that requires
elaboration of mathematical reasoning so that 80% or more of my ELL students demonstrate proficiency on
essay questions on the end of the year 8th grade geometry assessment.
Student Learning Goal(s) Planned Activity
Action
1. By October 1, I will assess ELL student
comprehension and knowledge with
formative assessments
2. By October 15th, I will share this data
with my department team and
instructional coach and solicit feedback
on instructional strategies related to
teaching mathematical literacy to ELL
students.
3. By October 30th, I will develop writing
objectives for each unit and integrate
them into unit assessments.
4. From November through May, after each
unit, I will disaggregate assessment data
for ELL students, focusing on
mathematical literacy. I will track their
progress and adjust instruction as
necessary.
Supports/Resources
from School/District
Formative geometry
assessment
Monthly department team
meetings
Monthly one-on-one data
analysis with instructional
coach and ELL specialist
Unit assessments
Timeline/Benchmark or
Frequency
1. Oct. 1: review formative assessment results
for my ELL students
Oct. 15: share formative assessment results
with department team and instructional
coach and identify at least three
instructional strategies related to building
mathematical literacy with ELL students.
• Evidence: meeting notes, 3 strategies
2. October 30th: developing writing objectives
for each unit
• Evidence: written objectives, essay
questions
3. November—June: Administer unit
assessments in three content areas and
analyze student performance on essay
questions
• Evidence: student data from essay
questions in at least three unit
assessments
11
Importance of Strategically
Collecting Artifacts
Artifacts should be a sample that demonstrates
educator performance and impact
o Aligned with educator goals, the Model System
Teacher Rubric or school goals
Number of artifacts to collect varies by educator
Artifacts can provide evidence of more than one
Standard-Indicator
o An annotated summary of Grade 5 unit assessment results can include 12
evidence of practice related to I.C. (Analysis), IV.A.1 (Reflective
Practice), and IV.C (Collaboration).
Lessons from Early Adopters:
Collecting Evidence
Quality not quantity
Guidelines and exemplars will help
Prioritize based on focus areas
13
Massachusetts Department of Elementary and Secondary Education
Products of Practice:
Observations & Feedback
14
Observations
The regulations require a minimum of one
unannounced observation.
The Model System recommends short,
frequent unannounced observations for all
educators, as well as at least one announced
observation for non-PTS educators and
struggling educators.
15
Why short, frequent
observations?
More opportunities to see patterns of practice
Flexibility in scheduling
Promotes ongoing conversation around
teaching and learning
Facilitates observations beyond the classroom
 Is 5-15 minutes enough?
16
Observation and Feedback
School-level Administrator Rubric (I-D-2):
Typically makes at least two unannounced visits to
classrooms each day and provides targeted,
constructive feedback to all educators. Acknowledges
effective practice and provides redirection and support
for those whose practice is less than proficient.
Superintendent Rubric (I-D-2): Typically makes at
least three unannounced visits to each school to
observe principal practice every year and provides
targeted, constructive feedback to all administrators.
Acknowledges effective practice and provides
redirection and support for those whose practice is
less than proficient.
Massachusetts Department of Elementary and Secondary Education
17
Principles of high quality
observations
Frequent
Focused
Inside/Outside the Classroom
Useful & Timely Feedback
The rubric is not an evaluation tool, but a
guide to help identify trends and patterns
of practice over time.
18
Feedback
“[O]bservers must learn how to capture
classroom events in literal notes, and to talk
productively with the teacher about it
afterward in a way that is evidence-based and
productively points toward actionable
improvement.” –John Saphier
19
Principles of Good Feedback
Verbal as well as written
Focused on a few key areas
Based on evidence
Tied to Standards of effective practice
Offers reinforcement for areas of effective
practice
Facilitates self-reflection on areas of practice
that need refinement and guides the teacher
in thinking beyond the lesson observed
Massachusetts Department of Elementary and Secondary Education
20
Tips & Strategies
21
1. PLAN
2. COMMUNICATE
EXPECTATIONS
3. ORGANIZE
22
1. PLAN
The more concrete the Educator Plan, the
easier it is to identify and collect artifacts
Identify common artifacts all or most
educators will be expected to collect (unit
assessments, parent-teacher logs, etc.)
Share examples of high-quality, valuable
evidence during faculty or team meetings
o Demonstrate example artifacts that provide
evidence of more than one Standard-Indicator
23
2. Communicate Expectations
OR
24
2. Communicate Expectations
Artifacts should be a sample that
demonstrates educator performance and
impact
Submitted evidence should be tied to educator
goals, Standards or Indicators, or school goals
Provide everyone with a clear idea of how and
when to share products of practice
o Email? Paper? Online cloud?
25
3. ORGANIZE
Calendar observations
Adopt a process for organizing artifacts and
observation notes by Standard/Indicator
and/or goals
o Paper-based, email-driven, or online repository
o Sample tools for evidence collection and
organization
26
Sample Tools for Evidence
Collection and Organization
Included in your packet
Sample Evidence Tool
Completed by . . .
Artifact Cover Page
Educator or Evaluator
(the person who
identifies the artifact)
Observation Evidence Collection Tool Observer/evaluator
27
28
Next Steps – Suggestions for Principals
 Read “Strategies and Suggestions for
Observations” (p. 39 of the School-Level
Planning & Implementation Guide)
 Identify options for collecting and organizing
evidence at your school and establish a protocol
for all educators
 Work with your administrative team to set a
calendar for observations and evaluations based
on the distribution of educators by plan type at
your school
Massachusetts Department of Elementary and Secondary Education
29
Resources
Massachusetts Model System for
Educator Evaluation
30
Massachusetts Department of Elementary and Secondary Education
School-Level Planning &
Implementation Guide
Content Overview
The Massachusetts Model System for Educator
Evaluation
Step 1: Self-Assessment
Step 2: Goal Setting and Plan Development
Step 3: Implementation of the Plan
Step 4: Formative Assessment and Evaluation
Step 5: Summative Evaluation
Appendices: Forms for Educator Evaluation, Setting
SMART Goals
Massachusetts Department of Elementary and Secondary Education
31
ESE Evaluation Resources
What’s coming?
− Summer 2012
 Guidance on District-Determined Measures
 Training Modules with facilitator guides, PowerPoint
presentations, and participant handouts
 List of approved vendors
 Updated website with new Resources section
 Newsletter
32
Massachusetts Department of Elementary and Secondary Education
ESE Evaluation Resources
What’s coming?
− Fall/Winter 2012
 Solicit and review feedback on Model System; update
 Research & develop student and staff feedback instruments
 Collect and disseminate best practices
 Collect and vet assessments to build a repository of district
measures
 Internal collaboration to support cross-initiative alignment
 EX: Support for use of rubric for teachers of ELLs aligned 33
to RETELL initiative
Massachusetts Department of Elementary and Secondary Education
For More Information and Resources:
Visit the ESE educator evaluation website:
www.doe.mass.edu/edeval
Contact ESE with questions and suggestions:
EducatorEvaluation@doe.mass.edu
Presenter:
Claire Abbott – cabbott@doe.mass.edu
Massachusetts Department of Elementary and Secondary Education
34