Professional Learning Team
Conference
June 2014
Each participant will leave the session
knowledgeable about inter-rater reliability
 What it is
 Why it’s important, and
 How to use evidence to consistently
and accurately rate teachers’
performance
 Demonstrate familiarity with district
rubric
 Describe proficient practices based
on district rubric descriptors
 Provide feedback to educators based
on evidence collection
 Calibration of evaluators
PARTICIPANTS WILL:
Establish a common vocabulary around
inter-rater reliability
Be able to explain levels of performance
with look-fors
Gain a deeper understanding of proficient
level of performance aligned to standards
CHINESE PROVERB:
What I hear, I forget
 5% retention after 24 hours
What I see, I remember
 30% retention rate if use of audio-visual
and demonstration
What I do, I understand
 80% retention rate with discussion and
practice by engagement
NORMS
Share experiences to enrich others
Ask questions
Learn by doing
Set aside any preconceived notions
about evaluating educators
Apply your own work
Participants will become familiar with look fors
of effective practice and be able to use that
knowledge to assess performance in three
focused areas:
 Student Engagement
 Questions, Prompts & Discussion
 Assessment
NATIONAL FOCUS ON TEACHER
EFFECTIVENESS AND EVALUATION
EFFECTIVENESS:
EVALUATION:
Teacher effectiveness is the single most
important school-related factor affecting
student learning.
However, most schools fail to evaluate
teachers in a meaningful way, with 94% of
teachers receiving the highest ratings.
Teacher evaluations in five urban school districts, based on data taken from a
report by The New Teacher Project:
http://widgeteffect.org/downloads/TheWidgetEffect.pdf.
Teacher evaluations in five urban school districts, based on data taken from
a report by The New Teacher Project:
http://widgeteffect.org/downloads/TheWidgetEffect.pdf
a systemic approach to
calibrate observations for
consistent and fair professional
practice ratings
At your table
define and describe
Student Engagement
Capture your collective thoughts on
chart paper
Educational author and former teacher, Michael
Schmoker shares in his book, Results Now,
a study that found of 1,500 classrooms visited,
85 percent of them had engaged less than
50 percent of the students
In other words, only 15 percent of the classrooms
had more than half of the class at least paying
attention to the lesson
You will see students...









Paying attention (alert, tracking with their eyes)
Taking notes (particularly Cornell)
Listening (as opposed to chatting, or sleeping)
Asking questions (content related, or in a game)
Responding to questions (whole group, small group, four corners,
Socratic Seminar)
Following requests (participating, Total Physical Reponses (TPR),
storytelling, Simon Says)
Reacting (laughing, crying, shouting, etc.)
Reading critically (with pen in hand
Interacting with other students
Michael Schmoker
 Activities aligned with the goals of the lesson
 Student enthusiasm, interest, thinking, problem-solving…
 Learning tasks that require high-level student thinking and are
aligned with lesson objectives
 Students are highly motivated to work on all tasks and are
persistent even when the tasks are challenging
 Students actively "working," rather than watching while the
teacher "works“
 Suitable pacing of the lesson: neither dragging nor rushed, with
time for closure and student reflection
When determining the level of performance for
Engaging Students in Learning, consider the
quality of activities, assignments, and resources
the instructor uses to pull students into the lesson
It is also important to understand how the lesson
uses student grouping and pacing when
determining the level of performance for
this component
At your table
define and describe
Assessment
Capture your collective thoughts on
chart paper
is to support
student learning and
to communicate that
learning to others
In order to support student learning
classroom assessment needs to:
 involve students deeply in the assessment
process
 provide specific, descriptive feedback during
the learning, and
 include evaluative feedback as required to
communicate and report progress over time
To ensure success of all:
1)
2)
3)
students need to know what they
already know
know what needs to be learned and
know what success looks like
Students also need to learn how to guide their
own learning through being involved in setting and
using criteria, giving themselves feedback for
learning (self-assessment), setting goals, collecting
evidence and communicating that evidence of
learning to others
 The teacher pays close attention to evidence of
student understanding
 The teacher poses questions specifically created to
elicit evidence of student understanding
 The teacher circulates to monitor student learning
and to offer feedback
 Students assesses their own work against
established criteria
 The teacher adjusts instruction in response to
evidence of student understanding (or lack of it)
BREAK
Work with your District partners to
identify the elements in
your rubric that focus on
 Student Engagement
 Question, Prompts & Discussion
 Assessment
While observing the video,
collect evidence that you will
later use to rate the teacher’s
performance using your
district rubric
You will use your district’s rubric
to rate the teacher’s performance
in the following areas:
Student Engagement
Question, Prompts & Discussion
Assessment
Video
https://www.teachingchannel.org/videos/surface-area-lesson#
In just a moment you will be
asked to review the data
you’ve collected,
cluster and code it for easy
interpretation, and rate the
teacher’s performance using
your district’s rubric
DATA
Evidence
DATA
Interpretation
DATA
Judgment
Feedback that
guides performance
Student: Um, how he has the equation.
CM: Exactly, how he has the equation so every calculation starts with his equation. I like
that too, good. Kay-Wen, keep it moving.
Student: He does it step by step.
Select student at
randon to participate
CM: What do you mean?
Prompts
Student: Well, for the circumference, see how he switched the d –
4:11
CM: Into a…?
Student: Where is it?
CM: Here. Into the diameter of 60, so that’s one step, that step’s called substitution.
Very good, I like that, Kay-Wen. What else do we like, Corrine?
Student: I like how it’s organized and that there’s no mistakes and he’s not skipping
steps.
CM: Absolutely, did any of you guys have to do a double take, like look at it like ugh,
what does that say? No, it’s very neat and organized, it’s clearly stated, very good.
Taylor, what else?
Feedback to reinforce behavior
rather than instruction
Student: Um, his answers are circled so that we know where to look for the actual
answer.
Fade
Teacher
circulates,
checking
understanding
and asking
clarifying
question to
deepen
understanding
Assessing
learning
10:44
Student: We’re using a basic strategy, but I think it might work.
CM: Let me, let me see. First of all, start your strategy off, what was your strategy?
Student: Our strategy was um, height by circumference would equal the surface area of
the cylinder and the reason is because –
CM: The whole cylinder? Which part?
Student: This part.
CM: Which part?
Student: The part around.
CM: Okay, is that it?
With an elbow partner, use the
verbatim transcript of the teacher’s
observation to identify and
organize evidence to accurately
and fairly rate the teacher’s
performance in the areas of
 Student engagement
 Questions, Prompts and Discussion
 Assessment
 Look at your evidence for each
component. Where does the
preponderance of evidence align
with levels of performance?
 Match your data with the rubric
performance level description
 Assign a performance level
Unsatisfactory
Basic
Proficient
Distinguish
1.
2.
3.
4.
Snickers
Milky Way
Kit Kat
Trix
64%
27%
9%
0%
STUDENT ENGAGEMENT
64%
Unsatisfactory
Basic
Proficient
Distinguish
27%
9%
ish
in
gu
Di
st
Pr
of
ic i
en
t
sic
Ba
isf
ac
to
r
y
0%
Un
sa
t
1.
2.
3.
4.
DISCUSS
88%
Unsatisfactory
Basic
Proficient
Distinguish
13%
ish
in
gu
Di
st
Pr
of
ic i
en
t
sic
0%
Ba
isf
ac
to
r
y
0%
Un
sa
t
1.
2.
3.
4.
DISCUSS
82%
Unsatisfactory
Basic
Proficient
Distinguish
18%
ish
in
gu
Di
st
Pr
of
ic i
en
t
sic
0%
Ba
isf
ac
to
r
y
0%
Un
sa
t
1.
2.
3.
4.
DISCUSS
 One thing you heard
today that you want to
remember
Two things you heard
that you want to act
on before the next
PLT Conference
Thank you
for your active
participation and
cooperation
