Educator Evaluation Pilot
July 19-21, 2011
Stonewall Resort, Roanoke, West Virginia
Overview
• The “Big Picture”
–Where are we headed?
–Where have we been?
–How will we get there?
• Two Day Agenda
• Next Steps
VISION:
West Virginia will have a
comprehensive and equitable
evaluation system that clearly
articulates, measures, rewards,
and develops educator
effectiveness
Where Have We Been?
Historical Perspective
2009
Standards
Adopted
2011
Evaluation
Task Force
Worked
2012
Revised
System
Piloted
Importance of Leadership
• July 20-21 – Stonewall Resort
• ALL Pilot schools leadership team
members
–
–
–
–
Principals and all assistant principals
Title I Directors and/or Assistant Superintendents
Teacher Leaders
Central office representative for non SIG pilot
schools
Clear Expectations
• Embrace the Opportunities:
•
•
•
•
Be “early adopter”
Engage in professional development
Work as part of a collaborative team
Provide input to policy makers
• Participate in the research study
• Communicate, partner and problem-solve
Day 1 Agenda
•
•
•
•
•
•
Expectations for Training
History Evaluation Creation
Conceptual Framework Overview
Self-Assessment
Observation
Student “Growth”
Day 2 Agenda
•
•
•
•
•
SMART Goals and Collaboration
Student Learning Goals
Professional Conduct
Research
Performance Assessment
What are the Next Steps?
Teacher Evaluation
Training Dates
• August 6-12 – Regional Training
• Participants: all teachers, administrators
and transformation specialists
• Fall and Spring Training
• On site visits
• Ongoing Technical Assistance
Historical Perspective
2013
2009
Standards
Adopted
2011
Evaluation
Task Force
Worked
2012
Revised
System
Piloted
Begin
Statewide
Scale Up
Thank You!
Expectations for
Professional Development
Expectations for
Professional Development
• Learning Target
– Understand how the new evaluation
system works to be able to implement
the processes yourself and to be able to
assist others in school
• Strategies to accomplish the goal
– Hands-on activities
– Multiple opportunities to master content
– Collaboration with tablemates
Expectations for
Professional Development
• Expectations for us
– Well prepared
– Clarity
– Consideration of the audience
• Expectations for you
– Active participation
– Regular feedback
– Focused attention
Teacher Evaluation Creation
Task Force Members’
Perspective
Evaluation Perceptions
& Experiences
Table Tasks
• Describe most recent evaluation
experience
– What was the purpose? What feedback
was provided? What was your role in
the process?
• What are the challenges of teacher
evaluation (not including time)?
• What will you need to learn to make
this PD beneficial?
A New Evaluation System:
The Rationale
A New Evaluation System:
The Rationale
• Learning Target: Know
– the three drivers behind a revised
evaluation system for principals and
teachers
Converging Forces
Conceptual Framework
Overview
Conceptual Framework
Overview
• Learning Targets: Understand
– the nexus between the WVPTS and the
Critical Standard Elements;
– how levels of teacher performance are
defined with rubrics;
– the tiered evaluation system with
progressions based on experience
Levels of Performance
Distinguished
Accomplished
Emerging
Unsatisfactory
Table Tasks
• Share one activity with distinguished
performance, one activity with
unsatisfactory performance
Levels of Performance
Distinguished
Accomplished
Emerging
Unsatisfactory
Distinguished
performance
describes
professional
teaching that
engages students
to be highly
responsible for
their own learning.
Performing at this
level involves
contributing to the
professional
learning of others
through teacher
leadership.
Accomplished
performance
describes
professional
teaching that
exhibits mastery of
the work of
teaching while
improving practice
and serving the
professional
community.
Emerging
performance
represents
teaching that
demonstrates
knowledge and
skills to implement
essential elements
albeit not always
successfully at
times.
Unsatisfactory
performance
describes teaching
that does not
convey sufficient
understanding of
concepts or the
successful
implementation of
essential elements.
Self Assessment –
Advance Progression
Self Assessment – Advance
Progression
• Learning Targets:
– Identify individual performance within
the established rubrics
Table Tasks
• Share an insight from doing this
activity about individual
performance
• Share reaction to critical standard
elements
• What implications does this have for
individual professional development
Table Tasks
• Discuss perception of how
comfortable teachers with 6+ years
of experience will be completing self
assessment
• Discuss how this process could be
used in relation to the PD plan for of
the school
• Discuss how this process could be
used in collaborative teams.
Observation
Observation
• Learning Targets:
– Understand that observation is
formative and offers a window into
instructional performance;
– Understand that observation, though
limited, is informative about certain key
aspects of instruction
Some specifics about
observation
• Not the evaluation
• Initial Progress-4, Intermediate-2,
Advanced if requested
• Class period or minimum of 30 mins
• One piece of a two part conversation
• Will be supported by evidence and
conversation
• Elements that contribute to the
research
Table Tasks
• What does observation tell you
about the critical standard elements?
• What elements do you still need to
know about to make a fair and
accurate assessment about
performance?
Evidence
Evidence
• Learning Targets:
– Understand that the educator plays an
active role in demonstrating
performance level by providing
evidence;
– Identify and classify evidence
Table Tasks
• Share the kinds of evidence that were
generated.
• Were any common among the table?
• Discuss whether you agree that
evidence “presented” was acceptable.
• Even though none are required, which
ones would be considered essential?
• Which ones should be brought to
conference with principal?
Evidence For Self
Assessment
Table Tasks
• Share the kinds of evidence that were
generated.
• Were any common among the table?
• Does the evidence convincingly support
the rating?
• Discuss whether you agree that
evidence “presented” was acceptable.
• Even though none are required, which
ones would be considered essential?
Student Growth 101
Juan D’Brot
Executive Director
Office of Assessment and Accountability
Student Growth 101
• Learning Targets:
– Understand how the school-wide growth
measure is developed
The WV Growth Model:
Changing Conversations
about Education
Juan D’Brot
Executive Director
Office of Assessment and Accountability
WV Growth Model
1.
2.
3.
4.
5.
What questions?
Purpose
Status vs. growth
What student growth data looks like
What school growth data looks like
First…
A challenge!
Stop me for questions…
Asking the Right Questions…
• Until you’ve defined the question,
you cannot examine the appropriate
data.
• Why?
– Different data answer different
questions
– Different questions lead to different
conversations
Fact #1: …increased use of student
assessment data as a “significant
factor” in (insert favorite accountability
topic here…)
• But what student data? That leads us to
start asking questions…
• But what questions?
• Whose questions?
WhoseWHY?
Questions?
• Researchers
To answer
questions about
• Policymakers
• Administrators
the student
• Business Leaders
• Teachers
How?
• Community Members
not declaring a verdict.
•By
Parents
Instead,• by
changing the
Guardians
conversation…
Components of the System should
Align with the Purpose of the System
• The revised teacher evaluation system is
intended to do what?
– Change conversations
– Provide support and direction
– Identify areas of best practice and need
• The revised consideration of student
learning (growth) is intended to do what?
– Change conversations
– Provide direction
– Reframe evidence of student learning
Bigger Question:
How do we align the
revision of both systems?
…Fact #2
Fact #2: Assessment Systems try
to be Everything to Everyone
• Danger: answering too many
questions with the same data
– Differentiated Instruction
– Student Proficiency
– Class Performance
– AYP
• Possible Solution: Can we come up
with a common question across
stakeholders?
The WV Growth Model:
Our Purpose
• To provide an answer to a few
common questions:
1. “How much did my student grow this
year?”
2. “What does this growth mean
compared to everyone else?”
3. “Is it enough growth?”
What do these three questions have in common?
Information about students
Status vs. Growth
• Before growth, let’s discuss status
– Status: A snapshot measure of a single
point in time – “Proficiency”
– Growth: Multiple snapshots across many
points in time
Status vs. Growth
A Balanced Approach
Low Growth
High
Performing
(Status)
Low
Performing
(Status)
High Growth
Quadrant 2
Quadrant 3
Is believed not to require
Does not require improvement
improvement because declining and is a potential site of interest
student growth is not recognized
for best practices
under status model
Quadrant 1
Quadrant 4
Legitimately requires
Is believed to require
improvement
improvement because high rates
of growth are not recognized
under the current status model.
May be a potential site of interest
for best practices.
Questions - Revisited
1. “How much did my student grow this year?”
(Time 2 – Time 1) – We can see this today
2. “What does this growth mean compared to
everyone else?” (Normative Component)
3. “Is it enough growth?” (Criterion Component CSOs)
How much did my student
grow this year?
–1st Question
» How much academic growth do individual students in
WV exhibit?
» Scale Scores – Tell very different stories
» Time 2 (2010 Scale Scores – Mastery = 550) – Time 1
(2009 Scale Scores – Mastery = 500)
2009 Scale Score
2010 Scale Score
Net “Growth”
525
575
50
550
575
25
425
575
125
375
400
25
What does this growth mean
compared to everyone else?
–Pre-Growth Model
»This year and last year (only 2 years
worth of comparison)
–Post-Growth Model
»Contextual consideration of growth
»Yields “Student Growth Percentiles”
»Think height
What is a Student Growth Percentile?
• Given (3 things):
– A student’s prior scale scores
– Academically similar students
– Current scale score
• A student’s current scale score represents a
percentile of growth – a Student Growth Percentile
• Think of it as the probability of a student’s current
achievement based on their past achievement:
– Pr(Current Achievement|Past Achievement)
• Growth percentiles describe the probability/rarity
of a student’s current achievement based upon
their prior achievement.
Vertical Axis: Number of Students at a Particular Scale Score
Vertical Axis: Number of Students at a Particular Scale Score
Vertical Axis: Number of Students at a Particular Scale Score
Vertical Axis: Number of Students at a Particular Scale Score
What does this student’s growth
mean compared to everyone else?
–2nd Question
» What does the academic growth of an individual
student in WV really mean?
» Scale Score Growth – Tells very different stories
» Time 2 – Time 1: Based on other students with similar
academic histories.
2009 Scale
Score
2010 Scale
Score
Net “Growth”
Growth
Percentile
525
575
50
40th
550
575
25
20th
425
575
125
99th
375
400
25
50th
But is it enough growth?
–3rd Question
» Is the academic growth of an individual student in WV
enough to make it to Mastery? What about staying at
Mastery?
» This year’s Growth Percentile isn’t enough
» Where would a student be next year if s/he exhibited:
• High Growth
• Typical Growth
• Low Growth
Our Growth Options
• How much growth? – Starts with the student.
Can be aggregated to drive school
conversations
–
–
–
–
Very Low Growth - Unsatisfactory
Lower Growth - Emerging
Typical Growth - Accomplished
Higher Growth – Distinguished
• Is it enough growth? – Can drive student
learning conversations
– Catching Up
– Keeping Up
– Falling Behind
Above
Mastery
Is it enough
growth?
Mastery
Partial
Mastery
How much
growth?
Novice
Novice
Partial
Mastery
Partial
Mastery
Mastery
Above Mastery
Mastery
Partial Mastery
Novice
Partial
Mastery
Partial Mastery
Partial Mastery
Partial Mastery
Novice
Partial Mastery
Partial Mastery
Mastery
Above Mastery
Mastery
Partial Mastery
Novice
Above Mastery
Mastery
Partial Mastery
Novice
Novice
Partial Mastery
Partial Mastery
Mastery
Above Mastery
Mastery
Partial Mastery
Novice
Partial
Mastery
Partial Mastery
Partial Mastery
Partial Mastery
From Student to School Growth
–For whom can we calculate growth?
» Only students in grades 4 – 11, in tested subjects
» Students must have at least 2 consecutive scores
» We examine RLA and Math
–How do we aggregate growth to the
school?
» Examine students by grade and by content for the state
» Identify all of the students in the school
» Take the median (middle) growth percentile of all students in
a school for each grade and content.
Let’s start with 5th grade
Student
Mathematics
Johnny
25th
Suzy
35th
Kenny
75th
Lori
85th
Juan
15th
Lisa
40th
Amelia
27th
Robert
40th
Lee
45th
Jan
62nd
Frank
51st
Mary
12th
5th Grade MEDIAN
Growth
40th
Let’s start with 5th grade
Student
Mathematics
Proficiency
Johnny
25th
Novice
Suzy
35th
Novice
Kenny
75th
Novice
Lori
85th
Novice
Juan
15th
Novice
Lisa
40th
Novice
Amelia
27th
Distinguished
Robert
40th
Distinguished
Lee
45th
Distinguished
Jan
62nd
Distinguished
Frank
51st
Distinguished
Mary
12th
Distinguished
5th Grade MEDIAN
Growth
40th
Distribution of Student Growth Percentiles by Scale Score
Grade 5 Mathematics
Demographi
cs
Aggregated Growth at the
School Level
• Remember the 4 quadrants
– High vs. Low Achievement
– Based on a Scale Score
– High vs. Typical vs. Low Growth
– Based on Student Growth Percentiles
• An example of a school level report
Moving on to school
• Look at each grade distribution of SGPs
• Take the median of a median
– 4th grade
– 5th grade
– 6th grade
Grade
Percentile
4th
35th
5th
40th
6th
37th
School Growth for MATH
37th
How it fits in with the
Evaluation System
• 5% of the total evaluation weight at
the school level
• School level growth?
– Lowest Growth (1-24th percentile)
– Lower Growth (25th – 34th percentile)
– Typical Growth (35th – 65th percentile)
– Higher Growth (66th – 99th percentile)
Percentile Bands within the
Evaluation System
Lowest Growth
Unsatisfactory
Low Growth
Emerging
Typical Growth
Accomplished
High Growth
Distinguished
Why the ranges?
• Empirical consideration of current
data
• Proposed percentile bands for first
pilot year
• Open for revision
• Aligns with 4 point rubrics
Questions?
Thank You
Juan D’Brot
(jdbrot@access.k12.wv.us)
Executive Director
Office of Assessment and Accountability
Student Learning in
Context
Student Learning in
Context
• Learning Targets:
– Know the definition of student learning;
– Identify examples and non-examples;
– Evaluate the quality of potential
measures
Wrap Up & Feedback
A New Landscape
Some Parting
Instructions
• Feedback Forms
• Principals and Assistant Principals
– Part Two
• Dine well and rest well!
• See you tomorrow at 8:00 a.m.