Educator Effectiveness
Initiatives
December 1, 2011
Superintendents’ Quarterly Meeting
Overview
1. Educator Effectiveness Reporting
2. Context for Educator Effectiveness Work
• What We Have
• What We Need
3. Implementing Educator Effectiveness Policies
4. “Common Exams” Design Process
5. Next Steps for 2011-12
12/1/2011 • page 2
Educator Effectiveness
Reporting
Two Types of Reporting
1. Federal Reporting required by SFSF Funding and
acceptance of ARRA funds
• The US Department of Education requires states to submit one
evaluation rating score per teacher or administrator
2. Public Reporting established by NC in Race to the Top
application
•
School level - number and percentage of teachers receiving
the various ratings on each standard
•
District level - number and percentage of principals/assistant
principals receiving the various ratings on each standard
12/1/2011 • page 4
Location of Data
District Report Card, Quality Teachers and Administrators Tab
12/1/2011 • page 5
Location of Data
Scroll to bottom of the screen
12/1/2011 • page 6
Location of Data
Link takes user to NCDPI Site
12/1/2011 • page 7
Accessing Data
On the site, users will select from several ranges of letters to
see PDF files for districts in that range of alphabet (for example,
A-D)
When a user opens a PDF file, she will see introductory text
The key messages emphasized in the text and on the headers of
all accompanying pages are:
•
Only probationary teachers, along with a small number of
career-status teachers, appear in the data
•
Because the evaluation instrument is intended to spur
professional growth, it is normal to see a distribution of
teachers across all rating categories
12/1/2011 • page 8
Sample Teacher Report
Rating categories
Table 2a: School Report of Aggregate Teacher Evaluation Ratings for All Standards
Name of school
School: Sample Elementary School
Creation Date: 22 September 2011
Filter Period: 1 August 2010 to 31 July 2011
Not Demonstrated (1)
Developing (2)
Proficient (3)
Accomplished (4)
Distinguished (5)
Standard
#
Standard1
Standard2
Standard3
Standard4
Standard5
The five
standards
%
0
0
0
0
0
#
0
0
0
0
0
%
0
0
0
0
1
#
0
0
0
0
11
Number of teachers
receiving developing
rating on Standard Five
%
5
3
5
5
5
#
55
33
55
55
55
%
1
3
2
3
1
#
11
33
22
33
11
%
3
3
2
1
2
Percentage of teachers
evaluated who received
developing rating on
Standard Five
12/1/2011 • page 9
33
33
22
11
22
Sample Principal/AP Report
Rating categories
Table 6b: District Report of Aggregate Principal/AP Evaluation Ratings for All Standards
Name of district
District: Sample County Schools
Creation Date: 19 September 2011
Filter Period: 1 August 2010 to 31 July 2011
Not Demonstrated (1)
Developing (2)
Proficient (3)
Accomplished (4)
Distinguished (5)
Standard
#
Standard1
Standard2
Standard3
Standard4
Standard5
Standard6
Standard7
%
0
0
0
0
0
0
0
The seven standards
#
0
0
0
0
0
0
0
%
0.0
1.0
0.0
0.0
0.0
0.0
0.0
#
0.0
1.4
0.0
0.0
0.0
0.0
0.0
%
16.0
4.0
10.0
9.0
8.0
15.0
14.0
Number of
administrators receiving
developing rating on
Standard Two
#
22.9
5.7
14.3
12.9
11.4
21.4
20.0
%
39.0
40.0
36.0
38.0
42.0
34.0
27.0
#
55.7
57.1
51.4
54.3
60.0
48.6
38.6
%
15.0
25.0
24.0
23.0
20.0
21.0
29.0
Percentage of
administrators evaluated
who received developing
rating on Standard Two
12/1/2011 • page 10
21.4
35.7
34.3
32.9
28.6
30.0
41.4
Context for Educator
Effectiveness Work
What We Have
Statewide NC Educator Evaluation System
•
•
All 115 school districts
Many charter schools
Standard, Statewide Measures of Student
Growth and Achievement
•
•
•
English Language Arts
Mathematics
Some years of Science
12/1/2011 • page 12
What We Need
1. A required, standard, objective measurement
of the effect each teacher has on each
student’s learning
2. Common measures of student growth in all
content areas
3. A common statewide process for identifying
“effective” teachers and leaders
12/1/2011 • page 13
What We Need
1. A required, standard, objective
measurement of the effect each
teacher has on each student’s
learning
Completed (Summer 2011)
Added Sixth and Eighth Standards
2. Common measures of student
growth in all content areas
In Progress (2011-12)
“Common Exam” Design Process
(Phase One of Three Completed)
3. A common statewide process for
identifying “effective” teachers
and leaders
Targeted for Completion (January
and February 2012)
Second Set of State Board Policies
on Educator Effectiveness
12/1/2011 • page 14
Implementing Educator
Effectiveness Policies
Decision Points
State Board of Education
Action Step
Date Completed/
Expected
Completion Date
Key Questions
still to be Answered
Add Sixth and Eighth
Standards
Summer 2011
--
Require Annual Evaluation
for All Teachers
Summer 2011
--
Define Effective and Highly
Effective Educators
January and
February 2012
--
Require Effective
Designation to Convert to
SP2 Licensure
January and
February 2012
--
12/1/2011 • page 16
Decision Points
State Board of Education
Action Step
Require Effectiveness
When Granting Tenure
Date Completed/
Expected
Completion Date
Key Questions
still to be Answered
January and
February 2012
--
Require Steps for
January and
Improvement for Ineffective February 2012
Teachers
--
Determine Rating
Categories for the Sixth
and Eighth Standards
January and
February 2012
Number of categories?
12/1/2011 • page 17
Decision Points
State Board of Education
Action Step
Design an Interim Model
(2011-12) for the Sixth
Standard
Date Completed/
Expected
Completion Date
January and
February 2012
Design a Permanent Model August 2012
(2012-13 and beyond) for
the Sixth Standard
Key Questions
still to be Answered
Only a teacher’s
individual contribution to
student growth, or
combined with measure
of school-wide growth?
Individual, team, and
school-wide growth
used for sixth standard?
How might we use
student survey results?
12/1/2011 • page 18
Decision Points
State Board of Education
Action Step
Date Completed/
Expected
Completion Date
Key Questions
still to be Answered
Create Process for Timing
of Student Growth Data
Reporting
Late Spring 2012
How can timing of
student growth
information and
requirements for
evaluation completion
be aligned?
Select Growth Model Used
for Sixth and Eighth
Standard Ratings
Late Spring 2012
Consider
recommendations made
by West Ed
Finalize Educator
Effectiveness Data
Reporting
January 2012
Final text to appear
before data tables
12/1/2011 • page 19
Timeline: 2011-12
• Sixth and eighth standards added
• Teachers and administrators with three years of data receive
sixth/eighth standard ratings (with no formal role in
evaluation)
• Pilot programs with team value-added component and
student surveys
• School-wide data serves as discussion point for teachers of
non-tested grades and subjects
• Teachers in tested grades and subjects without three years of
data continue to accumulate data toward three year mark
12/1/2011 • page 20
Timeline: 2012-13
• Teachers and administrators with three years of data receive
sixth/eighth standard ratings (with formal role in evaluation)
• Potentially implement team value-added component
• Potentially implement student survey results in Educator
Evaluation Instrument
• Roll out “Common Exams” statewide (each teacher begins to
accumulate his or her own growth data)
• Teachers in all grades and subjects without three years of
data continue to accumulate data toward three year mark
12/1/2011 • page 21
Timeline: 2013-14
• Teachers and administrators with three years of data receive
sixth/eighth standard ratings (with formal role in evaluation)
• The only teachers and administrators not receiving
sixth/eighth standard ratings are those without three years of
data
12/1/2011 • page 22
“Common Exams”
Design Process
Three-Phase Process
Teachers design
item specifications
for all currently
non-tested grades
and subjects
Teachers design
guidance for
administering and
scoring “Common
Exams”
Teachers review
items aligned to
specifications
12/1/2011 • page 24
Input from Educators
We asked educators:
What does meaningful
assessment in your
content area look like?
12/1/2011 • page 25
Preliminary Answers
1. Teachers in Healthful Living, advanced mathematics courses, and
some social studies and science courses believe that wellconstructed multiple-choice items can fully assess their content
skills and knowledge
2.
K-2 teachers want an ongoing assessment model that
measures student growth over the course of a year
3.
Arts teachers believe that performance tasks graded with
standardized rubrics are the best way to measure the “creation
of the arts” that is so important to their standards
12/1/2011 • page 26
Preliminary Concerns
1. How valid and reliable will assessment results be if performance
tasks with rubrics are used to measure student growth?
2. How much time will younger students be spending on
assessment?
3. If results from “Common Exams” are not part of the school
accountability model, where is the push to give teachers the
appropriate amount of time to cover the curriculum that will be
assessed?
4. How does the State measure student growth in subjects that have
switched to thematic standards with LEA-level curricula?
12/1/2011 • page 27
The Balancing Act
Given limited resources to dedicate to this work:
Freedom from Bias in
Results
High Levels of Reliability
Ability to Feed Results
into EVAAS or Another
Sophisticated Growth
Model
High Levels of Content
Validity, with Performancebased Tasks
12/1/2011 • page 28
The Balancing Act
Freedom from Bias in
Results
High Levels of Reliability
High Levels of Content
Validity, with Performancebased Tasks
Ability to Feed Results
into EVAAS or Another
Sophisticated Growth
Model
12/1/2011 • page 29
The Balancing Act
Freedom from Bias in
Results
High Levels of Reliability
Ability to Feed Results
into EVAAS or Another
Sophisticated Growth
Model
High Levels of Content
Validity, with Performancebased Tasks
12/1/2011 • page 30
Tradeoffs
Reduction of performance-based courses to
multiple-choice assessment is less costly
Involvement of teachers in grading
increases risk for bias and decreases
reliability but allows for performancebased tasks
12/1/2011 • page 31
Next Steps for 2011-12
Next Steps
Jan 2012
• State Board of Education discusses next set of educator
effectiveness policies, including the definitions of effective and
highly effective educators
• Educator effectiveness data goes live on NCDPI website (linked
from ABC School Report Cards)
• Educator Effectiveness Work Group meets again
Jan – May 2012
• Pilot programs with student surveys and team value-added
component take place
February 2012
• SBE votes on next set of educator effectiveness policies
12/1/2011 • page 33
Summary
1. Educator Effectiveness Reporting
2. Context for Educator Effectiveness Work
• What We Have
• What We Need
3. Implementing Educator Effectiveness Policies
4. “Common Exams” Design Process
5. Next Steps for 2011-12
12/1/2011 • page 34