Evaluating Growth
Recognizing and Supporting Educators
Through AchieveNJ
February 28th 2014

2

Nationwide Education Reform
– NCLB
– CCSS
– PARCC
– Race to the Top
– ESEA waiver
3

1.
College- and Career-Ready Expectations for All Students
• developing and administering annual, statewide, aligned, high-quality assessments, and corresponding academic achievement standards, that measure student growth
2.
State-Developed Differentiated Recognition, Accountability, and
Support
• systems to improve struggling schools… progress…closing gaps
3.
Supporting Effective Instruction and Leadership an evaluation system that will require
• continual improvement of instruction;
• use multiple valid measures in determining performance levels, including data on student growth for all students
4

Educator evaluation in New Jersey
Three Years in the Making
CCSS adopted
Educator
Effectiveness Task
Force formed
EPAC, Pilot 1 launched
$38 million
Race to the Top award for NJ
TEACHNJ Act passed
EPAC and external
Rutgers reports issued
All districts launch improved evaluations
PARCC pilot
2010 2011 2012 2013 2014
Task Force releases recommendations
1st round of evaluation regulations proposed
Pilot 2 selected
2nd round of evaluation regulations proposed

New Jersey’s evaluation framework
NOW
In-depth
Multiple measures
Multiple observers
Student growth counts
Comprehensive training required
Must be linked to professional development
Educator-driven

New Jersey’s evaluation framework
NOW
In-depth
Multiple measures
Multiple observers
Student growth counts
Comprehensive training required
Must be linked to professional development
Educator-driven

Evaluations use multiple measures
Practice
Teacher
Practice
Student Achievement
Student Growth
Objective
(SGO)
Student Growth
Percentile
(SGP)
Summative
Rating
8

What do SGPs and SGOs have in common?
SGP SGO
9

What do SGPs and SGOs have in common?
S G P S G O
10

What do SGPs and SGOs have in common?
S G P S G O
11

What do SGPs and SGOs have in common?
S G P S G O
12

How do SGPs capture growth?

All students can show growth.
• Student Growth Percentiles (SGP) measure how much a student has learned from one year to the next compared to peers with a similar NJ ASK performance history from across the state.
13

Proficiency
250
Advanced Proficient
200
Proficient
230
150
100 Partially Proficient
220
160
205
165
Gr. 3 Gr. 4
NJ ASK Scale Score by Grade
Gr. 5
 Maria
 Albert
14

SGP considers growth, not proficiency
Albert has taken the 5 th grade NJ ASK.
How does his score compare to those of his academic peers?
Albert’s Prior
Scores

3 rd Gr.
150
4 th Gr.
160
5 th Gr.
165
Academic Peers’
Prior Scores
     
3 rd Gr.
≈150
4 th Gr.
≈160
5 th Gr.
???






























 
15

Albert’s 5 th
250 Advanced Proficient
Grade NJ ASK Score
200
Proficient
150
100 Partially Proficient
Gr.3
160
Gr.4
Albert’s Academic Peers’ NJ ASK Scores
250 Advanced Proficient
165
Gr.5
200
Proficient
150
100 Partially Proficient
Gr.3
160
Gr.4
Gr.5
200
29%
110
70%
16

Albert’s Score

5 th Gr.
165
Academic Peers’ Scores
     
5 th Gr.
110 - 200
       
SGP
1% 70% 99%
17

How are student SGPs related to a teacher’s mSGP rating?
Albert’s SGP along with the
SGPs of all his teacher’s students are arranged from low to high.
















Student
Hugh
Eve
Clarence
Clayton
Earnestine
Helen
Clinton
Tim
Jennifer
Jaquelyn
Lance
Roxie
Laura
Julio
Selena
Ashlee
Albert
Mathew
Marcus
Charles
Milton
SGP Score
51
53
57
61
31
35
39
44
46
12
16
22
24
25
65
66
70
72
85
89
97
Median SGP Score
Albert’s teacher receives an effectiveness rating by taking the median SGP score - in this case, 51.
18
18

SGP conversion from 1-99 to 1- 4 mSGP Score
28
29
30
31
32
33
34
1 – 20
21
22
23
24
25
26
27
Evaluation
Rating
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2
2.1
2.2
2.3
2.4
mSGP Score
43
44
45
46
47
48
49
35
36
37
38
39
40
41
42
Evaluation
Rating
2.5
2.5
2.6
2.6
2.7
2.7
2.8
2.8
2.9
2.9
3
3
3
3
3 mSGP Score
58
59
60
61
62
63
64
50
51
52
53
54
55
56
57
Evaluation
Rating
3
3
3
3
3
3
3.1
3.1
3.2
3.2
3.3
3.3
3.4
3.4
3.4
mSGP Score
72
73
74
75
76
77
78
79
80 - 99
65
66
67
68
69
70
71
3.7
3.7
3.8
3.8
3.8
3.9
3.9
3.9
4
Evaluation
Rating
3.5
3.5
3.5
3.6
3.6
3.6
3.7
Based on her mSGP score, Albert’s teacher receives an mSGP evaluation rating of 3.
This is combined with other evaluation components in a summative rating.
19

How do SGOs capture growth?
A Student Growth Objective is a long-term academic goal that teachers set for groups of students and must be:
• Specific and measurable
• Aligned to New Jersey’s curriculum standards
• Based on available prior student learning data
• A measure of student learning between two points in time
20

How do we effectively set a long-term academic goal for students?
1.
Establish what students know and can do currently based on a range of information.
2.
Use an assessment that fairly and thoroughly measures what students should be able to know and do after instruction.
3.
Determine what an ambitious but achievable learning goal is as measured by the assessment based on information about prior learning.
21

1. Assessments
Traditional
Assessments
• District, school and departmental tests e.g., modified final exams, benchmark exams
• State and national exams (except the NJ
ASK), e.g. NOCTI, AP
Portfolio
Assessments
• Writing and reflection sample
• Laboratory research notebook
• Portfolio of work
• Project-based assessment
• Teaching Strategies
Gold
®
Performance
Assessment
• Lab Practicum
• Sight reading in music
• Dramatic performance
•
Skills demonstration
• Persuasive speaking
• DRA ™2
22

2. Prior student learning
Performance Data to Determine Students’ Starting Points
Grades/performance in current year
Prior-year test results that assess knowledge and skills that are pre-requisites to the current subject/grade
Test results in other relevant subjects from prior years
Students’ grades in previous classes
Beginning-of-course diagnostic tests or performance tasks
23

3. Ambitious and achievable SGO targets
Growth
• 75% of students will increase their performance by an average of 15% on 4/5 measures of fitness over the course of 4 months.
• 85% of students will decrease the score between their starting points and 100 by at least 50% during the SGO period.
Achievement
• 70% of students will score 80% on the social studies final assessment
• 90% of students will score 3/4 on at least 8 components of the art portfolio rubric .
24

Top Challenges
1.
Collecting quality baseline data that can be used to effectively set realistic targets
2.
Setting targets that are fair for a wide variety of students
3.
Developing, administering, and scoring an assessment
4.
Administrator – teacher collaboration
25

1. Baseline data taken from multiple high quality sources
8
9
6
7
10
11
12
3
4
5
1
2
Student ID
Prior Test
Scores
NJ ASK 8
Math
230
202
211
241
263
284
199
201
144
182
143
171
NJ ASK Math
Score
<200
200 – 249
200 – 300
Current Year Test Scores Markers of Future Success
Unit 1
90
91
57
50
58
62
78
100
90
95
85
90
Unit 2
85
88
75
58
58
83
83
97
95
95
86
92
Average
Score
98.5
92.5
95
85.5
91
87.5
89.5
66
54
58
72.5
80.5
Current Year Test
Score Average
<70
70 – 85
85 – 100
Number of Future
Success Markers
0 – 1
1 – 2
2 – 3
Participates in Class
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
No
No
Yes
No
Preparedness
Group
Low
Medium
High
Completes
Retakes
Yes
Yes
Yes
No
No
No
Yes
Yes
No
No
Yes
Yes
Completes
Homework
No
Yes
Yes
No
Yes
Yes
Yes
No
No
No
No
No
Total Points
Target Score on
Summative
70
80
90
1
0
2
3
0
2
1
3
1
2
2
3
Preparedness
Group
High
High
High
High
High
High
High
Low
Low
Low
Medium
Medium
26

2. Fair targets for a wide variety of students
• Use a growth measure, e.g. improvement on assessment such as DRA or MAP
80% students will show one year’s reading growth, or be on grade level, as measured by the DRA2
• Use tiered SGOs
Set different achievement targets for students starting at different places
27

Scoring Plan
Preparedness
Group
Low
Medium
High
Target Score on
Final
Assessment
70
80
90
Objective Attainment Level Based on Percent and Number of
Students Achieving Target Score
Exceptional
Attainment (4)
Full
Attainment (3)
Partial
Attainment (2)
Insufficient
Attainment (1)
>85% students
(31-36)
>85% students
(19-21)
>85% students
(8)
≥70% students
(25-30)
≥70% students
(15-18)
≥70 % students
(6-7)
≥55% students
(18-24)
≥55% students
(11-14)
≥55% students
(4-5)
<55% students
(0-17)
<55% students
(0-10)
<55% students
(0-3)
28

• Common assessments developed or chosen by teachers
• Modify existing assessments as necessary
• Develop a protocol for administering and scoring all assessments to ensure consistency
• If used at all, use pre-assessment to establish the level of preparedness and pre-requisite knowledge
• Don’t test more, increase the quality of what is already there
29

• Teacher-created
• Tailored
• Collaborative
• Process-based
• Flexible
According to the law:
• A teacher develops SGOs in consultation with his or her supervisor.
30

31

In what ways do SGOs capture what effective teachers regularly do?
 Teach a curriculum that is aligned to standards.
 Determine the needs of students using several methods including a variety of assessments.
 Differentiate instruction based on the needs of students.
 Set goals for students appropriate to their grade, subject, and readiness level.
 Use high quality assessments to measure student performance.
Formalize and document the process, and be recognized for doing these things well.
32

• Learning from those who have gone before us
• Learning from New Jersey’s educators
• Adjusting the system as needed
33