2013-14 Median Student Growth Percentile
Reports: An Overview
January 2015
Bottom Line Up Front
All districts received secure access to their 2013-14 teacher and principal/AP/VP
Median Student Growth Percentile (mSGP) data on January 8, 2015.
•
Teachers and school leaders have the highest impact on student achievement of
any in-school factor.
•
NJDOE has worked with NJ educators in taking a long and thoughtful approach
to implementing both evaluations and mSGP.
•
mSGP data is an important part, but only one part of an educator’s evaluation.
These scores will be used to finalize 2013-14 evaluations and to inform
educators’ ongoing professional development.
By statute, mSGPs (like all aspects of an individual’s evaluation) are confidential and should not
be shared publicly.
2
Context for SGPs in New Jersey
• Student Growth Percentile (SGP) methodology is not new; SGP was adopted
in the 2009-2010 school year and first used for school accountability in the
2011-2012 annual performance reports.
• TEACHNJ Act requires use of at least one measure of student growth in
evaluations (18A:6-123: “Standardized assessments shall be used as a
measure of student progress…”).
– AchieveNJ incorporates 2 measures of growth for teachers: mSGP and SGOs
and 3 measures for principals: mSGP, SGO Average, Administrator Goals.
• NJDOE has been building capacity to share mSGP for over 4 years,
including distribution of 2011-12 teacher data to pilot districts and 201213 teacher data to all districts.
• mSGP data is derived from roster and staff submission data provided by
districts through a submission process in place for many years.
3
Timeline of SGP Development in New Jersey
A thoughtful, multi-year approach to ensure data is accurate and usable
Evaluation Pilot
Federal Mandate for Stimulus
Advisory Committee
Funds: States Must Calculate
District SGP Profile
Provides Feedback
“Student Growth”; Link
Reports Deployed
on Usefulness of
Teachers to Students
via NJ SMART
SGP Data
2011-12 Teacher
Median SGP (mSGP)
Student SGPs
Reports Provided
Provided to All
to Pilot Districts for
Districts in NJ SMART
Learning Purposes
2010
NJ Adopts SGP
Methodology for
Calculating
Student Growth
2011
SGP Training
Begins for Districts;
SGP Video Released
2012
TEACHNJ Act
Passed; Growth
Measures Required
for Evaluation
2012-13 Teacher
mSGP Reports
Provided to All
Districts for Learning
Purposes and Data
Preview
2013
School SGPs Used in
School Performance
Reports per NJ’s
Federal ESEA Waiver
2014
2013-14 mSGP
Score
Verification &
Certification
Process
Completed by
Districts
2015
2013-14 Teacher &
Principal mSGP Reports
Provided to All Districts
for Use in Evaluations;
Score Verification Process
Announced
4
2013-14 mSGP Data
• The 2013-14 mSGP data counts for 30% of qualifying teachers’ and 20
or 30% of qualifying principals’ 2013-14 evaluations.
• Evaluation data of a particular employee shall be confidential in
accordance with the TEACHNJ Act and N.J.S.A. 18A:6-120.d and 121.d.
– Educator mSGP data should be handled in the secure manner one
would treat, handle, and store any part of a confidential personnel
record and should not be released to the public.
• While a dry run for teacher mSGP data was conducted last year to
improve roster verification processes, if educators identify a problem with
the 2013-14 mSGP score, the Department is offering options for
addressing the issue.
5
mSGP Qualification and Weighting: Teachers
To receive an mSGP score, teachers must:
1. Teach a 4th–8th grade Math or
Language Arts Literacy (LAL)
class, and
2. Be the teacher of record for at
least 60% of the course prior to
the NJ ASK assessment, and
3. Have at least 20 students with
valid SGP scores who are enrolled
in the class for at least 70% of
the school year before they take
the NJ ASK.
2013–14
30%
15%
Student Growth Percentile
Student Growth Objectives
55%
Teacher Practice
6
mSGP Qualification and Weighting: Principals
To receive an mSGP score, principals/APs/VPs must:
1. Be assigned to schools attended
by more than 20 separate
students who took the 4th- to 8thgrade Mathematics or Language
Arts NJ ASK; and
1. Be assigned to qualifying
students reported as having
attended the school for one full
year prior to the administration of
the NJ ASK, according to the data
submitted by the district in its
State Submission.
2013-14
Multi-Grade
Single-Grade
SGP Principals/APs/VPs SGP Principals/APs/VPs
(1 SGP grade in school)
(2 or more SGP grades in school)
20%
30%
20%
10%
20%
SGO Average
Administrator Goals
School SGP
30%
10%
10%
30%
20%
Principal Practice
Evaluation Leadership
7
Individual mSGP Report
Individual mSGP reports provide:
•
Background information about SGPs and mSGPs
•
The educator’s mSGP score for 2013-14; Chart depicting an example of a
teacher’s score is displayed below
•
Suggestions for how to interpret and utilize the data, as well as access to
additional resources
TEACHER
Language Arts Literacy
Mathematics
Overall
MSGP
46
68
52
# STUDENTS
ASSIGNED
22
22
22
MSGP SCORE
(1.0-4.0)
3.0
8
Addressing Data Issues
• The accuracy of mSGP Data is dependent on course roster and staff
submission data by the school district.
• Districts may expect to receive mSGP scores for educators who ended
up not qualifying – and vice versa.
• Incorrect district data submissions may have led to errors in mSGP
scores for individual educators.
• Guided by feedback from last year’s practice release, the Department
has created mSGP score verification and correction procedures.
•
•
Depending on the issue and the district’s ability to document evidence for a given
error, the mSGP score may be recalculated or removed from an educator’s
summative rating.
Detailed guidance has been posted here.
9
Potential Data Issues
• Educators expecting but not receiving the mSGP score
– Option A: For a teacher with 2 SGOs for 2013-14 OR for a principal/AP/VP,
recalculate summative rating using other component scores
– Option B: For a teacher with 1 SGO, eliminate summative rating
• Educators seeking to correct the mSGP score
– Option A: Provide documentation of errors, correct mSGP score, and
recalculate summative rating
– Option B: Remove mSGP score for a teacher with 2 SGOs OR for a
principal/AP/VP, and recalculate summative rating
– Option C: Remove mSGP score for a teacher with 1 SGO, eliminate
summative rating.
Note: State law requires multiple measures of student achievement; thus, if district data errors result in the
need to negate the mSGP score for a teacher – and that teacher did not set 2 SGOs – then the teacher cannot
receive a final summative rating for 2013-14. This scenario does not apply to principals, whose evaluations
already include both Administrator Goals and the SGO Average in addition to potential mSGP scores.
10
Evaluation Score Certification Tool
• Districts will have an opportunity to certify that all 2013-14 evaluation
data is correct or to make changes where necessary.
– The Department will release the 2013-14 Evaluation Score
Certification Tool, a new electronic application for districts to use in
certifying final 2013-14 summative ratings for all educators, in late
January.
– This interface will allow districts to review data, correct any errors
that occurred in the original NJ SMART submission, and certify the
accuracy of each staff member’s final score.
– Districts will have approximately one month to complete this process
after release of the tool.
11
Next Steps
• Districts: download secure mSGP reports and provide individual educator
reports confidentially, ideally through one-on-one conference
– Department resources: http://www.nj.gov/education/AchieveNJ/teacher/percentile.shtml
• Educators: review data, share questions/feedback with supervisor
• Districts Identifying Data Issues: consult Department guidance:
http://www.nj.gov/education/AchieveNJ/resources/1314mSGPdataprocedures.pdf
• Department: release 2013-14 Evaluation Score Certification Tool in coming
weeks
• Districts: certify all 2013-14 evaluation scores within 4 weeks of Tool access
12
Implementation Timeline: Common Core, State
Assessments, and Student Growth Data
‘10-’11
CCSS
curriculum
alignment
begins (K-2
math)
‘11-’12
CCSS curriculum
alignment
continues (K-12
ELA, additional
math)
CCSS aligned
questions
piloted in NJ
ASK
Setting the Context
‘12-’13
All curriculum
aligned to CCSS
NJ ASK aligned to
CCSS in ELA (3-8)
and Math (3-5)
2011-12 median
Student Growth
Percentiles
(mSGPs) released
to pilot districts
‘13-’14
‘14-’15
NJ ASK completely
aligned to CCSS
PARCC piloted in
classrooms across
1,276 schools
2012-13 mSGPs
released to all
districts as practice
exercise
Full PARCC
Implementation
2013-14 mSGP
data released
13
SGP and the PARCC Transition
Multi-Year Preparation
Growth, Not Proficiency
•CCSS adopted 4
years ago
•NJ ASK incrementally
aligned to CCSS in
content and rigor
•2014 NJ ASK scores
show little change
compared to prior
years despite
increased rigor
•81% of schools who
will use PARCC
engaged in pilot
testing technology
•To calculate SGP,
student growth is
compared with
growth of academic
peers taking the
same assessments
statewide
•SGP does not depend
on tests having a
consistent scale and
is not a criterionreference metric
Multiple Measures,
Lower Stakes
•mSGP is one of
multiple measures for
the educators in
tested grades and
subjects; others
include educator
practice, SGOs, and
additional goals and
evaluation leadership
for principals
•mSGP weight
reduced to 10% for
all educators for
2014-15 to recognize
adjustment to new
assessment
14
Appendix: Calculating SGP and mSGP
• The following slides depict and explain how SGP and mSGP scores are
calculated.
• For a video overview of SGP calculation, please access
http://survey.pcgus.com/njgrowth/player.html.
15
Calculating Student Growth Percentiles








All students can show growth.
• Student Growth Percentiles (SGPs)
measure how much a student has
learned from one year to the next
compared to students with a similar
NJ ASK performance history from
across the state (“academic peers”).
• SGP scores range from 1 to 99
16
Why Student Growth?
A student’s NJ ASK score does not tell the whole story.
NJ ASK Scale
250
Advanced Proficient
220
230
200
205
Proficient
160
150
100
165
Maria
Albert
Partially Proficient
Gr. 3
Gr. 4
Gr. 5
NJ ASK Scale Score by Grade
Under our current system, schools and parents might only notice that
Maria is “Proficient” and that Albert is “Partially Proficient.”
17
SGP Considers Growth, Not Proficiency
Albert has taken the 5th-grade NJ ASK.
How does his score compare to those
of his academic peers?
Albert’s Prior
Scores

Academic Peers’
Prior Scores

3rd Gr.
150
3rd Gr.
≈150
4th Gr.
160
4th Gr.
≈160
5th Gr.
165
5th Gr.
???


 






















 





18
Understanding Academic Peers
• Multiple consecutive years of data are required to calculate SGP.
– Since 3rd grade is the first year of the NJ ASK, 4th grade is the first year for
which we can calculate SGP.
• The more years of data available, the more precise an academic peer group is.
– By 8th grade, we have up to 6 years of scores for the calculation (as long as
students have taken the same assessments).
– If a student does not have an NJ ASK score for a given year, an SGP cannot
be calculated until he/she has 2 consecutive years of scores.
• The peer group includes all students in a grade cohort who have taken the exact
same assessments, i.e., all 7th graders who took the 3rd, 4th, 5th and 6th-grade
Math NJ ASK and the 7th-grade Math NJ ASK
• Academic peer groups are recalculated every year, considering all assessment
data for a given cohort of students.
19
Determining an SGP
NJ ASK Score
Albert’s 5th-Grade NJ ASK Score
250
Advanced Proficient
200
Proficient
150
100
160
165
Partially Proficient
Gr.3
Albert’s Academic Peers’ NJ ASK Scores
250
Advanced Proficient
200
Proficient
150
100
Gr.4
Gr.5
200
160
165
110
Partially Proficient
Gr.3
Gr.4
29%
70%
Gr.5
Albert scored 165. His academic peers scored between 110 and 200.
How did Albert do in comparison to them?
20
Determining an SGP
Albert’s Score

5th Gr.
165
Academic Peers’ Scores

5th Gr.
110 - 200




















SGP
1%
70%
99%
A comparison to his academic peers allows us to see that Albert actually
outperformed 70% of students who, up until this year, performed in a similar
manner to Albert. Albert’s SGP score would be 70.
21
Determining a Teacher’s mSGP Rating



Albert’s SGP is 

arranged along

with the SGPs of

all his teacher’s
students from 
low to high.







Student
Hugh
Eve
Clarence
Clayton
Earnestine
Helen
Clinton
Tim
Jennifer
Jaquelyn
Lance
Roxie
Laura
Julio
Selena
Ashlee
Albert
Mathew
Marcus
Charles
Milton
SGP Score
12
16
22
24
25
31
35
39
44
46
51
53
57
61
65
66
70
72
85
89
97
Albert’s teacher would
receive a median SGP score
of 51.
22
22
mSGP Technical Rules
Median this Year = 51
Student
Hugh
Eve
Clarence
Clayton
Earnestine
Helen
Clinton
Tim
Jennifer
Jaquelyn
Lance
Roxie
Laura
Julio
Selena
Ashlee
Albert
Mathew
Marcus
Charles
Milton
In future years, teachers should
know that:
SGP Score
12
16
22
24
25
31
35
39
44
46
51
53
57
61
65
66
70
72
85
89
97
•
If two or three years of data are
available, the Department will use the
best available score — either the
teacher’s median score of their current
roster or the median of all student
scores over the years available.
•
The mSGP score, along with the
observation and SGO scores, will be
placed on a 1.0 - 4.0 scale and
weighted appropriately to determine a
summative evaluation rating.
Median over 2 years = 56
Student
SGP Score
Hugh
Eve
John
Charles
Annie
Clarence
Clayton
Earnestine
Jake
Helen
Rachel
Clinton
Tim
George
Amber
Jennifer
Jaquelyn
Bobby
Lance
Roxie
Mike
Mel
Laura
Regina
Marissa
Julio
Faye
Selena
Ashlee
Jackie
Courtney
Albert
Matthew
Laura
Jack
Jared
Rick
Marcus
Charles
Michelle
Molly
Milton
12
16
16
20
20
22
24
25
27
31
33
35
39
41
42
44
46
50
51
53
55
56
57
58
60
61
63
65
66
67
68
70
72
77
78
80
84
85
89
92
95
97
23
mSGP Conversion from 1 - 99 to 1.0 - 4.0
mSGP Score
1 – 20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Evaluation
Rating
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2
2.1
2.2
2.3
2.4
mSGP Score
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
Evaluation
Rating
2.5
2.5
2.6
2.6
2.7
2.7
2.8
2.8
2.9
2.9
3
3
3
3
3
mSGP Score
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
Evaluation
Rating
3
3
3
3
3
3
3.1
3.1
3.2
3.2
3.3
3.3
3.4
3.4
3.4
mSGP Score
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80 - 99
Evaluation
Rating
3.5
3.5
3.5
3.6
3.6
3.6
3.7
3.7
3.7
3.8
3.8
3.8
3.9
3.9
3.9
4
Based on the mSGP score, Albert’s teacher receives a rating of 3 for this
component, which is then combined with other evaluation components to get a
summative rating.
24
SGP Conversion Chart Explained
mSGP Score
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
Evaluation
Rating
2.5
2.5
2.6
2.6
2.7
2.7
2.8
2.8
2.9
2.9
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.1
3.1
3.2
3.2
3.3
3.3
3.4
3.4
3.4
Why are all the values between 45 and 55 set
to the same score (3.0)?
• The Department believes that educators in
the middle of the mSGP distribution are
driving significant academic growth in their
students.
• Educators whose students achieve scores
in this range should be recognized by
receiving a rating on par with their impact.
25
SGP Conversion Chart Explained
mSGP Score
Evaluation
Rating
1 – 20
1.0
21
22
23
24
25
26
27
28
29
30
31
32
33
34
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
2.2
2.3
2.4
Why are the values at the extreme
ends of the distribution, 1-20 = 1
in this case (and 80-99 = 4), set
to the same score?
• When more than half of a
teacher's students are in the
top 20 percentile points on the
SGP scale it is an indication of
very high growth.
• When more than half of a
teacher's students are in the
bottom percentile points this is
an indicator of low growth to be
considered with other
evidence.
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
Evaluation
Rating
3.5
3.5
3.5
3.6
3.6
3.6
3.7
3.7
3.7
3.8
3.8
3.8
3.9
3.9
3.9
80 - 99
4.0
mSGP Score
26
SGP Conversion Chart Explained
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
Evaluation
Rating
3.5
3.5
3.5
3.6
3.6
3.6
3.7
3.7
3.7
3.8
3.8
3.8
3.9
3.9
3.9
80 - 99
4.0
mSGP Score
Why Decimals? Why Tenths?
• The use of decimals instead of whole
numbers enables the scale to
increase/decrease gradually, improving the
statistical efficiency of the conversion.
• This prevents large rating differences that
may not accurately reflect significant
differences in student learning.
27