School Performance
Framework
Sponsored by
The Colorado Department of Education
Summer 2010
Version 1.3
Materials
Outcomes: School Performance Framework
• Understand the purpose of the school and district performance
frameworks.
• Define and appropriately use terminology included in the school and
district performance framework reports.
• Interpret the metrics (by performance indicator) included in the SPF
and explain why each of the metrics were included.
• Describe the “cut-points” for each metric and how an overall plan
type assignment is determined.
• Interpret the performance of one local school as presented by the
School Performance Framework Report.
• Identify next steps in using the school performance framework
reports.
Agenda: School Performance Framework
Purpose and
Terminology
Indicator
Points and
Plan Type
Assignments
Overview
Performance
Data by
Indicator
Interpreting
the School
Performance
Framework
Using the
School
Performance
Framework
School Performance Framework
Purposes
• Focus attention on what matters most
• Provide a body of evidence related to state-identified
performance indicators to support school and district
performance management
• Support school and district efforts to evaluate their
performance
• Establish a common framework for the state to use to hold
schools accountable for performance
• Identify schools that need additional support (priority
improvement and turnaround)
Focus Attention on What Matters
All students will exit Colorado’s K-12
education system ready for
postsecondary education and
workforce success.
--Colorado’s Achievement Plan for Kids
(SB 08-212)
On what does the framework focus?
Four key performance indicators:
– Academic Achievement
– Academic Growth
– Academic Growth Gaps
– Postsecondary and Workforce Readiness
How does the framework focus
on these indicators?
• Assigns a rating to each of the performance
indicators so schools/districts know the extent to
which they meet state expectations.
– Exceeds, Meets, Approaching, Does not meet
• The ratings roll up to an overall evaluation of the
school/district’s performance. That evaluation
informs the plan type assigned to the school.
– Performance, Improvement, Priority Improvement,
Turnaround
Mean vs. Median
Percentage vs. Percentile
Percentiles
Percentiles
• Range from 1 - 99
• Indicate the relative
standing of a
student’s score to the
norm group. (i.e. how
a particular compares
with all others)
Growth Percentiles
• Indicate a student’s
standing relative to
their academic peers,
or students with a
similar score history
(how his/her recent
change in scores
compares to others’
change in scores).
Experiencing Student Growth
Percentile
• Using the Student Growth Card, form a group
with all of the “students” with the same 3rd grade
scale score – academic peers.
• Within your group get in order by 4th grade scale
score.
• Turn your cards over and compare your growth
percentiles.
Experiencing Median Growth Percentile
• Using your Student Growth card, identify your
school (A, B, C, D, E).
• Form a group with others from the same school.
• Put yourselves in order (in a line) by growth
percentile.
• Identify the person in the middle (median).
• What is the median student growth percentile for
your school?
Adequate Growth
• What is adequate growth?
• Based on catch-up and keep-up
growth.
Catching Up
To be considered to be Catching Up:
• The student scores below proficient
(unsatisfactory or partially proficient) in the
previous year
• The student demonstrates growth adequate
to reach proficiency within the next three
years or by tenth grade, whichever comes
first.
Calculating Adequate Growth for Students
Scoring Below Proficient: Catching Up
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
95
55
Not Proficient
Calculating Adequate Growth for Students
Scoring Below Proficient: Catching Up
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
85
85
Not Proficient
Calculating Adequate Growth for Students
Scoring Below Proficient: Catching Up
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
80
80
80
Not Proficient
Calculating Adequate Growth for Students
Scoring Below Proficient: Catching Up
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
76
76
76
76
Not Proficient
Calculating Adequate Growth for Students
Scoring Below Proficient: Catching Up
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
95
85
80
76
76 is the minimum-this
student’s adequate
growth value
80
85
80
76
76
76
Not Proficient
Calculating Adequate Growth for Students
Scoring Below Proficient: Catching Up
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
76
76
76
76
Not Proficient
Calculating Adequate Growth for Students
Scoring Below Proficient: Catching Up
6th grade
7th grade
55th percentile growth will not
be enough for this student to
catch up – her current growth
is not adequate.
8th grade
9th grade
10th grade
Proficient
76
76
76
55
76
55
55
Not Proficient
55
Keeping Up
To be considered to be Keeping Up:
• The student scores at the proficient or
advanced level in the previous year.
• The student demonstrates growth adequate
to maintain proficiency for the next three
years or until tenth grade, whichever comes
first.
Calculating Adequate Growth for Students
Scoring Above Proficient: Keeping Up
6th grade
7th grade
79
8th grade
9th grade
10th grade
Proficient
12
Not Proficient
Calculating Adequate Growth for Students
Scoring Above Proficient: Keeping Up
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
25
25
Not Proficient
Calculating Adequate Growth for Students
Scoring Above Proficient: Keeping Up
6th grade
7th grade
38
8th grade
9th grade
10th grade
Proficient
38
38
Not Proficient
Calculating Adequate Growth for Students
Scoring Above Proficient: Keeping Up
6th grade
7th grade
50
8th grade
9th grade
10th grade
Proficient
50
50
50
Not Proficient
Calculating Adequate Growth for Students
Scoring Above Proficient: Keeping Up
6th grade
7th grade
38
50
25
12
8th grade
9th grade
10th grade
Proficient
50
38
25
50
38
50 is the maximum this student’s adequate
growth value
50
Not Proficient
Calculating Adequate Growth for Students
Scoring Above Proficient: Keeping Up
6th grade
7th grade
50
8th grade
9th grade
10th grade
Proficient
50
50
50
Not Proficient
Calculating Adequate Growth for Students
Scoring Above Proficient: Keeping Up
6th grade
7th grade
79
50
79th percentile growth will
be enough for this student
to keep up – his current
growth is adequate.
8th grade
9th grade
79
10th grade
79
Proficient
79
50
50
50
Not Proficient
Median Adequate Growth
AGP
45
78
99
32
11
91
55
67
43
10
77
Sorted AGPs
Adequate
growth
percentiles
for all
catch-up
and keepup students
Search for the
middle value…
Median AGP
Median Adequate
Growth for this
school is 55
SPF Terminology
• Consider the SPF Terminology (Appendix of the
SPF Technical Guide).
• Use the following legend to mark each term:
–  = “I got it”
– ? = Could use further clarification
– * = New term or new definition for a familiar term
• With your table group
– Discuss ?
– Identify any unanswered questions for full group
Agenda: School Performance Framework
Purpose and
Terminology
Indicator
Points and
Plan Type
Assignments
Overview
Performance
Data by
Indicator
Interpreting
the School
Performance
Framework
Using the
School
Performance
Framework
1 year vs. 3 year data
• CDE has provided two different versions of the
School Performance Framework Reports.
– The most recent year of data (2009)
– The most recent three years of data (20072009)
• Read 1-year vs. 3-year Data (technical guide)
– Why are there two options?
– When would the 3-year version be used to
assign the school plan type?
School Performance Framework
Overview
• Page 1: Summary of school performance,
including plan type assignment and ratings
for each performance indicator
• Page 2: Details of school performance by
indicator and sub-indicator
• Page 3: Scoring guides including cut
scores
School Performance Framework,
Page 1
Plan Type
Indicator
Assignment
Ratings
Four possible
plan type
assignments
Four possible
indicator ratings
Performance Indicators
• Read the definitions of each of the following performance
indicators on the first page of the sample SPF
–
–
–
–
Academic Achievement
Academic Growth
Academic Growth Gaps
Postsecondary and Workforce Readiness
• Discussion: Why does the School Performance
Framework focus attention on these areas of
performance?
SPF: Performance Indicator
Summary, Page 1
Agenda: School Performance Framework
Purpose and
Terminology
Indicator
Points and
Plan Type
Assignments
Overview
Performance
Data by
Indicator
Interpreting
the School
Performance
Framework
Using the
School
Performance
Framework
Performance Indicators and Data
Performance Indicator
Performance Data
Academic Achievement
% proficient and advanced (reading,
mathematics, science and writing)
Academic Growth
Median student growth percentile
(reading, mathematics and writing)
Gaps in Academic Growth
Median student growth percentile in
reading, mathematics and writing for
disaggregated groups
Postsecondary and Workforce
Readiness
Graduation Rate, Dropout Rate,
Average Colorado ACT Composite
Score
Look at Page 2, first column
Page 2: Sub-Indicators
Sub-indicators for each performance indicator:
• Points (earned and eligible)
• Percent of points
• Ratings
• School Performance Data
Defining State Expectations
1. Performance data
2. Basis for comparison
3. Definition of what level of performance meets
state expectations
– Scoring Guide on page 3 of the SPF
– SPF Technical Guide includes reference data
Academic Achievement (status)
• Performance Data: the percent of students in
the school scoring at the proficient or advanced
achievement level
• Comparison: Compared to the distribution of
percent proficient and advanced scoring
students in all other schools* in the state.
• Scoring Guide: sets the percentile cut-points for
each rating.
* Does not included schools with N of less than 16 or Alternative Education Campuses
Example
• What would result in a rating of “Meets” for academic
achievement for high schools in mathematics? (p. 3)
Academic
Achievement
The school's percentage of students scoring proficient or advanced was:
• at or above the 90th percentile of all schools.
• below the 90th percentile but was at or above the 50th percentile of all schools.
• at or above the 15th percentile of all schools.
• below the 15th percentile of all schools.
Exceeds
Meets
Approaching
Does Not Meet
• Below the 90th percentile and at or above the 50th percentile of all* schools.
(scoring guide, page 3 of SPF)
• For 2009 greater than or equal to 26.32% and less than 49.94% proficient
and advanced
Math
N of Schools
15th percentile
50th percentile
90th percentile
Elem
Middle
High
1021
47.69
68.97
87.18
482
29.16
52.66
75.79
380
5.69
26.32
49.94
Practice
• What percent proficient and advanced would result in a
rating of “Approaching” for academic achievement for
high schools in reading?
Practice
• What percent proficient and advanced would result in a
rating of “Approaching” for academic achievement for
high schools in reading?
– At or above the 15th percentile and below the 50th
percentile of all* schools. (scoring guide, page 3 of
SPF)
– For 2009 greater than or equal to 41.51% and below
69.57% proficient and advanced (technical guide)
Academic Growth
• Performance data: Median Growth Percentile and Median
Adequate Growth
• Comparison: Two steps
1. Did my school meet adequate growth? (yes or no)
2. How did my school’s growth compare to state
expectations for growth?
• Scoring guide: sets the cut-points for median growth
percentile using two tables that depend on whether or not
the school met adequate growth
Adequate Growth
1. What was my school’s median growth
percentile?
2. What was my school’s median adequate
growth percentile?
3. Did my school meet adequate growth?
Scoring Guide for Academic
Growth
High School Writing Example
• What was the median growth percentile in
writing?
– 58.0
• What was median adequate growth in
writing?
– 82.0
• Did this school meet adequate growth?
– No
Cut-Points for Academic Growth
Did we meet median adequate growth?
NO, so use the NO scoring guide.
58.0
Practice: High School Reading
• What was the median growth percentile for
high school reading?
• What was median adequate growth for
high school reading?
• Did this school meet adequate growth?
Practice
• What was the median growth for high
school reading?
– 52.0
• What was median adequate growth for
high school reading?
– 35.0
• Did this school meet adequate growth?
– YES
Cut-Points for Academic Growth
Did we meet median adequate growth?
YES, so use the YES scoring guide.
52.0
Growth Gaps
• Performance data: Median Student Growth
Percentile for disaggregated student groups.
• Comparison: Two Steps
1. Did the growth of disaggregated groups in my
school meet adequate growth? (yes or no)
2. How did the growth of disaggregated groups in my
school compare to state expectations for growth?
• Scoring guide: sets the cut-points for median growth
percentile using two tables that depend on whether or
not the disaggregated group met adequate growth
Practice
• What was the median growth percentile for students who
were eligible for free/reduced lunch in reading?
• What was the median adequate growth percentile for
students who were eligible for free/reduced lunch in
reading?
• Did the students who were eligible for free/reduced lunch
meet adequate growth?
• What this school’s Growth Gaps rating for free/reduced
lunch students?
Practice
• What was the median growth percentile for students who
were eligible for free/reduced lunch in reading?
– 42.0
• What was the median adequate growth percentile for
students who were eligible for free/reduced lunch in
reading?
– 41.5
• Did the students who were eligible for free/reduced lunch
meet adequate growth?
– Yes
• What this school’s Growth Gaps rating for free/reduced
lunch students?
– Approaching
State Expectations for Postsecondary
and Workforce Readiness
Performance Data
Comparison
Graduation Rate
Above 80%
Drop-Out Rate
At or below state average
Colorado ACT Composite
At or above the state average
Technical Guide:
•How graduation and dropout rates are calculated.
•The state average dropout rate.
•The state average ACT Composite Score.
Note: State averages are different for 1-year vs. 3year metrics
High School Practice
• What was the graduation rate?
– 88.6%
• What rating did that earn?
– Meets (above 80% but below 90%)
• What was the dropout rate?
– 2.2%
• What rating did that earn?
– Meets (below state average of 5.1, but above 1%)
• What was the average Colorado ACT composite score?
– 16.4
• What rating did that earn?
– Does not meet (At or below 19)
Agenda: School Performance Framework
Purpose and
Terminology
Indicator
Points and
Plan Type
Assignments
Overview
Performance
Data by
Indicator
Interpreting
the School
Performance
Framework
Using the
School
Performance
Framework
Indicator Points
• Points earned on a scale of 0-4
Points Earned Rating
4
Exceeds
3
Meets
2
Approaching
1
Does not meet
0
Not eligible for points
• Sum sub-indicator points for total indicator
points.
• Convert to % (total indicator points/100)
Example
• Consider the Academic Growth Gaps indicator (high
school example)
• How many points was this high school eligible to earn?
– 48
• How many points were earned?
– 29
• What percentage of points were earned for this
indicator?
– 60.4
• In which sub-indicator content area did this school earn
the most points?
– Writing
Indicator Ratings
• Based on the % of points earned of the points
eligible for the indicator.
• Cut-points on page 3 of SPF
Cut-Points for each performance indicator
Cut-Point: The school earned … of the points eligible on this indicator.
• at or above 87.5%
Achievement; Growth; Gaps;
Postsecondary and Workforce
Readiness
Exceeds
• at or above 62.5% - below 87.5%
Meets
• at or above 37.5% - below 62.5%
Approaching
• below 37.5%
Does Not Meet
• What rating would the example high school example
earn with 60.4% of points for Academic Growth Gaps?
High School Practice
• How many points did this school earn out of
eligible points on the Academic Growth
indicator?
– 3 in reading + 2 in math + 3 in writing = 8 points out of
12 eligible (4 per subject)
• What percent of points did this school receive on
the Academic Growth indicator?
– 66.7%
• What rating did that earn on Academic Growth?
– Meets
Framework Points
• Each indicator is weighted differently in the overall plan
type assignment (page 1 SPF).
Indicator
ES/MS Weight
HS Weight
Academic Achievement
25
15
Academic Growth
50
35
Academic Growth Gaps
25
15
Postsecondary and Workforce
Readiness
0
35
• Multiply the percent of points earned by the weight for
each indicator (Eligible points).
• These are your framework points for each indicator.
High School Example
• What percent of points did this high school
earn on the Academic Growth indicator?
– 66.7%
• What framework points did this high
school earn on the Academic Growth
indicator?
– 66.7% x 35 framework points = 23.3
framework points
Arriving at a Plan Type Assignment
• Sum framework points for each indicator.
• Convert this sum to a percentage (of 100).
• Use % of framework points to determine the plan type
assignment.
• See “Cut-points for plan type assignment” on page 3 to
determine plan type assignment.
Cut-Points for plan type assignment
Cut-Point: The school earned … of the total framework points eligible.
Total Framework
Points
• at or above 60%
• at or above 45% - below 60%
• at or above 30% - below 45%
• below 30%
Performance
Improvement
Priority Improvement
Turnaround
High School Example
• What framework points did this school earn on
each indicator?
– 5.6 for Achievement, 23.3 for Growth, 9.1 for Growth
Gaps, 20.4 for Postsecondary & Workforce Readiness
• What total framework points did this school earn?
– 5.6 + 23.3 + 9.1 + 20.4 = 58.4
• What percent of framework points did this school
earn?
– 58.4 out of 100 eligible = 58.4%
• What plan assignment does this school receive?
– Improvement
Distribution of
Preliminary
School
Performance
Framework
Plan
Assignments:
All School
Levels
See Technical Guide pages 26-27 for distribution by school level.
Agenda: School Performance Framework
Purpose and
Terminology
Indicator
Points and
Plan Type
Assignments
Overview
Performance
Data by
Indicator
Interpreting
the School
Performance
Framework
Using the
School
Performance
Framework
Resources to Support SPF
Interpretation
• School Performance Framework Technical
Guide
• Guiding Questions for Interpreting the
School Performance Framework (page 18
of the technical guide)
Interpreting your School
Performance Framework Report
• Take out a School Performance Framework
report for a school in your district (or use the
elementary sample report in the technical
guide).
• Use the guiding questions for interpreting the
School Performance Framework Report.
• Do you have any questions about what you see
on your SPF report?
Agenda: School Performance Framework
Purpose and
Terminology
Indicator
Points and
Plan Type
Assignments
Overview
Performance
Data by
Indicator
Interpreting
the School
Performance
Framework
Using the
School
Performance
Framework
Next steps with the SPF Reports
• Identify for which indicators and sub-indicators
school performance
– does not meet state expectations, or
– does not meet school or district expectations.
• Drill down . . . into additional data related to
priority sub-indicators.
• Use this data to focus improvement planning
efforts.
• Use plan type assignments to target support for
schools.
Additional State Data Sources
• School Growth Summary, District Growth
Summary
• CSAP score reporting
• Colorado Growth Model (both public and private)
• Student-level CSAP files (from CTB)
• Student-level flat files (growth, CSAPA, PSWR)
from CEDAR
Additional Local Data Sources
• What additional local data sources could
you analyze to drill-down in areas where
the SPF indicates performance has not
met expectations?
• Consider:
– Interim/benchmark assessment results
– Progress monitoring assessment results
– End of unit assessment results
– Disaggregated by various student groups
More School and District Reports
coming soon. . .
• July: Revised (based on your feedback) School
Performance Framework Reports with 20072009 data.
• July: District Performance Framework Reports
with 2007-2009 data
• August: School and District Performance
Framework Reports (2008-2010 data)
Provide Feedback!!!
• Parking Lot
– + the aspects of this session that you liked or
worked for you.
– The things you will change in your work or
would change about this session.
– ? Questions that you have
–
Light bulb: ideas, a-has, innovations
• On-line Feedback:
www.surveymonkey.com/spffeedback