Understanding Value-Added
Lesson 2: How Value-Added Works
Office of Accountability
Recap: What is Value-Added?
 Value-Added is the District’s measure of elementary school and teacher
growth.
 Value-Added is a nationally recognized way of measuring growth.
Academic Growth = Student Learning
2012
2013
 Emphasizes continual student improvement
 Provides information to understand what drives continual
improvement
Office of Accountability
2
Measuring Growth, Not Attainment
In this school, the percent meeting state
standards is 25% in both Year 1 and Year 2.
Attainment is unchanged – but are students
learning?
210
220
230
240
250
Scale
Score
200
210
220
Office of Accountability
230
240
250
Meets State Standards
200
Analyzing growth provides this
information
(Year 2)
260
270
280
290
300
(Year 1)
260
270
280
290
3
300
Accounting for Student Populations
 Student academic growth varies by grade, prior performance, and
demographics.
 The goal of the Value-Added metric is to measure the school or teacher’s
impact on student learning independent of student demographic factors.
 Value-Added accounts for the following student factors:
Prior Reading Score
Low-Income Status
Prior Math Score
ELL Status
Grade Level
IEP Status
Gender
Homelessness
Race/Ethnicity
Mobility
 Controlling for the factors above gives proper credit for growth to low
attainment schools and schools that serve unique populations.
Office of Accountability
4
How it Works
 Value-Added is not a comparison to similar schools.
‐ We do not look for a comparison group of schools that match each other
on all 10 student factors…such a group might not exist.
 Rather, Value-Added compares growth of students in each school
to growth of students across the District, controlling for the list of
student factors.
 To do this, we utilize a regression methodology, developed in
collaboration between CPS and academic experts from the
University of Wisconsin.
Office of Accountability
5
What is Regression?
 By measuring the impact of each student factor, the regression
model isolates the impact of the teacher on student growth.
 In other words, some growth is explained by external factors. We
can measure the average impact of these external factors on
growth at the District level and subtract that impact from the
teacher’s absolute growth.
 The growth that is left over after removing the impact of these
factors is attributed to the teacher. This is the value added by the
teacher.
Office of Accountability
6
For More on Regression…
 Two other presentations on this topic are available at
http://cps.edu/Pages/valueadded.aspx
 For an illustrative example of regression, view the “Oak Tree
Analogy” presentation. This presentation illustrates the ValueAdded model by using an analogy of two gardeners tending to oak
trees.
 For technical details, view “Lesson 3: Technical Specifications of the
Value-Added Regression Model”
Office of Accountability
7
Some Things to Know

Tested Students
•

Mobile Students
•
•

Mobile students count towards the Value-Added score in each school they attended, but are
weighted in the analysis by the amount of time they were in the school during the year.
At the teacher-level, mobile students count towards the Value-Added score for each teacher
that provided instruction to that student, but are weighted in the analysis by the time they
were in the school and the amount of instruction provided by each teacher.
English Language Learners
•
•

All students making normal grade progression who took ISAT or NWEA MAP in both the
previous year and current year are included in analysis.
For ISAT: ELL students in Program Years 0 through 5 are excluded from the analysis.
For NWEA MAP: Students with an ACCESS literacy score below 3.5 are excluded.
Students with Disabilities
•
•
IEP status is differentiated by type of IEP.
For example, the impact of a severe and profound disability is considered separately from the
impact of a speech and language disability.
Office of Accountability
8
Value-Added Scores
Value-Added measures the difference between the growth of
students for whom a school or teacher provided instruction and the
growth of similar students across the District.
A positive score indicates a school or teacher whose
students are growing at a faster pace than similar
students.
Zero (0) is the District average. A score near zero
indicates a school or teacher whose students are
growing at about the same pace as similar students.
A negative score indicates a school or teacher whose
students are growing at a slower pace than similar
students.
Office of Accountability
9
Standardization of Scores
 Growth is measured in scale score points (for NWEA, these are
called “RIT” scores).
Student A “grew” by 35 scale score
points
200
210
220
230
240
 However, one scale score point of growth is more difficult to obtain
in some grade levels than others.
 As a result, standardization is used to ensure that all Value-Added
scores are on the same scale.
Office of Accountability
10
Standardization of Scores
 Standardization is a common statistical process. In this case, it is
used to convert scale score points to a standard scale.
 The unit of measure is the “standard deviation” which is a measure
of distance from the mean.
‐ i.e., how much does School A’s score deviate from the mean?
 This places all scores on the same scale, allowing for more precise
comparisons between scores at different grade levels.
Office of Accountability
11
The Standard Scale
Features of the Standard Scale
 Zero (0) is the District average.
 About 68% of scores fall between -1 and 1.
 About 95% of scores fall between -2 and 2.
 About 99% of scores fall between -3 and 3.
 Only about 1% of scores are less than -3 or more than 3.
34%
2.5%
Office of Accountability
13.5%
34%
13.5%
2.5%
12
Reading the Value-Added Reports
(School-Level Report)
Value-Added Score
Number of Students in the
calculation
Office of Accountability
Percentile: This is the percent of
scores that fall below this score.
Percentiles range from 0th to
99th
Confidence Interval: This is
explained in the next set of slides.
13
Confidence Intervals
 The Value-Added model controls for factors that CPS can measure,
but there are some factors that cannot be measured, such as:
‐ Motivation to learn
‐ Family circumstances
‐ Health
 In addition, the Value-Added model is a statistical estimation of the
school or teacher’s impact on student learning and therefore
contains a certain amount of random error.
 For these reasons, the Value-Added model includes confidence
intervals.
Office of Accountability
14
Real World Example: Political Polling
A Political Polling company surveys a representative random sample of
1,000 community households about for whom they are going to vote on
Election Day. The question they pose is:
If the election were held today, for whom would you cast your ballot?
The percentages of responses breakdown as follows:
 Candidate Jones would receive 54% of the vote
 Candidate Smith would receive 46% of the vote
 There is a +/- 3% margin of error
Office of Accountability
15
Confidence Intervals in Political
Polling
With the margin of error of +/- 3%, the range of the percentage of people who plan on
voting for each candidates is as follows:
Candidate Jones would receive between 51% and 57% of the vote.
43%
44%
45%
46%
47%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
Candidate Smith would receive between 43% and 49% of the vote.
The confidence intervals do not overlap. Therefore the race is NOT “too close to call.” We can predict
with a high degree of confidence that Candidate Jones will win the race.
Office of Accountability
16
Confidence Intervals in Value-Added
 A confidence interval is a range of scores around the Value-Added estimate.
The Value-Added estimate is 1.0.
 The confidence interval is ± 0.3.
 The confidence interval range is from 0.7 to 1.3.
 The district average (0) is not in the confidence interval,
so we are 95% confident that the school’s effectiveness
is different than the average (above average in this
example).

Example:
0.7
1.0
1.3
 We are 95% confident that the true Value-Added score falls within the
confidence interval range.
 The confidence interval is “n” dependent, meaning larger samples yield smaller
confidence intervals.
‐ This is because in larger samples, a score that is different from the average
is less likely to be due to random error alone.
Office of Accountability
17
Statistical Significance
 If the confidence interval does not include zero, we say that the score is
statistically significant, meaning we are 95% confident that the score is different
from zero.
 A color is associated with each score based on the statistical significance:
Green
• Entire confidence
interval is above
zero.
• Score is positive
and statistically
significant at the
95% confidence
level.
Office of Accountability
Yellow
• Confidence
interval includes
zero.
• Score is not
statistically
different from
zero at the 95%
confidence level.
Red
• Entire confidence
interval is below
zero.
• Score is negative
and statistically
significant at the
95% confidence
level.
18
How Confidence Intervals are
Reported
This is how Value-Added scores are displayed in the school-level
This school has a Value-Added score of
reports.
-0.5 in reading
(the score is ½ of a standard deviation
below the mean)
The confidence interval ranges from -1.9 to 0.8
Because the confidence interval includes zero, we say that this school is not
statistically different from zero at the 95% confidence level.
For that reason, the bubble is yellow.
Office of Accountability
19
For More Information
 More lessons and other resources for understanding ValueAdded are available at:
http://cps.edu/Pages/valueadded.aspx
 Lesson 2 (Part 2): Oak Tree Analogy
 Lesson 3: Technical Specifications of the Value-Added Regression Model
Office of Accountability
20