EVAAS Concepts:
NCEs and Standard Errors
Sandy Horn
January 2013
SAS® EVAAS® for K-12
Copyright © 2010, SAS Institute Inc. All rights reserved.
Similarities between the EVAAS Growth
Standard Model and the Predictive Model
For both models:
• Progress is measured relative to the
average progress for a grade/subject in
North Carolina in the year tested.
• Average Progress/Growth Standard is
0.0.
• Standard errors determine whether a
district, school, or teacher’s growth
measure is different from average.
(More on this, shortly)
2
Copyright © 2010, SAS Institute Inc. All rights reserved.
Differences between the EVAAS Growth
Standard Model and the Predictive Model
Growth Standard Model
 Only used with tests
administered in
sequential years.
Predictive Model
 Can be used with tests,
whether administered
sequentially or not.
 Does not predict where
students will score, so
doesn’t need three prior
test scores.
 Requires at least three
prior test scores to predict
where students will score,
relative to other NC
students who take the
same test. (Prior scores
don’t have to be in the
subject being analyzed.)
3
Copyright © 2010, SAS Institute Inc. All rights reserved.
Differences between the EVAAS Growth
Standard Model and the Predictive Model
Growth Standard Model
 Measures the difference
between a cohort’s
position in the state
distribution in a
grade/subject at the end
of one year and their
position in that
distribution at the end of
the next year.
 Reported in NCEs.
Predictive Model
 Measures the difference
between where students
would be expected to
score, assuming the
average progress,
statewide, and where they
did score, when tested.
 Reported in Scale Score
Points.
4
Copyright © 2010, SAS Institute Inc. All rights reserved.
Growth Standard Value Added Report
5
Copyright © 2010, SAS Institute Inc. All rights reserved.
Normal Curve Equivalent Units - NCEs
6
Copyright © 2010, SAS Institute Inc. All rights reserved.
NCEs and the Growth Standard
5th Grade:
NCE 37
6th Grade:
NCE 37
30 40 50 60 70
30 40 50 60 70
6th grade NCE 37 ― 5th grade NCE 37 = 0 = State Growth Standard
The State Growth Standard (0.0) is achieved when students do not
lose ground from year to year, relative to other students, across the
state, who take the same test. It signifies one year’s growth.
7
Copyright © 2010, SAS Institute Inc. All rights reserved.
Growth Standard Value Added Report
8
Copyright © 2010, SAS Institute Inc. All rights reserved.
What are Standard Errors, and Why aren’t
they all the same?
Standard Errors let us know how confident we can be that a sample
approximates a population. In other words, how sure can we be that our
students (a sample) made progress similar to or different from the progress
that is average for all the students in North Carolina (the population)?
If we have a lot of students and/or a lot of data points (test scores) and/or
somewhat complete data for our sample and so on, we can be pretty precise
in estimating the progress of that group of students. The standard error
would be small.
If we have few students and/or few data points and/or very spotty data, we
can’t be as certain of the estimated progress of that group of students. The
Standard Error would be large.
9
Copyright © 2010, SAS Institute Inc. All rights reserved.
Standard Errors are Protection against
Misclassification
Small negatives and small positives may not be
significantly different from Average. The standard error
lets you know whether the difference is significant or
not.
Unless there is strong evidence to the contrary,
standard errors ensure that an estimate is considered
“Average.” They protect districts, schools, and
teachers from being misclassified as different from
Average unless there is a significant difference.
10
Copyright © 2010, SAS Institute Inc. All rights reserved.
About those Standard Errors…
How much confidence do we have that the estimate is different from 0.0?
80%
80%
90%
90%
95%+
95%+
99%
99%
-3se
Does Not Meet
Expected Growth
-2se
-1se
0.0
+1se
Growth Standard
State Average
Meets Expected Growth
+2se
+3se
Exceeds
Expected Growth
11
Copyright © 2010, SAS Institute Inc. All rights reserved.
About those Standard Errors…
How much confidence do we have that the estimate is different from 0.0?
5th Grade:
NCE 37
6th Grade:
NCE 38.8
+1.8 NCEs above 0.0
0.0
Growth Measure:
1.8 NCEs/points
30 40 50 60 70
30 40 50 60 70
6th grade NCE 38.8 ― 5th grade NCE 37 = 1.8 NCEs
Is it different from the State Growth Standard?
12
Copyright © 2010, SAS Institute Inc. All rights reserved.
About those Standard Errors…
How much confidence do we have that the estimate is different from 0.0?
School 1:
Growth Measure: 1.8 NCEs
SE: 1.1 NCEs
+/- 2 SEs = 2.2 NCEs
School 2:
Growth Measure: 1.8 NCEs
SE: 0.8 NCEs
+/- 2 SEs = 1.6 NCEs
+2.2 NCEs
+1.6 NCEs
+1.8 NCEs above 0.0
-1.6 NCEs
Growth Std/State Avg (0.0)
-2.2 NCEs
School 1 Meets Expected Growth because its
Growth Measure is less than 2 SEs above or below
the Growth Standard.
School 2 Exceeds Expected Growth because
its Growth Measure is 2 SEs or more above
the Growth Standard.
It works just the same for the Predictive Model.
Just change NCE to Scale Score and Growth Standard to State Average.
Copyright © 2010, SAS Institute Inc. All rights reserved.
13
Growth Standard Value Added Report
14
Copyright © 2010, SAS Institute Inc. All rights reserved.