Value-added estimates, by definition, examine the improvement of student achievement over a
period of time, usually a given school year. By doing so, the estimates measure students’
progress over the time period rather than assign predetermined proficiency levels using student
results from one assessment. Our district is using Education Analytics to produce our valueadded estimates. Below is a description of the model being employed.
A general value-added model is defined by four equations: a "best linear predictor" value-added
model defined in terms of true student post and prior achievement and three measurement error
models for observed post and prior achievement:
Student achievement: y1i =  + y0i+ alty0ialt + 'Xi + 'Zi + 'Ti + ei
(1)
Posttest measurement error: Y1i = y1i + v1i
(2)
Same-subject pretest measurement error: Y0i = y0i + v0i
(3)
Other-subject pretest measurement error: Y0ialt = y0ialt + v0ialt
(4)
where:











y1i is true post achievement;
y0i and y0ialt are true prior achievement in the same subject and in the other subject (math
in the ELA model, ELA in the math model), with slope parameters  and alt;
Xi is a vector of characteristics of student i, with slope parameter vector ;
Zi is a vector of characteristics of student i's classroom, with slope parameter vector ;
Ti is a vector of teacher indicators, or, for mobile students, the amount of time that
student i is taught by a given teacher j;
is a vector of teacher value-added effects (where j is the value-added effect for teacher
j);
ei is the error in predicting post achievement given the explanatory variables included in
the model;
Y1i is measured post achievement;
v1i is measurement error in post achievement;
Y0i and Y0ialt are measured prior achievement; and
v0i and v0ialt are measurement error in prior achievement.

Substituting the measurement error equations (2), (3), and (4) into the student achievement
equation (1) yields an equation defined in terms of measured student achievement:
Measured achievement: Y1i =  + Y0i+ altY0i alt + 'Xi + 'Zi + 'Ti + i
(5)
where the error term i includes both the original error component and the measurement error
components:
Error in measured achievement: i = ei + v1i - v0i - altv0ialt
(6)
Estimating the measured student achievement equation (5) without controlling for pretest
measurement error yields biased estimates of all parameters, including the value-added teacher
effects. This bias stems from the fact that measurement error in prior achievement causes the
error term (6), which includes the measurement error components v0i and v0ialt, to be correlated
with measured prior achievement. The desired parameters, as defined in equation (1), can be
estimated consistently if external information is available on the variance of measurement error
for prior achievement; approaches for consistent estimation in the presence of measurement error
are described in detail in Wayne Fuller, Measurement Error Models (Wiley, 1987).
The value-added model used for APPR in 2012-13 with NWEA MAP differs slightly from this
general description in the following ways.



Only the same-subject pre-tests will be used to predict student performance, rather than
both pre-tests from multiple subjects.
The vector of classroom characteristics (Zi, which some interpret as peer effects) will not
be included in this year’s model.
While a large sample of state data is used, value-added is a relative measure and therefore
results are only relative to districts within the sample. As more districts use the NWEA
MAP and Education Analytics to produce their value-added estimates, this comparison
group will grow.
Investigations are currently on-going to determine whether these should be included in the
models for the following year.
There are a number of model features that Education Analytics includes in its in value-added
models—features designed to produce estimates that are highly valid and reliable. These are
outlined below.
Measurement Error Correction
The use of a standardized test to measure a student’s knowledge base on a subject is never
perfectly accurate, because it is impossible to ask every question in every format to test how
much the student knows and understands of the tested topic(s). Measurement error takes place
because no test is a perfect measure; two students of equal proficiency can still receive different
scores on the same test due to luck and chance. Although it is impossible to calculate an
individual student's measurement error on any given test, it is possible to estimate the overall
magnitude of measurement error, or the standard error of measurement; this information is
reported in the NWEA MAP technical report.
For non-adaptive tests, measurement error is typically designed to be minimized at particular
points of interest (for many NCLB tests this is at the proficiency cut point). This creates a
situation where measurement error is extreme at the upper and lower portions of the test scale.
Because the MAP is computer adaptive, the measurement error properties are close to ideal:
relatively low measurement error even in the extremes of the distribution when students are
possibly several grades ahead or behind their peers. No assessment is perfect and as such the
error in the test needs to be accounted for.
The value-added model uses this information to produce an estimate of a student's measurement
error, based on a student’s performance of the test and other student-level information. The
model then takes measurement error into account when producing measures of student growth
and school value-added results. If measurement error is ignored, then the resulting teacher
measures typically favor teachers who teach higher-achieving students at the expense of teachers
who teach lower-achieving students.
Shrinkage
Statistical models rely on large sample sizes for increased precision. This means that teachers
with fewer students are likely to have less precise results than those with a larger student
enrollment, because a performance measure based on fewer students is more likely to be
influenced by randomness. This, in turn, increases the likelihood that the average improvement
among a teacher's students is very high or very low when a teacher has a smaller number of
students.
A statistical technique called shrinkage addresses this problem by adjusting for the number of
students within a given classroom, so that teachers with a much smaller number of students have
shrunk value-added estimates to adjust for their wider distribution (or statistical variance) that
may occur simply as a result of having fewer students. Including shrinkage thus avoids
overrepresentation of teachers with small numbers of students at both the top and bottom of the
distribution of measured performance. Our value-added models apply statistical shrinkage to
improve the accuracy of value-added effect estimates.
Dosage/mobility
Student and teacher mobility are introduced into the core value-added model with only a slight
tweak in the traditional definition of the classroom variables included in a standard multilevel
(hierarchical) model. In such a model, a school or classroom variable is set to 1 if a student
attended a given classroom during the school year and 0 otherwise. In order to accommodate
students/teachers who changed schools/classrooms during the school year, the classroom
variable is redefined so that it measures the fraction of time that a student attended a course
taught by a given teacher during the school year. This variant of the value-added model is
referred to as the dose model. In some cases this leads to technical challenges of weak
identification among various combinations of teachers. In these cases the model uses
sophisticated algorithms to detect and address these situations. Education Analytics has
implemented dose value-added models in several districts and states.