Office of the Dean
Teachers for a New Era
Teacher for New Era Assessment Committee
Educational Expansions Project Outline
Research Questions
Are there significant differences in CMT scores for pupils in the classrooms
instructed by Neag prepared teachers in comparison to those in classrooms
instructed by non-Neag prepared teachers?
1. For Neag vs. non-Neag, what is student performance disaggregated by …
i. grade?
ii. subject area? (reading, math, writing)
iii. educational reference group?
2. Are there significant differences between pupils in the classrooms
instructed by Neag prepared teachers in comparison to those in
classrooms instructed by non-Neag prepared teachers in terms of …
i. Disaggregated by student gender?
ii. Disaggregated by student race/ethnicity?
iii. Disaggregated by program (e.g. special education, ELL)?
iv. Disaggregated by length of time in particular school?
v. Growth over time?
vi. Pupil aspirations?
vii. Pupil perspectives on the learning environment?
viii. Daily average attendance?
ix. Suspension rates?
2. For Neag vs. non-Neag, what is pupil performance disaggregated by …
i.
ii.
iii.
iv.
v.
teacher gender?
teacher race/ethnicity?
teacher preparation program completed?
content preparation?
teacher performance
vi.
vii.
viii.
ix.
x.
xi.
number of years in Neag instructed classroom
Longetivity and retention in teaching profession
Longetivity in grade or subject level taught
Longevity and growth within the profession
Teacher ability
Classroom environment
Additional Comments
1. Without further covariates on teacher quality, Neag vs. non-Neag seems
like an inappropriate comparison. We may presume that on measures of
academic ability the NEAG graduates are brighter than most teachers
from other institutions. To determine the "NEAG effect" we will have to
control for this. A most simplistic way to do this would be to divide the
non-NEAG group into those teachers from NEAG peer institutions and
others.
2. I have some concerns with measuring/comparing Neag graduates with
non-Neag graduates using CMT scores. Our graduates may, indeed, opt
for more challenging placements, making comparisons of scores difficult
to do. I think that individual students’ academic growth would be a better
comparison than either student performance on CMTs (#1) or group
changes on CMTs (#2). From what I’ve read about value-added
assessment to measure individuals’ growth—in order to account for
changes in population, etc.—it is necessary to have at least three years of
data, information about students’ background, and norm referenced
assessments. Sophisticated statistical modeling and analysis of
longitudinal test score data are necessary.
3. "Performance" on the CMT is generally defined in schools along five
levels: Below basic [1] to Advanced [5]. Studies should be run on the
Scale Score, which has a 100 to 400 range and allows for much more
variation. It is also an interval scale, which helps with tests of significance.
4. CMT4 was first administered last spring (2006) therefore we may not be
able to compare Scale Scores of the same content over the last two years.
Definitely, with the major changes in the test, we can't go back to CMT3 -some of the strands are different and not comparable. The only "anchor"
data would be the Degrees of Reading Power (DRP) which is
independently normed and can, by using the scale DRP Unit score, be
compared over time.
5. If we want to go after Neag vs Non-Neag comparisons, we'll need to hold
constant such variables as ethnicity, SES and school/student types --
SPED, ELL etc. It is also important to note that all ELL kids, regardless of
time in the program, must take the Mathematics sections of the CMT4
which could skew results. In addition, there's a data point in CMT that
allows for "date of entry to the school/district after 10/01/2006". This
allows you to flesh out kids who haven't been at the site for the entire
school year.
6. In addition to student performance, I think it is important to consider a
variety of other indicators. For example:
a. How long they stay in teaching
b. Participation in professional development activities
c. Educational leadership roles in the school, district, community,
state, etc.
d. Participation in professional organizations
e. Continued education
f. Service to other teachers—for example, mentoring pre-service and
new teachers, providing professional development
g. Activities with students outside of the classroom
7. The race/ethnicity n's for the minority students and teachers are going to
be very small.
8. Depending on how much data linkage we can get (statewide vs. districts),
we should decide whether to make a statewide comparison or
district/school-based comparison.