NWEA MAP Measures, Metrics
and Cut-Points for Alternative
Education Campuses
Colorado Department of Education
July 10, 2012
1
Agenda
• Purpose
• Selection of Measures Process & Considerations
• Metrics and Cut-Points for NWEA MAP Achievement
Results
• Metrics and Cut-Points for NWEA MAP Growth Results
• Calculating NWEA MAP Growth Results
• Timeline
2
Purpose
• Explain methodologies for calculating and
reporting NWEA MAP assessment results for
accountability purposes on the AEC SPF report
(metrics and cut-points).
• Answer questions related to NWEA MAP
assessment results for accountability purposes on
the AEC SPF report.
3
Selection of Measures Process
• AECs submitted the Selection of Measures Form July 2.
– http://www.cde.state.co.us/Accountability/Downloads/AECSel
ectionOfAccountabilityMeasures.xlsx
• There are a set of state required AEC measures:
–
–
–
–
Achievement: CSAP/TCAP % proficient/advanced
Growth: CSAP/TCAP Median Growth Percentiles
Postsecondary: Completion rate, Dropout rate, ACT composite
Engagement: Average daily attendance, Truancy rate
• Districts may select additional measures, subject to CDE
approval.
4
Considerations for Measures
• The three most recent years of data will be included.
• AECs will receive indicator ratings similar to the
traditional SPF:
– Does not meet, approaching, meets, or exceeds AEC norms
• Each indicator/sub-indicator:
– Uses adjusted AEC cut-points relative to each other
– Typically uses either normative cuts of 40th/60th/90th percentile
or head counts of 40%/60%/90%
• Where there are common measures across districts,
common metrics and cut-points will be used.
5
Considerations for Overall Results
• Based on the indicator ratings, AECs receive an overall
point total and plan type.
• AECs engage in the same UIP process as all other schools,
and have the same consequences for performance.
– Note: There is a modified UIP template for AECs to reflect the
modified performance indicators.
6
Metrics and Cut-Points for MAP Academic
Achievement (Status) Results
Of students continuously enrolled for at least 8 weeks
Metrics
% of students scoring at grade
level on spring assessment
% of students increasing at
least one grade level between
pre- and post-test
administrations
7
Cut-Points
•
•
•
•
Exceeds – at/above 90%
Meets – below 90% but at/above 60%
Approaching – below 60% but at/above 40%
Does Not Meet – below 40%
Metrics and Cut-Points for NWEA MAP
Academic Growth Results
% of students achieving target growth from one test administration to the next
Metrics
Cut-Points
Targets based on seasonal
assessment windows:
• Fall to Winter Growth
• Winter to Spring Growth
• Fall to Spring Growth
•
Exceeds – at/above 90%
•
Meets – below 90% but at/above 60%
•
Approaching – below 60% but at/above 40%
•
Does Not Meet – below 40%
Targets based on 8-week
enrollment intervals:
• 8 weeks
• 16 weeks
• 24 weeks, etc.
8
Calculating NWEA MAP
Growth Results
• The following slides have been adapted from documents
written by Dr. Jody Ernst:
– NWEA Use Guidance (07.05.12)
– NWEA AEC Research Findings to Inform How to Best Articulate
the Assessments Use for Purposes of AEC Accountability
– http://www.cde.state.co.us/cdedocs/OPP/NWEAAECWriteUp.p
df
9
Calculating NWEA MAP Growth
Results
• The results and norms from these studies are all based
on Colorado AEC MAP data from 2008-2010
• Insufficient data were available to calculate Colorado AEC
growth norms for Science, but Reading, Mathematics and
Language Usage norms have been published
10
NWEA MAP Characteristics
• Grades K-11 (or students with grade level
equivalents of K-11)
• Reading, Mathematics, Language Usage, Science*
• Fall, Winter, Spring assessment windows
• Student-level results are RIT scores
* As noted previously, Science growth targets are not available
11
Pulling NWEA MAP Data
• Export NWEA data for all subjects in which most
students in a grade take the assessment
• Include 3 years of data or as many years as are
available if your school only recently started
giving NWEA MAP
12
Formatting NWEA MAP Data
• For each student, compute the difference, in
weeks, between each testing occasion (i.e., fall to
winter, winter to spring, and/or fall to spring)
• Only results based on assessments that were at
least eight weeks apart may be used.
13
Calculating each Student’s Pre-test
Grade Equivalent
• Using the Fall/Winter/Spring RIT Ranges and
Fall/Winter/Spring RIT Grade Equivalents,
compute the grade level equivalent for each
student’s pre-test RIT score
• The following tables show the RIT ranges and
corresponding grade equivalents for the Fall MAP
administration
14
For example, if a student
receives a RIT score of 219,
they have a Reading level
equivalent to that of an 8th
grader (regardless of their
assigned grade). This value
is used as their pre-test
grade equivalent.
15
16
17
Calculating RIT Growth
• Calculate the observed amount of pre-test to
post-test growth for each student according to the
following:
– Fall to Winter = Winter RIT – Fall RIT
– Winter to Spring = Spring RIT – Winter RIT
– Fall to Spring = Spring RIT – Fall RIT
18
Calculating Target RIT Growth using
Seasonal Assessment Windows
• If most students are consistently enrolled in your AEC and
take the MAP assessment at seasonal intervals, the easiest
option for finding a student’s growth target is to use the
tables previously published by Jody Ernst.
• For each student, use the assessments that give the
longest window between pre- and post-test
administrations.
19
Calculating Target RIT Growth using
Seasonal Assessment Windows
• Using a student’s pre-test grade equivalent and the
duration between pre- and post-test administrations,
determine the appropriate growth target for that
student
• The following tables show the RIT growth targets for
each combination of starting grade equivalent and test
window.
20
For example, the
student at an 8th
grade equivalent for
Fall (RIT score of
219), is expected to
increase their Spring
RIT score by 3 points
(to a RIT of 222). For
AEC reporting
purposes, 3 RIT
points is this
student’s target
growth.
21
22
23
Calculating Target RIT Growth using 8week Enrollment Windows
• If students enroll in your AEC throughout the year and
for radically differing lengths of time and take the MAP
assessment as a pre- and post-test at irregular intervals,
consider using growth targets based on 8-week
enrollment windows.
• For each student, use the assessments that give the
longest window between pre- and post-test
administrations.
24
Calculating Target RIT Growth using 8week Enrollment Windows
• CDE is currently working to calculate growth
targets for by pre-test grade equivalent for 8 week
intervals.
• These results will be available within the next
week or two and will enable AEC’s opting into
NWEA and the 8-week interval method to complete
their Selection of Measures forms.
25
Determine if Target Growth Achieved
• Determine whether each student met their growth
target by comparing the actual growth obtained to their
target growth.
• If a student’s observed growth was greater than or equal
to their target growth, the student MET their target.
• If a student’s observed growth was less than their target
growth, the student DID NOT MEET their target.
26
Aggregating at the AEC Level
• Compute the percentage of students meeting their
target growth
Number of students that met their target growth
(being sure to use only one growth result per
student)
Total number of students with pre- and post-test
records (using one record per student)
27
Completing Selection of Measures Form
Exceeds AEC norms
at or above 90 percent of students
Meets AEC norms
below 90 percent but at or above 60 percent of students
Approaching AEC norms
below 60 percent but at or above 40 percent of students
Does Not Meet AEC norms
Use measure?
OPTIONAL
YES
To receive the indicator rating, the percent of students achieving their target growth was:
below 40 percent of students
NWEA MAPS Cut-point values for percent of students achieving target growth were:
Reading
Cut-point value for Exceeds
90.0
Cut-point value for Meets
60.0
Cut-point value for Approaching
40.0
Your school's actual percent of students achieving target growth was:
Percent achieving target
Total student count (if N ≥ 16)
66.7
27
• Fill in your AEC’s aggregated percentage of students meeting
their target growth
• Fill in your AEC’s total number of students with pre- and posttest records (denominator)
28
Assigning a Growth Rating
• CDE will assign a rating to your AEC’s reported
NWEA MAP growth
• This calculation is done separately for each of the
three subject areas by comparing the percentage of
students meeting their target growth against the
following state cut-points:
Cut-Points
•
•
•
•
29
Exceeds – at/above 90%
Meets – below 90% but at/above 60%
Approaching – below 60% but at/above 40%
Does Not Meet – below 40%
Process/Timeline
May
• CDE provides updated AEC
Application/Renewal form
and Selection of Measures
form.
July 2
• For schools seeking AEC
designation, districts/authorizer
submits (1) Application/Renewal
form and (2) Selection of
Measures form, with
supplemental measures,
metrics, cut-points and data, or
district proposed framework.
July 25:
NWEA Results to CDE
October 5
October 15
Aug. 15/Sept. 4
• SBE approves schools for AEC
designation. (Aug.)
• CDE release preliminary traditional
SPF. (Aug. 15)
• CDE reviews proposed AEC
selected measures or district
proposed framework, and
approves supplemental measures ,
metrics, and cut-points or
framework. (Sept. 4)
Nov. 15 / Dec. 17
• CDE generates AEC
• District submits final
• CDE makes final
framework report with state
and submitted supplemental
data.
• CDE provides initial plan type
assignment based on the AEC
framework.
accreditation category for
AEC based on framework.
• District submits any requests
to reconsider plan type
assignments.
recommendation to the State
Board re: school plan type
assignments. (Nov. 15)
• State Board finalizes school
plan type assignments. (Dec 17)
• CDE publishes SPFs to
SchoolView.org. (Dec. 17)
30
Resources
• Alyssa Pearson, Executive Director
– pearson_a@cde.state.co.us , 303-866-6855
• Marie Huchton
– huchton_m@cde.state.co.us , 303-866-6203
• Donna Morganstern
– morganstern_d@cde.state.co.us , 303-866-6209
• Somoh Supharukchinda
– supharukchinda_s@cde.state.co.us , 303-866-6778
• http://www.cde.state.co.us/Accountability/StateAccountabilityAE
Cs.asp
31