THE NEW MEAP & MME CUT SCORES
On Track to Career & College Ready
Questions may be submitted via the “chat” feature of Mistreamnet.
We will respond to as many of those questions as we can at the
conclusion of the presentation.
The PowerPoint is available for download in the Materials &
Resources box in the lower right corner of the Mistreamnet event
screen.
Wayne RESA
Bruce R. Fay, PhD | Assessment/Evaluation Consultant
What We Know





New cut scores approved by SBE in Sept. 2011
Major proficiency category cut scores only for
MEAP (not MEAP-Access) and MME only
MI-Access cut scores will not be changed
New cut scores in effect for 2011-12 SY tests
Methodology described in the SBE item
 Did
NOT use standard setting panels
 “Purely” statistical, but they:
 Used
several different methods, and
 Chose the results that made the most sense
What We Know – 2

They basically replicated and validated the 2005
ACT College Readiness Benchmark Study using only:
 Michigan
students (MME scores)
 Michigan post-secondary public institutions (22+15=37)


Defined “success” as a 0.50 probability of earning
a B or better in the corresponding course(s), which
turns out to be a 0.75 probability of a C or better
“Proficient” cut corresponds to this definition
“Advanced” cut is p = 2/3rds,
 “Partially Proficient” cut is p = 1/3rd

What We Know – 3


Linked MME results (11th grade) to “success” in first
semester freshman credit-bearing (non-remedial)
college courses for one (1) cohort of students
Linked MME to grade 3 – 9 MEAP results, such that
 Proficiency
in grade X tends to predict proficiency at
the next tested grade level, and
 Lack of proficiency in grade X tends to predict lack of
proficiency in the next tested grade level
What We Know – 4



Attempted to maximize classification consistency
(reliability of classifications)
As with the earlier ACT study, there will still be error
in these classifications/predictions
Historical re-calculations (retrospective look at data)
available to
 Schools
Oct. 31 (BAA secure site, embargoed)
 Public Nov. 3 (MI SCHOOL DATA website)
What We Don’t Know (yet)




Minor cut scores within proficiency categories
needed to calculate performance level change
How/when these will be determined (it will be done)
How MDE will present this to the press/public when
the results come out for the Fall 2011 MEAP tests
New MEAP-Access cut scores (these may/will change as
part of standard-setting this winter)

…?
What Does It Mean To Be
Career & College Ready?
A Short Quiz…
Which student is “College-Ready” According to ACT?
To read the chart: Amy’s ACT English score was 20; her math score was 22;
her reading score was 21; her science score was 24; and her ACT Composite was 22.
ACT English
ACT Mathematics
ACT Reading
ACT Science
ACT Composite
36
34
30
31
28
28
26
24
25
ACT Scale Score
22
22
19
16
13
10
7
4
1
Amy (2*)
These are real students' ACT profiles.
Bob (37*)
Cal (73*)
Deb (94*)
Student Name (fictitious)
* Percent of students with same composite score who met all 4 ACT College-Ready Benchmarks.
Eve (99*)
What you have to know to figure it out…
ACT Scale Score Cut Score
National Percentile Rank
Michigan Average
Class of 2010 National Average
Class of 2010 MI Average
ACT
ACT
ACT
ACT
ACT
English Mathematics Reading Science Composite
18
22
21
24 Not Used
37
63
53
80
18.3
20.5
18.9
19.3
21
19.7
19.4
21.3
19.7
19.7
20.9
19.9
19.3
21
19.7
(Class of 2010 is primarily Spring of 2009 results plus retak es.)
Michigan is one of a handful of states where 100% of students take the ACT.
Yes, Amy, the student with the lowest ACT Composite, is the only student who ACT
considers “College-Ready.”
Eve, with her ACT Composite score of 30, is not considered “College-Ready.”
* About 37% of students with an ACT Composite score of 24 are “College-Ready.”
Which College Courses Were Used To Establish
Michigan’s New Cut Scores?




Mathematics – College Algebra
Reading – Courses identified by 4-year universities;
Reading-heavy courses such as entry-level literature,
history, philosophy, or psychology for 2-year universities
Science – Courses identified by 4-year universities;
Entry level biology, chemistry, physics, or geology for 2year universities
Social Studies – Courses identified by 4-year
universities; Entry level history, geography, or economics
for 2-year universities
Which Students Were Used?

For the HS–College link, only students who:
 Had
valid MME scores (spring 2009)
 Attended one of 37 MI post-secondary public schools
 Enrolled in one or more of the identified courses as first
semester freshman (2010-11 SY)


Student taking more advanced mathematics courses,
such as calculus for example, were NOT included in
the study calculations.
If they had been, the cut scores would have been
even higher
Another View of “College Ready” (ACT CCR)
221,606 ways that ACT
subject area subscores can
be combined to produce
ACT composite scores
153,776 NOT ACT CCR
67,830 are ACT CCR
An ACT composite score of
28 is about equally likely to
be considered CCR or not
CO 2010*
English
Math
Reading
Science
Composite
National
20.5
21.0
21.3
20.9
21.0
Michigan
18.9
19.7
19.7
19.9
19.7
*Class of 2010 is primarily spring 2009 + retakes
National and Michigan
average composite score for
Class of 2010 below left
(ACT does not use composite
scores in determining if a student
is “college ready”)
Actual Data
3559 students
- 1434 ARE CCR
- 2125 NOT CCR
Less than 50% CCR at
composite score of 24
More than 50% CCR at
composite score of 25
CO 2010*
English
Math
Reading
Science
Composite
National
20.5
21.0
21.3
20.9
21.0
Michigan
18.9
19.7
19.7
19.9
19.7
The New Cut Scores
Mathematics Cut Scores
Assessment
Grade
Partially
Proficient
Proficient*
Advanced
MME
11
1093
1116
1138
MEAP
8
809
830
865
MEAP
7
714
731
776
MEAP
6
614
629
675
MEAP
5
516
531
584
MEAP
4
423
434
470
MEAP
3
322
336
371
* Old proficient cut score was grade level x 100
Reading Cut Scores
Assessment
Grade
Partially
Proficient
Proficient*
Advanced
MME
11
1081
1108
1141
MEAP
8
796
818
853
MEAP
7
698
721
760
MEAP
6
602
619
653
MEAP
5
501
521
565
MEAP
4
395
419
478
MEAP
3
301
324
634
* Old proficient cut score was grade level x 100
Science Cut Scores
Assessment
Grade
Partially
Proficient
Proficient*
Advanced
MME
11
1106
1126
1144
MEAP
8
826
845
863
MEAP
5
526
553
567
* Old proficient cut score was grade level x 100
Social Studies Cut Scores
Assessment
Grade
Partially
Proficient
Proficient*
Advanced
MME
11
1097
1129
1158
MEAP
9
899
928
960
MEAP
6
593
625
649
* Old proficient cut score was grade level x 100
What About Writing?


Since the writing tests are relatively new, the cut
scores were already set at a career and college
ready level, and did not need to be reset.
Proficient remains at Grade Level * 100
Approximate % correct needed to be proficient
on MME / MEAP – Old vs. New
Test
Grade
Math
Old
Math
New
Rdg
Old
Rdg
New
Sci
Old
Sci
New
SS
Old
SS
New
MME
11
42
58
51
57
46
65
39
63
MEAP
9
n/a
n/a
n/a
n/a
n/a
n/a
39
64
MEAP
8
35
59
45
64
40
75
n/a
n/a
MEAP
7
32
60
52
70
n/a
n/a
n/a
n/a
MEAP
6
35
58
42
61
n/a
n/a
43
66
MEAP
5
39
61
45
64
48
83
n/a
n/a
MEAP
4
29
58
45
64
n/a
n/a
n/a
n/a
MEAP
3
34
72
45
67
n/a
n/a
n/a
n/a
What Does This Mean?

On MME, the following probabilities of earning a B or
better in the associated college class(es):
“Advanced” ~= at least 2/3
 “Proficient” ~= 1/2 to 2/3
 “Partially Proficient” ~< 1/3



On MEAP, the degree to which a student is likely “on
track” to proficiency in the next tested grade in that
subject.
Remember…the tests themselves have not changed; the
distribution of scale scores for the 2011-12 test cycle
will be similar to the 2010-11 test cycle.
New Cut Scores Applied To
Old (2010-2011) Test Results
State of Michigan Retrospective Proficiency Rates
MEAP & MME 2010-11 Old & New Cut Scores
Michigan Old
100
Michigan New
95
91
90
84
87
85
80
84
84
82
79
78
80
78
78
78
75
70
63
Percent Proficient
85
64
66
64
60
63
56
74
61
57
54
52
50
42
40
35
37
39
36
36
33
29
30
28
26
28
17
20
14
10
0
3rd
4th
5th
6th
Math
7th
8th
11th
3rd
4th
5th
6th
Reading
7th
8th
11th
5th
8th
11th
Science
6th
9th
11th
Social Studies
Implications?




Proficiency rates going forward will drop
Schools / students should still be able to see relative
changes in performance (due to historical recalculation)
AYP may not be an issue if MDE is successful in
obtaining the ESEA Flexibility (NCLB waivers) from
USED (a whole separate issue)
Accreditation will not be effected (much) if/when
MSA is implemented due to the use of ranking
calculations that mostly do not depend on proficiency
(cut scores)
Final Thoughts – 1



The tests have not changed
The distribution of scale scores for 2011-12 will be
similar to the distribution for 2010-11
Cut Scores are not designed in to the tests; they are:
 Added
later by a separate standard-setting process
 After the test has been administered
 Represent a judgment to as to what level of
performance is considered “good enough” to meet
some definition

There is no a priori “correct” cut score
Final Thoughts – 2


Schools will continue to use state test results, in
conjunction with other information, to monitor and
evaluate the academic achievement of their
students and examine the effectiveness of curriculum
and instruction.
Parents will continue to see reports that show the
number of items their student answered correctly
and the associated scale score.
Final Thoughts – 3



If we were misleading students and parents about
college readiness before, the new cut scores do not
necessarily fix that – and we may be more likely to
suggest to a student that they are not ready when
they have a real chance at success
“College readiness”, for each student, depends on the
unique interaction of their choices of college and
program of study with their personal attributes, such
as maturity, work habits, financial support, and
motivation to succeed
(see David T. Conley, PhD, Professor, Univ of Oregon, for research in this area)
Final Thoughts – 4




For some students, Partially Proficient – High may be
adequate
For others, Advanced – Mid, or even Advanced –
High may be a more appropriate goal
In spite of MDE’s statistical methodology, we don’t
know how predictive these new cut scores will be of
college success going forward – only time & data will
tell
Being “career & college ready” is a complex issue
that can not be captured in a few test scores
Questions?
Bruce R. Fay, PhD
Assessment / Evaluation Consultant
FayB@resa.net
734.334.1384