COMPASS ANALYSIS
Executive Summary
A random sample of students was gathered (n=244) to understand what explains student
success in students' initial math courses at ISU. Analysis of the data suggests that passing
remedial and non-remedial courses is most likely influenced by students' scores on the
second COMPASS sub scores and on the number of college preparatory mathematics
courses successfully completed in high school. When non-remedial courses are analyzed
(n=54), SAT math scores and the number of college preparatory courses successfully
completed in high school explain grades in initial ISU math courses; the higher the SAT
math score and the higher number of college preparatory courses successfully completed,
the higher the grades in students' initial math courses. Further analysis indicates that the
latter of these, number of college preparatory courses taken in high school, is the most
significant factor.
Factors Explaining Success in Initial Math Courses at ISU
Background
The University Academic Advisement Committee (UAAC) requested that a study be
conducted so its members could better understand what different factors--SAT math
scores, COMPASS test scores, and high school preparation for collegiate math courses in
particular--explain success in collegiate math courses. UAAC requested that Assessment
and Accreditation, as part of the Division of Academic Affairs, conduct the study.
Methodology
Sample
A random sample of 305 students taking courses in 200505 was taken, of which 244
contained sufficient information for analysis.
Data
A request was submitted to the registrar's office and the Student Affairs Office of the
College of Arts and Sciences to gather numerous amounts of information. The following
information was received:
•
•
•
•
•
•
The students' university 991 number to control for multiple submissions of
information
The initial math course taken by each student
The grade in the students' initial math course, which was then transformed into
the appropriate grade point defined in the university catalog. For most purposes,
this served as the dependent variable for statistical purposes
Year of high school graduation; this was used as a possible control for the
argument that number of years elapsed since high school graduate could be a
factor in initial math grades
Term admitted to ISU
ACT and SAT math and English scores
•
The students' Compass scores (all sub-scores, if applicable, were supplied)
In addition, the high school transcripts for each student were supplied, allowing us to
gather the following information:
• Of high school math courses (pre-algebra, algebra, geometry, algebra 2, precalculus/trigonometry, calculus/AP calculus), how many such courses were taken
Dependent Variables
The subject of study is student success in initial math courses taken at ISU. Because
students take either remedial or non-remedial courses, and because remedial courses in
particular are grades on a bimodal scale, "S" meaning satisfactory and "U" meaning
unsatisfactory, it was necessary to conduct three separate studies:
•
•
•
A study of success in remedial courses, using "S" denoting success, and "U"
denoting lack of success
A study of student grades in non-remedial courses, using grade points as the
measure of those grades
A study of success (passing grades) and non-success (non-passing grades--Ds, Fs,
or W/WF's) in all courses
Independent Variables
The following were used as independent, or explanatory, variables for this analysis:
• Year of high school graduation (to control for time)
• SAT math scores
• COMPASS math placement scores1, in particular the first pre-algebra test and the
second algebra test
• High school college preparatory math courses taken (identified as "1" for course
taken, and "0" for course not taken); by "college-preparatory" we mean prealgebra, algebra 1, algebra 2, geometry, pre-calculus/trigonometry, calculus/AP
calculus.
• Number of college preparatory math courses not failed in the list above
• Number of college preparatory math courses in which students earned grades of C
or higher
• Number of years between high school graduation and initial math course taken
1
The COMPASS placement test is designed by ACT, Inc, “to help accurately place students in college
math courses…it indicates how well [a student] know(s) the background material needed to be successful
in…college level math course(s).”1 It is used by the mathematics department and College of Arts and
Sciences at Indiana State University as an advising tool to assist students in maximizing the chances for
success in initial registration in math courses.
The COMPASS test is divided into four specific tests, the first two of which are of special importance in
respect to assigning students to either remedial or non-remedial mathematics courses. The first test is a prealgebra test. Students who score below 44 on this test are assigned to Math 010, and those who score 44 or
higher on this test are assigned to Math 011. If students are successful enough on the pre-algebra test, ACT
enables students to proceed toward completion of the second test, which tests them on algebra. Students
who score between 1 and 40 on this test are encouraged to take Math 011 (with advisor discretion), and
those who score higher than 40 are encouraged to take non-remedial math.
Analytical Strategy
We use frequency statistics and cross-tabulations to present the data in simple form.
Second, in order to observe what factors explain success in remedial math courses, a
logistic regression was performed that tests for the probability of success, using (1) "S"
grades in remedial courses, or (0) "U" grades in remedial courses as the dependent
variable. We used the above independent variables to accomplish our purpose.
Third, in order to observe what factors explain success in remedial and non-remedial
courses, a logistic regression was also performed, using (1) as passing grades ("S" in
remedial courses and A, B, and C grades in non-remedial courses) and (0) for nonpassing grades ("U" in remedial courses and D, F, W, and WF).
Fourth, for non-remedial courses, we used the grade number assigned to each course (i.e,
4.0=A, 3.5=B+, etc.) as the dependent variable, and the above independent variables, in
tests of association and in a regression analysis.
In what follows, we attempt to answer each of the above research questions.
Analysis
Frequency Statistics
The following table reveals basic frequency statistics for the variables included in this
analysis:
Frequency Statistics
N
Valid
Mean
Missing
Median
Std. Deviation
Minimum
Maximum
Initial Math Course
Grade Number
231
24
2.1450
2.0000
1.45122
.00
4.00
Initial Remedial
Math
244
11
.05
.00
.225
0
1
Years Since HS
Graduation
238
17
1.9328
.0000
5.01682
.00
30.00
238
17
2.7815
1.0000
5.02090
-2.00
31.00
238
17
2001.79
2003.00
5.064
1972
2005
62.95
65.00
18.380
17
98
Years Between
Initial Math Course
and HS Graduation
Year Graduated
From High School
Compass Test 1
97
158
Compass Test 2
70
185
52.09
50.50
21.447
15
93
SAT Math Score
181
74
482.82
470.00
83.880
260
720
Total Number of
Math Courses in HS
164
91
4.45
5.00
.980
0
6
164
91
4.24
4.00
1.187
0
6
Total Number of
Math Courses in HS
D Grades Not
Included
Initial Math Course
Pass
244
11
.65
1.00
.477
0
Table One indicates that 35% of students in the sample fail their initial math course. Very
few students in this sample spent much time out of college between high school
graduation and initial matriculation into college, and students tended to wait one to three
years between their last math courses taken in high school and their initial math course.
Scores on the first COMPASS sub-test were fairly high, although this is no surprise since
only very few students took remedial math. Scores on the second COMPASS math test
were a bit lower. Students took between four and five college preparatory courses in high
school, meaning that the average student completed algebra 2 in high school.2 This
number appears to decline somewhat when D grades exclude students from this
designation. 65% of students taking their initial math course pass.
Table Two below reveals how many students passed and failed each of their respective
initial math courses:
Passing Initial Math Courses
Initial Math Course Pass
Did Not Pass Initial Math Course Passed Initial Math Course
Initial
Math
Course
Total
Total
MA010
0
2
2
MA011
1
10
11
MA102
44
81
125
MA 111
23
36
59
MA 112
2
3
5
MA 115
8
16
24
MA 131
7
11
18
85
159
244
For this specific sample of students, it appears that, with the exception of math remedial
courses, thirty percent or more of students fail their initial math courses.
Remedial Math Courses
Table Three found at the end of this report compares demographic and test score
information of remedial students with non-remedial students:
In respect to background in math, there appear to be no surprises. For students taking
remedial courses, SAT and COMPASS tests tend to be lower, as well as the number of
college-preparatory math courses taken in high school--students taking remedial courses
in their initial math experience tended on average to have completed geometry compared
2
1=pre-algebra, 2=algebra 1, 3=geometry, 4=algebra 2, 5=pre-calculus/trig, 6=calculus/AP calculus
1
to algebra 2 for those taking non-remedial courses as their initial collegiate math course
at ISU.
Table Four uses nonparametric methods to observe what best correlates with placement
in remedial math courses: The higher the SAT math score, the higher the Compass score
on test one (MCP1), the more years between initial math courses and high school
graduation, and the total number of college preparatory courses taken in high school, the
lower the probability that students will initially enroll in remedial courses work (which is
clearly not a surprise).
Explaining Factors Placing Students in Remedial Math Courses
Years Between
Initial Math Course
and HS Graduation
Mann-Whitney U
Wilcoxon W
Total Number of
Total Number Math Courses in
Year Graduated
HS D Grades Not
of Math
From High School Compass Test 1 SAT Math Score Courses in HS
Included
994.000
1462.000
247.500
455.000
526.500
641.000
1085.000
26887.000
325.500
521.000
617.500
732.000
-1.997
-.002
-2.877
-2.852
-2.967
-2.179
.046
.998
.004
.004
.003
.029
Z
Asymp. Sig. (2tailed)
a Grouping Variable: Initial Remedial Math
NOTE: Because only one student in the sample failed his/her initial remedial course, a
study of what explains success cannot be conducted. A prior non-randomized study
performed in 2004, however, indicated that the COMPASS test, particularly the first
subtest, was a good predictor of success in remedial courses.
Success in Initial Math Courses Taken at ISU
A comparison of those who passed or did not pass their initial math courses can be found
in Table Five at the end of this report. The frequency data in the table indicate that there
are differences, especially in respect to the COMPASS tests (first and second subtests)
and the SAT math score.
An initial logistic regression study indicates that there is a high likelihood that the second
sub-test of the COMPASS, and number of courses taken in high school explain the
probability of success in initial math courses. A stricter logistic regression analysis, much
like stepwise regression, indicates that the most significant of these variables is number
of courses taken in high school, especially when courses are counted that exclude D
grades earned in such courses.
Explaining Grades in Non-Remedial Courses
Observing possible correlations between the students' success in their initial math course
at ISU and the SAT math subtest, COMPASS subtests, and high school preparation for
collegiate math courses, is informative because it is easier to determine which are
significantly associated, thereby allowing us to better understand which can be good
predictors.
It is clear from the above table that SAT math scores, total number of math courses, and
the COMPASS 2 test are significantly associated with higher grades in students' initial
math courses at ISU.3 The higher the scores on these indicators, the higher the grade
students' initial math courses.
A stepwise regression analysis enables us to enter these variables in one equation, and
focus only on those variables that are both significantly associated with grades in initial
math courses and that independently explain that association (stepwise regression). A
regressions analysis that includes these variables in such an equation, found at the end of
this report, indicates that number of college preparatory math courses taken in high
school and SAT math scores explain success in non-remedial initial math courses taken
by students at ISU.
Summary
From the above analysis, the following appear to be reasonable statements:
1. When data for students taking remedial courses versus those taking non-remedial
courses are compared, the latter tend to have higher SAT scores, higher
COMPASS sub scores, and a higher number of college preparatory courses in
high school.
2. The random sample used in this study indicates that 30% or more of all students
taking non-remedial courses fail their initial math course at ISU
3. Higher scores on the second COMPASS subtest and number of college
preparatory courses taken in high school suggest a higher probability that students
will succeed in their initial math course taken at ISU (the sample includes
students taking both remedial and non-remedial courses). A more detailed logistic
regression, however, indicates that the most significant of these variables is
number of courses taken in high school, especially when courses are counted that
exclude D grades earned in such courses
4. A regressions analysis that includes SAT math scores, total number of math
courses, and COMPASS 2 test scores , found at the end of this report, indicates
that number of college preparatory math courses taken in high school and SAT
math scores explain success in non-remedial initial math courses taken by
students at ISU.
One concern for these findings is that the model generated by the stepwise regression
analysis indicates that there is a lower probability of students' passing their initial math
courses beyond what can be explained through this analysis. For example, according to
3
Compass test 1 was not significant at 0.082
the regression analysis, a student with a 483 on the SAT math test (the average according
to the frequency statistics) who took precalculus/trigonometry in high school would
receive 2.3 grade points (a C) in math. This may be comforting in respect to supporting
the notion that the analysis is close to the average grade points of all students taking nonremedial math courses initially (see frequency table at beginning of the report), but not so
comforting in that such a grade appears low for a student with a strong pre-collegiate
background.
A number of reasons for this might be subjects of discussion for UAAC, such as:
1. Given that 30% of students in this sample failed their initial math course
(excluding students who took remedial courses as their initial math course), is
there a reason why the failure rate is this high? Or, is this failure rate what is to be
expected?
2. Is the level of high school preparation in college preparatory math courses
adequate?
Group Statistics
Initial Remedial Math
Mean
Std. Deviation
Valid N (listwise)
Unweighted
Initial Math Course Was NonRemedial
Initial Math Course Was
Remedial
Total
Years Since HS Graduation
.0147
.12127
68
68.000
Years Between Initial Math
Course and HS Graduation
.6471
.80604
68
68.000
Compass Test 1
66.0000
16.91683
68
68.000
SAT Math Score
503.6765
72.70682
68
68.000
Total Number of Math Courses
in HS
4.7206
.87836
68
68.000
Total Number of Math Courses
in HS D Grades Not Included
4.5294
1.07162
68
68.000
Years Since HS Graduation
.1000
.31623
10
10.000
Years Between Initial Math
Course and HS Graduation
.9000
.73786
10
10.000
Compass Test 1
49.6000
15.07168
10
10.000
SAT Math Score
422.0000
59.02918
10
10.000
Total Number of Math Courses
in HS
3.7000
.67495
10
10.000
Total Number of Math Courses
in HS D Grades Not Included
3.6000
.84327
10
10.000
Years Since HS Graduation
.0256
.15908
78
78.000
Years Between Initial Math
Course and HS Graduation
.6795
.79762
78
78.000
Compass Test 1
63.8974
17.49329
78
78.000
SAT Math Score
493.2051
75.90995
78
78.000
Total Number of Math Courses
in HS
4.5897
.91787
78
78.000
Total Number of Math Courses
in HS D Grades Not Included
4.4103
1.08635
78
78.000
Weighted
Group Statistics
Initial Math Course Pass
Mean
Std. Deviation
Valid N (listwise)
Unweighted
Did Not Pass Initial Math Course Years Since HS Graduation
.0000
.00000
13
13.000
Years Between Initial Math
Course and HS Graduation
.9231
.49355
13
13.000
Compass Test 1
69.9231
9.05043
13
13.000
SAT Math Score
489.2308
37.74068
13
13.000
Total Number of Math Courses
in HS
4.2308
1.09193
13
13.000
Total Number of Math Courses
in HS D Grades Not Included
3.7692
1.36344
13
13.000
Compass Test 2
Passed Initial Math Course
41.0000
19.23972
13
13.000
Years Since HS Graduation
.0250
.15811
40
40.000
Years Between Initial Math
Course and HS Graduation
.6000
.74421
40
40.000
Compass Test 1
73.8500
11.30135
40
40.000
SAT Math Score
517.2500
68.19984
40
40.000
Total Number of Math Courses
in HS
4.8750
.79057
40
40.000
Total Number of Math Courses
in HS D Grades Not Included
4.8000
.79097
40
40.000
Compass Test 2
Total
52.8750
20.78977
40
40.000
Years Since HS Graduation
.0189
.13736
53
53.000
Years Between Initial Math
Course and HS Graduation
.6792
.70092
53
53.000
Compass Test 1
72.8868
10.84446
53
53.000
SAT Math Score
510.3774
62.97015
53
53.000
Total Number of Math Courses
in HS
4.7170
.90676
53
53.000
Total Number of Math Courses
in HS D Grades Not Included
4.5472
1.04819
53
53.000
49.9623
20.88518
53
53.000
Compass Test 2
Weighted
Correlations
Initial Math Course
Grade Number
Pearson Correlation Initial Math Course Grade
Number
SAT Math Score
1.000
.371
.616
.389
.340
.819
.371
1.000
.215
.510
.117
.231
.616
.215
1.000
.411
.232
.504
Compass Test 2
.389
.510
.411
1.000
.202
.328
.340
.117
.232
.202
1.000
.350
.819
.231
.504
.328
.350
1.000
Initial Math Course Grade
Number
SAT Math Score
.
.003
.000
.002
.006
.000
.003
.
.059
.000
.199
.047
Total Number of Math Courses in
HS D Grades Not Included
.000
.059
.
.001
.046
.000
Compass Test 2
.002
.000
.001
.
.072
.008
.006
.199
.046
.072
.
.005
.000
.047
.000
.008
.005
.
54
54
54
54
54
54
Year Graduated From High
School
Initial Math Course Pass
N
Initial Math
Course Pass
Total Number of Math Courses in
HS D Grades Not Included
Year Graduated From High
School
Initial Math Course Pass
Sig. (1-tailed)
Total Number of
Math Courses in
HS D Grades Not
Year Graduated
SAT Math Score
Included
Compass Test 2 From High School
Initial Math Course Grade
Number
SAT Math Score
54
54
54
54
54
54
Total Number of Math Courses in
HS D Grades Not Included
54
54
54
54
54
54
Compass Test 2
54
54
54
54
54
54
54
54
54
54
54
54
54
54
54
54
54
54
Year Graduated From High
School
Initial Math Course Pass
Model Summary(c)
Change Statistics
Model
R
R Square
Adjusted R Square
Std. Error of
the Estimate
R Square Change
F Change
df1
df2
Sig. F Change
1
.616(a)
.380
.368
1.11531
.380
31.874
1
52
.000
2
.663(b)
.440
.418
1.07058
.060
5.437
1
51
.024
a Predictors: (Constant), Total Number of Math Courses in HS D Grades Not Included
b Predictors: (Constant), Total Number of Math Courses in HS D Grades Not Included, SAT Math Score
c Dependent Variable: Initial Math Course Grade Number
ANOVA(c)
Model
1
Sum of Squares
Regression
Residual
Total
2
df
Mean Square
39.649
1
39.649
64.684
52
1.244
104.333
53
Regression
45.881
2
22.940
Residual
58.453
51
1.146
104.333
53
Total
F
Sig.
31.874
.000(a)
20.015
.000(b)
a Predictors: (Constant), Total Number of Math Courses in HS D Grades Not Included
b Predictors: (Constant), Total Number of Math Courses in HS D Grades Not Included, SAT Math Score
c Dependent Variable: Initial Math Course Grade Number
Durbin-Watson
1.695
Coefficients(a)
Model
1
Unstandardized Coefficients
(Constant)
Total Number of Math
Courses in HS D Grades
Not Included
SAT Math Score
2
(Constant)
Total Number of Math
Courses in HS D Grades
Not Included
SAT Math Score
Standardized
Coefficients
-1.143
.644
.769
.136
-3.643
1.237
.702
.134
.005
.002
-1.776
.082
5.646
.000
-2.944
.005
.563
5.241
.000
.250
2.332
.024
.616
a Dependent Variable: Initial Math Course Grade Number
Excluded Variables(c)
Model
Tolerance
1
2
SAT Math Score
.250(a)
2.332
.024
.310
.954
Compass Test 2
.163(a)
1.376
.175
.189
.831
.208(a)
1.901
.063
.257
.946
Compass Test 2
.047(b)
.357
.722
.050
.644
Year Graduated From High
School
.191(b)
1.812
.076
.248
.941
Year Graduated From High
School
SAT Math Score
a Predictors in the Model: (Constant), Total Number of Math Courses in HS D Grades Not Included
b Predictors in the Model: (Constant), Total Number of Math Courses in HS D Grades Not Included, SAT Math Score
c Dependent Variable: Initial Math Course Grade Number