Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Collegiate Assessment of Academic Proficiency:
Mathematics Test
Spring 2012 Administration
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 1 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Table of Contents
Executive Summary....................................................................................................................................... 3
Introduction .................................................................................................................................................. 4
Sample....................................................................................................................................................... 5
Results ........................................................................................................................................................... 7
CAAP Scores Overall .................................................................................................................................. 7
CAAP Scores by Student Classification, Sex, Effort, and GPA ................................................................... 8
Conclusion ................................................................................................................................................... 10
Appendix ..................................................................................................................................................... 11
Appendix A: Information about the CAAP Mathematics Test ................................................................ 11
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 2 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Executive Summary
The Collegiate Assessment of Academic Proficiency (CAAP) Mathematics Test is designed to
measure skills in mathematics. In coordination with the various Texas Tech University (TTU)
colleges, the Office of Planning and Assessment administered the Mathematics Test in select
courses across campus at the beginning of the spring 2012 semester. Courses were chosen
based on their enrollment of juniors and seniors to achieve a representative number from each
college.
A total of 487 Texas Tech University (TTU) students participated in the spring 2012
administration of the CAAP Mathematics Test. The answers of 400 participants were sent to
ACT for scoring. A final sample of 399 participants (177 juniors and 222 seniors) is included in
this analysis. The final sample seems to be a fairly good representation of the population of all
TTU juniors and seniors in terms of sex and ethnicity. The College of Agricultural Sciences, the
Rawls College of Business Administration, and the College of Human Sciences are
overrepresented and the other colleges are somewhat underrepresented in the final sample.
This should be considered in interpreting the results.
From the 2012 administration of the CAAP Mathematics Test it appears that TTU juniors (mean
= 58.5, SD = 3.8) score similarly on average to the national sample of juniors (mean = 58.1, SD =
4.3) and that the TTU seniors (mean = 59.1, SD = 4.2) score higher on average than the national
sample of seniors (mean = 56.6, SD = 4.0). This difference in scores for the seniors is statistically
significant (t = 9.20, p < .0001).
The CAAP Mathematics Test includes two subscales: Basic Algebra (17 items) and College
Algebra (18 items). In terms of Basic Algebra scores, the TTU juniors (mean = 15.8, SD = 2.3)
score slightly higher on average than the national sample of juniors (mean = 15.3, SD = 2.5).
This difference in Basic Algebra scores for the juniors is statistically significant (t = 2.65, p =
0.0082). In terms of College Algebra scores, the TTU juniors (mean = 15.1, SD = 2.7) score
similarly to the national sample of juniors (mean = 15.2, SD = 2.8). Comparisons of the seniors’
subscale scores both follow the same pattern as the overall Mathematics Test scores. On
average the TTU seniors score higher than the national sample in Basic Algebra scores (TTU
mean = 16.1, SD = 2.4; National mean = 14.5, SD = 2.5) and in College Algebra scores (TTU mean
= 15.5, SD = 3.0; National mean = 14.3, SD = 2.4).
For the TTU sample, a regression model was used to see if the variables student classification,
sex, GPA, and self-reported effort are significant predictors of a participant’s CAAP
Mathematics Test score. The results suggest that, while controlling for the other variables, sex
(t = -4.06, p < .0001), GPA (t = 5.25, p < .0001), and self-reported effort (t = -3.48, p = .0006; t =
2.25, p = 0.0253) are significant predictors and that student classification is not a significant
predictor (t = 0.84, p = 0.4017). The results suggest that on average male students, students
with higher GPA’s, and students reporting that they gave more effort score higher on the CAAP
Mathematics Test if all other variables are equal.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 3 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Introduction
The Collegiate Assessment of Academic Proficiency (CAAP) “is the standardized, nationally
normed assessment program from ACT that enables postsecondary institutions to assess,
evaluate, and enhance student learning outcomes and general education program outcomes.” 1
The CAAP offers six different modules: Reading, Writing Skills, Writing Essay, Mathematics,
Science, and Critical Thinking. 2 As per decision of the University Assessment Committee in fall
2010, the Office of Planning and Assessment administered the Mathematics Test of the CAAP
during the spring 2012 semester. The Mathematics Test “is a 35-item, 40-minute test designed
to measure students' proficiency in mathematical reasoning. The test assesses students’
proficiency in solving mathematical problems encountered in many postsecondary curricula. It
emphasizes quantitative reasoning rather than the memorization of formulas.” 3 Please find
more information on how the test is constructed and scored in Appendix A.
In coordination with the various colleges, the Office of Planning and Assessment administered
the Mathematics Test in select courses across campus. Courses were chosen based on their
enrollment by student classification and size. Juniors and seniors were the target group, so
courses with high junior and senior enrollment were selected to participate. Larger courses
were chosen from colleges with higher junior and senior enrollment and smaller courses were
chosen from colleges with lower junior and senior enrollment. Included in the sample were
courses from the following colleges: College of Agricultural Sciences and Natural Resources
(AG), College of Architecture (AR), College of Arts and Sciences (AS), College of Education (ED),
College of Human Sciences (HS), College of Mass Communications (MC), College of Visual and
Performing Arts (VP), Edward E. Whitacre Jr. College of Engineering (EN), Honors College (HR),
and Jerry S. Rawls College of Business Administration (BA). The CAAP was administered during
the regularly scheduled class time. This allowed most students from each class to participate
and helped reduce the bias that may have been created if students had been asked to
volunteer to take the survey on their own time.
A total of 487 Texas Tech University (TTU) students participated in the spring 2012
administration of the Mathematics Test. As juniors and seniors were the target group, 46
students from other student classifications were excluded. Of the remaining 441 participants,
41 students were randomly selected from the colleges that were overrepresented in the
sample to be excluded. The remaining 400 answer sheets were sent to ACT to be scored. ACT
returned scores for 399 students (1 of the selected answer sheets was completed in a manner
that it could not be scored). The final sample includes 177 juniors and 222 seniors.
1
http://www.act.org/caap/, accessed 3/21/2012
http://www.act.org/caap/about/modules.html, accessed 3/21/2012
3
http://www.act.org/caap/test/math.html, accessed 3/21/2012
2
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 4 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Sample
ACT provided scores for the 399 answer sheets of students who participated in the spring 2012
administration of the CAAP. Since the students’ R numbers were used as the student ID on the
Mathematics Test, the Office of Planning and Assessment was able to obtain demographic
information for the participants from Institutional Research (IR). The demographic information
obtained includes sex, ethnicity, student classification, and college. The following graphs
compare the final samples of juniors (n = 177) and seniors (n = 222) to the population of all TTU
juniors and seniors by sex, ethnicity, and college.
Sample and Population by Sex:
Juniors
Sample and Population by Sex:
Seniors
80%
80%
60%
60%
Sample
40%
Population
20%
Sample
40%
Population
20%
0%
0%
Female
Male
Female
Male
The samples of juniors and seniors both appear to be good representations of their respective
populations in terms of sex.
Sample and Population by Ethnicity: Juniors
100%
80%
60%
Sample
40%
Population
20%
0%
AAM
AI
AS
B
HI
M
NH/PI
NR
U
WH
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 5 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Sample and Population by Ethnicity: Seniors
100%
80%
60%
Sample
40%
Population
20%
0%
AAM
AI
AS
B
HI
M
NH/PI
NR
U
WH
The samples of juniors and seniors both appear to be good representations of their respective
populations in terms of ethnicity.
Sample and Population by College: Juniors
35%
30%
25%
20%
Sample
15%
Population
10%
5%
0%
AG
AR
AS
BA
ED
EN
HR
HS
MC
UC
UN
VP
Sample and Population by College: Seniors
35%
30%
25%
20%
Sample
15%
Population
10%
5%
0%
AG
AR
AS
BA
ED
EN
HR
HS
MC
UC
UN
VP
It appears that for both the juniors and the seniors the College of Agricultural Sciences, the
Rawls College of Business Administration, and the College of Human Sciences are
overrepresented and that the other Colleges are underrepresented in the sample. This should
be considered in interpreting the results.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 6 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Results
CAAP Scores Overall
The following table and graph give a summary of the juniors’ and seniors’ CAAP scores.
Summary of CAAP Scores by Student Classification
n
177
222
Junior
Senior
Mean
58.5
59.1
SD
3.8
4.2
Min
48
48
Median
59
59
Max
72
72
Histogram of CAAP Scores by Student Classification
60%
50%
40%
30%
Juniors
20%
Seniors
10%
0%
45
50
55
60
65
70
75
80
The TTU juniors (mean = 58.5, SD = 3.8) and seniors (mean = 59.1, SD = 4.2) performed similarly
on the CAAP. The ACT website has a summary of the scores from 13,392 juniors from 117
institutions and 9,867 seniors from 120 institutions that participated in the Mathematics Test in
fall 2011.4 On average the TTU juniors scored similarly to the national sample of juniors (mean
= 58.1, SD = 4.3). 5 This difference for the juniors is not statistically significant (t = 1.23, p =
0.2183). On average the TTU seniors scored higher than the national sample of seniors (mean =
56.6, SD = 4.0). 6 This difference for the seniors is statistically significant (t = 9.20, p < .0001).
4
http://act.org/caap/norms/, accessed 3/21/2012
http://act.org/caap/norms/pdf/11Table12.pdf, accessed 3/21/2012
6
http://act.org/caap/norms/pdf/11Table13.pdf, accessed 3/21/2012
5
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 7 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
The CAAP Mathematics Test includes two subscales: Basic Algebra (17 items) and College
Algebra (18 items). In terms of Basic Algebra scores, the TTU juniors (mean = 15.8, SD = 2.3)
score slightly higher on average than the national sample of juniors (mean = 15.3, SD = 2.5).
This difference in Basic Algebra scores for the juniors is statistically significant (t = 2.65, p =
0.0082). This pattern in Basic Algebra scores is different than the pattern found for the juniors’
overall Mathematics Test scores. In terms of College Algebra scores, the TTU juniors (mean =
15.1, SD = 2.7) score similarly to the national sample of juniors (mean = 15.2, SD = 2.8), and this
difference is not statistically significant (t = -0.47, p = 0.6368), as with the overall Mathematics
Test scores. Comparisons of the seniors’ subscale scores follow the same pattern as the overall
Mathematics Test scores. In terms of Basic Algebra scores, the TTU seniors (mean = 16.1, SD =
2.4) score higher on average than the national sample of seniors (mean = 14.5, SD = 2.5), and
this difference is statistically significant (t = 9.44, p < .0001). In terms of College Algebra scores,
the TTU seniors (mean = 15.5, SD = 3.0) score higher on average than the national sample of
seniors (mean = 14.3, SD = 2.4), and this difference is statistically significant (t = 7.32, p < .0001).
CAAP Scores by Student Classification, Sex, Effort, and GPA
The following table gives a summary of the overall sample by student classification, sex,
participants’ self-report of effort, and GPA. These variables will be included in a regression
model for predicting the CAAP Mathematics Test score.
Summary of Predictor Variables for CAAP Scores
Student Classification
n
Junior
177
Senior
222
Sex
n
Female
166
Male
233
Self-Reported Effort
n
Tried my best
171
Gave moderate effort
142
Gave little effort
24
Gave no effort
1
n
Mean
GPA
398
3.08
%
44.4
55.6
%
41.6
58.4
%
50.6
42.0
7.1
0.3
SD
0.52
The following table gives a summary of a regression model for predicting a participant’s CAAP
Mathematics Test score. For the variable “Self-Reported Effort”, the “Gave little effort” group
and the “Gave no effort” group were combined in the model because of the small number of
participants in the “Gave no effort” group. For this variable, the “Gave moderate effort” group
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 8 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
was used as the reference group for the “Tried my best” group and the “Gave little or no effort
group”.
Linear Regression Model for Predicting CAAP Mathematics Test Scores
Variable
Intercept
Student Classification
Sex
GPA
Gave little or no effort
Tried my best
df
1
1
1
1
1
1
Parameter Standard
Estimate
Error
52.82
1.19
0.34
0.40
-1.67
0.41
1.98
0.38
-2.75
0.79
0.94
0.42
t value
44.57
0.84
-4.06
5.25
-3.48
2.25
p value
< .0001
0.4017
< .0001
< .0001
0.0006
0.0253
95% Confidence Interval
Lower Limit Upper Limit
50.49
55.15
-0.45
1.12
-2.48
-0.86
1.24
2.72
-4.30
-1.19
0.12
1.75
Overall, the model was found to be statistically significant (F = 12.70, p < .0001, R2 = 0.16).
Within the model, the significant predictors of CAAP score were sex (t = -4.06, p < .0001), GPA (t
= 5.25, p < .0001), gave little or no effort (t = -3.48, p = 0.0006), and tried my best (t = 2.25, p =
0.0253). Specifically, this model suggests that on average male students score higher than
female students, students with higher GPA’s score higher, students reporting that they gave
little or no effort score lower than students reporting that they gave moderate effort, and
students reporting that they tried their best score higher than students reporting that they gave
moderate effort. Student classification was not a significant predictor of CAAP scores (t = 0.84,
p = 0.4017). The following graphs show the average CAAP scores by the significant predictor
variables from the model: sex, GPA, and self-reported effort.
Average CAAP Scores by Sex
CAAP Scores by GPA
75
75
70
70
65
60
65
58.2
59.3
60
55
55
50
50
45
45
Female
Male
1.50
2.00
2.50
3.00
3.50
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 9 of 12
4.00
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Average CAAP Scores by Self-Reported Effort
75
70
65
60
59.4
58.3
Tried my best
Gave moderate
effort
55
55.6
50
45
Gave little or no
effort
Conclusion
Overall it appears that TTU juniors score similarly on average to other juniors nationally and
that TTU seniors score higher on average than other seniors nationally on the CAAP
Mathematics Test. When considering the subscale scores, TTU juniors score higher on average
than other juniors nationally in terms of Basic Algebra scores, but they score similarly on
average to other juniors nationally in terms of College Algebra scores. As with the overall
scores, the TTU seniors score higher on average than other seniors nationally in terms of both
Basic Algebra and College Algebra scores. For the TTU sample it appears that on average male
students, students with higher GPA’s, and students reporting that they gave more effort score
higher, all else being equal, on the CAAP Mathematics Test. Student classification was not a
significant predictor of CAAP Mathematics Test score.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 10 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Appendix
Appendix A: Information about the CAAP Mathematics Test 7
7
http://www.act.org/caap/test/math.html, accessed 3/21/2012
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 11 of 12
Collegiate Assessment of Academic Proficiency: Mathematics Test
Spring 2012
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2012
Page 12 of 12