Online Senior Assessment 2010: Multicultural
Introduction
The second component of the OSA contains questions for the Multicultural core. The
first questions ask the participants about how they fulfilled their Multicultural core
requirement.
The following table shows the number and percentage of participants who selected
each response to the first question regarding where students took their core curriculum
course. The number of participants selecting each response adds up to more than the
755 total participants because those who did not select “I took my core curriculum class
at Tech” could select more than one of the other responses.
How did you complete your multicultural requirement?
% of all
% of all
Response
N
Responses Participants
I took a multicultural class through dual credit in
20
2.4%
2.6%
high school.
I took an advanced placement multicultural class
12
1.5%
1.6%
in high school.
1
0.1%
0.1%
I took a CLEP exam for multicultural credit.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 1 of 19
Online Senior Assessment 2010: Multicultural
How did you complete your multicultural requirement? (Cont.)
I received transfer multicultural credit for a class
172
20.8%
that I took at another institution.
558
67.6%
I took my core curriculum class at Tech.
I received multicultural credit for having done a
study abroad course (either through Texas Tech
63
7.6%
University or elsewhere).
826
100.0%
Total Responses
22.8%
73.9%
8.3%
For the analysis in this report the 755 participants will be divided into the “TTU” group
and the “ELSE” group. The TTU group will represent the 558 participants (i.e., 73.9%)
who selected “I took my core curriculum class at Tech” and the ELSE group will
represent the 197 participants (i.e., 26.1%) who selected one or more of the other
responses indicating that they took their multicultural core curriculum class elsewhere.
The following pie chart shows this division of the sample.
Multicultural
197 (26.1%)
TTU
ELSE
558 (73.9%)
The 197 participants in the ELSE group were also asked if the class they took outside of
Tech counted for their core curriculum credit. Of the 197 participants who reported
taking a Multicultural course elsewhere, 23 (i.e., 11.7%) reported that they did not know
if the course counted for their Multicultural core curriculum credit and 174 (i.e., 88.3%)
reported that the course did count for their Multicultural core curriculum credit. The 174
participants who reported the course taken outside of Tech did count for their
Multicultural core curriculum credit were also asked which one counted. The following
table shows the number and percentage of the 174 participants who selected each
response.
Which one?
Response
A dual credit class.
An advanced placement class.
A CLEP exam.
A class I took at another institution.
I don't know.
Total
N
%
11
6.3%
10
5.7%
1
0.6%
147
84.5%
5
2.9%
174 100.0%
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 2 of 19
Online Senior Assessment 2010: Multicultural
Results
The student learning outcomes for the Multicultural core are:
Demonstrate awareness and knowledge of cultural differences within one or
more distinctive sub-cultures of the United States, or
Demonstrate awareness and knowledge of cultural differences within one or
more global societies (Outside the U.S).
Both student learning outcomes apply to all 7 questions on the OSA. They are all
knowledge-type questions where one correct answer exists. A screenshot of the
questions is shown below. For analysis purposes, the answers were coded from 1 to 4
in the order they appear on the actual instrument. Attachment C shows how many
times each answer choice was selected by the different participants for all of the
Multicultural questions.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 3 of 19
Online Senior Assessment 2010: Multicultural
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 4 of 19
Online Senior Assessment 2010: Multicultural
Multicultural 1:
The chart below shows the distributions of answers to the first question for participants
who took their class for the Multicultural core requirement at TTU (blue) and participants
who took their class for the Multicultural core requirement elsewhere (red). Answer 2 is
the correct choice. It can be seen that a majority in both groups chose the correct
answer. However, there are also quite a few participants in both groups who chose one
of the other answers. Overall, a few more people in the TTU group chose the correct
answer as compared to the ELSE group (71.9% vs. 68.0%). This difference is not
statistically significant at the 0.05 level (see attachment A for details). This means that
on average students who take their class for the Multicultural core requirement at TTU
do not do better with this question on average than students who take their Multicultural
class elsewhere.
Multicultural 1
80.0%
71.9%
70.0%
68.0%
60.0%
50.0%
40.0%
TTU
30.0%
ELSE
24.4%
17.4%
20.0%
10.6%
10.0%
7.6%
0.2% 0.0%
0.0%
1
2
3
4
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 5 of 19
Online Senior Assessment 2010: Multicultural
Multicultural 2:
The chart below shows the distributions of answers to the second question for
participants who took their class for the Multicultural core requirement at TTU (blue) and
participants who took their class for the Multicultural core requirement elsewhere (red).
Answer 3 is the correct choice. It can be seen that a large majority in both groups chose
the correct answer. There are only a few participants in both groups who chose one of
the other answers, which may mean that this question is not a good discriminator of
Multicultural understanding. A few more people in the TTU group chose the correct
answer as compared to the ELSE group (93.9% vs. 92.9%). This difference is not
statistically significant at the 0.05 level (see attachment A for details). This means that
on average students who take their class for the Multicultural core requirement at TTU
do not do better with this question than students who take their Multicultural class
elsewhere.
Multicultural 2
100.0%
93.9% 92.9%
90.0%
80.0%
70.0%
60.0%
50.0%
TTU
40.0%
ELSE
30.0%
20.0%
10.0%
0.2% 0.0%
0.2% 1.0%
1
2
5.7% 6.1%
0.0%
3
4
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 6 of 19
Online Senior Assessment 2010: Multicultural
Multicultural 3:
The chart below shows the distributions of answers to the third question for participants
who took their class for the Multicultural core requirement at TTU (blue) and participants
who took their class for the Multicultural core requirement elsewhere (red). Answer 1 is
the correct choice. It can be seen that a large majority in both groups chose the correct
answer. There are only a few participants in both groups who chose one of the other
answers, which may mean that this question is not a good discriminator of Multicultural
understanding. Overall, a few more people in the TTU group chose the correct answer
as compared to the ELSE group (96.1% vs. 90.9%). This difference is statistically
significant at the 0.05 level (see attachment A for details). This suggests that on
average students who take their class for the Multicultural core requirement at TTU do
better with this question than students who take their Multicultural class elsewhere.
Multicultural 3
120.0%
100.0%
96.1%
90.9%
80.0%
60.0%
TTU
ELSE
40.0%
20.0%
1.6% 3.0%
0.2% 1.5%
2.2% 4.6%
2
3
4
0.0%
1
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 7 of 19
Online Senior Assessment 2010: Multicultural
Multicultural 4:
The chart below shows the distributions of answers to the fourth question for
participants who took their class for the Multicultural core requirement at TTU (blue) and
participants who took their class for the Multicultural core requirement elsewhere (red).
Answer 4 is the correct choice. It can be seen that less than half of both groups chose
the correct answer. The fact that so many participants chose the same wrong answer
(i.e., answer 2) might indicate a lack of understanding in this area or that the answer
choice was confusing. It appears that students probably did not understand the
difference between discriminatory action and a stereotype. More people in the TTU
group chose the correct answer as compared to the ELSE group (46.8% vs. 37.6%).
This difference is statistically significant at the 0.05 level (see attachment A for details).
This suggests that on average students who take their class for the Multicultural core
requirement at TTU do better with this question than students who take their
Multicultural class elsewhere.
Multicultural 4
50.0%
46.8%
44.2%
45.0%
39.8%
40.0%
37.6%
35.0%
30.0%
25.0%
TTU
20.0%
ELSE
14.7%
15.0%
11.3%
10.0%
5.0%
3.6%
2.2%
0.0%
1
2
3
4
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 8 of 19
Online Senior Assessment 2010: Multicultural
Multicultural 5:
The chart below shows the distributions of answers to the fifth question for participants
who took their class for the Multicultural core requirement at TTU (blue) and participants
who took their class for the Multicultural core requirement elsewhere (red). Answer 1 is
the correct choice. It can be seen that a large majority in both groups chose the correct
answer. There are also quite a few participants in both groups who chose one of the
other answers. More people in the TTU group chose the correct answer as compared to
the ELSE group (77.2% vs. 70.1%). This difference is statistically significant at the 0.05
level (see attachment A for details). This suggests that on average students who take
their class for the Multicultural core requirement at TTU do better with this question than
students who take their Multicultural class elsewhere.
Multicultural 5
90.0%
80.0%
77.2%
70.1%
70.0%
60.0%
50.0%
TTU
40.0%
ELSE
30.0%
20.0%
3.8% 5.1%
10.0%
7.9%
11.7%
11.1% 13.2%
0.0%
1
2
3
4
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 9 of 19
Online Senior Assessment 2010: Multicultural
Multicultural 6:
The chart below shows the distributions of answers to the sixth question for participants
who took their class for the Multicultural core requirement at TTU (blue) and participants
who took their class for the Multicultural core requirement elsewhere (red). Answer 1 is
the correct choice. It can be seen that a majority in both groups chose the correct
answer. There are also quite a few participants in both groups who chose one of the
other answers. More people in the ELSE group chose the correct answer as compared
to the TTU group (75.6% vs. 74.2%). This difference is not statistically significant at the
0.05 level (see attachment A for details). This means that on average students who take
their class for the Multicultural core requirement elsewhere do not do better with this
question than students who take their Multicultural class at TTU.
Multicultural 6
80.0%
74.2% 75.6%
70.0%
60.0%
50.0%
40.0%
TTU
30.0%
ELSE
20.0%
11.1%
10.0%
13.4%
15.7%
8.6%
1.3% 0.0%
0.0%
1
2
3
4
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 10 of 19
Online Senior Assessment 2010: Multicultural
Multicultural 7:
The chart below shows the distributions of answers to the seventh question for
participants who took their class for the Multicultural core requirement at TTU (blue) and
participants who took their class for the Multicultural core requirement elsewhere (red).
Answer 2 is the correct choice. It can be seen that a majority in both groups chose the
correct answer. There are also quite a few participants in both groups who chose one of
the other answers. Overall, a few more people in the TTU group chose the correct
answer as compared to the ELSE group (77.6% vs. 68.5%). This difference is
statistically significant at the 0.05 level (see attachment A for details). This means that
on average students who take their class for the Multicultural core requirement at TTU
do better with this question than students who take their Multicultural class elsewhere.
Multicultural 7
90.0%
77.6%
80.0%
68.5%
70.0%
60.0%
50.0%
TTU
40.0%
ELSE
30.0%
20.0%
23.9%
15.4%
5.7% 5.1%
10.0%
1.3% 2.5%
0.0%
1
2
3
4
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 11 of 19
Online Senior Assessment 2010: Multicultural
Multicultural Average:
The table below compares the differences between TTU and ELSE when the results are
averaged (e.g., if a student got 5 out of the 7 questions correct, his score will be 5/7 =
.71). Participants who took their course for the Multicultural core requirement at Texas
Tech have a higher mean than participants who took their course elsewhere. This
difference is statistically significant at the 0.05 level. This suggests that on average
students who take their Multicultural course at TTU perform better on the Multicultural
section of the OSA than students who take their Multicultural course elsewhere.
N
Multicultural
Overall
Core at TTU
Mean
SD
558
76.8%
17.6%
N
Core Elsewhere
Mean
SD
197
71.9%
18.8%
T-stat
2.772
P-value
0.006
The chart below shows the distributions of the average scores for participants who took
their class for the Multicultural core requirement at TTU (blue) and participants who took
their class for the Multicultural core requirement elsewhere (red). The distributions are
fairly similar. The main difference appears to be that the TTU group had quite a few
more participants who answered all of the questions correctly.
Multicultural Overall: TTU vs. ELSE
35.0%
31.0%
30.0%
26.2%
29.2%
27.5%
25.0%
20.8%
18.4%
16.8%
20.0%
TTU
13.3%
15.0%
10.0%
5.0%
0.0%0.0% 0.2%0.8%
3.5%
1.8%
5.0%5.5%
0.0%
0%
14.3%
28.6%
42.9%
57.1%
71.4%
85.7%
100%
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 12 of 19
ELSE
Online Senior Assessment 2010: Multicultural
The table below shows a comparison of the average scores for the participants
selecting each course option within the ELSE group (those selecting more than one
course option were excluded from the analysis). The table includes the F value and P
value for an analysis of variance comparing the means. Although the table shows
information for all five ELSE options, the groups with less than 15 participants (Dual
Credit, Advanced Placement, and CLEP Exam) were excluded from the analysis of
variance.
Dual Credit
Advanced Placement
CLEP Exam
Another Institution
Study Abroad
Total
N
13
8
1
163
58
243
Mean
72.5%
87.5%
57.1%
70.9%
76.6%
72.8%
St. Dev.
18.9%
11.9%
.
19.2%
16.2%
18.6%
F-value
4.060
P-value
0.045
Based on the mean, students who took their Multicultural course through advanced
placement are the highest-performing group and students who took their course at
another institution are the lowest-performing group, excluding the one student who took
a CLEP exam. The two means included in the analysis of variance are significantly
different at the 0.05 level. This suggests that on average for the students who take their
Multicultural course elsewhere, which course option they use to take their Multicultural
course makes a difference in how they perform on the Multicultural section of the OSA.
More specifically this suggests that on average students who take their Multicultural
course through study abroad do better on the Multicultural section of the OSA than
students who take their Multicultural course at another institution. The group that took
their Multicultural course through advanced placement did not have enough participants
to include in the analysis of variance, but it appears that they may be the best
performing group on average. This outcome makes sense when considering that
students who take an AP course need to pass an exam to receive credit, which
suggests that they performed well in their AP class.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 13 of 19
Online Senior Assessment 2010: Multicultural
The chart below shows the distribution of the average scores for the female participants
(blue) and the male participants (red). The male participants had higher overall scores
than the female participants (mean of 76.1% vs. 75.2%). This difference is not
significant at the 0.05 level (see attachment B). This means that on average male
students do not do better than female students with the Multicultural section of the OSA.
Multicultural Overall by Sex
35.0%
29.7%
29.0%
26.9%
26.3%
30.0%
25.0%
21.2%
20.0%
17.9%
17.0%
16.3%
15.0%
Female
Male
10.0%
5.0%
3.2%
1.3%
1.0%
0.0%0.0% 0.0%
4.8%5.4%
0.0%
0%
14.3%
28.6%
42.9%
57.1%
71.4%
85.7%
100%
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 14 of 19
Online Senior Assessment 2010: Multicultural
The following table shows the correlations between the overall average for the
Multicultural questions and time to complete OSA, GPA, SAT score, ACT score, transfer
credits, total credits earned, and age (p-values for the correlations are in parenthesis).
The correlations with GPA, SAT score, ACT score, transfer credits, and total credits
earned are all significant at the 0.05 level. These correlations suggest that on average
students with higher GPA’s, higher SAT scores, higher ACT scores, less transfer
credits, and more total credits do better on the Multicultural section of the OSA. Some of
these correlations are small and are more likely to be found statistically significant
because of the large sample size.
Multicultural
Overall
Correlation
P-value
N
Time
-0.064
(0.079)
755
GPA
0.112
(0.002)
755
SAT
0.307
(<0.001)
511
ACT
0.311
(<0.001)
388
Transfer
Credits
-0.115
(0.002)
755
Total
Credits
0.117
(0.001)
755
Age
-0.033
(0.370)
755
The following tables show the results of regression models for the overall average for
the Multicultural questions including all of the variables that have been explored in this
analysis. There are three separate regression models because not all of the
participants have an SAT score and ACT score. Since most students either have an
SAT score or an ACT score, there would be too many missing values if both scores
were included in the same regression model. The first model excludes both in order to
include most respondents in the analysis.
Multicultural Model 1
N
F
P-value
753
4.03
< 0.001
Variable
Coefficient P-value
Intercept
0.610 < 0.001
Time
-0.000020
0.095
Sex
-0.014
0.305
GPA
0.036
0.002
Transfer Credits
-0.00024
0.544
Total Credits
0.00031
0.324
Age
0.00012
0.925
Multicultural class taken at TTU
0.025
0.143
This first model excludes SAT and ACT score to include 753 of the 755 participants.
The model overall is significant at the 0.05 level (R2 = 0.0365). For this model, GPA is
the only significant predictor at the 0.05 level for the overall average for the Multicultural
questions. This predictor suggests that on average students with higher GPA’s do
better on the Multicultural section of the OSA when the other variables in the model are
held constant.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 15 of 19
Online Senior Assessment 2010: Multicultural
Multicultural Model 2
N
F
P-value
511
8.13
< 0.001
Variable
Coefficient P-value
Intercept
0.243
0.007
Time
-0.000016
0.309
Sex
-0.010
0.507
GPA
0.028
0.049
Transfer Credits
-0.00039
0.419
Total Credits
0.00012
0.758
Age
0.0049
0.032
Multicultural class taken at TTU
0.013
0.536
SAT
0.00029 < 0.001
This second model includes SAT score and excludes ACT score to include 511 of the
755 participants. The model overall is significant at the 0.05 level (R2 = 0.1147). For
this model GPA, age, and SAT score are the significant predictors at the 0.05 level for
the overall average for the Multicultural questions. These predictors suggest that on
average students with higher GPA’s, older students, and students with higher SAT
scores do better on the Multicultural section of the OSA when the other variables in the
model are held constant. When the SAT score is included, the variable GPA seems to
become less important. It is interesting that age becomes more important when the SAT
score is included.
Multicultural Model 3
N
F
P-value
387
6.27
< 0.001
Variable
Coefficient P-value
Intercept
0.308
0.004
Time
-0.000059
0.200
Sex
-0.032
0.094
GPA
0.023
0.231
Transfer Credits
-0.00048
0.392
Total Credits
-0.0000047
0.992
Age
0.0046
0.113
Multicultural class taken at TTU
0.0098
0.699
ACT
0.012 < 0.001
This third model includes ACT score and excludes SAT score to include 387 of the 755
participants. The model overall is significant at the 0.05 level (R2 = 0.1172). For this
model, ACT score is the only significant predictor at the 0.05 level for the overall
average for the Multicultural questions. This predictor suggests that on average
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 16 of 19
Online Senior Assessment 2010: Multicultural
students with higher ACT scores do better on the Multicultural section of the OSA when
the other variables in the model are held constant.
Limitations
It is interesting that students who take their Multicultural course at TTU seem to do
better on this measure of Multicultural understanding than students who take their
Multicultural course elsewhere. It is difficult, though, to measure Multicultural
understanding with only seven questions. As mentioned in the results, some of the
questions were answered correctly by a large majority of the participants in both groups
and may not be good discriminators of Multicultural knowledge.
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 17 of 19
Online Senior Assessment 2010: Multicultural
Attachments
Attachment A: Summary of Chi-Square Tests for Questions 1 - 7
TTU
(N=500)
correct incorrect
401
157
524
34
536
22
261
297
431
127
414
144
433
125
Multicultural 1
Multicultural 2
Multicultural 3
Multicultural 4
Multicultural 5
Multicultural 6
Multicultural 7
Else
(N=232)
correct incorrect
134
63
183
14
179
18
74
123
138
59
149
48
135
62
Chi Statistic
1.04
0.25
7.83
5.00
4.05
0.16
6.43
Chi Probability
0.3074
0.6163
0.0051
0.0253
0.0441
0.6897
0.0112
Attachment B: 2-Sample T-Test for Average Scores by Sex
N
Multicultural
Overall
441
Female
Mean
SD
75.2%
N
17.7%
312
Male
Mean
SD
76.1%
T-stat
18.5%
-0.684
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 18 of 19
P-value
0.494
Online Senior Assessment 2010: Multicultural
Attachment C: Number of Participants Selecting Each Answer for Each Question
Multicultural 1
Answer
TTU
1
59
2
401
3
97
4
1
Multicultural 4
Answer
TTU
1
12
2
222
3
63
4
261
Multicultural 7
Answer
TTU
1
86
2
433
3
32
4
7
ELSE
15
134
48
0
ELSE
7
87
29
74
Multicultural 2
Answer
TTU
1
1
2
1
3
524
4
32
Multicultural 5
Answer
TTU
1
431
2
21
3
44
4
62
ELSE
0
2
183
12
ELSE
138
10
23
26
Multicultural 3
Answer
TTU
ELSE
1
536
179
2
9
6
3
1
3
4
12
9
Multicultural 6
Answer
TTU
ELSE
1
414
149
2
62
17
3
75
31
4
7
0
ELSE
47
135
10
5
Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010
Page 19 of 19