Online Senior Assessment 2009: Social and Behavioral Sciences Introduction
The sixth component of OSA contains the questions for the Social and Behavioral
Sciences. The first question asks if students took their Social and Behavioral Sciences
course as a core requirement at TTU.
The following pie chart shows that out of the 704 participants, 496 (i.e., 70.45%)
indicated that they took their Social and Behavioral Sciences course for core curriculum
credit at TTU and 208 (i.e., 29.55%) indicated that they took it elsewhere.
The system stores some data for each of the participants and so it was possible to
identify the respondents who major in Social and Behavioral Sciences. The majors that
were identified and classified as Social and Behavioral Sciences majors are ANTH,
BECO, CFAS, COMS, ECO, ESS, GEOG, HDFS, HLTH, IECO, MCOM, POLS, PSY,
Office of Planning and Assessment, September 2009 Page 1 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences SOC, and SW. There were a total of 140 Social and Behavioral Sciences majors in the
sample. These participants represent the so-called “experts”.
Results
The student learning outcomes for the Social and Behavioral Sciences are:
- To identify and critique alternative explanations for claims about social issues
and human behavior.
- To recognize the appropriate methods, technologies, and data that social and
behavioral scientists use to investigate the human condition.
- To identify and appreciate differences and commonalities between cultures.
- To demonstrate knowledge of the origins and evolution of U.S. and Texas
political systems.
The first student learning outcome aligns well with questions 1, 4, and 5. The second
learning outcome aligns well with questions 2, 3, and 4. The third learning outcome
does not clearly align with any of the assessment questions. The fourth learning
outcome aligns well with question 6 through 10.
The Social and Behavioral Sciences portion of the assessment contains ten knowledge
questions. These are shown below as a screenshot from the actual instrument. For
analysis purposes, the answers were coded from 1 to 4 in the order they appear on the
actual instrument.
Office of Planning and Assessment, September 2009 Page 2 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Office of Planning and Assessment, September 2009 Page 3 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Office of Planning and Assessment, September 2009 Page 4 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Office of Planning and Assessment, September 2009 Page 5 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 1:
The chart below shows the distributions of answers for the first question for participants
who took their class for the Social and Behavioral Sciences core requirement at TTU
(bottom) and participants who took their class for the Behavioral and Social Sciences
core requirement elsewhere (top). Answer 3 is the correct choice. It can be seen that
just over half of both groups chose the correct answer. Overall, a few more people in
the “TTU” group chose the correct answer as compared to the “Else” group (61.49%
versus 55.29%). This difference is not statistically significant. Please see attachment A
for details. This means that students who take their class for the Social and Behavioral
Sciences core requirement at TTU do not do significantly better with this question than
the students who take their Social and Behavioral Sciences class elsewhere.
Office of Planning and Assessment, September 2009 Page 6 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 2:
The chart below shows the distributions of answers for the second question for
participants who took their class for the Social and Behavioral Sciences core
requirement at TTU (bottom) and participants who took their class for the Behavioral
and Social Sciences core requirement elsewhere (top). Answer 1 is the correct choice.
It can be seen that over two thirds of both groups chose the correct answer. Overall,
more people in the “TTU” group chose the correct answer as compared to the “Else”
group (79.64% versus65.38%). This difference is statistically significant. Please see
attachment A for details. This means that students who take their class for the Social
and Behavioral Sciences core requirement at TTU do significantly better with this
question than the students who take their Social and Behavioral Sciences class
elsewhere.
Office of Planning and Assessment, September 2009 Page 7 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 3:
The chart below shows the distributions of answers for the third question for participants
who took their class for the Behavioral and Social Sciences core requirement at TTU
(bottom) and participants who took their class for the Behavioral and Social Sciences
core requirement elsewhere (top). Answer 2 is the correct choice. It can be seen that
less than half of both groups chose the correct answer. Overall, more people in the
“TTU” group chose the correct answer as compared to the “Else” group (42.94% versus
34.13%). This difference is statistically significant. Please see attachment A for details.
This means that students who take their class for the Behavioral and Social Sciences
core requirement at TTU do significantly better with this question than the students who
take their the Behavioral and Social Sciences class elsewhere.
Office of Planning and Assessment, September 2009 Page 8 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 4:
The chart below shows the distributions of answers for the fourth question for
participants who took their class for the Social and Behavioral Sciences core
requirement at TTU (bottom) and participants who took their class for the Behavioral
and Social Sciences core requirement elsewhere (top). Answer 4 is the correct choice.
It can be seen that only a few in both groups chose the correct answer; this question
may be too difficult or misleading to make it a good discriminator of knowledge in the
Social and Behavioral Sciences. Overall, a few more people in the “Else” group chose
the correct answer as compared to the “TTU” group (10.10% versus 8.27%). This
difference is not statistically significant. Please see attachment A for details. This means
that students who take their class for the Social and Behavioral Sciences core
requirement at TTU do not do significantly better with this question than the students
who take their Social and Behavioral Sciences class elsewhere.
Office of Planning and Assessment, September 2009 Page 9 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 5:
The chart below shows the distributions of answers for the fifth question for participants
who took their class for the Social and Behavioral Sciences core requirement at TTU
(bottom) and participants who took their class for the Behavioral and Social Sciences
core requirement elsewhere (top). Answer 3 is the correct choice. It can be seen that
less than half of both groups chose the correct answer. Overall, a few more people in
the “TTU” group chose the correct answer as compared to the “Else” group (40.52%
versus 38.46%). This difference is not statistically significant. Please see attachment A
for details. This means that students who take their class for the Social and Behavioral
Sciences core requirement at TTU do not do significantly better with this question than
the students who take their Social and Behavioral Sciences class elsewhere.
Office of Planning and Assessment, September 2009 Page 10 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 6:
The chart below shows the distributions of answers for the sixth question for participants
who took their class for the Social and Behavioral Sciences core requirement at TTU
(bottom) and participants who took their class for the Behavioral and Social Sciences
core requirement elsewhere (top). Answer 1 is the correct choice. It can be seen that
just over half of both groups chose the correct answer. Overall, a few more people in
the “TTU” group chose the correct answer as compared to the “Else” group (54.84%
versus 53.85%). This difference is not statistically significant. Please see attachment A
for details. This means that students who take their class for the Social and Behavioral
Sciences core requirement at TTU do not do significantly better with this question than
the students who take their Social and Behavioral Sciences class elsewhere.
Office of Planning and Assessment, September 2009 Page 11 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 7:
The chart below shows the distributions of answers for the seventh question for
participants who took their class for the Social and Behavioral Sciences core
requirement at TTU (bottom) and participants who took their class for the Behavioral
and Social Sciences core requirement elsewhere (top). Answer 2 is the correct choice.
It can be seen that less than a third of both groups chose the correct answer. Overall, a
few more people in the “Else” group chose the correct answer as compared to the
“TTU” group (29.81% versus 28.02%). This difference is not statistically significant.
Please see attachment A for details. This means that students who take their class for
the Social and Behavioral Sciences core requirement at TTU do not do significantly
better with this question than the students who take their Social and Behavioral
Sciences class elsewhere.
Office of Planning and Assessment, September 2009 Page 12 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 8:
The chart below shows the distributions of answers for the eighth question for
participants who took their class for the Social and Behavioral Sciences core
requirement at TTU (bottom) and participants who took their class for the Behavioral
and Social Sciences core requirement elsewhere (top). Answer 1 is the correct choice.
It can be seen that around half of both groups chose the correct answer. Overall, a few
more people in the “TTU” group chose the correct answer as compared to the “Else”
group (54.23% versus 49.52%). This difference is not statistically significant. Please see
attachment A for details. This means that students who take their class for the Social
and Behavioral Sciences core requirement at TTU do not do significantly better with this
question than the students who take their Social and Behavioral Sciences class
elsewhere.
Office of Planning and Assessment, September 2009 Page 13 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 9:
The chart below shows the distributions of answers for the ninth question for
participants who took their class for the Social and Behavioral Sciences core
requirement at TTU (bottom) and participants who took their class for the Behavioral
and Social Sciences core requirement elsewhere (top). Answer 2 is the correct choice.
It can be seen that a large majority of both groups chose the correct answer, which may
make this question to be too easy to be a good discriminator of Social and Behavioral
Sciences knowledge. Overall, more people in the “TTU” group chose the correct answer
as compared to the “Else” group (98.59% versus 95.67%). This difference is statistically
significant. Please see attachment A for details. This means that students who take their
class for the Social and Behavioral Sciences core requirement at TTU do significantly
better with this question than the students who take their Social and Behavioral
Sciences class elsewhere.
Office of Planning and Assessment, September 2009 Page 14 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences 10:
The chart below shows the distributions of answers for the tenth question for
participants who took their class for the Social and Behavioral Sciences core
requirement at TTU (bottom) and participants who took their class for the Behavioral
and Social Sciences core requirement elsewhere (top). Answer 1 is the correct choice.
It can be seen that just over half of both groups chose the correct answer. Overall, a few
more people in the “TTU” group chose the correct answer as compared to the “Else”
group (61.49% versus 56.73%). This difference is not statistically significant. Please see
attachment A for details. This means that students who take their class for the Social
and Behavioral Sciences core requirement at TTU do not do significantly better with this
question than the students who take their Social and Behavioral Sciences class
elsewhere.
Office of Planning and Assessment, September 2009 Page 15 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Social and Behavioral Sciences Average:
The table below compares the differences between “TTU” and “Else” when the results
for all the questions are averaged (e.g., if a student got 8 out of the 10 questions
correct, his score will be 8/10 = .80). The mean is higher for students who took their
core requirement for the Social and Behavioral Sciences at TTU. This difference is
statistically significant.
Core Elsewhere
Core at TTU
pCore Area
N
Mean
stdev
N
mean
stdev
value
Social/Behavioral S.
207 0.48986
0.1705
496
0.53004 0.1544 0.0024
The chart below shows the distributions of the average scores for participants who took
their class for the Behavioral and Social Sciences core requirement at TTU (bottom)
and participants who took their class for the Behavioral and Social Sciences core
requirement core requirement elsewhere (top). The distributions are similar, but it looks
like the “Else” group centers closer to 4 out of 10 correct answers and “TTU” group
centers closer to 5 out of 10 correct answers.
Office of Planning and Assessment, September 2009 Page 16 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences The chart below shows the distributions of the average scores for those participants
who are majoring in the Behavioral and Social Sciences (bottom) and participants who
are not majoring in the Behavioral and Social Sciences (top). A larger percentage of the
Behavioral and Social Sciences majors answered more than half of the questions
correctly, while the non-majors seem to center at about 5 out of 10 correct answers.
Overall, the majors have a slightly higher average than the non-majors (54% vs. 51%).
This difference is not statistically significant at the .05 level. Please see attachment B for
details. This means that we cannot conclude that Behavioral and Social Sciences
majors perform better than non-majors on the assessment.
The overall average for the Behavioral and Social Sciences questions does not
significantly correlate with the total transfer credits, total credits earned, or the age of
participants. The overall average for the Behavioral and Social Sciences questions
does significantly correlate with participants’ GPA (see attachment C). There is a
statistically significant difference between the average overall performance of the
female and male participants (see attachment D). Male students perform better on this
measure than female students (mean of 54% versus 50%).
Office of Planning and Assessment, September 2009 Page 17 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Below is a chart showing the distribution of averages for the female participants (top)
and the male participants (bottom). It appears that there are few more male participants
answering with higher scores that may pull the male average up.
Comparison: 2008 vs. 2009
In the 2008 assessment the “TTU” performed practically the same as the “Else” group.
The Social Sciences and Behavioral Sciences were kept separate in the 2008 analysis
and “TTU” did about the same as the “Else” group in both sections (60.48% vs. 60.58%
in the Social Sciences and 43.3% vs. 41.6% in the Behavioral Sciences). In the 2009
assessment, there was a statistically significant difference between the performance of
the two groups with the “TTU” group doing better than the “Else” group (53.0% vs.
49.0%).
Office of Planning and Assessment, September 2009 Page 18 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Limitations
It is difficult to measure the breadth of knowledge taught in the Social and Behavioral
Sciences with ten questions. Many of the questions seem to have challenged the
participant. As mentioned in the results, a couple of the questions may have been too
simple or too complicated to be a good discriminator of Social and Behavioral Science
knowledge. Without more information about the courses taken elsewhere, it is difficult
to clarify what any differences between the “TTU” group and the “Else” group might
mean. Any differences may be the result of what type of Social and Behavioral
Sciences class was taken and what the emphasis of that class was.
Office of Planning and Assessment, September 2009 Page 19 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Attachments
Attachment A: Summary and Chi-Square Test for Questions 1 - 4
n = 208 n = Social/Behavioral Sciences 1 Social/Behavioral Sciences 2 Social/Behavioral Sciences 3 Social/Behavioral Sciences 4 Social/Behavioral Sciences 5 Social/Behavioral Sciences 6 Social/Behavioral Sciences 7 Social/Behavioral Sciences 8 Social/Behavioral Sciences 9 Social/Behavioral Sciences 10 correc
t Else incorrec
t 496 correc
t TTU incorrec
t Chi‐Square Probability percent correct Else percent correct TTU Better
? Significant
? 115 93 305 191 0.1258 55.29% 61.49% TTU no 136 72 395 101 <.0001 65.38% 79.64% TTU yes 71 137 213 283 0.0297 34.13% 42.94% TTU yes 21 187 41 455 0.4344 10.10% 8.27% Else no 80 128 201 295 0.6101 38.46% 40.52% TTU no 112 96 272 224 0.8093 53.85% 54.84% TTU no 62 146 139 357 0.6326 29.81% 28.02% Else no 103 105 269 227 0.2529 49.52% 54.23% TTU no 199 9 489 7 0.0179 95.67% 98.59% TTU yes 118 90 305 191 0.2392 56.73% 61.49% TTU no Office of Planning and Assessment, September 2009 Page 20 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Attachment B: 2-Sample T-Test for Average Scores of Majors (1)
vs Non-Majors (0)
Office of Planning and Assessment, September 2009 Page 21 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Attachment C: Simple Statistics and Correlations
Office of Planning and Assessment, September 2009 Page 22 of 23 Online Senior Assessment 2009: Social and Behavioral Sciences Attachment D: Average Performance and Gender
Office of Planning and Assessment, September 2009 Page 23 of 23