Online Senior Assessment 2009: Humanities Introduction
The first component of OSA contains the questions for Humanities. The first question
asks if students took their Humanities course as a core requirement at TTU.
The following pie chart shows that out of the 704 participants, 526 (i.e., 74.72%)
indicated that they took their Humanities course for core curriculum credit at TTU and
178 (i.e., 25.28%) indicated that they took it elsewhere.
The system stores some data for each of the participants and so it was possible to
identify the respondents who major in Humanities. The following majors were identified
and classified as Humanities majors: CLAS, ENGL, FREN, GERM, HAL, HIST, PHIL,
and SPAN. These participants represent the so-called “experts” in the sample. The
following table shows that there were a total of 54 Humanities majors in the OSA
sample. It also displays how many participants were in each of the major areas.
Office of Planning and Assessment, September 2009 Page 1 of 22 Online Senior Assessment 2009: Humanities Results
The student learning outcomes for Humanities are:
- Identify methodologies of historical, literary, philosophical, and/or aesthetic
research and recognize their applicability to everyday life.
- Develop analytical arguments in written and/or oral form.
- Evaluate events, ideas, and artistic expressions in terms of multiple cultural
contexts and value systems.
- Recognize ways in which the humanities are fundamental to the health and
survival of any society.
There does not seem to be a question on OSA that measures the first learning
outcome. The second learning outcome seems to align very well with the first selfassessment question. It looks like the third learning outcome aligns closely with the
second self-assessment question as well as all four knowledge questions. The fourth
learning outcome seems to relate to the third knowledge question. It is not clear which
learning outcome the third self-assessment question aligns with.
The first three questions ask the students to assess their own abilities. These three selfassessment questions are shown below. For analysis purposes, the answers were
coded from 1 = “definitely yes” to 5 = “definitely no”.
Office of Planning and Assessment, September 2009 Page 2 of 22 Online Senior Assessment 2009: Humanities Self-Assessment Question 1:
The chart below shows the distributions of answers for the first question for participants
who took their class for Humanities core requirement at TTU (bottom) and participants
who took their class for Humanities core requirement elsewhere (top). It can be seen
that the two groups chose very similar answers across the board. Both groups seem
pretty confident about their ability to think critically in written and verbal form. The group
“TTU” has a slightly higher mean than the group “Else” (1.36 versus 1.33). However,
this difference is not statistically significant at the .05 level. Please see attachment A for
details. This means that students who take their class for Humanities core requirement
at TTU do not do significantly better than students who take it elsewhere. Since the first
self-assessment question aligns with the second learning outcome, this means that on
average students who take their Humanities course at TTU meet this learning outcome
about the same as those who take the course elsewhere.
Office of Planning and Assessment, September 2009 Page 3 of 22 Online Senior Assessment 2009: Humanities Self-Assessment Question 2:
The chart below shows the distributions of answers for the second question for
participants who took their class for Humanities core requirement at TTU (bottom) and
participants who took their class for Humanities core requirement elsewhere (top). It can
be seen that the two groups again chose very similar answers. Both groups seem pretty
confident about their ability to recognize the possibility of multiple interpretations,
cultural contexts, and values. The group “TTU” has a slightly lower mean than the
group “Else” (1.37 versus 1.40). However, this difference is not statistically significant at
the .05 level. Please see attachment B for details. This means that students who take
their class for Humanities core requirement at TTU do not do significantly better than
students who take it elsewhere. Since the second self-assessment question aligns with
the third learning outcome, this means that on average students who take their
Humanities course at TTU meet this learning outcome about the same as those who
take the course elsewhere.
Office of Planning and Assessment, September 2009 Page 4 of 22 Online Senior Assessment 2009: Humanities Self-Assessment Question 3:
The chart below shows the distributions of answers for the third question for participants
who took their class for Humanities core requirement at TTU (bottom) and participants
who took their class for Humanities core requirement elsewhere (top). It can be seen
that the two groups again chose similar answers. Both groups seem pretty confident
about their ability to discuss ways in which the humanities shape or are shaped by
cultures and societies. The group “TTU” has a slightly lower mean than the group “Else”
(1.65 versus 1.70). However, this difference is not statistically significant at the .05 level.
Please see attachment C for details. This means that students who take their class for
Humanities core requirement at TTU do not do significantly better than students who
take it elsewhere. It is not clear which learning outcome the third self-assessment
question aligns with.
Office of Planning and Assessment, September 2009 Page 5 of 22 Online Senior Assessment 2009: Humanities The second part consists of four knowledge questions. These are shown below as a
screenshot from the actual instrument. For analysis purposes, the answers were coded
from 1 to 5 in the order they appear on the actual instrument.
Office of Planning and Assessment, September 2009 Page 6 of 22 Online Senior Assessment 2009: Humanities Office of Planning and Assessment, September 2009 Page 7 of 22 Online Senior Assessment 2009: Humanities Knowledge Question 1:
The chart below shows the distributions of answers for the first knowledge question for
participants who took their class for Humanities core requirement at TTU (bottom) and
participants who took their class for Humanities core requirement elsewhere (top).
Answer 3 is the correct choice. It can be seen that a large majority in both groups chose
the correct answer. However, there are also quite a few participants in both groups who
chose one of the other answers. Overall, a few more people in “TTU” chose the correct
answer as compared to the group “Else” (66.16% versus 65.17%). This difference is not
statistically significant. Please see attachment D for details. This means that students
who take their class for Humanities core requirement at TTU do not do significantly
better than students who take it elsewhere. Since the first question aligns with the third
learning outcome, this means that on average students who take their Humanities
course at TTU meet this learning outcome at similar levels compared to students who
take the course elsewhere.
Office of Planning and Assessment, September 2009 Page 8 of 22 Online Senior Assessment 2009: Humanities Knowledge Question 2:
The chart below shows the distributions of answers for the second knowledge question
for participants who took their class for Humanities core requirement at TTU (bottom)
and participants who took their class for Humanities core requirement elsewhere (top).
Answer 5 is the correct choice. It can be seen that about half of both groups chose the
correct answer. For this question a few more people in “Else” chose the correct answer
as compared to the group “TTU” (52.81% versus 46.01%). This difference is not
statistically significant. Please see attachment D for details. This means that students
who take their class for Humanities core requirement at TTU do not do significantly
better than students who take it elsewhere. Since the second question aligns with the
third learning outcome, this means that on average students who take their Humanities
course at TTU meet this learning outcome at similar levels compared to students who
take the course elsewhere.
Office of Planning and Assessment, September 2009 Page 9 of 22 Online Senior Assessment 2009: Humanities Knowledge Question 3:
The chart below shows the distributions of answers for the third knowledge question for
participants who took their class for Humanities core requirement at TTU (bottom) and
participants who took their class for Humanities core requirement elsewhere (top).
Answer 1 is the correct choice. It can be seen that a large majority in both groups chose
the correct answer. There are only a few participants in both groups who chose one of
the other answers, which may make this question a poor discriminator of humanities
knowledge. Overall, a few more people in “TTU” chose the correct answer as compared
to the group “Else” (89.54% versus 88.20%). This difference is not statistically
significant. Please see attachment D for details. This means that students who take
their class for Humanities core requirement at TTU do not do significantly better than
students who take it elsewhere. Since the third question aligns with the third learning
outcome, this means that on average students who take their Humanities course at TTU
meet this learning outcome at similar levels compared to students who take the course
elsewhere.
Office of Planning and Assessment, September 2009 Page 10 of 22 Online Senior Assessment 2009: Humanities Knowledge Question 4:
The chart below shows the distributions of answers for the fourth knowledge question
for participants who took their class for Humanities core requirement at TTU (bottom)
and participants who took their class for Humanities core requirement elsewhere (top).
Answer 5 is the correct choice. It can be seen that a large majority in both groups chose
the correct answer. There are only a few participants in both groups who chose one of
the other answers, which may make this question a poor discriminator of humanities
knowledge. Overall, a few more people in “TTU” chose the correct answer as compared
to the group “Else” (90.87% versus 88.76%). This difference is not statistically
significant. Please see attachment D for details. This means that students who take
their class for Humanities core requirement at TTU do not do significantly better than
students who take it elsewhere. Since the fourth question aligns with the third learning
outcome, this means that on average students who take their Humanities course at TTU
meet this learning outcome at similar levels compared to students who take the course
elsewhere.
Office of Planning and Assessment, September 2009 Page 11 of 22 Online Senior Assessment 2009: Humanities Knowledge Questions Average:
The table below compares the differences between “TTU” and “Else” when the results
for all the knowledge questions are averaged (e.g., if a student got 3 out of the 4
questions correct, his score will be ¾ = .75). The mean is almost the same, but slightly
higher for students who took their core requirement for Humanities elsewhere. However,
this difference is not statistically significant. This means that “TTU” performs about the
same as “Else” in the Humanities.
Core Elsewhere
Core Area
Humanities
N
Core at TTU
pmean
stdev
N
mean
stdev
value
178 0.73736
0.2714
526 0.731464 0.2512
0.791
The chart below shows the distributions of the average scores for participants who took
their class for Humanities core requirement at TTU (bottom) and participants who took
their class for Humanities core requirement elsewhere (top). The distributions are fairly
similar, but it looks like there is a bigger group in “Else” that got a top score. However,
this group also has more people performing 50% or below than “TTU”.
Office of Planning and Assessment, September 2009 Page 12 of 22 Online Senior Assessment 2009: Humanities The chart below shows the distributions of the average scores for those participants
who are majoring in the Humanities area (bottom) and participants who do not have a
major that is part of Humanities (top). It is interesting to see that while over 50% of the
majors reach 100%, the non-majors do not even have 40% in that area. Overall, the
majors have a much higher average than the non-majors (83% vs 73%). This difference
is statistically significant at the .05 level. Please see attachment E for details. This
means that students who major in Humanities perform significantly better than students
who major in areas.
The correlations between the overall average for the Humanities questions and total
transfer credits, total credits earned, and age are all fairly low and none of them are
significant (see attachment F). There is however a fairly strong correlation between the
average score and GPA which is statistically significant. Also, there is a statistically
significant difference between average overall performance and gender (see attachment
G). Female students perform significantly better than male students (mean of 75%
versus 71%).
Office of Planning and Assessment, September 2009 Page 13 of 22 Online Senior Assessment 2009: Humanities The chart below shows the distributions of the average scores for the female and male
participants. It appears that the significant difference in average scores of the female
and male participants is due to a few more female participants getting 4 out of 5
questions correct and a few more males getting 1 or 2 out of 5 questions correct.
Although the results are statistically significant, there does not appear to be a large
difference between the performance of the female and male participants.
Office of Planning and Assessment, September 2009 Page 14 of 22 Online Senior Assessment 2009: Humanities Comparison: 2008 vs. 2009
In the 2008 assessment, the “TTU” group did not significantly differ from the “Else”
group in responses to the self-assessment questions or knowledge questions. This
assessment found similar results; the “TTU” group did not significantly differ from the
“Else” group in responses to the self-assessment questions or knowledge questions.
Looking at the knowledge questions averages, there is a smaller gap between the
“TTU” group and the “Else” group in 2009, than there was in 2008 (73.2% versus 73.7%
in 2009 and 71.5% versus 66.1% in 2008). The scores for both groups are slightly
higher in 2009 than in 2008. Overall, the differences in results from 2008 and 2009 are
too small to claim that they are due to more than just chance.
Limitations
It is important to recognize the difficulty of measuring a participant’s knowledge in the
Humanities with seven questions. The three self-assessment questions can help to
measure broader knowledge with fewer questions, but it is difficult to interpret and
compare the results (what does “Definitely Yes” and “Somewhat Yes” mean and do they
mean the same thing for each participant surveyed). Also, the student might not have
an accurate perception of his or her knowledge. As noted in the results, a couple of the
knowledge questions (questions 3 and 4) were answered correctly by almost 90% of
both the “TTU” group and the “Else” group. It may be that some of the questions were
too easy to discriminate between those with a greater understanding of the humanities.
Overall, though, that the Humanities majors did significantly better than everyone else
does suggest that the measure did help capture participants’ understanding of the
Humanities.
Office of Planning and Assessment, September 2009 Page 15 of 22 Online Senior Assessment 2009: Humanities Attachments
Attachment A: 2-Sample T-Test – Self-Assessment Question 1
Office of Planning and Assessment, September 2009 Page 16 of 22 Online Senior Assessment 2009: Humanities Attachment B: 2-Sample T-Test – Self-Assessment Question 2
Office of Planning and Assessment, September 2009 Page 17 of 22 Online Senior Assessment 2009: Humanities Attachment C: 2-Sample T-Test – Self-Assessment Question 3
Office of Planning and Assessment, September 2009 Page 18 of 22 Online Senior Assessment 2009: Humanities Attachment D: Summary and Chi-Square Test for Questions 1 - 4
n = 178 n = Else 526 TTU correct incorrect correct incorrect Chi‐Square Probability percent correct Else percent correct TTU Better? Significant? Humanities 1 116 62 348 178 0.8094 65.17% 66.16% TTU no Humanities 2 94 84 242 284 0.1163 52.81% 46.01% Else no Humanities 3 157 21 471 55 0.6181 88.20% 89.54% TTU no Humanities 4 158 20 478 48 0.41 88.76% 90.87% TTU no Office of Planning and Assessment, September 2009 Page 19 of 22 Online Senior Assessment 2009: Humanities Attachment E: 2-Sample T-Test for Average Scores of Majors (1)
and Non-Majors (0)
Office of Planning and Assessment, September 2009 Page 20 of 22 Online Senior Assessment 2009: Humanities Attachment F: Simple Statistics and Correlations
Office of Planning and Assessment, September 2009 Page 21 of 22 Online Senior Assessment 2009: Humanities Attachment G: Average Performance and Gender
Office of Planning and Assessment, September 2009 Page 22 of 22