Online Senior Assessment 2013: Natural Sciences
INTRODUCTION
The fourth component of the OSA contains questions for the Natural Sciences core. The first
questions ask the participants about how they fulfilled their Natural Sciences core requirement.
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 1 of 13
Online Senior Assessment 2013: Natural Sciences
The following table shows the number and percentage of participants who selected each
response to the first question regarding where students took their core curriculum course. The
number of participants selecting each response adds up to more than the 1,578 total
participants because participants were instructed to select all that apply.
Response
How did you complete your multicultural requirement?
% of Sample
n
I took a multicultural class through dual credit in high school.
50
3.2%
I took an advanced placement multicultural class in high
school.
59
3.7%
I took a CLEP exam for multicultural credit.
13
0.8%
I received transfer multicultural credit for a class that I took
at another institution.
557
35.3%
I took my core curriculum class at Tech.
899
57.0%
For the analysis in this report the 1,578 participants will be divided into the “TTU” group and
the “ELSE” group. The TTU group will represent the 899 participants (57.0% of the sample) who
selected “I took my core curriculum class at Tech.” The participants who selected one or more
of the other responses were asked if the class they took outside of Tech counted for their core
curriculum credit. The ELSE group will represent the 606 participants (38.4% of the sample)
who reported that a class taken outside of Tech did count for their Natural Sciences core
curriculum credit. These 606 participants were asked which course counted. The following
table shows the number and percentage of the sample who selected each response.
Which one?
Response
A dual credit class.
An advanced placement class.
A CLEP exam.
A class I took at another institution.
I don't know.
[did not answer]
Total
n
47
42
2
502
6
7
606
% of Sample
3.0%
2.7%
0.1%
31.8%
0.4%
0.4%
38.4%
73 participants (4.6% of the sample) reported that they did not know if a course taken outside
of Tech counted for their Natural Sciences core curriculum credit. These 73 participants will be
excluded from any TTU vs. ELSE group analyses.
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 2 of 13
Online Senior Assessment 2013: Natural Sciences
The system stores data for each of the participants which makes it possible to identify the
respondents who major in programs related to the Natural Sciences core. The following majors
were identified and classified as Natural Sciences majors: Animal Sciences, Biochemistry,
Biology, Cell and Molecular Biology, Chemistry, Environmental Crop and Soil Sciences,
Geosciences, Horticultural Turf Grass Science, Microbiology, Physics, and Zoology. The
following table shows that there were a total of 171 Natural Sciences majors in the OSA sample.
It also displays how many participants were in each of the Natural Sciences majors.
Natural Sciences Majors
Major
Animal Sciences
Biochemistry
Biology
Cell and Molecular Biology
Chemistry
Environmental Crop and Soil Sciences
Geosciences
Horticultural and Turf Grass Sciences
Microbiology
Physics
Zoology
Total
n
27
23
49
11
19
4
16
7
7
6
2
171
% of sample
1.7%
1.5%
3.1%
0.7%
1.2%
0.3%
1.0%
0.4%
0.4%
0.4%
0.1%
10.8%
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 3 of 13
Online Senior Assessment 2013: Natural Sciences
RESULTS
The student learning outcomes for the Natural Sciences are:




Demonstrate knowledge of the scientific method and to contrast it with other ways
of understanding the world.
Demonstrate knowledge of the tools and methods used by scientists to study the
natural world.
Explain some of the major theories in the Natural Sciences.
Describe how Natural Sciences research informs societal issues, including ethics.
The Natural Sciences section of the OSA contains six knowledge questions. The first learning
outcome aligns with the fourth, fifth, and sixth question. The second learning outcome aligns
with the fourth and fifth question. The third learning outcome aligns with the first, second, and
third question. The fourth student learning outcome aligns with the second question. A
screenshot and summary of responses for each question is shown on the following pages.
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 4 of 13
Online Senior Assessment 2013: Natural Sciences
Natural Sciences Question 1:
The chart below shows the distributions of answers for the first question for participants who
took their Natural Sciences course at TTU (red) and participants who took their Natural Sciences
course elsewhere (blue). Answer 4 is the correct choice. It can be seen that almost half of both
groups chose the correct answer. Overall, a few more people in the ELSE group chose the
correct answer as compared to the TTU group (50.5% vs. 44.5%). This difference is not
statistically significant at the 0.05 level. This means that on average students who took their
Natural Sciences course elsewhere do not do significantly better with this question than
students who took their course at TTU. Since the first question aligns with the third learning
outcome, this suggests that on average students who took their Natural Sciences course at TTU
meet this learning outcome similar to students who took their course elsewhere.
Metabolism
19.5%
17.2%
Heredity
19.6%
19.8%
Homeostasis
10.6%
9.7%
TTU
44.5%
50.5%
Movement
Cellular organization
0.0%
ELSE
5.9%
2.8%
20.0%
40.0%
60.0%
80.0%
100.0%
Percent of Sample
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 5 of 13
Online Senior Assessment 2013: Natural Sciences
Natural Sciences Question 2:
The chart below shows the distributions of answers for the second question for participants
who took their Natural Sciences course at TTU (red) and participants who took their Natural
Sciences course elsewhere (blue). Answer 4 is the correct choice. It can be seen that less than
half of both groups chose the correct answer. Overall, a few more people in the ELSE group
chose the correct answer as compared to the TTU group (44.9% vs. 40.9%). This difference is
not statistically significant at the 0.05 level. This means that on average students who took
their Natural Sciences course elsewhere do not do significantly better with this question than
students who took their course at TTU. Since the second question aligns with the third and
fourth learning outcomes, this suggests that on average students who took their Natural
Sciences course at TTU meet these learning outcomes similar to students who took their course
elsewhere.
38.6%
33.2%
Increased mutation rates in bacteria
4.9%
5.4%
Decreased mutation rates in bacteria
13.9%
14.0%
Selection against antibiotic-resistant bacteria
40.9%
44.9%
Selection for antibiotic-resistant bacteria
Does not have an effect on the evolution of
bacteria
TTU
ELSE
1.7%
2.5%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Percent of Sample
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 6 of 13
Online Senior Assessment 2013: Natural Sciences
Natural Sciences Question 3:
The chart below shows the distributions of answers for the third question for participants who
took their Natural Sciences course at TTU (red) and participants who took their Natural Sciences
course elsewhere (blue). Answer 1 is the correct choice. It can be seen that a majority of both
groups chose the correct answer, with about one fourth of both groups choosing another
answer. Overall, a few more people in the TTU group chose the correct answer as compared to
the ELSE group (72.1% vs. 71.3%). This difference is not statistically significant at the 0.05 level.
This means that on average students who took their Natural Sciences course at TTU do not do
significantly better with this question than students who took their course elsewhere. Since the
third question aligns with the third learning outcome, this suggests that on average students
who took their Natural Sciences course at TTU meet this learning outcome similar to students
who took their course elsewhere.
72.1%
Oceanic crust is continuously being destroyed
by subduction.
Oceanic crust first began to form 160 million
years ago; before that no oceans existed.
Older oceanic crust exists but has not been
dated.
There is no difference between oceanic and
continental crust.
0.0%
71.3%
13.7%
13.2%
10.7%
TTU
12.0%
ELSE
3.6%
3.5%
20.0%
40.0%
60.0%
80.0%
100.0%
Percent of Sample
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 7 of 13
Online Senior Assessment 2013: Natural Sciences
Natural Sciences Question 4:
The chart below shows the distributions of answers for the fourth question for participants who
took their Natural Sciences course at TTU (red) and participants who took their Natural Sciences
course elsewhere (blue). Answer 4 is the correct choice. It can be seen that a large majority of
both groups chose the correct answer. Overall, a few more people in the ELSE group chose the
correct answer as compared to the TTU group (79.0% vs. 78.3%). This difference is not
statistically significant at the 0.05 level. This means that on average students who took their
Natural Sciences course elsewhere do not do significantly better with this question than
students who took their Natural Sciences course at TTU. Since the fourth question aligns with
the first and second learning outcomes, this suggests that on average students who took their
Natural Sciences course at TTU meet these learning outcomes similar to students who took
their course elsewhere.
non-control group
control group
conclusion
4.4%
5.0%
11.0%
9.4%
TTU
6.2%
6.6%
ELSE
78.3%
79.0%
hypothesis
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Percent of Sample
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 8 of 13
Online Senior Assessment 2013: Natural Sciences
Natural Sciences Question 5:
The chart below shows the distributions of answers for the fifth question for participants who
took their Natural Sciences course at TTU (red) and participants who took their Natural Sciences
course elsewhere (blue). Answer 2 is the correct choice. It can be seen that a majority of both
groups chose the correct answer. Overall, more people in the ELSE group chose the correct
answer as compared to the TTU group (76.9% vs. 73.6%). This difference is not statistically
significant at the 0.05 level. This means that on average students who took their Natural
Sciences course elsewhere do not do significantly better with this question than students who
took their Natural Sciences course elsewhere. Since the fifth question aligns with the first and
second learning outcomes, this suggests that on average students who took their Natural
Sciences course at TTU meet these learning outcomes similar to students who took their course
elsewhere.
22.5%
non-control group
21.3%
73.6%
control group
conclusion
hypothesis
76.9%
2.2%
TTU
1.3%
ELSE
1.7%
0.5%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Percent of Sample
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 9 of 13
Online Senior Assessment 2013: Natural Sciences
Natural Sciences Question 6:
The chart below shows the distributions of answers for the sixth question for participants who
took their Natural Sciences course at TTU (red) and participants who took their Natural Sciences
course elsewhere (blue). Answer 2 is the correct choice. It can be seen that a large majority of
both groups chose the correct answer. Overall, a few more people in the TTU group chose the
correct answer as compared to the ELSE group (86.8% vs. 84.3%). This difference is not
statistically significant at the 0.05 level. This means that on average students who took their
Natural Sciences course at TTU do not do significantly better with this question than the
students who took their Natural Sciences course elsewhere. Since the sixth question aligns with
the first learning outcome, this suggests that on average students who took their Natural
Sciences course at TTU meet this learning outcome similar to students who took their course
elsewhere.
absolutely proved
4.9%
7.9%
86.8%
84.3%
supported
non-supported
absolutely disproved
0.0%
TTU
6.3%
5.3%
ELSE
2.0%
2.5%
20.0%
40.0%
60.0%
80.0%
100.0%
Percent of Sample
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 10 of 13
Online Senior Assessment 2013: Natural Sciences
Natural Sciences Average:
The table below compares the differences between TTU and ELSE when the results for all the
questions are averaged (e.g., if a student got 3 out of the 6 questions correct, his score will be
3/6 = .50). The mean is almost the same, but slightly higher for students who took their core
requirement for the Natural Sciences elsewhere. This difference is not statistically significant at
the 0.05 level. This means that on average TTU scores similar to ELSE in the Natural Sciences.
n
Natural Sciences
Overall
899
Core at TTU
Mean
SD
66.0%
Core Elsewhere
n
Mean
SD
22.9%
606
67.8%
22.3%
t-stat
p-value
-1.50
0.0671
The chart below shows the distributions of scores for participants who took their class for the
Natural Sciences core requirement at TTU (red) and participants who took their class for the
Natural Sciences core requirement elsewhere (blue). The distributions are similar, but it looks
like the ELSE group had a few more participants who answered all of the questions correctly.
Natural Sciences Questions: TTU vs. ELSE
30.0%
27.9% 28.2% 28.0%
25.9%
Percent of Sample
25.0%
19.3%
20.0%
16.4%
14.7%
15.0%
11.6%
10.8%
10.0%
ELSE
7.4%
4.3%
3.6%
5.0%
1.0%0.8%
0.0%
0%
17%
33%
TTU
50%
67%
83%
100%
Percent Correct
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 11 of 13
Online Senior Assessment 2013: Natural Sciences
The table below shows a comparison of the average scores for the participants selecting each
course option within the ELSE group.
A dual credit class.
An advancement placement class.
A CLEP exam.
A class I took at another institution.
I don’t know.
[did not answer]
Total
n
47
42
2
502
6
7
606
Mean
65.2%
78.2%
75.0%
67.2%
50.0%
65.4%
67.8%
SD
20.2%
22.5%
35.4%
22.2%
23.6%
2.1%
22.3%
Based on the mean, students who took their Natural Sciences course through advanced
placement are the highest scoring group and students who took a dual credit class are the
lowest scoring group, excluding the students that did not know which course counted for their
core credit.
The following table shows a summary of the correlations between the percentage of Natural
Sciences questions answered correctly and the following variables of interest: gender, age, SAT
score, ACT score, transfer credit hours, total credit hours, cumulative GPA, Humanities major,
Mathematics major, Natural Sciences major, and Social and Behavioral Sciences major.
Gender
Age
SAT score
ACT score
Transfer hours
Total credit hours
Cumulative GPA
Humanities major
Mathematics major
Natural Sciences major
Social/Behavioral major
Correlation
-0.05
-0.01
0.39
0.34
-0.03
0.06
0.15
0.10
0.02
0.15
-0.02
p-value
0.0321
0.6938
<.0001
<.0001
0.2143
0.0165
<.0001
0.0001
0.3292
<.0001
0.5289
n
1,575
1,578
897
743
1,578
1,578
1,576
1,578
1,578
1,578
1,578
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 12 of 13
Online Senior Assessment 2013: Natural Sciences
The following tables summarize the regression model using these same variables of interest
with Natural Sciences scores on the OSA as the outcome variable. Note that the variables SAT
score and ACT score were combined into one variable, SAT/ACT score, to include more students
in the one model. This variable was created by using a conversion table from the ACT website
to convert ACT scores to the SAT score range (see http://www.act.org/solutions/college-careerreadiness/compare-act-sat/).
Regression model predicting Natural Sciences score
n
F-value
p-value
R2
1,183
26.08
< .0001
0.1820
Variable
B
SE
β
Intercept
-0.1896
0.0818
Gender
-0.0093
0.0131
-0.02
Age
0.0063
0.0024
0.08**
SAT/ACT score
0.0005
0.0000
0.38***
Transfer credit hours
0.0000
0.0002
0.01
Total credit hours
0.0002
0.0003
0.02
Cumulative GPA
0.0186
0.0112
0.05
Humanities major
0.0724
0.0293
0.07*
Mathematics major
-0.0137
0.0191
-0.02
Natural Sciences major
0.1088
0.0207
0.15***
Social and Behavioral Sciences major
0.0558
0.0152
0.11***
Note: * p < 0.05; ** p < 0.01; *** p < 0.001; gender: 0 = male, 1 = female
The model overall is significant at the 0.05 level (R2 = 0.1820). For this model age, SAT/ACT
score, Humanities major, Natural Sciences major, and Social and Behavioral Sciences major
were significant predictors at the 0.05 level for Natural Sciences scores on the OSA. The model
suggests that when controlling for the other variables in the model older students, students
with higher SAT/ACT scores, students with a Humanities related major, students with a Natural
Sciences related major, and students with a Social and Behavioral Sciences related major, on
average, score higher on the Natural Sciences section of the OSA.
Office of Planning and Assessment, Devin DuPree and Gail Alleyne Bayne, June 2013
Page 13 of 13