Social Sciences and Diversity
Assessment Report—4/27/2011
This draft from Gary Johnson (not the entire subcommittee). Data analysis provided by Susan Ratwik.
Social Sciences
LSSU graduates will be able to think critically and analytically about the causes and consequences of
human behavior.
Explanation of Outcome
Albert Einstein once said that “the whole of science is nothing more than a refinement of everyday
thinking.” Einstein’s point applies to the social sciences as well as the natural sciences. Each of these
disciplines engages in the systematic study of some domain of human behavior. Both the methods and
theories of the social sciences are, ultimately, a refinement of the human capacity for critical and
analytical thinking. It is this capacity that is valuable for all students, whether they are students of
business, engineering, nursing, or any of the other fields in which LSSU grants degrees. While the
discipline-specific content of a social science course may be valuable to students from some particular
major, it is not the specific content of these courses that is of enduring value for students across all
majors. It is, rather, the discipline-transcending skills that may be applied across all domains of human
behavior. By engaging these skills in more that one discipline, students discover that they are applicable
in more than one domain. This insight should prepare them to engage in a lifetime of critical and
analytical thinking across all domains of human behavior.
Assessment Recommendation
The Social Sciences and Diversity Subcommittee, in each of the past two years, has recommended that
the best and most practical assessment tool for the social sciences was ETS’s MAPP Test, now referred to
as the Academic Proficiency Profile (APP).
APP’s questions encompass all of the social sciences. Results may be compared with results from other
institutions. However, administering APP both when students enter and when they leave will allow
longitudinal assessment as well.
The APP test provides valuable data for assessing the overall success of our set of general education
courses in achieving the social sciences outcome. In addition, if it is administered to a large enough
sample of students, APP results could be combined with campus data so that we could assess the
effectiveness of specific courses. Controlled for ACT scores, we would gain valuable information about
the individual courses students use in satisfying the social sciences requirement.
Since nineteen different courses from six different disciplines may be used to satisfy the Social Sciences
requirement, we do not recommend the development of methods of assessment other than APP, at least
for General Education purposes.
1
Course Offerings, 2010-2011
The nineteen different courses that may be used to fulfill the social sciences requirement are listed below,
together with relevant data for 2010-2011.
Soc Sci
Gen Ed
Course
No.
Sect.
F2010*
Enrolled
All Sect.
F2010*
Completion
Rate
F2010*†
GE Outcome
on Syllabus
S2011*
No.
Sect.
S2011*
Enrolled
All Sect.
S2011*
Total
Sect.
2010-11
Total
Enrolled
2010-11
ECON 201
2
65
94%
N
1
50
3
115
ECON 202
1
40
93%
N
1
57
2
97
GEOG 201
1
43
100%
1
43
GEOG 302
1
23
100%
1
23
HIST 101
1
35
94%
1
35
1
47
2
63
ECON 208
ECON 209
ECON 302
HIST 102
HIST 131
Y
2
63
47
100%
HIST 132
POLI 110
1
Y
2
88
2
88
2
123
93%
Y
1
61
3
184
PSYC 101
2
206
91%
Y
1
82
3
288
PSYC 155
1
126
91%§
Y
1
95
2
221
SOCY 101
2
184
77%
Y
2
189
4
373
Y
2 (o-l)
30
2 (o-l)
30
1
17
28
1624
POLI 160
POLI 241
SOCY 102
SOCY 113
1
17
Totals
16
925
100%
Y=7; N=2
12
699
* Blank cell indicates that the course was not offered that semester
† Completion rate with D- or better. N’s and W’s are not counted here, either in the enrollment figure or as unsuccessful
completions. Only F’s and I’s are counted as unsuccessful completions.
§
Numerous incompletes included in unsuccessful completions that may have turned into passing grades
2
Assessment Results as of Spring 2011
Summaries of Scaled Score Comparison to ETS Norm
1. Fall 2009 Compared to ETS Entering Freshmen
Fall 2009 LSSU freshmen were significantly above average on the social science scale relative to the
appropriate ETS freshman comparison group. The effect size (using Cohen’s d, the difference
between LSSU mean and ETS mean divided by the population standard deviation), however, was
relatively small. LSSU students were .21 standard deviations above the ETS comparison norm. An
effect size of .20 is considered small.
ETS Entering
Freshmen 1
LSSU Freshmen Fall 2009 (N = 406)
Scale
Social Sciences
1
M
SD
p (< .05 is significant)
d (Effect Size)
M
SD
112.39
5.32
0.000
.21
111.2
5.7
ETS Entering Freshmen (No hours completed )Baccalaureate (Liberal Arts) Colleges I and II, Individual Scores
2. Fall 2010 Compared to ETS Entering Freshmen
Fall 2010 LSSU freshmen were also significantly above average on the social science scale relative to
the appropriate ETS freshman comparison group. The effect size (using Cohen’s d, the difference
between LSSU mean and ETS mean divided by the population standard deviation) was, again,
relatively small, however. The LSSU Fall 2010 freshmen were .29 standard deviations above the
ETS comparison norm. An effect size of .20 is considered small.
ETS Entering
Freshmen 1
LSSU Freshmen Fall 2010
Scale
Social Sciences
M
SD
p (< .05 is significant)
d (Effect Size)
M
SD
112.86
5.56
0.000
.29
111.2
5.7
3
3. Spring 2010 Seniors Compared to ETS Seniors
Spring 2010 LSSU seniors scored significantly above average on the social science scale relative to
the appropriate ETS senior comparison group. The effect size (using Cohen’s d, the difference
between LSSU mean and ETS mean divided by the population standard deviation) was small,
however. The LSSU Spring 2010 seniors were .25 standard deviations above the ETS comparison
norm. An effect size of .20 is considered small.
LSSU Seniors Spring 2010
Scale
Social Sciences
ETS Seniors
M
SD
p (< .05 is significant)
d' (Effect Size)
M
SD
116.10
5.65
0.001
.25
114.5
6.3
2
ETS Seniors (More than 90 semester hours or more than 145 quarter hours) Baccalaureate (Liberal Arts)
Colleges I and II, Individual Scores
4. Spring 2010 LSSU Seniors Compared to Fall 2009 and Fall 2010 LSSU Freshman
The table below shows the mean scaled scores for LSSU Seniors and LSSU Freshmen. The means
were compared with one-way analyses of variance and Bonferroni adjusted post hoc analyses. The
LSSU seniors’ means were significantly above the LSSU freshman means for the social sciences.
Analysis of Variance
Freshmen
Freshmen
F2009
F2010
M
Scale
Social
Sciences
2
SD
112.39 5.32
M
112.86
SD
5.56
Seniors S2010
M
SD
116.10 5.65
4
Seniors
Significantly
higher than
Freshmen
Yes
F
25.45 0.000
p
Senior Exit Survey on General Education Outcomes – Spring 2010
Do you feel that your educational experience at LSSU prepared you to think critically and
analytically about the causes and consequences of human behavior?
Cumulative
Frequency
Valid
Missing
Percent
Valid Percent
Percent
Did not prepare me
3
2.7
3.1
3.1
Prepared me poorly
15
13.6
15.6
18.8
Prepared me adequately
53
48.2
55.2
74.0
Prepared me well
25
22.7
26.0
100.0
Total
96
87.3
100.0
System
14
12.7
110
100.0
Total
Summary and Recommendation
LSSU seniors scored significantly better on the APP social science questions that both their ETS senior
comparison group and two different groups of LSSU freshmen. The senior exit survey provided
additional support believing that the social sciences requirement is accomplishing its objective.
As we continue administering the APP social science questions, we should—as planned—begin assessing
the effectiveness of individual social science courses though APP scores, ACT scores, and campus data.
5
Diversity
LSSU graduates will be able to view the world from cultural perspectives other than their own.
Explanation of Outcome
Understanding diversity is not a matter of advocacy on behalf of any specific group or category
of people. Nor is it a matter of learning a specific theory of political, social, or economic
relations—such theories may be only transitory products of particular schools of thought. The
enduring skill that will help students throughout their lives—both professional and personal—is
a capacity to see the world through the eyes of those who are fundamentally different in some
way. One should be able to hone this skill in any course that provides a relatively in-depth
examination of any of the divergent cultures that exist within and across societies, and that are
sometimes also associated with differences in race, gender, class, religion, national origin, and
sexual orientation. Students who develop this skill should be able to utilize it throughout their
lives and across cultural boundaries with which they were previously unfamiliar. The diversity
that exists today is not necessarily the diversity that will exist tomorrow.
Assessment Recommendation
The APP test does not address the diversity outcome. The Social Sciences and Diversity
Subcommittee therefore recommended that we assess our diversity requirement through local
questions that would be administered with APP. If both incoming freshmen and graduating
seniors are tested with a good set of local questions, we should have good data for assessing the
overall success of our set of general education courses. In addition, as with APP, a large enough
sample will allow us to assess the effectiveness of specific courses.
Since twelve different courses from seven different disciplines may be used to satisfy this
requirement, we do not recommend the development of methods of assessment other than the
locally developed APP questions, at least for General Education purposes.
A set of local questions was administered on an experimental basis in 2009-10 for both the
incoming freshmen and the graduating seniors. These same questions were given to the Fall
2010 entering freshmen, and they are also being given to Spring 2011 seniors. If these questions
appear to be useful, the number of these questions should probably be expanded in the future.
6
Course Offerings, 2010-2011
The thirteen different courses that may be used to fulfill the diversity requirement are listed below,
together with relevant data for 2010-2011.
Diversity
Gen Ed
Course
No.
Sect.
F2010*
Enrolled
All Sect.
F2010*
Completion
Rate
F2010*†
BUSN 308
1
9
89%
ECUC 250
1
35
74%§
GE Outcome
on Syllabus
S2011*
GEOG 306
No. of
Sect.
S2011*
Enrolled
All Sect.
S 2011*
Total
Sect.
2010-11
Total
Enrolled
2010-11
1
9
Y
1
26
2
61
Y
1
76
1
76
HIST 203
HLTH 328
1
39
100%
Y
1
35
2
74
6 (2 o-l)
189
90%
Y
5 (3 o-l)
131
11(5o-l)
320
9
272
Y=4
8
268
17
540
NATV 225
POLI 333
POLI 334
SOCY 103
SOCY 213
SOCY 225
SOCY 226
SOCY 321
Totals
* Blank cell indicates that the course was not offered that semester
† Completion rate with D- or better. N’s and W’s are not counted here, either in the enrollment figure or as unsuccessful
completions. Only F’s and I’s are counted as unsuccessful completions.
§
Numerous incompletes included in unsuccessful completions that may have turned into passing grades
Assessment Results as of Spring 2011
There are two types of local diversity questions. The first type consists of “attitude” questions
that attempt to measure students’ attitudes through self-reports. The strength of these questions
is their attempt to tap directly into student’s attitudes. The potential weakness of these questions
is that students know what attitudes are considered “correct” and may simply provide the
answers they know are valued.
The other type of question, what might be called “factual questions,” attempt to measure
students’ knowledge either about other cultures or about the views of the world that derive from
being a member of another culture. These questions, which are an attempt to get at students’
7
knowledge of other cultures and their capacity to think like others, were an experiment in these
first three administrations. If they appear to be successful, more questions should be added.
Attitude Questions
These self-report attitudes were measured using a four-point Likert-type scale. In Fall of 2009, a
four-point scale was presented horizontally, with “Strongly Disagree” presented left of a “1” and
“Strongly Agree” presented to the right of a “4.” With more than four options available on the
ETS scoring sheet, with options beginning with “0,” some students treated this as a six-point
scale, providing answers from “0” through “5.”
Since these messy results cannot be reliably interpreted, no results are presented here for Fall
2009. The options were recast for the Spring 2010 and Fall 2010 administrations to eliminate the
possibility of confusion (0=Strongly agree; 1=Agree; 2=Disagree; 3=Strongly disagree).
Comparing the Fall 2010 freshman results with the Spring 2010 senior results, the “attitude” test
items generally showed a shift toward more positive attitudes, but only Question 11 (“I am
knowledgeable about other cultures and lifestyles”) showed a significant change.
Crosstab Additional Question11
I am knowledgeable about other cultures and lifestyles.
Strongly
Seniors May
2010
Count
% within
Strongly Agree
Agree
Disagree
Disagree
Total
12
86
36
6
140
8.6%
61.4%
25.7%
4.3%
100.0%
16
205
207
39
467
3.4%
43.9%
44.3%
8.4%
100.0%
Code
Freshmen Fall Count
2010
% within
Code
The proportions were significantly different, χ 2 (3, N = 467) = 24.81, p < .001.
8
Crosstab Additional Question12
I feel comfortable around people from different cultures and
lifestyles.
Strongly
Agree
Seniors
May 2010
Count
% within
Strongly
Agree
Disagree
Disagree
52
78
8
37.1%
55.7%
5.7%
173
264
28
37.0%
56.4%
6.0%
Total
2
140
1.4% 100.0%
Code
Freshmen Count
Fall 2010
% within
3
468
.6% 100.0%
Code
The proportions were not significantly different, χ 2 (3, N = 608) = .84, p < .841.
Crosstab Additional Question13
I often get discouraged when interacting with people from different
cultures.
Seniors
May 2010
Freshmen
Fall 2010
Count
% within Code
Count
% within Code
Strongly Agree
Agree
Disagree
Strongly Disagree
4.00
Total
3
10
78
49
0
140
2.1%
7.1%
55.7%
35.0%
.0%
100.0%
4
30
298
136
0
468
.9%
6.4%
63.7%
29.1%
.0%
100.0%
The proportions were not significantly different, χ 2 (3, N = 608) = 4.0, p < .262.
9
Crosstab Additional Question14
I feel that I can share aspects of my culture or lifestyle with other
people.
Strongly Agree
Agree
Disagree
Strongly Disagree
Total
44
90
5
1
140
31.4%
64.3%
3.6%
.7%
100.0%
145
297
25
1
468
31.0%
63.5%
5.3%
.2%
100.0%
Seniors May 2010 Count
% within
Code
Freshmen Fall
2010
Count
% within
Code
The proportions were not significantly different, χ 2 (3, N = 608) = 1.52, p < .677.
Factual Questions
All four of the factual questions were answered correctly by a significantly higher percentage of
Spring 2010 Seniors than by Fall 2010 Freshmen.
Question
#
% Answering Correctly
χ2
(1df)
Freshmen
Seniors
Fall 2010 Spring 2010
Q7
51.7
66.4
9.44
Q8
39.0
53.6
9.40
Q9
38.5
49.3
5.14
Q10
59.0
79.1
18.82
Questions are presented below in Appendix A.
10
Total
N
p
Significant
Difference
608
607
607
607
.002
.002
.023
<.000
Yes
Yes
Yes
Yes
Senior Exit Survey on General Education Outcomes – Spring 2010
Do you feel that your educational experience at LSSU prepared you to view the world from
cultural perspectives other than your own?
Cumulative
Frequency
Valid
Valid Percent
Percent
Did not prepare me
6
5.5
5.9
5.9
Prepared me poorly
13
11.8
12.7
18.6
Prepared me adequately
49
44.5
48.0
66.7
Prepared me well
34
30.9
33.3
100.0
102
92.7
100.0
8
7.3
110
100.0
Total
Missing
Percent
System
Total
Summary and Recommendation
Spring 2011 seniors believed they were more knowledgeable about other peoples and cultures
than did Fall 2010 freshmen. The senior exit survey provided additional support from self-report
data. And on some specific items that relate to knowledge of other cultures, Spring 2011 seniors
actually demonstrated greater knowledge than Fall 2010 freshmen.
We have reason to believe that LSSU’s diversity requirement is accomplishing its objective. As
we continue to assess the effectiveness of this requirement, we should refine our attitude
questions and expand the number of factually based questions.
11
Appendix A
Diversity Questions
7. Your sister has been killed in a tragic automobile accident. You and your family oppose cremation of
her body. You also insist that she be buried within 24 hours, and you would prefer that she be buried
without embalming and without a coffin. What is the most likely reason that you and your family
have these preferences?
0.
1.
2.
3.
You are atheists
You are Muslims
You are members of the Green Movement
You are recent immigrants to the United States
8. You consider yourself a patriotic American. Your grandfather, who was a U.S. marine, died in the
Battle of Iwo Jima. Your father served in the U.S. army in Vietnam, and you did a tour of duty in
Iraq. Nevertheless, you always have mixed feelings when you watch the American flag wave in the
breeze. Which of the following is the most likely explanation for your ambivalence?
0.
1.
2.
3.
Your family has always been poor
You are Jewish
You are Native American
You are a socialist
9. Capital punishment continues to be a contentious issue in the United States. There are highly
conflicting views about whether the death penalty should continue to be employed. You understand
that there are good arguments on both sides of this issue, but you oppose capital punishment because
of the history of its use in America. You are most likely to feel this way if you are
0.
1.
2.
3.
Catholic
Protestant
Arab-American
African-American
10. If you consider Yom Kippur (Day of Atonement) to be the holiest day of the year, you are
0.
1.
2.
3.
Morman
Jewish
Muslim
Catholic
11. I am knowledgeable about other cultures and lifestyles.
0.
1.
2.
3.
Strongly agree
Agree
Disagree
Strongly disagree
12
12. I feel comfortable around people from different cultures and lifestyles.
0.
1.
2.
3.
Strongly agree
Agree
Disagree
Strongly disagree
13. I often get discouraged when interacting with people from different cultures.
0.
1.
2.
3.
Strongly agree
Agree
Disagree
Strongly disagree
14. I feel that I can share aspects of my culture or lifestyle with other people.
0.
1.
2.
3.
Strongly agree
Agree
Disagree
Strongly disagree
13