Wabash National Study of Liberal Arts
Education
First Year Summary Report
Introduction
The Wabash National Study of Liberal Arts
Education, being conducted at the Center of Inquiry
in the Liberal Arts at Wabash College, is a largescale, longitudinal study investigating critical factors
that affect the outcomes of liberal arts education. It is
designed to help colleges and universities improve
student learning and enhance the educational impact
of their programs. The two fundamental goals of the
study are to learn what teaching practices, programs,
and institutional structures support liberal arts
education, and to develop methods of assessing
liberal arts education.
The study focuses on the development of seven
outcomes associated with undergraduate liberal arts
education and the educational conditions and
experiences that foster these outcomes. The selected
outcomes include:
•
•
•
•
•
•
•
Effective reasoning and problem solving
Inclination to inquire and lifelong learning
Integration of learning
Intercultural effectiveness
Leadership
Moral reasoning
Well-being.
A total of nineteen institutions participated in the
Wabash Study in the first year. They included eleven
liberal arts colleges, three regional universities, three
research universities, and two community colleges.
The study sample contains both private
and public institutions, as well as
religiously-affiliated, single-sex, and
minority-serving schools. Participating
institutions exhibit a wide range of
selectivity, tuition costs, and geographic
variety 1 (see Appendix).
Methodology
The first-year battery of instruments was
administered at the beginning of the 2006
fall semester and in the latter part of the
2007 spring semester. At Hamilton, firstyear students completed the fall
administration at a single proctored sitting
during orientation, and in late March and
early April completed the spring
administration at two proctored sittings
and several individual or small group
sittings in the institutional research office.
In the fall a total of 332 first-year students
completed useable instruments for a cohort
response rate of 67%. In the spring, 170
of the original 332 respondents
participated, for a final cohort response
rate of 34%. Each student was paid $50
for participating each semester, and
1
From a description of the Wabash National Study
at: http://www.liberalarts.wabash.edu/nationalstudy
additional incentive prizes were raffled off for those
who participated in the spring.
Through a series of instruments the following tests
and scales were completed by the participants:
•
•
•
•
•
•
•
•
•
CAAP Critical Thinking Test
Defining Issues Test (moral reasoning)
Need for Cognition Scale
Milville-Guzman Universality-Diversity
Scale
Ryff Psychological Well-Being Scale
Socially Responsible Leadership Scale
Life Goals Scale
Orientation Towards Learning Scale
National Survey of Student Engagement
(spring only)
Results
Good Practices
Using a method called factor analysis, the Wabash
researchers were able to construct good practice
scales, or groupings of survey items that were highly
correlated and addressed a similar construct. Table 1
shows the good practice scales in which Hamilton
scored significantly higher than the mean of either the
liberal arts colleges (LACs) or the entire group of
colleges and universities. Practices that were rated
higher by Hamilton students among both groups
included “overall quality of faculty teaching” and
“co-curricular involvement.” The remaining practices
listed in Table 1 had higher Hamilton scores just
among the entire group of institutions. Table 2 shows
the scales in which Hamilton scored significantly
lower than the mean of either group. Those practices
included “frequency of contact with student affairs
staff” among both groups and “avoiding negative
diversity experiences” among all institutions.
of interactions with peers,” “cooperative
learning,” “integrated experiences,”
“course-related diversity experiences,” and
“positive diversity experiences.”
Table 1. Individual Good Practices in
Which Hamilton Scored Higher Than the
Group Averages
Sig. higher
than other
LACs
Good Practice Scales
Sig. higher
than all
other
institutions
Quality of non-classroom
interactions with faculty
Faculty interest in
teaching and student
development
X
Frequency of feedback
received from faculty
X
Overall quality of faculty
teaching
X
X
X
In class challenge and
expectations of faculty
X
Frequency of higher-order
exams and assignments
X
Co-curricular involvement
X
X
X
Diversity experiences
Table 2. Individual Good Practices in
Which Hamilton Scored Lower Than the
Group Averages
Good Practice Scales
Sig. lower
than other
LACs
Avoiding negative
diversity experiences
Frequency of contact with
student affairs staff
Sig. lower
than all
other
institutions
X
X
X
Individual good practices in which there was no
difference between Hamilton and the two groups
included “frequency of contact with faculty,”
“academic challenge and effort,” “positive influence
2
Factor analysis was also used to create broader good
practices scales in order to further group the practices
into constructs that are more easily benchmarked and
allow a more summative review of the findings of the
study. Table 3 shows that Hamilton scored
significantly higher than the mean of both groups in
“effective teaching and interaction with faculty,” and
higher than just the entire group in “degree to which
student reports working hard academically, feeling
challenged in class activities, and called on to
integrate material.”
Table 3. Broad Good Practices in Which Hamilton
Scored Higher Than the Group Averages
Broad Good Practice
Scales
Effective teaching and
interaction with faculty
Degree to which student
reports working hard
academically, feeling
challenged in class
activities, and called on to
integrate material
Sig. higher
than other
LACs
Sig.higher
than all
other
institutions
X
X
Table 4. Hamilton’s Rank on Broad Good
Practices Scales
Rank
among
LACs
Rank
among all
institutions
Effective teaching and
interaction with faculty
2 of 11
2 of 19
Frequency with which
student interacts with
others on campus
8 of 11
10 of 19
6 of 11
7 of 19
7 of 11
8 of 19
7 of 11
9 of 19
Broad Good Practice
Scales
Degree to which student
reports working hard
academically, feeling
challenged in class
activities, and called on to
integrate material
Degree to which student
reports positive
experiences and
interactions with diverse
others and ideas
Degree to which students
reports interacting with
peers
X
Broad good practices in which there was no
difference between Hamilton and the two groups
included “frequency with which student interacts with
others on campus,” “degree to which student reports
positive experiences and interactions with diverse
others and ideas,” and “degree to which student
reports interacting with peers.” There were no broad
good practices in which Hamilton had a significantly
lower score. Table 4 shows Hamilton’s rank within
the two groups on each of the broad dimensions.
Learning Outcomes
By administering the instruments at the
beginning of the year and near the end of
the year, the researchers were able to
develop measures of change in certain
cognitive and reasoning skills over the
first year of college. Specifically, the
CAAP test of critical thinking, the DIT
moral reasoning test, the universalitydiversity scale, and the need for cognition
scale all allow for measurement of higherorder thinking and intellectual rigor in
addressing complex issues and problems.
Table 5 shows how Hamilton ranked
among the other liberal arts colleges in
scores on each of the fall and spring tests,
and the amount of change from fall to
spring. The only change in rank from fall
to spring for Hamilton was on the CAAP
critical thinking test, where Hamilton
dropped two spots. And, while the relative
3
ranks on each exam are around the middle to the
upper third of the group, the ranks of the actual
amount of change are near the bottom
more comprehensive analysis of the entire
curriculum and campus culture can be
made.
Table 5. Hamilton's Rank Among Liberal Arts
Colleges on Selected Learning Outcomes
Change
(fallspring)
rank among
LACs
Fall rank
among
LACs
Spring
rank
among
LACs
9 of 11
3 of 11
5 of 11
Need for Cognition
10 of 11
4 of 11
4 of 11
Universality-Diversity
(full scale)
Moral Reasoning (DITP)
8 of 11
6 of 11
6 of 11
9 of 11
4 of 11
4 of 11
Item
Critical Thinking
(CAAP)
Summary
Overall, there were mixed results for Hamilton from
the first year of the Wabash Study. In relative terms,
the area in which Hamilton is strongest is the
faculty’s skill in the classroom and their ability to
connect with students academically. These findings
agree with other institutional research projects, such
as NSSE and the senior survey, where Hamilton
students were found to have a much higher level of
satisfaction with the faculty than students at peer
institutions.
In other areas important to a liberal arts education,
Hamilton does not do as well. The level of academic
challenge was not as highly rated at Hamilton
compared to the other liberal arts colleges, and
Hamilton does even worse in the quality and quantity
of peer interaction and in having positive experiences
with, and opinions of, diverse people and ideas.
These findings, however, only relate to the first year
experience. Any conclusions or plans of action
derived from these data should be placed in that
context. The Wabash Study will continue with
student interviews over the next two years, and will
again administer these same interests when the fall
2006 entering cohort are seniors. After that time a
Report completed by:
Gordon Hewitt
Office of Institutional Research
October 26, 2007
4
Appendix
Participating Institutions in 2006-2007:
Liberal Arts Colleges
Alma College
Bard College
Columbia College (SC)
Connecticut College
Coe College
Gustavus Adolphus College
Hamilton College
Hope College
Wabash College
Whittier College
Regional Universities
Butler University
San Jose State University
University of North Carolina Wilmington
Research Universities
University of Kentucky
University of Michigan
University of Notre Dame
Community Colleges
Ivy Tech Community College
Kirkwood Community College
5