Warren Wilson College – Overview of First-Year Results
Introduction
This overview highlights findings for Warren Wilson College students in the first year of the Wabash National
Study. Because this is an overview, it touches on a fraction of the information about your students’
responses included in the data tables and data files we sent you last fall. We hope you can use this
overview to spark conversations about your students, and we hope it piques your curiosity about the more
detailed information contained within the rest of your Wabash National Study first-year report.
This report includes five sections:
Section 1 – Summary of how much your students changed on some of our outcome measures. Outcomes
are simply students’ skills and capacities—such as critical thinking, moral reasoning, or interest in
contributing to the arts—that should be improved by their work at Warren Wilson College.
Section 2 – Summary of your students' experience of Warren Wilson's teaching and learning environment.
Although students’ reports about, for example, the degree to which they receive prompt feedback or the
extent to which their assignments require them to integrate information are subjective—and may not align
with faculty and staff perceptions of what happens in the classroom—they do matter. All of the student
experiences we discuss in this overview predict growth on a variety of independent outcome measures.
Section 3 – Summary comparison of your students' reports about how much time they spend preparing for
class.
Section 4 – Summary of your students' reports about their binge drinking.
Section 5 – Short review of variability in student experiences within your institution. Although asking how
student experiences at Warren Wilson College compare with those at other small colleges can be
informative, it is far more important to ask about the range of student experiences within Warren Wilson.
We look forward to working with you to review and clarify this information so that you can use it to benefit
your students.
Section 1 – Overview of Warren Wilson Students’ Growth on Outcome Tests and
Surveys
The Wabash National Study measures student development in twelve areas. The following table
summarizes whether your students were more likely to improve, decline, or remain unchanged on these
outcomes during their first year at Warren Wilson College. The table also compares the changes your
students experienced to those of students at other small institutions in the study. 1
Students' Change on Wabash National Study Outcome Measures
Outcome
Critical Thinking
Psychological Well-Being
Size of First-Year Change
at Warren Wilson College
Size of First-Year
Change at Small
Institutions Overall
Outcome Change
Percentile Rank among
All Institutions2
Moderate improvement
No effective change
90
Small improvement
No effective change
88
Openness to Diversity/Challenge
Small decline
Small decline
71
Socially Responsible Leadership
No effective change
No effective change
63
Contribution to the Arts
No effective change
No effective change
60
Political and Social Involvement
Small decline
Small decline
40
Positive Attitude Toward Literacy
Small decline
No effective change
29
No effective change
No effective change
24
Small decline
Small decline
24
Moderate decline
Moderate decline
24
Small decline
No effective change
22
Need for Cognition
Contribution to the Sciences
Academic Motivation
Universality-Diversity Awareness
Moral Reasoning
No effective change
Moderate improvement
Overall, Warren Wilson students were somewhat less likely to grow on our outcomes than students at other
Wabash Study schools. Your students grew on two outcomes over their first year of college: critical thinking
and psychological well-being. On average, students at other small colleges did not change on either of these
two measures. Like students elsewhere, Warren Wilson students did not change or declined on many of our
outcome measures, including leadership, need for cognition, political and social involvement, and academic
motivation. It is worth noting that your students did not gain on moral reasoning while students at other small
colleges tended to show moderate improvement in this area. Additionally, Warren Wilson students declined
more than students in the comparison group on positive attitude toward literacy and universality-diversity
awareness.
The outcome on which Warren Wilson students improved most was critical thinking. The outcome on which
they declined most was academic motivation. The following tables include more detailed information about
how your students changed on these two outcomes compared to students at other institutions in the study.
Warren Wilson College’s First-Year Change in Critical Thinking
The CAAP Critical Thinking Test measures students’ skills in clarifying, analyzing, evaluating, and extending
arguments. The largest potential change is 40 points.
Institution Type3
Large institution
Small institution
Small institution
Small institution
Warren Wilson College
Small institution
Small institution
Small institution
Small institution
Small institution
Small institution
Large institution
Small institution
Large institution
Large institution
Small institution
Large institution
Small institution
Mean Change from
Fall to Spring
Effect Size4
Description of Effect Size5
3.36
2.80
1.65
1.73
1.69
1.61
1.20
1.20
1.15
1.24
1.22
0.78
0.96
0.88
0.71
0.71
0.52
0.61
0.71
0.64
0.36
0.34
0.31
0.31
0.24
0.24
0.24
0.21
0.21
0.20
0.19
0.19
0.17
0.15
0.14
0.13
Very large improvement
Large improvement
Moderate improvement
Small improvement
3
Large institution
Small institution
Large institutions overall
Large institution
Small institution
Small institutions overall
Large institution
Large institution
Small institution
Small institution
Small institution
Small institution
Small institution
Large institution
Small institution
Large institution
Large institution
Small institution
Small institution
Small institution
Small institution
Small institution
Large institution
Small institution
0.68
0.57
0.52
0.54
0.39
0.34
0.34
0.28
0.25
0.18
0.14
0.18
0.12
0.11
0.09
-0.05
-0.06
-0.11
-0.62
-0.84
-0.78
-2.00
-2.49
-3.02
0.13
0.13
0.13
0.10
0.08
0.07
0.07
0.07
0.05
0.04
0.04
0.03
0.02
0.02
0.02
-0.01
-0.01
-0.02
-0.13
-0.15
-0.16
-0.42
-0.51
-0.52
No effective change
Small decline
Moderate decline
Large decline
Warren Wilson College’s First-Year Change in Academic Motivation
This scale measures students’ interest in working hard, getting good grades, and engaging challenging
intellectual material. The largest potential change is four points.
Institution Type
Mean Change from
Fall to Spring
Effect Size
Small institution
Small institution
Small institution
Small institution
Large institution
Large institution
Small institution
Large institution
Small institution
Large institution
Small institution
Small institution
Small institution
Large institution
Large institution
Small institution
Small institution
Small institution
-0.06
-0.06
-0.05
-0.09
-0.09
-0.10
-0.12
-0.13
-0.13
-0.14
-0.16
-0.14
-0.14
-0.16
-0.16
-0.15
-0.17
-0.20
-0.09
-0.10
-0.10
-0.16
-0.16
-0.16
-0.21
-0.22
-0.22
-0.24
-0.24
-0.25
-0.25
-0.26
-0.28
-0.29
-0.29
-0.32
Description of Effect Size
No effective change
Small decline
Moderate decline
Large institution
-0.18
-0.32
Small institutions overall
-0.18
-0.33
Small institution
-0.20
-0.34
Small institution
-0.20
-0.36
Large institutions overall
-0.21
-0.36
Small institution
-0.20
-0.37
Large institution
-0.21
-0.38
Small institution
-0.23
-0.38
Large institution
-0.23
-0.39
Small institution
-0.23
-0.40
Large institution
-0.24
-0.40
Small institution
-0.24
-0.40
Small institution
-0.23
-0.42
Small institution
-0.25
-0.43
Small institution
-0.24
-0.43
Small institution
-0.26
-0.45
Warren Wilson College
-0.25
-0.46
Large institution
-0.27
-0.46
Small institution
-0.25
-0.46
Large institution
-0.26
-0.46
Small institution
-0.25
-0.47
Small institution
-0.26
-0.49
Large institution
-0.30
-0.50
Small institution
-0.31
-0.54
Small institution
-0.33
-0.58
Large decline
Large institution
-0.33
-0.63
Large institution
-0.36
-0.64
For more detail on how your students changed on our outcome measures, please see Tables 11–15 in the
Wabash National Study first-year report data tables.
Section 2 – Summary of Student Experiences In and Out of the Classroom
In our research thus far, we have identified four broad categories 6 of teaching practices and institutional
conditions that promote student growth on a wide variety of qualities, including critical thinking, moral
reasoning, leadership, openness to diversity and challenge, political and social involvement, and positive
attitude toward literacy. These good practices and conditions are:




Good teaching and high-quality interactions with faculty
Academic challenge and high expectations
Diversity experiences
Deep learning
We have subdivided each of these four good practice areas into clusters of related activities; for example,
good teaching and high-quality interactions with faculty includes experiences like receiving prompt feedback
and having teaching that is clear and organized.
The table below lists our groups of good practices and indicates whether your students reported
experiencing higher, lower, or similar levels of these practices and conditions compared to students at other
small Wabash National Study institutions.7 The table also shows the proportion of students who reported
experiencing high levels of these good practices.8
Level of High-Impact Experiences Reported by Warren Wilson Students
Comparison to
Students at Other
Small WNSLAE
Institutions
Similar
% of Warren Wilson
Students who
Reported High
Levels
36%
Faculty interest in teaching and student development
Similar
44%
Prompt feedback
Similar
15%
Quality of nonclassroom interactions with faculty
Similar
28%
Teaching clarity and organization
Lower
33%
Academic Challenge and High Expectations
Lower
8%
Academic challenge and effort
Lower
0%
Frequency of higher-order exams and assignments
Lower
24%
Challenging classes and high faculty expectations
Similar
19%
Integrating ideas, information, and experiences
Similar
32%
Diversity Experiences
Similar
6%
Meaningful interactions with diverse peers – I
Similar
5%
Meaningful interactions with diverse peers – II
Similar
15%
NSSE Deep Learning
Similar
22%
Higher-order learning
Lower
26%
Integrative learning
Similar
18%
Good Teaching and High-Quality Interactions with Faculty
Reflective learning
Similar
42%
Compared to students at other small colleges, Warren Wilson students reported similar levels of many of the
good practices, including faculty interest in teaching, challenging classes, diversity experiences, integrative
learning, and reflective learning. However, your students reported receiving fewer assignments that required
them to write or engage in more intellectually demanding activities such as synthesis, contrasting topics, or
pointing out the strengths and weaknesses of arguments. Warren Wilson students also reported having a
significantly lower academic workload—e.g., time spent on academic work, the number of books read, the
number of papers written—compared with students at other small schools. In fact, none of your students
indicated experiencing high levels of academic challenge and effort. Finally, Warren Wilson students' reports
of teaching clarity and organization were lower than those of students in the comparison group. In addition to
promoting growth on many of our outcomes, teaching clarity and organization promotes student persistence.
It is also useful to examine students' responses to individual questions about good practices to see how they
compare to students at other small institutions in the study. In the following tables, we highlight your
students’ responses on a few good practice questions.9
Most students at Warren Wilson said they often engaged in reflective learning activities such as trying to
understand someone else’s views or examining the strengths and weaknesses of their own views. Your
students reported these activities more often than students at the comparison institutions.
How often have you tried to better understand
someone else's views by imagining how an
Very Often
issue looks from his or her perspective?
Warren Wilson College
30%
Other 30 Small Institutions
23%
Often
Sometimes
Never
46%
21%
3%
41%
33%
3%
How often have you examined the strengths
and weaknesses of your own views on a topic Very Often
or issue?
Warren Wilson College
22%
Often
Sometimes
Never
45%
32%
1%
Other 30 Small Institutions
20%
37%
37%
6%
Warren Wilson students’ reports of teaching clarity and organization generally were lower than their peers at
the other small institutions. This was true for your students’ responses about how often course goals were
explained, whether assignments were helpful in learning the course material, and how often professors’
class presentations were well organized.
Course goals and requirements were clearly
explained.
Very Often
Often
Sometimes
Rarely
Warren Wilson College
21%
56%
19%
4%
0%
Other 30 Small Institutions
36%
47%
14%
2%
<1%
Very Often
Often
Sometimes
Rarely
Never
Warren Wilson College
12%
58%
26%
5%
0%
Other 30 Small Institutions
26%
50%
21%
3%
<1%
Very Often
Often
Sometimes
Rarely
Never
Warren Wilson College
15%
55%
27%
1%
1%
Other 30 Small Institutions
23%
54%
21%
2%
<1%
Faculty gave assignments that helped in
learning the course material.
The presentation of material was well
organized.
Only twelve percent of your students said Warren Wilson very much emphasizes spending time studying or
doing academic work. Furthermore, compared to students at the other small institutions, Warren Wilson
students typically spent fewer hours per week preparing for class.
Does your institution emphasize spending
significant amounts of time studying and on
academic work?
Warren Wilson College
Very Much
Quite a Bit
Some
Very Little
12%
43%
37%
8%
Other 30 Small Institutions
44%
43%
12%
1%
Although less than 25% of Warren Wilson students reported often discussing class ideas with faculty outside
of class, 78% said they often talked with peers or family about class ideas. The frequency of such peer
conversations was higher at Warren Wilson than at the other small institutions.
How often have you discussed ideas from
your readings or classes with faculty outside
of class?
Warren Wilson College
Other 30 Small Institutions
How often have you discussed ideas from
your readings or classes with others outside
of class (students, family members,
coworkers,etc.)?
Warren Wilson College
Other 30 Small Institutions
Very Often
Often
Sometimes
Never
1%
21%
57%
21%
9%
21%
47%
23%
Very Often
Often
Sometimes
Never
35%
43%
20%
3%
27%
40%
30%
3%
Never
Your students also were more likely than students at other small schools to say that their out-of-class
experiences have helped them translate classroom knowledge into action.
My out-of-class experiences have helped me
translate knowledge and understanding from
the classroom into action.
Warren Wilson College
Other 30 Small Institutions
Strongly
Agree
Agree
Neutral
Disagree
Strongly
Disagree
23%
45%
27%
5%
0%
16%
46%
29%
7%
2%
Section 3 – Time on Task
One experience that predicts growth on virtually all of our outcome measures is how much time students
spend preparing for class. Unfortunately, students typically do not spend as much time preparing for class
as we imagine. Fifty-four percent of students who participated in the Wabash National Study reported
spending 15 hours or less per week preparing for class. The figure below shows the amount of time your
students reported spending preparing for class compared to students at other study institutions.
Time Spent Preparing for Class
On average, Warren Wilson students reported spending less time preparing for class than students at other
Wabash Study institutions, with 47% indicating that they spent 10 hours or less per week preparing for class.
Section 4 – Binge Drinking
Our analysis of Wabash National Study data shows that binge drinking (defined as having five or more
drinks in one sitting) diminishes student growth during their first year of college. Specifically, we have found
that binge drinking is associated with significant reductions in the growth of socially responsible leadership,
openness to and interest in diversity, well-being, and academic motivation. We have also found that this
negative impact occurs regardless of the kinds of educational experiences that these students have when
they do not drink. Good practices do not appear to inoculate students against the negative effects of binge
drinking.
As the figure below shows, Warren Wilson students reported binge drinking slightly less than students at
other Wabash Study institutions, however 36% of your students still said that they binge drink at least once a
week.
Frequency of Binge Drinking
Section 5 – Variation within Warren Wilson College
Much of this overview has focused on how Warren Wilson students compare to their peers at other small
institutions in the Wabash National Study. While those comparisons can be useful, it is also critical to
examine the variation within Warren Wilson College. The box plot below highlights the range and variation of
students’ scores within the institutions in the Wabash National Study.
Range of Good Teaching and High-Quality Interactions with Faculty across all Wabash
National Study Institutions
In the box plot, the top horizontal line indicates the score at the 95th percentile, and the lowest horizontal
line the score at the 5th percentile. The top of the rectangular box represents the 75th percentile, the
horizontal line in the middle represents the median or 50th percentile.
The box plots illustrate a key point—across virtually all of our measures, the differences among students at
your institution is much larger than the typical difference between your institution and any other institution in
the study. There are students at your institution who are having experiences as good as those of the best
students at any other institution. The opposite is also true. There is no singular “Warren Wilson College
experience.” Some Warren Wilson students are having a terrific experience, and some are not. The same is
true at every institution in the study.
It is important to explore the range of student experiences within Warren Wilson College. Which students are
experiencing the highest and lowest levels of the good teaching, academic challenge, or other important
experiences? How can you increase the number of students having high levels of these experiences? What
could you do to more deeply engage the students who report the lowest levels of these experiences?
Footnotes
change is calculated as the difference between students’ fall and spring scores. The descriptions
indicate the magnitude of the change according to the National Survey of Student Engagement’s revised
Cohen’s d interpretations: 0.1=small effect size, 0.3=medium effect size, 0.5=large effect size, 0.7=very
large effect size. (See “Contextualizing NSSE Effect Sizes: Empirical Analysis and Interpretation of
Benchmark Comparisons" at http://nsse.iub.edu/pdf/effect_size_guide.pdf.) For more detailed information
about outcome changes, please see Tables 11–15 in the Wabash National Study First-Year Report data
tables.
1First-year
Institutions were rank ordered from largest positive change to largest negative change for each outcome.
Institutions at the 100th percentile are at the top of this ranking, while institutions at the 0th percentile are at
the bottom.
2
Peer groups include institutions that entered the study in 2006, 2007, and 2008. The “small institutions
overall” group includes students from all of the liberal arts colleges: Allegheny College, Alma College,
Alverno College, Augustana College, Bard College, Bennington College, Blackburn College, Brandeis
University, Carleton College, Coe College, College of the Holy Cross, Columbia College, Connecticut
College, Drew University, Franklin College, Gustavus Adolphus College, Hamilton College, Hampshire
College (one cohort in 2006 and another in 2008), Hobart and William Smith Colleges, Hope College, Lasell
College, Marlboro College, New College of Florida, Oxford College of Emory University, Ripon College,
Wabash College (one cohort for each of the three years), Warren Wilson College, Wheelock College, and
3
Whittier College. The “large institutions overall” group includes students from all of the research universities,
regional universities, and community colleges: Butler University, Delaware State University, Fairfield
University, Ivy Tech Community College, Kirkwood Community College, Millersville University, North
Carolina Agricultural and Technical State University, Salem State College, San José State University,
University of Kentucky, University of Michigan, University of North Carolina Wilmington, University of Notre
Dame, University of Rhode Island (one cohort in 2007 and another in 2008), and Worcester Polytechnic
Institute.
4The
effect sizes listed in the tables are Cohen’s d values. These are calculated as the mean score from the
spring minus the mean score from the fall, divided by the pooled standard deviation of the fall and spring
data.
5The
descriptions of the magnitude of the effect sizes are based on the National Survey of Student
Engagement's revised Cohen's d categories, as follows: 0.1=small effect size, 0.3=medium effect size,
0.5=large effect size, 0.7=very large effect size.
6These
four categories were derived from survey questions in the Wabash National Study about student
experiences. The questions come from both the Student Experiences Survey and the National Survey of
Student Engagement (NSSE). The deep learning questions all come from the NSSE.
7Comparisons
are the difference between your students’ average spring score and the average spring score
of their peers at other small WNSLAE institutions. When the difference is positive and statistically significant
at or below the 0.05 level, it is classified as “higher.” When the difference is negative and statistically
significant at or below the 0.05 level, it is classified as “lower.” Differences that are not statistically significant
at or below the 0.05 level are not considered large enough to indicate variability and are called “similar.”
8 We
classify students with benchmark scores of 75 or greater on a good practice as experiencing high
levels of that good practice. We calculate benchmark scores on a 100-point scale. A benchmark score of
100 means that a student gave the highest possible response on every question on a scale. For example, a
benchmark score of 100 on the integrating ideas, information, experiences subscale would mean that
students gave the highest score, whether that is "strongly agree," "very often," or "very much," to the nine
questions on that scale. A benchmark of 67 on a good practice scale means that your students typically
choose "3" or "often" from four alternatives ranging from 1=never to 4=very often.
9
Percentages in these and other tables in the overview may not add to 100% due to rounding.