Warren Wilson College – Overview of First-Year Results Introduction This overview highlights findings for Warren Wilson College students in the first year of the Wabash National Study. Because this is an overview, it touches on a fraction of the information about your students’ responses included in the data tables and data files we sent you last fall. We hope you can use this overview to spark conversations about your students, and we hope it piques your curiosity about the more detailed information contained within the rest of your Wabash National Study first-year report. This report includes five sections: Section 1 – Summary of how much your students changed on some of our outcome measures. Outcomes are simply students’ skills and capacities—such as critical thinking, moral reasoning, or interest in contributing to the arts—that should be improved by their work at Warren Wilson College. Section 2 – Summary of your students' experience of Warren Wilson's teaching and learning environment. Although students’ reports about, for example, the degree to which they receive prompt feedback or the extent to which their assignments require them to integrate information are subjective—and may not align with faculty and staff perceptions of what happens in the classroom—they do matter. All of the student experiences we discuss in this overview predict growth on a variety of independent outcome measures. Section 3 – Summary comparison of your students' reports about how much time they spend preparing for class. Section 4 – Summary of your students' reports about their binge drinking. Section 5 – Short review of variability in student experiences within your institution. Although asking how student experiences at Warren Wilson College compare with those at other small colleges can be informative, it is far more important to ask about the range of student experiences within Warren Wilson. We look forward to working with you to review and clarify this information so that you can use it to benefit your students. Section 1 – Overview of Warren Wilson Students’ Growth on Outcome Tests and Surveys The Wabash National Study measures student development in twelve areas. The following table summarizes whether your students were more likely to improve, decline, or remain unchanged on these outcomes during their first year at Warren Wilson College. The table also compares the changes your students experienced to those of students at other small institutions in the study. 1 Students' Change on Wabash National Study Outcome Measures Outcome Critical Thinking Psychological Well-Being Size of First-Year Change at Warren Wilson College Size of First-Year Change at Small Institutions Overall Outcome Change Percentile Rank among All Institutions2 Moderate improvement No effective change 90 Small improvement No effective change 88 Openness to Diversity/Challenge Small decline Small decline 71 Socially Responsible Leadership No effective change No effective change 63 Contribution to the Arts No effective change No effective change 60 Political and Social Involvement Small decline Small decline 40 Positive Attitude Toward Literacy Small decline No effective change 29 No effective change No effective change 24 Small decline Small decline 24 Moderate decline Moderate decline 24 Small decline No effective change 22 Need for Cognition Contribution to the Sciences Academic Motivation Universality-Diversity Awareness Moral Reasoning No effective change Moderate improvement Overall, Warren Wilson students were somewhat less likely to grow on our outcomes than students at other Wabash Study schools. Your students grew on two outcomes over their first year of college: critical thinking and psychological well-being. On average, students at other small colleges did not change on either of these two measures. Like students elsewhere, Warren Wilson students did not change or declined on many of our outcome measures, including leadership, need for cognition, political and social involvement, and academic motivation. It is worth noting that your students did not gain on moral reasoning while students at other small colleges tended to show moderate improvement in this area. Additionally, Warren Wilson students declined more than students in the comparison group on positive attitude toward literacy and universality-diversity awareness. The outcome on which Warren Wilson students improved most was critical thinking. The outcome on which they declined most was academic motivation. The following tables include more detailed information about how your students changed on these two outcomes compared to students at other institutions in the study. Warren Wilson College’s First-Year Change in Critical Thinking The CAAP Critical Thinking Test measures students’ skills in clarifying, analyzing, evaluating, and extending arguments. The largest potential change is 40 points. Institution Type3 Large institution Small institution Small institution Small institution Warren Wilson College Small institution Small institution Small institution Small institution Small institution Small institution Large institution Small institution Large institution Large institution Small institution Large institution Small institution Mean Change from Fall to Spring Effect Size4 Description of Effect Size5 3.36 2.80 1.65 1.73 1.69 1.61 1.20 1.20 1.15 1.24 1.22 0.78 0.96 0.88 0.71 0.71 0.52 0.61 0.71 0.64 0.36 0.34 0.31 0.31 0.24 0.24 0.24 0.21 0.21 0.20 0.19 0.19 0.17 0.15 0.14 0.13 Very large improvement Large improvement Moderate improvement Small improvement 3 Large institution Small institution Large institutions overall Large institution Small institution Small institutions overall Large institution Large institution Small institution Small institution Small institution Small institution Small institution Large institution Small institution Large institution Large institution Small institution Small institution Small institution Small institution Small institution Large institution Small institution 0.68 0.57 0.52 0.54 0.39 0.34 0.34 0.28 0.25 0.18 0.14 0.18 0.12 0.11 0.09 -0.05 -0.06 -0.11 -0.62 -0.84 -0.78 -2.00 -2.49 -3.02 0.13 0.13 0.13 0.10 0.08 0.07 0.07 0.07 0.05 0.04 0.04 0.03 0.02 0.02 0.02 -0.01 -0.01 -0.02 -0.13 -0.15 -0.16 -0.42 -0.51 -0.52 No effective change Small decline Moderate decline Large decline Warren Wilson College’s First-Year Change in Academic Motivation This scale measures students’ interest in working hard, getting good grades, and engaging challenging intellectual material. The largest potential change is four points. Institution Type Mean Change from Fall to Spring Effect Size Small institution Small institution Small institution Small institution Large institution Large institution Small institution Large institution Small institution Large institution Small institution Small institution Small institution Large institution Large institution Small institution Small institution Small institution -0.06 -0.06 -0.05 -0.09 -0.09 -0.10 -0.12 -0.13 -0.13 -0.14 -0.16 -0.14 -0.14 -0.16 -0.16 -0.15 -0.17 -0.20 -0.09 -0.10 -0.10 -0.16 -0.16 -0.16 -0.21 -0.22 -0.22 -0.24 -0.24 -0.25 -0.25 -0.26 -0.28 -0.29 -0.29 -0.32 Description of Effect Size No effective change Small decline Moderate decline Large institution -0.18 -0.32 Small institutions overall -0.18 -0.33 Small institution -0.20 -0.34 Small institution -0.20 -0.36 Large institutions overall -0.21 -0.36 Small institution -0.20 -0.37 Large institution -0.21 -0.38 Small institution -0.23 -0.38 Large institution -0.23 -0.39 Small institution -0.23 -0.40 Large institution -0.24 -0.40 Small institution -0.24 -0.40 Small institution -0.23 -0.42 Small institution -0.25 -0.43 Small institution -0.24 -0.43 Small institution -0.26 -0.45 Warren Wilson College -0.25 -0.46 Large institution -0.27 -0.46 Small institution -0.25 -0.46 Large institution -0.26 -0.46 Small institution -0.25 -0.47 Small institution -0.26 -0.49 Large institution -0.30 -0.50 Small institution -0.31 -0.54 Small institution -0.33 -0.58 Large decline Large institution -0.33 -0.63 Large institution -0.36 -0.64 For more detail on how your students changed on our outcome measures, please see Tables 11–15 in the Wabash National Study first-year report data tables. Section 2 – Summary of Student Experiences In and Out of the Classroom In our research thus far, we have identified four broad categories 6 of teaching practices and institutional conditions that promote student growth on a wide variety of qualities, including critical thinking, moral reasoning, leadership, openness to diversity and challenge, political and social involvement, and positive attitude toward literacy. These good practices and conditions are: Good teaching and high-quality interactions with faculty Academic challenge and high expectations Diversity experiences Deep learning We have subdivided each of these four good practice areas into clusters of related activities; for example, good teaching and high-quality interactions with faculty includes experiences like receiving prompt feedback and having teaching that is clear and organized. The table below lists our groups of good practices and indicates whether your students reported experiencing higher, lower, or similar levels of these practices and conditions compared to students at other small Wabash National Study institutions.7 The table also shows the proportion of students who reported experiencing high levels of these good practices.8 Level of High-Impact Experiences Reported by Warren Wilson Students Comparison to Students at Other Small WNSLAE Institutions Similar % of Warren Wilson Students who Reported High Levels 36% Faculty interest in teaching and student development Similar 44% Prompt feedback Similar 15% Quality of nonclassroom interactions with faculty Similar 28% Teaching clarity and organization Lower 33% Academic Challenge and High Expectations Lower 8% Academic challenge and effort Lower 0% Frequency of higher-order exams and assignments Lower 24% Challenging classes and high faculty expectations Similar 19% Integrating ideas, information, and experiences Similar 32% Diversity Experiences Similar 6% Meaningful interactions with diverse peers – I Similar 5% Meaningful interactions with diverse peers – II Similar 15% NSSE Deep Learning Similar 22% Higher-order learning Lower 26% Integrative learning Similar 18% Good Teaching and High-Quality Interactions with Faculty Reflective learning Similar 42% Compared to students at other small colleges, Warren Wilson students reported similar levels of many of the good practices, including faculty interest in teaching, challenging classes, diversity experiences, integrative learning, and reflective learning. However, your students reported receiving fewer assignments that required them to write or engage in more intellectually demanding activities such as synthesis, contrasting topics, or pointing out the strengths and weaknesses of arguments. Warren Wilson students also reported having a significantly lower academic workload—e.g., time spent on academic work, the number of books read, the number of papers written—compared with students at other small schools. In fact, none of your students indicated experiencing high levels of academic challenge and effort. Finally, Warren Wilson students' reports of teaching clarity and organization were lower than those of students in the comparison group. In addition to promoting growth on many of our outcomes, teaching clarity and organization promotes student persistence. It is also useful to examine students' responses to individual questions about good practices to see how they compare to students at other small institutions in the study. In the following tables, we highlight your students’ responses on a few good practice questions.9 Most students at Warren Wilson said they often engaged in reflective learning activities such as trying to understand someone else’s views or examining the strengths and weaknesses of their own views. Your students reported these activities more often than students at the comparison institutions. How often have you tried to better understand someone else's views by imagining how an Very Often issue looks from his or her perspective? Warren Wilson College 30% Other 30 Small Institutions 23% Often Sometimes Never 46% 21% 3% 41% 33% 3% How often have you examined the strengths and weaknesses of your own views on a topic Very Often or issue? Warren Wilson College 22% Often Sometimes Never 45% 32% 1% Other 30 Small Institutions 20% 37% 37% 6% Warren Wilson students’ reports of teaching clarity and organization generally were lower than their peers at the other small institutions. This was true for your students’ responses about how often course goals were explained, whether assignments were helpful in learning the course material, and how often professors’ class presentations were well organized. Course goals and requirements were clearly explained. Very Often Often Sometimes Rarely Warren Wilson College 21% 56% 19% 4% 0% Other 30 Small Institutions 36% 47% 14% 2% <1% Very Often Often Sometimes Rarely Never Warren Wilson College 12% 58% 26% 5% 0% Other 30 Small Institutions 26% 50% 21% 3% <1% Very Often Often Sometimes Rarely Never Warren Wilson College 15% 55% 27% 1% 1% Other 30 Small Institutions 23% 54% 21% 2% <1% Faculty gave assignments that helped in learning the course material. The presentation of material was well organized. Only twelve percent of your students said Warren Wilson very much emphasizes spending time studying or doing academic work. Furthermore, compared to students at the other small institutions, Warren Wilson students typically spent fewer hours per week preparing for class. Does your institution emphasize spending significant amounts of time studying and on academic work? Warren Wilson College Very Much Quite a Bit Some Very Little 12% 43% 37% 8% Other 30 Small Institutions 44% 43% 12% 1% Although less than 25% of Warren Wilson students reported often discussing class ideas with faculty outside of class, 78% said they often talked with peers or family about class ideas. The frequency of such peer conversations was higher at Warren Wilson than at the other small institutions. How often have you discussed ideas from your readings or classes with faculty outside of class? Warren Wilson College Other 30 Small Institutions How often have you discussed ideas from your readings or classes with others outside of class (students, family members, coworkers,etc.)? Warren Wilson College Other 30 Small Institutions Very Often Often Sometimes Never 1% 21% 57% 21% 9% 21% 47% 23% Very Often Often Sometimes Never 35% 43% 20% 3% 27% 40% 30% 3% Never Your students also were more likely than students at other small schools to say that their out-of-class experiences have helped them translate classroom knowledge into action. My out-of-class experiences have helped me translate knowledge and understanding from the classroom into action. Warren Wilson College Other 30 Small Institutions Strongly Agree Agree Neutral Disagree Strongly Disagree 23% 45% 27% 5% 0% 16% 46% 29% 7% 2% Section 3 – Time on Task One experience that predicts growth on virtually all of our outcome measures is how much time students spend preparing for class. Unfortunately, students typically do not spend as much time preparing for class as we imagine. Fifty-four percent of students who participated in the Wabash National Study reported spending 15 hours or less per week preparing for class. The figure below shows the amount of time your students reported spending preparing for class compared to students at other study institutions. Time Spent Preparing for Class On average, Warren Wilson students reported spending less time preparing for class than students at other Wabash Study institutions, with 47% indicating that they spent 10 hours or less per week preparing for class. Section 4 – Binge Drinking Our analysis of Wabash National Study data shows that binge drinking (defined as having five or more drinks in one sitting) diminishes student growth during their first year of college. Specifically, we have found that binge drinking is associated with significant reductions in the growth of socially responsible leadership, openness to and interest in diversity, well-being, and academic motivation. We have also found that this negative impact occurs regardless of the kinds of educational experiences that these students have when they do not drink. Good practices do not appear to inoculate students against the negative effects of binge drinking. As the figure below shows, Warren Wilson students reported binge drinking slightly less than students at other Wabash Study institutions, however 36% of your students still said that they binge drink at least once a week. Frequency of Binge Drinking Section 5 – Variation within Warren Wilson College Much of this overview has focused on how Warren Wilson students compare to their peers at other small institutions in the Wabash National Study. While those comparisons can be useful, it is also critical to examine the variation within Warren Wilson College. The box plot below highlights the range and variation of students’ scores within the institutions in the Wabash National Study. Range of Good Teaching and High-Quality Interactions with Faculty across all Wabash National Study Institutions In the box plot, the top horizontal line indicates the score at the 95th percentile, and the lowest horizontal line the score at the 5th percentile. The top of the rectangular box represents the 75th percentile, the horizontal line in the middle represents the median or 50th percentile. The box plots illustrate a key point—across virtually all of our measures, the differences among students at your institution is much larger than the typical difference between your institution and any other institution in the study. There are students at your institution who are having experiences as good as those of the best students at any other institution. The opposite is also true. There is no singular “Warren Wilson College experience.” Some Warren Wilson students are having a terrific experience, and some are not. The same is true at every institution in the study. It is important to explore the range of student experiences within Warren Wilson College. Which students are experiencing the highest and lowest levels of the good teaching, academic challenge, or other important experiences? How can you increase the number of students having high levels of these experiences? What could you do to more deeply engage the students who report the lowest levels of these experiences? Footnotes change is calculated as the difference between students’ fall and spring scores. The descriptions indicate the magnitude of the change according to the National Survey of Student Engagement’s revised Cohen’s d interpretations: 0.1=small effect size, 0.3=medium effect size, 0.5=large effect size, 0.7=very large effect size. (See “Contextualizing NSSE Effect Sizes: Empirical Analysis and Interpretation of Benchmark Comparisons" at http://nsse.iub.edu/pdf/effect_size_guide.pdf.) For more detailed information about outcome changes, please see Tables 11–15 in the Wabash National Study First-Year Report data tables. 1First-year Institutions were rank ordered from largest positive change to largest negative change for each outcome. Institutions at the 100th percentile are at the top of this ranking, while institutions at the 0th percentile are at the bottom. 2 Peer groups include institutions that entered the study in 2006, 2007, and 2008. The “small institutions overall” group includes students from all of the liberal arts colleges: Allegheny College, Alma College, Alverno College, Augustana College, Bard College, Bennington College, Blackburn College, Brandeis University, Carleton College, Coe College, College of the Holy Cross, Columbia College, Connecticut College, Drew University, Franklin College, Gustavus Adolphus College, Hamilton College, Hampshire College (one cohort in 2006 and another in 2008), Hobart and William Smith Colleges, Hope College, Lasell College, Marlboro College, New College of Florida, Oxford College of Emory University, Ripon College, Wabash College (one cohort for each of the three years), Warren Wilson College, Wheelock College, and 3 Whittier College. The “large institutions overall” group includes students from all of the research universities, regional universities, and community colleges: Butler University, Delaware State University, Fairfield University, Ivy Tech Community College, Kirkwood Community College, Millersville University, North Carolina Agricultural and Technical State University, Salem State College, San José State University, University of Kentucky, University of Michigan, University of North Carolina Wilmington, University of Notre Dame, University of Rhode Island (one cohort in 2007 and another in 2008), and Worcester Polytechnic Institute. 4The effect sizes listed in the tables are Cohen’s d values. These are calculated as the mean score from the spring minus the mean score from the fall, divided by the pooled standard deviation of the fall and spring data. 5The descriptions of the magnitude of the effect sizes are based on the National Survey of Student Engagement's revised Cohen's d categories, as follows: 0.1=small effect size, 0.3=medium effect size, 0.5=large effect size, 0.7=very large effect size. 6These four categories were derived from survey questions in the Wabash National Study about student experiences. The questions come from both the Student Experiences Survey and the National Survey of Student Engagement (NSSE). The deep learning questions all come from the NSSE. 7Comparisons are the difference between your students’ average spring score and the average spring score of their peers at other small WNSLAE institutions. When the difference is positive and statistically significant at or below the 0.05 level, it is classified as “higher.” When the difference is negative and statistically significant at or below the 0.05 level, it is classified as “lower.” Differences that are not statistically significant at or below the 0.05 level are not considered large enough to indicate variability and are called “similar.” 8 We classify students with benchmark scores of 75 or greater on a good practice as experiencing high levels of that good practice. We calculate benchmark scores on a 100-point scale. A benchmark score of 100 means that a student gave the highest possible response on every question on a scale. For example, a benchmark score of 100 on the integrating ideas, information, experiences subscale would mean that students gave the highest score, whether that is "strongly agree," "very often," or "very much," to the nine questions on that scale. A benchmark of 67 on a good practice scale means that your students typically choose "3" or "often" from four alternatives ranging from 1=never to 4=very often. 9 Percentages in these and other tables in the overview may not add to 100% due to rounding.