BEYOND THE CLASSROOM REPORT 2013-2014 DIVISION OF UNIVERSITY AFFAIRS STEPHEN F. AUSTIN STATE UNIVERSITY Table of Contents Introduction .................................................................................................................................................. 2 Mission .......................................................................................................................................................... 2 Methodology................................................................................................................................................. 2 Program Learning Outcomes ........................................................................................................................ 3 Division of University Affairs ......................................................................................................................... 3 Ethics and Values .......................................................................................................................................... 3 Retention of First-Year Students................................................................................................................... 4 Key Performance Indicators .......................................................................................................................... 5 Saliency of Program Learning Outcomes ...................................................................................................... 6 Our Engagement Model .............................................................................................................................. 10 Our Shared Values: The SFA Way Honor Code ........................................................................................... 12 ACE Workshops ........................................................................................................................................... 16 Week of Reflection...................................................................................................................................... 17 2013 Omicron Delta Kappa Leadership Conference ................................................................................... 18 Freshman Leadership Academy .................................................................................................................. 19 Skills and Qualities Desired by Employers .................................................................................................. 20 The Pine Log………………………………………………………………………………………………………………………………………….24 Health Clinic…………………………………………………………………………………………………………………………………………..26 Career Services………………………………………………………………………………………………………………………………………27 Closing the Loop…………………………………………………………………………………………………………………………………….28 Page | 1 Introduction The Division of University Affairs at Stephen F. Austin State University (SFASU) is composed of nineteen departments, including: Campus Recreation, Divisional Operations, Office of the Dean of Student Affairs, Counseling Services, Career Services, Testing Services, Disability Services, Health Services, Veterans’ Resource Center, Student Rights and Responsibilities, Multicultural Affairs, Orientation Programs, Student Affairs Programs, Student Services, Residence Life, Baker Pattillo Student Center, Student Publications, University Police and the Office of the Vice President for University Affairs. Mission The Division of University Affairs supports SFA's academic mission by providing our students with co-curricular and extracurricular PROGRAMS that create opportunities for students to engage in active and high-impact learning; we design and produce ACTIVITIES that strengthen students' connections to the university and to each other; we provide targeted SERVICES that support our students' pursuit of their personal and academic goals, and we develop and maintain the FACILITIES necessary to fulfill this mission. Methodology Data are collected through various means throughout the Division of University Affairs. These include surveys, focus groups, and rubric analyses of artifacts of learning, pre T test/post T test and mixed methods projects. All programs are required to develop and test learning outcomes, and each major program is required to assess these learning outcomes each year. Additionally, all departments must assess all programs at least once every three years. All learning outcomes are derived by and connected to the mission of University Affairs and of Stephen F. Austin State University. This mission is further articulated through nine program learning outcomes. These are: Citizenship, Communication, Cultural Competence, Ethics and Values, Leadership, Life Skills, Self Knowledge, Scholarship & Inquiry, and Wellness. These have been developed to express the kind of knowledge and skill our programs are meant to develop. The division has also advanced four Key Performance Indicators. These are measured in all areas. These performance indicators are driven from the mission of university affairs and help to determine the effectiveness of the division’s programs in meeting this mission. These include measuring students’ commitment to persistence, awareness of resources, commitment to diversity and awareness of co-curricular learning. Page | 2 Program Learning Outcomes Division of University Affairs Communication Citizenship Cultural Competence Ethics and Values Leadership Life Skills Self-Knowledge Scholarship & Inquiry Wellness Page | 3 Students gain skills to help them to communicate effectively and professionally, using written, oral, and artistic means, as well as current technology. Students learn to listen actively and effectively, to read with comprehension, to evaluate ideas, and to form meaningful interpersonal relationships. Through civic engagement, community service and classroom experiences, students acquire a contextually rich understanding of complex issues and increased sense of social responsibility. Students are prepared for the challenges and opportunities of a democratic society, and to fulfill their duties as citizens of the state, nation, and world. By interacting with others who are different from themselves, students gain an increased appreciation for diversity and attain valuable skills for success in a pluralistic society. Beyond tolerance, students increase their understanding of the beliefs, values and worldviews of others. Students define, refine and clarify their values, learn to keep congruence between these beliefs and their actions and to balance their personal interests with the interests of others and of society as a whole. Through a variety of experiences, students will learn to maximize their own individualized talents and skills to collaborate with others and to motivate them toward a common goal. From maintaining and balancing a budget to effectively managing one's time, students learn to build strategies for leading a productive life in college, in graduate school, in their career and beyond. Through experiences and reflection, students discover a sense of purpose, gain the ability to realistically appraise themselves enhance self-esteem and to become increasingly independent and interdependent. These skills are useful in picking a career and in helping the student to set and meet both educational and personal goals. A broad-based educational foundation allows student to think critically, creatively and independently and to experience intellectual growth. Students become committed to better understanding and to advancing the knowledge of their chosen field they are prepared for the rigors of graduate study, for their careers, and for lifelong learning. Students learn to practice healthy behavior to attain a satisfying and productive lifestyle, learning to balance their physical, emotional and spiritual well-being with the demands of their life as a student and beyond. Retention of First-Year Students In the University’s Strategic Plan 2013: Preparing for the Future, a goal was established to have first-year retention at 70% by Fall of 2012 (Initiative 3: Strategy 3). This was a very bold goal. At the time, the university’s retention rate was approximately 63%. In the fall of 2013, we have finally reached the 70% goal. While many factors contributed to this change, it is clear that raising admission standards has made a significant impact. It should not be overlooked, however, that raising admission standards tends to impact whether a student may be unable to continue based on poor academic performance. It does not impact whether students want to stay at the university when they have that chance. Many programs and services of the Division of University Affairs have significantly impacted the engagement of students within the university, and, by extension have impacted the first-time/full-time retention rate. Retention of First-Time/Full-Time Students 72 70 68 66 64 62 60 Column1 Page | 4 2010-11 2011-12 2012-13 2013-14 64.1 63.4 67.3 70 Key Performance Indicators Questions related to our Key Performance Indicators are embedded in each Campus Labs survey. This produces data from a wide variety of students who answer these questions in a number of different contexts. Last year, more than 3,750 students responded to these questions. The tremendous diversity within surveys makes duplication less likely. Additionally, there are at least 1,000 unique respondents which can produce a confidence level of 97%. 5 4.8 4.7 4.6 4.8 4.8 4.7 4.7 4.6 4.71 4.6 4.5 Indicator #4 4.4 4.3 4.33 4.3 4.2 4.2 4.1 Indicator #3 Indicator #2 4.1 Indicator #1 4 3.8 3.6 2010-2011 2011-2012 2012-2013 2013-2014 Indicator # 1: I am committed to completing my degree at SFA. Indicator #2: It's important to interact with others who are different from me such as those of different ethnicities, cultural backgrounds, or abilities. Indicator #3: When I have questions about SFA, I am usually able to easily resolve them or find someone to help me find the answer. Indicator #4: I have gained skills through student organizations, activities, and other cocurricular experiences that have helped me to understand what I am learning in the classroom. Analysis: The dramatic drop in indicator three is the most disconcerting element of our Key Performance indicators project. This indicator, however, is still relatively high. Indicator four, which is related to how aware students are of what they are learning in co-curricular programs all decreased slightly (continuing a three-year downward trend). While possible explanations are highly speculative, it is interesting to note that this drop co-occurred with a more than a 33% decrease in Peer Involvement Advising sessions. The Involvement Center is a key resource in informing students about programs that can assist them and frames cocurricular experiences as learning opportunities rather than just ways to spend ones time after class. Page | 5 Saliency of Program Learning Outcomes The nine program outcomes were established in 2008 and have been measured each year on the Beyond the Classroom survey. This year, 1,202 students completed the survey. The figures represent the mean score for the statements below. Respondents: 1,320 students in 2014 1,202 students in 2013 1,187 students in 2012 1,027 students in 2011 1,089 students in 2010 1,161 students in 2009 “Participating in co-curricular activities has given me skills that I can use throughout my life.” “As a result of participating in cocurricular activities, I feel that I am better at evaluating information to determine what I believe.” “As a result of participating in cocurricular activities, I believe that I better understand the cultures of others.” Page | 6 “As a result of participating in cocurricular activities, I feel better able to communicate my thoughts and ideas to others.” “As a result of participating in cocurricular activities, I pay closer attention to important issues facing my community, the nation or the world.” “As a result of participating in cocurricular activities, I more carefully consider the ethical implications of the decisions I make.” * “As a result of participating in cocurricular activities, I feel more capable of leading others.” Page | 7 “As a result of participating in cocurricular activities, I better understand my individual strengths and weaknesses.” “As a result of participating in cocurricular activities, I feel that I am better prepared to maintain a healthy lifestyle.” *This question was reworded in 2012. Analysis: Nearly all of our saliency measures were relatively unchanged this year with the exception of two. Ethics was considerably up (+.18) and wellness was somewhat down (-.11). Ethics is still slightly down from two years ago. Last year we focused on creating an SFA Way pledge that was used by Student Government Association. This year we are broadening that to all sponsored student organizations in student affairs. It will be interesting to see if this measure improves. Wellness is down .36 percent since 2012. It is hard to know to what this can be attributed, but it will be important to make staff across the division aware that we need to find ways to demonstrate learning in wellness across our programs. Page | 8 Three-Year Trends 5 4.5 Life Skills 4 Scholarship 3.5 Cultural Competency 3 Communication 2.5 Citizenship 2 Ethics 1.5 Leadership 1 Self Knowledge 0.5 Wellness 0 2012 2013 2014 Relative Position of Learning Outcomes 4.1 Life Skills 4 Scholarship 3.9 Cultural Communication 3.8 Citizenship Ethics 3.7 Leadership Self-Knowledge 3.6 Wellness 3.5 2014 Analysis: This year, we appear to have had an overall drop-off in the level at which students are aware they are learning from co-curricular programs. However, the relative position of the learning outcomes appears to be very similar to recent years. Next year, we should focus on helping to bolster students’ understanding of what they are learning in programs. Additionally, while rephrasing the question related to ethics seems to make a difference, ethics would still be the lowest area of saliency in our learning outcomes. We need to continue to stress ethical issues in our programs. Page | 9 Our Engagement Model We have expanded our engagement model to capture engagement as a precursor to involvement, cognizant of the ways that these developmentally progress. For a student to become a leader, they must first become involved. For them to become involved, they must have become engaged through an awareness of a program and some sort of peripheral contact. The model below aligns data from the Beyond the Classroom survey to demonstrate our effectiveness in this key area. As we move forward, we would like to increase the number of students who identify as involved and as leaders. Page | 10 - A popular cheer at athletics events is “Give ‘em the axe, big ‘Jacks!” Two years ago, we took that cheer quite literally and gave nearly 2,400 axe handles to our first-year students. Additionally, we started a new tradition of allowing students the opportunity to run out with the team at football games carrying their axe handles. DROP IN PIA Sessions Page | 11 Personalizing an axe handle has long been a tradition on our campus. We believed that by giving students an axe handle we could connect them to the university’s history but also allow them to personalize the experience to reflect their own personality and values. This section looks at the impact that these axe handles have on the engagement of our students. Our Shared Values: The SFA Way Honor Code A key resource in improving students’ awareness of ethical considerations is The SFA Way. For the past two years, we have measured the number of students who can correctly identify the “five root principles” of The SFA Way. We also looked at how involvement in various student organizations related to knowledge about our honor code. Page | 12 Non-cognitive Factors in Student Success One of the primary factors we believe to be salient in student success is Locus of Control, which refers to the extent to which individuals believe they can affect the factors that lead to their success. Students with an internal locus of control believe that their hard work, skills and abilities will lead to success. Students with an external locus of control tend to ascribe success to luck, fate or powerful others. Locus of Control was assessed asking students to rate their agreement with the question, “One of the most important factors in success is good luck.” Agreement with this statement would tend to indicate an external locus of control, whereas disagreement would tend to indicate a more external locus of control. The data were analyzed by ethnicity and some trends were apparent. There appears to be a strong connection between ethnicity and locus of control. For example, African American students were 9% higher than White students and 4% higher than Hispanic students in their likelihood to agree or strongly agree with the statement that luck was an important factor in success. Hispanic students were 5% higher than White students in their agreement. One interesting note is that the percentage of students whose answer indicates an externalized locus of control increased from 17% last year to 20% this year. This is surprising given the impact of raised admission standards has had on any number of other outcomes for first-year students. Page | 13 Family Support Mean 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Family cares how I do in school Family is encouraging me Family expects me to fail Adequate financial support from family White 3.92 3.9 1.37 2.79 African American 3.94 3.86 1.38 2.55 Hispanic 3.82 3.79 1.6 2.69 Commitment Mean 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Likely to transfer Not really sure why I'm attending SFA Confident I can graduate White 1.7 1.13 3.75 African American 1.9 1.14 3.84 Hispanic 1.91 1.17 3.69 Page | 14 Involvement Mean 4 3.5 3 2.5 2 1.5 1 0.5 0 Intend to get involved Want to be a leader Expect to learn from involvement Best to wait to get involved White 3.49 3.01 3.5 2.78 African American 3.5 3.14 3.58 3.08 Hispanic 3.52 3.2 3.55 3.01 Academics Mean 4 3.5 3 2.5 2 1.5 1 0.5 0 Certain of major choice Fail to turn in assignments in HS Procrastinate Expect to make As and Bs White 3.52 1.51 2.12 3.62 African American 3.49 1.72 2.11 3.66 Hispanic 3.57 1.66 2.08 3.68 Page | 15 ACE Workshops In response to the challenges faced by first-generation students, we developed a program called, “Achieving College Excellence” (ACE). First-year students are recruited during orientation. Each year, between 350 and 450 students attend. There are workshops on ACEing College Reading, ACEing College Math and ACEing College Study Skills. Additionally, we have a presentation on Making Success a Choice (intended to combat external locus of control), and we invite a faculty member who has overcome significant odds to achieve their education as a keynote speaker. This program is assessed by sending a survey to students who sign up for the program and to students who follow through and attend. We selected a cohort of students who indicated that neither parent had completed college and who could be verified as having completed the pre-test, attended the program and completed the post-test (N=17). A paired t-test was used to compare their answers before and after the educational intervention. The program seems to be effective in changing mindsets and increasing motivation. The data below demonstrate that the program makes an impact in a short period of time. We selected a cohort of students who could be verified as having completed the pre-test, attended the program and completed the post-test (N=50). A paired t-test was used to compare their answers before and after the educational intervention. First-Generation Students PRE 29% neither parent had completed a two-year degree or higher. POST 25% neither parent had completed a two-year degree or higher. I am confident that my study skills will be sufficient for succeeding in college. Pre Post 3.72 4.31 Significance P=0.00 Confidence 95% I am confident that my reading skills will be sufficient for succeeding in college. Pre Post 4.21 4.43 Significance P=0.06 Confidence 90% I am capable of succeeding in college. Pre 4.68 Page | 16 Post 4.88 Significance P=0.031 Confidence 95% Week of Reflection The Week of Reflection is an annual program that encourages students to think about what they are learning at SFA and how it is changing them. Through interactive events and an annual keynote address from a prominent public figure, students are challenged to think beyond consumerist notions of what education is for. Additionally, the program is intended to spark a joy for learning that can be a catalyst for internal motivation for learning and provide an antidote for external notions of locus of control. Students receive a reflection guide at each event to help prompt their learning. This past year, 92% of the students who attended events during the Week of Reflection and who responded to our survey said that they had taken time to read and think about our reflection questions. The program includes six Learning and Development outcomes… Participants will think about what they’ve learned and how it is changing them. Participants will indicate that the program made them think about their learning in a new way. Participants will express an increased commitment to their personal goals. Participants will express an increased commitment to their educational goals. Participants will express an increased desire to learn from their classes. Participants will express an increased desire to learn from their cocurricular experiences. Results Learning and Development Outcome The participant will think about what they’ve learned and how it is changing them. The participant will indicate that the program made them think about their learning in a new way. The participant will express an increased commitment to their personal goals. The participant will express an increased commitment to their educational goals. The participant will express an increased desire to learn from their classes. The participant will express an increased desire to learn from their cocurricular experiences. Page | 17 Mean in 2013 3.72 Mean in 2014 3.69 3.44 3.67 3.77 3.72 3.87 3.69 3.59 3.46 3.74 3.62 Qualitative Data • The program taught me that there is a lot to look at deep under the surface of everything that may come my way. • I learned that it is good to take time and reflect on your life. It prepares you for your future and allows you to make changes for the better! • I work so hard that I get lost in “busy.” Sitting back and seeing how much I have accomplished makes me tear up. 2013 Omicron Delta Kappa Leadership Conference Each year, the SFA Circle of Omicron Delta Kappa puts on the Fall Leadership Conference. This is a key element in the common intellectual theme. Last year, the theme was “Civility and The SFA Way.” More than 100 students attended the conference. Learning and Development Outcome I learned something at this event which will improve my leadership skills. Mean 4.51 Events like this one make me want to get more involved or stay involved on campus. What I learned from this event connects to what I am learning or expect to learn in my classes. 4.40 Page | 18 4.14 Please provide details of what you learned that relates to improved leadership skills: Andrew Slack (keynote speaker) inspired me to lead with more love. Being courageous, and be willing to step up, take the lead and make a difference. I feel like I was able to see my strengths together with my future goals. I realize why I want to do what I want to do, because of my strengths. I learned about my five strengths that I received when taking the strengths quiz. I believe that this event readdressed my strengths and motivated me more to offset my strengths even more. I learned that what I thought were my flaws are actually my strengths and how to use them to the best of my ability. Freshman Leadership Academy The Freshman Leadership Academy expanded to four sections of SFA 101 this year. Since the program’s inception in 2004 as a student leadership retreat, it has grown each year. The class planned and executed a high-impact project. They studied the issue of the plight of Rroma people in Romania, raised $2500 to start a jobs program for children preparing to leave the orphanages. Learning and Development Outcome I have gained skills that will help me lead others more effectively. I have made at least one friend in this class whom I plan to stay connected with afterwards. I know more about resources available to me as a student at SFA. I had the opportunity to put my leadership skills into practice. I have learned about my own individual leadership style. I better understand how to be an ethical leader. I am committed to The SFA Way. I have discussed concepts from the class with someone unrelated to the class (parents, friends from home, SFA students who not in the class). Page | 19 Mean 4.28 4.63 4.22 4.13 4.15 4.12 4.38 4.06 Skills and Qualities Desired by Employers Each year, the National Association of Colleges and Employers (NACE) conducts a “Job Outlook Survey” asking employers to rank the skills they find most desirable when hiring college graduates. This year, in the Beyond the Classroom survey, we asked involved students if they were learning these skills, and if so, if they were learning them from their classes or through cocurricular programs. Of the 1,202 students who completed the Beyond the Classroom report, 710 (59%) reported that they had participated in a student organization in the previous year. This group’s responses were analyzed as to whether or not they had gained the skills employers most desired in the NACE survey. In the past, students’ experiences outside of the classroom were often referred to as “extra-curricular” programs. The results make a strong case for the newer term, “cocurricular” for these experiences. In many cases, students are as likely (if not more likely) to gain the skills that employers value most through their cocurricular experiences. NACE Skills Desired by Employers/Source Work in Teams Class Cocurricular Not at all 68.15 Class 63.7 51.3 4.54 Page | 20 Verbally Communicate Cocurricular Not at all 53.98 6.11 Solve Problems Class Cocurricular Not at all Obtain/Process Info Class Cocurricular 85.83 73.7 46.2 33.89 4.07 Plan/Prioritize Work Class Not at all Cocurricular Not at all 3.15 Analyze Data Class Cocurricular Not at all 82.87 76.67 44.91 24.26 7.59 4.81 Knowledge of Job Class Cocurricula Not at all 74.35 Computer Proficiency Class Cocurricular 69.35 32.96 24.91 10.19 Write/Edit Reports Class Cocurricular Not at all 22.87 19.07 Sell/Influence Others Class 50.65 81.02 Page | 21 Not at all Cocurricular Not at all 49.17 18.8 9.44 Page | 22 Analysis: In a time in which confidence in higher education is often waning, two observations jump out from this data. First, students strongly communicate that they are gaining the skills that employers say they want the most. There is also a clear, direct relationship between level of involvement and attaining these skills. More students who are involved say they are gaining these skill from their involvements outside of the classroom than students in general, and a greater percentage of student leaders say they are gaining these skills compared to both involved students and students in general. This direct relationship makes a strong case for the role of cocurricular involvement in the learning and development of students in college. Page | 23 During the past few years, readership of The Pine Log has dropped precipitously. Along with that finding has come a decrease in the numbers of students who believe the publication is relevant to them. As a result, this fall The Pine Log will move to once-weekly publication with an increased emphasis on original content stories posted to a re-designed and re-energized web site. To drive readers to the publication, a social media editor is being added to the staff. Among the responsibilities of this position is utilizing various forms of social media to convince the campus community of the newspaper's relevance to their daily lives. A new program utilizing members of campus organizations to help distribute The Pine Log to the campus community is also in the works. Page | 24 Readership of The Pine Log 100 50 0 2009 2010 2011 Read All/Most 2012 2013 2014 Read Occasionally On a positive note, students who do regularly read The Pine Log say that it makes them more informed about what is going on at the university and slightly less than half say that it makes them engage more actively in events and programs on campus. Nearly a third of students say that it also informs them of national and international issues. 90 80 70 60 50 40 30 20 10 0 82.59 28.98 10.37 Know more about what's going on at SFA More actively engage in campus events/programs Know more about national and international issues Know more about what's going on at SFA More actively engage in campus events/programs Know more about national and international issues Page | 25 Analysis: Last year, the Health Clinic began billing student insurance. Students who were not insured continued to be seen in the Health Clinic. We were concerned that students might mistakenly believe that if they did not have insurance that they would not receive care in the Health Clinic. We are pleased that the percentage of students who visited the health clinic rose. Page | 26 Career Services Career Services exists to empower students and alumni to achieve life-long career success. We fulfill this mission through individualized assistance, diverse career development programs, and collaboration with both internal and external partners concentrated on career goal achievements. Page | 27 In the 2012-2013 Beyond the Classroom report, we published data that demonstrated that readership of The Pine Log student newspaper was declining. In response, the Office of Student Publications employed a number of strategies to improve. The newspaper was reduced from twice weekly to once per week. Additionally, emphasis was placed on improving the online and social media presence of the paper. Data published in this year’s Beyond the Classroom report demonstrate that this approach is working and should continue to develop in the coming year. For many years we have collected data on the percentage of students who engage in cocurricular opportunities. We have identified a concern that we lacked capacity to increase the number of involved students. In response, we created a new sponsored student organization called, “Purple Haze Association.” This organization was designed to take as many students as desired to participate. In the second year of existence, the organization now has more than 300 active members. In the 2011-2012 Beyond the Classroom report, we reported a first-time, full-time retention rate of 63.4%. The Freshman Leadership Academy provides a significant means of engagement and provides first-year students with high impact experiences. Research has demonstrated that participation in high-impact experiences tends to lead to greater persistence. In response, we nearly doubled the size of Freshman Leadership Academy from two sections with a total of 46 students in 2012 to four sections with a total of 83 in 2013. The impact of this growth on retention of participants will continue to be studied this year. Each year Campus Recreation sends a user satisfaction survey to all students as well as faculty, staff and alumni members. The survey focuses on a wide variety of items from programs offered to the cleanliness of the facility. A resounding response was noticed in regards to patron satisfaction of facility hours. After looking deeper into the specific comments it was determined that the previous opening time of noon on Saturdays seemed to be the main issue. After looking at the pros and cons of changing the time, it was decided that a test period would be used to see if more participation occurred and if positive feedback would follow. Following the change of opening the facility at 10a, we noticed a significant increase of patrons using the gym from 10 a.m. to noon. We have since made the change permanent and have worked to find other creative ways to work within our budget while providing extra hours to our constituents. Club Sports added leadership workshop opportunities because of the amount of students who responded to their survey that said leadership was a primary motivation for joining sport clubs. An important institutional goal has been to improve the persistence of first-year students. Internal research has shown that the retention of historically under-represented groups has been significantly below that of other students. In response, the Office of Multicultural Affairs Page | 28 created the AxCel Mentorship program last year. More than 100 students participated as mentees and 20 students served as mentors. Page | 29