Summary of Residence Life Programming Foundations of Excellence: Improvements Prepared by Todd Thomas and Spencer Walrath Background Ninety percent of UNI freshmen live in residence halls. Consequentially, Residence Life programming is tailored to freshmen and specifically to those living oncampus. Approximately 400 out of the ~1,500 freshmen live in a Springboard house, which are houses exclusive to freshmen. The Department of Residence’s Residence Life staff employ a relational model for teaching eight core competencies and information to all students living on campus, as opposed to an academic/formalized structure. This relational model involves Residence Life staff personally interacting with students in an effort to convey programming concepts and accomplish the goals of Residence Life, as opposed to an academic/formalized structure of regularly meeting in a classroom or testing information conveyed. Most internal assessments by Residence Life check to see if Residence Life staff and programs are perceived to be challenging and supportive in the eight primary competencies identified as important by programming staff. The eight core competencies taught through Residence Life Programming are: Citizenship Health- Keep mind, body and spirit as healthy as possible Diversity- Understand and communicate well with different kinds of people Management- Manage money, time and other resources well Service- Contribute to house, campus, city, and the world Scholarship Academics- Acquire successful academic skills and attitudes Uniqueness- Explore what makes them unique Major/Minor- Find the major, minor and programs that match unique talents, interests and backgrounds Deadlines- Meet degree deadlines and requirements The eight competencies above represent an internal curriculum developed by Residence Life staff through years of feedback, discussion, assessments, literature, and recommendations by peer institutions. Off-campus students and non-traditional students living in University Apartments may utilize Residence Life programming by participating in events, reading material created by Residence Life staff, and purchasing Panther Planners created by the 2 Department of Residence. Residence Life staff also collaborate with a variety of other campus departments to address issues that impact both on and off-campus students. For example, the Residence Life staff provides the Springboard curriculum which was designed just for on-campus freshmen and making that available before the start of the fall semester to all UNI freshmen. Professional Development and Involvement Professional Residence Life staff attend the Association of College and University Housing Officers (ACUHO) conferences once or twice a year, which includes programming sessions. Every year professional staff take away a variety of ideas and materials for use in the current or upcoming school year’s programming. Students in Residence Life attend the National Association of College and University Residence Halls (NACURH) conferences two to three times a year for programming and leadership, sharing assessment information, or learning assessment information from other institutions. Most professional development activities are related to student development, which includes residence programming. UNI has also given a variety of presentations at national and regional conferences, such as information on the PAIR program, the Citizens and Scholars model, and diversity programming. Several presentations have been given special awards and UNI’s Residence Life program serves as a model for many other colleges and universities throughout the nation. Current Assessment Inventory Residence education occurs through three primary methods: Relationship building/teaching Passive information (bulletin boards, fliers, e-mails, and web resources) Programs and events (house, hall, or campus-wide) Regardless of the method in which education takes place, Residence Life either utilizes or has access to the following assessments. For an additional evaluation of each assessment, review Appendix A: Residence Life Programming Evidence Evaluation Matrix. Each evidence item below is listed along with a brief overview. All assessments listed are done annually unless specifically noted. Department of Residence Formal Assessments The five assessments below utilize a formalized, structured assessment based on data collection and analysis. Association of College and University Housing Officers – International/Educational Benchmarking, Inc. (ACUHO-I/EBI) Resident Assessment 3 This assessment provides insight into resident perceptions of satisfaction and is used as a benchmarking indicator, comparing resident perceptions to previous years and other peer institutions, as well as analysis and cross-tabulation or filtering on data. House surveys Each student in the residence halls has an opportunity to complete a House survey, asking questions related to Resident Assistant (RA) performance, programming, and hall resources (facilities, hall staff, etc.). House surveys all utilize a core set of questions and then optionally include hall-specific questions as determined by the hall’s Residence Life Coordinator (RLC). Immersion Project New for the fall of 2008 is the Immersion Project. Residence Life programming staff identified several key topics that would be helpful for students and then immersed them in information about that topic through events, bulletin boards, fliers, table tents, electronic media (myUNIverse News, e-mail, etc.) and more. Some sample topics would include friendship in August/ September, finals in December, etc. A pre-test before starting the Immersion Project was created and administered in the spring of 2008. A post-test will be conducted in early April to measure whether or not the immersive programming efforts made an impact on students. Peer Advisor In Residence (PAIR) survey The PAIR program provides student academic advisors in residence halls and is coordinated through the office of Academic Advising. This program helps fulfill the Scholarship competency of Residence Life programming and offers academic help to students. An annual survey of the PAIR program and its advisors is administered in the spring through Residence Life. Questions are created with input from Residence Life and Academic Advising. Springboard Dive-In Days survey Incoming freshmen living in freshmen-only Springboard houses have had the option to arrive at UNI before the start of the fall semester and participate in Dive-In Days. DiveIn Days provide freshmen a chance to learn about additional college-related topics and meet with other students in their house. This assessment focused on the amount of value students perceived from each session attended and provided two general questions on the best part of Dive-In Days and how it might be improved. For fall of 2009, Dive-In Days will be opened to all freshmen regardless of whether a student resided in a Springboard house as well as being open to off-campus students. Dive-In Days will take place the Saturday prior to the start of school for the fall, so this assessment will change for the coming school year. 4 Department of Residence Informal Assessments The three assessments listed below utilize an informal, relational, and observational assessment based on the experience and expertise of Residence Life staff. Residence Life Coordinator (RLC) meetings with Residence Life staff RLCs meet with residence programming staff to discuss programming progress on the eight core competencies, develop strategies for improvements, and identify successes. During these meetings, the RLCs develop the desired outcome of programming efforts and the criteria used to evaluate the success of that programming in concrete and measurable ways. For instance, the desired outcome of an event may be to increase residents’ awareness of a particular topic. The criteria for success may be having a certain level of participation, having students attend a future event, etc. Resident Assistant (RA) meetings with RLC staff RLCs meet weekly or bi-weekly with the Resident Assistants (RAs) in their hall. Many topics are covered during these meetings, including Residence Life programming. President’s Council The President’s Council is a group of student representatives from each residence hall. It was formed as a feedback mechanism for the Department of Residence staff to bring forward ideas and gather opinions. Residence Life has utilized this group for input on programming efforts. External Formal Assessments Used by Residence Programming: The two assessments listed below utilize a formalized, structured assessment based on data collection and analysis and are administered by entities outside the Department of Residence. National Survey of Student Engagement (NSSE) The National Survey of Student Engagement (NSSE) has been administered through the Office of Academic Assessment since the spring semester of 2006. Prior to 2006, the College Student Experiences Questionaire (CSEQ) was employed. NSSE results are used, along with other data, for assessment of learning in the Liberal Arts Core at UNI. They also provide information on student experiences that can be useful to other program areas and departments at UNI. Some topics pertain to general student demographics, habits, and attitudes, which could apply to Residence Life programming. American College Health Association-National College Health Assessment (ACHA-NCHA) 5 UNI’s Wellness Education department utilizes the ACHA-NCHA survey conducted annually each spring. Residence Life is able to utilize general statistical information as well as attitudes of students that may impact programming efforts. Health-related assessment information is applicable since health is one of the core competencies for Residence Life programming. Analysis Residence Life currently utilizes a variety of methods for assessing first year students at UNI, both within the Department of Residence (internally) and through other University departments (externally). Surveys, such as ACUHO-I/EBI Resident Assessment, cover many topics pertinent to Residence Life programming, but focus on satisfaction rather than actual behavior improvements. Asking students if their behavior has changed as a result of programming is the current method used for determining programming effectiveness and is addressed in a couple different surveys (the House survey and Immersion Project survey). This survey evaluates perceived changes rather than actual outcomes. The biggest challenge facing Residence Life is to not only use existing assessments but both to utilize that information in an applicable fashion and to create assessments that measure residents’ actual change in behavior. After analyzing the evidence provided and speaking with Residence Life staff, it is our recommendation that for Performance Indicator 9.1 “To what degree does Residence Life programming include systematic assessment?” be rated as Medium. Our recommendation for this is based around the following: Assessing the success of Residence Life programming efforts through surveys is difficult. Many informal assessments are done through RAs, students, and the expertise and experience of RLCs , which then should provide feedback to Residence Life staff regarding their programming effectiveness. Many surveys utilized by Residence Life have a small sub-set of application for programming efforts. While some surveys are highly systematic, they are not necessarily systematically assessing the information useful to Residence Life. Residence Life programming would like to assess whether behavior changed as a result of programming efforts, whereas several surveys ask whether a student is satisfied with how they received programming, which are two different types of assessment. A student may not be satisfied with programming but still change their behavior as a result of it, or vice-versa. Participation rates for some surveys (House survey and Dive-In Days) had poor and/or varied participation. This input makes utilizing these assessments difficult since the data collection may be skewed or an inaccurate representation of students. 6 Despite the difficulty in assessing programming effectiveness, the current assessment practices provide benefit to Residence Life and could continue to be refined through review by other departments and the Recommendations listed. It is our recommendation for Performance Indicator 9.2 “To what degree have assessment results been used to improve existing practices?” be rated as High. With a few exceptions, the assessments listed earlier are utilized regularly by Residence Life staff as a decision-making tool for improving programming. Some examples of findings made after reviewing current assessments include: ACHA-NCHA confirmed the continued need for alcohol education. House surveys identified that career-related issues could be improved and revisions will be made to improve passing along this information to students. The ACUHO-I/EBI survey identified improving the areas of managing time and related academic excellence. Changes in these areas include Houses recently being identified for positive academic achievement, the PAIR program’s responsibilities were changed, and grade-related bulletin boards were mandated. RLCs are given priorities gathered from the previous year's ACUHO-I/EBI results, as well as other related assessment information, such as the House survey results. In addition, RLCs provide feedback they have gathered from RA staff and through their expertise and experiences. Recommendations Keeping in mind the goals of Residence Life programming, “Build communities of outstanding citizens and scholars through example, challenge and support,” the following recommendations have been created: Springboard houses should be utilized to capture impressions of services offered, programming effectiveness, and as a focus group for other Foundations of Excellence assessing, since all students living in those houses are new to UNI and fit the description of students examined in Foundations of Excellence. Program participation rates should be utilized. Attendance of events and programs is currently recorded for participation; however, these participation rates are not utilized as an indicator of interest in a particular topic or for historical prediction of attendance in the future. Students attending programming events should be given the option of providing basic feedback on that event through a brief survey for immediate feedback. External resources should be examined to improve assessing the goals of Residence Life programming. Grant funding or UNI special allocation money could be pursued to specifically find opportunities to quantify traits related to behavior for trending over time. National surveys should also be researched that 7 could be used as a standard for benchmarking Residence Life effectiveness. Collaboration with UNI’s Strategic Marketing Services or other consulting companies should also be considered. Examine utilizing other electronic methods to convey programming information to students as a supplement to programming efforts. Examples include using a Facebook group page, blogs, the Department of Residence website, and/or the UNI cable TV station. Currently, marketing of survey participation primarily occurs through e-mail solicitation and personal requests from RAs. Using additional methods could help to increase participation rates, share assessment information, and communicate helpful information to freshmen as they arrive on campus. Coordinate efforts between the Department of Residence Marketing department, Public Relations, and Residence Life. These efforts could help increase participation in assessments and convey applicable results to the UNI community. A more formalized assessment review process should be examined to ensure improvements based on formalized assessment data are being completely reviewed for future changes. The creation of an assessment review committee, comprised of RLCs, RAs, Residence Life staff, and perhaps other non-Residence Life staff that may be able to help with information analysis would be useful. Applicable assessment findings should also be shared with the campus community or other departments for their feedback and knowledge. House surveys should be coordinated centrally as a means to compare results between different halls and increase participation rates. Providing a incentives to complete this survey should also increase participation, which has proven successful with other surveys within the Department of Residence. Evaluate the timing of surveys and the populations affected. Also, examine how Department of Residence surveys coincide with University surveying efforts, if possible. This examination would help ensure students are not over-surveyed and information is not duplicated among other University assessments. The NSSE survey should be evaluated for future application to Residence Life programming. Special thanks to Drake Martin in the Department of Residence for his helpful information, responsiveness, and useful insight into Residence Life programming. Respectfully submitted, Todd Thomas Spencer Walrath 8 Appendix A: Residence Life Programming Evidence Evaluation Matrix FoE: Improvement Dimension Prepared by Todd Thomas and Spencer Walrath Evidence Item ACUHO-I/EBI Resident Assessment PI 9.1- Assessment PI 9.2 Use of Assessment To what degree does this evidence To what degree have item include systematic assessment assessment results been used (none to very high)? to improve existing practices? Very high Medium House survey Medium Very High Immersion Project Very high High Peer Advisor In Residence (PAIR) survey Very high High Springboard Dive-In Days survey Medium Medium RLC meetings with Residence Life staff Medium High RA meetings with RLC staff Low Medium President’s Council None Low National Survey of Student Engagement (NSSE) ACHA-NCHA Very high None Very high Medium Notes EBI data is not tailored to Residence Life programming, so many questions are generic/broad in nature; mostly measures satisfaction rather than growth/change Varied rates of participation; questions change over time, so less for benchmarking and more for immediate feedback; very useful for RAs and how they are perceived; surveying performed individually by RLCs with the results shared with each RLC Will be evaluated for success based on the results from comparing the April 2009 post-test to the April 2008 pre-test; survey is administered through myUNIverse New as of 2008; survey results helped to adjust PAIR from having Office hours to speaking at House meetings; surveyed through myUNIverse Only given to Freshmen in Springboard houses that chose to arrive for Dive-In Days; Used mostly to capture impressions of Dive-In Days presentations; survey is changing for 2009 Meetings occur 10 times a year; very applicable and practical to daily operations of Residence Life; utilizes the expertise of RLCs and data they collect for discussion Residence programming is one aspect of these weekly/bi-weekly meetings and provides helpful insights into programming effectiveness and topics for future programming The President’s Council primarily is used as an optional source of feedback for upcoming topics that may be related to Residence Life programming NSSE information was reviewed in the past, but has not been used recently; may contain helpful information for programming Health-related items are used for verification of programming efforts; participation rates have been between 10-20% of a random sampling of 4,000 students; reviewed annually