Student Retention
Software Platforms
October 2013
In the following report, Hanover Research examines student attrition and the use of technology in institutional retention strategies. The report provides a description of the potential benefits of automating certain aspects of a student retention strategy, describes the prevalence of such strategies at postsecondary institutions across the country, and offers some key considerations when selecting the most appropriate software for an institution. The report concludes with several profiles of institutions that have incorporated technological platforms into their student retention initiatives. Hanover Research | October 2013
TABLE OF CONTENTS
Executive Summary and Key Findings ................................................................................ 3
Introduction ........................................................................................................................... 3
Key Findings ........................................................................................................................... 4
Section I: Student Retention Software ............................................................................... 6
Technology in Student Retention Strategies ......................................................................... 6
Prevalence of Early Warning Systems ............................................................................... 7
Critical Considerations when Selecting Retention Software ............................................. 7
Common Student Retention Platforms ................................................................................. 9
Starfish Retention Systems Early Alert ............................................................................ 10
EBI MAP‐Works ................................................................................................................ 12
Campus Labs Beacon ....................................................................................................... 14
Ellucian ............................................................................................................................. 15
Section II: Institutional Profiles ........................................................................................ 17
Youngstown State University ............................................................................................... 17
Baylor University .................................................................................................................. 19
Bowie State University ......................................................................................................... 21
© 2013 Hanover Research | Academy Administration Practice
2
Hanover Research | October 2013
EXECUTIVE SUMMARY AND KEY FINDINGS
INTRODUCTION The statistics surrounding student attrition are staggering: nation‐wide, more than 30 percent of students that matriculate to colleges or universities withdraw before they earn a degree.1 The economic consequences of attrition are also severe, and can be substantially costly for the individual, the institution, and society as a whole.2 Though the institutional costs of student attrition are most often discussed as lost tuition revenue, the actual cost to the institution is much greater. A recent study by the American Institutes for Research found that, in addition to lost tuition and ancillary income, nearly 20 percent of all institutional expenditures are directed toward students that will ultimately not earn a degree, amounting to approximately $18,000 for each student withdrawal.3 Given these high institutional costs, colleges and universities around the country have launched comprehensive programs to help retain their admitted students. Despite their inherent complexity, the factors that contribute to student attrition, retention, and overall academic success have been widely studied, allowing postsecondary institutions to create programs designed to contribute to positive academic outcomes and, ultimately, help students earn degrees.4 However, research has shown that students that are most likely to benefit from academic interventions are often the least likely to participate, limiting the efficacy of such programs.5 More recently, colleges and universities have begun to implement automated analytics software that can help identify students that pose the greatest risk for attrition, allowing for institutional retention resources to be allocated more efficiently and effectively. These systems interface and automatically gather data from institutions’ student information systems (SIS), learning management systems (LMS), and enterprise resource planning (ERP) systems, and generally condense data from both academic sources, such as grades, and non‐academic sources, such as level of campus engagement, to warn the appropriate campus personnel if a student is at‐risk. Most contemporary student retention software programs also allow academic advisors and other campus administrators a platform for communication and managing student interventions to completion. 1
Johnson, N. “The Institutional Costs of Student Attrition.” The American Institutes for Research. 2012, p. 1. http://www.deltacostproject.org/resources/pdf/Delta‐Cost‐Attrition‐Research‐Paper.pdf 2
Swail, W.S. “The Art of Student Retention: A Handbook for Practitioners and Administrators.” Educational Policy Institute. 2004, p. 9. http://www.educationalpolicy.org/pdf/art_of_student_retention.pdf 3
Johnson, N. Op cit. p. 5. 4
Chacon, F., Spicer, D., and Valbuena, A. “Analytics in Support of Student Retention and Success.” EduCause Center for Applied Research. April 10, 2012. p. 1. http://net.educause.edu/ir/library/pdf/ERB1203.pdf 5
Nelson, J. “What’s the Big Deal about Early Alert Systems?” CREDO. October 27, 2010. http://www.credohighered.com/blog/whats‐the‐big‐deal‐about‐an‐early‐alert‐system/ © 2013 Hanover Research | Academy Administration Practice
3
Hanover Research | October 2013
In this report, Hanover Research examines student attrition and the use of technology in institutional retention strategies. The report provides a description of the potential benefits of automating certain aspects of a college or university’s student retention strategy, describes the prevalence of such strategies at postsecondary institutions across the country, and offers some key considerations for selecting the most appropriate piece of software for an institution. The report also includes summaries of the key functions and features of some of the more common retention software platforms on the market. This report concludes with profiles of three institutions that have incorporated technological platforms into their student retention initiatives. Accordingly, the report comprises the two following sections: 
Section I: Student Retention Software describes the theory underlying the use of student retention software, the prevalence of such retention strategies at U.S. institutions, and critical considerations for selecting the most appropriate software platform for a college or university. The section concludes with a description of the basic features and functionality of some of the more common student retention platforms. 
Section II: Institution Profiles provides practical examples from three universities that have incorporated software platforms into their student retention strategies. Lessons are drawn from secondary literature and interviews with administrative personnel. KEY FINDINGS 
Recent evidence shows that poor academic performance accounts for only 15 percent of student withdrawals in higher education nation‐wide. Accordingly, many colleges and universities are deploying student retention strategies that include both cognitive and non‐cognitive factors to be more representative of the entire student experience. 
Early identification of at‐risk students can be an effective tool to improve persistence and may contribute to increased graduation rates. While early identification has traditionally involved an assessment of mid‐term grades, student retention software platforms can help identify at‐risk students early in the semester, allowing for more efficient academic interventions. 
Most student retention software programs offer similar services, usually including an “early warning system” for at‐risk students; a forum for communication between advisors, faculty, and students; and platform to manage student academic interventions. Given the similarity of these products, colleges and universities should select the retention software that most closely aligns with their existing infrastructure, retention strategies, and institutional goals. 
Some student retention platforms – such as EBI’s MAP‐Works and Campus Lab’s Beacon – use the results of student self‐assessment surveys to help identify at risk students. Other retention platforms allow for integration with popular student assessment surveys, such as the Noel‐Levitz College Student Inventory. © 2013 Hanover Research | Academy Administration Practice
4
Hanover Research | October 2013

Some student retention platforms – such as Starfish Retention Solution’s Early Alert and MAP‐Works – allow system administrators to monitor the efficacy of different types of student interventions. Such analytic platforms can help streamline an institution’s overall retention initiative, and allow resources to be dedicated to the most effective intervention strategies. 
Many colleges and universities implementing student retention software programs choose to pilot their platforms on a small scale before introducing the system campus‐wide. This pilot period has been conducted with either a limited number of students enrolled in the system, or a limited number of advisory personnel given access to system data. © 2013 Hanover Research | Academy Administration Practice
5
Hanover Research | October 2013
SECTION I: STUDENT RETENTION SOFTWARE
In this section, Hanover Research explores the use of technology to assist student retention initiatives, with an emphasis on third‐party retention software. This section provides a description of the potential for student retention to help identify at‐risk students and to focus institutional resources toward the most effective and efficient student interventions. The section also includes a description of the prevalence of student retention initiatives at the nation’s colleges and universities and discusses the most critical consideration when choosing the appropriate third‐party retention software. The section concludes with a description of the basic features and functions of some of the more common student retention platforms. TECHNOLOGY IN STUDENT RETENTION STRATEGIES In a 2002 study conducted for the Lumina Foundation, researchers determined that, all others factors being equal, colleges and universities with greater resources typically have higher rates of retention and graduation.6 While factors such as the institutional culture, the efficacy of instruction, and the dedication of faculty were found to be influential, an institution’s ability to dedicate monetary and human resources to retention was the most significant factor in helping students graduate. The report indicated that “Lower‐performing schools had staff as or more dedicated than those at better performing schools, and offered a quality education. It’s just that other schools were able to pile on resource after resource in who comes, who stays, and who completes.”7 Unfortunately, resources available for student retention initiatives are often quite limited. A 2009 College Board study of four‐year colleges found that student retention initiatives at U.S. institutions are typically under‐staffed, with an average of only 0.29 full‐time equivalents (FTEs) dedicated to the administration and coordination of these efforts.8 In an effort to identify the most at‐risk students, and to direct available resources towards their retention, many institutions evaluate mid‐term grades and attempt to determine which students are struggling academically. However, research indicates that academic problems account for only about 15 percent of student withdrawals, with family responsibilities, financial troubles, and personal reasons being much more likely reasons for students to leave higher education before earning a degree.9 Consequently, many postsecondary institutions have begun to utilize student retention software platforms to efficiently identify at‐risk students and focus available resources toward those students that pose the most significant risk for attrition. Modern student 6
Swail, W.S. Op cit. p. 7. Ibid. 8
“How Colleges Organize Themselves to Increase Student Persistence: Four‐Year Institutions.” College Board. 2009, p. 6. http://professionals.collegeboard.com/profdownload/college‐retention.pdf 9
Johnson, N. Op cit. p. 10. 7
© 2013 Hanover Research | Academy Administration Practice
6
Hanover Research | October 2013
retention platforms typically interface with an institution’s SIS, LMS, or ERP to analyze data that are more representative of the entire student experience, and create a forum for communication regarding intervention activities. While placing significant emphasis on a student’s reported grades, most student retention software programs reviewed in the preparation of this report also include data related to: 






Demographics High school grades College entrance exam results Financial aid status Family educational background Campus engagement Interactions with advisors, faculty, and other campus personnel PREVALENCE OF EARLY WARNING SYSTEMS In October 2010, the John N. Gardner Institute for Excellence in Undergraduate Education administered a national survey to assess the efficacy of seven common strategies employed at colleges and universities to increase retention and improve academic performance of undergraduate students. 10 Amongst these strategies was the use of “early warning/ academic alert systems” used to identify students that are considered the most likely to prematurely withdraw from an institution. Overall, survey results indicate that early warning systems are quite common amongst U.S. postsecondary institutions. Of the 420 respondents, nearly 98 percent of private institutions and 86 percent of public institutions reported employing some form of early warning or academic alert system to identify struggling students. However, such systems were found to be considerably more common amongst smaller institutions with less than 5,000 enrolled students, as compared to large institutions. Further, the study found that approximately 70 percent of all U.S. institutions employ continuous student monitoring over the course of the semester, rather than relying solely on mid‐term grades.11 CRITICAL CONSIDERATIONS WHEN SELECTING RETENTION SOFTWARE While third‐party student retention software can help identify at‐risk students and streamline an institution’s overall retention strategy, the number of software applications available can complicate the selection of the most appropriate system for a given university. Most of the commercial student retention systems on the market offer similar services: early warning of those students that are most likely to withdraw or to lag academically; a platform for communication between students, faculty, advisors, and other campus 10
Barefoot, B., Griffin, B., and Koch, A. “Enhancing Student Success and Retention throughout Undergraduate Education: A National Survey.” John N. Gardner Institute for Excellence in Undergraduate Education. 2012. p. 1. http://www.jngi.org/wordpress/wp‐content/uploads/2012/04/JNGInational_survey_web.pdf 11
Ibid. p. 25‐31. © 2013 Hanover Research | Academy Administration Practice
7
Hanover Research | October 2013
retention personnel; and customized analytics to help manage and evaluate the efficacy of retention strategies and interventions. Given the relative similarity of services, colleges and universities should evaluate how each system meets institutional needs, goals, and objectives. Dr. Hossein Hakimzadeh, the Director of Informatics at Indiana University South Bend and founder of the Retainology Consortium, has compiled a rubric of the most relevant criteria in selecting retention software.12 Dr. Hakimzadeh recommends evaluating each program’s anticipated performance in a number of disparate areas, ranging from the system’s overall predictive reliability to its ability to scale to the university’s student body. These criteria are summarized in Figure 1.1 below. Before beginning the selection process, however, colleges and universities should attempt to develop an understanding of the reasons for student attrition, and how the appropriate technology will fit within the overall retention strategy.13 Figure 1.1: Key Considerations for Evaluating Retention Systems CATEGORY EVALUATION CRITERIA ACCEPTABLE LEVEL OF SERVICE Simplicity Interfaces are simple, intuitive. System responsiveness should be nearly instantaneous. Scalability does not require unreasonable increase in the cost of hardware or software. Fixed and reasonable cost of integration or migration of data to the new retention system. No hidden costs for additional software licensing. Fixed and reasonable. Fixed and reasonable. Fixed and reasonable. Faculty is able to provide recommend remediation actions for at risk students. Simple, intuitive interface which allows faculty, advisors, as well as students to communicate. Ability to send automatic and/or on‐demand notifications to various user groups. Ability of the faculty to obtain follow up information about the students that they have been flagged as at‐risk. All users should be able to view appropriate, accurate and actionable reports. Speed Ease of Use Scalability Cost of integration with existing systems Costs Communication Cost of supporting software tools and infrastructure. Cost of Software Cost of hardware and network Cost of maintenance Ability to incorporate faculty recommendations Ability to communicate with individual or groups of students Automatic notifications Integration with Retention Strategy Ability to follow up on at‐risk students Actionable and timely reports Source: Retainology Consortium14 12
Hakimzadeh, H. “A Guide for evaluating and Selecting an Early Warning Student Retention System.” Indiana University South Bend. https://retain.iusb.edu/retain/public/Guide_For_Evaluating_Early_Warning_and_Retention_Software.pdf 13
Ibid. p. 7. 14
Ibid. p. 3‐6. © 2013 Hanover Research | Academy Administration Practice
8
Hanover Research | October 2013
While some of the recommended evaluation criteria – such as the system’s predictive reliability and validity – seem somewhat obvious, other criteria relate to more esoteric aspects of the software and may not be readily apparent before the piloting the system. For example, while most institutions may budget for the cost of software licensing, a number of hidden costs, such as requisite hardware upgrades, may make the system substantially pricier than anticipated.15 COMMON STUDENT RETENTION PLATFORMS In this section, Hanover Research provides an overview of the basic features and functionality of four of the most common student retention platforms employed at postsecondary institutions. The current body of secondary literature related to retention software applications is quite limited. Accordingly, the information provided in this section generally summarizes details provided by the software developers and, in some instances, by colleges or universities using the system. In total, Hanover Research discovered 12 student retention software platforms developed by 11 different firms currently on the market (Figure 1.2). In general, each of these platforms provides the same basic functionality. All of the products listed in Figure 1.2 act as an “early warning system,” identifying at‐risk students based on academic and non‐
academic data; automatically mine data from an institutions SIS, LMS, or enterprise resource planning (ERP) system; allow for direct communication between students and campus personnel; and allow academic advisors to track student progress. Figure 1.2: Third‐Party Retention Software SOFTWARE DESIGNER Blackboard Campus Labs Ellucian Ellucian EMAS Hobsons Jenzabar EBI Pharos QuScient SmartEvals Starfish Retention Solutions SOFTWARE Blackboard Analytics Suite
Beacon
Colleague Retention Alert
Banner Student Retention Performance
Retention Pro
Retain
Finish Line
Map Works
Pharos 360
ProRetention
DropGuard
Early Alert
15
Ibid. p. 4. © 2013 Hanover Research | Academy Administration Practice
9
Hanover Research | October 2013
STARFISH RETENTION SYSTEMS EARLY ALERT Starfish Retention System’s Early Alert is an early warning and student tracking system designed to identify at‐risk students and help manage student retention activities.16 The software provides automated analysis of data from academic and administrative management systems, and provides immediate notifications to the appropriate campus personnel when an at‐risk student is identified. In order to provide a more holistic assessment of student risk factors, the system also relies on the manual input of data – by academic faculty, advisors, coaches, and residence hall directors – to most effectively predict those students that are most likely to withdraw.17 Product Features While identifying at‐risk students may be the most apparent benefit of the Early Alert system, the software can be further integrated into an institution’s overall retention strategy. In order to allow an institution to allocate resources most efficiently, the system’s analytic and notification protocols can be customized to more closely track a specific cohort of students or those students taking a particular course, and allows for the automatic prioritization of “flags” to identify students who are of greatest risk.18 The system is also capable of generating analytic reports comparing student achievement to different types of interventions, helping institutions determine which aspects of their retention strategy are most and least effective.19 Early Alert is capable of automatically mining data from various institutional management systems and automatically identifying at‐risk students based on common indicators of attrition, such as academic performance and attendance. The system also allows authorized personnel to manually flag an individual student, notifying other relevant personnel throughout the university that the student is at‐risk. Institutions may also choose to incorporate the results of faculty surveys into the systems analytics, thereby gathering student academic data prior to midterm exams.20 The Early Alert identification system is customizable, capable of automatically identifying at‐
risk students based on the set of indicators that are most commonly associated with student withdrawal at a specific institution. Starfish has created two unique models to customize analytics. The Early Alert “Flag Lab” creates a database of customized indicator sets used or developed by other institutions across the country, and allows these flags to be incorporated into the identification and alert system.21 Alternatively, Starfish will develop 16
“Early Alert Solution Overview.” Starfish Retention Solutions. p. 1. http://www.starfishsolutions.com/data/document/pdf/StarfishEarlyAlert.pdf 17
“Starfish Early Alert.” Starfish Retention Solutions. http://www.starfishsolutions.com/sf/earlyalert.php. 18
Ibid. 19
“Early Alert Solution Overview.” Op cit. p. 1. 20
Schaffhauser, D. “Youngstown State U. Expands Usage of Student Retention Software.” Campus Technology. May 28, 2013. http://campustechnology.com/articles/2013/05/28/youngstown‐state‐u‐expands‐usage‐of‐student‐
retention‐software.aspx 21
“Early Alert Solution Overview.” Op cit. p. 2. © 2013 Hanover Research | Academy Administration Practice
10
Hanover Research | October 2013
custom flags based on a requested set of indicators and corresponding dataset, and can typically test and employ the flag in one day.22 The Starfish Early Alert program can mine data from several student success systems, including the Noel‐Levitz College Student Inventory, Redrock’s AdvisorTrac student counseling manager, and the Smarthinking tutoring and student support manager.23 Early Alert also integrates with some of the more common learning management and student information systems, and can be integrated with other proprietary technological systems, as demonstrated in Figure 1.3. Figure 1.3: Early Alert Compatible Management Systems MANAGEMENT SYSTEM TYPE Learning Management Systems Student Information Systems COMPATIBLE SYSTEMS • Blackboard Learning System 8.0 and higher • WebCT Vista 4.0 and higher • WebCT Campus Edition 6.0 and higher • ANGEL 7.3 and higher • Moodle 1.8 and higher • Desire2Learn 9.4.1 and 10.0 • Oracle / Peoplesoft
• Sungard Banner • Sungard PowerCAMPUS • Datatel Colleague • Jenzabar • Campus Management System CampusVUE Source: Starfish Retention Solutions24 Starfish provides full customer support throughout the implementation and operation of the Early Alert system.25 As part of the regular licensing fee, Starfish provides installation and training services, as well 24‐hour emergency support and comprehensive product and technical support during extended business hours. Starfish also offers a line of fee‐based “Strategic Services” to help institutions customize their Early Alert system and interpret system data.26 Early Alert is a web‐based application, hosted entirely on Starfish servers and accessible through most web browsers.27 The program has been designed with SSAE 16 Certification, ensuring secure communication between users, the university, and Starfish, as well as compliance with the Family Educational Rights and Privacy Act (FERPA).28 The system allows for a single, streamlined log‐in with some of the most common academic and personnel 22
Ibid. “Starfish Early Alert Specifications.” Starfish Retention Solutions. http://www.starfishsolutions.com/sf/earlyalertspecs.php 24
Ibid. 25
“Client Success Services.” Starfish Retention Solutions. http://www.starfishsolutions.com/sf/clientsuccess.php 26
Ibid. 27
Ibid. 28
“Starfish Early Alert Specifications.” Op. cit. 23
© 2013 Hanover Research | Academy Administration Practice
11
Hanover Research | October 2013
management systems, including Blackboard and PeopleSoft, and can be incorporated into the log‐in of other proprietary systems.29 EBI MAP‐WORKS Educational Benchmarking Incorporated’s MAP‐Works platform is a comprehensive software application designed to improve student retention by identifying at‐risk students early in the term, and providing relevant information to the faculty and staff responsible for assisting these students.30 MAP‐Works uses data‐driven analytics to identify at‐risk students early in the fall semester, allowing university personnel to intervene before academic, behavioral, or financial factors lead to student withdrawal.31 The system was designed in partnership with Ball State University in 2003, and incorporates research‐based findings from over 20 years of experience with Ball State’s “Making Achievement Possible” (MAP) program.32 The EBI MAP‐Works retention platform and EBI Benchmarking Assessment system were employed by more than 1,500 colleges and universities during the 2010‐2011 academic year.33 Product Features The MAP‐Works platform uses two primary sources of data to identify at‐risk students: student characteristics and student survey results. Student characteristics typically include information such as gender, ethnicity, course schedule, high school grade point average, and college entrance exam scores. 34 Student characteristics are generally manually uploaded by system administrators in comma‐delimited file format, or can be automatically mined from the institution’s student information system or enterprise resource system.35 Participating institutions typically administer the MAP‐Works “First‐Year Transition Survey” to incoming freshmen during the first four weeks of the fall term.36 These surveys collect information in three additional categories: academic integration, self‐assessment, and social integration. Figure 1.4 displays the various types of data collected in the MAP‐Works risk assessment. 29
Ibid. “MAP‐Works.” EBI MAP‐Works. http://www.webebi.com/mapworks 31
“The Foundation of MAP‐Works: Research and Theoretical Underpinnings.” EBI MAP‐Works. p. 3. http://indstate.edu/studentsuccess/pdf/MAP‐Works%20Foundation%20Oct%202012%20.pdf 32
[1] Hickey, L. “MAP‐Works Student Retention Platform Reports Over $25 Million in Institutional Savings in 2012.” EBI MAP‐Works. February 12, 2013. http://www.webebi.com/community/news/151/map‐works‐student‐
retention‐platform‐reports‐over‐25‐million‐in‐institutional‐savings‐in‐2012 [2] “The Foundation of MAP‐Works Research and Theoretical Underpinnings of MAP‐Works.” Op cit. p. 3. 33
“Our Clients.” EBI MAP‐Works. http://www.webebi.com/about/clients 34
Moore, S., et. al. “Using Comprehensive Assessment to Intervene with and Retain Students.” EBI MAP‐Works. p. 5. http://slra.osu.edu/posts/documents/osaac‐effectively‐intervene‐with‐and‐retain‐students.pdf 35
“Frequently Asked Questions.” EBI MAP‐Works. http://www.webebi.com/mapworks/faqs 36
“The Foundation of MAP‐Works: Research and Theoretical Underpinnings.” Op cit. p. 8. 30
© 2013 Hanover Research | Academy Administration Practice
12
Hanover Research | October 2013
Figure 1.4: Information Included in the MAP‐Works Risk Assessment DATA CLASSIFICATION Student Characteristics COLLECTION METHODOLOGY • Integrated from SIS/ERP • Manually uploaded Academic Integration • Survey results Self‐Assessment • Survey results Social Integration • Survey results INFORMATION • Gender
• Race/ethnicity • Entrance exam scores • Credit hours • High school GPA • Academic self‐efficacy • Core academic behaviors • Advanced academic behaviors • Commitment to higher education • Communication skills • Analytical skills • Self‐discipline • Time management • Health and wellness • Peer connections
• Living environment • Roommate relationships • Homesickness Source: EBI MAP‐Works37 Once student data are uploaded, the MAP‐Works system uses a regression algorithm to determine the relative risk that students will prematurely withdraw from the institution.38 Students are classified in one of four “Risk Indicator” groups ranging from “low risk” to “extremely high risk,” which are automatically updated when additional student data become available. Accordingly, students can move from one Risk Indicator group to another as their academic performance or relative risk factors change over the course of the year.39 MAP‐Works then allows relevant faculty and staff to view student Risk Indicators and detailed survey results, and creates a platform for planning and monitoring the results of interventions. The system also provides analytical tools to help assess the efficacy of interventions, and allows administrators to compare outcomes across different academic years and benchmark against peer institutions. 40 Students are also provided with automatically‐generated reports that identify differences between the student’s personal expectations and the habits and behaviors necessary to achieve these goals.41 Faculty and staff may also establish lines of contact with students directly through MAP‐Works, using the platform’s DirectConnect feature.42 37
Moore, S., et. al. “Using Comprehensive Assessment to Intervene with and Retain Students.” Op cit. p. 5. Ibid. p. 10 39
Ibid. 40
“MAP‐Works Implementation Process.” EBI MAP‐Works. http://www.webebi.com/mapworks/process 41
Ibid. 42
“Frequently Asked Questions.” EBI MAP‐Works. Op cit. 38
© 2013 Hanover Research | Academy Administration Practice
13
Hanover Research | October 2013
MAP‐Works is a web‐based application hosted entirely on EBI servers, typically requiring no additional hardware infrastructure to implement.43 According to EBI, the system seamlessly integrates and mines data from all ERP and SIS systems.44 EBI offers on‐campus training and consulting to help institutions implement and customize the MAP‐Works platform. Training is typically conducted by educational professionals who are skilled in MAP‐Works optimization and have successfully implemented the platform at their campus.45 CAMPUS LABS BEACON Campus Labs’ Beacon is a web‐based analytics platform that uses academic and non‐
cognitive data to predict a student’s future academic success.46 The system collects and analyzes data from a number of different sources to provide a holistic assessment of a student’s entire campus experience. The Beacon platform is designed to help postsecondary institutions retain students by employing a three‐part strategy: collecting and analyzing the most relevant student data, sharing that data with the appropriate people across the campus, and connecting data to identify students that are typically overlooked.47 Product Features The Beacon platform automatically generates reports for students, faculty, and advisors to help colleges and universities make data‐driven decisions concerning their retention strategies based on five types of data that are considered “the strongest predictors of student retention and persistence,” including:48 




Demographic information Academic records Campus involvement Student strengths Key indicator areas Though Campus Labs does not explicitly detail the criteria for all “key indicator areas,” these typically include a student’s personal information, such as the level of familial support they receive.49 The system automatically collects and continuously updates information from a number of campus information systems, producing an up‐to‐date picture of a student’s academic progress and the likelihood that he or she will withdraw from the university. The system also allows campus personnel to manually upload information, such as periodic 43
Ibid. “MAP‐Works Implementation Process.” EBI MAP‐Works. Op cit. 45
“Training and Consulting.” EBI MAP‐Works. http://www.webebi.com/mapworks/training 46
“Beacon Product Card.” Campus Labs. p. 2. http://www.campuslabs.com/download‐beacon/ 47
White, J.D. “Identifying More At‐Risk Students with an Expanded Data Set.” Campus Labs. 2012, pp. 8‐9. http://www.campuslabs.com/pdf/be‐041612.pdf 48
“Beacon: Building Bridges to Student Success.” Campus Labs. http://www.campuslabs.com/products/beacon/ 49
“Beacon Product Card.” Op cit. p. 2. 44
© 2013 Hanover Research | Academy Administration Practice
14
Hanover Research | October 2013
reports from residence hall advisors, academic faculty, and advisors, which can be automatically analyzed and included in student risk assessments.50 In addition to automatically mined student data, the Beacon system relies on their proprietary Student Strengths Inventory survey (SSI) to gather information related to a student’s motivation factors and academic habits. 51 The SSI, which is administered electronically and typically takes approximately 10 minutes to complete, asks students to self‐report their strengths on 48 key non‐cognitive areas, and is used to generate individualized reports pertaining to the student’s future academic performance and their risk factor for withdrawal.52 The SSI is designed as an assessment tool for incoming college freshman. Campus Labs has designed and is currently piloting a similar assessment survey for second year students. ELLUCIAN Ellucian has developed several student retention platforms designed to be used in conjunction with their popular student information system, Banner. Perhaps the most apparent benefit of the Ellucian platforms is that they are embedded in the Banner platform. Additionally, Ellucian has developed a line of “Student Success and Retention Planning” consulting services designed to help postsecondary institutions align resources and expertise to minimize attrition and improve students’ academic outcomes.53 Colleague Retention Alert Colleague Retention Alert interfaces with an institution’s student information system to identify at‐risk students and alert the student and appropriate campus personnel that they may benefit from additional campus assistance. 54 Unlike the other Ellucian platforms described in this report, Colleague Retention Alert uses non‐academic data sources – such as attendance records, family problems, and illnesses – to identify those students that are most likely to withdraw. 55 Once the system identifies at‐risk students, automatically generated emails are delivered to the student, academic advisors, and other responsible campus personnel. The Colleague Retention Alert system allows case managers to automatically or manually generate flags for students, and accommodates personalized notes from the student, faculty, or other authorized university employees.56 50
White, J.D. Op cit. p. 10. “Beacon Product Card.” Op cit. p. 2. 52
“About the SSI.” Campus Labs. http://www.campuslabs.com/products/beacon/about‐the‐student‐strengths‐
inventory/ 53
“Ellucian Student Success and Retention Planning Services.” Ellucian. http://www.ellucian.com/Solutions/Ellucian‐
Student‐Success‐and‐Retention‐Planning‐Services/ 54
“Colleague Retention Alert.” Ellucian. http://www.ellucian.com/Solutions/Colleague‐Retention‐Alert/ 55
“Datatel Launches Complete Retention Solution to Identify At‐Risk Students and Preserve Enrollment.” Business Wire. January 14, 2008. http://www.businesswire.com/news/home/20080114005012/en/Datatel‐Launches‐
Complete‐Retention‐Solution‐Identify‐At‐Risk 56
“Datatel Retention Alert.” Datatel. 2009, p. 3. http://hlc.southeast.edu/public/149‐
Retention%20Alert%20Software%20Description.pdf 51
© 2013 Hanover Research | Academy Administration Practice
15
Hanover Research | October 2013
Banner Student Retention Performance Ellucian’s Banner Student Retention Performance is an application that can be integrated within an institutions Banner Student information system to automate advanced analytics based on academic data. The system allows postsecondary administrators to quickly and effectively examine student success indicators, make informed determinations about program efficacy, assess academic trends over time, and support institutional retention strategies and student success initiatives.57 In addition to tracking individual student's academic performance, Ellucian has developed a series of “Key Performance Indicators” that allow colleges and universities to track their overall progress towards institutional goals. 58 Using the system’s embedded scorecards, dashboards, reports, and analytics, Banner Student Retention Performance can help institutions identify at‐risk students, evaluate retention and completion rates, conduct program evaluations, gauge the efficacy of retention strategies, and compare the relative educational outcomes of different student groups.59 Course Signals Ellucian’s Course Signals is an early warning and intervention system designed to alert students when they are at risk of underperforming in a course and facilitates early faculty engagement.60 Based on a predictive model built and piloted by Purdue University in 2007, the system relies on multiple data inputs, assessing a student’s performance in a specific course in comparison to several established risk factors, such as low overall grade point average, first‐generation college students, and transfer status.61 As with all Ellucian software packages, Ellucian provides consultation services to help institutions customize their Course Signals analytics and integrate the program with the overall institutional imperatives and retention strategies.62 57
“Banner Student Retention Performance: Make Informed, Strategic Decisions to Improve Student Success.” Ellucian. p. 1. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&ved=0CDQQFjAB&url=http%3
A%2F%2Fwww.ellucian.com%2FSolution‐Sheets%2FBanner‐Student‐Retention‐
Performance%2F&ei=YrwLUuujAaiMyAGEjoDwCQ&usg=AFQjCNGXTmGx2XT‐
STXw3i5rSLFQTnLVSQ&sig2=jsECwjXbguvdU‐_ZvfGKRA 58
“Banner Student Retention Performance Handbook.” SunGARD Systems. April 2011. 7‐2. http://our.saintmarys.edu/~banner/bdoc8x/srp1x/srp10000hb.pdf 59
“Banner Student Retention Performance.” Ellucian. http://www.ellucian.com/Solutions/Banner‐Student‐Retention‐
Performance/ 60
“Ellucian Course Signals: redefining ‘Early’ to Support Student Success.” Ellucian. p. 1. http://www.ellucian.com/Solution‐Sheets/Ellucian‐Course‐Signals/%E2%80%8E 61
Ibid. pp. 1‐3. 62
Ibid. p. 3. © 2013 Hanover Research | Academy Administration Practice
16
Hanover Research | October 2013
SECTION II: INSTITUTIONAL PROFILES
In this section, Hanover Research provides profiles of several institutions that have implemented student retention platforms as part of their overall retention strategy. The institutions examined in this section were selected based on profiles included on the retention software websites, as well as the availability of secondary literature detailing the retention programs and the willingness of relevant university personnel to discuss system implementation with Hanover Research staff. This section includes profiles of Youngstown State University, Baylor University, and Bowie State University. YOUNGSTOWN STATE UNIVERSITY Youngstown State University is a public university located in eastern Ohio, with an enrollment of approximately 13,300 undergraduate and 1,100 graduate students.63 The Youngstown State student body is made up of students with particularly challenging retention demographics, with only about 10 percent of students living on campus, more than 50 percent of students being first‐generation college attendees, and nearly 90 percent of students receiving financial aid. 64 Despite well‐established academic services for struggling students, an internal study of the University’s 2004 cohort found that less than 15 percent of students had graduated within four years of matriculating, and less than 40 percent had earned a degree within six years.65 In order to improve student academic outcomes and increase the graduation rate, administrators at Youngstown State decided to integrate the Starfish Retention Solutions Early Alert platform into their institutional student success and retention initiatives.66 Before moving to the Early Alert platform, Youngstown State used an in‐house system to identify at‐risk students during their freshman year. This system, which used a series of paper surveys administered to faculty members, was plagued by low professor response rates, with only about 5 percent of faculty completing surveys in a given semester.67 The system’s other primary data input, student’s mid‐term grades, produced more consistently reliable data, but often came too late for campus personnel to stage an efficient and substantive intervention. 68 Following an extensive evaluation of third‐party retention 63
“Search for Schools, Colleges, and Libraries.” National Center for Education Statistics. http://nces.ed.gov/globallocator/ 64
Plunkett, J. “Youngstown State University Adopts the Starfish System to Support Campus Initiative to Improve Graduation and Completion Rates.” PRWeb. May 22, 2013. http://www.prweb.com/releases/Starfish/YSU/prweb10754694.htm 65
Lazar, A. “Improving Student Retention Using Starfish Retention Solutions Software.” Youngstown State University. p. 3. http://www.uakron.edu/dotAsset/b71ff750‐9da4‐49fb‐9735‐30d7f9d3d706.pdf 66
Schaffhauser, D. “Youngstown State U Expands Usage of Student Retention Software.” Op. cit. 67
Egleton, T. and Beatrice, J. “Early Alert Systems: Closing the Communication Loop for At‐Risk Students.” Youngstown State University. p. 17. http://www.ohioaacrao.org/professional_development/pdf/2012_Presentations/2012_starfish.pdf 68
Beatrice, J. Executive Director of Student Life. Youngstown State University. Telephone interview. August 16, 2013. © 2013 Hanover Research | Academy Administration Practice
17
Hanover Research | October 2013
management software solutions, administrators selected Early Alert as they felt it best fit their needs with respect to:69 




A user friendly functionality and interface Integration with their current SIS Alignment with existing academic support procedures Excellent customer service Faculty driven, rather than student self‐reporting Prior to launching the Early Alert program institution‐wide, Youngstown State staged an approximately two‐year pilot‐period to streamline academic interventions with the new data‐driven approach. The system was initially introduced to only five freshman development seminars in the fall of 2011, and gradually scaled up to include the entire freshman class in fall of 2012. According to Jonelle Beatrice, Youngstown State’s Executive Director of Student Affairs and the campus administrator responsible for overseeing the system’s implementation, this pilot period was invaluable for coordinating the University’s intensive response to at‐risk students.70 The system was ultimately rolled‐out University‐
wide during the spring of 2013, and more than 4,000 flags were raised for at‐risk students over the course of the semester. Without the pilot period, Ms. Beatrice does not think the University would have been able to launch such a large‐scale response to help at‐risk students.71 Though the Early Alert system is designed to interface with the University’s SIS, Ellucian’s Banner, Ms. Beatrice says that getting the most out of the platform does require some additional tooling. Youngstown State worked with their IT personnel and faculty from the Computer Science and Information Systems Department to develop an intermediary program to pull the desired data from the SIS and enter it into the Early Alert platform. Ms. Beatrice says that Starfish Retention Systems provided a high‐level of support throughout the development of this intermediary software, and has continued to offer exceptional customer service.72 The Center for Student Progress at Youngstown State has customized the Early Alert platform to help direct students toward the most appropriate type of intervention. Currently, the University’s system produces five unique flags based on different student data inputs, as demonstrated in Figure 2.1.73 The faculty and administration at Youngstown State are particularly fond of one flag, the “kudos” flag, which alerts the student, peer mentors, and academic advisors when a student is excelling academically. Ms. Beatrice says 69
Beatrice, J. “Case Study: Youngstown State University – Utilizing Starfish to Support Campus Initiatives to Improve Graduation and Completion Rates.” Starfish Retention Solutions. Webinar. http://dostarfish.com/services/CaseStudies/YSUCaseStudy/YSUCaseStudy_player.html 70
Beatrice, J. Telephone interview. Op cit. 71
Ibid. 72
Ibid. 73
Egleton, T. and Beatrice, J. Op cit. p. 10. © 2013 Hanover Research | Academy Administration Practice
18
Hanover Research | October 2013
that this sort of positive reinforcement is unique to the Early Alert system, and has been an especially powerful motivator for students.74 Figure 2.1: Early Alert Flags at Youngstown State University FLAG TYPE Poor Attendance / No Attendance Low Grades Tutorial Referral Needs CSP Outreach Kudos DESCRIPTION Indicates any student who is not attending class on a regular basis, or has not attended class at all. Indicates a student who has not been successful in tests or class assignments. Indicates a student would benefit from a tutor. Indicates a student would benefit from a CSP intervention. Indicates a student is demonstrating good academic progress. Source: Youngstown State University75 Though it is still too early to determine if the system has increased first‐to‐second year retention or improved the University’s graduation rate, a number of academic indicators suggest that the system is making a difference. In the one semester in which the system was used institution‐wide, Ms. Beatrice says that Youngstown State students saw a significant increase in overall grade point average and the average number of credit hours earned per semester, a decrease in non‐attendance course failures, and a 3 percent increase in the course completion rate.76 Faculty participation in the periodic surveys also increased nearly 37 percent from the previous paper system, contributing significantly to the system’s viability.77 BAYLOR UNIVERSITY Baylor University is a private, coeducational university located in Waco, Texas. The University enrolls a total of approximately 12,500 undergraduate and 2,500 graduate students.78 Prior to implementing the MAP‐Works platform during the fall semester of 2012, Baylor had approximately seven years of experience using a home‐grown early‐warning system to identify at‐risk students early in their freshman years.79 However, as University personnel began to research the underlying causes of attrition at Baylor, the inherent limitations of the system, which relied on information about student performance electronically submitted by professors, became apparent. During the spring semester of 2011, a committee comprising staff from the Success Center, the department of Information Technology, and the Office of Academic Enrollment was formed to investigate the potential of third‐party software to help identify at‐risk students based on a more representative data set. 74
Beatrice, J. Telephone interview. Op cit. Egleton, T. and Beatrice, J. Op cit. p. 10. 76
Beatrice, J. Telephone interview. Op cit. 77
Egleton, T. and Beatrice, J. Op cit. p. 17. 78
“Search for Schools, Colleges, and Libraries.” National Center for Education Statistics. Op cit. 79
Unless otherwise noted, information in this section provided by: Vanderpool, S. Assistant Vice Provost for Academic Enrollment Management. Baylor University. Telephone interview. August 19, 2013. 75
© 2013 Hanover Research | Academy Administration Practice
19
Hanover Research | October 2013
According to Dr. Sinda Vanderpool, Baylor University’s Assistant Vice Provost for Academic Enrollment Management, the committee identified two primary functions that the student retention platform should fulfill: identifying at‐risk students based on cognitive and non‐
cognitive data, and enable effective communication between all campus personnel that are involved in the various components of academic success. In order to successfully integrate the platform into the overall advising system, the committee also sought a program that interfaced with the student information system, Ellucian’s Banner, and would meet the ease‐of‐use demands of the relevant campus personnel. After nearly 18 months of research, the committee decided that EBI’s MAP‐Works platform best‐suited the institution’s needs. In less than a month from purchasing the software agreement, the University’s IT department, with the help of a paid consultant from EBI, had installed the platform and had incorporated the program into its retention strategy for incoming and transfer students. Unlike many other institutions implementing a new student retention platform, Baylor elected not to pilot the system with a limited number of students. Instead, the University registered nearly 3,700 students in the system, and chose to place strict limitations on the number of campus personnel that could directly access the system. During the first year of implementation, only staff from the Success Center and a select number of academic and residence hall advisors were granted access to the system, totaling between 50 and 60 campus personnel. Dr. Vanderpool believes this strategy was widely successful in allowing the University to develop a full scale response to at‐risk students, while limiting the long‐
term detriment that may have arisen from challenges associated with early implementation. For the second year of implementation, Baylor has enrolled approximately 200 advisors and faculty member on the platform. The MAP‐Works platform relies heavily on data from a series of surveys to identify at‐risk students. Accordingly, Dr. Vanderpool indicates that the single most important component for effective system operation is ensuring that students complete the self‐assessment survey. During the fall semester, Baylor launched a large‐scale campaign to encourage incoming freshman and transfer students to take the online survey. Students were alerted of the survey during orientation activities and through an email alert, and were continuously encouraged to take the survey during the compulsory first‐year seminar and by residence hall staff. After leaving the fall survey open for three weeks, Baylor saw a 78 percent freshman response rate. However, the response rate for the spring survey fell to only 40 percent. While Dr. Vanderpool says some decrease in student response was anticipated, she indicates that Baylor will modify its spring outreach campaign in subsequent years to encourage more students to take the survey. Dr. Vanderpool says that Baylor’s success implementing the MAP‐Works platform can largely be attributed to one strategic decision: to implement the system on a small scale, while maintaining widely inclusive practices and effective communication throughout the process. During the selection process, Baylor actively sought the input of a wide‐range of University actors – from advisors to academic faculty – which generated a certain excitement and increased buy‐in from disparate departments throughout the institution. Baylor’s limited roll‐out of the implementation, however, allowed the University to develop © 2013 Hanover Research | Academy Administration Practice
20
Hanover Research | October 2013
internal systems on a smaller scale, as well as develop feasible solutions for unanticipated system glitches. Though hesitant to draw any direct causal relationship with the MAP‐Works platform, Dr. Vanderpool notes that after only one year of using the system, the University will likely have the highest fall‐to‐fall retention rate on record. BOWIE STATE UNIVERSITY Bowie State University is a historically black college located in the Baltimore‐Washington metropolitan area, and is part of the University System of Maryland.80 The institution has a total enrollment of approximately 4,500 undergraduate students, with a high‐proportion of first‐generation and economically disadvantaged students.81 Spurred by a recent decline in the second‐year retention rate – which fell to near 70 percent – the University launched a comprehensive student retention strategy with two primary goals: to track a wider range of student data related to student success, and to bring at‐risk students to the attention of resource staff before they withdraw.82 In 2011, Bowie State University implemented two software platforms developed by Starfish Retention Solutions, Early Alert and Connect, to automatically generate data analytics and communicate information about at‐risk students to academic advisors in the University’s Student Success and Retention Center. In addition to identifying at‐risk individuals within the student body, Bowie State uses Early Alert’s analytic capacity to identify retention trends at the class, department, and college levels, in order to assess systemic factors that may contribute to student performance and to measure progress towards institutional goals.83 Bowie State then uses Connect to establish lines of contact between at‐risk students and appropriate campus resources, and manage academic interventions.84 In order to gather information from the University’s ERP and SIS systems – PeopleSoft and Blackboard – Bowie State developed proprietary “middleware” to extract and transfer relevant data to the web‐based Early Alert platform.85 This proprietary middleware collects student data related to demographics, socioeconomic status, program involvement, academic performance, community involvement, and intermediate and final grades. Though Early Alert is designed to directly interface and collect data from a variety of ERPs, SISs, and LRMs, Bowie State decided to collect information using their middleware to allow for an easier transition to a new LMS in the near future.86 80
“USM Institutions.” University System of Maryland. http://www.usmd.edu/institutions/ [1] “Search for Schools, Colleges, and Libraries.” National Center for Education Statistics. Op cit. [2] Forsythe, R., et. al. “Two Case Studies of Learner Analytics in the University of Maryland System.” EduCause Review Online. August 13, 2012. http://www.educause.edu/ero/article/two‐case‐studies‐learner‐analytics‐
university‐system‐maryland 82
Ibid. 83
Chacon, F.,et. al. Op cit. p. 3. 84
Forsythe, R. et al. Op cit. 85
Ibid. 86
Chacon, F. et al. Op cit. p. 6. 81
© 2013 Hanover Research | Academy Administration Practice
21
Hanover Research | October 2013
Though Bowie State administrators have found the Early Alert system to be “…an efficient tool for monitoring individual data relevant to student retention and for managing the actions of faculty and staff in response to these data,” it is still too early to determine if the system has had a tangible effect on second‐year retention.87 So far, the University has had some difficulty recruiting a sufficient number of users into the system. Though the number of effective users doubled during the second year after system implementation, the system had only 441 registered user profiles during the spring 2012 semester.88 87
88
Forsythe, R. et al. Op cit. Chacon, F. et al. Op cit. p. 6. © 2013 Hanover Research | Academy Administration Practice
22
Hanover Research | October 2013
PROJECT EVALUATION FORM
Hanover Research is committed to providing a work product that meets or exceeds partner expectations. In keeping with that goal, we would like to hear your opinions regarding our reports. Feedback is critically important and serves as the strongest mechanism by which we tailor our research to your organization. When you have had a chance to evaluate this report, please take a moment to fill out the following questionnaire. http://www.hanoverresearch.com/evaluation/index.php CAVEAT
The publisher and authors have used their best efforts in preparing this brief. The publisher and authors make no representations or warranties with respect to the accuracy or completeness of the contents of this brief and specifically disclaim any implied warranties of fitness for a particular purpose. There are no warranties which extend beyond the descriptions contained in this paragraph. No warranty may be created or extended by representatives of Hanover Research or its marketing materials. The accuracy and completeness of the information provided herein and the opinions stated herein are not guaranteed or warranted to produce any particular results and the advice and strategies contained herein may not be suitable for every partner. Neither the publisher nor the authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Moreover, Hanover Research is not engaged in rendering legal, accounting, or other professional services. Partners requiring such services are advised to consult an appropriate professional. © 2013 Hanover Research | Academy Administration Practice
23
Hanover Research | October 2013
1750 H Street NW, 2nd Floor Washington, DC 20006 P 202.756.2971 F 866.808.6585
www.hanoverresearch.com © 2013 Hanover Research | Academy Administration Practice
24