Using The College Persistence Questionnaire To Enhance Commitment Among NGCSU Cadets Back In The Antediluvian Days Why the Increased Emphasis on Retention 1. Financial and legislative pressures on colleges. 2. Global economy and automation: Heighten the need for skilled versus semi- and unskilled labor 3. Schools are becoming more sensitive to students’ needs. 4. Death of a dream: Failure to graduate often leads to reductions in students’ income and quality of life Some Students Have Special Requirements What Do We Need To Know About Our Students? It Depends Upon Your Responsibilities Working With Individual Students: What Counselors, Advisors and Faculty Need To Know 1. Which cadets are most at-risk? 2. What factor(s) are causing a particular cadet to leave the Corps? 3. We need scale scores and responses to individual items. CPQ Advisor Portal https://www.beckdavidson.com/welcome/welcome.aspx Working With Groups of Students: What Policy-Makers and Instructors Of Persistence-Oriented Courses Need To Know 1. What factor(s) distinguish students at their schools who will persist and will not persist in their educations? 2. Are our retention programs effective? Six Revelations On The Path To An Effective Retention Program (Part 1) Revelation What That Means 1. We have a retention problem. Yes, every school has a retention problem. Some schools have a greater problem than others. Nevertheless, every school includes students that do not graduate who have the capacity to graduate. 2. Someone needs to do something. Retention problems will not just disappear; they cannot be ignored except at great cost, they must be addressed. Following this revelation, a policy maker often tasks someone or a group to deal with the retention issue. 3. We need to know what works. You know there is a problem. Now you need to determine what it is and what to do about it. A common practice is for schools to model their initial retention activities after a sister school that claims success or a program reported in the empirical literature. Six Revelations On The Path To An Effective Retention Program (Part 2) Revelation What That Means 4. We need to know if our activities decrease attrition. Many schools believe that their retention-reducing activities are effective. Few schools, however, could provide a skeptic with information demonstrating that their activities are attenuating attrition or having beneficial effects. Realizing this is a big step forward. 5. We need to focus our resources, time, and energy on those activities that will have the greatest impact on our students. Knowing what affects retention at your school is not enough. We must prioritize. There are more variables that affect persistence decisions than any school can address. Resources are limited so we must concentrate our energies on those variables that most strongly influence retention at our institution. 6. We need to establish an ongoing data collection system to evaluate our progress, augment our strengths, and overcome our weaknesses. Students change, schools change, programs change. An inplace assessment system will allow us to demonstrate our achievements, make steady improvements, and accommodate changes to our educational systems. Strategies For Improving Retentioin Rely On Someone’s Professional Expertise Subject matter experts (SMEs) have invaluable experience. Most of us were employed because we are military and/or academic SMEs. Rely On Someone’s Professional Expertise Problems With SMEs: SMEs can make mistakes. Some SMEs may not fully appreciate the consequences of their decisions. Rely On Someone’s Professional Expertise King MacArthur Marshall Problems With SMEs: Even the most expert of SMEs sometimes disagree. How do you determine whose position to support? Look To The Literature For Guidance In comparison to continuing-generation (CG) students firstgeneration (FG) students tend to be: 1. Female 2. Older 3. Have dependents 4. From a lower SES FG students also tend to have: 1. Lower standardized test scores 2. Poor math, reading and critical thinking skills 3. Low self-images 4. Higher attrition rates 5. Poor grades 6. Low levels of academic and social integration Problem With Relying On The Literature The Golden Path To Effective Intervention: Attitude Can You Imagine Someone’s Photograph In This Box? Aiming for 100% Retention is an unrealistic and counterproductive goal. Not everyone should be a military officer. The Golden Path To Effective Intervention: Attitude Take the offensive! We have nothing to be defensive about regarding retention. Our mission is to clear a path for others to follow. The Golden Path To Effective Intervention: Empirical Evidence Data Driven Decision Making Goals In Developing the CPQ 1. Greatest Unmet Need: A better way of determining how students respond to the first 6 or 8 weeks of college 2. We need an instrument of great breadth that is applicable at many schools and many types of students. 3. The instrument must provide information pertinent to personnel working with individuals as well as groups of students. 4. The instrument must identify the factors that most strongly govern retention at a particular school or for a particular individual. 5. The instrument must not be too long. 6. Data must be immediately available; we can’t wait. 7. The instrument must have a mechanism for addressing issues pertinent to a particular school. Development of the CPQ-1: Procedure 1. Conducted an extensive review of the retention literature, at least 150 studies 2. Identified at least 85 variables (WOW) that had been associated with retention 3. Wrote items addressing these 85 variables 4. Automated the administration and data recording system 5. Administered the questionnaire to more than 2000 students Simplify, Simplify, Simplify: Why Thoreau Would Like Factor Analysis 1. Reduced 85 variables to 6 factors 2. Scales tend to be more reliable than single items 3. Findings reported in Journal of College Student Retention Reasons For Developing CPQ-2 1. Several CPQ-1 scales could benefit from additional items. 2. Version 1 did not assess financial pressures, the motivation to perform academic work, or the student’s belief that he or she was capable of performing successfully. CPQ Components: Student Background Form The Student Background Form consists of three types of items: 1. Demographic and Family: (e.g., sex, ethnicity, marital status, parent’s education) 2. Work and Financial Resources: (e.g., hours working, sources of income) 3. Reasons For Attending: (e.g., reputation, location, friends) Principal Components Analysis: Participants and Procedure 1. Two thousand eight hundred and thirty-eight undergraduates from Angelo State University, Appalachian State University, Greenville Technical College, and Troy UniversityMontgomery served. 2. Students responded online, most taking less than 40 minutes to complete the 82-item questionnaire. Results 1. Responses were converted to 5-point “favorability” scores, depending on whether the answer indicated something positive or negative about the student’s college experience. The solution produced ten factors with eigenvalues greater than 1.0. 2. A principal components analysis was performed on the favorability scores of the 82 items using a direct oblimin rotation. 3. All items with pattern coefficients of .40 or higher were retained for further analysis. Results Just Keep On Coming 5. A second principal components analysis with a direct oblimin rotation was conducted on the resultant 54 items to ensure that the deletion of questions did not cause substantial changes in the pattern coefficients. 6. The findings of the two analyses were similar. Item deletion did not have a pronounced effect on the coefficients. 7. All correlations between components were less than .32. 8. Alpha levels were in excess of .70 for all factors. CPQ Components: Student Experience Form 1. Institutional Commitment: Loyalty, intention to reenroll, confidence in school choice 2. Degree Commitment: Personal importance, students’ supportive network, value of degree 3. Academic Integration: Positive views of instruction, instructors, and own intellectual growth; awareness of connections between academics and career. 4. Social Integration: Sense of belonging, shared values, and similarity to others; positive involvement behaviors. 5. Collegiate Stress: Feelings of distress, pressure, and sacrifice. CPQ Components: Student Experience Form 6. Academic Motivation: Interest and enjoyment in academic tasks; willingness to spend extra time. 7. Scholastic Conscientiousness: Timely performance of academic responsibilities. 8. Academic Efficacy: Confidence in academic skills and outcomes. 9. Financial Strain: Financial worries and difficulties; sense of disadvantage relative to others. 10. Advising Effectiveness: Positive views of advising and school communication processes. Did Anyone Notice? That The Ten Student Experience Scales Of The CPQ Assess Many Of The Variables Composing Institutional Effectiveness CPQ Components: Institution Specific Form This form addresses questions not covered by the CPQ but are important to the Corps of Cadets. 1. How confident are you that the Corps of Cadets is right for you? 2. How do you feel about the Corps’ PT requirement? 3. How likely is it that you will participate in the Corps next semester? 4. How do you feel about the policy of requiring cadets to wear a military uniform while on campus? CPQ Components: Institution Specific Form 5. How do you feel about mandatory quarters for beginning Cadets? 6. During your last year in high school, on the average how much time did you spend in exercise or sports? 7. How likely is it that you will complete eight semesters (or equivalent) of military science? 8. How do you feel about the residency policy where you must withdraw from the university for one academic year if you drop out of the Corps of Cadets? How Well Does The Field Predict Retention? ACT Meta-Analysis 13 academic and non-Academic variables explain 17% of the variability of college retention across students. Validation Study: Participants and Procedure 1. Seven hundred and forty-nine freshmen from three comprehensive universities responded online to the CPQ. 2. CPQ was administered six to eight weeks into the first semester of the freshman year. 3. Most students took less than 40 minutes to complete the questionnaire. 4. Return for next Fall semester was the criterion variable. Does the CPQ Predict Retention? Identifying At-Risk Students (Individual Items) Measures Nagelkerke R2 HSR + SAT + Ethnicity + Sex + Age + Class Size (df = 6) .08 CPQ – Background Variables, not usually in student database (df = 12) .18 Student Experience Items (df = 54) .56 Using The CPQ To Understand An At-Risk Group: Individual Items Compared to other freshmen Student Support Services students reported that: 1. They had a bigger influence on such matters as course offerings, rule and regulations, and registration procedures. 2. They were more satisfied with the academic advisement they received. 3. They more strongly preferred tasks that make extensive use of their own ideas and allowed expression of their individuality. 4. Fewer of their close friends were at Appalachian State. 5. It was more difficult for their families to handle the cost of college. CPQ Items Predicting Institutional Commitment: Community College Data Argues For Targeting Item Business Nursing Arts & Sciences 1 Degree Commitment Degree Commitment Academic Integration 2 Degree Commitment Degree Commitment Academic Integration 3 Degree Commitment Degree Commitment Support Services 4 Academic Integration Academic Integration Acad. Conscientiousness 5 Academic Integration Academic Integration Support Services 6 Academic Integration Academic Integration Social Integration The Importance Of Student Experiences: Participants and Procedure 1. 701 freshmen from three comprehensive universities. 2. CPQ was administered six to eight weeks into the first semester of the freshman year. 3. Most students took less than 40 minutes to complete the questionnaire. How Well Does The CPQ Predict Retention? Regress retention (return the following Fall semester) on the following predictor variables Predictor Variables Variables typically in student database (df=15). Variables from Background Form not typically in database (df=12) CPQ Student Experience Scales Nagelkerke R Square .13 .23 .38 Where We Are Going Degree Commitment .22 Academic Integration .24 .14 .33 .38 .23 .11 .05 -.07 Social Integration .30 Support Services Satisfaction Institutional Commitment .10 Academic Conscientiousness .19 Summary and Implications 1. CPQ is a valid predictor of whether freshmen will return for their sophomore years. 2. CPQ identified why individual students dropped out. Summary and Implications 3. CPQ identified those variables that best distinguished those students who would from those students who would not continue their educations at these three institutions. 4. This information would be very helpful to policy makers deciding what retention programs to develop. What Retention Activities At Your School Do You Believe Have Been Successful? A Sample Mentoring programs have been successful. Small class sizes, instructors taking personal interest in students-increasing engagement. Low student to advisor ratios in many programs. Reaching more students by e-mail rather than US mail. Tutoring The Student Solutions Center has shown some success. What Retention Activities At Your School Do You Believe Have Been Successful? A Sample Meeting 1:1 with students to discuss their method of study, getting them to set up study plans, teaching them to underline or otherwise ID key terms in test questions, eliminating wrong answers by crossing them out, dealing with test anxiety. All efforts have had some impact, however, all could be improved. The Summer Bridge Program. Academic support in freshman dorms has been very beneficial for at-risk or students – We have increased the number of dorm activities to foster the growth of community and sense of home for the students. Setting The Stage For Tomorrow Let Me Invite You To List Three Retention Activities That You Believe Have Been Successful at NGCSU College Persistence Questionnaire Identifies Factors Influencing Commitment Of Military Cadets Goals Of This Investigation 1) Determine if the CPQ predicts commitment to the Corps 2) Identify those factors that are most strongly associated with commitment 3) Offer guidelines for increasing the retention of Cadets at NGCSU 4) Identify those cadets most at-risk for discontinuing their educations CPQ Components 1. Student Background Form: Consists of three types of questions: Demographic and Family items (sex, ethnicity, marital status, parent’s education), Work and Financial Resources items (hours working, sources of income), and Reasons for Attending Higher Education items (reputation, location). 2. Student Experience Form: Ten psychometrically validated scales that assess students’ interactions with the academic and social environments. More About CPQ Components 3. Ten Student Experience Form Scales: Institutional Commitment, Degree Commitment, Academic Integration, Collegiate Stress, Social Integration, Academic Motivation, Scholastic Conscientiousness, Academic Efficacy, Financial Strain, and Advising Effectiveness. 4. Institutional Specific Form: Additional questions pertaining to the Corps of Cadets. Participants and Procedure 1. 582 Cadets from NGCSU responded in groups online to the CPQ. 2. CPQ was administered six to eight weeks into the first semester of the freshman year. 3. Most students took less than 40 minutes to complete the questionnaire. Commitment To Corps And NGCSU Overlap 1. Created Commitment To Corps scale from Institution specific items. 2. Correlation of Commitment to Corps and Institutional Commitment scale was .71. 3. Created single scale, Commitment Scale (CS), from corps and institutional items. 4. Cronbach’s alpha of CS was .78. Does The CPQ Predict Commitment To The Corps? Concurrent Validity: Regress CS scores on items from the Student Background and Student Experience Forms as well as three items from the Institution Specific Form, excluding items composing the CS scale. F(68, 513) = 6.75, p < .001, R = .69, R2 = .47 Student Background Form Cadets who chose NGCSU because of the following factors were more committed than cadets not endorsing these attributes. 1. Academics 2. Overall reputation 3. Appealing location Student Background Form Although these correlations were not strong, they may: 1. Aid in recruiting and identifying the types of prospective students who are likely to stick. 2. Indicate that cadets with certain background characteristics may need additional services once they matriculate. Student Experience Form 1. Eight of nine Student Experience scales were statistically significant, p < .01, two-tailed. 2. The most robust correlations with CS scores were the Academic Integration, Degree Commitment, and Social Integration scales. This result provides a guide to those seeking to augment commitment to the Corps. Improvements in the qualities underlying these scales will be particularly effective in enhancing commitment. 3. A complementary approach is to develop an intervention for a variable that is “networked” with other variables (e.g., advising). Where We Are Going Degree Commitment .22 Academic Integration .24 .14 .33 .38 .23 .11 .05 -.07 Social Integration .30 Support Services Satisfaction Institutional Commitment .10 Academic Conscientiousness .19 Student Background Form, Student Experience Form, Institution Specific Form: Comparisons 1. Student Background Form: Questions entered in isolation accounted for 16% of the variance. 2. Student Experience Form: Questions entered in isolation accounted for 43% of the variance. 3. Institution Specific Items: Questions assessing attitudes about requiring cadets to wear their uniform on campus, their views on PT, and whether they exercised in high school accounted for 15% of the variance when entered in isolation. Two Important Takeaways 1. The Student Experience Form was a much better predictor of commitment than any items on the Student Background Form. This finding indicates that experiences during the first six to eight weeks have a profound impact on commitment. 2. It also argues against attempting to determine commitment during orientation or at an earlier date. Limitations And Recommendations 1. This study needs to be followed by an analogous investigation in which retention (return for the sophomore year) is the outcome measure. 2. The sample size was small for those regressions employing individual items as predictors. Nevertheless, given the strength of the findings, we can be confident of the reliability of the outcomes. 3. It is encouraging that Student Experience items tend to be better predictors of commitment than Student Background items. Student Experience items are more subject to change. Limitations And Recommendations 4. Efforts to augment cadet retention should focus on increasing Academic Integration, Social Integration, and Degree Commitment. 5. An alternative strategy is to network these variables and commitment by improving Advising Effectiveness. 6. The CPQ was administered only once in the Fall semester. Many cadets drop out after the Spring semester. Administering the CPQ in the Spring might identify students who were now at-risk but were not at-risk in the Fall. Furthermore, changes in scales scores could prove diagnostic. What Does This Tell You About These Cadets? 2 1.5 1 0.5 0 -0.5 -1 -1.5 -2 Smith Jones FALL 1.75 -0.5 SPRING 1.1 1.1 Intervention Strategies: Transforming Cadets Into Officers Working With Individual Students: What Counselors, Advisors and Faculty Need To Know 1. Which cadets are most at-risk? 2. What factor(s) are causing a particular cadet to leave the Corps? 3. We need scale scores and responses to individual items. Two Measures of Retention Institutional Commitment Score Advantages: Provides a means of identifying at-risk students after 6 weeks of college. The Student Experience variables that are correlated with Institutional Commitment are subject to change. Disadvantage: Is not perfectly correlated with who actually returns Returning Sophomores Advantage: Indicates who actually returns. Disadvantage: It is too late to help at-risk students when this data becomes available. Beck’s List of Invariably Violated Principles 1. The earlier the diagnosis the better the prognosis 2. More information generally enhances the prediction of at-risk students 3. The more time you have to gather information the more useful information you can acquire What Do You Need To Do To Reduce Attrition? Not An Easy Task 1. Identify at-risk students within first 6 to 8 weeks of college 2. Determine why a student is at-risk 3. Craft a powerful message that addresses that student’s issues 4. Quickly alter the direction of that student’s life A Tale Of Two Portals The Student and Advisor Portals were developed because early users of the CPQ requested an online system. Student Portal: Administers the CPQ to the student and saves the information to a database. Advisor Portal: Retrieves information from the database and uses this information to identify at-risk students and indicate why that individual is at-risk. Advisor Portal: Identifying At-Risk Students The Advisor Portal provides school personnel with a tool for making one-on-one sessions with students helpful, effective, and fruitful in augmenting retention rates. It identifies individual students who are most at-risk and supplies an indepth analysis of the problems each one is experiencing. Testimonials of advisors and counselors who have used it are uniformly and overwhelmingly positive in describing its value in their sessions with individual students. Working With Groups of Students: What Policy-Makers and Instructors Of Persistence-Oriented Courses Need To Know 1. What factor(s) distinguish students at their schools who will persist and will not persist in their educations? 2. Are our retention programs effective? Meta Analysis: Intervention Effects On College Performance And Retention 1. Robbins, Oh, Le, and Button (2009) 2. Examined the effects of interventions (academic skill, self-management, socialization, and first-year experience) on academic performance and retention. Results: Intervention Effects On College Performance And Retention 1. Academic skill training (e.g., study habits, learning strategies) had the strongest effects on academic performance. Self-management skill training (e.g., self-regulatory, emotional control) had a lesser effect. 2. Self-management training had the strongest effect on retention. Socialization interventions also showed a significant effect on retention. 3. First-year experience programs had “weak to negligible effects on academic performance and retention.” Why Do First Year Courses Have Such Modest Effects On Retention? Interpreting The Data They may: 1. Be ineffective. 2. Have very broad goals which are difficult to measure. 3. Be very diverse. Programs may very different goals. 4. Have very little structure or guidance as to objectives. Instructors may simply do what they want to do and the institution may hope that this has some benefit. What Works In Student Retention? Student And Institutional Characteristics Item Mean Level of student preparation for college-level work Adequacy of personal financial resources Student study skills Amount of financial aid available to students 3.90 3.90 3.80 3.65 Level of student motivation to succeed Level of student commitment to earning a degree Level of job demands on students 3.64 3.56 3.52 Student low socio-economic status Student educational aspirations and goals Student personal coping skills 3.49 3.36 3.34 What Works In Student Retention? Retention Activities Item Mean Academic advising center Increased number of academic advisors Advising interventions with selected populations Comprehensive learning assistance center/lab 3.98 3.98 3.93 3.92 Supplemental instruction Programs for first-generation students Required on-campus housing for freshmen 3.91 3.90 3.86 Reading center/lab Tutoring Summer bridge program 3.86 3.84 3.83 What Activities Differentiate Schools With High Versus Low Retention Rates? Item Increased number of academic advisors Top Bottom 43% 30% Advising with selected populations Supplemental instruction Summer bridge program 96% 75% 75% 82% 64% 49% Program for honors students Integration advising and first-year programs Training for non-faculty academic advisors Faculty mentoring 91% 72% 84% 74% 73% 48% 67% 45% Living/learning communities (residential) Staff mentoring 87% 57% 60% 31% What Are The Three Most Important Retention Activities? Item % Selected Freshman seminar/university 101 (credit) Supplemental instruction Tutoring Living/learning communities (residential) 24% 16% 15% 14% Advising with selected populations Placement in courses based on test scores Academic advising center 14% 13% 12% Summer orientation Early warning system 11% 10% Which Attrition-Reducing Activities Should We Use? That depends upon: 1. What variables differentiate cadets who become military officers from those who dropout. 2. What the empirical literature indicates about the effectiveness of various attrition-reducing practices. 3. What resources and personnel we can direct towards retention. How Do We Know If Our Actions Are Hitting The Mark? That Brings Us To Measurement Issues Your Main Outcome Measures 1. Grades 2. Institutional Commitment 3. Sophomore Retention Your Palette Of Measures There must be a logical relationship between your objectives and your measures. Your available indices include: Measures Retention Academic Integration Graduation Rates Financial Strain GPA Social Integration Institutional Commitment Degree Commitment High School Rank, Standardized Tests Collegiate Stress Background (e.g., sex, parental education) Academic Motivation Work and Finances (e.g., hours working) Scholastic Conscientiousness Reasons To Attend (e.g., friends, location) Academic Efficacy Institution Specific Items Advising Effectiveness Activities With Multiple Objectives Activities Training for non-faculty academic advisors Faculty mentoring Staff mentoring Freshman seminar/university 101 (credit) Early warning system Measures: These activities are very generally stated. Therefore, they should have very diverse effects. If successful, a broad range of indices will be impacted. The measures taken should depend upon the particular objectives of the activities. Some Attributes, Activities And Their Measures Activities Financial resources Measures Financial Strain Motivation to succeed Academic Integration, Academic Motivation, Scholastic Conscientiousness Commitment to Degree Commitment, Academic Integration earning a degree Job demands Educational aspirations Work and Finances Section, Financial Strain Degree Commitment, Academic Integration Coping skills Collegiate Stress, Scholastic Conscientiousness, Academic Efficacy High School Rank, Standardized Tests, Multiple Student Experience Scales Preparation for college-level work Assessing At-Risk Or Special Populations Activities Students of low socio-economic status Advising interventions with selected student populations Programs for first-generation students Programs for honors students Programs for minority students Measures: These activities attempt to address problems common to atrisk or special populations. Measures on the Student Background Form (e.g., parents’ education, ethnicity) and student database could help identify these groups. The goals of these programs vary considerably, therefore, the measures used to assess their effectiveness varies accordingly. Academic Enhancement Activities Activities Student study skills Comprehensive learning assistance center/lab Supplemental instruction Reading center/lab Tutoring Placement of students in courses based on test scores Program for honors students Measures: Academic Integration, Academic Efficacy, Academic Motivation, Scholastic Conscientiousness, Collegiate Stress. Social Integration Activities Activities Required on-campus housing for freshmen Living/learning communities (residential) Program for international students Summer bridge program Integration of advising with first-year transition programs Summer orientation programs Measures: A major objective of these activities appears to be to enhance the extent that students identify with and feel part of the academic community. Other objectives of these programs should be measured, in addition to the Social Integration scale of the CPQ. Advising Activities Activities Academic advising center Increased number of academic advisors Advising interventions with selected student populations Measures: Advising is critical because it networks with so many other variables. The Advising Effectiveness scale of the CPQ should be used to assess these endeavors. Other variables, depending on the goals of the program should also be measured.. What Is A Reasonable Assessment Strategy? 1. Gather attrition-related information from your Institutional Research Office. 2. Administer the CPQ to obtain Student Background and Experience variables. 3. Combine the Institutional Research and CPQ variables into a dataset. 4. Determine who returns the following Fall semester. 5. Annually prepare Institutional Commitment and Retention reports. 6. Examine data across years to anticipate trends. For Your Attention. Do Not Be A Stranger. This Is A beginning, Not An Ending!