Kevin Eagan, Sylvia Hurtado, Bryce Hughes, & Tanya Figueroa, UCLA Association for Institutional Research Annual Forum Orlando, FL May 28, 2014 Colleges and universities have been challenged to produce an additional one million STEM degrees over the next decade The NSF, NIH, and institutions have invested heavily in interventions which have been shown to improve academic performance and retention in STEM Supplemental instruction and faculty mentoring are two cost-effective types of interventions often already provided at institutions To examine the effect of supplemental instruction and faculty mentoring on STEM identity, intentions to enroll in a STEM graduate program, and commitment to a STEM career To utilize a quasi-experimental statistical modeling technique to better isolate the effects of these interventions on the three outcomes of interest Developed by Deanna Martin at the University of Missouri-Kansas City Targets “at-risk courses” as opposed to “at-risk students” Peer-facilitated sessions focused on problem solving and enhancing course material Voluntary; not remedial Supplemental instruction has been shown to improve academic performance and term-toterm retention rates in single-institution studies Intentional support, as opposed to happenstance faculty-student interactions Consists of professional and personal support activities Faculty mentoring also improves academic performance and retention However, students who typically seek faculty mentoring are students already positioned to succeed Situated Learning Theory (Lave and Wenger, 1991) STEM as a community of practice Learning as legitimate peripheral participation Through the process of learning new members become more central to the community through identifying with the community Social Learning Theory (Bandura, 1971) Learning is a cognitive and behavioral process that occurs through both observation and modeling Learning results from a dynamic interaction between cognition, environment, and behavior Theory of Planned Behavior (Ajzen, 1991) Intentions are a crucial precursor to behavior Data Source and Sample Longitudinal Dataset ▪ 2004 CIRP Freshman Survey ▪ 2008 CIRP College Senior Survey NIH and NSF funding augmented participation of MSI’s and STEM-producing institutions 4,166 longitudinal student cases who intended to major in STEM across 237 institutions Dependent Variables STEM identity – Four-item factor ▪ Becoming an authority in my field ▪ Making a theoretical contribution to science ▪ Receiving recognition from others for contributions to my field ▪ Finding a cure for a health problem Commitment to a STEM career (dichotomous) Intentions to pursue STEM graduate study (dichotomous) Independent Variables Participation in Supplemental Instruction Receipt of Faculty Mentoring – 9-item factor Each is dichotomized for the propensity score matching analysis ▪ Supplemental instruction: Students who participated (frequently or occasionally) versus non-participants ▪ Faculty mentorship: Above average (>50) mentorship versus average or below average (<50) Control variables Background characteristics Pre-college academic preparation Pre-college aspirations and expectations Initial measures of STEM identity, plans to pursue a STEM career, and expectations of pursuing graduate study in STEM (i.e. Pretest) Analytic Strategy Missing data addressed through EM algorithm Propensity score matching ▪ Probit regression ▪ Precollege characteristics and experiences predicting mentorship and supplemental instruction participation ▪ Nearest neighbor matching ▪ T-tests conducted with matched sample for each intervention for each of the three outcomes Limitations Secondary data analysis Propensity score matching only as good as the variables available Two outcomes measure intentions rather than actual behavior HS GPA (+) STEM identity as a freshman (+) HPW talking with teachers outside of class in HS (+) Race: Latino vs. White (-) Race: Asian American vs. White (-) HS GPA (+) Mother’s education (+) STEM identity as an incoming freshman (+) Reason for attending college: Prepare for graduate school (+) HPW: Talking with teachers outside of class in HS (+) Major: Engineering or computer science (-) Concerns about ability to pay for college (-) STEM Graduate School STEM Identity Aspirations STEM Career Aspirations Mean Mean Mean Difference Sig. Difference Sig. Difference Sig. Supplemental Instruction 0.63*** 0.05* 0.03 Mentorship 1.10*** 0.06** 0.04* STEM Identity STEM Graduate School Aspirations STEM Career Aspirations Mean Mean Mean Difference Sig. Difference Sig. Difference Sig. Supplemental Instruction 0.63*** 0.05 0.06* Mentorship 1.30*** 0.07** 0.06* Supplemental instruction as a way to establish a community of practice Strengthens students’ STEM identity Increases likelihood to plan to enroll in STEM graduate programs Particularly beneficial for URM STEM identity development Faculty Mentorship Benefits of mentorship extend even after accounting for the types of students likely to receive or seek out mentorship Mentorship even more impactful for URM students’ STEM identity development Undergraduate research can be a resourceintensive intervention Supplemental instruction and faculty mentoring are additional important STEM persistence tools Structure or web of opportunity in STEM Lends support for further expansion of supplemental instruction offerings and for broader access to intentional faculty mentoring Propensity score matching provides a method for reducing bias due to participant selfselection when assessing STEM interventions Examine effects of mentorship, supplemental instruction, and other interventions on longer-term outcomes STEM graduate program enrollment Entry into STEM workforce Administrative Staff: Dominique Harrison Faculty/Co-PIs: Sylvia Hurtado Kevin Eagan Graduate Research Assistants: Tanya Figueroa Bryce Hughes Undergraduate Research Assistants: Paloma Martinez Robert Paul Papers and reports are available for download from project website: http://heri.ucla.edu/nih Project e-mail: [email protected] This study was made possible by the support of the National Institute of General Medical Sciences, NIH Grant Numbers 1 R01 GMO71968-01 and R01 GMO71968-05, the National Science Foundation, NSF Grant Number 0757076, and the American Recovery and Reinvestment Act of 2009 through the National Institute of General Medical Sciences, NIH Grant 1RC1GM090776-01. This independent research and the views expressed here do not indicate endorsement by the sponsors.