Kevin Eagan, Sylvia Hurtado, Bryce Hughes, & Tanya Figueroa, UCLA
Association for Institutional Research Annual Forum
Orlando, FL
May 28, 2014
Colleges and universities have been challenged
to produce an additional one million STEM
degrees over the next decade
 The NSF, NIH, and institutions have invested
heavily in interventions which have been shown
to improve academic performance and retention
in STEM
 Supplemental instruction and faculty mentoring
are two cost-effective types of interventions
often already provided at institutions



To examine the effect of supplemental
instruction and faculty mentoring on STEM
identity, intentions to enroll in a STEM
graduate program, and commitment to a
STEM career
To utilize a quasi-experimental statistical
modeling technique to better isolate the
effects of these interventions on the three
outcomes of interest
Developed by Deanna Martin at the University of
Missouri-Kansas City
 Targets “at-risk courses” as opposed to “at-risk
students”
 Peer-facilitated sessions focused on problem
solving and enhancing course material
 Voluntary; not remedial
 Supplemental instruction has been shown to
improve academic performance and term-toterm retention rates in single-institution studies





Intentional support, as opposed to
happenstance faculty-student interactions
Consists of professional and personal support
activities
Faculty mentoring also improves academic
performance and retention
However, students who typically seek faculty
mentoring are students already positioned to
succeed

Situated Learning Theory (Lave and Wenger, 1991)
 STEM as a community of practice
 Learning as legitimate peripheral participation
 Through the process of learning new members become
more central to the community through identifying with
the community

Social Learning Theory (Bandura, 1971)
 Learning is a cognitive and behavioral process that occurs
through both observation and modeling
 Learning results from a dynamic interaction between
cognition, environment, and behavior

Theory of Planned Behavior (Ajzen, 1991)
 Intentions are a crucial precursor to behavior

Data Source and Sample
 Longitudinal Dataset
▪ 2004 CIRP Freshman Survey
▪ 2008 CIRP College Senior Survey
 NIH and NSF funding augmented participation of
MSI’s and STEM-producing institutions
 4,166 longitudinal student cases who intended to
major in STEM across 237 institutions

Dependent Variables
 STEM identity – Four-item factor
▪ Becoming an authority in my field
▪ Making a theoretical contribution to science
▪ Receiving recognition from others for contributions to
my field
▪ Finding a cure for a health problem
 Commitment to a STEM career (dichotomous)
 Intentions to pursue STEM graduate study
(dichotomous)

Independent Variables
 Participation in Supplemental Instruction
 Receipt of Faculty Mentoring – 9-item factor
 Each is dichotomized for the propensity score
matching analysis
▪ Supplemental instruction: Students who participated
(frequently or occasionally) versus non-participants
▪ Faculty mentorship: Above average (>50) mentorship
versus average or below average (<50)

Control variables
 Background characteristics
 Pre-college academic preparation
 Pre-college aspirations and expectations
 Initial measures of STEM identity, plans to pursue
a STEM career, and expectations of pursuing
graduate study in STEM (i.e. Pretest)

Analytic Strategy
 Missing data addressed through EM algorithm
 Propensity score matching
▪ Probit regression
▪ Precollege characteristics and experiences predicting
mentorship and supplemental instruction participation
▪ Nearest neighbor matching
▪ T-tests conducted with matched sample for each
intervention for each of the three outcomes

Limitations
 Secondary data analysis
 Propensity score matching only as good as the
variables available
 Two outcomes measure intentions rather than
actual behavior



HS GPA (+)
STEM identity as a freshman (+)
HPW talking with teachers outside of class in
HS (+)









Race: Latino vs. White (-)
Race: Asian American vs. White (-)
HS GPA (+)
Mother’s education (+)
STEM identity as an incoming freshman (+)
Reason for attending college: Prepare for
graduate school (+)
HPW: Talking with teachers outside of class in
HS (+)
Major: Engineering or computer science (-)
Concerns about ability to pay for college (-)
STEM Graduate
School
STEM Identity
Aspirations
STEM Career
Aspirations
Mean
Mean
Mean
Difference Sig. Difference Sig. Difference Sig.
Supplemental
Instruction
0.63***
0.05*
0.03
Mentorship
1.10***
0.06**
0.04*
STEM Identity
STEM Graduate
School
Aspirations
STEM Career
Aspirations
Mean
Mean
Mean
Difference Sig. Difference Sig. Difference Sig.
Supplemental
Instruction
0.63***
0.05
0.06*
Mentorship
1.30***
0.07**
0.06*

Supplemental instruction as a way to establish a
community of practice
 Strengthens students’ STEM identity
 Increases likelihood to plan to enroll in STEM graduate
programs
 Particularly beneficial for URM STEM identity
development

Faculty Mentorship
 Benefits of mentorship extend even after accounting for
the types of students likely to receive or seek out
mentorship
 Mentorship even more impactful for URM students’ STEM
identity development

Undergraduate research can be a resourceintensive intervention
 Supplemental instruction and faculty mentoring are
additional important STEM persistence tools
 Structure or web of opportunity in STEM
Lends support for further expansion of
supplemental instruction offerings and for
broader access to intentional faculty mentoring
 Propensity score matching provides a method
for reducing bias due to participant selfselection when assessing STEM interventions


Examine effects of mentorship, supplemental
instruction, and other interventions on
longer-term outcomes
 STEM graduate program enrollment
 Entry into STEM workforce
Administrative Staff:
Dominique Harrison
Faculty/Co-PIs:
Sylvia Hurtado
Kevin Eagan
Graduate Research Assistants:
Tanya Figueroa
Bryce Hughes
Undergraduate Research Assistants:
Paloma Martinez
Robert Paul
Papers and reports are available for download from project website:
http://heri.ucla.edu/nih
Project e-mail: herinih@ucla.edu
This study was made possible by the support of the National Institute of General Medical Sciences, NIH Grant Numbers 1 R01
GMO71968-01 and R01 GMO71968-05, the National Science Foundation, NSF Grant Number 0757076, and the American Recovery
and Reinvestment Act of 2009 through the National Institute of General Medical Sciences, NIH Grant 1RC1GM090776-01. This
independent research and the views expressed here do not indicate endorsement by the sponsors.