MSU Retention Study
Preliminary Findings
Acknowledgements
• Chris Gilstrap – Graduate Student (Computer
Science and B.S. in statistics).
• Chris Fastnow
• Phil Gaines & Erika Swanson
• University of Central Florida.
• Greg Young and Allen Yarnell for supporting
study.
Purpose of the Study
 Develop an understanding of who (incoming freshman)
are more likely to “persist” and who are more likely to
“withdraw” from MSU using incoming variables (H.S.
GPA, H.S. Percentile, ACT Verbal, ACT Math, etc).
 Develop a “retention profile” for departments and key
centers on campus.
 Develop a “profile” of persisters and withdrawers.
 Develop an understanding of why persisters and
withdrawers leave MSU (what variables may affect
departure?).
• Develop a sense of MSU’s culture or orientation
towards retention (primarily faculty/staff based).
The Odyssey of Data Mining
“Data mining means the discovery of knowledge
from (a large amount of) data.
Data mining should provide not only predictions
but also knowledge such as rules that are
comprehensible (that permit “next steps”).
H. Tsukimoto.
Data Mine
• Data mines should be triangulated with
qualitative research.
• Outcomes from data mining should “inform
and track” the process not direct the process.
• Data mining can inform an opportunity for
improvement and change.
Preliminary Facets in our “Data Mine”
• Decision Tree/Logistical Regression of
persisters and withdrawers
• Descriptive Analysis “college/department”
retention profiles
• Descriptive Analysis from data set compiled
from survey sent to freshman class of 2007.
• Inferential Analysis set compiled from survey
sent to freshman class of 2007.
Predictive Modeling
In essence we want to know:
• What must we influence to “load the coin flip?”
• Answer probably lies in some (but not
exclusively) incoming variables.
• If we understand what variables are likely to put
a student “at risk” we can respond with a
meaningful intervention by influencing incoming
variables with “environmental variables.”
Logistical Regression/Decisions Trees
• Used UCF as a model.
• Attribute 2-4% increase in retention rate using this method.
• Efficient - target those “most” at risk with intervening
methods.
–
–
–
–
–
Student success center
Meet with an advisor at least once per semester usually twice.
“Knight” success program
Residential advising
Targeted communication, etc.
• Believe “cross influencing of methods/intervention” is
contributing to an improving retention rate 11% increase in
12 years.
Logistical Regression/Decision Trees
• Model improves over time as understanding
of influences of incoming variables improves
(i.e. training the model).
• UCF can now predict with 90% accuracy of
who their persisters and withdrawers are
Input Variables Considered
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Gender
High School GPA
Ethnicity
High School Rank
Age
High School Size
Citizenship
High School Percentile
Home County
ACT Math
Home State
ACT English
Home Region
SAT Verbal
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Academic Status (Full/Part Time)
SAT Composite
College
ACT Equivalent
Department
Major
Concentration
Survey Questions 1-100
University College (yes/no)
Lived in Residence Halls (yes/no)
ACT Composite
Residence Status
SAT Math
Admission Term
Predictive Modeling
Without University GPA - MSU
Logistical Regression
Predicted to withdraw and withdrew
Predicted to persist but withdrew
0.04
0.19
Predicted to withdraw but persisted
Predicted to persist and persisted
0.11
0.61
Decision Tree
Predicted to withdraw and withdrew
0.10
Predicted to withdraw but persisted
0.12
Predicted to persist but withdrew
0.19
Predicted to persist and persisted
0.58
What we still need to do….
• Continue to run log reg/trees using other
“unconsidered” variables in an attempt to
bolster predictability and train the models.
• Develop “campus-cultural” appropriate
interventions.
• Identify “beta” group to test retention
interventions.
Retention Profile of Campus
Departments/Offices
“Run the retention” numbers by college or office
affiliation (i.e. College of EHHD, Residence Life) or
some other definable category (instate versus
out-of-state).
– Analysis was directed towards students who withdrew
from MSU based upon affiliation with a department
or office (or other categorical variable).
– Data captured for “first-time freshman.” Changes (i.e.
residency, change in major, etc) were not accounted
for in the analysis.
Ret. Rate
2003
#
% of WithD Ret. Rate
2004
#
% of WithD. Ret. Rate
2005
#
% of WithD.Ret. Rate
2006
#
% of WithD. Ret. Rate
2007
#
% of WithD.
resd. Status
instate
out
wue?
0.7
0.62
0.81
1485
493
206
2184
0.665919
0.28003
0.058505
0.7
0.65
0.71
1487
613
84
2184
0.64746
0.311393
0.035356
0.69
0.64
0.76
1499
582
155
2236
0.649008
0.292626
0.051955
0.71
0.64
0.77
1427
682
107
2216
0.60768
0.360529
0.036138
0.71
0.63
0.78
1363
659
83
2105
0.598894
0.369439
0.027667
0.71
46
91
2046
0.041943
0.061211
0.886906
0.43
0.61
0.69
54
92
2038
0.044673
0.052075
0.916952
0.39
0.54
0.69
67
70
2099
0.057081
0.044972
0.908785
0.56
0.58
0.7
124
72
2020
0.080117
0.044405
0.889868
0.49
0.48
0.71
180
48
1872
0.139091
0.037818
0.822545
0.8
0.7
0.64
0.75
0.8
0.7
0.59
0.63
0
105
342
157
114
390
288
116
671
0
0.027205
0.125561
0.097937
0.041854
0.109118
0.128251
0.060673
0.371106
0
0.76
0.65
0.74
0.71
0.74
0.71
0.71
0
0.62
103
335
185
124
401
316
96
0
624
0.035878
0.170174
0.069811
0.052192
0.151321
0.133004
0.040406
0
0.344151
0.74
0.63
0.64
0.76
0.76
0.75
0.75
0
0.61
93
384
225
137
353
338
99
0
607
0.033771
0.198436
0.113128
0.045922
0.118324
0.118017
0.034567
0
0.330628
0.68
0.69
0.61
0.64
0.77
0.76
0.64
0
0.66
95
351
168
115
387
316
112
0
672
0.04464
0.15978
0.096211
0.060793
0.130705
0.111366
0.059207
0
0.335507
0.69
0.7
0.68
0.71
0.76
0.67
0.68
0
0.64
91
280
182
112
365
286
99
0
690
0.042742
0.127273
0.088242
0.049212
0.132727
0.143
0.048
0
0.376364
admit
Gen. Studies 0.39
Non Trad.
0.55
Trad.
college
ag
art
bus
ed
eng
ls
nurse
GS
UC
first gen.
yes
0.001451
0.001451
no
0.001397
0.001397
0.001468
0.001468
0.001515
0.001515
on campus
No
0.59
0.72
379
1804
0.232272
0.755037
0.61
0.7
347
1837
0.196415
0.799855
0.59
0.7
340
1869
0.194693
0.783101
0.62
0.71
341
1875
0.190279
0.798458
0.59
0.7
295
1810
0.183258
0.822727
part
0.72
0.43
2011
172
0.841674
0.146547
0.71
0.46
2000
184
0.8418
0.144209
0.71
0.47
1985
251
0.80398
0.185796
0.71
0.54
1942
274
0.82699
0.185081
0.72
0.57
1855
250
0.78697
0.162879
Overall
0.69
Yes
FT_PT
full
0.68
0.68
0.69
0.69
Potential Intervention Groups
•
•
•
•
•
•
•
Out-of-State Students
Provisional Admits
Non Traditional Admits
College of Art, Business, University Studies
First Generation Students
“Off-campus” residential students
Part-time Students
What we still need to do…
• Conduct a “progression analysis” of students as they
move through the system (where did they “land”?).
• Information provides for a good “beta” group to begin
testing retention interventions.
Example:
– freshman living off campus must participate in a program
to help them “bond” with the institution
– Out-of-state residents could be paired with on campus
mentors to host “groups by state”
– Develop a “major stalking” program for University Studies
students.
Bean Model
• From Indiana University
• One of the major “theorists” of student
persistence/retention
– Tinto , Pascarella, Bean
• Bean uses “socialization” model to under-pin
student persistence theory
• Essentially he wants to know if “institutional
selection” promotes student retention or
socialization methods influence retention
methods.
Bean Model
• Variables:
–
–
–
–
–
–
–
–
–
–
–
–
Goals (is it important to receive your degree?)
Utility (how good is your education for earning a job?)
Alienation
Faculty/Staff Contact
Social Life
Finances
Opportunity to Transfer
Outside Friends
Institutional Fit
Institutional Commitment
Drop Out Syndrome
Grades
Survey Methods
Entering Class Fall 2007
• Population Group (N=2107)
• Emailed/called persisters (n= 1446)
– 338 responded (23.3%)
• Called/Emailed withdrawers (n=661)
– 104 responded (15.7%)
• Postcard sent to withdrawers
• Still collecting …
Significant Tests Between Groups
“Persisters” and “Withdrawers”
Differences at the .001 level
1.How difficult was it to leave MSU and transfer to
another university or college as good as this one
(from an academic stand point)?
Persister = 3.12
Withdrawer =3.64
2. How useful do you think your education at MSU
will be for securing future employment?
Persister = 4.44
Withdrawer = 3.28
Significant Tests Between Groups
“Persisters” and “Withdrawers”
Differences at the .01 level
All in all, how good an education do you think
you can get at MSU?
Persister = 4.2
Withdrawer =3.61
Significant Tests Between Groups
“Persisters” and “Withdrawers”
Differences at the .05 level
How difficult was it to leave MSU (i.e. loss of
friends, change of routine, the application
process, etc.) and transfer to another
college/university?
Persister = 3.55
Withdrawer =3.05
Significant Tests Between Groups
“Persisters” and “Withdrawers”
Differences at the .10 level
1.Is your academic program at MSU exciting?
Persisters = 3.73 Withdrawers =3.37
2. About how many times per semester have you met
with a FACULTY member outside the classroom and
spoken to them (for 10 minutes or more)?
Persisters = 2.43 Withdrawers = 2.24
3. To what extent have you discussed leaving MSU with
your friends?
Persisters = 1.84
Withdrawers = 2.61
Next Steps
• Continue to refine analysis of datasets
• Conduct interviews and focus groups on
campus
• Triangulate findings
• Develop invention strategies
• Propose intervention strategies to SPOC.
The Power of “Interventions”
Thoughts/Questions