Multi-Institutional Data
Predicting Transfer Student
Success
Denise Nadasen
Anna Van Wie
Institutional Research
University of Maryland University College
Outcomes for this Session
• You will learn about the:
– Goals for this grant and the research
project
– Process for integrating a multi-institutional
data base
– Research questions, methods, and findings
– Lessons learned and next steps
2
Goals of the grant
•
•
•
•
•
•
•
Collaborate with the community colleges
Define research questions and variables
Build a dataset for transfer students
Explore predictor/outcome variables
Predict student success
Report the results at national conferences
Use the results to inform policy and
practice to better serve transfer students
3
Collaborative Partners
• UMUC is an online institution that enrolls over
90,000 diverse students each year worldwide
• Prince George’s Community College is located
within two miles of UMUC’s Academic Center
and enrolls over 37,000 diverse students.
• Montgomery College is located within 10 miles
of UMUC’s largest regional center, and enrolls
over 35,000 diverse students.
4
The Team
• PI– President, Provost
• Sponsor – Institutional Research
• Partners
– Montgomery College and Prince George’s Community
College
– Undergraduate retention and data mining specialist
– External evaluators
• Researchers:
– Cheoleon Lee, Jing Gao, Futoshi Yumoto, Husein
Abduhl-Hamid
• Data Mining Specialists
– Stephen Penn, The Two Crows
5
The Student Population
• Students enrolled at UMUC between
2005 and 2011
• PG and MC transfer students
– Direct compare (32,000)
– National Student Clearinghouse (12,000)
– UMUC records (8,000)
6
Merging Multi-Institutional Data
• Protect this data!
• Balance institutional-specific protocols with
research-based definitions
• Address data anomalies
• Distinguish student level vs. course level
• Define LMS data
– Limits on data extract
7
KDM
• Integrates student data
– Community College and UMUC SIS
– Demographic
– Courses
– Performance
– Classroom behavior (LMS)
• 300 source and derived variables
• Gather from disparate sources
• One time snapshot
8
MC-PGCC-UMUC Transfers
UMUC students who transferred from MC or PGCC and
were matched in the BASE file
PGCC
students and
class data
MC students
and class
data
Prior
Work
derived data for
transfer students
Base Extract
UMUC undergrad students enrolled
between Spring 2005 and Spring 2011
WT
extract
Classroom activity
Data Warehouse
UMUC students from
PeopleSoft Daily Update
PeopleSoft
WT
Live SIS with UMUC students
Online
Classroom
Question
• What barriers would your institution face in merging
multi-institutional data?
10
Research Goals
•
•
•
•
Define outcome variables
Define predictor variables
Model the student lifecycle
Determine the success and failure
factors
• Develop and implement interventions
• Impact outcomes
11
Outcome Variables
•
•
•
•
•
Successful course completion (percent)
First term GPA (dichotomized)
Reenrollment in next term (Y/N)
Retention (12 month window – Y/N)
Student Classification (Slackers,
Splitters, Strivers, and Stars)
12
Transfer Student Progressions
Transfer
Four-Year Institution
cc
Demog and
Other
Academic
Work
First
Semester
Semester 2
Last
Semester
Graduate
School
cc
Transfer
13
Research Studies
14
Which variables contribute to the
prediction of online course success?
The data:
• 4,558 new, undergraduate, first bachelordegree seeking enrollments in 15 UMUC
online gateway courses in Spring 2011.
• Transfer data on students from partner
institutions, Montgomery College (MC) and
Prince George's Community College (PGCC).
Methodology
• Exploratory factor analysis (EFA) was used to
identify key covariates.
• Logistic regression was used to predict
course success.
Findings
• Total number of transferred credits is the best
predictor of course success
– pseudo R2 value around .12
• GPA from transferred credits is the second best
predictor of course success
– pseudo R2 around .11
• Semester course load contributes less to
course success than other covariates.
Findings
• Four of five predictors derived from
online student behavior show a strong
contribution to successful course
completion.
Final Predictive Model
Total number of transfer credits
Summary of students’ week 0 behavior prior to the first day
Significant
Variables
GPA from transferred credits
Semester course load
Amount of time since students attended the last institution
Read a conference note
Significant Online Entering a class
behaviors
Created a conference note
Created a response note
Which variables predict retention in
an online environment?
• The same data set for the prediction of
course success
• Add in retention status from Summer
2011, Fall 2011, and Spring 2012.
Methodology
• Logistic regression
• Preliminary analysis focused on the
evaluation of covariates (as identified in
the previous analysis) predictors based
on the students’ coursework behavior,
and course success.
Findings
• The covariates and student behavior
variables made less of a contribution to
this model than the prediction of course
success.
• These results indicate that course
success may be a good predictor of
retention.
What is the relationship between prior
academic coursework and UMUC first
semester gateway course on reenrollment?
The population:
• Students new to UMUC in Fall 2008 to Fall
2010
• Took ACCT220, BMGT110, CMIS102,
GVPT170, or PSYC100 in their first semester.
Methodology
• Association algorithm Apriori to determine
relationships between courses in previous
academic work and re-enrollment rates.
• The algorithm indicates when a certain
condition is found another condition can be
expected.
Findings
Significant Course Relationships
Community College Course Disciplines
First UMUC
Course
Math or Business
ACCT 220
Math or Business
CMIS 102
Business or Science
BMGT 110
Science or communication
GVPT 170
Math or communication
PSYC 100
We cannot assume causality
Current Studies
26
Examine CC Courses
• Explore relationship between CC
courses and first term GPA
• Identify courses of interest
• Developmental Ed sequencing
• Successful completion of CC course
• Mixing course level and student level
data
27
Predicting First Term GPA
• What CC variables predict first term
GPA of 2.0 or higher?
• Course Efficiency
• CC courses
– English, Math, Speech, Computer, Honors, Online Course, Remedial
• Demographics
– Age, Gender, Race, Marital Status, Cohort,
Community College Origin, Terms skipped
28
The Population
• 9,063 students from MC and PGCC
• Mostly Single, African-American, and
female
• Most do not skip terms
• Most get A’s and B’s at CC
• Most have >2.0 at UMUC
• Only PGCC offered online courses
29
Predictors of Success
Age
Gender
Race
Marital Status
C.C. Courses
English
Math
Speech
Success
@ UMUC
Computer
Honors
On-line
Remedial
Course Efficiency
Logistic Regression
30
Question
• What CC variables do you think are good predictors?
31
Findings
• Predictive variables:
– Age, marital status, and under-represented
minorities have predictive power
– Math and Honors courses have positive
effects
– Remedial and Online have a negative
effect
– Course efficiency has a positive effect
32
Predicting Student Clusters
• Dataset includes all PGCC and MC
students who transferred
• Student level derived variables
• Cluster students based on retention and
first term GPA at UMUC
• Predict clusters from prior CC work and
demographic variables
33
Retention
No
Retention
Yes
Success Quadrants
Strivers
Stars
Slackers
Splitters
GPA < 2.0
GPA > 2.0
34
Stay Tuned ….
• Data mining continues
– So far, Stars appears to have
distinguishing features
• Focus on top 50 CC courses and
combinations of courses as predictors
• Focus on performance in gateway
courses at UMUC as outcomes
35
Summary of Findings
• Positive effects
– Transfer credit, prior GPA, math, honors,
course efficiency, online activity, age,
marital status
– Course success can predict retention
• Negative effects
– remedial, online, minority status
36
Interventions
• Identify areas of risk
• Collaborate with CC
• Develop intervention strategies
– Advising
– Messaging
– Learning community
– Course development
37
3 Projects Synergizing
PAR
Kresge
Civitas
38
Next Steps
•
•
•
•
Examine course success at the CC
Implement/evaluate interventions
Update KDM with more data
Develop, understand, and explain
predictive models to identify at risk
students at the CC
39
Lessons Learned
•
•
•
•
Long term plan for data up front
Get a project manager
Manage expectations
Communicate progress far and
wide
40
Questions
• Anna anna.vanwie@umuc.edu
• Denise Denise.nadasen@umuc.edu
41