Predictive Analytics
Reporting (PAR) Framework:
Overview, Applications, Results
Ellen Wagner
Chief Research and Strategy Officer
June 18, 2014
@edwsonoma
Are You “Scorecard-Ready”?
http://collegecost.ed.gov/
Performance Based Funding and
US Post-Secondary Institutions
http://www.ncsl.org/research/education/performance-funding.aspx
A Stronger Nation Through Higher Education:
Lumina Foundation, April 2014
http://strongernation.luminafoundation.org/report/
While “Big Data” raise expectations,
student data drive big decisions in .edu
How Do Institutions Deal With the
“Apples to Zebras” Problem?
National
Non-profit
Multi-institutional
Collaborative
Institutional Effectiveness
+
Student Success
PAR: “standard, equal, normal”
The Predictive Analytics Reporting
(PAR) Framework
• PAR is a “massive data” analysis effort using predictive analytics
to identify drivers related to loss and momentum and to inform
student loss prevention
• PAR member institutions voluntarily contribute de-identified
student records to create a single federated database.
• Descriptive, inferential and predictive analyses have been used
to create benchmarks, institutional predictive models and to
map student success interventions to predictor behaviors
PAR Framework video introduction
https://www.dropbox.com/s/ll6qmo9fru869un/
PAR_1080p_storyeyed.mp4
PAR distributes efforts associated with
analysis and modeling processes
• Analysis and model building is an
iterative process
• Around 70-80% efforts are spent
on data exploration and
understanding.
PAR’s Common Data Definitions Enable
Shared Understandings and Results.
PAR uses structured, readily available data
from all of its members for generalizability
• Common data
definitions = reusable
predictive models and
meaningful
comparisons.
• Openly published via a
cc license @
https://public.datacook
book.com/public/institu
tions/par
PAR Input data are available for ALL
students from ALL US institutions
Student
Demographics
& Descriptive
Gender
Race
Prior Credits
Perm Res Zip Code
HS Information
Transfer GPA
Student Type
Course Catalog
Subject
Course Number
Subject Long
Course Title
Course Description
Credit Range
Student Course
Information
Course Location
Subject
Course Number
Section
Start/End Dates
Initial/Final Grade
Delivery Mode
Instructor Status
Course Credit
Lookup Tables
Credential Types Offered
Course Enrollment Periods
Student Types
Instructor Status
Delivery Modes
Grade Codes
Institution Characteristics
Student
Financial
Information
FAFSA on File – Date
Pell Received/Awarded –
Date
Student
Academic
Progress
Curent Major/CIP
Earned Credential/CIP
Possible Additional **
Placement Tests
NSC Information
SES Information
Satisfaction Surveys
College Readiness Surveys
Intervention Measures
** Future
PAR’s Actionable Benefits/Outcomes
IDENTIFY:
Benchmarks
Show how institutions
compare to their peers in
student outcomes, by
scaling a multiinstitutional database
for benchmarking and
research purposes.
TARGET:
Predictive models
Identify which students
need assistance, by using
in-depth, institutional
specific predictive models.
Models are unique to the
needs and priorities of our
member institutions based
on their specific data.
TREAT:
Intervention measures
Determine best ways to
address weaknesses
identified in benchmarks
and models by scaling
and leveraging a
member, data and
literature validated
framework for examining
interventions within and
across institutions
(SSMx).
Feedback loops for enabling
institutional performance improvements
Performance
Benchmarks
Measurable
Results
Action
Common
Data
Definitions
and Data
Warehouse
Intervention
Benchmarks
Predictive
Models
Scalable cross institutional improvements enabled by
Collaboration via PAR
Descriptive and Predictive Insight
PAR Benchmarks
Descriptive Analytics
Cross Institutional
Student/degree/major level insight into:
1. What did the retention look like for
students entering in the same
cohort
2. How does your institution compare
to peer institutions / institutions in
other sectors
3. How did performance vary by
student attributes
PAR Models
Predictive Analytics
Institutional Specific insight into:
1.
2.
3.
4.
What students are being retained
over time?
Which students are currently at risk
for completing and why?
Which factors are directly correlated
to student success?
What is the predicted course
completion rate for a particular
program?
Collaborative Benchmarking
Student-level data
+
common data definitions
=
deeply drillable comparative reports
Partners determine measures and content
INSTITUTIONAL SPECIFIC
PREDICTIVE MODELS
Institution X
Predicting retention aimed at taking action finding the most important factors
Actionable information at the student
level
1st, 2nd and 3rd most important
factors contributing to risk
PAR
anonymized ID
Risk they will not be
retained
Student Success Matrix (SSMX) Review
• Inventorying & categorizing student success interventions/
supports using a common framework
– Based on known predictors of risk and success
– In the context of the academic life cycle
• Addresses “Now What?” by linking predictions to action
– Enables cross institutional benchmarking
– Supports local and cross institutional cost/ benefit
analyses.
©PAR Framework 2013
SSMX Progress
From this
To this
Launched June 2013
Student Success Matrix (SSMx)
Publically available, 1,400+ downloads
https://public.datacookbook.com/public/institutions/par
Launched April 22, 2014
Members only, managed environment
©PAR Framework 2013
Comprehensive view –
completed SSMx
©PAR Framework 2013
©PAR Framework 2013
Examine interventions by predictor
category
©PAR Framework 2013
©PAR Framework 2013
Isolate interventions
Find gaps
©PAR Framework 2013
Applying Interventions at the Greatest
point of Need/Value
• A fundamental objective for developing common language
and frameworks for reviewing student interventions is so that
the most effective interventions can be applied at the points
of greatest need to effectively remediate risk at the student
level.
• PAR has paved the way for creating common understanding of
student risk and common tools for diagnosing risk, but the
road to developing consistent and applied measurement to
student impact of intervention will take time and vigilance.
From Hindsight to Foresight
PAR Futures
• PAR, Inc., a 501.c.3 non-profit educational organization launching
Dec 9, 2014 as an Analytics-As-A-Service (AAAS) provider.
• PAR will focus on benchmarks, predictive models, the student
success intervention mapping and measurement, “Rosetta Stone”
cross-walks to other data projects and platform providers.
• New reports that emphasize pathways to achieving outcomes (e.g.
Adult learners, PLAs, CBE).
• New reports that consider “big issues” impact on learning
outcomes, e.g., online-blended-onground programs; for-profitpublic-private institutions.
• Support/resources/services for community of research and
practice.
Thank you!
http://parframework.org
@PARFRamework