ANALYTICS: A GAME
CHANGER FOR HIGHER
EDUCATION
Great Plains IDEA Spring Meeting
April 7, 2014
3:00-4:30
Dr. Linda Baer, Senior Consultant, i4 Solutions
What happens to learning when we move from the
stable infrastructure of the 20th century to the fluid
infrastructure of the 21st century where technology
is constantly creating and responding to change?
—Thomas & Brown (2011)
A New Culture of Learning
RAPIDLY CHANGING
LANDSCAPE
Actionable Analytics
Rapid conversion of large data sets to actionable
information that is pushed to advisors, faculty, staff,
administrators and learners to drive learner success and
institutional productivity.
• Learner Success = learning outcomes + persistence +
completion
• Drive learner success up while driving costs down
3
Strategic Intelligence for Higher Education
What’s the best that can happen?
What will happen next?
What if these trends continue?
Why is this happening?
What actions are needed?
Where exactly is the problem?
How many, how often, where?
The institutions that will succeed—indeed,
thrive—in this era will be those that
constantly innovate. The beginning steps in
institutional transformation:
•Flipped Courses
•Open Learning Initiative (OLI)
•Math Emporiums
•Blended Courses
•Massive Open Online Courses (MOOCs)
•Competency-based Education*
NEW COURSE MODELS
—Mehaffy (2012)
http://www.educause.edu/ero/article/challenge-and-change
Institution/C
orporate
Partners
Courses
Students
Enrolled
edX
May 2012
MIT-Harvard US$60M
28
60-
1M+
Udacity
ex-Stanford
February 2012
US$15.3M
10
30
750K+
Coursera
Stanford
April 2012
US$16M
100
universities,
196 countries
500
500M
Provider
Launch
/Capital
MOOCs are university inspired…
Provider
Strategies / Tools
Authenticating Learning
edX
MITHarvard
• short videos
• online quizzes, labs,
and textbook
• virtual office hours
• Proctored exams offered by
Pearson
• edX Certificate of Achievement
Udacity
exStanford
• short videos
• interactive quizzes
• inquiry-based
learning
• Proctored exams offered by
Pearson
• California State University
accepts college credit
Coursera • noted faculty
Stanford • lecture capture
• American Council on
Education (ACE) working with
Coursera to provide credit
• Antioch University accepts
college credit
…but not all the same
• Increasing number of public-private joint ventures
• Augment existing skills, resources
• 2tor: online platform to expand graduate programs
•
•
•
•
Technology and infrastructure
Fieldwork sites
Creates instructional material with faculty
Capital investment
• Shares tuition revenue
• “School-as-a-service”
JOINT VENTURES
Premier post-baccalaureate distance education alliance
that sponsors inter-institutional academic programs and
develops policy and practice models for inter-institutional
distance education programs
• Inter-institutional team developed curricula
• Credit
• Academic standards
• “Common price” per credit
• Home institution, teaching institution and central
alliance management
Great Plains Interactive Distance
Education Alliance
• $99/month (+ $39/course)
or $999/year for 10 courses
• Required college courses
• Start any time; no required
meeting times
• Individualized, on-demand
support (online)
• Transfer credits to partner
college(s)
COURSE PROVIDERS
• Tutoring and
mentoring
• Available on
demand, 24x7
• Matches mentors
and mentees;
flexible scheduling
• Shared,
synchronous
experiences
PRIVATE LABEL
RECLAIM TIME
Direct2Degree: Kentucky Technical and
Community College System
• Learning happens everywhere
• peer, volunteering, work
• Badging:
• Recognition for skills and
achievements
• Display across PSP* learning/networking environments
• Credentials
• move students quickly into the workforce with marketable
skills and increased earning capability
• Programs with multiple “completion points” achievable within
short time-frames
BADGES AND STACKABLE CREDENTIALS
* personal, social, professional
• Based on Lumina’s degree qualifications profile
• 120 defined competencies
• Organized as mastery triads
o
o
o
Foundational
Personal and social skills
Content knowledge
• Demonstrate mastery by completing tasks
COMPETENCY-BASED
College for America (SNHU)
Self-paced, online associate’s degree program, $2500/year
Coach
Online
Networks
Accountability
Partner
Mentor
Individual Mastery Plan:
Self-directed student
progression through key
competencies using
curated e-resources
SUPPORT MODEL
College for America (SNHU)
• Emerging now are a set of rapidly developing
technology tools to support students as they
progress through a course and program.
• Depending on the individual student’s
understanding of specific ideas, concepts, and
operations, these tools have the capacity to
• personalize the learning experience for every
student
• chart individual pathways through course
materials
STUDENT SUCCESS SOLUTIONS
Education
Planning
Counseling and
Coaching
Risk Targeting and
Intervention
Transfer and
Articulation
Legacy
ERP/SIS/LMS
Vendor point
solutions
Homegrown
point
solutions
Sinclair’s
MAP
Valencia’s
LifeMap
Austin Peay’s
Degree Compass
Central Piedmont’s
Online Student Profile
WICHE’s Predictive
Analytics Reporting
Direct-tostudent
© 2010 Bill & Melinda Gates Foundation
• Cross-institutional online advising
and
degree attainment support system
• Real-time exploration of
• Course choices and effect of choices on degree program
• Courses from other campuses that meet degree requirements
• Lets advisors know which students are off-track
• Decreased time to graduation; Increased graduation rate
• Reduced cost to student, state and support programs
ACADEMIC JOURNEY SYSTEM
University of Hawaii 10 campuses
• Students know where
they stand
• 8% improvement in
student retention rate
(since 2007)
• Percent of students
on the correct path to
major:
•
22% (2000)
•
95% (2010)
CLEARER PATH TO GRADUATION
Arizona State University
• Map-in starting point
and
destination
• Routes to completion
• Time to destination –
progress
• Fuel for the journey
• Travel time to “norm”
for the destination
• Highway for optimizing
student success
EDUCATION AND CAREER
POSITIONING SYSTEM
Lone Star Community College
http://studentalignment.com/home.html
Analytics: Graduate
and Online Models
METRICS FOR BEST ONLINE GRADUATE
EDUCATION PROGRAM RANKINGS
• Student Engagement
• Interact with instructors and classmates
• Instructors are accessible and responsive
• Instructors create an experience rewarding enough to stay
enrolled and complete in a reasonable amount of time
• Admissions Selection
• Entering students have proven aptitudes, ambitions and
accomplishments to handle the demands of rigorous course
work
• Peer Reputation
• Industry opinion accounts for intangible factors on program
quality
• Degrees with strong perceptions of quality among academics
may be held in higher regard among employers
http://www.usnews.com/education/online-education/articles/2014/01/07/about-thetop-online-education-programs-rankings-2014
METRICS FOR BEST ONLINE GRADUATE
EDUCATION PROGRAM RANKINGS
• Faculty credentials and training
• Strong online programs employ instructors with academic
credentials one would expect rom a campus-based program
• Invest resources to train these instructors on how to teach
distance learners
• Student services and technology
• Incorporates diverse online learning technologies which allows
greater flexibility for students to take classes from a distance
• Outside of classes, a strong support structure provides learning
assistance, career guidance and financial aid resources
commensurate with quality campus-based programs
http://www.usnews.com/education/online-education/articles/2014/01/07/about-the-top-online-educationprograms-rankings-2014
•
•
•
•
•
•
Student selection and admissions
Mentoring and advising
Financial support
Program environment
Research experience
Curricular and administrative processes and
procedures
Graduate Student Metrics
Policies and Practices to Promote
Student Success
Sowell, Bell and Kirby Ph.D. Completion and Attrition: Policies and
Practices to Promote Student Success. Council of Graduate Schools. 2010
Graduate Program Evaluation Metrics
• Time-to-degree (master’s and doctoral)
• Completion rate (master’s and doctoral)
• 2 year master’s
• 6-8 year for doctoral
• Advancement to candidacy rate
• Number of degrees awarded per year
• Percent doctoral students receiving full support
• Competitiveness of stipends with respect to AAU institutions
• Student placement in context of program goals
• Master’s 2 years after completion
• Doctoral 5 years after completion
• Benchmark performance against national criteria by discipline
Association of American Universities http://www.aau.edu
• Expert electronic
coaching
• Uses academic
information, goals,
psycho-social factors,
real-time data
• Individually personalized
messages
PREDICTION AND INTERVENTION
ECoach at University of Michigan
McGraw Hill Learn Smart
Starfish Solutions
PREDICTION AND INTERVENTION
Knewton
Course Signals
Purdue
PREDICTION AND INTERVENTION
Predictive Analytics Reporting (PAR) Framework
• Funded by Bill & Melinda Gates Foundation in 2011, 2012
• Managed by WICHE Cooperative for Educational
Technologies, operated by WCET core project team
• 20+ institutional partners
▫ 4-year schools
▫ Community colleges
▫ For-profit institutions
• 12.5M+ course level records
• 1.7M+ student level records
• In-kind donations to date
▫ Blackboard
▫ iData
▫ Starfish
http://wcet.wiche.edu/advance/par-framework
predictors
learner characteristics
learner behaviors
academic integration
social-psychological integration
other learner support
course/program characteristics
instructors behaviors
PREDICTIVE ANALYTIC REPORTING
FRAMEWORK (PAR)
https://par.datacookbook.com/public/institutions/par
Student
Demographics
& Descriptive
Gender
Race
Prior Credits
Perm Res Zip Code
HS Information
Transfer GPA
Student Type
Course Catalog
Subject
Course Number
Subject Long
Course Title
Course Description
Credit Range
Student
Course
Information
Course Location
Subject
Course Number
Section
Start/End Dates
Initial/Final Grade
Delivery Mode
Instructor Status
Course Credit
Student
Financial
Information
FAFSA on File – Date
Pell Received/Awarded –
Date
Lookup Tables
Credential Types Offered
Course Enrollment Periods
Student Types
Instructor Status
Delivery Modes
Grade Codes
Institution Characteristics
Student
Academic
Progress
Current Major/CIP
Earned Credential/CIP
Possible Additional **
Placement Tests
NSC Information
SES Information
Satisfaction Surveys
College Readiness Surveys
Intervention Measures
DATA INPUTS
** Future
PAR Student Success Matrix (SSMx)
Literature-based tool for benchmarking student
services and interventions
https://par.datacookbook.com/public/institutions/par
http://www.grad.illinois.edu/sites/default/files/researchschemagraphic.png
From Data to the Student
Who needs
additional
support?
Identify them
with a suite
of predictive
models
Why?
Cluster
analysis to
segment
students
combined
with
qualitative
data
What should
we do and
when?
How do we
enable
support staff
to help?
Combine what
we know
about the why
with what has
historically
been effective
and design
comprehensiv
e intervention
strategies
Create
dashboards and
timely
communications,
provide
recommended
resources, and
engage all roles
in the
development
How do we
continuously
get better at
this?
Create a
mechanism for
on-going data
collection and
a feedback
loop to inform
all parts of the
process
The rarely articulated implication of all of this data floating
around is that un-augmented human cognition is no longer
sufficient. Every day there is more to know, more ways to
know it, and heightened expectations—by students, faculty
members, alumni, football coaches, trustees, regulators,
elected officials— that senior managers will do something
efficacious with what they know. ... The best tool in [their]
battle against ignorance is advanced analytics.
—Thornton May, 2011
The New Know: Innovation Powered by Analytics
IMPLICATIONS:
WRIT LARGE
• Driving all of this change will be the capacity
of analytics to improve student success
• Chart the progress of individual students
through a course
• Identify specific learning outcomes by
individual student
• Track with enormous precision a student’s
journey through the curriculum.
ANALYTICS
Fall 2007 Cohort
Number of
Signals
Courses
Retention Rate
Cohort
Size
1 year
2 year
3 year
4 year
Number of
Average
Signals
SAT Score
Courses
No Signals
5,134
83.44%
73.14%
70.47%
69.40% No Signals
At least 1
1,518
96.71%
94.73%
90.65%
87.42% At least 1
1 instance
1,311
96.57%
94.13%
89.70%
86.50% 1 instance
2 or more
207
97.58%
98.55%
96.62%
93.24% 2 or more
SIGNALS AFTER 4 YEARS
1155
1129
1133
1102
Mentor faculty
Lead faculty
REDEFINING ROLES
Discipline mentors
Evaluators
Four questions…
1. Who is collecting what data, for whom, and how?
2. How is research about students being used to
develop activities and programs that promote
student success?
3. Are student success activities mandated or
optional? How many students participate?
4. How are leaders investing in student success on
their campuses? Policies, practices, resources,
people, long term commitment, ongoing research
on the issues that matter.
Institutional Culture
 Student-centered
 Visionary leadership at many levels
 Evidence-driven
 Culture of experimentation and inquiry
 Results-oriented
 “Culture eats strategy for lunch!”
In the 21st century, this ancient course
model stands in stark contrast to the
large-scale courses, the collaborative
courses, and the programmed courses
that have now begun to appear.
—George Mehaffy, Challenge and Change,
EDUCAUSE Review, September/October 2012
Using Analytics to Unlock Your
Campus Potential
DISCUSSION
References
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
AACU High Impact Practices http://www.aacu.org/leap/index.cfmC&U
Association of Community College Trustees. 2013. Student Success Toolkit. http://governance-institute.org/toolkit
Baer, Linda and John Campbell. 2012. From Metrics to Analytics, Reporting to Action: Analytics’ Role in Changing the Learning Environment. In Game
Changers, edited by Diana Oblinger. http://www.educause.edu/library/resources/chapter-4-metrics-analytics-reporting-action-analytics%E2%80%99-rolechanging-learning-environment
Bean, John P. and Barbara Metzner 1985 A Conceptual Model of Nontraditional Undergraduate Student Attrition in Educational Research Winter, 1985,
Vol.55, No 4, 485-540.
Compete College America www.completecollegeamerica.org
Crow, Michael. No More Excuses in EDUCAUSE Review, vol. 47, no. 4 July/August 2012 http://net.educause.edu/ir/library/pdf/ERM1241P.pdf
Davenport, Thomas. http://www.slideshare.net/sasindia/keynote-thomas-davenportanalyticsatwork
Retrieved November 23, 2013
Gilbert, C., M. Eyring, and R. N. Foster. “Two Routes to Resilience.” Harvard Business Review, December, 2012, 65–73. http://hbr.org/2012/12/tworoutes-to-resilience/ar/1.
Graduate School Metrics. ssociation of American Universities http://www.aau.edu
Jones. Dennis. 2013. Outcomes-Based Funding: The Wave of Implementation in September 2013. National Center for Higher Education Management
Systems
Kamenetz Anya. 2012 Fast Company 2012 Most Innovative Companies 2012 Southern New Hampshire University
http://www.fastcompany.com/3017340/most-innovative-companies-2012/12southern-new-hampshire-university
University of Illinois Graduate Model. http://www.grad.illinois.edu/sites/default/files/researchschemagraphic.png
Kuh. George and Jillian Kinzie, John H. Schuh, Elizabeth J. Whitt and Associates. 2010. Student Success in College: Creating Conditions that Matter
National Commission on Higher Education Attainment. 2013. An Open Letter to College and University Leaders: College Completion Must Be Our Priority.
American Council on Education. Washington D.C.
Norris, Donald and Robert Brodnick, Paul LeFrere, Joseph Gilmour, Linda Baer, Ann Hill Duin and Stephen Norris. 2013. Transforming in an Age of
Disruptive Change. Strategic Initiatives, Inc. and the Society for College and University Planning
Predictive Analytics Reporting Framework. info@parframework.org
Sowell, Bell and Kirby Ph.D. Completion and Attrition: Policies and Practices to
Promote Student Success. Council of Graduate Schools. 2010
Student Success Matix (SSMX ) A model classifying influences on student success within the PAR Project WCET Annual Meeting Presentation
Mindy Sloan,
Ashford University, Karen Swan, University of Illinois Springfield, Michelle Keim, Bridgepoint Education, Heidi Hiemstra, Predictive Analytics Reporting
(PAR) Framework. November 15, 2013
A
Shugart, S. M. 2012. The Challenge to Deep Change: A Brief Cultural History of Higher Education. Planning for Higher Education,
December 28. Retrieved January 15, 2013, from the World Wide Web: http://mojo.scup.org/forum/topics/the-challenge-to-deep-change-abrief-cultural-history-of-higher.
Tinto, Vincent. 2012a. Leaving College: Rethinking the Causes and Cures of Student Attrition. University of Chicago Press.
Tinto, Vincent. 2012b. Completing College: Rethinking Institutional Action. University of Chicago Press