BIG DATA AND
ACCREDITATION
CHEA 2014 Summer Workshop
Positioning Accreditation: Role, Responsibilities and Expectations
June 25, 2014
Dr. Linda Baer, Senior Consultant, i4 Solutions
Implementing analytics and applying it to
make data driven decisions is a major
differentiator between high performing and
low performing organizations.
Big Data: The Next Frontier for Innovation, Competition and Productivity
McKinsey Global Institute 2012
Analytics: The Game
Changer for Higher
Education
“No More Excuses”
Goals of Arizona State University
2002
• Increasing graduate numbers
• Graduation rates
• Freshman retention rates
• Expand ethnic and economic
diversity
Outcomes in 2011
• Increased enrollment 30% in 10
years
• Increased minority enrollment as %
of total population by 52%
• Increased degrees awarded by 52%
• Increased 6-year grad rate by 19%
• Increased freshman persistence to
84% up 9%
Solutions
• Comprehensive use of analytics
http://net.educause.edu/ir/library/pdf/ERM1241P.pdf
 If you use these analytical tools, you will know
•
•
•
•
where you are,
what you’re doing,
if what you are doing is working or not
whether or not you need to be doing new things
customized to fit your particular school or demographic
• infinitely more information to help students be successful
“No More Excuses”
“…to address societal imperatives, higher education must
begin by transforming its own culture, which is reflected in
the questions we ask (and those we don’t), the
achievements we measure and highlight (and those we
ignore), and the initiatives we support (or don’t support).”
-- Dr. Freeman Hrabowski
President of University of Maryland, Baltimore County
Hrabowski, et al., (2011, p16)
Why Study Data Analytics?
• Expectations for accreditation and external
accountability are increasing, so that it is no longer
sufficient for institutions to have assessment plans.
Instead, institutions strive to build a culture of evidence
with examples of how assessment results are used to
improve student learning.
• Very seldom do institutions now complete their
reaccreditation without including language about work
that needs to be done regarding the collection of
student learning outcomes assessment evidence and
using that evidence to improve.
Baker, Gianina R., Natasha A. Jankowski, Staci Provezis, and Jillian Kinzie. Using
Assessment Results: Promising Practices of Institutions That Do It Well. NILOA July
2012
•
•
•
•
What is big data?
Why is it important?
How is big data tied to accreditation?
What does big data have to do with
student success?
QUESTIONS
DATA ARE CHANGING EVERYTHING
• Big Data has captured public attention as a
critical tool to advance data analytics,
visualization and customized services to
consumers. This trend is affecting higher
education with more data available on student
readiness, progression and success. While
much is known about what works to improve
student persistence, completion, and
interventions that increase the likelihood of
success, it takes local will to make sense of
and act on this information.
What is Big Data?
• Big data is a blanket term for any collection of data
sets so large and complex that it becomes difficult to
process using on-hand database management tools or
traditional data processing applications.
• The challenges include capture, curation, storage,
search, sharing, transfer, analysis and visualization.
• More data may lead to more accurate analyses.
• More accurate analyses may lead to more confident
decision making
http://en.wikipedia.org/wiki/Big_data
• Most institutions are collecting evidence of student
learning, but it is not clear how those results are being
used to improve student outcomes
• “Mountains of data, very little action.”
• Institutions are at many levels of capacity in the use of
data and analytics to improve decision making
• Institutions need to link data with action and targeted
outcomes
What Accreditors Need to
Know about Big Data
While Big Data raise expectations,
student data drive big decisions in .edu
Why is it important?
Connecting the Dots
Assessment
Accreditation
Student
Success
Analytics
Accountability
•
•
•
•
Assessment
Accountability
Accreditation
Analytics
•
•
•
•
Concrete Measures
Concrete Targets
Concrete Quality Assurances
Concrete Practices
Connecting the Dots
17
• Advanced academic quality
• Demonstrate accountability through consistent, reliable
information about academic quality and student
achievement that fosters continuing public confidence and
investment
• Encourage where appropriate self scrutiny and planning for
change and needed improvement
• Employ appropriate and fair procedures in decision making
• Demonstrate ongoing review of accreditation practices
• Possess sufficient resources that are predictable and
stable
Accreditation
Accreditation
http://www.chea.org/pdf/chea-at-a-glance_2012.pdf
• Institutional performance
• Program performance
• Student performance
Assessment
Assessment
• Proven metrics
• Progress over time
• Culture of inquiry
• Internal and external drivers
Accountability
Accountability
Definition: Analytics is the use of data,
statistical analysis and explanatory
and predictive models to gain insights
and act on complex issues.
Analytics
Analytics
http://www.educause.edu/library/resources/2012-ecar-study-analytics-highereducation
ANALYTICS
STRATEGIC
QUESTION
DATA
ANALYSIS
AND
PREDICTION
INSIGHT AND
ACTION
http://www.educause.edu/library/resources/2012-ecar-study-analyticshigher-education
Strategic Intelligence for Higher Education
What’s the best that can happen?
What will happen next?
What if these trends continue?
Why is this happening?
What actions are needed?
Where exactly is the problem?
How many, how often, where?
Past
Information
Insight
Present
What happened? What’s happening
Now?
(Alerts)
(Reporting)
How and why
did it happen?
(Modeling,
Experimental
design)
What’s the next
best action?
(Recommendation)
Future
What will happen?
(Extrapolation)
What’s the
best/worst that can
happen?
(Prediction,
Optimization,
Simulation)
Key Questions Addressed by
Analytics
Analytics at Work. Davenport, Harris and Morison.2010
Obligation of Knowing
We now have models for improving student
success. What is our obligation to do
something about it?
– Maximize the availability and use of data
– Unlock the silos
– Involve faculty in meaningful ways
– Change approach to the facilitation of
learning
– Empower students to know how they are
doing and what they can do to improve
Critical is the issue of “closing the loop” of using
assessment evidence to improve student
learning and inform curricular decisions. After
scouring the literature and discussing the
question, Banta and Blaech 2011 found that only
six percent of cases actually could link to student
learning improvement.
Linking Analytics to
Student Success
http://www.teaglefoundation.org/teagle/media/library/documents/resources/NILOA.pdf?ext=.pdf
Implications for
Accreditation
Higher Learning Commission
4.C. The institution demonstrates a commitment to educational
improvement through ongoing attention to retention,
persistence, and completion rates in its degree and certificate
programs.
1. The institution has defined goals for student retention,
persistence, and completion that are ambitious but attainable
and appropriate to its mission, student populations, and
educational offerings.
2. The institution collects and analyzes information on
student retention, persistence, and completion of its programs.
3. The institution uses information on student retention,
persistence, and completion of programs to make
improvements as warranted by the data.
http://www.higherlearningcommission.org/Information-for-Institutions/criteria-and-corecomponents.html?highlight=WyJwZXJzaXN0ZW5jZSIsImNvbXBsZXRpb24iXQ==
CRITERIA
• The institution has defined
goals for student retention,
persistence, and
completion.
• The institution collects and
analyzes information on
student retention,
persistence, and completion
of its programs.
• The institution uses
information on student
retention, persistence, and
completion of programs to
make improvements as
warranted by the data.
EVIDENCE
 Measures, metrics, targets
 Data warehouse, mining,
analysis, assessment,
accessibility, action
 Improvement activities,
interventions, resource
allocation
Example of a Retention, Persistence and Completion
•
•
•
•
•
Beyond reporting to action
Competencies and skills
Progress in programs
How to improve student achievement
How to continue to link to national professional
standards and expectations
Professional Accreditation
and Analytics
WHAT ARE INSTITUTIONS
DOING WITH BIG DATA?
•
•
•
•
•
•
Early predictions of “at-risk” students
Likelihood of success in courses, programs of study
Course sequencing
Likelihood of attending class next week
“Students like you…” scenarios
Relevant, readily available information about where students
are in a class for students, advisors and faculty
• Targeted interventions to match student needs
• Early alerts and kudos
• Improved alignment of student’s interests, skills, and
career/training opportunities
Areas Where Analytics Can Improve
Student Success
predictors
learner characteristics
learner behaviors
academic integration
social-psychological integration
other learner support
course/program characteristics
instructors behaviors
PREDICTIVE ANALYTIC REPORTING
FRAMEWORK (PAR)
https://par.datacookbook.com/public/institutions/par
Student
Demographics
& Descriptive
Gender
Race
Prior Credits
Perm Res Zip Code
HS Information
Transfer GPA
Student Type
Course Catalog
Subject
Course Number
Subject Long
Course Title
Course Description
Credit Range
Student
Course
Information
Course Location
Subject
Course Number
Section
Start/End Dates
Initial/Final Grade
Delivery Mode
Instructor Status
Course Credit
Student
Financial
Information
FAFSA on File – Date
Pell Received/Awarded –
Date
Lookup Tables
Credential Types Offered
Course Enrollment Periods
Student Types
Instructor Status
Delivery Modes
Grade Codes
Institution Characteristics
Student
Academic
Progress
Current Major/CIP
Earned Credential/CIP
Possible Additional **
Placement Tests
NSC Information
SES Information
Satisfaction Surveys
College Readiness Surveys
Intervention Measures
DATA INPUTS
** Future
Action Behind Analytics
Know What Works
• Bridging Programs
• Learning Communities
• Active Learning
• First Year Experiences
• Course redesign
• Capstone courses
• Supplemental Instruction
• Intrusive Advising
35
Customize Interventions
• Longitudinal data system
to track course patterns
• Multiple intervention
strategies – academic and
student support
• Build from learning
management systems
• Act soon enough to make
a difference
Analytics Maturity Index
http://www.educause.edu/ecar/research-publications/ecar-analytics-maturity-index-highereducation
• How does big data affect the assurance of
quality?
• What do accreditors need to know?
• What are institutions doing with big data that is
important to accreditors as they assess
academic quality?
• As an accreditation reviewer, what evidence do
you need from the institution?
• Should there be a template for guiding
institutions and peer reviewers?
QUESTIONS FOR ACCREDITORS
QUESTIONS?
lindalbaer0508@gmail.com
References
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
AACU High Impact Practices http://www.aacu.org/leap/index.cfmC&U
Association of Community College Trustees. 2013. Student Success Toolkit. http://governance-institute.org/toolkit
Baer, Linda and John Campbell. 2012. From Metrics to Analytics, Reporting to Action: Analytics’ Role in Changing the Learning
Environment. In Game Changers, edited by Diana Oblinger. http://www.educause.edu/library/resources/chapter-4-metricsanalytics-reporting-action-analytics%E2%80%99-role-changing-learning-environment
Bean, John P. and Barbara Metzner 1985 A Conceptual Model of Nontraditional Undergraduate Student Attrition in Educational
Research Winter, 1985, Vol.55, No 4, 485-540.
Compete College America www.completecollegeamerica.org
Crow, Michael. No More Excuses in EDUCAUSE Review, vol. 47, no. 4 July/August 2012
http://net.educause.edu/ir/library/pdf/ERM1241P.pdf
Davenport, Thomas. http://www.slideshare.net/sasindia/keynote-thomas-davenportanalyticsatwork
Retrieved November
23, 2013
Gilbert, C., M. Eyring, and R. N. Foster. “Two Routes to Resilience.” Harvard Business Review, December, 2012, 65–73.
http://hbr.org/2012/12/two-routes-to-resilience/ar/1.
Graduate School Metrics. ssociation of American Universities http://www.aau.edu
Jones. Dennis. 2013. Outcomes-Based Funding: The Wave of Implementation in September 2013. National Center for Higher
Education Management Systems
Kamenetz Anya. 2012 Fast Company 2012 Most Innovative Companies 2012 Southern New Hampshire University
http://www.fastcompany.com/3017340/most-innovative-companies-2012/12southern-new-hampshire-university
University of Illinois Graduate Model. http://www.grad.illinois.edu/sites/default/files/researchschemagraphic.png
Kuh. George and Jillian Kinzie, John H. Schuh, Elizabeth J. Whitt and Associates. 2010. Student Success in College: Creating
Conditions that Matter
National Commission on Higher Education Attainment. 2013. An Open Letter to College and University Leaders: College
Completion Must Be Our Priority. American Council on Education. Washington D.C.
Norris, Donald and Robert Brodnick, Paul LeFrere, Joseph Gilmour, Linda Baer, Ann Hill Duin and Stephen Norris. 2013.
Transforming in an Age of Disruptive Change. Strategic Initiatives, Inc. and the Society for College and University Planning
Predictive Analytics Reporting Framework. info@parframework.org
Sowell, Bell and Kirby Ph.D. Completion and Attrition: Policies and Practices to
Promote Student Success. Council of Graduate Schools. 2010
Student Success Matrix (SSMX ) A model classifying influences on student success within the PAR Project WCET Annual Meeting
Presentation
Mindy Sloan, Ashford University, Karen Swan, University of Illinois Springfield, Michelle Keim, Bridgepoint
Education, Heidi Hiemstra, Predictive Analytics Reporting (PAR) Framework. November 15, 2013
A
Shugart, S. M. 2012. The Challenge to Deep Change: A Brief Cultural History of Higher Education. Planning
for Higher Education, December 28. Retrieved January 15, 2013, from the World Wide Web:
http://mojo.scup.org/forum/topics/the-challenge-to-deep-change-a-brief-cultural-history-of-higher.
Tinto, Vincent. 2012a. Leaving College: Rethinking the Causes and Cures of Student Attrition. University of
Chicago Press.
Tinto, Vincent. 2012b. Completing College: Rethinking Institutional Action. University of Chicago Press