2015 SHEEO Higher Education Policy Conference:
Changing Higher Education to Serve the New American Student
Transforming State Systems
Audrey Hovannesian, EdD
Associate Director of Assessment
CSU Office of the Chancellor
• Established in 1887
• 23 Campuses
• Over 440,000
students
• Over 40,000 staff
and faculty
• Largest campus
enrollment 37,000
whom we teach
enrollment rates of 18- to 24-year-olds
in degree-granting institutions
1970
top 26%
Source: National Center for Education Statistics
enrollment by Hispanics, African-Americans, and Asians:
15%
1976
19%
1990
2009
top 41%
34%
2009
enrollment by students eligible for financial aid:
30%
1976
46%
1990
66%
2009
Graduation Initiative
K-12
opportunity
quality
success
baseline
51%
2025 target
60%
16%
24%
Gap by ethnicity
14 points
7 points
Gap by Pell eligibility
11 points
5 points
Transfer four-year rate
70%
76%
Transfer two-year rate
27%
35%
Six-year graduation rate
Four-year graduation rate
Graduation Rates by Ethnicity and participation in High-Impact Practices
73%
65%
Source: CSU Northridge
Institutional Research
August, 2010
68%
69%
2
3+
63%
55%
49%
38%
0
1
2
Latino/a
3+
0
1
not Latino/a
Participation in HIPs CSU Systemwide
Source: AAC&U and
NSSE Special Analysis
May, 2009
53
Native
51
44
Transfer
39
34
25
17
12
Research
with Faculty
10
7
Study
Abroad
Service
Learning
Internship
Capstone
First-Year Seminars and Experiences
Intrusive Advising
Learning Communities
Undergraduate Research
Service Learning, Community-Based Learning
Internships
Capstone Courses and Projects
Common Intellectual Experiences
Diversity/Global Learning/Study Abroad
Collaborative Assignments and Projects
Writing-Intensive Courses
Early Alert
Mentor Services
Summer Bridge
Summer Transition (non-EOP summer bridge experiences)
Supplemental Instruction
Coordinated Student On-Campus Employment
Pedagogy Sphere
Integrated HIP
• Rubric/scoring
system
• Available to the
public
• Comments/feedback
please
Minimum
Definition
Measures of
Campus Climate
record
creation
record management
assessment
HIP Concerns
High Impact Practice Delivery
Selection Bias
Equity Issues
Second Year
Third Year
Fourth Year…
Sixth Year
Lack Valid
Metrics
First Year
Too Quick to
Label HIPs
Programs too
small to scale
CMS
capture
store
display
Academic and Student Success Programs (“ASSP”)
Gates-funded “Preparing to Scale High-Impact Practices”
Student Success Data Dashboard
“Data Readiness” and “Action Research” grants
Support for campus-based “Dashboard Leaders”
Preparing to Scale High Impact Practices (HIPs) Project
Timeframe: Short-term 8 month project
Goal: Scale HIPs and track HIP participation and levels of engagement in baseline
student data
JAN
Project Campuses
Pilot Partners
CSU/Gates
Foundation Preparing
to Scale High Impact
Practices Project
Model Campus
Inform initial
tracking
JAN
process
MAR
HIP Implementation Model
CSU Fullerton Strategic Plan
Goal #2:
“Improve student persistence, increase graduation rates
university-wide and narrow the achievement gap for
underrepresented students”
Objective:
“Increase participation in HIPs and ensure that 75% of
CSUF students participate in at least two HIPs by
graduation”
CSU Fullerton Model
Create a HIPs
TaskForce
Develop a
System to Scale
High Impact
Practices
Inventory
Campus HIPs
Identify the HIPs
which have large
scale benefits for
student success.
Create systems
to track HIPs in
student records.
Identify HIP variables
to assess levels of
student participation
and engagement.
Assessment of HIPs (Positive Findings)
Undergraduate
Research
Freshman
Programs
Residence
Halls
Supplemental
Instruction
Study Abroad
Programs
Timeline
•
•
•
•
•
•
•
•
•
•
•
August 19, 2014: Introduction Webinar
September 19, 2014: Application deadline
October 1, 2014: Pilot partners selection announced
October 19-21, 2014: IUPUI Assessment Conference
December 1, 2014: Task #1 deadline
December 12, 2014: Pilot Partner Webinar
February 24, 2015: Task #2 deadline
March 6, 2015: Pilot Partner Webinar
March 26th and 27th 2015: AAC&U meeting in San Diego.
April 21, 2015: Final TaskForce Report Due
May 2015: Culminating event discuss project deliverables and
next steps.
STEP 1: Create a HIPs TaskForce
STEP 1: Create a HIP TaskForce
HIPs TaskForce Creation
Process
Step 1:

Begin the taskforce building process by following the normal
channels for faculty nomination to committees.
Step 2:
 Recruit administrators and technical staff from areas already known
to be associated with HIPs and/or data collection and analysis.
Step 3:
 Remain open to the possibility of adding additional specialists from
campus to address specific needs in the process of identifying and
tracking the HIPs.
HIPs TaskForce Creation
Takeaways
• Slow process getting faculty involved
• Control the size of group
• Utilize those already working on HIPs
California State University
High Impact Practices
Leadership Retreat
HIPs Q-Sort
Q-Sort Results: Group A & B
Statement
1. Close and then eliminate gaps in retention and graduation rates, as well as
student achievement on campus among students from underrepresented groups.
2. Help students develop skills that will allow them to adapt and innovate in
shifting workplace contexts.
3. Raising overall student GPA.
4. Encourage faculty and staff to reorient their thinking so as to make the
institution ready for students (rather than emphasizing the need to make
students ready for college).
5. Increase graduation (both 4 and 6 year) and retention rates.
6. Increase students' self-efficacy and persistence.
7. Provide more (and more effective) interventions and supports for students
experiencing academic obstacles or difficulty.
8. Improving recruitment and enrollment services by focusing on greater and
more equitable student access.
9. Impact economic development in the community.
10. Help align university faculty and resources with the university’s vision and
values.
11. Increase and enhance student-to-student mentoring and learning
opportunities.
12. Increase meaningful interactions among faculty and students.
13. Increase the frequency and quality of multicultural experiences for students.
14. Better prepare students for transitions toward meaningful careers or other
post-graduate plans.
15. Increased student understanding of how community involvement will help
them to grow as individuals.
16. Encourage faculty to reflect on their teaching and how it might become more
innovative, effective, and engaging.
17. Increase university staff and faculty recognition that we are best when we
take a "whole person" approach.
18. To provide us with a framework and a language for talking about and
organizing activities, programs, and processes that promote student success.
19. Deepen academic content understanding for students.
20. Better support our students' transition to college.
21. Increase sense of belonging to and engagement with our college.
22. Deepen student involvement in the community, by promoting a spirit of giving
back.
23. Increase student motivation and commitment to be life-long learners.
24. Increase opportunities for undergraduate research and research collaboration
with faculty.
25. Foster students' sense of justice and social responsibility by helping them to
understand their place in their communities and their world and how they may
contribute to and affect change in both.
26. Help students develop as leaders in their families, workplaces, and
communities.
27. Increase student opportunities for authentic learning experiences that
connect them more deeply to their academic area and career field.
28. Improve students' satisfaction with the institution - both their academic and
non-academic experiences.
Factor
A
1
Factor
B
4
3
0
-4
1
-1
2
-1
4
1
3
3
2
-3
-2
-3
-2
-4
-1
0
0
2
-1
3
1
-1
1
0
-3
2
1
-2
-3
-1
0
0
-1
0
-1
-2
2
3
-3
3
1
-2
-1
2
-2
1
-1
4
1
-2
0
Task 1: Create a HIP TaskForce
Associate Dean of Undergraduate Education
Associate Vice President for Academic Affairs Operations
Area Coordinator, Residential Life
Director of Institutional Research
Interim Director, Institute for Civic and Community Engagement (ICCE)
Faculty Director of General Education
Assistant Dean of Students
Dean of Undergraduate Education and Academic Planning
AVP for Academic Planning
CIO
Director of Institutional Assessment and Research
Associate Director for Service Learning
AVP for Inclusive Excellence
Associate Provost for Research and Dean of Graduate Studies, Chair
Vice President, Division of Student Affairs
Vice President, Division of Information Technology Services
Director, Project Management and Assessment, Information Technology Services
Director, Office of Student Engagement
Dean of Students
Director of Orientation/First Year Experience, Student Affairs
AVP and Dean of Undergraduate Studies
Director of Retention Projects, Undergraduate Studies
Associate Dean of Graduate Studies and Director of the Office of Student
Research
Director of Institutional Research
Chair of the Faculty Senate
faculty in Art, Senate member
Academic Programs, project lead
Institutional Research
Student Affairs
Library
RISS
IT
Faculty in Business
Dean, Undergraduate Studies
Faculty Center Director (while Elisa is on sabbatical)
Director, Director, First Year Programs
Faculty Director, Service Learning Faculty Director
Faculty Fellow for Undergrad Research
Director, IITS
Assistant Director, Institutional Planning & Analysis
Undergraduate Studies Student Support Professional
Faculty Fellow for 21st Century Learning will join us as available
Registrar
Associate Provost for Academic Affairs
Director, Office of Undergraduate Research
Director, Enrollment & One-Stop Services
Director Center for Community Engagement
Coordinator, Student Life & Cultural Centers
Director of Institutional Research and Academic Resources
Director, Institutional Research
Chair, Department of Biology
Faculty, Geological and Environmental Sciences
Director, Student Life and Leadership
Director, Academic Advising Services
Associate Vice President, Student Affairs
University Registrar
Director, Chico Student Success Center
Information Resources
Chair, Communications Studies
MegaDirector, Career Center
Director Educational Opportunity Program
Financial Aid
Chair- Academic Senate
Director, University Housing and Food Services
Faculty, Communication Design
Director, First Year Experience Program
Chair, Comparative Religion and Humanities
Dean of Undergraduate Education
STEP 2: Inventory High Impact Practices
STEP 2: Inventory HIPs
A Daunting Task, Un-Daunted
1. Review HIPs established by Kuh (2008) &
others
2. Review characteristics of HIPs in Kuh &
O’Donnell (2013)
3. Convene diverse group to brainstorm re:
Graduation Initiative Team: 7 faculty, Senate
Chair, AVP Enrollment Management, EOP,
FYE, IR, Student Success, Advising Directors,
Registrar, other professional staff … 20 people
Chico State HIPs
Inventory:
(1) HIP; (2)
Estimated Impact;
(3) “Trackability;”
and (4) Local
example
Task 2: HIP Inventory
Early Alert
Pedagogy:
First-Year Experience
Teaching
Intrusive Advising
Practices
Learning Communities
Mentor Services
Summer Bridge
Summer Transition (non-EOP summer bridge experiences)
Supplemental Instruction
Undergraduate Research
Interventions: Helps
Service Learning, Community-Based Learning
students resume on
Internships
Capstone Courses and Projects
their educational path
Common Intellectual Experiences
Diversity/Global Learning/Study Abroad
Collaborative Assignments and Projects
Writing-Intensive Courses
Practices: Adds value
Coordinated Student On-Campus Employment
to a students
Pedagogy Sphere
educational path
STEP 3: HIP Inventory IR Review
Identified HIPS
•
•
•
•
•
•
•
•
•
Service Learning
First Year Experience
Undergraduate Research
Internships
Study Abroad
Supplemental Instruction
Peer Mentoring Programs
Learning Communities
Campus Employment
Created Codes
Label
DESC
HIP001
HIP002
HIP003
HIP004
HIP005
HIP006
HIP007
HIP008
HIP009
HIP010
HIP011
HIP012
Internship
Pre-College Experience: ESE
Pre-College Experience: ESM
Pre-College Experience: SOAR
Pre-College Experience: Summer Bridge
Service Learning
Study Abroad
Student Employment
Writing-Intensive Course
Co-Curricular Service Learning
HIP013
OSR: Peer Lab Participant
HIP014
OSR: Peer Lab Leader
HIP015
HIP016
HIP017
HIP018
HIP019
HIP020
HIP021
HIP022
HIP023
HIP024
HIP025
Other HIPS entry in
progress
OSR: Peer Consultant Leader
OSR: Peer Consultant Participant
OSR: Workshop Attendee
OSR: Symposium Presenter
OSR: Faculty Student Grant (Student Recipient)
OSR: Student Grant (Recipient)
OSR: Peer Statistics (Specialist)
OSR: Peer Statistics (Attendee)
OSR: Journal Submission (Paper Accepted)
OSR: Summer Program (Applicant Accepted)
OSR: CSUSB Research Competition (Participant)
OSR: CSU Research Competition (Participant)
OSR: Course re-design (Participant)
Collected Data in Peoplesoft
Scaling HIPs at CSUSB
We are
here
Analysis
•
•
•
•
•
GPA
Retention Rates
Graduation Rates
Engagement
Other: Qualitative and quantitative data on
effectiveness of individual HIPs
Cal Poly Pomona HIP Survey Results
High Impact Practices
On-campus employment
Collaborative/team projects
First year experience
Capstone course/project
Internships
Undergraduate research
Peer mentoring
Service-learning
Writing Intensive courses
Study Abroad / global learning
Learning Communities
Residence Hall Living
Summer transition programs
Supplemental Instruction
Student clubs
Leadership Development
Tutoring
Diversity programming
Field trips
Sophomore Seminar
Advising
Common intellectual experience
Volunteer activities
% of
Responses
65
56
54
54
50
46
42
40
29
27
25
15
15
8
2
2
2
2
1
1
1
1
1
Data Collection Comment
Payroll, MyBAR
Should find a way to collect data
Course Enrollment
Course Enrollment
Course Enrollment, S4
Office of UG Research, MyBAR
MyBAR
Course Enrollment, S4
Should find a way to collect data
International Center, MyBAR
Should find a way to collect data
MyBAR
Should find a way to collect data
Too few responses to warrant attention
MyBAR
Too few responses to warrant attention
Too few responses to warrant attention
Too few responses to warrant attention
Too few responses to warrant attention
Too few responses to warrant attention
Too few responses to warrant attention
Too few responses to warrant attention
S4
Current/
Future
Current
Future
Current
Future
Future
Curent
Future
Future
Future
Future
Future
Future
Future
Current
Future
Institutional Research and
Academic Resources is:
• Conducting meetings
• Gathering information
about data tracking and
organization
• Supporting CSU
Strategic Use of Data
Initiative
• Using questionnaire to
guide meetings
Questionnaire Highlights:
• Define the program in which your students participate.
• How are students assisted by your program?
• What impact are you trying to make?
• What do you think is the best measure of success of
your program?
• What do students need to do to be considered active?
How long? How is participation verified?
• How are you storing data? How far back does it go?
• How can we pull/gather your data? Do you get regular
reports or data dumps? From where?
Cameron Stevenson, CSUSM
Institutional Planning & Analysis
CSUSM’s
Inventory
Of HIPs Data
Spring 2015 Exploratory Analysis
• First Year Seminar & Learning Communities 
• Summer Transition 
• Supplemental Instruction 
• On-Campus Employment 
• Study Abroad ~
• Capstones ~
High priority for further analysis
~ Low priority for further analysis
Next Steps at CSUSM
•
Analytics, data models, and visualization tools
•
Business process for tracking in student records
•
Asking different questions… Who? When? How?
Task 3: HIP Inventory IR Review
Measures
GPA
Retention
Persistence
Graduation
Repeatable grade rates
Several measures of academic, civic engagement, and psycho-social well-being
# of units completed
STEP 4: HIP Tracking in Electronic Records
Aromatization of HIP Tracking
SFSU Model
Campus Dashboard
Electronic Student Record
System Dashboard
Student Success Destination:
CSU HIPs Dashboard
HIP Model: Step 3- Status Designation
HIP Model: Step 4- Development
Humboldt Model
L. Castellino 2015
56
Step
7
Step-by-Step
Fullerton Model
Step A: Create a Taskforce
Step B: Campus-wide HIP Inventory
Step C: Pre-HIP Status designation process
(Fullerton Phase II)
Step D: HIP Development Process
Step E: HIP Designation
Step 1- Assess and dedicate resources
Step 2- Identify your available data.
Step 3- Complete decomposition study
on student performance.
Step 4- Complete literature review on
best practices.
Step 5- Inform stakeholders, programs
& students. Communicate x 12
Step 6- Fullerton Model
Step 7- Three- to five-year
assessment cycle.
Step 8- Sunset what isn’t working.
Step 9- Redeploy resources as needed.
HIP Model: Step 5-Assessment
• Began in 2013-2014
• $7.2 million annual
continuous
allocation from the
Chancellor
• 15 CSU campuses
• 44 distinct High
Impact Practice
programs
• HIP “nursery”
ASSP Practitioner Learning Community
Engages participants in iterations of a short-term Improvement Science model to
test change and identify improvement.
ACT: Programs
complete improvement
plans based on external
and internal
assessment findings
PLAN: Identify ASSP
campus programs and
create transparent
practitioner learning
community environment
P-D-S-A
Plan-Do-Study-Act
Testing Changes
The Plan-Do-Study-Act
(PDSA) cycle is shorthand
for testing change in the
real work setting- by
planning it, trying it,
observing the results, and
acting on what is learned.
This is the scientific
method used for actionoriented learning.
ACT: Programs
complete improvement
plans based on external
and internal
assessment findings
DO: Engage in annual
assessment cycle,
conduct campus visits,
hold digital and in=person
meetings/trainings
ASSP
Practitioner Learning Community
Setting Aims
Improvement requires setting aims. The aim should be time-specific
and measureable; it should also define the specific population of
students that will be affected.



450,000 CSU students
Assist CSU Graduation Initiative Goals & identify effective HIPS
Annual assessment cycle of 44 ASSP programs
Selecting Changes
All improvement requires making changes, but not all changes result in
improvement. Organizations therefore must identify the changes that
are most likely to result in improvement.






Define HIPs within the CSU
Identify common HIP measures, program components, outcomes, metrics
Validate existing taxonomies
Identify national HIP partners
Create a HIP library of related literature
Increase HIP research, publication, and presentations
Establishing Measures
Teams use quantitative and qualitative measures to determine if a
specific change actually leads to improvement.



CA CSU Student Success & Accountability Metrics
ASSP HIP Operational Outcome Sets
ASSP HIP Student Outcome Sets
ASSP Program Rubric
HIP Implementation Model
National HIP Efforts
www.calstate.edu