A Tale of Two Cities: Comprehensive
Assessment of College Student Readiness
Paul Gore, Ph.D.
University of Utah
Paul.gore@utah.edu
Student Readiness
There is concern that college students are not
prepared to meet the challenges of attaining
a college education.
• Average six year graduation rate at two-year
institutions = 34% (Swail, 2004)
• Average six-year graduation rate at four-year
institutions = 53% (Carey, 2004)
• First to second year retention
• Traditional 73.4%; Selective 81.7%;Two year
colleges 52%
Student Readiness
Student Readiness
Percent of Students Receiving Bachelor's Degree
by Remedial Course Taking Behavior
70
60
58
Percent
50
40
30
27
20
17
10
0
No Remediation
One or two Math
Courses
Any Remedial
Reading
Student Readiness
• Predicting student success
– Primary focus on cognitive factors (ACT, GPA)
– Pre-enrollment situational (SES, first generation)
– Post-enrollment situational (Housing, hours
worked on campus, campus climate)
– Non-cognitive/motivational (engagement,
achievement motivation, self-efficacy)
• Your institutions probably focuses on all 4
Student Readiness
• In 2004, Robbins and his colleagues
published a meta-analysis of noncognitive prediction of student
outcomes
• Their work included calculations of
incremental validity
Student Readiness
High School GPA
Non Cognitive Factors
Persistence and Success
Overview of the SRI
The Student Readiness Inventory (SRI) was
designed to measure the content domains that
surfaced in the Robbins meta-analysis
•
•
•
•
It is a low stakes, self-report instrument
Has 108 items that are face-valid
Ten content domains exist as scales
Each scale measures a construct that allows for
intervention
• The instrument contains two risk indices that
predict probability of success (performance and
persistence)
Overview of the SRI
Overview of the SRI
1st Sem
GPA
2nd Sem 2nd Sem 1st Year
GPA
Retention Retention
Academic Discipline
0.28
0.29
0.11
0.13
Academic Self-Confidence
0.17
0.19
0.06
0.09
Commitment to College
0.12
0.10
0.12
0.12
General Determination
0.12
0.12
0.06
0.06
Social Connection
0.05
0.04
0.10
0.09
ACT Composite Score
0.41
0.46
0.11
0.17
High School GPA
0.45
0.49
0.15
0.20
SES
0.16
0.19
0.07
0.10
*
*
*
*
*
*
*
Overview of the SRI
Institutional Application
• Programs designed to promote the transition of firstyear students abound
• Over 75% of all colleges and universities have an
FYE seminar course
• Many colleges and universities lack a systematic,
integrated, and coordinated set of programs
• Effort was undertaken to bring together first-year
campus programs around a common focus and to
use data to drive coordination efforts
Institutional Application
• Project developed by workgroup on first-year
including advising, faculty, FYE, Native
American and Multicultural Student Centers,
Assessment, LAC, New Student programs
• Goal: proactive, intrusive interventions which
could influence first year student success.
Institutional Application
Requirements
• Data from assessment of student success tool must be
usable by both student services/faculty and students
• Data must be presented in a user friendly, accessible format
• Data must be provided early enough for dissemination and
use by both student services and students themselves
• Results must connect students to university resources
• Different areas of campus must be willing to provide
programming to meet the needs identified by the students
success tool
Institutional Application
1.
ID early and make direct contact with students who were
determined to be at risk for either failure to persist at NAU or
failure to remain in good academic standing
2.
Use the information to match student needs with specific
university resources and services
3.
Embed the student profiles in FYE seminars to facilitate
understanding of the connection between student needs and
campus resources
4.
Use composite indices of retention and academic success to
build a customized retention prediction equation for NAU
Institutional Application
1.
Direct contact with students at risk



2.
Composite indices in SRI used
Identified group of reachable students based on resource
decisions. Interventions varied
NAU used retention risk index and did not allocate their
resources to the lowest percentiles
Connect students to campus resources using scale scores


Direct contact via email or postcards encouraging students to
access services or participate in events. SRI not explicitly
referenced
Examples: Students with potential academic problems invited to
LAC; Students with high scores on academic discipline and goal
striving invited to meet with Res Life and New Student Programs
to participate in leadership opportunities
Institutional Application
3.
Including student SRI profiles in lesson and instructional
activities in FYE seminar courses




Traditional 3-credit FYE for special admit students
One-credit FYE course for regularly admit students
Scores used regularly as opportunities for reflection in class
Students met one-on-one with instructor to develop student
success action plan
Institutional Application
Institutional Application
Academic discipline
General determination
Goal striving
Commitment to college
Study skills
Communication skills
Social connection
Social activity
Academic self-confidence
Emotional control
0
200
400
600
800
1000
1200
1400
low
Number of Students
1600
med
1800
high
very high
NO SRI
11%
MSC
7%
STAR
5%
No
Co
nta
ct
N
AS
S
INVALID
7%
3%
Official Freshman Cohort n=3171
LAC
6%
3%
SSS
4%
LC
4%
General GSSC
21%
SL 2
EPS Class
5%
GSSC
SA LOW
9%
FYE Class
15%
%
Institutional Application
Resource Use by GPA
3
2.95
2.91
2.87
2.9
2.86
2.82
GPA
2.8
2.7
2.76
2.67
2.67
2.63
2.62
2.6
2.53
2.5
2.4
2.3
0-4
5-19
20+
Recreation Resources
0
1
2+
Social Resources
0
1+
Academic Referrals
0
1-3
4+
Advising/Career Sessions
Institutional Application
Resource Use by Retention
1
0.88
0.9
0.85
Retention
0.8
0.77
0.84
0.77
0.76
0.72
0.71
0.69
0.7
0.6
0.6
0.52
0.5
0.4
0-4
5-19
20+
Recreation Resources
0
1
2+
Social Resources
0
1+
Academic Referrals
0
1-3
4+
Advising/Career Sessions
Institutional Application
• Final Predictive Model of GPA (33% of
variance)
–
–
–
–
–
–
–
–
–
–
Academic risk
Academic services
Academic services X Academic risk
Recreational resources
Social Resources
Academic referrals
Academic referrals X Academic risk
Advising/Career Services
Ethnicity
Gender
Institutional Application
• Final Predictive Model of Retention
(13% of variance)
– Retention risk
– Academic services
– Social resources
– Academic referrals
– Advising/Career services
Institutional Initiative
• Establish baseline understanding of U
of U students’ strengths and
weaknesses
• Develop strategic data-use plan with
academic and student support services
• Provide training as needed to support
outreach and early warning efforts
Institutional Initiative
• When compared to the national sample
– 7.8% of students fall in the lowest quartile
on the Retention Risk Index (N=175)
– 9.2% of students fall in the lowest quartile
on the Performance Risk Index (N=206)
Institutional Initiative
Academic Discipline
<25%
General Determination
25% - 75%
>75%
Goal Striving
SRI Scale
Commitment to College
Study Skills
Communication Skills
Social Activity
Social Connection
Academic Self-Confidence
Steadiness
0
500
1000
1500
Number of Students
2000
2500
Pre-Enrollment
First Semester
Long-Term
Assessment
Admissions Index/SRI
Data Processing
Early Outreach
Advising
First-Year Seminars
College Success
Class
Special Population
Services Outreach
Early Warning System
Other University
Support Services
Retention/Graduation
Career Services
Leadership/Mentors
Utilization/
Engagement
Program Refinement
Institutional Initiative
• Push back
– Students don’t like taking the instrument
– Too much time out of orientation (20
minutes)
– Not enough time available in advising
– Tracking concerns
– Support exists at the administrative level
but not the front line