Holistic Course Placement and
Whole-Student Model of Early
Intervention
Kevin Li, Dean of Instruction
Sara Schupack, Director of Developmental Education
Lawrence Buonaguidi, Quality Assurance Coordinator
Steve Robbins, Director of Research Innovation, Academic and Workforce Success
The Innovations 2013 Conference
March 12, 2013; Dallas, Texas
Part 1: Holistic Course Placement
ETS/CCC Collaboration Objectives
• Combine COMPASS and Psychosocial Measure to accelerate
student progress and college completion
– Students placed within the upper-band and combined with
high psychosocial scores can enroll in the next level course
• Allow more individualized, actionable plans for students tied to
curricular goals
• Non-cognitive survey and in-class behavioral ratings will be used
to trigger Early Alert and intervention strategies
• Create supplementary services tied to Early Alert and to
behavioral monitoring
Background
• Two new studies from the Community College Research Center
(CCRC) at Teachers College, Columbia University suggest that a
significant number of students may be accelerated if provided
proper supports
• Placement tests alone do not yield strong predictions of how
students will perform in college
• Success requires intervening with at-risk students to ensure
completion and success
An Alternative Measure Based on Grounded
Research: Non-Cognitive Assessment
• A meta-analysis was conducted as an attempt to integrate the
psychological and educational literatures regarding the
postsecondary outcomes of achievement and retention
(Robbins, S., Lauver, K., Le, H., Davis, D., Langley, R., &
Carlstrom, A., 2004)
• Examined the relationship between psychosocial and study skill
factors (PSFs) across 109 studies
• Results indicated relationships between retention and
academic goals, academic self-efficacy, and academic-related
skills
An Alternative Measure Based on Grounded
Research: Non-Cognitive Assessment
• Respect the whole student (examine cognitive, non-cognitive,
behavioral factors)
• Focus on the unique individuality of students
• Focus on factors educators can control and empower students
to modify: such as motivation, social connectedness, as
opposed to socio-economic, situational factors
• CCC found that behavioral monitoring and intervention was key
to student success
[HAND-OUT for group exercise 1]
Group interaction 1: 5 min.
• Form a 3-4 person group
• Look over the hand-outs
• Discuss what “non-cognitive” or “psycho-social skills” mean to
you and how they are manifested.
• How would you assess, monitor, and address these
components at your campus?
• Report out
[HAND-OUT for group exercise 1]
Domain
Academic Skills
Behaviors, beliefs,
and skills that
directly facilitate
academic success.
Motivation
Commitment to,
drive toward, and
perceived
importance of
academic success.
Self-Management
Able to anticipate
and respond to
pressure and stress
related to college
life.
Social Support
Perceived
availability of
resources to
support academic
success.
Facet
Definition
Example Items
Organization
Being planful in one’s actions, including effective use
I make a schedule for getting my school work done.
of time management and organizational tools and
I take due dates seriously.
strategies.
Behavioral Compliance
Complying with expected, academic-related
behaviors that directly impact academic success.
I attend almost all of my classes.
I complete the reading that is assigned to me.
Academic Self-Efficacy
Belief in one’s ability to perform and achieve in an
academic setting.
I'm confident that I will succeed in my courses this
semester.
I can do well in college if I apply myself.
Degree Goals
One of my life goals is to graduate college.
Clear goal commitment and plan of action to achieve
The benefit of a college education outweighs the
post secondary goals
cost.
Institutional Commitment
Attitudes toward and attachment to college or the
institution.
This is the right school for me.
I’m proud to say I attend this school.
Energy & Determination
Propensity to work hard in attaining academic goals.
Nothing can stop me from graduating from college.
I push myself very hard to succeed.
Stress Sensitivity
Tendency to react to daily hassles academic stress
I get stressed out easily when things don't go my
way.
I am easily frustrated.
Coping
Using adaptive strategies to deal with academic
stressors or pressures.
When feeling stressed about the amount of
homework I have, I try to get organized to get on top
of my homework.
When worried about a test, I try to get prepared for
the test regardless.
Test Anxiety
Negative reactions to the pressures and experience
of academic testing.
When taking a test, I think about what happens if I
don't do well.
Before a test, my stomach gets upset.
Connectedness
A general sense of belonging and engagement in
college life.
I feel connected to my peers.
People understand me.
Institutional Support
If I don't understand something in class, I ask the
The ability to identify and use resources and supports instructor for help.
that lead to success in college.
I know how to find out what's expected of me in
classes.
Instrumental Support
External factors that could enable or impede
academic success.
Family pressures make it hard for me to commit to
school.
People close to me support me going to college.
Risk Indices: Separate indices for both
classroom and enrollment success. Based
on background, cognitive , and psychosocial information and supported by
statistical relationships with success.
Background Information: Communicate
key student information from both
SuccessNavigator and SIS to
faculty/advisor.
Domain Scores: Four general areas of
student strengths and weaknesses.
Scores are presented normatively.
Feedback: Determine by more specific
“facet” scores (see next page).
Action Plans: Suggested interaction
with programs and services on campus.
Risk Indices: Separate indices for both
classroom and enrollment success. Based
on background, cognitive , and psychosocial information and supported by
statistical relationships with success.
Background Information: Communicate
key student information from both
SuccessNavigator and SIS to
faculty/advisor.
Domain Scores: Four general areas of
student strengths and weaknesses.
Scores are presented normatively.
Feedback: Determine by more specific
“facet” scores (see next page).
Action Plans: Suggested interaction
with programs and services on campus.
Risk Indices: Separate indices for both
classroom and enrollment success. Based
on background, cognitive , and psychosocial information and supported by
statistical relationships with success.
Background Information: Communicate
key student information from both
SuccessNavigator and SIS to
faculty/advisor.
Domain Scores: Four general areas of
student strengths and weaknesses.
Scores are presented normatively.
Feedback: Determine by more specific
“facet” scores (see next page).
Action Plans: Suggested interaction
with programs and services on campus.
The “Success Navigator” - Purpose
• Low Stakes designed to assess the dispositions and
noncognitive skills of incoming college students
• Can be used with or without academic markers (SAT, HS GPA,
course placement score)
• Three primary intended uses:
– Identifying likelihood of persistence to degree and
academic failure
– Informing course placement decisions
– Developmental Feedback for advising, FYE, etc
The “Success Navigator” - Logistics
• Length: 20-25 minute on-line assessment
• Items
– Initially, ~120 self-report items
– Incorporating anchoring vignettes and forced-choice items
– Demographic and life event information
• Administration: Varies per institution: could be at home,
during new student orientation, or in first-year experience
course
Sample of Holistic Course Placement Chart
Existing Reading Placement
COMPASS
0 - 20
Conference Required
21 - 37
Foundational Studies 1
38 - 49
Foundational Studies 2
50 - 64
Reading 099
65 - 79
Reading 125
80 and above
Reading Not Required
COMPASS
0 - 20
21 - 29
30-37
38 - 42
43-49
50 – 57
58-64
65 - 72
73-79
80 and above
Future Holistic Reading Placement
Success Navigator
NA
NA
>= .8
NA
>= .8
NA
>= .8
NA
>= .8
Course placement
Conference Required
Foundational Studies 1
Foundational Studies 2
Foundational Studies 2
Reading 99
Reading 099
Reading 125
Reading 125
Reading Not Required
Reading Not Required
Expected Outcomes of Holistic Placement
• More accurate placement
• Acceleration: Less time spent in developmental education
• Help us get to know our incoming students early and
efficiently
• More nuanced understanding of the whole student in a
meaningful way
• Provide data to help develop placement and intervention
programs
Part 2: Whole-Person Model of
Early Alert
Whole-person model: 3 domains of assessment
Whole-student/Whole-person Early Alert Model
Research shows that behavioral monitoring
and intervention are key to student success
Math
Readiness
Effort Level
High
Medium
Low
High
.96
.91
--
Medium
.94
.77
.82
Low
.86
.61
.38
Note. N = 411. Success rates were not calculated for cells with
insufficient sample sizes (denoted by a dash).
Group interaction 2: 10 minutes
• Form a group of 3-4
• Look over the hand-outs.
• Think about your students and your programs: What are ways
that you could best support the whole student on your campus?
What collaborations across departments and units could you
encourage?
• What are different ways that you can reach out to at-risk
students?
• What are different ways that you can monitor students’
behaviors in order to intervene with a robust support program?
• Report out
Use the Whole Student Model to
Drive Intervention Strategies
• Map support services to individual students’ needs based on
non-cognitive scores
• Feedback loop to share non-cognitive report with students to
arrive at action plans for students
• Consider other strategies to further student success, such as
supplementary instruction, early alert and support other new
and existing initiatives
[HAND-OUT for group exercise 2]
Comprehensive Early Interventions
•
•
Model and coach at-risk students toward a set of desirable behaviors.
Based on previous research (Li et al., 2012), low-level dev ed math students can
increase the likelihood of passing the course by 30% if they exhibit desirable inclass behaviors:
– Active participation in groupwork (Student is actively engaged during group
work; helps other students with assignments; does his/her fair share of the
work, etc)
– Active participation in lecture (Student's alert, attentive during class,
asks/answers questions, etc)
– Attendance (Attends class, stays for whole period, etc)
– Completion of homework assignments (Student completes assignments
thoroughly and completely, turns assignments in on time)
[HAND-OUT for group exercise 2]
Interventions: Yes we can!
• Three basic implications on early interventions:
• As educators, we can motivate our students, shape and
reinforce their intentionality and academic determination.
(Surrounding non-cognitive assessment)
– Outward behaviors are indicators of inwards states such as
motivation and engagement
– We have the opportunity and capability to coach our
students towards a set of desirable classroom/academic
behaviors.
[HAND-OUT for group exercise 2]
Comprehensive Early Interventions
• Map support services to non-cognitive psychological constructs:
Academic Skills, Self Management, Motivation, Social Support
Non-Cognitive
construct
Academic Skills
(Behaviors, beliefs and
skills that facilitate
success)
Self Management
(Able to anticipate and
respond to pressure
and stress)
Motivation
(Commitment to, drive
towards, and
perceived importance
of success.)
Social support
(Perceived availability
of resources)
Psycho-social Skills
Refer Student to Existing
natural fit support services
at CCC
Study skills
Study skills workshop
Time management skills
Study skills workshop
Emotional control
Wellness Center
Resilience
Meeting with advisors,
tutors and instructors
Degree intention
Transfer center, advising
Class attendance
Advisors, tutors, instructors
Engagement
Advisors, tutors, instructors
Help seeking
New student orientations,
advisors, service referrals
Peer support
Student clubs and
organizations
Facet
•Organization
•Behavioral Compliance
•Academic Self-Efficacy
•Stress Sensitivity
•Coping
•Test Anxiety
•Degree Goals
•Institutional Commitment
•Energy & Determination
•Connectedness
•Institutional Support
•Instrumental Support
Recommendations: The 3 Pillars of Success
Academic
Readiness
•
•
•
•
Behavioral
Factors
Instruction and
Institutional Resources
Place students more accurately using multiple measures, including non-cognitive
skills for possible acceleration
Smarter, strategic interventions: coordinate efforts to support the whole student
Institutional commitment: Faculty, advisors, and staff collaborate in early alert
program
Generate fine-grained reporting to empower frontline workers and drive further
research and program development
References
ACT, Inc. (2002). Creating seamless educational transitions for urban African American and Hispanic students. Iowa City, IA: Author.
Scott-Clayton, J. (2012). Do High-Stakes Placement Exams Predict College Success? Predicting Success in College: The Importance of
Placement Tests and High School Transcripts. CCRC Working Paper No. 41. New York: Community College Research Center,
Teachers College, Columbia University.
ACT, Inc. (2004). Schools involving parents in early postsecondary planning. Iowa City, IA: Author.
ACT, Inc. (2008). Student Readiness Inventory user’s guide. Retrieved from ACT, Inc. website:
http://www.act.org/sri/pdf/UserGuide.pdf
ACT, Inc. (2008). What we know about college success: Using ACT data to inform educational issues. Iowa City, IA: Author.
ACT, Inc. (2008). Student Readiness Inventory Toolshop. Retrieved from ACT, Inc. website:
http://www.act.org/sri/studentguide/toolshop.html
Allen, J. & Robbins, S. (2010). Effects of interest-major congruence, motivation, and academic performance on timely degree
attainment. Journal of Counseling Psychology, 57, 23-35.
Allen, J., Robbins, S., Casillas, A., & Oh, I.-S. (2008). Third-year college retention and transfer: effects of academic performance,
motivation, and social connectedness. Research in Higher Education, 49(7), 647-664.
American Association of Community Colleges (AACC). (2009). Community college fast facts. Retrieved July 10, 2009 from
http://www2.aacc.nche.edu/research/index.htm
Belfield, C.R., Crosta, P.M. (2012). Predicting Success in College: The Importance of Placement Tests and High School Transcripts. CCRC
Working Paper No. 42. New York: Community College Research Center, Teachers College, Columbia University.
References (cont.)
Berkner, L., & Choy, S. (2008). Descriptive summary of 2003-04 beginning postsecondary students: Three years later. Washington, DC:
U.S. Department of Education, Institute of Education Sciences.
Bragg, D. D. (2001). Community college access, mission and outcomes: Considering intriguing intersections and challenges. Peabody
Journal of Education. 76(1), 93-116.
Community College Research Center. (2002a). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008,
from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8
Community College Research Center. (2002b). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008,
2008, from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8
Habley, W. & McClanahan, R. (2004). What Works in student retention – All survey colleges. Iowa City, IA: ACT, Inc.
Harris, S. (1991) Can’t you guys read? Cartoons on academia. New Brunswick, NJ: Rutgers University Press.
Herzog, K. & Miller, M. P. (Eds.). (1985). Scholarship: More great cartoons from the Kappan. Bloomington, IN: Phi Beta Kappa
International Inc.
Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college
outcomes: Constructing the Student Readiness Inventory. Educational and Psychological Measurement, 65, 482-508.
Li, K, & Cordes, C. (2010). Individualizing the Delivery of Math Developmental Education. Talk presented at the 34th Annual National
Association of Developmental Education Conference, Columbus, OH.
Lotkowski, V., Robbins, S., & Noeth, R. (2004). The role of academic and non-academic factors in improving college retention.
Retrieved from ACT, Inc. website: www.act.org/research/policy/index.html.
McCabe, R.H. (2000). No one to waste. Washington, DC: Community College Press.
NCES (2004). Remedial education at degree-granting postsecondary institutions in fall 2000 (NCES 2004-010).
NISOD (2012). Assessing College-readiness assessments. Webinar delivered on 3/28/2012.
References (cont.)
Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco: Jossey-Bass.
Peterson, C. H., Casillas, A., & Robbins, S. B. (2006). The Student Readiness Inventory and the Big Five: Examining social
desirability and college academic performance. Personality and Individual Difference, 41, 663-673.
Porchea, S., Allen, J., Robbins, S., Phelps, R. (in press). Predictors of long-term enrollment and degree outcomes for community
college students: Integrating academic, psychosocial, socio-demographic, and situational factors. Journal of Higher Education.
President’s Commission on Higher Education. (1947). Organizing higher education. Washington, DC: Author.
Robbins, S. B., Allen, J. Casillas, A., Akamigbo, A., Saltonstall, M., Campbell, R., Mahoney, E. & Gore, P.A. (2009). Associations
of resource and service utilization, risk level, and college outcomes. Research in Higher Education, 50(1), 101-118.
Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social,
and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598616.
Robbins, S. B., Lauver, K., Le, H., David, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict
college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288.
Robbins, S., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by
motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology,
94, 1163-1184.
Swail, W. S. (2004, January 23). Legislation to improve graduation rates could have the opposite effect. The Chronicle of Higher
Education, B16.
Refining Course Placement
Practices and Driving toward
Student Success using a Holistic
Model
Correspondence regarding this presentation
should be addressed to:
Steve Robbins, Director, Research Innovation, ETS
srobbins@ets.org
Kevin Li, Dean of Instruction
Wilbur Wright College,
One of the City Colleges of Chicago
kli@ccc.edu