Creating “Turning Points” in the Lives of
Youth Residing in High Risk Communities:
Participation and Response to School-Based
Mentoring and Impact on Academic Outcomes
Marcia Hughes
May 31, 2006
Purpose of the Study
Examine a youth program that was designed to help
participants from high risk communities do well in school,
and plan and prepare for post secondary education.
The Intervention Project: GEAR UP
(Gaining Early Awareness and Readiness for Undergraduate Programs)
• Federally-funded: US Department of Education
• Collaborative effort: University of Connecticut & Hartford Public Schools
The study asks for whom was the program effective,
in what ways, and under what circumstances.
Acknowledgements
Principal Investigators: John C. Bennett, Sara Harkness,
Kay A. Norlander-Case, Charles W. Case
Advising Committee: Sara Harkness, Preston Britner,
Orv Karan, and Charlie Super
Undergraduate, Graduate Students, Participating Youth:
Many thanks to all the undergraduate and graduate students
who have served on the project, & to all the participating youth.
Background:
Evolution of Youth development field (Small & Memmo, 2004)
- Prevention: target a particular youth population deemed at risk of problem
- Fix-it approach: Runs counter to what is known about
human motivation
- Resilience: Risk and protective mechanisms exist together
- basis of mentoring programs
- assist with coping, problem solving
Youth Development Field:
- Positive, asset building orientation
- Builds on strengths rather than categorizing according to deficits
- Get involved, develop competencies, experience success as intervention
Background: “Process is Product” (McLaughlin, 2000)
Developmental outcomes of interest:
- Learning to be productive; learning to connect; learning to navigate
(Gambone, Klem, Connell, 2002)
- Constructive use of time, commitment to learn, positive values, & social
competence (Small & Memmo, 2004)
- Confidence, character, connection, competence, contribution (Pittman,
Irby, Tolman, Yohalem, & Ferber, 2003)
Getting “Buy-in” and Sense of Belonging:
- “Just opening the doors and getting youth involved marks a major
accomplishment.” (McLaughlin, 2000)
- Belonging is of equal significance and utility in youth programs as social,
behavioral, and academic outcomes (Barkdull, 2004)
Background: Broad Based Comprehensive Strategy
Contemporary models of youth development and problem prevention:
Incorporates all 3 paradigms, framed in a developmental-ecological
model:
•Prevention
•Resilience
•Positive youth development
(Catalano, Boglund, Ryan, Lonczak, & Hawkins, 1998; Gambone et al., 2002; Kerpelman,
2004; Small & Memmo,2004)
“The State of the Field” (Catalano et al, 1998)
Evaluation of 25 National programs:
Promote bonding
•Foster resilience
•Promote social competence
•Promote emotional competence
•Promote cognitive competence
•Promote behavioral competence
•Promote moral competence
•Foster self-determination
•Foster spirituality
•Foster self-efficacy
•Foster clear and positive identity
•Foster belief in the future
•Provide recognition for positive behavior
•Provide opportunities for prosocial involvement
•Foster prosocial norms
Background: Research and Program Evaluation
In order to advance the Youth Development Field, calling for:
- More meaningful measures (Gambone et al, 2002; McLaughlin, 2000)
- Assessing the “whole” child (Catalano et al, 1998; Riggs & Greenber, 2004)
- Linking the chain of effects (Catalano et al, 1998; Gambone et al, 2002)
- Getting inside the “Black Box”: Document/describe/understand
day-to-day realities and challenges of youth programs (Kalafat &
Illback, 1998; Koss-Chioino & Vargas, 1999; Small, 2005)
Research Model
PROBLEM BEHAVIORS
RISK FACTORS &
PROGRAM
PROMOTING PROCESSES PARTICIPATION &
RESPONSIVENESS
Individual
level
Family
level
School &
Community
level
- Social Support
- Emotional bond/support
- Academic Support
- Cognitive Skills
- Self-Awareness
- Belonging
- Sense of Competence
- Sense of Control/
Coping Strategies
- Improved relationships
Last 2 years of
project participation
Post-Secondary
Outcomes
- University or
Community College
- Technical School
- Working
- Retained
- Drop out
Research Questions
1) By what means, if at all, have the components of the project
fostered processes that facilitated positive change for the youth?
2) How are youths’ level of participation and response to the
intervention modified by individual adolescent needs and individual
situations at home and at school? Specifically:
• What are the overlapping problem behaviors and support?
• What are the overlapping risk factors and promoting processes?
• In what ways did the project mitigate risk exposure?
• In what ways did the project promote positive processes?
3) How are differences in youth’s participation and responsiveness
related to their post-secondary status (i.e., long-term goal of the project)?
GEAR UP: Current Status
• Longitudinal
project, began in 1999:
- Two cohorts of students in the 6th & 7th grades (N=130)
- Have now completed the 11th and 12th grades (N=110)
• Indicators
of Success:
- Continued high rate of participation (Approx. 82%)
- Low school dropout rate (Approx. 10%)
- High rate of students accepted for post secondary education (Approx. 50%)
Intervention Model: Program Description
(Harkness, Hughes, Muller, & Super, 2004)
•Site-based program: Ongoing presence in the schools
•Continuity across contexts of home and school
•Transitional support: early adolescence - early adulthood
•One-to-one mentoring assignments
•Strong infrastructure: close monitoring and accountability
Intervention Model: Practicing Principles
(Harkness, Hughes, Muller, & Super, 2004)
•Intervene with youth in the context of their environments
•Not one-size-fits-all intervention/Environmental Fit
•Ethnographic process of inquiry, collaborator, not “expert”
•Focus: strengths, resources, & broad indices of development
•Process first, outcomes second
Methodology: Study Sample
Wide range of participants:
From the student who is excelling academically to the
student who appears to be near drop-out.
Total # of participants/subjects in this study:
• 45 from older cohort: 23 Male/20 Female
• 34 from younger cohort: 22 Male/12 Female
• Total: 79 Subjects: 45 Male/32 Female
Ethnic Distribution:
• 65 are Hispanic (all but 5 are Puerto Rican)
• 12 are Black (all but 3 are African-American)
• 2 are White
Methodology: Data Collection
Data Collection (secondary data)
• Documentation of process data for each participant
• Collected by mentors/practitioners
• In the form of a “service note” and “summary reports”
Instruction on Data Collection
• Service notes: Every 2 weeks with feedback
• Summary reports: End of each semester
Analyzed data for the last 2 years of the project
• Older Cohort: Fall semester 2003 - Summer 2005
• Younger Cohort: Fall semester 2004 - Present (2006)
“Checklist” for organizing observations and impressions
Summary Reports: Different Children, Different Needs
Methods of Analysis
I Analysis of Qualitative Data: Coding system; analyzing service notes
1) Responsiveness 2) Problem behavior/risk factors 3) Promoting processes
II Factor Analysis: Reduced the large number of variables (i.e., codes)
down to a few factors for each of the 3 subsets of variables -responsiveness,
problem/risk factors, and promoting processes
III Cluster analysis: Students were “clustered” into groups based on
similar profiles across the factor scales (i.e., on responsiveness,
problem/risk factors, and promoting processes)
IV Chi-Square to test the effect of cluster group (each group with its
own “profile” across responsiveness, problem/risk factors, and promoting
processes ) on post secondary outcomes
Coding System for Analysis of Qualitative Data
Student participation &
responsiveness
(R)
Problem behaviors
and risk factors
(A, B, C)
Promoting
processes
(E, F, G)
1. Social/instrumental support
2. Mentor bond/emotional support
3. Academic support
4. Cognitive skills
5. Self-awareness
6. Belonging and membership
7. Improved sense of competence
8. Improved sense of control
9. Improved relationships
1. Problem behaviors:
motivational, behavioral,
academic, interpersonal,
resistance, stress
2. Risk: individual level
3. Risk: family level
4. Risk: school level
1. Individual
2. Family
3. School
Improved sense of competence/ability; positive school
experience; looks for attention and approval from mentor
Goal oriented and goal setting; self-awareness
including self evaluation and discussion of future self
Improved sense of control & improved sense of competence:
Hopeless and giving up but with sustained effort (versus avoidance),
reduction of negative chain reactions and “corrective” experience
Results of Qualitative Analysis:
Fine tuning and reliability of coding system
Coding system: fine tuning
• For each service note, code was noted if present (0 or1)
• Progress/report cards to confirm related mentor reports
• Double-coded 18 cases
Reliability:
• Aggregated data by obtaining % across observations
• Calculated alphas: two sets of coded data for same youth
• Mean reliability = .92
Factor Analyses: Description
Data Screening
• Aggregated data by obtaining % across observations
• One outlier that was removed
• Transformed variables where necessary (skewness or kurtosis)
Separate Analysis (responsiveness, problem behaviors/risk factors, promoting processes)
• Principal Components Analysis: Decide on number of factors
• Factor Analysis: Varimax rotation to maximize the spread in loadings
Factors
• Responsiveness to Project: 4 Factors/16 “observed” variables
• Problem Behaviors/Risk Factors: 4 Factors/17 observed variables
• Promoting Processes: 3 Factors/10 observed variables
4 Factors (59%)
Academic
Engagement
(α=.82)
Goal Setting
(α=.66)
Relatedness
(α=.69)
Special
Relationship
(α=.47)
Observable variables: Student Responsiveness
2.1 Accepts
2.2 Asks
tutoring from mentor
for tutoring assistance
Loadings
.62
.49
7.1
Improved beliefs about ability, attitude/behavior
.73
7.4
“Corrective” experience
.75
7.5
Positive school experience
.61
8.2
Engaged in problem solving, sustained effort
.81
2.3
College application process
.48
4.1
Increased self-awareness
.49
4.2
Constructing and believing in possible selves
.52
8.1
Goal setting, planning, and decision making
.71
8.4
Follow through, tasks/next steps, short term plans
.55
1.1.1
Relate to one another, common interests, & ideas
.60
5.2
Several good relationships within project
.91
3.1
Engage in meaningful conversation/debate
.50
1.1.3
Confides in mentor/asks for personal guidance
.54
1.1.5
Excitement / inspired by mentor
.47
4 Factors (60%)
Problem
Behaviors
(α=.79)
School
Disruption
(α=.77)
Stress and
Strain
(.71)
Low Academic
Motivation
(α=.54)
Observable variables: Problem Behavior/Risk Factors
Loadings
A.7 Breaking rules: tardy, skipping class or school
.53
B.2 Drug or alcohol abuse
.60
B.6 Delinquency/crime
.67
B.7 Alienation/rebelliousness
.64
C.2 Home regulation: chaotic
.69
C.5 Ineffective discipline
.46
D.6 Negative peer influence
.56
A.8 Disruptive in class/behavioral issues
.68
A.9 Not taking responsibility for behavior
.62
A.16 School discipline: suspension
.74
D.6 Negative peer influence
.45
A.13 Indicators of problems coping
.73
B.9 Mental health difficulties, low self-esteem
.74.
C.1 Family stress/strain
.52
C.4 No/limited knowledge or support of academic status
.54
A.3 Inconsistent motivation, not following through, distracted
.61
A.6 Avoidance/procrastination
.52
A.11 Failing grade(s), difficulty with class
.53
3 Factors (67.3%)
Goal Oriented
(α=.83)
High Academic
Motivation
(α=.81)
Connected
(α=.62)
Observed variables: Promoting Processes
Loadings
E.4
Self-regulated: goal-oriented & purposeful
.63
E.6
Enduring set of values, balanced perspective
.87
E.7
Good coping, problem-solving, & fortitude
.78
E.8
Positive attitude toward school
.58
E.3
Positive academic achievement
.79
E.4
Self-regulated: goal-oriented & purposeful
.56
E.9
Positive beliefs about self and future
.70
E.1 Positive attitude, responsive to others, pro-social
.49
E.2
Attachment to others/intimate relationship
.45
F.2
Parental warmth, encouragement, cohesion
.65
F.6 Stable, integrated family routines, interdependency
.59
Cluster Analyses:
Responsiveness, Problem/Risk factors, Promotive Processes
Factor Scales on the following 11 factors:
Student
Participation &
Response to
Project
Problem Behavior
& Risk Factors
1. Academic Engagement
2. Goal Setting
3. Relatedness
4. Special Relationship
5. Problem Behaviors
6. School Disruption
7. Stress and Strain
8. Low Academic Motivation
Promoting
Processes
9. Goal Oriented
10. High Academic Motivation
11. Connected
Results of Cluster Analysis
CLUSTERS:
1
2
3
4
# OF PARTICIPANTS:
26
12
27
9
Sig
4.9
7.6
10.6
8.4
.00
1. Academic Engagement
2. Goal Setting
15.8
21.4 21.1 18.3
.03
3. Relatedness
10.6
10.3
8.4
9.2
.31
4. Special Relationships
8.0
11.9
12.3
33.9
.00
5. Problem Behaviors
.64
.12
.67
.77
.20
6. School Disruption
.15
.21
.18
.31
.53
7. Stress and Strain
4.9
4.9
5.5
11.1
.00
8. Low Academic Motivation
7.9
6.4
15.5
8.5
.00
9. Goal Oriented
4.1
20.4
9.0
7.6
.00
10. High Academic Motivation
6.3
26.6
11.9
6.7
.00
11. Connected
4.6
8.1
9.3
10.0
.00
Chi-Square: Count/Percentage within Groups (χ2 =.35, p<.00)
Post
Secondary
Status
Group 1
Group 2
Goal
Setting
College
20%
83.3%
68%
55.6%
(5)
(10)
(17)
(5)
Technical
School
16%
0%
4%
0%
5
Working
28%
16.7%
20%
44.4%
18
(7)
(2)
(5)
(4)
8%
0%
1
0%
3
0%
8
9
71
Retained
(4)
33.8%
25
37
(4%)
0%
(7)
Total
Total
(1)
(2)
Drop Out
Group 3
Group 4
Academic
Special
Engagement Relationship
4%
(1)
12
25
Conclusions
Using process data obtained from a comprehensive
program that had noted success, was able to meet the
challenges that are current in the youth development field:
•
• Used meaningful measures
• Assessed the whole child
• Demonstrated that developmental outcomes can be
linked to long term academic outcomes
• Illuminated the day-to-day processes of the program
Conclusions
• For
whom, in what ways, and under what circumstances?
• Was able to identify patterns of change among subgroups
• Meaningful overlap on student responsiveness, problem
behaviors and risk factors, and promoting processes.
• Results support a developmental-ecological framework:
• In some cases, the project was able to mitigate exposure to risk.
• In other cases, the project was able to promote positive
development.