TUFTS
UNIVERSITY
November 27, 2012
Wendy M. Wheeler
Interim Executive Director
Thrive Foundation for Youth
1010 El Camino Real, Ste 250
Menlo Park, CA 94025
Dear Wendy:
Re: Final Report- Revised
Promoting the Thriving Journey for America’s Adolescents: Igniting the Sparks for Positive
Youth Development through Enhancing Selection, Optimization, and Compensation
Enclosed please find our revised, final report for the grant “Promoting the Thriving Journey for
America’s Adolescents.” We are grateful for your continued support of research that promotes
positive development in children and adolescents across the United States.
Sincerely yours,
Edmond P. Bowers, Ph.D.
Research Assistant Professor
Director, Project GPS
Institute for Applied Research in Youth Development
307 Lincoln Filene Building
Tufts University
Medford, MA 02155
Enclosure
cc:
Richard M. Lerner, Jacqueline V. Lerner, Heidi Johnson
1
Final Report
Promoting the Thriving Journey for America’s Adolescents: Igniting the Sparks for Positive
Youth Development through Enhancing Selection, Optimization, and Compensation
Overview
The Thrive Foundation for Youth funded the Institute for Applied Research in Youth
Development from May 1, 2009 to August 31, 2012 to develop and test tools for youth-serving
professionals that were designed to increase their capacities to foster Selection of goals (S),
Optimization of means and resources (O) for attaining goals, and Compensation (C; SOC) skills
in youth. This document provides an overview of the tasks performed by the team of researchers
at the Institute of Applied Research on Youth Development (IARYD) in order to achieve the five
primary goals of the grant. These goals involved developing the following deliverables:
Goal #1: A Mentor Manual for Coaching Goal Management
Goal #2: Assessment Rubrics—Measurement of the Thriving Journey
Goal #3: Videotaped Exemplars
Goal #4: Evaluation of the Overall Tool Kit
Goal #5: Release of Versions 2.0 of Deliverables 1, 2, and 3 which, together, constitute the
“Goal Management Playbook.”
The results of these activities are evidenced in the work of Project GPS. The core components of
Project GPS are based on a translation of SOC theory and research. The project uses the
metaphor of a car’s GPS navigation system – you “choose your destination” and the GPS (your
SOC skills in this case) provides “strategies” to arrive at your destination (in this case, achieving
a goal). In Project GPS, “G” stands for “Goal Selection,” and reflects Selection skills. “P” stands
for “Pursuit of Strategies,” and reflects Optimization skills. “S” stands for “Shifting Gears,” and
reflects Compensation skills.
This final report also provides initial results concerning the validity and utility of the Project
GPS suite of tools. We believe that our efforts have helped to develop materials that will bring
the Thrive Foundation closer to reaching millions of children in diverse youth-serving programs
across the nation.
2
Goal 1. Mentor Manual for Coaching Goal Management
In order to create the most effective and empirically valid tools to increase the capacity of
mentors to assist youth in the growth of SOC (Selection of goals, Optimizations of means and
resources, and Compensation), a thorough review of the existing intentional self-regulation or
goal-management literature was produced. The review was based on what is considered the
cutting edge of goal-related theory – the work of action theorists such as Paul Baltes, Alexandra
Freund, Jochen Brandtstädter, and Jutta Heckhausen. As Paul Baltes has explained, all human
development rests on enacting the “fundamental pragmatics of life,” which he has described
using the SOC model of goal selection, pursuit, and maintenance.
A “practitioner-friendly” review was produced and delivered to Thrive on February 22, 2012.
This review included the latest research produced by IARYD researchers and other scholars in
the field of intentional self-regulation, such as material that appeared in volumes of New
Directions in Child and Adolescent Development and Advances in Child Development and
Behavior, and in a special issue of the Journal of Adolescence. The Thrive team plans to translate
this review into short reports suitable for the practitioner population.
Decisions about the structure and purpose of the Mentor Manual, or Playbook, was greatly
influenced by the literature review as well as by the expertise provided by having several former
school teachers on the Project GPS team at Tufts. In particular, we developed a mentor manual
that was designed to help mentors foster positive performance and task-oriented goal behaviors
with their youth, encouraging them to focus on initiative and motivation and mastery and skill
development.
The materials generated for the mentor manual were ultimately aggregated into three resource
books: 1. Background and Research on Project GPS; 2. The GPS Rubrics; and 3. Activities and
Videos.
As a whole, these resource books are designed to help mentors create individualized experiences
for youth, and to engage in appropriate and effective conversations about the goal-management
process.
Book 1 of the Mentor Manual provided youth-serving professionals with an overview of Project
GPS, a review of the evidence linking GPS skills to positive youth development (PYD), and
detailed descriptions of each of the GPS processes: Goal Selection, Pursuit of Strategies, and
Shifting Gears.
Goal Selection: A young person who has positive purpose is also one who is on a
thriving path. Therefore, young people need to understand the importance of selecting
positive goals and of having the skills to make good choices.
Pursuit of Strategies: Adolescents need to develop strategies to attain their goals. They
need to be able to make goal-specific plans and to develop appropriately the resources –
from practicing a skill to recruiting the help of others – to achieve their goals.
3
Shifting Gears: Youth must be able to switch to a new strategy when their initial strategy
fails to help achieve their goal. In these circumstances, they need to judge when it is
reasonable to stay with their original goals and when it is prudent to select a new goal, for
instance, when the chance to attain the initial goal is lost.
The resources in Book 3 include 30 activities that provide guidance for developmentally
appropriate conversations about what GPS is and what types of strategies and behaviors will
move youth toward reaching their goals and put them on a thriving path. A series of four
activities introduces youth to the concepts of goal management and to the use of the GPS rubrics
for self-assessment and self-reflection. The next 26 activities are divided into two categories, one
of each type of activity for each of the 13 GPS Skills. The two sets are “Getting the Feel” and
“Trying it Out.” These activities build upon constructivist learning theories and are derived from
PYD theory.
Getting the Feel activities introduce each skill through a hands-on challenge in which youth need
to use that GPS skill in order to succeed. Youth play a game together, engage in visualization or
artistic activities, or do something else that provides them with the opportunity to experience the
role and usefulness of that specific GPS skill. Each hands-on experience is followed-up using a
series of discussion questions derived from the Experiential Learning Cycle model. These
questions are used to structure debriefing questions that help youth reflect on the challenge and
understand the value of that skill. The first set of questions assesses “what happened?,” asking
the youth to express key aspects of their experience of the activity and to assess how they felt
and what they observed. The second set of questions assesses “so what?,” asking the youth to
name why the experience mattered and to assess the significance of the decisions they made
during the experience. The third set of questions asks “now what?,” as youth take what they
learned from the experience and consider how to apply it to their own work on their own goals.
Trying it Out activities then walk youth step-by-step through applying that skill to their own
goals, after the Getting the Feel activity has demonstrated the importance of the skill. In these
activities, youth work with their mentors on tangible products that will help them organize their
efforts as they work towards their goal(s). For example, they make lists, charts, reports, or art
projects in the form of puzzles or maps. The content of these activities is based on whatever goal
the youth themselves have selected to work on, whether it’s applying to college, shooting a
basketball through the hoop, or getting a summer job. These activities help the youth come to
understand themselves as active producers of their own development, individuals who can use
these skills to practice gaining strength in the means that will help them reach their ends, thus
building agency. All Trying it Out activities also include a suggestion for a follow-up task after
each activity for an extended learning opportunity.
Each lesson plan grows out of a constructivist model (Bruner, 1973), complete with plentiful
suggestions for adapting the activities to suit the particular situation and needs of the individual.
Suggestions are included for adapting the materials by age; for an individual or group; or for
youth with different learning styles (e.g., artistic or verbal). Youth actively engage in their own
learning, as they share their goals, strengths, struggles, thoughts, and feelings. They have an
opportunity for reflection after each activity.
4
Goal 2: Assessment Rubrics/Growth Grids – Measurement of the Thriving Journey
The core tools for Project GPS are the rubrics or “growth grids.” The growth grids provide a
standardized way for youth and mentors to discuss GPS skills and the Five Cs of PYD and
Contribution. Growth grids were also designed to give mentors a snapshot on how youth in their
programs are doing, and what their youth’s goal-management skills look like. The growth grids
were also designed to help mentors assess skill development. This tool enables mentors to assess
how well youth in a program have benefitted from their involvement with mentors. The growth
grids, however, are not just measurement tools. They can also serve as powerful motivators for
change in youth (Andrade, 2000; Goodrich,1996; Marzano & Haystead, 2008; Moskal, 2003;
Popham, 1997; Stiggins, 2001). An essential feature of Project GPS is having both mentorscored growth grids, in which the mentors assess the youth, as well as youth-scored growth grids,
in which the youth do a self-assessment. With the growth grids as a guide, the mentor and youth
can compare their assessments of the youth’s GPS Skills and PYD – and discuss where they
share opinions or where they differ. Youth can see where their greatest strengths lie, and where
their biggest challenges exist as they move on a path towards thriving.
The growth grids break down the different aspects and skills of GPS and PYD to help both youth
and their mentors reflect on the youth's strengths and areas for improvement by providing a
standard of performance needed to attain a specific score. Each of the growth grids shares the
same “1 to 5” scoring scale. The youth moves up in the scoring scale as they improve along two
axes: skill initiative and skill competence. In other words, youth need to have both the initiative
to try to use a skill and the competence to implement that skill effectively.
Over the course of the project, several instantiations of the growth grids had been constructed
and the merits and problems with each had been discussed within the IARYD team, and with the
Thrive team and other youth-serving professionals. Initial reviews with youth-serving
professionals and the Thrive team resulted in a pilot set of growth grids. The growth grids were
then piloted with a sample of mentors and youth from Oregon and North Carolina. After the
conclusion of the pilot, these rubrics were revised by the joint Thrive-IARYD team, based on the
empirical findings IARYD obtained and reports that Thrive received from the field. This revised
set of rubrics was included in the Evaluation of the Project GPS toolkit. As part of the overall
evaluation, the IARYD team also conducted qualitative analyses of rubrics that were revised and
annotated by “highly involved” Project GPS evaluation participants.
These responses were included in a final round of growth grid revisions that were performed by
the Thrive team in the Winter and Spring of 2012. This set of growth grids has been
implemented in Thrive-affiliated programs in the Summer and Fall of 2012. We note here that
these growth grids were not revised on the basis of research by the IARYD team.
Goal 3. Videotaped exemplars
Exemplars were models of successful use of GPS skills to foster the Five Cs of PYD and
Contribution. The IARYD team recruited youth from several youth-serving organizations with
which team members and Tufts students and colleagues had affiliations.
5
After we identified promising youth to serve as exemplars, we pre-interviewed these possible
exemplars by phone or in person. This pre-interview process helped to ensure that the youth
would have experiences and stories that are a good match for the project, and that they could
describe those experiences in a lively and detailed way. After reviewing these pre-interviews
with Dr. Cheryl K. Olson, the supervisor of the video exemplar development, we conducted and
videotaped more detailed interviews with twelve young people. These interviews were edited
and appropriate B-roll and stock footage were added to create a final set of videos for review by
the Thrive team. This review resulted in a final set of 11 videos of exemplars; these individuals
collectively exemplified the 13 GPS skills as well as the Five Cs of PYD and Contribution.
In addition to the videos, the IARYD-Thrive team also created a set of discussion guides to
accompany each video. The guides were designed to help youth identify how these exemplars
use GPS skills to achieve different types of goals, grow in the Five Cs of PYD, contribute to their
communities, and use the same skills to achieve goals.
Goal 4. Evaluation of the Overall Toolkit
The Project GPS Evaluation involved recruiting and training youth-serving professionals around
the country to use the Project GPS materials. Conducting an evaluation of a new set of materials
with a variety of programs around the country led us to make several decisions about the design
of the evaluation.
First, we decided to present the GPS materials as additional resources that groups and mentors
could use to complement the work that they were already doing. This decision was made in
regard to the multitude of models used by programs and the multitude of different ways sites
implement those models. However, we required that, at a minimum, programs and mentors had
to conduct the introductory activities with youth, so that the young people would be able to
complete the growth grids accurately.
We also conducted the evaluation under the assumption that the variety of programs we recruited
would use the materials in different ways. The key point was that we collected details about how
the programs and mentors used the materials.
The design of the study was meant to examine 1. how different youth-serving organizations use
the Project GPS tools in their programs; and 2. whether differences in the use of Project GPS
tools affect ISR development in youth. The goal of the evaluation was to revise the set of
Version 1.0 Project GPS tools in order to create a Version 2.0 set of tools.
Given the scholarly goals of the Project, these results will be submitted to peer-reviewed
journals. To ensure the highest scientific quality of these products, we have undertaken complex
analyses that have required a team effort and substantial time. As indicated in section A.iv below,
these publications are currently being prepared for submission over the last quarter of 2012 and
the first two quarters of 2013.
6
Participant Sample
Of the 26 program sites that began the Project GPS process, only two programs did not continue
through the entirety of the study. Twenty four sites representing 15 nonprofits were in the final
participant pool. From February, 2011 to January, 2012, we analyzed 415 mentor-youth pairs
from these 24 different sites across the United States. That is, there were 415 dyads (pairs) of
mentors and youth, although many mentors had more than one mentee in the study. In all, 115
mentors participated in the study, along with 415 of their mentees. Of those 415 mentor-youth
pairs, 86% of mentors (357) and 85.3% of mentees (354) participated in all three waves of data
collection. 6.5% of mentors (27) and 10% of youth (41) participated in two waves of data
collection. Finally, 7.5% of mentors (21) and 4.8% of youth (20) participated in one wave of data
collection.
Of the 415 youth who participated in the study, 51% were male, and 49% were female. The
average age of participants was 14.14 years old at Wave One (SD = 2.26). Participants’ grade in
school ranged from “5th Grade” to “12th Grade or higher.” Of these 415 participants, 0.3% of
participants were in 5th Grade, 27% of participants were in 6th Grade, 22% of participants were in
7th Grade, 18% of participants were in 8th Grade, 8% of participants were in 9th Grade, 8% of
participants were in 10th Grade, 9% of participants were in 11th Grade, and 8% of participants
were in 12th Grade or higher.
Figure 1. Histogram of Grade in School of Participating Youth
The sample was diverse with regard to race/ethnicity. Of 415 participants, self-reported
race/ethnicity was 7% Asian/Pacific Islander, 28% Black/African American, 32%
Hispanic/Latino, 22% White/Caucasian, 1% Native American, 7% Multiethnic/Multiracial, and
3% Other.
7
Figure 2. Race/Ethnicity of Participating Youth
As a proxy for socio-economic status, we asked participants to report the highest level of
education their mother had completed at the time of the survey. Accordingly, 21% (n = 75
participants) reported that they did not know their mother’s level of education, and 15% (n = 62
participants) did not reply. Of the 278 participants who reported that they knew their mother’s
level of education, 10% had “8th Grade Education or Less,” 10% had “Some High School,” 18%
had “High School Diploma/G.E.D.,” 18% had “Some College,” 33% had “A College Degree,”
and 11% had “A Master’s Degree or Higher.”
At the start of the study, 333 mentors reported on the length of their relationships with each
youth. Of these 333 mentors, 3.6% (n = 12) reported having just met their mentee; 22.8% (n =
76) reported having known their mentee a few weeks; 30.9% (n = 103) reported having known
their mentee for several months; 20.1% (n = 67) reported having known their mentee about a
year; and 22.5% (n = 75) reported having known their mentee for several years. Based on the
prevailing view in the field of mentoring, these results indicate that over almost 75% of our
mentor sample had known their mentees long enough to accurately report relationships
characteristics and youth attributes.
In terms of program dosage, the frequency of mentor-reported contact with youth in general was
stable across the three times points. However, this stability does not preclude individual mentormentee relationships changing in frequency of contact. At Time 1, 1.2% (n = 4) reported seeing
their mentee less than one time per month; 10.2% (n = 34) reported seeing their mentees one
time per month; 10.8% (n = 36) reported seeing their mentee several times per month; 24.7% (n
= 82) reported seeing their mentee once per week; and more than half of the sample 53.0% (n =
176) reported seeing their mentee several times per week or more.
Groups varied in the length of time between points of measurement in Project GPS. Seventy-nine
percent (n = 328) of mentor-youth pairs completed their three times of measurements in less than
three months, and 21% (n = 87) of mentor youth pairs completed their three times of
measurements in more than three months. On average, as seen in the graph below, participants
8
completed the three surveys over the course of three months. Mentor-mentee pairs within each
youth-serving group completed the surveys within a two week window during each wave of data
collection, which accounted for small variations in time-lag within groups (0-14 days).
Figure 3. Time of participation in Project GPS for each mentor-mentee pair
Groups also varied in their mentoring practices, such that some mentors met with their youth
individually, whereas others met in groups. In our study, 86% of mentors met with their youth in
groups, and 14% met with their youth individually. Participants in group-mentoring programs
varied greatly in the number of participants in their group, ranging from 1 to 15 participants per
mentor. Tenacity, a mentoring program which had 23 mentor participants, had an average of 7.7
participants per group represented in our study, with a median of seven participants per group. In
turn, 38 other group mentors who participated in our study had smaller groups, ranging from 112 with an average of 3.3 participating mentees per mentor. The median and mode for group size
of non-Tenacity groups was 1.
Data Analyses
We will present the findings from the evaluation is several sections. The first section will review
the factor structure of the GPS and PYD rubrics as indicated in the pilot testing and in the
evaluation. The second section will present the responses of mentors and mentees to the utility of
the Project GPS materials. Finally, the third section will present the mean levels scores of youth
on the GPS and PYD rubrics. Please note that the second and third sections are presented for the
whole sample of participants, for the participants as differentiated by length of their use of the
GPS materials (less than three months versus more than three months), and for the type of
program structure (one-on-one versus group mentoring).
9
A. The Structure and Validity of the GPS and PYD Rubrics
i. Pilot
Our pilot study was conducted in the Fall of 2010 with groups in North Carolina and Oregon.
There were 152 unique mentor/mentee pairs that participated in the pilot. Of these groups, 69 of
these pairs included youth older than 14 years of age, while 83 included youth younger than 14
years of age. The results indicated that the GPS, Five Cs of PYD, and Contribution rubrics were
reliable measures, and that scores on these measures predicted scores on youth-reported
questionnaire items pertaining to Selection, Optimization, and Compensation, and to Positive
Youth Development, respectively. Mentors rated youth in a more reliable manner, that is, with
high levels of consistency, while youth displayed greater variability (as expected) in their
responses.
Tables 1- 4. Reliabilities of Growth Grids in Pilot Sample
Table 1. Older youth, mentor-scored rubric
Rubric Item
G
P
S
Competence
Confidence
Character
Caring
Connection
Contribution
Cronbach’s alpha
.88
.91
.86
.85
.87
.91
.90
.76
.90
Table 2. Older youth, self-scored Rubric
Rubric Item
G
P
S
Competence
Confidence
Character
Caring
Connection
Contribution
Cronbach’s alpha
.65
.80
.52
.42
.62
.56
.43
.45
.78
10
Table 3. Younger Youth, Mentor Scored Rubrics
Rubric Item
GPS
Competence
Confidence
Character
Caring
Connection
Contribution
Cronbach’s alpha
.92
.85
.86
.86
.85
.76
.86
Table 4. Younger youth, self-scored Rubrics
Rubric Item
GPS
Competence
Confidence
Character
Caring
Connection
Contribution
Cronbach’s alpha
.63
.66
.73
.80
.49
.45
.63
Mentor and youth factors for GPS and the Five Cs of PYD and Contribution were, however, each
significantly positively correlated. That is, each of the latent G, P, S, (or GPS) factors were
positively correlated with each of the Cs of PYD and Contribution factors. The strength of these
correlations varied across rater, age group, and factor pairing. These pilot findings suggested that
the rubrics were suitable for larger-scale evaluation. For older youth, most of the correlations
were significant; only a few correlations between these scales were not significant.
For younger youth, scores on the GPS scale correlated with all Five Cs of PYD and Contribution
scale scores for both mentor and youth rated pilot data.
The full set of the correlations within each set of rubrics are presented in Appendix A.
ii. Evaluation – Time 1
We chose to model the Time 1 Project GPS data using the Confirmatory Factor Analysis (CFA)
framework, rather than the related Exploratory Factor Analysis (EFA) framework. There are
several reasons why we chose to use CFA. First, given the findings from the 4-H Study of PYD,
we expected that younger youth would have a single-factor GPS structure, while older youth
would have three discrete G, P, and S factors. We also expected that data from both age groups
would fit a structure of the Five Cs of PYD and Contribution. CFA allows for the testing of these
hypotheses, while EFA does not. In addition, CFA allows us to test for invariance across time,
that is, the procedure allows us to assess if the structure of GPS or PYD changes or remains
constant. Finally, CFA allows us to better understand the relationship among factors; for
instance, we can test whether Shifting Gears is predictive of levels of Confidence.
11
In general, the CFAs illustrated that the Time 1 Project GPS data fit models that were reasonably
similar to our theory and research-based hypotheses. However, there were several notable
differences in each model. Below, we present a brief review of each of the four models: 1. Older
youth, self-scored; 2. Older youth, mentor-scored; 3. Younger youth, self-scored; and 4. Younger
youth, mentor-scored. Technical data regarding model fit, item loadings, and factor correlations
are presented in Appendix B.
Older youth, self-scored
As shown in the table below, there were a total of 215 older youth participants (51.8% of
particpants), ranging from 14-21 years of age. Among older youth, 57 were 14 years old
(26.4%), 42 were 15 years old (19.9%), 43 were 16 years old (20.5%), 33 were 17 years old
(15.8%), and 16 were 19-21 years old (7.5%).
Older youth, self-scored data fit a model in which each of the Five Cs of PYD and Contribution
can be discretely modeled as separate ‘C’ factors. In terms of GPS, S could be modeled as a
distinct factor, while G and P were best modeled as a single factor. This finding means that, for
this age group, it is more accurate to use the G and P rubrics to measure a combined factor
representing elements of both concepts, rather than trying to measure G or P separately. The G/P
factor and the S factor were significantly correlated with each PYD factor, with correlations
ranging from .59 for G/P with Caring to .93 for S to Contribution.
Older youth, mentor-scored
Older youth, mentor-scored data fit a model that is identical to the older youth, self-scored data.
This finding means that each of the Cs of PYD and S can be discretely modeled as separate
factors, while G and P were best modeled as a single factor. Again, this finding means that it is
more accurate to use the G and P rubrics to measure a combined factor representing elements of
both concepts, rather than use G or P separately. G/P and S correlated significantly and strongly
with each PYD factor, ranging from .74 for G/P to Character to .95 for S to Connection.
Since youth-scored and mentor-scored versions of the older youth rubrics had the same factor
structure at Time 1, scores across raters of any of these rubrics can be compared. This
comparison may be useful for organizations seeking to understand if there are variations in how
youth and mentors rate youth behavior. In addition, organizations that wish to focus on one or a
few of the PYD Cs may do so (since each of these factors is distinct); however, if these
organizations want to understand how GPS is related to their PYD focus, the organization should
use each of the G, P, and S rubrics, and, when modeling the data, combine the G and P rubrics to
represent the factor we discussed above.
From a modeling and measurement perspective, the G/P and S factors cannot be combined into a
single factor. The goal of Confirmatory Factor Analysis (CFA) is to provide a best-fitting, mostparsimonious model that tests the extent to which latent, underlying factors (in this case, an
individual’s ability to Shift Gears, for instance) predict an individual’s scores on items designed
to test that underlying factor (in this case, the “S” columns).
12
Our original theoretical model included distinct G, P, and S factors for older youth. CFA
analyses indicated that the G and P latent factors were correlated at nearly 1.0, suggesting that
they may be manifestations of an underlying GP factor. We wanted to test this possibility. In
order to do so, we combined the data into a single GP factor and tested whether this new model
had statistically significant poorer fit that the prior model. It did not, and therefore, we retained
the GP factor.
Following this step, the resulting GP factor and the S factor were also strongly correlated. We
considered that a single GPS latent factor may be predicting scores on the GP and S factors.
Therefore, we combined all factors into a single GPS factor, and once more tested to see if the
resulting model demonstrated significantly worse fit. In this case, the new model did have worse
fit, and therefore, we could not recommend combining the factors into a single GPS factor.
We recognize that this two-factor solution is less parsimonious and less user friendly than a
single factor solution. While our analyses indicate modeling older youth data using a single GPS
factor is a significantly worse model than the stipulated two-factor approach, the correlations
among these two factors are still high. This finding may inform future research with refined
tools, which, given our iterative design process for the rubrics, may result in a parsimonious
single factor GPS model.
Younger youth, self-scored
There were a total of 200 younger youth participants, ranging from 10-13 years old. Of younger
youth, one was 10 years old (0.4%), 34 were 11 years old (17%), 83 were 12 years old (41.5%),
and 82 were 13 years old (41.1%).
The younger youth data did not have the same factor structure across raters. In addition, these
factor structures did not match those of the older youth. For younger, self-scored youth data,
GPS was measurable as a single factor, as predicted. PYD did not differentiate into Five Cs and
Contribution, as it had in older youth data. Instead, the Cs collapsed into three factors, each
representing new factors comprised of two Cs each: competence/confidence, character/caring,
and contribution/connection. These pairs are substantively logical and are consistent with high
correlations found among these factors in the 4-H Study (Bowers, et al., 2010). The GPS factor
correlated significantly and strongly with each of the three PYD factors (range = .77 to .81).
These findings may influence the way organizations use the rubrics for younger youth. Since, for
instance, character and caring are represented as a single factor, and organizations that wish to
focus on improving the way youth rate their character should, at a minimum, assess the items
associated with both character and caring. For younger youth at Time 1, the character items
potentially represent a “prosocial behaviors” factor that also is informed by their responses on
caring items. Each of the combined C factors can be interpreted as theoretically meaningful,
rather than simply measurement artifacts. For instance, younger youth confidence and
competence are likely closely related across a variety of domains, as youth begin to explore their
skills, talents, and abilities.
13
Younger youth, mentor-scored
The younger youth, mentor-scored data followed a modeling pattern similar to younger youth,
self-scored data. GPS was again measurable as a single factor, and PYD again did not
differentiate into Five distinct Cs and Contribution. However, in these data, there were two
factors comprised of two Cs each – connection/competence, confidence/contribution – and two
distinct factors for the remaining Cs: caring and character. The single GPS factor was
significantly and strongly correlated to each of these 4 factors (range = .74 to .81).
These findings indicate that mentors also do not fully differentiate PYD in younger youth into
Five Cs and Contribution. Because mentors and youth combine different Cs into new factors, it
will be difficult for organizations – if these Time 1 patterns continue – to compare the PYD
scores of youth between raters. Regardless of the findings of future waves, organizations should
collect as much data as is manageable for their organization, and then analyze and interpret the
findings for younger youth with caution.
There were three general trends in the modeling of Time 1 Project GPS data:
1. Older youth, whether mentor or youth scored, had Five Cs of PYD and Contribution
factors, but a G/P and an S factor.
2. Younger youth, whether mentor or youth scored, lacked clear differentiation among
the Five Cs of PYD and Contribution, but had one GPS factor.
3. Youth scores indicated more variability in response patterns, while mentor response
patterns were general more uniform in nature.
iii.
Invariance of GPS Growth Grids
An important step in assessing development through latent variable data analytical procedures
is invariance testing. In order to compare latent factors across time, factors must be equivalent
in several ways. If the factors are not invariant across time, then the longitudinal relations
among these factors do not represent the development of a single latent characteristic of an
individual (in this case, the development of GPS skills), but rather the longitudinal relations of
three separate factors. Since we are interested in assessing the development of GPS, we
conducted invariance tests on the data.
There are three levels of invariance that are typically required for longitudinal assessment.
Each of these three levels is a progressively more restrictive. The first level is termed
configural invariance. Here, the pattern of freed and fixed parameters is set to be equivalent
across groups. This may also be termed a longitudinal CFA. The second level is termed weak
invariance. Here, the factor loadings on the items are equated across time. Finally, the third
standard level of invariance testing is termed strong invariance. Here, the means of each
indicator are equated across time.
The invariance tests we conducted on the GPS data are presented below in four samples:
younger youth, self-scored, and younger youth, mentor-scored, older youth, self-scored, and
older youth, mentor-scored. We tested invariance using the criterion suggested by Cheung and
Rensvold (2002; i.e., Δ CFI < .01 for each level of invariance). The models testing configural,
14
weak, and strong invariance displayed acceptable model fit in all four samples. Tables 5 to 8
present these findings, in addition to the each model’s fit, based on the alternative null model
approach (e.g., Widaman, 2003). The results indicate that the pattern of fixed and free
parameters, factor loadings, and indicator means can be adequately measured as being
equivalent across time within scoring group. Thus, in future analyses (such as, perhaps,
those involving the relations between GPS and PYD), we can accurately ask how variations in
the GPS factors, over time, can influence outcomes, given that the factors themselves can be
modeled as equivalent along the above-stated parameters. Technical data regarding item
loadings and factor correlations are presented in Appendix B.
Tables 5 – 8. Model Fit Indices for Tests of Invariance
Table 5. Younger youth, Self-scored Invariance Testing and Model Fit
Step
Invariance/ Normal
equality
Theory
description test
Chi sqr
alternative
0 Longitudinal null
Min. Fit
Function
Chi sqr
df
1282.234 1282.234
CFI
Change
CFI
NNFI
Change
Chi-sqr
Change
df
p
201
Measurement invariance
1 Longitudinal configural
145.516
145.516
114
0.971
0.949
2 Longitudinal weak
153.521
153.521
124
0.973
0.956
-0.002
8.005
10
0.6283
3 Longitudinal strong
186.313
186.313
158
0.974
0.967
-0.001
32.792
34
0.5267
Table 6. Younger youth, Mentor-scored Invariance Testing and Model Fit
Step
description
0 Longitudinal
Invariance/ Normal
equality
Theory
test
Chi sqr
alternative
null
Min. Fit
Function
Chi sqr
df
18205.61 18205.61
CFI
NNFI
Change
CFI
Change
Chi-sqr
Change
df
p
201
Measurement invariance
1 Longitudinal
configural
147.4
147.4
114
0.998
0.997
2 Longitudinal
weak
155.084
155.084
124
0.998
0.997
0.000
7.684
10
0.6597
3 Longitudinal
strong
184.675
184.675
158
0.999
0.998
0.000
29.591
34
0.6836
Table 7. Older youth, Self-scored Invariance Testing and Model Fit
Step
Invariance/ Normal
equality
Theory
description test
Chi sqr
alternative
0 Longitudinal null
Min. Fit
Function
Chi sqr
df
6105.958 6105.958
CFI
NNFI
Change
CFI
Change
Chi-sqr
Change
df
p
836
Measurement invariance
1 Longitudinal configural
806.864
806.864
648
0.970
0.961
2 Longitudinal weak
816.913
816.913
670
0.972
0.965
-0.002
10.049
22
0.9859
3 Longitudinal strong
907.932
907.932
719
0.964
0.958
0.008
91.019
49
0.0003
15
Table 8. Older youth, Mentor-scored Invariance Testing and Model Fit
Step
Invariance/ Normal
equality
Theory
description test
Chi sqr
alternative
0 Longitudinal null
Min. Fit
Function
Chi sqr
df
40348.52 40348.52
CFI
NNFI
Change
CFI
Change
Chi-sqr
Change
df
p
832
Measurement invariance
iv.
1 Longitudinal configural
769.162
769.162
648
0.997
0.996
2 Longitudinal weak
857.714
857.714
670
0.995
0.994
0.002
88.552
22
0.0000
3 Longitudinal strong
920.038
920.038
709
0.995
0.994
0.001
62.324
39
0.0102
Future directions for publications based on the Project GPS Growth Grid Data
The papers below are expected to be completed and submitted over the final quarter of 2012
and the first six months of 2013.
1. Theoretical Paper – Foundations of GPS
2. Empirical paper 1 – Invariance of GPS and PYD rubrics for youth across three waves
3. Empirical paper 2 – Relations between mentor and youth rated GPS across three waves.
4. Empirical paper 3 – Time lag as a moderator of relation between GPS and PYD
4.Empirical paper 4 – Mentoring relationship closeness X Ability to mentor predicting youth
GPS, PYD, Goal attainment
5. Empirical paper 4 – Structural vs. process measures in predicting youth outcomes
6. Empirical paper 5 – Domains and success of goal youth sets -- relations to changes in GPS
and Cs of PYD
7. Empirical paper 6 – Implementation and usefulness of tools to changes in GPS and goal
attainment
16
B. Youth and Mentor Responses to the Utility of GPS Materials
Not all participants answered each question at each wave. Participants were instructed not to
answer questions if they did not apply to their program. In addition, the GPS Project did not
mandate that participants make use of all of the resources provided to them, so the number of
responses varied greatly depending on the question. The responses at Times 1, 2, and 3 are not
always from the same sample of youth or mentors; the responses reflect the percentage of youth
or mentors who answered that question at that time point. However, as indicated earlier, a large
proportion of youth and mentors participated at all three times.
We provide the results of responses to several questions that index the usability of the growth
grids based on: 1. The whole sample; 2. Short-term (less than three months total) versus longterm (more than three months total) use of the GPS materials; and 3. One-on-one versus group
mentoring structure. Long and short duration were defined by the time lag between Time 1 of
measurement, and Time 3 of measurement for each mentor-mentee pair.
The mean time lag for all participants was 0.26 years (approximately 95 days). We therefore
established a cutoff at 0.30 years (110 days), in which participants whose time lag was more than
110 days were considered “Long Time Lag,” and participants whose time lag was less than 110
days were considered “Short Time Lag.” The average lag-time across groups was 0.26 years (95
days, or 47.5 days between each wave of measurement). As such, 290 mentor-mentee pairs were
in the Short Time-lag group, and averaged 0.22 years total time-lag (80 days, or 40 days between
each wave of measurement). In turn, 78 mentor-mentee pairs were in the Long Time-lag group,
and averaged 0.42 years total time-lag (154 days, or 77 days between each wave of
measurement).
1. Introduction of the Growth Grids
Of the 309 mentor-mentee pairs in which the mentor responded to this item, 28.5% (n = 88) used
both Session A and Session B from the GPS manual; 11.3% (n = 35) used only Session A from
the manual; 3.2% (n = 10) used only Session B from the manual; 31.7% introduced the GPS
skills and rubrics with their own activities; and 25.2% (n = 78) reported not introducing the GPS
skills and rubrics to the youth. Follow-up analyses will determine whether these differences were
a result of the hierarchical structure present in some programs in which one associate presented
curriculum while youth had their own mentors separate from the curriculum.
17
2. Use of GPS Language at Meetings
A key measure of the degree of Project GPS implementation would be a reported increase in the
use of GPS language and ideas. At Time 2, we asked mentors how often they brought up GPS
with youth during meetings and 71.0% of mentors at the second time of measurement said they
brought up GPS skills either “rarely” or “never” with their youth. However, those who
participated over a long duration were more likely to have brought up GPS. For example, 26.1%
of mentors with a short duration brought up GPS “sometimes,” “often,” or “always,” as
compared with 42.5% of long-duration mentors. These differences can also be seen between
mentor-youth pairs that were individual, as opposed to group mentoring programs. That is,
54.8% of individual mentors-youth pairs reported that they brought up GPS “sometimes,”
“often,” or “always,” as compared with 22.9% of group-mentors.
Time 2 Responses
Figure 4. All mentors reported use of GPS language
n = 276
18
Short versus longer term use of materials
Figure 5. Mentors in programs using materials for less than three months reported use of GPS
language
n = 234
Figure 6. Mentors in programs using materials for more than three months reported use of GPS
language
n = 40
19
One-on-One versus Group mentoring
Figure 7. Mentors in one-on-one mentoring programs reported use of GPS language
n = 31
Figure 8. Mentors in group mentoring programs reported use of GPS language
n = 227
20
Time 3 Responses
At Time 3, 51.2% of mentors at the third time of measurement said they brought up GPS skills
either “rarely” or “never” with their youth. However, again, those who participated over a long
duration were more likely to have brought up GPS. For example, 46.3% of mentors with a short
duration brought up GPS “sometimes”, “often” or “always”, as compared with 62.2% of longduration mentors. These differences can again be seen between mentor-youth pairs that were
individual, as opposed to group mentoring programs. 74.4% of individual mentors-youth pairs
reported that they brought up GPS “sometimes”, “often” or “always”, as compared with 42.7%
of group-mentors. On the whole, 48.8% of mentor-youth pairs at Time 3 reported that they
brought up GPS “sometimes”, “often” or “always”, as opposed to only 29.0% at Time 2.
Figure 9. All mentors reported use of GPS language
n = 291
21
Short versus longer term use of materials
Figure 10. Mentors in programs using materials for less than three months reported use of GPS
language
n = 246
Figure 11. Mentors in programs using materials for more than three months reported use of GPS
language
n = 45
22
One-on-One versus Group mentoring
Figure 12. Mentors in one-on-one mentoring programs reported use of GPS language
n = 43
Figure 13. Mentors in group mentoring programs reported use of GPS language
n = 218
23
3. Usefulness of Rubrics
At the conclusion of the Project, we asked youth if the rubrics were helpful in achieving their
goals. We found that 73.6% of youth reported that the rubrics were either “Somewhat Useful,”
“Useful,” or “Very Useful” in achieving their goals. This distribution differed slightly between
groups who used the rubrics for more than three months (63.5%) and groups who used the
rubrics for less than three months (75.7%). Youth in one-on-one mentoring pairs were more
likely to report the rubrics were “Useful” or “Very Useful” in helping them to achieve their goals
(56.4%) than youth in group mentoring programs (43.9%).
Figure 14. All youth reported usefulness of GPS rubrics
n = 301
24
Short versus longer term use of materials
Figure 15. Youth in programs using materials for less than three months reported usefulness of
GPS rubrics
n = 235
Figure 16. Youth in programs using materials for more than three months reported usefulness of
GPS rubrics
n = 63
25
One-on-One versus Group mentoring
Figure 17. Youth in one-on-one mentoring programs reported usefulness of GPS rubrics
n = 39
Figure 18. Youth in group mentoring programs reported usefulness of GPS rubrics
n = 255
26
We also asked mentors if they thought the rubrics were useful at the conclusion of the project,
and 40.2% of mentors reported that the rubrics were either “Somewhat Useful,” “Very Useful,”
or “Extremely Useful.” This distribution differed noticeably between groups who used the
rubrics for more than three months (59.4%) and groups who used the rubrics for less than three
months (37.6%). Mentors of individuals were more likely to report that the rubrics were either
“Somewhat Useful,” “Very Useful,” or “Extremely Useful” (50.0%) as compared with mentors
in group-mentoring programs (38.7%).
Figure 19. All mentors reported usefulness of GPS rubrics
n = 268
27
Short versus longer term use of materials
Figure 20. Mentors in programs using materials for less than three months reported usefulness of
GPS rubrics
n = 236
Figure 21. Mentors in programs using materials for more than three months reported usefulness
of GPS rubrics
n = 32
28
One-on-One versus Group mentoring
Figure 22. Mentors in one-on-one mentoring programs reported usefulness of GPS rubrics
n = 30
Figure 23. Mentors in group mentoring programs reported usefulness of GPS rubrics
n = 235
29
The significant percentage of mentors reporting that the rubrics were not useful, and the
indication that the differences might be related to the structure of implementation, are also
apparent in the themes that emerged from our analysis of three open response items. While we
did not specifically ask mentors to report why they thought the rubrics were useful or not,
mentors were asked to provide additional thoughts about the materials, what they would change
about Project GPS, and what their favorite thing about Project GPS was.
For example, of the 30 respondents who gave additional thoughts about the materials:
o 26 % had negative comments on the rubrics, with responses stating that the
materials were redundant, the language difficult to follow, and that they were
lengthy.
 “Language of the rubrics is too advanced and abstract for our middle
schoolers”
 “The rubrics were redundant and time consuming for the students to
complete and they often complained. As a mentor, I found that the
wording of the rubrics did not always accurately describe the level that a
particular student was at in his/her development.”
o 17% expressed a desire to have more time with their mentees to integrate project
GPS
 “We would benefit from using these over a much longer period of time;
especially with a 13 year old.”
 “Need to conduct this activity over periods of longer duration. This
assessment was too cramped to provide any real benefit beyond getting an
orientation to some concepts.”
o 17% expressed difficulties understanding Project GPS or incorporating it into
their meetings with youth
 “With how (my program) is run, I wasn't able to use many of the activities
listed because they wouldn't be successful with our large class size and the
challenges we face with our students in terms of behavior management.”
Similarly, of the 91 participants that responded to the question “What would you change about
Project GPS?”
o 16% indicated that the survey was too long or redundant
 “The rubrics are redundant and very time-consuming. The students loose
interest in completing them and don't understand the significance of
completing them.
o 15% indicated that the language of the rubrics was too advanced for youth
 “Rubric language too complex for students”
In turn, 77 participants that responded to the question “What was your favorite thing about
Project GPS?”
o 8% chose to speak about how it made students self-reflect
 “It makes students think about themselves and what drives them”
 “Finding out what students thought about their own life skills, abilities and
self-confidence”
30
4.
Utility of the Activities
At the conclusion of the project, we also asked mentors if they thought the activities were useful.
Accordingly, 37.1% of mentors reported that the activities were either “Very Useful” or
“Extremely Useful.” This distribution differed noticeably between groups who used the activities
for more than three months (17.7%) and groups who used the activities for less than three
months (39.7%). Mentors of individuals were more likely to report that the activities were either
“Very Useful,” or “Extremely Useful” (43.8%), as compared with group program mentors
(35.2%).
Figure 24. All Mentors reported usefulness of GPS activities
n = 143
31
Short versus longer term use of materials
Figure 25. Mentors in programs using materials for less than three months reported usefulness of
GPS activities
n = 126
Figure 26. Mentors in programs using materials for more than three months reported usefulness
of GPS activities
n = 17
32
One-on-One versus Group Mentoring
Figure 27. Mentors in one-on-one mentoring programs reported usefulness of GPS activities
n = 16
Figure 28. Mentors in group mentoring programs reported usefulness of GPS activities
n = 125
33
We also asked mentors to what degree they found the activities engaging. In general, mentors
found the activities either “somewhat” (57.4%) or “very engaging” (32.8%). Mentors who used
the materials over a shorter duration were more likely to find them to be “very engaging” or
“extremely engaging” (46.5%), as compared mentors who used the activities over a longer
period (11.1%). A greater proportion of mentors in group mentoring programs reported that the
activities were “very engaging” or “extremely engaging” (39.5%) than mentors in individual
relationships (23.5%)
Figure 29. All Mentors reported engagement level of GPS activities
n = 61
34
Short versus longer-term use of materials
Figure 30. Mentors in programs using materials for less than three months reported engagement
level of GPS activities
n = 43
Figure 31. Mentors in programs using materials for more than three months reported engagement
level of GPS activities
n = 18
35
One-on-One versus Group Mentoring
Figure 32. Mentors in one-on-one mentoring programs reported engagement level of GPS
activities
n = 17
Figure 32. Mentors in group mentoring programs reported engagement level of GPS activities
n = 43
36
Qualitative responses confirmed the generally positive reviews of the activities. For example, of
the 77 participants that responded to the question, “What is your favorite thing about Project
GPS?”
o 26% had positive comments on the activities.
 “The activities, resources, ideas that help us to think about how to work
with our students on their goals.”
 “The activities I chose ended up working very well and were engaging for
the students. They enjoyed doing something different in class and didn't
really notice they were even learning with some of them.”
o 17% mentioned that they liked the goal setting aspect of the program.
 “Learning about goal setting and realization strategies.”
5.
Utility of the Videos
Only 22 mentors responded to the items about the utility of the exemplar videos. While, the
responses were generally positive, we do not think that the data provided by this subsample of
mentors is representative of the general sample of mentors. Based on anecdotal feedback from
programs and mentors, we will seek to improve mentor access to the videos, as well as to more
fully integrate the videos into the Project GPS materials.
6.
Completion of Goals
In addition to impacting youth GPS skills and measuring the change in those skills, another
important outcome indexed by Project GPS was whether youth reported higher rates of
successful goal attainment at each point of data collection. We asked youth at Time 1 of
measurement if they had completed any goals within the last three months. We then asked the
youth at Time 2 and Time 3 if they achieved any goals that they had set for themselves since the
last time they were surveyed.
Only 27.7% of youth reported having completed a goal within three months prior to Time 1 of
measurement. In contrast, 68.3% of youth accomplished a goal they had set by Time 2, and
62.2% accomplished a goal between Time 2 and Time 3. Responses to this question differed at
Times 2 and 3 between groups who used the rubrics for more than three months (73.1% at Time
2, and 70.9% at Time 3) and groups who used the rubrics for less than three months (67.5% at
Time 2, and 60.7% at Time 3).
There were also some very slight differences between those who had individual-mentors and
group mentors. Individual mentors reported slightly higher goal achievement at all three times of
measurement (40.9% at Time 1, 72.4% at Time 2, and 66.7% at Time 3) for individualmentoring pairs, as opposed to (26.8% at Time 1, 68.3% at Time 2, and 62.5% at Time 3) for
group-mentoring pairs.
We asked mentors parallel questions at Time 2 and Time 3 about whether youth had achieved
any goals that they had set for themselves since the last time they were surveyed. Mentors
37
reported that 66.7% of youth accomplished a goal they had set by Time 2, and 64.3%
accomplished a goal between Time 2 and Time 3. Responses to this question differed at Times 2
and 3 between groups who used the rubrics for more than three months (93.2% at Time 2, and
86.7% at Time 3) and groups who used the rubrics for less than three months (60.0% at Time 2,
and 58.5% at Time 3). There were also greater differences in goal achievement reported by
mentors in one-on-one versus group mentoring relationships than those reported by youth.
Individual mentors reported higher goal achievement at both times of measurement (76.4% at
Time 2, and 82.6% at Time 3) for individual-mentoring pairs, as opposed to (70.9% at Time 2,
and 64.0% at Time 3) for group-mentoring pairs.
C. Mean Levels of GPS and PYD at Times 1, 2, and 3
On average and at each time point, all youth perceived their GPS skills, Five C attributes, and
Contribution as higher than their mentors perceived them. Appendix C presents the means and
standard deviations of GPS, the Five Cs, and Contribution at Times 1, 2, and 3.
As seen in the appendix, on average, both the older and younger youth reported an increasing
ability and desire to learn the GPS skills, but that they sometimes need their mentor's help to
learn how to use each skill (i.e., about half the time). Older youth reported a significant increase
in G and S skills over the course of the project, but they did not report a significant increase in P
skills. Younger youth reported a significant increase in GPS skills over the course of the project.
Both the mentors of older and younger youth initially saw their mentees as motivated, but
needing a good deal of help to learn the GPS skills. The mentors of younger youth and older
youth also reported significant increases in youth motivation to use, and mastery of, the GPS,
and G, P, and S skills over the course of the project However, regardless of age, youth
consistently reported higher levels of GPS skills than their mentors at all three time points.
In terms of the Five Cs of PYD, youth on average indicated that they were able to show these
skills and attributes at least half the time at all three time measurement occasions, but they
needed the help of their mentors in certain situations. Mentors reported significant increases in
the youth Caring, Competence, Confidence, and, Connection over the course of the project, but
they did not report significant increases in youth Character. Youth also did not report significant
improvement in Character, but they also did not report significant changes in Caring or
Confidence in themselves that their mentors reported. Youth did, however, report significant
increases in Competence and Connection over the course of the project. Especially encouraging
was the significant growth and motivation both youth and mentors reported in youth desire and
ability to contribute back to their community. The youth reported being motivated to give back to
the community (Contribution), but they needed help from their mentors to figure out where, who,
and how to help. In general, the mentors and youth saw some consistent growth in the Cs, and
that the youth were motivated to grow in the Five Cs of PYD and Contribution; however, they
needed help from their mentors to learn how to build each skill and attribute.
38
Short versus longer-term use of materials
In general, the mean scores for GPS, the Five Cs of PYD, and Contribution were on a similar
upward trajectory for both short-and longer-term use pairs (see Appendix C). Across time points,
mentors in the longer term use pairs reported significantly higher youth G, P, and S scores than
mentors in shorter term use pairs. However, mentors of longer time-use youth did not report
significantly higher GPS scores than mentors of shorter time-use youth. Youth in the longer term
use group only reported significantly higher levels of Shifting Gears skills. The Five Cs of PYD
and Contribution were also consistently rated significantly higher by mentors in the longer use
group than those in the shorter use group. Youth in longer term groups rated themselves initially
higher on average in Confidence and Contribution. The mentors in the longer use group also
reported significant changes in Character, Competence, Confidence, and Contribution that were
not reported by mentors in the short-term use groups.
One-on-One versus Group Mentoring
Younger youth in individual mentor-youth pairs reported a slightly lower “Overall GPS” score
across the three time points than younger youth in group mentor-youth pairs, most notably at
Time 2 (a mean difference of .54). Mentors agreed, also reporting a significantly lower “Overall
GPS” score, with an average difference of .51 across all time points. The mentors in the group
mentoring programs also reported a significant increase in younger youth GPS skills over the
course of the project. Mentors of older youth in individual mentor-youth pairs also reported
significantly lower G, P, and Contribution scores across the three time points than youth in group
mentor-youth pairs, but mentors in both groups reported significant changes in G, P, and S skills
over the course of the project. General upward trends were similar between groups the Five Cs of
PYD and Contribution; however, group mentors reported significant increases in Caring,
Competence, Confidence, and Contribution that were not evident in mentors from one-on-one
mentoring pairs. This lack of significant differences in one-on-one pairs may be due to limited
power due to small sample sizes.
Age Differences in the Five Cs of PYD and Contribution Scores
There was very little difference between younger and older youth in self-reported Five Cs of
PYD and Contribution levels. However, mentors reported higher Five Cs scores in older youth,
indicating that they were more demonstrative of PYD and thriving indicators. This difference
increased as the study progressed, with older youth scoring significantly higher on the Five Cs
than their younger youth counterparts by Time 3. It is therefore possible that the GPS program
had had a larger impact on older youth. Mentor and youth reported Contribution did not vary
significantly by age of participants.
D. Time sensitivity of growth grids to detect change
A series of ordinary least squares regressions were conducted to examine the relationship
between changes in time and mentor-reported changes in G, P, S and GPS scores from Time 1 to
Time 3, controlling for the initial mentor-reported level of G, P, S, or GPS. Results indicated that
time lag was a significant predictor of changes in G, P, and S skills in older youth from Time 1 to
39
Time 3 (B = 1.442, B = 1.002, and B = 1.844, respectively). The longer the gap between Time 1
and Time 3 data collection points, the greater the reported increase in G, P, and S skills. There
were no significant changes due to time lag in GPS skills across any time frame. These results
indicated that on average, and controlling for initial reported levels of the skill, a one unit
increase on the G growth grid will take approximately 8.3 months, a one unit increase in P skills
will take almost a year, and a one unit increase on the S growth grid will take approximately 6.5
months.
Goal 5. Release of Versions 2.0 of Deliverables 1, 2, and 3, which together will constitute the
“Goal Management Playbook”
The feedback we have received over the course of the evaluation has led to changes in the
growth grids’ language and structure. These changes were made to better align the youth and
professional versions, better distinguish the skills from each other, and better define the
evidence that should be observed to report a particular level of a skill. Focus groups were used to
revise the rubrics that were used in the evaluation. They were also used to revise the “growth
grids” that are currently being used by Thrive grantees. The changes in the “growth grids” that
resulted from the second set of qualitative analyses from January, 2012 are available from
Edmond Bowers upon request.
As indicated, the latest iteration of the rubrics is currently being used in Thrive partnering
programs. We have also begun to move forward with revising and streamlining the mentor
manual and associated activities and videos in order to provide a more defined GPS Playbook.
We are drafting a follow-up proposal to examine the implementation, validity, and effects of this
revised suite of tools. This proposal will be submitted to several foundations and government
agencies.
Concluding Remarks
The IARYD team is deeply grateful to the Thrive Foundation for making this work possible. We
hope that we have made some significant contributions toward improving the work of youthserving professionals. However, we recognize that much more work has to be done to improve
the adoption and impact of these materials for a wider audience. Therefore, we promise to impact
academia by submitting several theoretical and empirical publications in the coming months. We
will also present this empirical work at conferences, both in the U.S. and internationally. A
complementary promise is to continue to work to impact the front-lines of youth development
work by continuing to collaborate with programs to refine and improve tools we have developed.