1
Response to Intervention Request for Proposals:
Recasting the Secondary School Classroom as a Context for Positive Youth Development
Principal Investigators:
Joseph P. Allen, Ph.D.
Robert C. Pianta, Ph.D.
University of Virginia
Proposed Dates of Funding:
March, 2007 – February, 2010
Proposed Costs:
$1,088,214 total direct costs across 3 years
($1,251,445 total direct + indirect costs across 3 years)
2
I.
Goals for Proposed Research: Major Questions
The goal of this project is to enhance the developmental quality of the single setting in
which youth under age 18 spend the greatest number of their waking hours: The secondary
school classroom. We propose to implement and assess an intervention designed to change the
nature of student-teacher interactions within the classroom so as to enhance adolescent
engagement, motivation, and ultimately, achievement and related outcomes. We will do so via a
novel intervention that combines recent research in principles of adolescent social development
and motivation, with a unique, web- and video-conferencing-based system for providing ongoing
training and support to teachers. We propose to accomplish three specific goals central to the
Foundation’s Request for Proposals: 1) development and evaluation of a model of support for
teachers to improve the capacity of secondary classroom settings to promote positive youth
development, 2) further validation and enhancement of an intervention-linked observational
assessment tool for gauging the capacity of secondary classroom settings to promote student
motivation, engagement, and achievement, and 3) identification of specific setting qualities that
are both subject to change via intervention and closely linked to subsequent student achievement,
so as to guide future research and intervention.
Our overarching premise is simple: For all of the attention paid to structural features of
schools and to the content of what they teach (e.g., see Felner, Favazza, Shim, & Brand, 2001;
Whitehurst, 2002), perhaps the single greatest mediator of adolescents’ academic outcomes—the
extent to which they are motivated and engaged by the interactions that take place between
themselves and their teachers within the classroom—has received relatively little attention. This
is true despite strong evidence that, from students’ perspectives, interactions with teachers are
frequently unsatisfying, unmotivating, and likely to lead to disengagement from schooling (e.g.,
Roeser, Eccles, & Sameroff, 2000). Our growing knowledge about adolescent social
development and about how to effectively support teachers and change their practices now
suggests several practical, effective, and theoretically-based approaches for dramatically
enhancing adolescent engagement and motivation in such interactions. This project explicitly
links recent theoretical perspectives on adolescent social development with teachers’
professional development, and with pragmatic approaches to altering student-teacher interactions
to make classrooms more valuable developmental settings for youth. In this respect, we are
testing theories related to mechanisms of change at two levels: 1) intensive and interactive
professional development as a context for changing teachers’ classroom interactions with
students, and 2) resulting changes in motivational supports and interactions as a mechanism for
enhancing classroom capacity to promote adolescent development.
We will evaluate a unique and effective approach to providing teachers with knowledge,
support, and feedback to alter both their interactions with students and the motivational supports
available in classroom settings. In turn we expect improved teacher-student interactions to alter
students’ motivation and engagement within the classroom, and ultimately, student achievement.
This proposal targets 80 relatively new teachers within a large, highly diverse school division.
Teachers with between 2 and 5 years of teaching experience will be assigned randomly to one of
two groups: Intervention and “business as usual” controls. Fifteen students will be assessed
within each classroom (Total N = 1200). Interventions will be implemented using a combination
of workshop and video-conference-based consultation methods in Year 1 of the intervention, and
all intervention teachers will receive a fall ‘booster’ workshop at the beginning of Year 2.
Intervention fidelity will be closely monitored using regular classroom observations collected via
3
videotape. Regular assessments of student achievement, motivation, attitudes and perceptions of
classroom experiences will also be obtained. Analyses will examine effects of the intervention on
classroom processes, teacher behaviors, and student outcomes. Analyses of longitudinal patterns
of student outcome change will be examined both directly and in relation to mediating
mechanisms of motivation and student-teacher interactions in classroom settings, which are
hypothesized to be significantly enhanced in the group receiving the intervention. Analyses will
also examine moderation of intervention effects as a consequence of teacher characteristics
(training, attitudes, experience), fidelity of implementation, and student characteristics (e.g.,
demographic characteristics, baseline performance).
Because this pragmatic intervention is based on state-of-the-art social developmental
theory, we are not only testing effects of an intervention for solving problems related to the
underperformance of a key setting for youth, we are also testing a theory about the mechanisms
by which that setting affects development and a theory about the mechanisms needed to bring
about change within classrooms (Pianta, 2006a).
II.
Theoretical and Empirical Rationale
The conceptual justification for this project rests on three primary arguments: 1) that
school settings, particularly classrooms and the teacher-student interactions within them, are key
potential assets to youth and that these classrooms are currently underperforming in this role, 2)
that the setting-level mechanism of students’ social and instructional interactions with teachers
can either produce or inhibit student motivation for achievement; and 3) that motivationproducing interactions with teachers can be improved by an intervention that increases teachers’
knowledge of adolescent development and provides them with ongoing, pragmatic support and
feedback to apply that knowledge in classrooms on a daily basis.
A.
Underperformance of the Classroom Setting as a Context for Youth
Development
There is little question that academic achievement, personal well-being, and civic-related
outcomes for adolescents are in dire need of improvement and enhancement (Carbonaro &
Gamoran, 2002; National Center for Education Statistics, 2003a). Students spend one-quarter of
their waking hours in schools, most of it in classrooms. Yet for all of the resources devoted to
schooling, the characteristics of secondary classrooms as settings for positive youth development
are often sorely lacking. Social and task-related disengagement and alienation reported by
adolescents often result from classroom experiences that are disconnected from youths’
developmental needs (Crosnoe, 2000; Dornbusch, Glasgow, & Lin, 1996; Eccles, Lord, &
Midgley, 1991). Youth often report a sense of disinterest in the goals of school and little
motivation to perform academically. They describe school experiences as irrelevant and lacking
appropriate and meaningful challenges. These tendencies are exacerbated dramatically for youth
attending schools in low-income communities, rural communities, large schools, and for those
with histories of poor achievement or problem behavior (e.g., Crosnoe, 2001; Eccles, Lord,
Roeser, Barber, & Jozefowicz, 1997).
Even more disconcerting is recent evidence from observational studies of large samples
of 5th grade classrooms, that the nature and quality of the instructional and social supports
actually offered to early adolescents in classrooms is generally low, and even lower for the
groups noted above (NICHD ECCRN, 2006). Moreover, findings from studies of large and
diverse samples of middle schools demonstrate quite clearly that competitive, standards-driven
4
instruction in de-contextualized skills and knowledge contributes directly to this sense of
alienation and disengagement (Eccles et al., 1997; Shouse, 1996). With regard to achievement
outcomes, there is recent evidence from large-scale national assessments such as the National
Assessment of Educational Progress (see National Center for Education Statistics [NCES],
2003b) that middle and high school youth are underperforming and that performance gaps
related to culture, race, and income are not closing despite years of rhetoric and attention (NCES,
2003b).
Symptomatic of the fundamental incapacity of schools to engage youth is the dropout
rate. When fewer than 60% of 9th graders in certain demographic groups (NCES, 2003b) actually
graduate 4 years later, when for 10 years decreasing the dropout rate has been a singular focus of
most secondary schools, and when the average annual dropout rate remains near 10% and ranges
up to almost 30% for recently-immigrated Latinos, it is strikingly clear that the high school
classroom as a setting for youth development is fundamentally broken. In fact, this epidemic is
such an embarrassment to the education system and state/local government that such results are
often masked and not publicized widely—witness the frequency with which graduation rates are
reported only for 12th graders, or on an event-basis (NCES, 2003b), not for the cohort entering 9th
grade 4 years earlier. Perhaps successful efforts to combat this epidemic are not to be found in
high-stakes, threat-based measures to keep youth from leaving school, but rather in applying
basic social science research and theory on qualities of school settings that keep both teachers
and youth engaged with one another and motivated to attain reciprocally satisfying goals.
No wonder then that current reform efforts at the Federal and state levels are focusing
intently on secondary education, through expanded and added requirements for teachers (e.g., a
degree in the field in which they teach), extension of the school day and year, creating socially
and administratively smaller schools (Felner et al., 2001), and a host of interventions aimed at
making schooling more relevant (e.g., work experiences, internships, college courses; see
Department of Education, 2004). In short, secondary education is being re-invented. In the
context of this effort, it is critical that student motivation and engagement be viewed as perhaps
the primary focus and means by which outcomes can be improved, particularly given the
dramatic failure of efforts aimed toward reducing dropout rates. Cowen (1999) has cast this as a
shift in perspective from prevention of pathology to promotion of wellness. It is this wellness
framework within which the present project is theoretically and operationally situated: This
project operates on the assumption that secondary classroom settings underperform in part
because teachers are typically ill-equipped to manage and enhance normative, developmental
processes central to maximizing youth motivation.
It is ironic that as adolescents’ capacities to engage with the larger world are soaring,
their engagement in the process of learning how that world works, via our educational system is
plummeting (Eccles & Roeser, 2003; Roeser, Eccles, & Sameroff, 1998). Engagement in school
begins to decline early in adolescence and by entry into high school, this decline is pronounced
to the point where more than half of high school students from all types of schools report that
they do not take their school or their studies seriously (Marks, 2000; Steinberg, Brown, &
Dornbusch, 1996). Further, adolescents bring their peers along with them: doing well in school
switches from being a positively valued behavior among peers in childhood to a somewhat
negatively valued behavior by mid-adolescence. Yet, engagement and intrinsic motivation
become pivotal in adolescence, as students at this age have the means to not only withdraw
5
energy from educational pursuits but also the ability to drop out altogether (National Research
Council [NRC], 2004).
Strikingly, under many conditions dropouts actually decrease in their levels of delinquent
and deviant behavior once they leave the school setting (Gottfredson, 2001). In the short-term,
dropping out removes a flailing adolescent from a largely aversive environment and produces
modest negative consequences. Unfortunately, beyond the short-term the effects of dropping out
of high school are both devastating and lifelong. Notably, however, the loss of motivation in
high school occurs for higher functioning adolescents as well—though parental pressure and
advanced abilities to delay gratification often allow these students to do passable quality work,
albeit with little interest or excitement (Steinberg et al., 1996). The result is that a recent
committee of the National Research Council charged with recommending ways to improve
academic outcomes for youth recently issued a primary recommendation, “that high school
courses and instructional methods be redesigned in ways that will increase adolescent
engagement” (NRC, 2002).
Although there have been promising efforts to reform schools to improve such outcomes,
most focus on the structural features of schools (e.g. “schools within a school” models; see
Felner et al., 2001) and instructional content aspects of the educational process (e.g., mandating
that teachers have content area degrees, see Whitehurst, 2002). In fact, secondary schooling is
viewed as weak enough to justify it as a primary focus of the next set of school reform initiatives
guided by No Child Left Behind. Notwithstanding these reform initiatives, youth report that
their experiences in classroom settings are lacking in meaningful challenges, supportive
relationships, and competence and motivation-building experiences necessary to promote
positive youth development (Crosnoe, 2001; Csikszentmihalyi & Schneider, 2000; Marks, 2000;
NRC, 2002; Roeser et al., 2000).
The central thesis informing the proposed project is that the capacity of schools to
support youth development, particularly for “high risk” youth, depends on the extent to which
the essence of schooling—the interactions among students and teachers within a classroom--is
structured to be a developmentally meaningful and challenging experience (NRC, 2002).
Adolescents’ (and teachers’) motivation and engagement in classroom processes are highly
dependent upon the quality of the working relationship they share—a relationship that forms as a
consequence of instructional and social interactions. It is these interactions that embody the
capacity of the classroom to promote positive development and these interactions which will be
the focus of this work.
B.
Motivation in the Classroom
When reflexive obedience to authority and belief in the relevance of grades begin to lose
their sway as primary motivators for many students entering adolescence (NRC, 2004), what
occurs within the central setting of the modern high school—the classroom—may be crucial to
holding their interest. Big schools, small schools, tracked schools, untracked schools, rigidly
structured schools and loosely organized schools all fundamentally rise or fall on the success of
what occurs within the classroom (e.g. Crosnoe, 2001; Resnick et al., 1997). The results of recent
large-scale, national evaluations of schooling effects, using sophisticated multi-level modeling
approaches, establish that the vast majority of variance in student achievement growth via
schooling is due to effects within the classroom (e.g., Nye, Kostantopolous, & Hedges, 2004).
Furthermore, students’ interactions and relationships with individual teachers are reported to be
6
the most salient of schooling features beyond the world of peers (Marks, 2000; Resnick et al.,
1997). Indeed, as a context for youth development, the high school classroom absorbs the
greatest percentage of the adolescent’s waking hours relative to any other setting outside the
home.
Ironically, close observation of most any secondary school in America reveals that
adolescents—both at-risk and high functioning—often display remarkably high degrees of
motivation and engagement within the school setting. Rarely, however, does this occur within the
classroom. High school hallways and lunchrooms literally brim over with youthful energy,
excitement, and enthusiasm. Intense interactions occur in sports and extracurricular activities,
and interactions with peers dominate students’ perception of the social ecology of school. It is
only when these students enter their classroom that energy levels decline precipitously and it is
rare that a given student will “connect” with a teacher or material in classroom or subject area in
such a way that they perform at high levels of capacity or “flow” (Csikszentmihalyi, 1988;
Csikszentmihalyi, 2000). Yet, the level of student engagement within the classroom is a critical
component of academic success, particularly for at-risk students (Finn & Rock, 1997; Shernoff,
Csikszentmihayi, Schneider, & Shernoff, 2003), and the extent to which student motivation can
be developed toward true mastery of material has fundamental implications not only for
educational outcomes but also for broader aspects of personality development (Dweck, 1986;
Dweck & Leggett, 1988).
The classroom setting looks equally bleak from the perspective of teachers, who are also
dropping out and becoming more disengaged. Fifteen percent of the entire teaching workforce
turns over every year. Rates of teachers leaving the profession are increasing. And those who
stay report a sense of malaise and frustration—they feel their job is getting harder and they have
fewer tools with which to work and feel effective (Hart, Stroot, Yinger, & Smith, 2005). The
relatively new teachers targeted by this proposal (with 2-5 years of teaching experience) are in
some sense still youth themselves, and as subject to becoming discouraged and dispirited by a
profession they may have entered with optimism and high hopes. They were selected for
intervention in part because their youth is likely to lend them flexibility to develop new
approaches to structuring their classrooms. In addition, however, they have many potential years
of teaching ahead of them, thus leveraging the effects of the initial intervention investment for
the future. A modest, but intended, side-benefit of the proposed intervention is to enhance the
development of these youthful teachers (many of whom will fit within the population of youth up
to age 25 targeted by the William T. Grant Foundation), by enhancing their job involvement and
motivation.
A fundamental principle of the proposed project in addressing the chronically resourcestarved classroom in the American high school is that modifying the classroom as a setting to
engage teachers and youth more fully may be the single best way to unleash and expand the level
of human resources (e.g., energy, effort, and enthusiasm) available to the educational process.
From this perspective, this intervention addresses the long-recognized, inherent, and chronic
“resource crisis” in schools by recognizing that high school classrooms frequently contain vast,
untapped (i.e. wasted) human resources seen as ennui on the part of both students and teachers
(Sarason, 1982). These resources continue to exist within classrooms, but it is only by altering
the nature of the interactions within those classrooms that their potential can be realized.
C.
Determinants of Youth Motivation
7
Increasingly, research on the social development of both normal and at-risk youths
suggests ways of understanding adolescent social development and motivation that can inform
educational practices in the classroom. Below, we organize and distill this knowledge into three
features of classrooms likely to influence adolescents’ levels of motivation. These are based on
recent research in several areas of adolescent psychosocial development that have focused on
adolescents’ needs to achieve a sense of connection and meaning within settings that also
promote a sense of autonomy and competence (Allen, Hauser, Bell, & O’Connor, 1994a; Allen,
Kuperminc, Philliber, & Herre, 1994b; Allen et al., 2002; Ryan & Deci, 2000). This perspective
suggests that an adolescent’s motivation in any setting will partly depend on the extent to which
that setting meets his/her needs for a sense of relatedness, of autonomy and competence, and of
the relevance of what occurs within that setting.
Relational Supports. As a behavior setting, the classroom runs on two primary types of
relationships: the relationship between the student and the teacher, and the relationships of
students with one another. It is not an overstatement to suggest that most adolescents live for
their social relationships (Collins & Repinski, 1994). Indeed, lack of relationships to others in
school is one of the single best predictors of the propensity to drop out of school (Cairns, Cairns,
& Neckerman, 1989; Lee & Burkam, 2003). Yet, the qualities of these relationships are
frequently afterthoughts in battles over curricula, testing, school structure and funding.
Positive relationships with adults are perhaps the single most important ingredient in
promoting positive youth development. The link between parental relationship quality and youth
development is so well-established as to be a truism (Allen, Moore, Kuperminc, & Bell, 1998;
Sroufe, Egeland, Carlson, & Collins, 2005; Steinberg, 2001). Teacher-student relationship
quality has received far less attention (see Pianta, Hamre, & Stuhlman, 2002, for a
comprehensive review), yet, teachers often spend more time interacting with adolescents each
day than do parents, even when teaching students only for a single subject. Collectively,
secondary school teachers consistently provide the single greatest source of exposure to potential
adult influence that most adolescents receive.
When teachers learn to make modest efforts to form a personal connection with their
adolescent students—such that the students feel known —they can dramatically enhance student
motivation in school and emotional functioning outside of school (Roeser et al., 1998; Skinner,
Zimmer-Gembeck, & Connell, 1998). A recent phone survey found adolescents reporting that
they would learn a great deal more if they felt their teachers cared about them personally (Public
Agenda, 1997). Similarly, at-risk adolescents report that a close and supportive relationship with
a teacher is a key feature distinguishing those who succeed in school from those who do not
(Resnick et al., 1997).
For students who find the anonymity of high schools alienating, teachers who notice a
student who seems lost in class, who respond positively to a student by name when the student
makes a comment, or who do a small favor for a student, can often dramatically increase student
motivation with a remarkably modest investment of time and attention. When relationships
function well, the resulting increases in motivation to comply with basic school norms also
appears likely to lead to reductions in problematic behavior outside of school (Bryant,
Schulenberg, Backman, O’Malley, & Johnston, 2000). Pianta, Hamre, and Stuhlman (2002) in
their comprehensive review of the literature on the nature and significance of student-teacher
relationships conclude that for adolescents, the dimensions of closeness, connection, and
8
affiliation are a critical feature of classroom interactions, even more so than avoidance of
relational conflict.
A vast range of studies has reported associations between youths’ sense of social
connection and outcomes ranging from higher achievement scores, to greater student
engagement and more positive academic attitudes (Bryk, Lee, & Holland, 1993; Bryk &
Driscoll, 1988; Connell & Wellborn, 1991; Crosnoe, Johnson, & Elder, 2004; Ryan & Deci,
2000; see also, NRC, 2004, for extended review of other similar findings). Most students,
unfortunately, do not feel their teachers care about them personally (Public Agenda, 1997).
Notably, even for relatively highly motivated late adolescents in college, recent experimental
work has shown that a sense of isolation can significantly reduce energy for intellectual pursuits
and that this reduction is powerful enough to temporarily depress results on IQ tests (Baumeister,
Twenge, & Nuss, 2002), while increasing irrational and risk-taking behavior (Twenge, Catanese,
& Baumeister, 2002).
Ironically, students are not alone in feeling alienated and isolated within secondary
school classrooms. Their teachers often share these feelings. Presumably, secondary school
teachers enter their profession with some degree of affinity for working with and interacting with
adolescents (Darling-Hammond, Holtzman, Gatlin, & Heilig, 2005). Yet, for every 10 new
teachers entering the profession each year, 2 to 3 will have become discouraged and changed
professions within one year (see Hart et al., 2005). The motivation of these teachers is in turn
likely to be highly influenced by the quality of their classroom interactions with youth. Young
teachers, of course, experience the same needs for connection, competence, and autonomy as do
the youth they teach (Deci, 1989). Yet, the nature of the classroom is such that it is often highly
isolating for teachers. No one other than adolescents—or the occasional administrator in a
position to judge their performance—is ever likely to be able to observe and interact with a
teacher about what goes on within his/her classroom. Breaking down the isolation experienced
by secondary school teachers—both by making interactions with students more positive and by
building in meaningful and non-judgmental peer interactions that are closely linked to what
occurs within classrooms—potentially can bring about major changes in teacher motivation and
development as well (Hart et al., 2005).
Building a healthy, academically and personally supportive relationship with students is
not an impossible task—some teachers do it instinctively—and it is a task that typically brings
gratification to teachers that more than makes up for the energy it requires of them. But for many
teachers, structuring their classrooms to facilitate the development of healthy relationships with
students takes a conscious effort and significant guidance (Pianta, 1999; Pianta, 2006b). One
facet of this proposal is to focus upon turning the classroom setting into a purposefully socially
engaging environment that motivates students and teachers to work together toward mutually
satisfying goals. This proposal begins with everyday instructional and social interactions through
which successful teachers establish a productive and satisfying interpersonal climate and then
seeks to enhance the value of these interactions for promoting relationships. Hamre and Pianta
(2005) recently demonstrated the powerful effects of enhancing such student-teacher interactions
to close the achievement gap in first grade—in this project we aim to harness and build upon the
potential of these interactions in secondary classrooms.
Our own focus groups of recent high school graduates have found that, when asked to
think of the characteristics of their best teachers in high school, youths very frequently
mentioned individuals who had taken small extra steps that showed that they cared about the
9
students as individuals. These steps did not involve the kind of massive, unsustainable efforts
sometimes heroically depicted in cinema, but rather events as small as consistently drawing a
student out in a supportive way in a class, or small, friendly gestures or jokes shared with a
student. Even within the regular teaching curriculum within the classroom, youths noted that
some teachers made clear that they knew and cared about their students. These teachers also
made their classrooms safe environments emotionally and intellectually, where ridicule—from
other students or from teachers—were explicitly not part of the curriculum. In short, in spite of
the potential alienation built into the modern high school, some teachers have made clear that
they can transform their classrooms into humanly and humanely rewarding environments. Our
well-validated observational coding system is designed to capture these behaviors and anchor
them concretely. This facilitates high quality research on classroom setting characteristics, but
even more importantly, it provides concrete indicators that allow such behaviors to be modeled,
taught, and reinforced to new teachers in the field on an ongoing basis.
In addition to the teacher-youth relationship, peer interactions have tremendous potential
as motivational forces for adolescents (Hartup & Stevens, 1997; Seidman et al., 2001), though it
has long been recognized that these influences are frequently not toward positive academic
outcomes (Bishop, Bishop, Gelbwasser, Green, & Zuckerman, 2003; Coleman, 1961).
Nonetheless, evidence suggests that peer processes can be built into classroom interactions in
productive ways both large and small, if a conscious, thoughtful effort is made in this regard
(Crosnoe, Cavanaugh, & Elder, 2003). Research now demonstrates that the power of these
interactions can be directed toward (or away) from academic purposes depending on the
structure of the classroom (Berndt & Keefe, 1996). The traditional competitive organization of
the classroom—with more energetic, able, or assertive students receiving most of the attention
and rewards—has long been known to potentially create a “reverse-peer-culture” in which those
who excel are seen as making life relatively harder for the rest of the class (Coleman, 1961). In
addition, the extent to which teachers’ routinely use differential treatment of students and
competitive techniques in classrooms has been linked not only to poorer academic attitudes but
also to lower student self-esteem (Roeser & Eccles, 1998). Conversely, more cooperative
methods, particularly those which in some way reward students for the learning of all students
within their group, have been consistently found to increase levels of student engagement and
achievement (Slavin, 1996). The extremes of either style, of course, rarely occur within
classroom settings, but significant gradations between these two styles clearly exist and suggest
opportunities to increase the extent to which peers become positive motivating forces within the
classroom setting.
Autonomy/Competence Supports. Teens are engaged by challenges that are within
reach and that provide a sense of self-efficacy and control—experiences that offer challenges
viewed as adult-like but for which appropriate scaffolding and support are provided (Bandura,
Barbaranelli, Caprara, & Pastorelli, 1996; Eccles et al., 1993). One of the most tragically
avoidable errors that some secondary school teachers make is to assume that youth strivings for
autonomy and self-expression represent negative forces to be countered rather than positive
energy to be harnessed. This basic misunderstanding of adolescent development (one often
promoted in teacher education courses and reinforced by school policies) then takes form in
highly controlling and punitive classroom and school settings and in instruction that is highly
teacher-driven and discouraging of exploration and curiosity. This mismatch of classroom and
development, driven by profound misunderstanding of teens, results in schools narrowing, rather
than expanding, the “space” in which zones of proximal development can be created for youth.
10
Teens intrinsically seek to control their environments, but in almost all circumstances
they do so in ways that attend to relevant environmental constraints. Thus, a teen might argue
endlessly, rudely, and even viciously with a parent over a seemingly minor issue of parental vs.
teen control—where the teen sees some possibility of the argument paying off. Yet, one rarely
sees a teenager struggling endlessly with their pet over control, or persisting in trying to open a
clearly locked door, or even arguing with a referee on a sports field for more than 30 seconds.
Teens want autonomy. They want it desperately. But they are more than capable of recognizing
structural limits to their autonomy and control, and to seek autonomy within those structures.
Teens also are highly motivated by their own developing competence—as it offers the promise
of greater future autonomy. The key then, is to direct teens’ autonomy/control strivings in ways
that develop their confidence while maintaining clear environmental controls.
The extent to which the classroom as a setting provides a structure that allows student
effort to be quickly recognized and rewarded may be one of the single most important features in
harnessing student autonomy strivings. Highly proximate goals tend to best maximize student
motivation (Bandura & Schunk, 1981). Perhaps the most powerful example of this effect is seen
in the almost ridiculously high levels of motivation provided from the second-by-second (but
otherwise superficial and meaningless) reinforcement of video games. Youths will spend hours
transfixed by moving dots on a computer screen—provided those dots are sufficiently responsive
to their own inputs. Contrast this highly proximate motivation with the very long-delayed
reinforcement provided by school programs that may enhance career potential only many years
in the future. Effective teachers bridge this gap through instruction that provides highly proximal
goals within the classroom. Attention and concentration are scaffolded/supported by high quality
instruction. Involvement is rewarded by active engagement in meaningful activities. And
homework is directly linked to learning by being relevant and providing immediate
consequences for completion vs. non-completion. Together, these activities can begin to engage
the “video-game effect” in which settings that are highly contingent and meaningful in the very
short-term can elicit high levels of motivation and effort—the exact opposite of what occurs in
many middle and high school classrooms (Eccles et al., 1993).
The degree to which a student’s effort in the classroom immediately “matters” to that
student (i.e. it affects something that the student cares about) is fundamental to motivation. Yet
normal variation in classroom feedback/responsiveness is tremendous (NICHD ECCRN, 2005).
Teachers may lecture as a monologue for 45 minutes and return graded papers only several
weeks after they are turned in. Alternatively, they may ask students to participate in thoughtful
ways on a minute-by-minute basis and grade assignments and provide feedback almost instantly.
Observations of 5th grade classrooms demonstrate that the average student interacts with their
teacher fewer than 4 times during a given one-hour period (NICHD ECCRN, 2006), a rate that is
not conducive to building systems of interaction that serve as regulatory mechanisms with
respect to students’ affective, motivational, or learning-related experiences (Pianta, 1999).
Teachers can also support student competence development by having high expectations
within their classrooms. Research to date suggests that the degree of the “academic press” within
the classroom has effects on achievement in addition to the effects of relational supports
(Phillips, 1997). Similarly, high teacher expectations for students have been found to predict
achievement scores, particularly for more at-risk students (Madon, Jussim, & Eccles, 1997).
Finally, teachers also have many opportunities to provide adolescents with meaningful
choices and autonomy in classrooms in ways that do not threaten teacher authority. They do not
11
always recognize these opportunities, however, or their importance to adolescent motivation.
Supporting student autonomy does not mean giving up teacher control. On the contrary,
autonomy can be supported by giving students choices of partners for group projects, types of
projects to perform, etc. (Allen et al., 1994b; Anderman & Midgley, 1998). Students who are
making choices are by definition not passive, but must to some degree become engaged in the
learning process (Deci, 1991). The fundamental challenge to teachers in this regard to is to
understand adolescents’ developmental push for autonomy so that they can then seek to guide
and direct it.
Relevance. Bronfenbrenner (1979) has eloquently described the importance youth attach
to whether what occurs in a setting such as a classroom, is actually linked to what happens in
other important settings beyond it. Too often, the high school curriculum and the rationales
behind it are taken as a “given” without recognition that these rationales need to be made clear to
each new cohort of students. Drawing even very distal connections between what occurs within
high school and the larger “real world” can alter student behavior. For example, involving
students in significant, real-world, voluntary community service and then discussing it within the
classroom in an ongoing way, has been found to reduce failure rates by 50%, in randomly
controlled trials, with similarly profound effects upon other behaviors in youths’ lives as well
(Allen, Philliber, Herrling, & Kuperminc, 1997). Centuries ago, late adolescents were
commanding armies and running countries (Barzun, 2000). Today, an ever more competent
generation of adolescents is confined to a classroom for hours a day with little vision of how
what occurs within that classroom relates to the larger world. Consciously addressing the
relevance of what occurs within the classroom to the students’ future options in that larger world
is critical to engaging otherwise restless young minds.
On a smaller scale, teachers may increase the relevance of the classroom by making
repeated, explicit ties between curricular material and real-world applications and engaging peer
group processes in learning (given the intrinsic meaningfulness of peer interactions to youth).
The key factor here is that the real-world connections must be made in ways that are meaningful
as perceived by the student. Stating that principles of trigonometry can be used to determine the
length of a ladder may be of little interest or relevance to a student who has never cared much
about the length of ladders and can’t easily imagine this being important to his/her future. In
contrast, connecting school work to actual careers in meaningful ways (e.g., describing how
trigonometry gets used in designing jet fighter planes and skyscrapers, and exposing students to
professionals working in these fields) can significantly enhance students’ sense of the
meaningfulness of what they are being taught, and hence of their motivation to learn it.
D.
Altering the Motivational Character of the Classroom
Although some teachers intuitively apply the motivational principles described above to
guide their interactions in classrooms, there is ample evidence from observational studies of
classroom settings and from surveys of youth that there is wide variation in these youthsupporting capacities. On average, these principles are not evident in most such interactions
(NICHD ECCRN, 2005; NRC, 2004). Although these same motivational principles also apply at
other ages (and indeed many have been built into the system for improving elementary
classrooms that serves as the foundation for the proposed work) in adolescence, two factors
make the motivational qualities of the classroom setting paramount. First, the struggles for
connection, autonomy and competence, and relevance all become developmentally focal issues
in adolescence. They can literally drive adolescents (for good or for bad) more strongly than
12
almost any other motivational force. Second, in adolescence, motivation is arguably the critical
variable that will determine academic outcomes. This is in contrast to childhood, where
conformity and obedience can often bring about adequate academic progress, and it is unlike
college, where selection procedures have already eliminated many less motivated adolescents.
Not coincidentally, adolescence is also the age where vast numbers of students drop out of the
educational system, bringing tremendous lifelong costs to themselves and society (NCES,
2003b).
As we note above, this project is based largely on the idea that teacher-student
interactions are the core asset in classroom settings, particularly for at-risk youth (Crosnoe et al.,
2003; Pianta, 2005). The motivational and instructional/organizational features of these
interactions account for the effects of schooling on youth achievement and social development
and take specific form in classrooms in relational supports, autonomy/competence supports, and
in clear demonstrations of the relevance of material presented. We label these as “motivational
qualities of interactions” in Figure 1 below. As we and others have demonstrated in elementary
and preschool settings, interactions between children and teachers are the mechanism by which
experiences in school produce achievement and social adjustment, above and beyond the
contributions of family environments and genetic endowments (NICHD ECCRN, 2004; Pianta,
2006a). This concept and our approach to intervention are depicted below.
Intervention
Intervention
Targets
Setting Motivational
Outcomes
Youth
Outcomes
Motivational
Quality of
Interactions
in Setting
Teacher
Motivation/
Effort
“My
Teaching
Partner”
Intervention
Student
Academic
Success
Student
Engagement
Student
Motivation/
Effort
Instructional/
Organizational
Qualities of
Setting
Figure 1. Conceptual Model for Proposed Project
In this diagram, the central, and putatively causal, role of the motivational properties of
student-teacher classroom interactions is depicted as the primary means by which motivational
resources in the classroom can be developed to lead to increased student engagement and
academic success. Notably, both explicit motivational characteristics of the setting (e.g., the
quality of the teacher-student connection) and more traditional instructional qualities (i.e.
organization and structure of material presented) are seen as contributing to motivational
13
outcomes (engagement, and student and teacher motivation and effort) in the classroom and
ultimately to educational outcomes as well. Our conceptual model does not ignore the role of
content-focused instructional quality. However, in adolescence, we recognize that although
instructional quality continues to have a significant direct link to student outcomes, much of the
variance in student outcomes is determined by the adolescent’s motivation within the classroom
setting. To put it more colloquially, you can lead a teen to teachers, but you can’t make them
think. Making adolescents want to think is a primary goal of the proposed intervention.
Our intervention program (MyTeachingPartner, or MTP) alters the dynamics of the
classroom both socially and cognitively so as to increase the likelihood that the adolescents
within it will be engaged, motivated, and work hard to think. The model has been successfully
applied in an array of elementary school classrooms (see description below); this proposal is to
apply this model to the secondary school classroom, while further enhancing it via attention to
adolescent developmental needs, and carefully evaluating its impact.
The model conceptualizes motivational and organizational/instructional capacities as
setting-level parameters that are closely intertwined with the qualities of student-teacher
interactions within the classroom. In this model, the intervention will target student-teacher
interactions as teachers understand them. Teachers participating in the intervention will be given
information on how to interpret the motivational properties of their interactions with students and
then be given feedback and coaching on their actual interactions so as to increase their
motivational value. As those interactions shift and re-organize around the motivational targets
described earlier (relational support, autonomy/competence, etc.) we hypothesize two effects –
the motivational properties of the setting itself will change as will the motivation/effort of
individuals within the setting (students and teachers). The motivation of individual students will
mediate effects of the intervention on achievement and related outcomes, while setting level
motivational properties (as reflected in collective student engagement) will indicate the effects of
the intervention at the setting level.
Pianta (1999) has outlined a systems model of student-teacher relationships that posits
that teacher knowledge and beliefs are predictably related to social and instructional interactions;
that interactions with students feed-back into teachers’ and students’ beliefs and feelings about
one another, and that this system of connected components has consequences for students’
learning, motivation, and engagement (see Pianta et al., 2002, for a comprehensive review of this
work). Changing this organized system of beliefs, knowledge, emotions, and interactions is in
part the challenge in this study. Ultimately if the goal is to change the mechanisms by which
classrooms enhance youth development (i.e., social and instructional interactions with teachers)
the proposed intervention must focus not only on changing interactions but also on changing
underlying beliefs and emotional experiences of teachers. This is why we have chosen an
intervention approach specifically designed to address the dynamic and interrelated aspects of
teacher-student interactions and relationships.
The “My Teaching Partner” Program. The My Teaching Partner program (MTP;
Pianta, Kinzie, Pullen, Justice, Lloyd, & Fan, 2003) consists of an intervention process that
increases teachers’ knowledge of adolescent social development and applies this knowledge with
ongoing personalized coaching and feedback that supports teachers’ appropriate responses to
adolescents’ relational, motivational, and academic needs. The intervention integrates
information-based training delivered through workshops together with personalized review of
classroom observations using a carefully designed web- and video-conference assessment and
14
feedback process. MTP consultants provide direct, individualized, regular, and systematic
feedback to teachers based on validated, observational assessment of the classroom environment.
As such, MTP is designed to break through the isolation of teachers in most high school
classrooms, whose work is almost never viewed by other adults trained to help, except when
those adults also are in supervisory/evaluative roles. The MTP Consultancy process functions by
increasing teachers’ a) capacity to observe the qualities of their interactions with students and the
contingencies involved, b) awareness of the meanings of these interactions in terms of their
contributions to motivational, relational, and competence-enhancing processes, and c) reflection
on the teachers’ own personal motivations and tendencies in these interactions and their impact
on interactive behaviors.
Program mechanisms. We emphasize two key features of the MTP approach to
addressing “capacity” problems in secondary classroom settings. First, MTP uses “face-to-face”
training sessions conducted over live, web-cam based internet connections, using user-friendly
technologies that have been thoroughly ‘debugged’ at scale in real-time school settings. In the
proposed project, we will specifically target teachers with fewer than 5 years of teaching
experience for exposure to this intervention approach. This group is at considerably elevated risk
for leaving the profession, despite the fact that evidence suggests recently-trained teachers
potentially bring unique resources to bear upon the challenges of secondary education (Hart et
al., 2005).
Central to this effort is an objective framework for understanding and organizing the
dimensions of student-teacher social and instructional interactions. In fact, Pianta has argued that
validated, objective, standardized observational assessment of classroom interactions is the key
to teacher professional development aimed at improving classroom effects on development
(2005). Thus the second key feature of the MTP approach is that the feedback and support
provided to teachers during the ongoing consultation process is directly tied to validated,
standardized observational systems shown to assess aspects of teachers’ social and instructional
interactions that predict growth in student performance. This system is known as the Classroom
Assessment Scoring System, or CLASS (Pianta, La Paro, & Hamre, 2004). The vocabulary we
use to describe those interactions is based on the CLASS-Secondary, an upward extension of the
CLASS (Pianta et al., 2004).
The dimensions of interaction assessed by the CLASS elementary version predict growth
in literacy and math as well as reduced teacher-child conflict and problem behavior from pre-k
through 5th grade (Hamre & Pianta, 2005; Howes et al., in press; NICHD ECCRN, 2004, 2006).
The CLASS, originally designed for elementary settings, has been recently redesigned and
adapted for secondary settings, and is now being used in K-12 teacher quality assessments in
Ohio, and in research on teacher education programs conducted as part of the Carnegie
Corporation’s Teachers for a New Era initiative (Hart et al., 2005). MTP Consultants rely on the
dimensions of interaction presented as observational rating scales in the CLASS and the detailed
descriptions of scale points on those dimensions that are provided in the CLASS manual when
they provide feedback to teachers. The CLASS is one of the most current and widely-used
standardized assessments of social and instructional interactions in classrooms (Hart et al., 2005;
NICHD ECCRN, 2002, 2005; McCaslin et al., 2005).
The CLASS-Secondary version, or CLASS-S, is explicitly designed to capture precisely
those aspects of classroom interactions that we hypothesize above to be resources for adolescent
engagement and motivation. As such, it builds on and incorporates all of the strengths of the
15
CLASS system at elementary levels, while adding specific dimensions conceptualized and
operationalized to maximize adolescent engagement. The CLASS-S scales are organized into
two overarching dimensions, similar to those reported in factor analyses of the elementary
version: Motivational Processes (e.g., Sensitivity, Relational Supports, Autonomy Supports) and
Organizational/Instructional Processes (e.g., Effective Behavior Management, Content
Knowledge, Quality of Feedback, Cognition Supports). Importantly, the CLASS–S serves as an
indicator of setting-level properties directly targeted by the motivational components of the
intervention (i.e., the scales reflecting the Motivational Processes factor) as well as other aspects
of classroom interaction that we believe are likely to be indirectly linked to student motivation
(i.e. Organizational/Instructional Processes) but important to understanding achievement
outcomes nonetheless.
We also emphasize a critical feature of this project, that MTP coaching is based upon
explicit, standardized, reliable, and valid indicators of student-teacher interactions that are
directly linked to the youth motivational processes we outlined earlier. The investigative team
has been leading the refinement of the CLASS-Secondary (CLASS-S), which has undergone
extensive field testing in studies of new and experienced teachers and has already been shown to
differentiate teachers judged to be “exemplary” from others in pilot work. The CLASS-S also
forms the basis of the video-text examples of “high quality teaching” on the MTP website, that
are in turn used by individual teachers and consultants as an empirically-based and objectively
defined model of “best practice” in real classrooms.
In sum we hypothesize that although classrooms are complex social systems and studentteacher relationships are also complex, multi-component systems, the nature and quality of social
and instructional interactions between teachers and children can be changed by providing
knowledge about developmental processes relevant for classroom interactions and personalized
feedback/support to teachers for observing interactive behaviors and cues and responding to
them in ways consistent with their developmental meaning and significance.
Program efficacy. The MTP model of professional development for teachers and its
influence on teacher-student interactions and child achievement has been tested with teachers of
younger children in a recent randomized trial with more than 220 teachers and 960 children. An
ongoing randomized controlled trial of My Teaching Partner’s approach to consultation (Pianta,
et al., 2003) has examined the efficacy of web-based inservice-training that provides experienced
pre-K teachers with CLASS-focused feedback on their practice and trains them in CLASS
definitions of implementation.
Comparisons of teachers who received this consultation support (N = 72) with those
receiving no consultation (N = 51) show significant differences, on average, in gains from Fall to
Spring of the academic year for the intervention group on CLASS dimensions of Productivity (t,
2-tailed, = 1.98; p = .05), Use of Effective Instructional Learning Formats (t, 2-tailed, = 5.49; p <
.001), and Students’ Engagement in Learning (t, 2-tailed, = 4.79; p < .001). Teachers who did
not receive consultation support showed lower levels of observed Positive Climate across the
year (t, 2-tailed, = -2.17; p = .03). These findings provide evidence of the efficacy of the MTP
approach to inservice professional development that provides information and feedback on
interaction based on a validated system for observing and defining such practices. Our next
concern was whether this type of training contributed to children’s learning in the domains of
language and literacy.
16
Analyses of results from direct assessments of children’s language and early literacy
skills show that changes in teacher-student interactions due to MTP were in turn associated with
gains in direct assessments of young children’s performance in language and literacy. For
example, when teachers showed greater increases in positive interactions with children, gains
were greater on alphabet knowledge (r = .34, p = .01) and rhyme awareness (r = .28, p = .04)
while gains in effective behavior management predicted gains in alphabet knowledge (r = .30, p
= .03). Increases in concept development predicted higher scores on word awareness (r = .29, p =
.03) while teachers who provided more engaging instructional formats had children who scored
higher on rhyme awareness (r = .29, p = .04). Finally, higher gains on teachers use of highquality language modeling were associated with greater gains for children on word awareness (r
= .28, p = .04). Finally, teachers reported greater levels of job satisfaction and self-efficacy when
exposed to consultation (Hadden, Dudding, Chase, & Pianta, 2006). The MTP team has also
applied the program to teacher education students during their student teaching, and in other
classroom settings, gathering experience and information that will influence the current project.
Thus for work with teachers of younger children, MTP support positively influenced
teachers’ interactions in classrooms, which in turn were related to improved achievement. These
findings are consistent with related literature suggesting that observation of teacher-child
interaction, knowledge about development in language and literacy, and effective
implementation of instruction, contributes to higher ratings of observed sensitivity, language
stimulation, and implementation in quasi-experiments (Cassidy, Buell, Pugh-Hoese, & Russell,
1995; Howes, Galinsky, & Kontos, 1998; Rhodes & Hennessy, 2000). It is this link between
support for teachers, improvements in the classroom setting, and better child outcomes that we
intend to replicate with this trial in secondary schools.
Finally, there is also evidence that the MTP Consultancy support serves as a mentoring
program for teachers that may reduce loneliness and isolation by establishing a supportive
collegial relationship with someone who understands their work in depth (Liang, Tracy, Taylor,
& Williams, 2002). Such mentoring has previously been found to enhance young adults’
commitment to the mission of educational institutions (Ferrari, 2004).
In sum, the efficacy of the MTP program in elementary school classrooms has now been
documented on several different levels. This proposal is to replicate and extend this intervention
into secondary school classrooms by applying the same set of intervention techniques and the
same types of assessment/training tools as have already been documented effective in the
primary grades. To maximize the impact of the MTP program, the proposed intervention further
enhances the original MTP approach by now adding an intensive focus on the specific
determinants of adolescent motivation. The key theoretical tool of this enhanced approach (the
CLASS-S System) has already been field-tested and included in ongoing evaluations of
classrooms across several states.
III.
Specific Hypotheses and Research Questions to be Addressed
This proposal will explore several overarching questions about the efficacy of the My
Teaching Partner intervention and its underlying theoretical/observational foundation, using a
combination of observational, teacher report, student report, and aggregated student outcome
data. We begin with basic questions about program effects and then move from these to examine
underlying mechanisms by which classroom setting qualities may be linked to student outcomes
17
and may mediate any observed programmatic effects. We phrase these follow-up questions
under the assumption there will be effects of the MTP program, given the strong past track
record of the program. However, even in specific instances where no program effects are found,
we will still be able to use the rare combination of observational and multi-reporter classroom
data to enhance our understanding of critical aspects of the functioning of the secondary school
classroom as it relates to student outcomes, so as to inform our own and others’ future
intervention efforts.
1. What is the overall impact of the MTP intervention at each of the levels of analysis
considered in Figure 1 above (e.g., quality of teachers’ behaviors with students, student
engagement and motivation, teacher motivation, and student behavioral and achievement
outcomes)?
After beginning with this overarching question, we then address each of the specific
pathways outlined in Figure 1 in turn:
2. To what extent does observed improvement in motivational qualities of teachers’
behaviors toward students (e.g., teacher efforts to connect with students, make relevance of
material more salient, etc.) as a result of teachers’ exposure to the intervention predict increases
in students’ motivation and engagement?
3. To what extent does observed improvement in the instructional/organizational qualities of
teachers’ behaviors toward students as a result of the intervention predict increases in students’
motivation and engagement?
4. Do changes in student engagement and teacher and student motivation in the classroom
predict changes in individual student achievement and related outcomes? Changes in student and
teacher motivation will be assessed and examined both at the classroom level (via observational
and aggregated student measures) and in terms of individual students’ levels of motivation,
recognizing that student achievement may be influenced both by changes within the individual
student as well as by changes in the overall quality of the classroom interaction.
5. Are any of the relationships examined in the first four questions moderated by
characteristics of students (prior performance, gender, poverty status, race), teachers (prior
training, beliefs about youth, attitudes), or classroom/schools (poverty level, racial/ethnic makeup)?
It is important to note that in keeping with the purpose of this RFP, our proposal is not
primarily focused upon assessing impacts upon individual students as primary outcomes. Rather,
as noted above, mediational processes such as classroom interactions and student engagement
assessed at the classroom level are the primary outcomes. Nonetheless we are quite interested in
whether intervention effects on setting-level mediators also lead to student-level outcomes (e.g.,
higher achievement for both individual students and for classrooms of students). If so, then these
impacts are critical indicators of the direction in which setting-level changes should be pursued.
In short, we are seeking not simply to change settings in ways that have theoretical links
to intended outcomes, but also to go on to examine which kinds of setting change are actually
associated with those outcomes. Thus, characteristics such as increased relational warmth within
the classroom are believed to be likely to increase student engagement and motivation, and
ultimately achievement. However, given the paucity of studies that have actually sought to
experimentally manipulate such characteristics, research is clearly needed to investigate whether
such increased setting warmth actually ‘flows through’ to lead to intended outcomes. From this
18
perspective, our goal is not simply to change the classroom as a behavior setting, but also to use
an experimental manipulation to examine which features of the classroom as a behavioral setting
are most important for youth development.
We view the outcome of this work as the continued, iterative, and empirically-based
improvement of the proposed intervention and the accompanying observational/training system,
both of which can be powerful tools to increase capacity of settings serving youth.
IV.
A.
Research Design and Methods
The Intervention
We plan to work with 80 recently trained teachers with two to five years of teaching
experience, conducting a random assignment study of an intervention that combines workshop
training in critical principles of adolescent social development with the MTP Consultation
process of support and feedback for improved teacher-student interactions as defined by CLASSS.
Workshop on Reinventing the Classroom using MTP. The goal of this workshop is to
provide teachers with foundational knowledge about the specific motivational principles outlined
above and a concrete and motivating action-plan for working together with our consultant to use
these principles to transform their classroom experience.
The initial workshop, like all of the intervention that follows, also employs the same key
principles involved in the intervention. It seeks to begin to increase teachers’ sense of
connection, efficacy, and relevance. As such, the workshop diverges as much as possible from
the typical lecture format of much in-service training. The goal is not to gain a slight increment
in teacher skill levels or knowledge, but to begin the process of getting teachers to rethink the
nature, structure, and process goals of their classroom as a behavior setting. The workshop has
several different goals.
1. Introduce teachers to adolescent motivational issues, from the adolescent’s perspective. Basic
information will be presented about both the importance of working first and foremost to
maximize adolescents’ motivation and the basic principles of social/developmental research
that support our approach to understanding adolescent motivation. Such information is likely
to have only limited value if presented solely didactically. Therefore, we also increase the
salience and relevance of this information by making extensive use of video clips of actual
high school-age youth and recent graduates describing classroom characteristics that did or
did not motivate them. Further, we also use video of average and highly successful teachers to
illustrate these points and to model for teachers how these principles might apply to their own
classrooms.
2. Give teachers a chance to practice/discuss the application of these principles to their
classrooms. Many of the principles presented in the workshop, upon reflection, will seem like
common sense to new teachers. The critical issue, then, becomes how does one apply these
principles to transform what occurs within the classroom. Specifically, we expect even young,
relatively novice teachers to approach this didactic material with some skepticism as to its
relevance to their own classrooms. We intend to welcome and encourage them to voice this
skepticism—modeling how we want them to treat adolescents in their own classrooms—so
that we may address it directly and use it to further refine our presentation of our program.
19
Our next step then, is to break teachers into small groups and have them work with the
specific trainer who will serve as their partner in the MTP program. Here, teachers ‘practice’
what they have learned by discussing different ways to present material, different problems
that arise in implementing these ideas and solutions to these problems. This component of the
program—which emphasizes exchange of information with their trainer and other teachers—
asks teachers to draw from their own teaching experiences, their own (relatively recent)
experiences as adolescents in classrooms, and the experiences of the other teachers in their
group. In this way, we seek to have the workshop build teachers’ own sense of their
relationship with their MTP trainer and their potential connection/relationship with their
students, their sense of efficacy/control in teaching, and the relevance of what is being taught
to their actual classroom experiences.
3. Introduce teachers to the MTP system of consultation. Although the system is designed to be
extremely user-friendly, even for young teachers with little to no computer experience, a
significant component of the workshop is walking teachers through the MTP remote
consultation process and use of the MTP website. Thus, a part of the workshop involves
introducing teachers to this web and video-conference-based system. However, again
following the principle of maximizing the relevance of what we do, we imbed much of this
training with real hands-on descriptions of substantive material. For example, when being
shown how to access edited/annotated examples of their own classroom behavior, we use
examples of teacher behavior specifically designed to illustrate points we are making about
adolescent development. Unlike most one-time in-service workshops, this workshop is
intended as only the first stage of training, to be followed by the far more personalized and
focused ongoing consultation sessions via the My Teaching Partner program.
The final concrete act of the workshop is to build in a set and regular time for the MTP
consultations to take place. It is our experience that setting this planning time in stone, at the
outset of the school year, is critical to making the intervention occur under the real-world
pressures faced by secondary school teachers.
My Teaching Partner Consultation during Academic Year 1. The MTP Consultation
intervention uses videotaped review and video-conference based interactive dialogue regarding
classroom interaction by experts in both teaching and adolescent development and has
demonstrated success in altering classroom processes at the primary level (Pianta, 2005, 2006).
The program is based on a four-stage “consultancy cycle” which serves as an iterative, highly
personalized process of teacher feedback and development. The cycle is intended to produce a
stream of communication between consultant and teacher based on observation of that teacher’s
interactions with students in the classroom and a process of joint reflection on strategies to
improve those interactions, using CLASS-S as the focus for quality. Thus a Consultancy Cycle
consists of four steps:
Teacher videotapes a classroom activity/lesson/discussion involving instruction of
students.
Consultant reviews the classroom observation video and then posts selected video clips,
edited to maximize didactic value, along with written prompts on a private, secure website for
review by the teacher.
Teacher views edited video and responds to prompts within an online journal.
20
Teacher and consultant participate in an online videoconference to discuss the edited
classroom video and issues related to classroom performance and determine goals for future
cycles.
On the next page you will see a diagram of this cycle. Note how information about the
teacher’s interactions with students is captured on video and flows to the consultant, who
analyzes it. Feedback is provided in terms of the online posting of the analyzed video and
consultant comments in the form of prompts. Finally, discussion of the feedback and further
sharing of information takes place through the online journaling and videoconferencing iChat™
meeting.
In the proposed work, teachers will have many opportunities over the academic year to
receive feedback and support from the consultant and to offer their insight and feedback in
return. Because of the extended nature of contact between the consultant and the teacher,
consultants must recognize that the consultancy relationship builds over time, in terms of the
capacity to give and receive feedback, particularly feedback that can be interpreted as critical.
For this reason, consultants must attend, in the initial stages, to the need to build a relationship
with the teacher and focus initially on support, slowly transitioning to more challenging aspects
as there is evidence the relationship with the teacher can tolerate such challenges.
Step 1: Teacher videotapes an
activity within the classroom
Step 4: Teacher and consultant
engage in videoconference; discuss
the classroom video and determine
goals for future sessions
Step 2: Consultant edits
and posts the video along
with printed prompt
Step 3: Teacher responds to
edited video and prompt in
online journal format
Figure 2. The MTP Consultancy Cycle
The four steps in the consultancy cycle repeat twice per month, with each cycle extending
across a two-week period.
Week One
Step 1: The teacher captures 30-40 minutes of video of the classroom. The teacher mails the
mini-digital videocassette containing the recording of the observation session to the
consultant using a prepaid mailer within 24 hours.
21
Step 2: The consultant edits the recorded observation video into a 3-4 minute segment using the
guidelines described in Steps of the Consultancy Cycle – Step Two. The edited video is then
posted along with written prompts on the MTP website. The consultant views, edits and posts
video clip at least 48 hours prior to the scheduled videoconference. The teacher is notified by
email that the edited video and prompts have been posted.
Week Two
Step 3: The teacher views the edited video and prompts and posts responses in the online journal
on the MTP website before the scheduled videoconference. This allows the consultant time to
read the responses and prepare for the videoconferencing session.
Step 4: The consultant reviews the teacher responses to the prompts in the online journal prior to
the videoconference. The videoconference is scheduled within 10 days of the recorded
classroom observation. During the videoconference, the teacher and consultant discuss the
edited video and teacher responses to the prompts and issues related to implementation of the
activity. They jointly determine goals for future sessions. After the videoconference, the
consultant sends the teacher a conference summary via email.
The MTP consultancy process is completely manualized. Moreover, a key component of
the MTP Consultancy intervention is a regular, weekly meeting of MTP Consultants with one
another, the purpose of which is continued clarification of goals, maintaining fidelity, care
studies, and group consultation/supervision. This meeting ensures quality control as well as
fidelity and consistency of implementation.
In the MTP approach, support and feedback to teachers can be effective at improving and
raising the value of adolescent-teacher interactions, whatever actual curriculum the teacher is
using in the classroom. An important supplementary resource for the intervention is the
MyTeachingPartner website, which has hundreds of videotaped segments of classroom
interaction corresponding to the different dimensions of the CLASS that have been
independently coded as exemplifying “high quality” (in terms of being rated at the level of 6 or
7). These videos are important teaching tools for consultants to use with teachers in terms of
demonstrating examples of high quality teacher-child interactions that map onto the motivational
supports described earlier. Repeated and directed viewing of these videos is often a part of the
MTP Consultancy.
My Teaching Partner Booster Workshop during Academic Year 2. In their second
year of enrollment in the study, following their participation in the MTP Consultation process the
prior year, each teacher receiving the intervention will receive additional training in the form of
an MTP Booster Workshop during the fall of the academic year. The workshop will consist of
brief extensions of the knowledge/principles outlined in the original workshop, group
observation and discussion of videos of classroom interactions particularly in relation to the
quality of those interactions (using the CLASS-S as a guide) and the consequences of those
interactions for student motivational processes. The workshop will also include group
discussions of strategies to promote motivational processes and collaboration/consultation
among the intervention teachers. This will be a one-day workshop scheduled in mid-Fall.
B. Participants and Settings
This study will involve 80 second through fifth year teachers within one large school
district and 1200 (15 per teacher) students. Young teachers are selected because they are both
22
likely to be open to new input regarding their teaching methods and because they will be
teaching for many years into the future. First-year teaches are excluded, because the practical
task demands (i.e. writing lesson plans, preparing materials) of the first year of teaching are
typically so great that little energy is available to take in additional input. Teachers will be
randomly assigned within schools to receive the workshop-MTP protocol or standard in-service
training provided by that school district. For ease of implementation and comparison, all teachers
will be in the content area of English/Social Science, although ultimately this approach is seen as
generalizing beyond these areas.
We plan to work with two Virginia public school systems: the Fairfax County Public
Schools (FCPS, see letter of support) with the Hanover County Public Schools (HCPS, see letter
of support) as a back-up. Both districts are large enough to sustain the entire intervention.
Fairfax County Public Schools (FCPS) is the largest school system in the Commonwealth
of Virginia, the 12th largest in the United States, and one of the most diverse. FCPS enrolls large
numbers of immigrants from all over the world, and significant proportions of African American
(11%), Latino (16%) and Asian-American (17%) students, as well as a large (approximately
30%) sample of students on free-reduced school lunch (a marker of family poverty status).
FCPS 21 high schools which typically enroll between 2,000-3,000 students each.
FCPS hires more than 300 new secondary teachers each year, and although rates change
from year to year, the district expects to have between 300-400 secondary teachers with between
2 and 5 years of experience who are also teaching English or Social Studies.
Hanover County Public Schools (HCPS) is smaller than Fairfax but nonetheless suitable
for the study if for any reason planning for implementation in FCPS does not proceed as
expected (our experience in any large-scale research involving school systems is that it is
essential to have alternatives available in case of unforeseen changes in school board policies,
administration, local politics, etc.). HCPS is expected to have between 100 and 150 teachers of
secondary English-Social Studies with between 2 and 5 years of experience next year. The
HCPS system is somewhat more rural than FCPS, and about half the student population is on
free-reduced lunch.
A major advantage of both HCPS and FCPS is that they have collaborated with Pianta on
several other intervention trials, and they have extensive and well-supported data warehousing
capacities so that data collected by the school system on students and on teachers can be made
readily available to the project.
C.
Measures
All data will be obtained for both participating and randomly assigned non-participating
teachers and one classroom of their students in the program. Care will be taken so that
intervention and control teachers are never teaching the same students, so as to minimize any
contamination of the control group. Data collection will occur on several levels.
Classroom environment assessments will consist of reliable coding of observations of
teacher student interactions using the CLASS-S system, along with teacher ratings and aggregate
student ratings of classroom environment using standardized assessment instruments. Student
measures will be aggregated across all assessed students in the classroom for setting-level
analyses.
Observations in classrooms.
23
The Classroom Assessment Scoring System-Secondary version (Pianta et al., 2004) will
be the primary source of standardized observational data on teacher-student interactions in the
classroom. The CLASS is one of the most current and widely-used standardized assessments of
social and instructional interactions in classrooms (NICHD ECCRN, 2002, 2005; McCaslin et
al., 2005). The CLASS-Secondary version, or CLASS-S, is designed to capture precisely those
aspects of classroom interactions that we hypothesize to be resources for adolescent engagement
and motivation. The CLASS-S consists of a set of global 7-point rating scales with behaviorally
anchored scale points providing detailed descriptions of a specific dimension of classroom
process and its scaling from low to high. The CLASS-S scales are organized into two
overarching dimensions, similar to those reported in factor analyses of the elementary version:
Motivational Processes (e.g., Sensitivity, Relational Supports, Autonomy Supports), and
Organizational/Instructional Processes (e.g., Effective Behavior Management, Content
Knowledge, Quality of Feedback, Cognition Supports). A separate scale for Student Engagement
is also used to measure the collective level of student enthusiasm, persistence, and participation.
These scales have been shown to predict student achievement and social outcomes in a number
of studies of large numbers of 5th graders (NICHD ECCRN, 2005) and in recent work with
teachers in secondary settings. All scales are coded by trained coders who have met certification
criteria for reliability. Coding takes place in the Center for Advanced Study of Teaching and
Learning at the University of Virginia. Coders meet weekly to conference difficult questions,
jointly code to maintain reliability, and address logistic and related issues. All coding is done
blind to intervention condition.
Teacher and student-reports of motivational processes.
We have extensive experience using a range of student and teacher report scales to assess
the primary constructs of interest in this study (e.g. perceptions of relational supports,
motivational processes, experience of autonomy and competence). Below we outline in brief a
number of these scales; our selection criterion for each includes evidence of reliability and
validity as well as suitability for use in a brief assessment protocol (to minimize data collection
demands on teacher and student time). Given that a number of the scales below are both brief
and somewhat overlapping in content, we plan to assess their factor structure and consider them
as indicators of broader latent constructs or factors (as outlined further in Analyses, below). For
student scales, we plan to aggregate results within classrooms, both using simple additive
approaches as well as advanced statistical techniques (e.g., HLM), that allow for examination of
the degree to which there is reliability and consensus within a classroom about features of that
classroom (Chan, 1998; Raudenbush, 1991).
We will program the items from these scales into a web-interface so that teachers and
students will complete the instruments by logging onto a secure website and responding to the
items displayed on the site. We have used this approach in the current MTP intervention study
being conducted statewide in Virginia and have extensive experience in the programming and
interface aspects of the task. This approach significantly increases student confidence in the
anonymity of their answers (particularly with respect to their teacher) and also greatly reduces
opportunities for coding/data entry errors.
Relationship climate. We employ a battery of brief measures used successfully in other
research with secondary school students to tap qualities of the classroom relationship climate
being targeted by the MTP intervention. Although this is a sizeable number of measures, each
24
measure is brief and together they inventory a range of the critical aspects of the relationship
climate within the classroom.
Student trust in and obligation to teacher authority (Gregory, 2005, adapted from Tyler
& Degoey, 1995). This 7–item measure uses a 4-point Likert-type scale to assess the extent to
which the student accepts teacher authority and trusts the teacher to use this authority wisely. For
example, students are asked to respond to the statement: “This teacher can be trusted to make
good decisions for everyone in class.” This scale has been linked to student compliance with
authority figures and to teacher perceptions of students as cooperative and not defiant (Gregory,
2005, Tyler & Degoey, 1995). The adapted scale has been found to have an alpha of 0.91
(Gregory, 2005).
Respect from teacher. This 4–item measure uses a 5-point Likert-type scale to assess the
degree to which the student feels respected by a given teacher. (Skinner, et al., 1998; alpha =
.77). For example, students are asked to respond to the statements: “This teacher listens to my
ideas.” “This teacher interrupts me when I have something to say” (reverse coded).
Teacher negative feedback from TTI (Weinstein et al., 1987). This 4–item measure uses a
4-point Likert-type scale to assess the degree to which feedback is communicated by the teacher
in an overly negative or critical manner. Student reports on this measure have been linked to
their trust in and willingness to comply with teacher authority (Gregory, 2005, Wentzel, 2002).
The scale has an alpha of .70 (Wentzel, 2002).
Teacher affection. This 3–item measure uses a 4-point Likert-type scale to assess the
degree to which the student feels cared for by a teacher (Skinner, et al., 1998, alpha= .71). For
example, students are asked to respond to the statements: “This teacher likes me,” and “This
teacher doesn’t seem to enjoy having me in his/her class” (reverse-coded). Gregory (2005)
reports that with high-risk adolescents, perceived teacher caring was a strong predictor of trust in
a teacher’s authority.
Teacher-student relationship scale (Roeser, Midgley, & Urdan, 1996). This 4–item
measure uses a 5-point Likert-type scale to assess the overall quality of the student-teacher
relationship from the students’ perspective. For example, students are asked to respond to the
statement, “In this classroom, the teacher really cares about students as individuals.” Roeser and
colleagues (1996) report an alpha of .81, and links between this scale and adolescents’ sense of
belonging in school which, in turn, predicted academic achievement.
Control/Efficacy/Autonomy. The proposed scales include the following:
Behavioral expectations scale (Gregory & Weinstein, 2004). This scale has 4 items and
addresses classroom order and rule enforcement. It has been adapted from a study which found
in a national longitudinal sample that student-reported behavior expectations in the school
predicted significant growth in math achievement across the years of high school (Gregory &
Weinstein, 2004). Items in the adapted scale include “This teacher makes sure the class is
orderly” and “Misbehaving students often get away with it” (reverse-coded) Gregory (2005)
found it to have adequate internal consistency (alpha = 0.65).
Academic press (Midgley et al., 2000). This 6-item scale assesses students’ perceptions
that their teacher presses them to achieve understanding of classroom material, using a 5-point
Likert-type scale ranging from 1= “Not at all true” to 5 = “Very true.” It has been found to have
25
both strong psychometric properties (Midgley et al., 2000) and links to student outcomes and
motivation (Middleton, 2002).
Classroom promotion of adolescent autonomy and relatedness (Allen et al., 1994b).
Twelve items, utilizing a 4-point scale, assess the extent to which the classroom was viewed as
promoting adolescent autonomy and relatedness with other students and teachers. Items tap
youths’ perceptions of the extent to which their teacher likes them and listens and takes into
account what they have to say. A teacher version of this questionnaire contains the same items as
the student version and asks teachers to fill it out as they thought their class on average would fill
it out. This measure is adapted from scales (α = .87) previously used in the evaluation of a
pregnancy and dropout prevention program and found to predict program efficacy in reducing
future risks in both areas (Allen et al., 1994b).
Relevance. We plan to use the following well-validated measure.
Relevance of school for future success. This 6-item scale assesses students’ beliefs that
doing well in school will (vs. will not) help them achieve success in the future, using a 5-point
Likert-type scale ranging from 1= “Not at all true” to 5 = “Very true.” This scale uses a reversecoding of the “skepticism about the relevance of school for future success” scale developed and
found to have strong psychometric properties by Midgley et al. (2000) and links to student
outcomes and motivation (Middleton, 2002). Example item: “Doing well in school won’t help
me have a satisfying career when I grow up.”
Student Motivation/Effort. These will be assessed from both teacher and classroom
aggregated-student rating scales, from repeated observational assessments based on videotapes
of classroom interactions, and from hard behavioral indices.
Student mastery motivation. This 5-item scale assesses students’ emphasis upon mastery
of academic material on a 5-point Likert-type scale ranging from 1= “Not at all true” to 5 =
“Very true.” The scale was found to have strong psychometric properties (Midgley et al., 2000)
and links to student outcomes and motivation (Middleton, 2002). Example item: “It’s important
to me that I thoroughly understand my class work.”
Educational aspirations. Cook et al. (1996) devised a simple and effective instrument to
measure educational aspirations. The validity of this instrument, which consists of just three
items, was demonstrated in a study showing significant improvement in aspirations among boys
participating in the New Hope anti-poverty program (Huston et al., 2001).
Student academic efficacy (Midgley et al., 2000). This 5-item scale focuses on students’
perceptions of their competence to do their class work. It displays good internal consistency,
using a 5-point Likert-type scale ranging from 1= “Not at all true” to 5 = “Very true.” Example
item, “Even if the work is hard, I can learn it.” It has been linked to independently assessed
teacher behavior (Middleton, 2002).
Cheating behavior (Midgley et al., 2000). This 3-item scale assesses students’ use of
cheating in class on a 5-point Likert-type scale ranging from 1= “Not at all true” to 5 = “Very
true.” It has been found to have strong psychometric properties and links to student outcomes
and motivation (Middleton, 2002). Example item: “I sometimes cheat on my class work.”
Disruptive behavior. This 5-item scale assesses students’ engagement in behaviors that
disrupt or disturb the classroom, on a 5-point Likert-type scale ranging from 1= “Not at all true”
to 5 = “Very true.” Found to have strong psychometric properties (Midgley et al., 2000) and
26
links to student outcomes and motivation (Middleton, 2002). Example item: “I sometimes don’t
follow my teacher’s directions during class.”
Behavioral indicators of student engagement. These will include administrative records
on classroom tardiness and disciplinary referrals based on classroom behavior, and teacher
ratings of student homework completion and effort on assignments.
Student Engagement. We will use both self-report and observational indicators to
measure students’ engagement in classroom processes and learning.
Student engagement. Engagement concerns how much the adolescent feels a sense of
belonging in school. This will be measured with What My School Is Like, measure, consisting of
five questions from the New Hope Study (Huston et al., 2001). In that study of 521 low-income
children ages 9-15, internal consistency of the measure was high (α = .84) and validity was
demonstrated by strong correlations with school achievement and other measures of academic
motivation. The questionnaire also includes six questions to assess students’ attachment to their
school and their involvement in extracurricular activities. These questions were taken from a
school “bonding” questionnaire devised by Cernkovich and Giordano (1992) that significantly
predicted adolescents’ juvenile delinquency in that study.
Observed student engagement CLASS-S. The 7-point rating scale for Student Engagement
from the CLASS-S will be used as an observational assessment of overall engagement by
students in the classroom. It is not an individual student-level scale but instead codes
engagement of students at the classroom level, and will function at the classroom level in
analyses. The scale reflects the extent to which student volunteer, display enthusiasm and effort,
persist in addressing learning challenges, and are focused. Reliability is calculated based on
inter-observer agreement, with Kappas above .65 in several studies (see NICHD ECCRN, 2005,
as an example).
Teacher Motivation/Effort. Teachers’ own degree of motivation to teach and the effort
they experience in relation to teaching are important proximal outcomes for the intervention and
also possible mediators or moderators of student outcomes.
Personal teaching efficacy (Midgley et al., 2000). This 7-item scale assesses teacher’s
beliefs they can effectively teach all students, using a 5-point Likert-type scale ranging from 1=
“Strongly Disagree” to 5 = “Strongly Agree.” It has been found to have strong psychometric
properties and links to student outcomes and motivation (Middleton, 2002). Example item: If I
try really hard, I can get through to even the most difficult student.”
Teacher motivation. The classic measure of job involvement and motivation in the field
of organizational behavior will be used to assess teacher motivation (Ruh, White, & Wood,
1975). This 14 item instrument uses 5-point Likert-scales and yields assessment of two related
dimensions: Job involvement, captured with 9 items (e.g., “How much do you look forward to
coming to work each day?”) and Motivation, captured with 5-items (e.g., “How often do you
really want to work hard at your job?”).
Student and Setting Outcomes
Student outcomes will include both those determined by teachers (i.e., course grades),
and objective measures such as standardized achievement test scores (aggregated at the
classroom level) will also be obtained. Because we will be sampling teachers (and students) from
within one school division, we will negotiate access to school records including transcripts,
27
attendance, referrals for special education, disciplinary actions, and performance on annual
standards-based assessments, all of which will be used in analysis of student performance at the
individual, classroom, and school levels, when appropriate.
Both Fairfax County Public Schools and Hanover County Public Schools collect
standardized achievement data from students annually in 9th-12 grades, the grade levels that will
be the focus of this project. They administer the Virginia Standards of Learning tests (SOLs),
which are closely linked to prescribed classroom curricula, at the end of each course/semester in
high school, and the SOLs can serve as a key indicator of academic success of the students in a
value-added modeling approach. More specifically, each student is tested each year in whatever
English/Social Studies course they are taking, and the following year they take the next course in
that sequence (Virginia mandates that students take an English and Social Studies/History course
each year for the full year). Thus we can use the SOLs for each student in the corresponding
subject area from the prior year as a baseline for evaluating relative growth with regard to their
performance in that subject area SOL the subsequent year.
In addition to the state-mandated SOLs, that can be used as described above, each district
uses a set of “benchmarking” tests throughout the year so teachers can gauge students’
performance toward the SOL proficiency levels. We will obtain the scores from these
benchmarking tests (these are standardized achievement tests, some developed by test publishers,
others by district psychometricians) to provide a within-course metric of growth in that subject
area. For example, FCPS administers a district-standardized benchmarking test 3 times per year
prior to the SOLs in specific subject areas (e.g., English and Social Studies), the scores of which
will be available to us through the data warehouse.
In addition, school records will also provide us with information on
principal/administrator evaluations of teacher performance for which we will blindly rate.
Assuming that we are able to obtain permission to do so, we will further ask that school
supervisors/administrators provide information on teacher performance using any of a number of
standardized rating scales that assess their views of teachers’ relationships with students, quality
of instruction, potential for engaging students, and contributions to the school. We currently have
a group of these measures identified and will select final item sets based upon the amount of time
administrators agree to provide for this rating task.
Student and Teacher Characteristics
We will gather demographic information on teachers via a brief questionnaire and on
students (ethnicity, poverty/income) and on classrooms (class size, academic level, etc) directly
from the schools.
We will also assess teachers’ personality using the short form of the NEO Five-Factor
Personality Scale (Costa & McRae, 2003). This measure is currently being used with early
career teachers in the teacher education participant pool at the University of Virginia and
preliminary analyses are showing that it demonstrates good variability and differences across the
five factors as well as an interpretable pattern of correlations with observations of these teachersto-be in student teaching settings.
Intervention Fidelity
Intervention fidelity will be assessed in terms of teacher compliance with requirements to
send in regular video-taped observations, as well as with records of teacher participation in both
28
the workshop and the repeated video-conferencing consultation part of the intervention. Given
our intent-to-treat approach, however, all teachers selected for the intervention will be included
in assessments of the overall efficacy of the intervention; however, follow-up analyses will
consider whether lack of compliance may have reduced apparent program effects for some
teachers. Based on past experience, we expect to find consistent, high levels of compliance with
the intervention, as most teachers find it to be helpful, pleasant, and non-stressful.
Project Timeline
Year 1
_
Task
Spring Summer
Hire/train personnel
Baseline student achievement
assessments
Organize/conduct teacher
workshops (late summer)
MTP interactive training
Ongoing student and teacher
assessments
Cohort 1 student achievement
and post-assessments
Baseline cohort 2 student
achievement assessments
Teacher booster session
Cohort 2 student achievement
and post- assessments
Data coding
Analysis and dissemination
V.
Year 2
_
Year 3
Intervention
Academic Year 1
Intervention
Academic Year 2
Fall
Winter
Fall
Winter








Spring Summer
Spring Summer
_
Fall
Winter
























Data Analysis Plan
First we describe our overall approach to data analysis and then present the specific approaches
that will be used for each of the primary research questions described earlier.
A.
General Analytic Approach
Initial reliability analyses, simple plots of data, and data transformations will first be
conducted to assess the psychometric adequacy of each of the measures being used. We next
begin the process of assessing the adequacy of the initial assignment of variables to constructs
outlined above in the “Measures” section. We begin with the theoretically generated groupings
of measures outlined above, which will then be examined using cluster and factor analytic
techniques and ultimately assessed for their adequacy via confirmatory factor analysis. As part
of this process, we will also determine whether each of the assessed outcome measures provides
independent information. For repeated measures, factor analysis of the initial assessments will
29
be conducted, and the resulting factor structure will then be tested for adequacy with follow-up
assessments using confirmatory factor analysis.
All analyses will proceed recognizing the hierarchically organized nature of the study
data, with multiple observations obtained for students nested within classrooms within schools.
Analyses will also recognize that for many constructs (e.g. observed classroom processes,
student and teacher indicators of motivational processes) we will have repeated assessments
across an academic year. Our primary approach to data analysis utilizes techniques that account
for this multilevel data structure including latent growth curve analyses within both hierarchical
linear models (HLM) (Raudenbush & Bryk, 2002; Burchinal, Bailey, & Snyder, 1994; Singer &
Willett, 2003) and structural equations models (Muthen & Curren, 1998; Curran, Bauer, &
Willoughby, 2005). Growth curves estimated for classroom observations, and student and
teacher characteristics will describe patterns of change over time and will assess whether the
MTP intervention is related to higher student motivation and larger gains in student outcomes.
Hierarchical linear modeling is particularly well-suited for the analysis of these data because it
estimates variance at multiple levels (e.g., within-subject (Level 1 data), between-subjects (Level
2 data), between classrooms (Level 3)) simultaneously, thus allowing for the modeling of each
source of variation while taking into account the statistical characteristics of the others.
B. Primary Analyses
We will begin by addressing the overarching question in this study--is the My Teaching
Partner intervention effective in altering student engagement and motivation within the
classroom—by comparing students and classrooms in the MTP and control conditions on
changing levels of motivational processes across the academic year. The primary outcomes are
measured at both the teacher and student level (with students nested within classrooms).
Direct Program Effects on Teacher Interactions. The effectiveness of the workshop
plus ongoing consultation in altering the interactions within the classroom setting will be
evaluated by assessing the teacher’s classroom behaviors and associated student-teacher
interactions. The type of analysis varies as a function of the amount and type of data collected.
For example, the teacher’s instructional behaviors and implementation are assessed repeatedly
during the intervention year. Video tapes will be sent biweekly to UVA and coded. The
CLASS-S scales will be analyzed using piecewise regressions within hierarchical linear models.
Higher order auto-regressive models will describe patterns of change for each individual to
account for dependencies in the data due to repeated assessments. The model will include as
predictors: treatment group, class size and poverty level, and time of observation. The treatment
effects analyses will test the extent to which changes in the classroom over the course of the year
(coded using CLASS-S) and spring teacher outcomes vary as a function of whether teachers
were randomly assigned to the intervention vs. control group. Covariates will include teacher
and student characteristics. Nesting of children within the classroom and teachers will be
accounted for within these growth curve models via SEM and HLM, as appropriate. Nesting of
classrooms within schools will also be assessed, though this effect is expected to be quite small
after accounting for other covariates in the study (see “Power” section below). Growth curve
analyses will allow us to assess not only whether the endpoint of the intervention at the end of
Year 1 is higher for those in the MTP condition, but also to consider whether and how the
30
intervention has lead to changing patterns of classroom setting characteristics over the course of
the academic year.
Direct Program Effects on Student Outcomes. The most substantively important test
of program effectiveness and setting changes involves assessing change in the actual functional
motivational characteristics of the setting (e.g., does it enhance student motivation) and in related
student achievement outcomes. Student engagement will be assessed repeatedly (at
approximately 15 points, spaced 2-weeks apart) via the CLASS-S system and analyzed using the
same procedures as described above for teacher behaviors. Similarly, self-report student
motivational assessments will be obtained at 4 points during each intervention year (September,
November, February, and April) and analyzed accordingly. Student achievement outcomes on
the Standards of Learning tests will be collected in the spring of the year prior to the intervention
and again in the spring of the intervention year and will consist of standardized-test performance
and related pre- and post- indicators. In addition we have the within-year benchmarking tests
also available for use in growth modeling.
Mediated Effects. Following initial analyses of overall intervention effects, we will
conduct a series of analyses designed to identify critical process mediators of intervention effects
(Questions 2-4 from the Specific Questions section). Although such analyses can obviously take
place and provide real value even in the absence of treatment group effects, they will have the
greatest value as a way of beginning to identify the critical “active ingredients” that underlie a
successful MTP intervention.
These follow-up analyses will focus upon assessing which aspects of the classroom
setting, as assessed via the CLASS-S observation system, best predict changes in actual levels of
student engagement, and student (and teacher) motivation in the classroom. Thus, in the
presence of MTP effects, we will explicitly examine the hypothesis that motivational processes
are the setting-level features most affected by the “active ingredients” of the program (Question
2). Similarly, we will consider the extent to which changes in the instructional/organizational
qualities of the classroom might also contribute to changes in student engagement and
motivation within it (Question 3). Even in the absence of strong programmatic effects, it will be
possible to assess the validity of the theoretical model presented and to refine and improve the
program over time, given what we expect to be considerable variation across classrooms in
constructs depicted in our model. Tests of mediation will be conducted by determining whether
including the across-time mean CLASS Motivational and Organizational/Instructional Processes
scores account for the observed differences in changing levels of student motivation. In essence,
we will be observing the extent to which teachers enact the motivational and
instructional/organizational behaviors identified as important in the CLASS-S system, and then
observing the extent to which these behaviors are linked to qualities of student engagement and
motivation in the classroom.
Next, we will consider one of the central propositions underlying the proposed
intervention: that changes in levels of student engagement and motivation will actually “flow
through” to higher levels of student achievement over the course of an academic year (Question
4). We will utilize the same analytic approach described above, but now apply it to considering
whether our indicators of student engagement within the classroom and of student and teacher
motivation predict student achievement outcomes and mediate the effects of MTP-induced
changes in classroom motivational and instructional/organizational qualities on student
outcomes.
31
We also recognize that the MTP intervention may alter different aspects of classroom
characteristics at different rates (i.e. teachers may quickly learn how to make the relevance of
material clear to students, whereas learning to with students may take place over the course of
the year). We plan to use these asynchronies as further opportunities to break into the “black
box” of the intervention to examine how it works. Although such analyses are obviously
exploratory, we will be able to at least begin to assess which short-term changes that occur via
the MTP program best predict future short-term changes in student motivation. This approach
thus adds a second means of testing mediation. Rather than simply examining how differences
in classroom characteristics across classrooms are related to student change (a valuable strategy,
but one that introduces additional confounds), we will also be able to look at how differences in
classroom characteristics within the same classroom, examined at different points in time, are
linked to short-term changes in student motivation. This approach also has its limitations, but
when combined with the more traditional between-classroom approach to assessing mediation
described above, it increases the likelihood that we will be able to triangulate in on a stronger
understanding of the critical aspects of the MTP intervention.
Moderator Effects. All analyses will begin with the assumption that outcomes may be
influenced by larger contextual factors (i.e. MTP may have a greater impact with some
populations of students or within some types of classrooms), and these moderator effects will be
explicitly considered in analyses (Question 5). In addition, a final set of analyses will add
implementation fidelity, and teacher knowledge and attitudes as mediators and moderators of
treatment effects.
Year 2 Analyses. In Year 2 of the intervention, a similar analytic strategy will be
employed, although we are expecting less change over the course of the year in classroom
characteristics (because the intensive period of the intervention is focused on Year 1). We will
nevertheless conduct the parallel analyses to all of those described above, being particularly alert
to any signs of decay of the intervention (and its implications for outcomes) in this follow-up
year. Although decay of intervention effects is clearly undesirable, any such decay observed,
when linked with accompanying changes in student outcomes, can provide still further evidence
about the workings of the intervention and the areas to be targeted with future follow-ups and
booster programs. From this perspective, even when the natural (if undesirable) decay process
occurs, it will still at least further our understanding of the intervention and of areas to target to
prevent such decay in the future.
C.
Attrition Effects and Missing Data
We work toward a goal of “zero-attrition-tolerance”: i.e., to keep missing data in any
given year to an absolute minimum, and have had significant success with this in our evaluation
of MTP to date. To the extent that the amount of attrition is non-trivial, assessments of the nature
of missingness (e.g., missing at random, missing completely at random, etc.) and its impact will
be made using current techniques for modeling incomplete longitudinal data within the analytic
framework described above (McArdle & Hamagami, 2001). More generally, we will handle
missing data with what are becoming increasingly standard procedures involving full maximum
likelihood analyses (for “MAR” data), and by including covariates (for variables systematically
predictive of missing data), and with multiple imputation procedures. We will carry out these
analyses with Mplus where procedures for missing data imputation are readily available
(Arbuckle, 1996; Little & Rubin, 1987; B. Muthen et al., 1987; L. K. Muthen & Muthen, 2001;
Rubin, 1987). In all cases, we will consider results of analyses both with and without special
32
handling of missing data, so as to assess the generality of our results across statistical
methodologies.
D.
Power Analyses
Statistical power for the proposed study was assessed using Optimal Design software
(Raudenbush et al., 2005b) and final estimates were reviewed in consultation with the William
T. Grant Foundation statistical consulting group for intervention research (Raudenbush, personal
communication). Given that power analyses for a multi-level design such as that proposed can
be inordinately complex, we present an example of a conservative estimate of our power for one
of our primary analyses--the impact of the classroom setting upon student achievement—and
then note ways in which other similar analyses might deviate from these estimates. Our analyses
proceed under the assumption that 80% of the variance in post-test results (at the classroom
level) will be predicted by pre-test results, and that 85% of variance will be between students
within classrooms and 15% will be between classrooms. All of these assumptions are considered
in the reasonable to conservative range based upon actual data collected in secondary schools
(Bloom et al., 2005; Raudenbush et al., 2005a; Raudenbush, personal communication). We
recognize there will also be an additional nesting of classrooms within schools, which may or
may not have a substantive impact on our findings. Given that classroom randomization will
occur within schools, that the number of schools will be small, and that the intervention does not
at all proceed at the school level, or use school personnel beside the individual teacher involved,
school effects over and above classroom effects and student background factors may well be
negligible; however, we will test this assumption once data are collected. For purposes of a
conservative power estimate, we assume a standard error of effect size of .07 at the school level
(equivalent to a school-level effect size variance of .005 ). We test for a modest effected expect
size of .25, which, again, is considered reasonable to conservative for this type of research, and
within the range of “small” effects as described by Cohen (Bloom et al., 2005; Cohen, 1988;
Raudenbush et al., 2005a).
Under these assumptions, we find we have 93% power to detect our modest expected
effect size of .25. Even for a very small effect size (.20), we still have greater than 80% power,
thus leaving a margin of safety in case of a modest loss in sites across an academic year. If
within school effects turn out to be negligible and are able to be disregarded, power rises
accordingly. We also expect that multiple-repeated assessments of critical student and classroom
characteristics (e.g., student motivation, classroom characteristics) will yield more precise
estimates of these constructs, thus increasing the overall power for analyses based upon them.
At the same time, we recognize that power to detect moderator effects will be somewhat lower.
As a result, only larger moderator effects will be likely to be detected—although it is only when
moderating effects are sizeable that they become of significant substantive interest to the study.
E.
Dissemination
We intend to disseminate results of this proposed study simultaneously in several
different venues. The basic scholarly results from the project are expected to be of interest to toptier scholarly journals in fields of education, child development, and community psychology.
The two principal investigators on this proposal have an extensive record of publishing results in
the highest quality journals in these fields and would expect to do so with results from this
inquiry. Importantly, both investigators are also involved with national organizations that
provide direct and wide access of findings to large numbers of possible consumers. For example,
33
Allen, a former William T. Grant Faculty Scholar, is an elected Governing Council Member of
the Society for Research in Adolescence and is also Associate Editor of Child Development.
Pianta is closely involved with the Carnegie-supported network of teacher education programs
through the Teachers for a New Era initiative, as well as part of an NICHD-NCATE initiative
aimed at identifying and disseminating knowledge about development to teacher educators.
Pianta is also a member of the Society for Prevention Research and Institute of Education
Sciences investigator networks.
The expected practical results of the study—an enhanced and further validated version of
the My Teaching Partner program (enhanced iteratively based upon results of this research)—
also will be disseminated to school districts and to teacher-training programs in several ways,
relying on the established network of dissemination and training contacts available through the
Center for Advanced Study of Teaching and Learning.
CASTL has an extensive network of contacts in the field at the state and district level
(e.g. VA, WV, MA, WY) as a consequence of training and professional development
partnerships. CASTL maintains an active connection with that network through its website and
newsletter, and linked participants will receive practitioner- and trainer-friendly versions of
results. CASTL is currently working with several states implementing pilot trials of professional
development interventions such as the one proposed in for this project and information about the
findings and the methods of professional development will be disseminated through that vehicle
as well. CASTL is also a member of the Carnegie Corporation’s Teachers for a New Era
Initiative, a network of 11 teacher-education institutions that meets regularly and through which
results will be disseminated and procedures made available. Importantly this network will make
available the opportunity to extend and replicate the results of this work using teachers-intraining.
Finally, the PI's and other CASTL investigators are members of several research
networks involving investigators from several other institutions and are regular consultants to a
number of agencies, non-profits, and Foundations that will receive reports of the results of the
study. We expect this study to result in a further refined and improved manual for intervention
and coding system, as well as updated web-based training materials. These materials will be
suitable for widespread dissemination, including to other school districts with which we may not
even be actively working at the time. In this regard, the intervention has the potential both to
continue to grow in controlled fashion with careful evaluations, but also to start to be
disseminated far more broadly as well.
In sum, the PI's expect the proposed project to yield a well-validated, highly refined
approach to classroom-level intervention. We then intend both to move this approach up the
next step in scale, gaining experience in far larger, yet still manageable, dissemination
procedures, while simultaneously making its results and basic methods fare more broadly
available to a national audience across several disciplines.
34
References
Allen, J. P., Hauser, S. T., Bell, K. L., & O'Connor, T. G. (1994a). Longitudinal assessment of
autonomy and relatedness in adolescent-family interactions as predictors of adolescent ego
development and self-esteem. Child Development, 65(1), 179-194.
Allen, J. P., Kuperminc, G., Philliber, S., & Herre, K. (1994b). Programmatic prevention of
adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in
the teen outreach program. American Journal of Community Psychology, 22(5), 617-638.
Allen, J. P., Marsh, P., McFarland, C., McElhaney, K. B., Land, D. J., Jodl, K. M., et al. (2002).
Attachment and autonomy as predictors of the development of social skills and delinquency
during midadolescence. Journal of Consulting & Clinical Psychology, 70(1), 56-66.
Allen, J. P., Moore, C., Kuperminc, G., & Bell, K. (1998). Attachment and adolescent
psychosocial functioning. Child Development, 69(5), 1406-1419.
Allen, J. P., Philliber, S., Herrling, S., & Kuperminc, G. P. (1997). Preventing teen pregnancy
and academic failure: Experimental evaluation of a developmentally based approach. Child
Development, 68(4), 729-742.
Anderman, L. H., & Midgley, C. (1998). Motivation and middle school students. ERIC Digest,
EDO-PS-98-5.
Arbuckle, J. L. (1996). Full information estimation in the presence of incomplete data. In G. A.
Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling: Issues and
techniques (pp. 243-277). Mahwah, N.J.: Lawrence Erlbaum Associates.
Bandura, A., Barbaranelli, C., Caprara, G. V., & Pastorelli, C. (1996). Multifaceted impact of
self-efficacy beliefs on academic functioning. Child Development, 67(3), 1206-1222.
Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-efficacy and intrinsic interest
through proximal self-motivation. Journal of Personality and Social Psychology, 41, 586598.
Barzun, J. (2000). From dawn to decadence: 500 years of western cultural life 1500 to the
present. London: Harper Collins.
Baumeister, R. F., Twenge, J. M., & Nuss, C. K. (2002). Effects of social exclusion on cognitive
processes: Anticipated aloneness reduces intelligent thought. Journal of Personality and
Social Psychology, 817-827.
Berndt, T. J., & Keefe, K. (1996). Friends' influence on school adjustment: A motivational
analysis. In J. Juvonen & K. R. Wentzel (Eds.), Social motivation: Understanding children's
school adjustment. Cambridge studies in social and emotional development (pp. 248-278).
New York: Cambridge University Press.
Bishop, J., Bishop, M., Gelbwasser, L., Green, S., & Zuckerman, A. (2003). Nerds and freaks: A
theory of student culture and norms. In D. Ravitch (Ed.), Brookings papers on education
policy: 2003. Washington, DC: Brookings Institution Press.
Bloom, H. S., Richburg-Hayes, L., & Black, A. R. (2005). Using covariates to improve
precision: Empirical guidance for studies that randomize schools to measure the impacts of
educational interventions. New York: Manpower Development Research Corporation.
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and
design. Cambridge, MA: Harvard University Press.
Bryant, A. L., Schulenberg, J., Bachman, J. G., O'Malley, P. M., & Johnston, L. D. (2000).
Understanding the links among school misbehavior, academic achievement, and cigarette
use: A national panel study of adolescents. Prevention Science, 1(2), 71.
Bryk, A. S., Lee, V. E., & Holland, P. B. (1993). Catholic schools and the common good.
35
Bryk, A. S., & Driscoll, M. (1988). The high school as a community: Contextual
influences and consequences for teachers. Madison, WI: University of Wisconsin, National
Center on Effective Secondary Schools.
Burchinal, M. R., Bailey, D. B., & Snyder, P. (1994). Using growth curve analysis to evaluate
child changes in longitudinal investigations. Journal of Early Intervention, 18, 422-442.
Cairns, R. B., Cairns, B. D., & Neckerman, H. J. (1989). Early school dropout: Configurations
and determinants. Child Development, 60, 1437-1452.
Carbona, W. J., & Gamoran, A. (2002). The production of achievement inequality in high school
English. American Educational Research Journal, 39, 801-827.
Cassidy, D. J., Buell, M. J., Pugh-Hoese, S., & Russell, S. (1995). The effect of education on
child care teachers’ beliefs and classroom quality: Year one evaluation of the TEACH Early
Childhood Associate Degree Scholarship Program. Early Childhood Research Quarterly, 10,
171-183.
Cernkovich, S. A., & Giordano, P. C. (1992). School bonding, race, and delinquency.
Criminology, 30(2), 261-289.
Chan, D. (1998). Functional relations among constructs in the same content domain at different
levels of analysis: A typology of composition models. Journal of Applied Psychology, 83(2),
234.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Coleman, J. S. (1961). The adolescent society: The social life of the teenager and its impact on
education. New York: Free Press.
Collins, W. A., & Repinski, D. J. (1994). Relationships during adolescence: Continuity and
change in interpersonal perspective. In R. Montemayor, G. Adams, & T. P. Gullotta (Eds.),
Personal relationships during adolescence (pp. 7-36). San Francisco: Sage Publications.
Connell, J. P., & Wellborn, J. G. (1991). Competence, autonomy, and relatedness: A
motivational analysis of self-system processes. In M. Gunnar & L. A. Sroufe (Eds.), Self
processes in development: Minnesota symposium on child psychology, vol. 23 (pp. 43-77).
Hillsdale, NJ: Erlbaum.
Cook, T. D., Church, M. B., Ajanaku, S., Jr., Shadish, W. R., Jeong-Ran, K., & Cohen, R.
(1996). The development of occupational aspirations and expectations among inner-city
boys. Child Development, 67, 3368-3385.
Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and the
NEO Five-Factory Inventory (NEO-FFI): Professional manual. Odessa, FL: Psychological
Assessment Resources, Inc.
Cowen, E. (1999). In sickness and in health: Primary prevention’s vows revisited. In D. Cicchetti
& S. L. Toth (Eds.), Rochester Symposium on Developmental Psychopathology, Volume 9:
Developmental Approaches to Prevention and Intervention (pp. 1-24). Rochester, NY:
University of Rochester Press.
Crosnoe, R. (2000). Friendships in childhood and adolescence: The life course and new
directions. Social Psychology Quarterly, 63, 377-391.
Crosnoe, R. (2001). Academic orientation and parental involvement in education during high
school. Sociology of Education, 74, 210-230.
Crosnoe, R., Cavanagh, S., & Elder, G. H., Jr. (2003). Adolescent friendships as academic
resources: The intersection of friendship, race, and school disadvantage. Sociological
Perspectives, 46(3), 331-352.
Crosnoe, R., Johnson, M. K., & Elder, G. H., Jr. (2004). Intergenerational bonding in school: The
behavioral and contextual correlates of student-teacher relationships. Sociology of Education,
77(1), 60-81.
36
Csikszentmihalyi, M. (1988). The flow experience and its significance for human
psychology. New York: Cambridge University Press.
Csikszentmihalyi, M. (2000). Beyond boredom and anxiety: Experiencing flow in work and play.
San Francisco: Jossey-Bass, Inc.
Csikszentmihalyi, M., & Schneider, B. (2000). Becoming adult: How teenagers prepare for the
world of work. New York: Basic Books.
Curran, P. J., Bauer, D. J, & Willoughby, M. T. (2004). Testing main effects and interactions in
latent curve analysis. Psychological Methods, 9(2), 220-237.
Darling-Hammond, L., Holtzman, D. J., Gatlin, S. J., & Heilig, J. V. (2005). Does teacher
preparation matter? Evidence about teacher certification, Teach for America, and teacher
effctiveness. Education Policy Analysis Archives, 13(42). Retrieved February 2, 2006 from
http://epaa.asu.edu/epaa/v13n42/.
Deci, E. L. (1989). Self-determination in a work organization. Journal of Applied Psychology,
74(4), 580.
Deci, E. L. (1991). Motivation and education: The self-determination perspective. Educational
Psychologist, 26(3), 325.
Dornbusch, S. M., Glasgow, K. L., & Lin, I-C. (1996). The social structure of schooling. Annual
Review of Psychology, 47, 401-429.
Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 10401048.
Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and
personality. Psychological Review, 256-273.
Ecles, J., Lord, S., & Midgley, C. (1991). What are we doing to early adolescents? The impacts
of educational contexts on early adolescents. American Educational Journal, August, 521542.
Eccles, J. S., Lord, S. E., Roeser, R. W., Barber, B. L., & Jozefowicz, D. M. H. (1997). The
association of school transitions in early adolescence with developmental trajectories during
high school. In J. Schulenberg, J. L. Maggs, & K. Hurrelmann (Eds.), Health risks and
developmental transitions during adolescence (pp. 283-321). New York: Cambridge.
Eccles, J. S., Midgley, C., Wigfield, A., Buchanan, C. M., Reuman, D., Flanagan, C., et al.
(1993). Development during adolescence: The impact of stage-environment fit on young
adolescents' experiences in schools and in families. American Psychologist, 48(2), 90-101.
Eccles, J. S., & Roeser, R. W. (2003). Schools as developmental contexts. In G. R. Adams & M.
D. Berzonsky (Eds.), Blackwell handbook of adolescence. Blackwell handbooks of
developmental psychology (pp. 129-148). Malden, MA: Blackwell Publishers.
Felner, R., Favazza, A., Shim, M., & Brand, S. (2001). Whole school improvement and
restructuring as prevention and promotion: Lessons from project STEP and the project on
high performance learning communities. Journal of School Psychology, 39, 177-202.
Ferrari, J. R. (2004). Mentors in life and at school: Impact on undergraduate protege perceptions
of university mission and values. Mentoring and Tutoring, 12(3), 295-305.
Finn, J. D., & Rock, D. A. (1997). Academic success among students at risk for school failure.
Journal of Applied Psychology, 82(2), 221-234.
Gottfredson, D. C. (2001). Schools and delinquency. Cambridge, UK: Cambridge University
Press.
Gregory, A. (2005, June). School climate and implementation of a preventive intervention.
Poster prsentd in a multi-level modeling symposium at the biennial meeting of the society for
community Research and Action, University of Illinois, Urbana-Champaign, IL.
37
Gregory, A., & Weinstein, R. S. (2004). Connection and regulation at home and in
school: Predicting growth in achievement for adolescents. Journal of Adolescent Research.
V. 19, 4. 405-427.
Hadden, S., Dudding, C., Chase, J., & Pianta, R. (2006) MyTeachingPartner: An innovative
approach to professional development for early childhood educators. Unpublished
manuscript, University of Virginia, Charlottesville, VA.
Hamre, B. K., & Pianta, R. C. (2005). Can instructional and emotional support in the first grade
classroom make a difference for children at risk of school failure? Child Development, 76(5),
949-967.
Hart, P., Stroot, S., Yinger, R., & Smith, S. (2005). Meeting the teacher education accountability
challenge: A focus on novice and experienced teacher studies. Mount Vernon, OH: Teacher
Quality Partnership.
Hartup, W. W., & Stevens, N. (1997). Friendships and adaptation in the life course.
Psychological Bulletin, 121, 355-370.
Howes, C., Burchinal, M., Pianta, R., Bryant, D., Early, D., Clifford, R., & Barbarin, O. (in
press). Ready to learn? children’s pre-academic achievement in pre-kindergarten programs.
Developmental Psychology.
Howes, C., Galinsky, E., & Kontos, S. (1998). Child care caregiver sensitivity and attachment.
Social Development, 4, 44-61.
Huston, A. C., Duncan, G. J., Granger, R., Bos, J., McLoyd, V., Mistry, R., et al. (2001). Workbased antipoverty programs for parents can enhance the school performance and social
behavior of children. Child Development, 72, 318-336.
Lee, V. E., & Burkam, D. T. (2003). Dropping out of high school: The role of school
organization and structure. American Educational Research Journal, 40(2), 353-393.
Liang, B., Tracy, A. J., Taylor, C. A., & Williams, L. M. (2002). Mentoring college-age women:
A relational approach. American Journal of Community Psychology, 30(2), 271-288.
Little, R. J., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: Wiley.
Madon, S., Jussim, L., & Eccles, J. (1997). In search of the powerful self-fulfilling prophecy.
Journal of Personality and Social Psychology, 791-809.
Marks, H. M. (2000). Student engagement in instructional activity: Patterns in the elementary,
middle, and high school years. American Educational Research Journal, 37(1), 153-184.
McArdle, J. J., & Hamagami, F. (2001). Latent difference score structural models for linear
dynamic analyses with incomplete longitudinal data. In L. M. Collins & A. G. Sayer (Eds.),
New methods for the analysis of change. (pp. 139-175). Washington, DC: American
Psychological Association.
McCaslin, M., Burross, H. L., & Good, T. L. (2005, January 2). Change and continuity in student
achievement from grades 3 to 5: A policy dilemma. Education Policy Analysis Archives,
13(1). Retrieved February 2, 2006 from http://epaa.asu.edu/epaa/v13n1/.
Middleton, M. J. (2002). Beyond motivation: Middle school students' perceptions of press for
understanding in math. Contemporary Educational Psychology, 27(3), 373.
Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., Gheen,
et al. (2000). Manual for the patterns of adaptive learning scales (pals). Ann Arbor, MI:
University of Michigan.
Muthen, B. O., & Curran, P. J. (1997). General longitudinal modeling of individual differences
in experimental designs: A latent variable framework for analysis and power estimation.
Psychological Methods, 2, 371-402.
Muthen, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are
not missing completely at random. Psychometrika, 52(3), 431-462.
38
Muthen, L. K., & Muthen, B. O. (2001). Mplus: Statistical analysis with latent
variables. User's guide. Unpublished manuscript, Los Angeles, CA.
National Center for Education Statistics. (2003a). Overview and inventory of state education
reforms: 1990-2000. Washington, DC: U.S. Department of Education, Institute of Education
Sciences.
National Center for Education Statistics. (2003b). The condition of education 2003. Washington,
DC: U.S. Department of Education, Institute of Education Sciences.
National Research Council. (2002). Achieving high educational standards for all. Washington,
DC: National Academy Press.
National Research Council. (2004). Engaging schools: Fostering high school students'
motivation to learn. Washington, DC: National Academies Press.
NICHD Early Child Care Research Network [ECCRN]. (2002). The relation of global first-grade
classroom environment to structural classroom features and teacher and student behaviors.
The Elementary School Journal, 102(5), 367-387.
NICHD Early Child Care Research Network. (2004). Social functioning in first grade:
Associations with earlier home and child care predictors and with current classroom
experiences. Child Development, 75, 1639-1662.
NICHD Early Child Care Research Network. (2005). A day in third grade: A large-scale study of
classroom quality and teacher and student behavior. The Elementary School Journal, 105,
305-323.
NICHD Early Child Care Research Network. (2006). Experiences in fifth grade classrooms and
stability from first and third grade. Manuscript in preparation. University of Virginia,
Charlottesville.
Nye, B., Konstantopoulos, S., & Hedges, L. (2004). How large are teacher effects? Educational
Evaluation and Policy Analysis, 26, 237-257.
Phillips, M. (1997). What makes schools effective? A comparison of the relationships of
communitarian climate and academic climate to mathematics achievement and attendance
during middle school. American Educational Research Journal, 34(4), 633.
Pianta, R. C. (1999). Enhancing Relationships Between Children and Teachers. Washington,
D.C.: American Psychological Association.
Pianta, R. (2005). Standardized observation and professional development: A focuson
individualized implementation and practices. In M. Zaslow and I. Martinez-Beck (Eds.),
Critical issues in early childhood professional development (pp. 231-254). Baltimore, Paul
H. Brookes Publishing.
Pianta, R. (2006a). Professional development interventions to support effective teaching.
Manuscript in preparation. University of Virginia, Charlottesville.
Pianta, R. (2006b). Schools, schooling, and developmental psychopathology. In D. Cicchetti and
D. Cohen (Eds.), Developmental Psychopathology, Volume 1: Theory and Method (pp. 494529). Hoboken, NJ: John Wiley & Sons, Inc.
Pianta, R. C., Hamre, B., & Stuhlman, M. (2002). How schools can do better: Fostering stronger
connections between teachers and students. In J. E. Rhodes (Ed.), New Directions for Youth
Development: A Critical View of Youth Mentoring (pp. 91-107). San Francisco: Jossey-Bass.
Pianta, R., Kinzie, M., Pullen, P., Justice, L., Lloyd, J., & Fan, X. (2003). Web training: Pre-k
teachers, literacy, and relationships. Grant #1 R01 HD046061-01. Washington, DC:
National Institute of Child Health and Human Development.
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2004). Classroom Assessment Scoring System
[CLASS]. Unpublished measure, University of Virginia.
39
Public Agenda. (1997). Getting by: What American teenagers really think about their
schools. New York: Author.
Raudenbush, S. W. (1991). A multilevel, multivariate model for studying school climate with
estimation via the em algorithm and application to us high-school data. Journal of
Educational Statistics, 16(4), 295.
Raudenbush, S., & Bryk, A. (2002). Hierarchical lineral model: Applications and data analysis
methods (2nd ed.). New York: Sage.
Raudenbush, S. W., Martinez, A., & Spybrook, J. (2005a). Strategies for improving precision in
group-randomized experiments. New York: William T. Grant Foundation.
Raudenbush, S. W., Spybrook, J., Liu, X., & Congdon, R. (2005b). Optimal design for
longitudinal and multilevel research, version 1.55, software documentation. New York:
William T. Grant Foundation.
Resnick, M. D., Bearman, P. S., Blum, R. W., Bauman, K., Harris, K. M., Jones, J., Tabor, J., et
al. (1997). Protecting adolescents from harm: Findings from the National Longitudinal
Study of Adolescent Health. Journal of the American Medical Association, 278, 823-832.
Rhodes, S., & Hennessy, E. (2000). The effects of specialized training on caregivers and children
in early-years settings: An evaluation of the foundation course in playgroup practice. Early
Childhood Research Quarterly, 15, 559-576.
Roeser, R. W., & Eccles, J. S. (1998). Adolescents' perceptions of middle school: Relation to
longitudinal changes in academic and psychological adjustment. Journal of Research on
Adolescence, 8(1), 123-158.
Roeser, R. W., Eccles, J. S., & Sameroff, A. J. (1998). Academic and emotional functioning in
early adolescence: Longitudinal relations, patterns, and prediction by experience in middle
school. Development & Psychopathology, 10(2), 321-352.
Roeser, R. W., & Eccles, J. S., & Sameroff, A. J. (2000). School as a context of early
adolescents’ academic and social-emotional development: a summary of research findings.
The Elementary School Journal, 100, 443-471.
Roeser, R. W., Midgley, C., & Urdan, T. C. (1996). Perceptions of the school psychological
environment and early adolescents' psychological and behavioral functioning in school: The
mediating role of goals and belonging. Journal of Educational Psychology, 408-422.
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: J. Wiley and
Sons.
Ruh, R. A., White, K., & Wood, R. R. (1975). Job involvement, values, personal background,
participation in decision making, and job attitudes. Academy of management journal, 8, 300312.
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic
motivation, social development, and well-being. American Psychologist, 55(1), 68-78.
Sarason, S. B. (1982). The culture of the school and the problem of change. (2nd ed.) Boston:
Allyn and Bacon.
Seidman, E., Chesir, D., Friedman, J. L., Yoshikawa, H., Aber, J. L., Allen, L., et al. (2001). The
risk and protective functions of perceived family and peer microsystems among urban
adolescents in poverty. Manuscript submitted for publication, New York University.
Shernoff, D. J., Csikszentmihalyi, M., Schneider, B., & Shernoff, E. S. (2003). Student
engagement in high school classrooms from the perspective of flow theory. School
Psychology Quarterly, 158-176.
Shouse, Roger C. ( 1996). Academic press and sense of community: Conflict, congruence, and
implications for student achievement. Social Psychology of Education. Vol. 1, No. 1, 47-68.
40
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling
change and event occurrence. New York: Oxford Press.
Skinner, E. A., Zimmer-Gembeck, M. J., & Connell, J. P. (1998). Individual differences and the
development of perceived control. Monographs of the Society for Research in Child
Development, 63(2-3).
Slavin, R. E. (1996). Research on cooperative learning and achievement: What we know, what
we need to know. Contemporary Educational Psychology, 21(1), 43-69.
Sroufe, L. A., Egeland, B., Carlson, E. A., & Collins, W. A. (2005). The development of the
person: The Minnesota study of risk and adaptation from birth to adulthood. New York: The
Guilford Press.
Steinberg, L. (2001). We know some things: Parent-adolescent relationships in retrospect and
prospect. Journal of Research on Adolescence, 11(1), 1-19.
Steinberg, L., Brown, B. B., & Dornbusch, S. M. (1996). Beyond the classroom: Why school
reform has failed and what parents need to do. New York: Simon and Schuster.
Twenge, J. M., Catanese, K. R., & Baumeister, R. F. (2002). Social exclusion causes selfdefeating behavior. Journal of Personality & Social Psychology, 83(3), 606-615.
Tyler, T. R., & Degoey, P. (1995). Trust in organizational authorities: The influence of motive
attributions on willingness to accept decisions. In R. M. Kramer & T. R. Tyler (Eds.), Trust
in Organizations (pp. 331-356). Thousand Oaks, CA: Sage.
Weinstein, M. C., Coxson, P. G., Williams, L. W., Pass, T. M., Stason, W. B., & Goldman, L.
(1987). Forecasting coronary heart disease incidence, mortality, and cost: The coronary heart
disease policy model. American Journal of Public Health, 77(11), 1417-1426.
Wentzel, K. (2002). Are effective teachers like good parents? Teaching styles and student
adjustment in early adolescence. Child Development, 73(1), 287-301.
Whitehurst, G. J. (2002). Research on teacher preparation and professional development. White
House Conference on Preparing Tomorrow’s Teachers. Washington, DC.
www.ed.gov/adminis/tchrqual/learn/preparingteachersconference/whithurst.html