Teacher Classroom Management
Practices: Effects on Disruptive
or Aggressive Student Behavior
2011 Society for Research on Educational
Effecti eness Conference
Effectiveness
Confe ence
Regina M. Oliver
Vanderbilt University
A k
Acknowledgements
l d
t

Co-Authors



Joseph H. Wehby, Vanderbilt University
D i l J.
Daniel
J Reschly,
R
hl Vanderbilt
V d bil University
U i
i
S
Special
i l Th
Thanks
k to
t

Mark Lipsey
Obj ti
Objectives
off Review
R i



Examine the effects of teachers
teachers’
universal classroom management
practices to reduce disruptive,
aggressive behavior
Greater understanding of what
teachers can expect with use of
combined effective classroom
management practices—
“Bang
g for the Buck”
What does it look like?
R
Research
hQ
Questions
ti
1
1.
2.
3.
4.
Do teachers’
teachers universal classroom
management practices reduce problem
behavior in classroom with students in
K-12?
What components make up the most
effective and efficient classroom
management programs?
Do differences exist between grade
levels in effects and classroom
management components?
Does treatment fidelity affect the
outcomes observed?
Method

Meta-analysis was used to
systematically identify the
magnitude and direction of effects
across studies and to determine
what particular features of studies
contribute to these effects.
D fi iti
Definition


“A collection off classroom
procedures implemented by
teachers in classroom settings with
all students, for purposes of
preventing and reducing
inappropriate behavior as well as
teaching
a
g prosocial
p o o a behavior”
b a o
Does not include social skills
programs
p
g
alone or academic
instructional procedures
I l i C
Inclusion
Criteria
it i
1.
2.
3
3.
The intervention
Th
i t
ti
mustt b
be d
delivered
li
d
universally to all subjects. Pull-out or
small group interventions (e.g., small
group social skills) were not eligible.
Interventions that began treatment
outside of the classroom and then
transferred it into the classroom were
not eligible.
Additional treatment components (e.g.,
(e g
parent training) were allowed provided
there was at least one outcome variable
measuring
i
ttreatment
t
t effects
ff t with
ith
students.
I l i C
Inclusion
Criteria
it i C
Continued
ti
d

Children in grades K-12


General or special education setting


No Pre-K
Residential, clinic, or day treatment
settings not included
Must have dependent measure of
inappropriate student behavior in
the classroom
I l i C
Inclusion
Criteria
it i C
Continued
ti
d




Experimental or quasi-experimental
with control group
Random
d
assignment, matching,
h
ANCOVA, or report pre-test—posttest data
Post-test only, non-equivalent
comparisons not included
Single subject studies NOT included
S
Search
hP
Procedures
d

Database Search:


Terms used:




classroom management, classroom organization,
classroom
l
structure,
t
t
b
behavior,
h i
outcomes,
t
evaluation,
l ti
effects, environment, climate, structure
Author search:


PsychoINFO, ERIC, ProQuest, ProQuest Dissertations
Brophy, Evertson, Canter, Kounin, Kellam, van Lier
Emailed Carolyn Evertson
Website search: www.comp.org
Hand search:



Journal of Educational Psychology
Journal of Applied Developmental Psychology
Behavioral Disorders
S l ti off Studies
Selection
St di






IInitial
i i l search
h produced
d
d 5,134
5 134 titles
il
94 abstracts reviewed
25 selected and screened
Screening reliability was conducted on
100% with 96% average agreement.
12 studies selected for final inclusion
Coding
gp
protocol used to extract data w//
84% point-by-point inter-rater agreement
and discrepancies re-coded
Classroom Organization and Management
Program COMP
Evertson et al., 1988

7 Studies
S di







Organizing the classroom
Planning and teaching rules and procedures
Managing student work and improving student
accountability
Maintaining good student behavior
Planning and organizing
Conduction of instruction and maintaining
momentum
Getting the year off to a good start
Additional Research Question Added: Is there a significant
difference in effects between COMP and Non-COMP Studies?
“G d B
“Good
Behavior
h i G
Game””

3 Studies
St di
(Ialongo et al., 1993; Dolan et al., 1999;
van Lier et al., 2004)






Group contingency used as universal
classroom management
Class split into teams
Classroom rules
Reinforcement
f
and
d consequences
Ialongo et al. also had parent training
component
p
Dolan et al. had additional treatment
group for Mastery Learning (not analyzed)
M lti
Multi-component
t

2 Studies
S di

Proactive classroom management strategies
(Hawkins et al., 1991)





Frequent use of encouragement and praise
ICPS social skills curriculum
Interactive teaching (2nd grade only)
Parent training
School, classroom and parent components
(Gottfredson et al
al., 1993)

Teachers trained in classroom management
based on Evertson’s work
St ti ti l P
Statistical
Procedures
d



Standard
St
d d mean difference
diff
effect
ff t
sizes calculated on dependent
measures of disruptive,
disruptive
inappropriate, or aggressive student
behavior in the classroom
CMA and SPSS software used for
the analysis
I
Intermediate
di
calculations
l l i
d
done iin
Excel
Adj t
Adjustments
t

Group combination (n = 2)
Pretest adjustment (n = 2)
Covariate adjustment (n = 1)

Issue:
u nested
d data
da a


Issues:




COMP studies
t di reported
t d classroom
l
measures as a mean
Other studies reported individual
students measures as a mean
Need to have equivalent effect sizes
across studies
Non-COMP studies should be
adjusted to classroom level
Clustering Adjustment—Non COMP
Studies (N = 5)


Effect
ff
sizes based on individual
student measures adjusted to
classroom level effect sizes to have
equivalent effect sizes with COMP
studies
ICC = .05 and ICC = .10 used in
calculation and both reported
ESclusteradj
l t dj =
ES sm
.05
ES sm
ESclusteradj =
.10
“Standard Difference Between the
Mean Classroom Effect Size”



Effect
ff
sizes in final
f
analysis would
not be typical ESsm
Finall effect
ff
sizes would
ld b
be
classroom level and larger than
typical ESsm due to smaller standard
deviations at classroom level
Can not be compared with ESsm
RESULTS
Results
Research Question 1:
1.
Do teachers’ universal classroom
management practices reduce
problem
bl
b
behavior
h i iin classroom
l
with
ih
students in K-12?
M i Eff
Main
Effects
t Analysis
A l i
ESclassroom
SE
z
p
ICC
ESsm
.80
0.15
( 05 0.74)
(-.05–
0 74)
5.44
0.00
.05
.18
.71
71
0 13
0.13
(-.03–1.56)
5 53
5.53
0 00
0.00
.10
10
.22
22
A l
Analyses
C
Continued
ti
d


Sensitivity analysis in CMA did not
indicate any one study had a
greater impact on results
“Trim and fill” procedure in CMA
indicated no publication bias
R
Research
hQ
Questions
ti
2
2, 3
3, & 4
4:
•
•
•
What components make up the
most effective and efficient
classroom management programs?
Do differences exist between grade
levels in effects and classroom
management components?
Does treatment fidelity affect the
outcomes observed?
H t
Heterogeneity
it Analysis
A l i
Q
df
p
I2
ICC
13.72
11
0.25
19.83
.05
10 67
10.67
11
0 47
0.47
0 00
0.00
.10
10
M d t Analysis
Moderator
A l i


Heterogeneity Analysis did not
support moderator analysis
Studies
d
d
did
d not report sufficient
ff
data to conduct moderator analysis
to answer questions related to
differences in grade level and
treatment fidelity
P t Hoc
Post
H Research
R
hQ
Question
ti 5
5:

Is there a significant
f
difference
ff
in
effects between COMP and NonCOMP Studies?
M d t Analysis—Post
Moderator
A l i P t Hoc
H
Variable
i bl Mean ES
SE
-95%
CI
+95%
CI
z
p
Classroom
Other
.88
(ICC=.05)
.29
.22
.41
6.36
.00
Other
.66
(ICC=.10)
.22
.23
1.10
3.01
.00
COMP
.75
.18
.40
1.10
4.23
.00
No Difference Between Effect Sizes
Based on Treatment Characteristics
Qbetween
df
p
ICC
0.38
1
.54
.05
0.07
1
.54
.10
Di
Discussion
i


Teachers’
T
h
’ universal
i
l classroom
l
management practices have a positive
effect on decreasing problem behavior in
the classroom compared with standard
classroom practices.
Overall mean effect size of .80 or .71 can
be interpreted to mean that in general,
students in treatment group performed
better than students in control group
Di
Discussion
i


No statistically significant
f
difference
ff
between COMP and non-COMP
studies
Effect of treatment needs to be
considered as a comparison
between “treatment as usual”
rather than “no
no treatment”
treatment
Li it ti
Limitations





Single subject data not included
Studies with no student outcome
d
data
reported
d not included
l d d
Lack of data to do additional
moderator
d
analyses
l
((e.g., effects
ff
by
b
grade level)
L k off treatment fidelity
Lack
fid li data
d
reported in studies
St di published
Studies
bli h d after
ft analysis
l i
F t
Future
Research
R
hQ
Questions
ti






What components make up the most
efficient and effective CM package?
What is the best way to measure classroom
management implementation?
What is the acceptable level of CM
implementation (treatment fidelity)?
What level needs to be in place before
children are identified as “nonresponders”?
What variables moderate effects of CM?
What is the best way to prepare inservice
and preservice teachers?
C t t Information
Contact
I f
ti

Regina M. Oliver
regina.m.oliver@vanderbilt.edu
Thank You!