Direct Instruction Implementation,
New Hanover County Summer School, 1999
Executive Summary of Key Findings
March 4, 2000
Frances B. Bessellieu
New Hanover County Schools
John S. Rice
Martin A. Kozloff
Kerri Seidel
The Watson School of Education
Center for Educational Research and Innovation
The University of North Carolina at Wilmington
TABLE OF CONTENTS
Introduction
Mean Placement Lesson and Mean of Lessons Mastered
by Curriculum, Race, and Gender
Participation Skills and Socio-Affective Development
Summary and Conclusion
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
1
2
3
6
13
Introduction
This executive summary report identifies and describes key findings from the
implementation of a Direct Instruction (henceforth, DI) (Science Research Associates
[SRA]) curriculum during the 1999 summer school session, in New Hanover County
public schools. A total of 486 students and 59 teachers – working at 20 schools in the
district – participated in this implementation.
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
2
Mean Placement Lesson and Mean of Lessons Mastered
by Curriculum, Race, and Gender
TABLE I
Mean Placement Lesson and Mean of Lessons Mastered
by Curriculum and Race (Black and White Students)
Language
for Learning
Reading
Mastery I
Reading
Mastery II
Reading
Mastery III
Fast Cycle
Corrective
Reading
Column
Totals
% of Total
Students*
Black Students
White Students
Mean
Placement
Lesson
Mean
Placement
Lesson
Mean No.
Row
of Lessons Totals
Mastered
29.24
(N = 96)
42.85
(N = 89)
3.2
(N = 9)
70
(N = 14)
1
(N = 2)
1
(N = 11)
221
13.3
174
16
178
14.1
34
17
18
39
23
7.5
27
23.7
(N = 78)
34.2
(N = 89)
4.2
(N = 25)
70
(N = 4)
37.7
(N = 21)
1
(N = 16)
243
Mean No.
of
Lessons
Mastered
16
18
13.3
16
23.5
6.9
54%
454
49%
* Percentages greater or less than 100% due to rounding
Language for Learning: A total of 174 black and white students participated in
the Language for Learning curriculum during the 1999 summer school session. The
most immediately striking datum is the extremely disproportionate representation of
blacks, vis-a-vis whites, relative to the distributions of these groups in the larger school
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
3
populations and in society more generally: 78 black students (45 percent of the 174
students enrolled in the LFL curriculum), and 96 white students (55 percent).1 As Table
1 indicates, the Mean Placement Lesson for black students was 23.7, whereas the
same score for whites was 29.2. Whites, then, on average placed 5.5 lessons higher
than did blacks. Over the course of summer session, however, the mean number of
lessons mastered was 16 for blacks and 13.3 for whites. On average, then, the gap
between black and white students, in terms of lessons mastered, decreased by 2.7
lessons. This is a positive result, indicating that, given a longer span of instructional
time, the well-known gaps between white and black student achievement will, at
minimum, become less glaringly disproportionate (a disproportionality reflected in the
overall distribution of summer school students, by race).
Reading Mastery I: A similar pattern appears among those students receiving the
Reading Mastery I curriculum. There were 178 students in RMI, overall, exactly evenly
divided between white and black students (N = 89 of each group). Again, the
disproportionality of representation is striking. And again, there was a discrepancy
between the Mean Placement Lessons for each group: the black students’ mean score
of 34.2 is 15.7 lessons lower than white students’ mean of 42.9. Over the same period
of time, however, black students mastered, on average, 2 more lessons than did white
students (18:16, respectively). As was the case with the students in the Language for
Learning curriculum, then, the data suggest that longer instructional time will gradually
reduce the disparities between black and white students’ achievement.
Reading Mastery II: As Table 1 demonstrates, the pattern evident for the first two
curricula varies in RMII, and does so in two noteworthy ways. First of all, although black
students are even more disproportionately represented, here, the Mean Placement
Lesson for black students (N = 25; 74 percent of the 34 students in RMII) is one point
higher than for white students (N = 9; 26 percent of RMII students): 4.2 versus 3.2,
respectively. Secondly, the mean number of lessons mastered for black students is 8
tenths of a point less than the mean for white students: 13.3 versus 14.1, respectively.
These are not especially large differences, to be sure, but the most salient issue is what
the data will “look like” over time: e.g., will the small discrepancies become magnified or
smaller with longer instructional time. These findings suggest that there is less skill
discrepancy among students placed in Reading Mastery II, than among students in the
earlier curricula. They also suggest that when black and white students learn to read
fairly well (RMII level), they read equally well.
1
This, of course, is not surprising; indeed, it is an all-too-familiar, obdurate, and very troubling pattern. It
is the hope of all involved in this initiative that DI will, at the very least, bring about proportional
representation in at-risk student population.
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
Reading Mastery III: The problem of disproportionate representation is not at work
among the students receiving the RMIII curriculum; this is the only curriculum in which
that is the case. Of the 18 students at this level, 4 (24 percent) were black, and 14 were
white. Moreover, the Mean Placement Lessons (70) are identical for each group. Black
students did, however, master, on average, one less lesson than did whites – a finding
that, as indicated above, points to the need to pay careful attention across time. One
fewer lesson need not be read as problematic, of course. But, over the course of a
school year, were that disparity to persist, it would clearly be a problem.
Fast Cycle: In the Fast Cycle curriculum, blacks were again – and even more so than in
other curricula – disproportionally represented. 21 of the 23 (91 percent) students
receiving FC instruction were black. The black students’ Mean Placement Lesson was
considerably higher than the mean for the two white students2 – 37.7 compared with 1 –
but the black students also mastered substantially fewer lessons (23.5 versus 39, a
difference of 15.5 lessons). These differences, of course, may well be attributable to
teacher differences, a failure to recognize a misplacement, or a number of other factors.
As subsequent pages will show, it is clear that there are significant differences in the
number of lessons covered. Furthermore, the disproportionate placement of black
students in FC is probably a good sign. Fast Cycle is an accelerated reading program
that combines RMI and RMII. It is a desirable finding that so many black students
placed in FC rather than RMI or RMII.
Corrective Reading: 59 percent of the 27 students (N = 16) in the Corrective Reading
curriculum were black; 41 percent (N = 11) were white. The Mean Placement Lesson for
both groups was 1; the black students’ mean of lessons mastered is 6 tenths of a point
off the pace of the white students (6.9 and 7.5, respectively). Earlier observations
regarding differences of less than one point between groups apply here, as well. Given
the condensed time frame of summer school, and the differences among teachers – in
terms of comfort and fluency with, and (in some cases) enthusiasm for, DI, it would be
premature to draw binding conclusions about small differences.
2
It is, of course, all but meaningless to report the mean score for two students – especially when it is the
case that both scored a 1 on the placement test for Fast Cycle. We report these findings in this way for
purposes of consistency in reporting the results. The data are illustrative, however, as comments in the
body of the report indicate.
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
Participation Skills
and Socio-Affective Development
In addition to helping to implement a curriculum backed by lengthy supporting
research literature, we also wanted to address another key issue related to Direct
Instruction’s effects on student achievement and the classroom, and school,
environment: namely, what, if any, impact does the DI curriculum have on the creation
and maintenance of the much-vetted notion of a “learning community” that will foster in
children the shared meanings, motives, and norms that will then facilitate mastery of
language and reading?
Table 2, which runs the next four and one-half pages, summarizes teachers’
responses to a series of items designed to shed tangible light upon the working
hypothesis just described. The instrument, “Participation Skills and Socio-Affective
Development” (see Appendix B for both the instrument and the Code Book), quite
simply asks teachers to tell us about their perceptions of student behaviors, particularly
changes in behaviors that are strongly associated with a positive and productive
learning environment.
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
TABLE 2: Participation Skills and Socio-Affective Development
(The children):
Come when
signaled
Get ready for
lesson
Respond
when teacher:
says “listen”
... says “get
ready”
... says
“watch”
... holds up
one finger (gs)
Very much
Improved
141 (29.9%)
Much
Improved
139 (29.4%)
A Little
Improved
84 (17.8%)
Virtually No
Improvement
12 (2.5%)
Not a Problem
Before
96 (20.3%)
Not Applicable
133 (28.3%)
142 (30.2%)
94 (20%)
14 (3%)
87 (18.5%)
134 (28.5%)
149 (31.7%)
92 (19.6%)
18 (3.8%)
77 (16.4%)
130 (27.7%)
147 (31.3%)
92 (19.6%)
15 (3.2%)
80 (17%)
6 (1.3%)
129 (27.4%)
148 (31.5%)
93 (19.8%)
16 (3.4%)
78 (16.6%)
6 (1.3%)
119 (25.4%)
118 (25.2%)
73 (15.6%)
18 (3.8%)
73 (15.6%)
67 (14.3%)
Row Totals
472
(14 missing)
470
(16 missing)
470
(16 missing)
470
(16 missing)
470
(16 missing)
468
(18 missing)
“GS” refers to group turn signals; the same items appear later in relation to individual turn signals.
TABLE 2: Participation Skills and Socio-Affective Development (Cont’d)
Very much
Improved
Much
Improved
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
A Little
Improved
Virtually No
Improvement
Not a Problem
Before
Not Applicable
Row Totals
... moves
finger along
word (gs)
... holds finger
nd
on 2 ball
under sound
(gs)
115 (24.6%)
112 (23.9%)
73 (15.6%)
16 (3.4%)
69 (14.7%)
83 (17.7%)
468
(18 missing)
109 (23.3%)
99 (21.2%)
65 (13.9%)
12 (2.6%)
58 (12.4%)
125 (26.7%)
468
... drops her hand (gs)
116 (24.9%)
136 (29.2%)
75 (16.1%)
17 (3.6%)
73 (15.7%)
49 (10.5%)
... claps (gs)
95 (20.3%)
103 (22%)
65 (13.9%)
9 (1.9%)
74 (15.8%)
122 (26.1%)
... says “do it” (gs)
97 (20.7%)
115 (45.2%)
72 (15.4%)
8 (1.7%)
70 (14.9%)
107 (22.8%)
... says “say it fast”
101 (21.6%)
96 (20.5%)
58 (12.4%)
9 (1.9%)
66 (14.1%)
138 (29.5%)
112 (24%)
96 (20.6%)
61 (13.1%)
9 (1.9%)
56 (12%)
132 (28.3%)
(18 missing)
466
(20 missing)
468
(18 missing)
469
(17 missing)
(gs)
... says “what word?”
468
(18 missing)
(gs)
466
(20 missing)
“GS” refers to group turn signals; the same items appear later in relation to individual turn signals.
TABLE 2: Participation Skills and Socio-Affective Development (Cont’d)
Very much Improved
Much Improved
A Little Improved
Virtually No
Not a Problem Before
Not Applicable
Row Totals
73 (15.9%)
39 (8.5%)
458
Improvement
(The children):
143 (31.2%)
114 (24.9%)
Put finger on correct
place when asked
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
72 (15.7%)
17 (3.7%)
(28 missing)
Restate rule or
150 (31.8%)
142 (30.1%)
56 (11.9%)
10 (2.1%)
51 (10.8%)
63 (13.3%)
133 (28.3%)
145 (30.9%)
79 (16.8%)
11 (2.3%)
77 (16.4%)
25 (5.3%)
115 (24.4%)
97 (20.6%)
49 (10.4%)
12 (2.5%)
33 (7%)
166 (35.2%)
teacher’s statement
Follow instruction to
(14 missing)
do independent task
Do
independent
472
(14 missing)
113 (24%)
131 (27.8%)
66 (14%)
7 (1.5%)
64 (13.6%)
90 (19.1%)
146 (30.9%)
114 (24.2%)
63 (13.3%)
9 (1.9%)
131 (27.8%)
9 (1.9%)
when asked
Show attachment to
470
(16 missing)
work
Sustain an action
472
471
(15 missing)
materials
472
(14 missing)
TABLE 2: Participation Skills and Socio-Affective Development (Cont’d)
Very much Improved
Much Improved
A Little Improved
Virtually No
Not a Problem Before
Not Applicable
Row Totals
471
Improvement
Sustain attention
121 (25.7%)
147 (31.2%)
103 (21.9%)
25 (5.3%)
74 (15.7%)
1 (.2%)
155 (32.9%)
117 (24.8%)
65 (13.8%)
23 (4.9%)
110 (23.4%)
1 (.2%)
136 (29.6%)
87 (19%)
60 (13.1%)
11 (2.4%)
142 (30.9%)
23 (5%)
during lessons
Interact appropriately
(15 missing)
with other children
Show attachment to
471
(15 missing)
materials outside DI
459
(27 missing)
Lesson
Express affection for
153 (33.6%)
116 (25.4%)
42 (9.2%)
15 (3.3%)
129 (28.3%)
156 (34.1%)
109 (23.9%)
54 (11.8%)
10 (2.2%)
128 (28%)
116 (25.3%)
124 (27.1%)
79 (17.2%)
19 (4.1%)
120 (26.2%)
teacher, other children
Put materials away in
456
(30 missing)
proper place
Behave appropriately
1 (.2%)
457
(29 missing)
by place
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
458
(28 missing)
TABLE 2: Participation Skills and Socio-Affective Development (Cont’d)
Very much Improved
Much Improved
A Little Improved
Virtually No
Not a Problem Before
Not Applicable
Row Totals
Improvement
(The children):
149 (32.7%)
106 (23.3%)
50 (11%)
16 (3.5%)
134 (29.5%)
Help other children
119 (26%)
130 (28.4%)
71 (15.5%)
24 (5.3%)
97 (21.2%)
Respond quickly to
110 (24%)
141 (30.8%)
94 (20.5%)
25 (5.5%)
88 (19.2%)
3,546
3,420
2,000
407
2,408
1,269
(27.2%)
(26.2%)
(15.3%)
(3.1%)
(18.5%)
9.7%)
455
Freely share materials
(31 missing)
16 (3.5%)
457
(29 missing)
458
teacher signals
Column Totals
(28 missing)
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
13,050
As we have already indicated, the purpose to gathering these data was to get a preliminary, but solid, look at the
hypothesis that Direct Instruction’s effectiveness these past thirty-plus years issues not only from its scripted and logically-integrated
lessons and overall curricula, or its emphasis upon building firm foundations in phonemic awareness and decoding skills, but also
because it fosters the creation of a genuine learning community, and does so in several different ways – some of which we alluded
to in the introductory remarks to this third part of the Report. In addition to the brisk lesson pacing, frequent and repeated
opportunities for children to experience academic successes, incorporation of both choral and individual responses – all of which
focus children’s attention and activity on the tasks of learning and lesson mastery – DI also teaches students what we have begun to
think of as extremely basic, but essential, behavioral components that make it possible for instruction and learning to occur. Some
significant minority of all children – and a higher proportion of children that develop reading, language, and behavioral problems –
come to school utterly unprepared for what is expected of them, and will be expected of them for years. We have in mind, here,
such things as: taking turns; following oral (or written) directions; staying in one physical location when a lesson is underway;
recognizing that a lesson is underway; respecting others (by not interrupting them when they are speaking), and so on.
Children must master these seemingly banal, but absolutely essential behavioral components – components that make
teaching and learning quite literally possible – if they are to have any real shot at mastering their curriculum. It is no secret that the
children most likely to be selected to populate some sub-section of the larger “at-risk” category most often (although, to be sure, not
always) hail from family backgrounds in which behaviors such as these are not available as models for these children to emulate. As
such, they enter school with a double deficit: on the one hand, they are facing – in many cases – disadvantages relative to their
classmates that were exposed, in general and repeated ways, to the value of learning and the importance of a good education
(parents that read to them frequently, took them to cultural events, and so on); on the other hand, they have also not been exposed
to these behaviors which translate into a necessary condition for instruction and learning. Direct Instruction appears to cover both of
these deficits, and, by doing so, lays the behavioral foundation for intellectual growth. Both dimensions, the behavioral and the
intellectual, rely upon success in their counterpart; they are integrally, even symbiotically linked. Children that do not have the
behavioral skills to keep up with the curriculum become progressively more marginalized; as this occurs, they also are at higher risk
for becoming behavioral problems. They are not engaged with others or with the lessons, do not feel themselves a part of, or
connected to, what is going on around them. As such, they fall further and further behind intellectually and get more and more
isolated from and disruptive to the learning community.
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington
SUMMARY AND CONCLUSIONS
Table2 – “Participation Skills and Socio-Affective Development” – which covered the preceding four and one-half pages
– summarizes teachers’ reports of any discernible changes in their students’ mastery of these essential behavioral components.
Although we had anticipated seeing some improvement, we were pleasantly surprised at the findings. As the final row of the Table
demonstrates, of the 13,050 possible responses to the various items comprising this instrument, 3,546 responses (27.2 percent) fell
under the “Very Much Improved” category; 3,420 (26.2 percent) were “Much Improved.” 53.4 percent of the teachers’ responses,
then, point to considerable positive gain in their students’ mastery, if you will, of the necessary behavioral components constituting
an environment favorable to teaching, learning, and achievement. If we include – and it is reasonable to do so (even bearing in mind
the caveat that the behavior it still not satisfactory) – those showing “A Little Improve[ment]” (N = 2,000; 15.3 percent of the total),
then the positive outcomes increase to 68.7 percent. Conversely, the number of reports of “Virtually No Improvement” tallied 407 –
but 3.1 percent of the total.
If the gains were so highly positive over a time-frame of 18 days of classes, it seems reasonable to assume that the
positive effects will multiply when stretched out over the course of an entire school year (or a student’s school career). These, again,
are extremely positive and hopeful findings.
Although there was every reason to expect some positive outcomes from the implementation of Direct Instruction
language and reading curricula – given the thirty-plus years of research that has invariably shown those curricula, and DI overall, to
be highly effective – it would have been nothing short of hubris to expect the findings to be as across-the-board positive as they
have turned out to be. After only eighteen days, as this Report has shown, student achievement trajectories – especially for the
beginning curricula (Language for Learning and Reading Mastery I) – have begun to point in the direction of greater language and
reading competency for all students, and in the direction of a reduction in the disproportionate representation of minority students
among students at-risk academically.
The additional – and we think extremely noteworthy and optimistic – finding from this research/implementation was
presented in the section on “Participation Skills,” in which the data indicate, and quite convincingly so, that this curriculum helps to
cultivate the behavioral skills that are a necessary condition for effective instruction and learning to occur.
In summary, the overall results of this initiative are highly positive. All of the indicators point to the likelihood that we will
see continuous improvement across each of these arenas. In turn, it is reasonable to expect these improvements to have a highly
positive impact upon the overall culture of the school.
Data Analysis and Report Preparation by John S. Rice
Center for Educational Research and Innovation
The University of North Carolina – Wilmington