Elish-Piper & L’Allier, NRC 2007
Page 1
Does Literacy Coaching Make a Difference?
The Effects of Literacy Coaching on Reading Achievement in Grades K-3
in a Reading First District
Susan K. L’Allier, Ed.D.
Associate Professor
Department of Literacy Education
Northern Illinois University
sklallier@niu.edu
Laurie Elish-Piper, Ph.D.
Professor
Department of Literacy Education
Northern Illinois University
laurieep@niu.edu
Paper Presented at the 57th Annual Meeting of the National Reading Conference
November 29, 2007
Austin, TX
Paper Summary
Purpose:
This study examined the effects of literacy coaching on student reading achievement in grades K-3
in a Reading First school district. Specifically, the study investigated the effects of the amount,
type, and content of coaching on student reading achievement at the teacher level, as well as for all
students and for English Language Learners.
Perspectives / Theoretical Framework:
Teacher quality significantly impacts student achievement in school (Darling-Hammond, 1999;
Joyce & Showers, 1995; Wayne & Youngs, 2003). Furthermore, research demonstrates that teacher
expertise makes a powerful contribution to student reading achievement (Bembry, Jordan, Gomez,
Anderson, & Mendro, 1998; Wharton-McDonald, Pressley, & Hampston, 1998). For example, in a
study of first-grade reading instruction, researchers found that students with the lowest abilities who
were taught by exemplary teachers scored as well as students with average abilities who were taught
by less skilled teachers (Pressley, Wharton-McDonald, Allington, Block, Morrow, Tracey, Baker,
Brooks, Cronin, Nelson, & Woo, 2001). Based on this and similar research studies (e.g., WhartonMcDonald, et. al, 1998), Allington (2006) argues that the best way to increase student reading
achievement is to improve the quality of classroom reading instruction. Moreover, Swartz (2005)
contends that improving the quality of classroom reading instruction may decrease the need for
remedial reading programs and special education services.
A promising method of improving classroom reading instruction is the use of literacy coaches who
provide on-going, job-embedded professional development for classroom teachers (International
Reading Association, 2004; Toll, 2005, 2006). Literacy coaching has been identified as one of the
key strategies to improve reading achievement by the Reading First initiative (International Reading
Association, 2004; Toll, 2005; 2006).
While there is a push for literacy coaching in the schools, there is limited research to document the
impact of literacy coaching on student reading achievement. Most of the available studies on
literacy coaching focus on the roles and responsibilities of coaches (e.g., Bean, Swan, & Knaub,
2003; Dole, 2004; Deussen, Coskie, Robinson, & Autio, 2007; Poglinco, Bach, Hovde, Rosenblum,
Saunders, & Supovitz, 2003). Several other frequently-cited studies about literacy coaching focus
on the implementation of new programs rather than on coaching itself (e.g., Lyons & Pinnell, 2001;
Elish-Piper & L’Allier, NRC 2007
Page 2
Norton, 2001). Other studies on coaching tend to be descriptive in nature without empirical data to
document the effects of literacy coaching on student achievement (e.g., Killion, 2003). One of the
few empirical studies is an evaluation study on literacy coaching (Swartz, 2005) indicating that
coaching contributes significantly to student reading achievement in grades K-4, more than either a
traditional professional development program or a highly prescriptive reading program. The small
number of empirical research studies, however, is not sufficient to document the impact of literacy
coaching on student reading achievement (Dole, 2004; Deussen, et al., 2007).
Methods:
This study included three groups of participants from a Reading First school district. Participants
included 12 literacy coaches, 121 classroom teachers in grades K-3, and 3,029 students in grades K3. The district enrollment is 95.7% minority, 79% low income, and 44% English Language
Learners. The district was in the initial year of the Reading First Grant during the year this study
was conducted.
Data were collected from the literacy coaches in the form of weekly Coaching Logs which
contained data about the amount of time and number of interactions (frequency) coaches spent (a)
with each teacher, (b) on each type of coaching activity, and (c) on the content of each coaching
activity. The Coaching Logs were collected each week from January through June of 2007. In the
fall of 2006 and spring of 2007, classroom teachers collected student test score data by
administering the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) to all students.
Frequency distributions were completed to provide an initial summary of the coaching and
assessment data at the teacher, grade, coach, and across-coach levels. Then hierarchical linear
modeling (HLM) (Raudenbush & Bryk, 2002) was used to analyze the impact of coaching on
students’ reading achievement. A two level model was used with the Student at Level 1 and the
Teacher at Level 2; this type of analysis takes into account that students are nested within
classrooms. The dependent measures were the total gain scores and the gain scores of the various
subtests while the predictor (independent) variables were the amount of coaching time, the amount
of time spent on various coaching activities (e.g., conferencing and demonstration teaching), and the
amount of time spent coaching about various aspects of literacy (e.g., comprehension and guided
reading). Analyses examined the impact of the predictors on the gain scores of students at the
various grade levels and on students who are English Language Learners.
Data Sources:
The Coaching Log and student test scores were the data sources for this study. The Coaching Log
documented all coach/teacher interactions; each interaction was categorized according to the
specific type of coaching activity (e.g., demonstration teaching, teacher observation, postobservation conferencing, co-planning of lessons) and to the content of the activity (e.g.,
comprehension, phonics, shared reading). The Coaching Log also documented additional coaching
activities that did not involve interactions with a specific teacher (e.g., ordering literacy materials
such as leveled readers, inputting assessment data into the district’s assessment management
system, and attending required Reading First meetings). Prior to the beginning of the study, the
researchers worked with the coaches to revise the list of coaching activities used in the earlier pilot
study (L’Allier & Elish-Piper, 2006) to more accurately reflect the activities in which these coaches
were typically engaged, and then they trained the coaches to use the previously-piloted coding
system for documenting their time. A three-week pilot period was completed during the Fall 2006
semester to ensure that the coaches were able to log their activities appropriately. A follow-up
meeting with the coaches was held to clarify questions about the data collection process. Coaches
Elish-Piper & L’Allier, NRC 2007
Page 3
completed and turned in electronic copies of their logs on a weekly basis from January 2007
through the end of the school year in June 2007. Fall and spring scores from the Dynamic
Indicators of Basic Early Literacy Skills (DIBELS) served as the assessment data.
Results:
The coaches spent 48% of their time working with teachers, primarily in the activities of observing,
conferencing, modeling, co-teaching / co-planning, and administering assessments for specific
teachers. Fifty-two per cent of the coaches’ time was spent on other activities such as inputting
assessment data into the district’s assessment management system, writing Reading First reports,
and attending professional development sessions about literacy coaching.
Teachers at the kindergarten level received an average of 13.39 hours of coaching during the 5
month period of the study (SD 8.07). First-grade teachers received an average of 13.89 hours of
coaching (SD 10.95); second-grade teachers received an average of 15.20 hours of coaching
(SD10.06); and third-grade teachers received the most coaching with an average of 20.69 hours (SD
13.05).
On average, the total gains on the DIBELS by students at the kindergarten, first-grade, secondgrade, and third-grade levels were statistically significant (at the < .0001 level). The percentage of
variance of student gain due to teacher differences was calculated for each grade level:
kindergarten 35.98%; first grade 19.79%; second grade 33.28%, and third grade 26.99%.
Kindergarten
More specifically, the data for each grade level and each subtest were analyzed and the following
results were identified. At the kindergarten level, students made statistically significant gains on all
of the DIBELS subtests: Phoneme Segmentation Fluency, Letter Naming Fluency, Letter Sound
Fluency, Nonsense Word Fluency, and Word Usage Fluency. Students in the English classrooms
had scores that were significantly higher than their peers in Bilingual classrooms on the Letter
Naming Fluency subtest (12.3 points higher at <.01 level), the Word Usage Fluency subtest (7.61
points higher at <.05 level), and the Total Gain (26.3 points higher at the <.0001 level).
When the effects of coaching were analyzed at the kindergarten level, the number of coaching hours
focused on conferencing was found to be statistically significant in relation to students’ Total Gain.
The amount of time coaches spent conferencing with teachers accounted for 20.19% of the teacher
variance and 7.26% of the total variance in Total Gain. Moreover, the data indicated that for every
hour of literacy coaching related to conferencing a teacher received, his/her students scored 18.59
points higher in terms of Total Gain. In addition, the number of total coaching hours approached
significance as a predictor, accounting for 18.39% of the teacher variance and 6.62% of the total
variance in Total Gain. It was determined that for every hour of literacy coaching a teacher
received, his/her students scored 1.79 points higher in terms of Total Gain on the DIBELS.
First Grade
Analysis of the first-grade data revealed that the students made statistically significant gains on each
DIBELS subtest (i.e., Phoneme Segmentation Fluency, Nonsense Word Fluency, Oral Reading
Fluency, Retelling Fluency, and Word Usage Fluency), as well as on the Total Gain. Students in
English classrooms significantly outperformed their peers in Bilingual classrooms on the Oral
Reading Fluency subtest (7.23 points higher at <.0001 level), Retelling Fluency subtest (7.69 points
higher at <.05 level), and Word Usage Fluency subtest (6.66 points higher at <.0005 level). On
Elish-Piper & L’Allier, NRC 2007
Page 4
Total Gain, students in the English classrooms scored significantly higher than their peers in
Bilingual classrooms (22.5 points higher at <.01 level).
Coaching variables were analyzed at the first-grade level, and the results indicated that coaching
hours spent on conferencing and administering assessments approached significance as predictors of
students’ Total Gain on the DIBELS. More specifically, coaching hours spent on conferencing
accounted for 9.09% of the variance in student Total Gain due to teacher differences and 1.80% of
the total variance in students’ Total Gain. Additionally, for each hour of literacy coaching focused
on conferencing that a teacher received, his/her students made a 10.24 point increase on Total Gain.
The number of coaching hours spent on administering assessments accounted for 9.55% of the
teacher variance and 1.89% of the total variance in Total Gain. Furthermore, each hour of coaching
spent administering assessments for a specific teacher resulted in a 5.91 point gain for that teacher’s
students on Total Gain on the DIBELS.
Second Grade
Second-grade students in the study made statistically significant gains on each DIBELS subtest (i.e.,
Oral Reading Fluency, Retelling Fluency, and Word Usage Fluency) and on the Total Gain.
Students in English classrooms scored significantly higher than their peers in Bilingual classrooms
on the Oral Reading Fluency subtest (8.36 points higher at <.05 level) and Retelling Fluency subtest
(5.72 points higher at <.05 level). Furthermore, students in the English classrooms outperformed
students in Bilingual classrooms on the DIBELS Total Gain (59.82 points higher at <.01 level).
Analysis of coaching variables at the second-grade level indicated that coaching hours spent
administering assessments, conferencing, modeling, and observing were significant predictors of
students’ Total Gain, as were total coaching hours. More specifically, the number of coaching
hours spent administering assessments accounted for 29.85% of teacher variance and 9.93% of total
variance on students’ Total Gain on the DIBELS. Furthermore, for each hour of coaching spent on
administering assessments for a specific teacher, his/her students made a 10.8 point increase on the
Total Gain. Coaching hours spent conferencing explained 12.22% of teacher variance and 4.07% of
total variance on the students’ Total Gain. The data indicated that for every hour of conferencing a
teacher received, his/her students’ Total Gain increased by 8.45 points. Modeling accounted for
10.30% of teacher variance and 3.44% of total variance on Total Gain. Moreover, for every hour of
modeling a coach provided to a teacher, his/her students’ Total Gain scores increased by 8.74
points. The number of coaching hours focused on observation accounted for 9.54% of teacher
differences and 3.17% of total variance on students’ DIBELS Total Gain. More specifically, for
each hour of literacy coaching focused on observation, students made a 4.92 point improvement on
Total Gain. Overall, the number of coaching hours was a significant predictor of student Total Gain
at the second-grade level. The total number of coaching hours accounted for 11.57% of teacher
variance and 3.85% of total variance on Total Gain. For each hour of coaching received, a teacher’s
students made a 0.8 point improvement on Total Gain on the DIBELS.
Third Grade
At the third-grade level, students made statistically significant gains on the Oral Reading Fluency
subtest and on the Retelling Fluency subtest (both at the <.0001 level), but they did not make
statistically significant gains on the Word Usage Fluency subtest. There were no significant
differences in the scores of students in English classrooms as compared to those in Bilingual
classrooms.
Elish-Piper & L’Allier, NRC 2007
Page 5
The third-grade data were analyzed regarding the effects of various coaching variables; however,
none of them were statistically significant in relation to student gains on the DIBELS. It is
interesting to note that the district did not meet requirements for Adequate Yearly Progress (AYP)
at the third-grade level during 2006-7 (the year of the study); however, the third-grade students did
make statistically significant gains on the DIBELS. This finding, coupled with the requirements of
AYP, indicates that statistical significance does not necessarily equate to educational significance.
Furthermore, on average, teachers at the third-grade level received more coaching than teachers at
the other grade levels in this study; however, this did not translate into the identification of any
coaching variables (e.g., those focused on actions such as conferencing, administering, and
modeling or those focused on content such as comprehension, vocabulary, and guided reading) that
accounted for any of the variance in student gains.
Discussion:
The amount of variance in student test scores due to teacher differences was generally high in this
data set; therefore, it appears that the literacy coaching approach offers promise as a way to help all
teachers provide quality reading instruction. These findings also indicate that the hours a literacy
coach spends conferencing contribute to the significant gains in students’ DIBELS scores.
Interestingly, the content of the conferencing did not appear to be an important factor affecting
student test scores. The effect of administering assessments was significant at the first and second
grade levels. From the current data set it is not possible to determine why this aspect of literacy
coaching contributed to student gains. Possible explanations are: the accurate assessment results
allowed teachers to plan and implement appropriate instruction, or teachers were able to spend more
time teaching rather than assessing when the literacy coach administered the assessments. Further
research is needed to understand the impact of administration of assessments as a coaching variable.
Limitations and Implications:
Two important variables were not measured in this study. First, the quality of literacy coaching was
not identified or analyzed; therefore, it is not possible to determine if some of the results of this
study were due to exemplary coaching, while others may be due to poor coaching. Furthermore,
teacher confidence in the coach was not assessed in this study. Finally, this study was conducted
across one school year; thus, it was not possible to determine the long-term effects of literacy
coaching on student achievement.
Based on the findings from this study, the following implications are offered. First, literacy
coaching programs and schedules should include ample time for conferencing with teachers since
this variable was found to be a significant contributor to student gains. Second, data-based coaching
models offer a promising approach to ensure that all children receive effective reading instruction
because coaches will be able to use data to differentiate coaching to decrease variance in teacher
instructional competence. Two implications for future research also arose from the study. Further
research is needed to determine how having literacy coaches administer assessments affects student
reading gains. In addition, examination of the effects of other coaching variables, such as the
quality of coaching, on student reading gains is warranted.
NOTE: This study was funded by an Elva Knight Research Grant of the International Reading
Association
Elish-Piper & L’Allier, NRC 2007
Page 6
References
Allington, R.L. (2006). What really matters for struggling readers: Designing research-based program (2nd
ed.). New York: Allyn & Bacon.
Bean, R.M., Swan, A.L., & Knaub, R. (2003). Reading specialists in schools with exemplary reading
programs: Functional, versatile, and prepared. The Reading Teacher, 56, 446-455.
Bembry, K.L., Jordan, H.R., Gomez, E., Anderson, M., & Mendro, R.L. (1998, April). Policy implications of
long-term teacher effects on student achievement. Dallas, TX: Dallas Public Schools. Retrieved July
12, 2005 from Dallas Independent School District Web site:
http://www.dallasisd.org/depts/inst_research/aer98rm1/longitudinal_teacher_effects.htm
Block, C.C., & Israel, S.E. (2005). Reading first and beyond: The complete guide for teachers and literacy
coaches. Thousand Oaks, CA: Corwin.
Darling-Hammond, L. 1999. Solving the dilemmas of teacher supply, demand and standards: How we can
ensure a competent, caring, and qualified teacher for every child. New York: National Commission
on Teaching & America's Future.
Dole, J.A. (2004). The changing role of the reading specialist in school reform. The Reading Teacher, 57,
462-471.
Deussen, T., Coskie, T., Robinson, L., & Autio, E. (2007). “Coach” can mean many things: Five categories
of literacy coaches in Reading First (Issues & Answers Report, REL 7007-No. 005). Washington,
DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education
Evaluation and Regional Assistance, Regional Educational Laboratory Northwest. Retrieved July 5,
2007, from http://ies.ed.gov/ncee/edlabs
International Reading Association (2004). The role and qualifications of the reading coach in the United
States. A position statement of the International Reading Association. Newark, DE: International
Reading Association.
Joyce, B., & Showers, B. (1995). Student achievement through staff development (2nd ed.). New York:
Longman.
Killion, J. (2003). Use these 6 keys to open doors to literacy. Journal of Staff Development, 24(2), 10-16.
L’Allier, S.K. & Elish-Piper, L. (December, 2006). An Initial Examination of the Effects of Literacy
Coaching on Student Achievement in Reading in Grades K-3. Paper presented at the 56th Annual
Meeting of the National Reading Conference, Los Angeles, CA.
Lyons, C.A., & Pinnell, G.S. (2001). Systems for change in literacy education: A guide to professional
development. Portsmouth, NH: Heinemann.
Norton, J. (2001). A storybook breakthrough. Journal of Staff Development 22(5), 22-25.
Poglinco, S.M., Bach, A.J., Hovde, K., Rosenblum, S., Saunders, M., & Supovitz, J.A. (2003). The heart of
the matter: The coaching model in America’s choice schools. Philadelphia: University of
Pennsylvania Consortium for Policy Research in Education.
Pressley, M., Wharton-McDonald, R., Allington, R.L., Block, C.C., Morrow, L., Tracey, C., Baker, K.,
Brooks, G., Cronin, J., Nelson, E., & Woo, D. (2001). A study of effective first-grade reading
instruction. Scientific Studies of Reading, 5, 35-58.
Raudenbush, S.W., & Bryk (2002). Hierarchical linear models (2nd ed.). Thousand Oaks, CA: Sage.
Swartz, S.L. (2005). The foundation for comprehensive early literacy learning research report 1994-2003.
Retrieved July 12, 2005, from the Foundation for Early Literacy Learning Web site: http://www.cellcom/foundationresearch report1.htm
Toll, C.A. (2005). The literacy coach’s survival guide: Essential questions and practical answers. Newark,
DE: International Reading Association.
Toll, C.A. (2006). The literacy coach’s desk reference: Processes and perspectives for effective coaching.
Urbana, IL: National Council of Teachers of English.
Wayne, A.J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review
of Educational Research, 73, 89-122.
Wharton-McDonald, R., Pressley, M., & Hampston, J.M. (1998). Literacy instruction in nine first-grade
classrooms: Teacher characteristics and student achievement. Elementary School Journal, 99, 101128.