OCM BOCES Region
Leveled Literacy Intervention (LLI)
2011-2012
Description of OCM BOCES Leveled Literacy Intervention
The OCM BOCES Leveled Literacy Intervention Region completed its 3rd year in June.
The site was comprised of 12 districts this year, with 76 teachers in 34 school buildings
delivering small group literacy instruction. Frances Malave, Reading Recovery Teacher
Leader and Leveled Literacy Interventionist and Lisa Schlegel, Leveled Literacy
Interventionist and Staff Developer, provided LLI training to 15 new participants and
maintained ongoing professional development with 61 continuing LLI participants for a
total of 76 LLI teachers in our region. Other responsibilities of the Leveled Literacy
Interventionists are to provide one on one visits and small group learning around live
lessons with all LLI teachers, facilitate Planning Committee meetings, collect and approve
data on all students served, and continue to develop literacy understandings by engaging
in life-long learning sessions such as attending professional development at Lesley
University facilitated by experts in the literacy field, participating in LLI webinars, and
attending literacy conferences.
What Is Leveled Literacy Intervention?
LLI is a short-term, small-group, supplementary intervention system developed by Irene
Fountas and Gay Su Pinnell. LLI is designed to serve struggling readers in grades K-2,
although some school districts may use the intervention with older students. LLI is built
around the text gradient designed by Fountas and Pinnell which designates leveled texts
from A (the easiest texts) to Z (the most difficult).
LLI is described as a system because it includes reading, writing, and word study, each
used together and systematically across increasingly difficult levels of texts. Three LLI
systems currently exist:
-Orange, recommended for Kindergarten, levels A to C;
-Green, recommended for Grade 1, levels A to J; and
-Blue, recommended for Grade 2, levels C to N.
The K-2 LLI systems are designed to provide 14 to 20 weeks of daily, intensive extra
reading lessons (beyond classroom service) that result in accelerated progress with
flexible decision making about student entry and exit. Students enrolled in LLI
participate in daily (five times per week), 30-minute literacy lessons taught by an LLI
trained teacher. The recommended group size is three students at a time with one
teacher to allow for close observation and differentiated response to student strengths
and needs. Because of restrictions or limited resources, some school districts may alter
these requirements but the goal of 1:3 and 5 lessons per week is highly recommended
for the strongest outcomes. The objective of LLI is to help students reduce the gap
1
between their current instructional reading level and their expected instructional reading
level.
The intended term of the intervention ranges from about 14 to 18 weeks after which
time the students are released if approximate grade level expectations are met. At the
end of this time period, students who have made progress but do not yet meet expected
instructional reading levels for grade and time are re-evaluated. The evaluation may
suggest more time in LLI lessons or an alternative intervention (such as individual
tutoring). Students who have not received a full program of services and are in the
intervention at the end of the school year may continue to receive service at the
beginning of the next school year. Some students who participate in the intervention
may have achievement levels more than a year below grade level. Although these
students may not achieve grade level performance in the short term, steady progress
may warrant longer term service if resources allow for service to continue.
Background on LLI Data Collection, 2011-2012:
In response to LLI Planning Committee feedback in the fall of 2011, the data collection
process and timing was revised to assist the group in consideration of the question:
Is Leveled Literacy Intervention an effective and
efficient intervention in Central New York?
The initial data collection tool (Survey Monkey format) from 2010-11 was revised with
questions added to gather additional information. To better collect student specific
information an additional tool in spreadsheet format was also revised by the committee.
Thus, the data collection process evolved to consist of two tools, one for program data
and one for student progress data, which were completed in two time intervals.
At mid-year, student progress data was collected. Planning Committee members were,
at their request, notified of who had completed the data form. This mid-year data
collection occurred over a range of several weeks. At the end of the school year,
teachers updated the student progress spreadsheet, which was forwarded to Lynn
Radicello. Again, planning committee members were notified of the status of data
completion for their district. (It should be noted that some of this data was incomplete).
Additionally, at the end of the year, teachers completed the data collection tool regarding
the program through Survey Monkey. All data was aggregated to create the regional
profile.
Individual district and teacher data is not analyzed here. If you are a planning
committee member and would like the raw data to conduct further analysis specific to
your district, please contact Lynn Radicello.
2
Leveled Literacy Intervention Program Data
Program data was reported for 392 LLI students taught by 45 LLI teachers in 20 schools
through survey monkey link. OCM BOCES LLI participants reported data voluntarily.
Leveled Literacy Intervention Teachers
The table below shows the total number of teachers trained by OCM BOCES LLI
Interventionists since 2009.
Training by
OCM BOCES Region
% of Teachers
100% (n= 103)
During the 2011-2012 school year, there were 76 LLI participants and only 45 teachers
reported their Professional Title at their assigned schools through surveymonkey link.
24.4% (11) reported Reading Teacher as their primary title and 35.6 % (16) reported
AIS as their primary title. The second highest percentage 28.9% (13) identified their title
as Special Education Teachers. 11.1% (5) are classroom teachers.
3
Teachers responded to what other role(s) they have in the school. 42.2% have no other
role, 31.1% teachers identified as Title 1 or reading teachers, 13.3% selected special
education teacher, 2.2% participants were staff developers, and none chose ELL, School
or District Administrator, or Reading Recovery teacher as their other role.
4
Students Served
Teachers reported the number of the instructional groups they served for 2011-2012
school year. 24.4% teachers served 2 groups and 24.4% served 3 groups, 20% served
1 group, 11.1% served 5 groups, 8.9% served 4 groups, 6.7% served 6 groups, 2.2%
served 8 groups, and 2.2% served 10 groups.
5
Teachers reported serving 392 students. Of these students, 195 (50%) reached
Fountas’ and Pinnell’s grade level expectation and needed no further support, 142 (36%)
students needed further support after receiving 18 weeks of the LLI intervention. 126
(32%) students needed longer support and stayed in an LLI group for more than 18
weeks. Included in this group are special education students for whom it is intended to
be a yearlong program. 33 (.08%) students received incomplete intervention and 15
(.04%) moved out of district while receiving services.
Category*
Total students served in LLI groups (See Chart A)
Students who accelerated and reached the
average reader in the classroom (See Chart B)
Students who received the full 18 weeks and were
recommended for further support (does not
include special education students who were
provided a yearlong LLI program) (See Chart C)
Students who received longer than 18 weeks of
LLI service (includes special education students
who were provided a yearlong LLI program) (See
Chart D)
Students who moved out of the district while
being served (See Chart E)
Students who were removed from LLI under
unusual circumstances with fewer than 18 weeks
instruction (See Chart F)
Students who did not receive the full number of
weeks of intervention because of insufficient time
(See Chart G)
Students who were considered for retention (See
Chart H)
Number Percentage
392
100%
195
50%
142
36%
126
32%
15
.04%
28
.07%
33
.08%
28
.07%
*Individual students may be counted in more than one category
6
Number of teachers responding
Chart A
Number of Students
7
Number of teachers responding
Chart B
Number of Students
8
Number of teachers responding
Chart C
Number of Students
9
Number of teachers responding
Chart D
Number of Students
10
Chart E
11
Chart F
12
Chart G
13
Chart H
14
Reasons for the 46.7% teachers for not filling in the open slot for another student:
 I am only permitted to work w/ students on my caseload, no gen ed students. I didn't have
another student who fit.
 no 3rd graders were low enough for the blue box
 We had no available vacancy
 There were no other students available to fill slot
 I only service special ed. students
 only one student moved into my LLI group when another left for other services. He did not
receive the full 18 weeks under my instruction, but from a reading teacher.
 I am only able to service my special needs students
 no one left
 No one moved out
 no vacancies this year
 If there was a move in or someone or we rearranged groups. Otherwise we were not searching
for students. We want to maintain the integrity and focus of our group, knowing that is about
their ability and the skills being taught more than filling a spot.
 If there was an student that fit that level
 Students were shifted to adjust levels being taught
 never had a vacancy
15
Number of teachers responding
Teachers reported on the types of assessment used and the degree to which they used
these assessments for LLI placement and exit from the intervention.
16
Concluding Thoughts from Trainers perspective:
We have found a great appreciation for LLI among teachers, students & parents who
have experienced the intervention in our districts. Teachers have implemented the
intervention with fidelity as closely as they can within the constraints of their buildings.
They have used varying models of the intervention in order to meet the needs of their
students. When we look at the data, we need to be mindful of the progress students
have made, which we will further discuss in the next section of this report.
17
Leveled Literacy Intervention Student Progress Data
Introduction
After gathering information about the LLI program, we analyzed the data regarding
individual student progress. Teachers were requested to complete a spreadsheet midyear and at the end of the year providing information about student progress. This data
was aggregated into a regional profile. Data from a total of 216 students across 11
districts were included. This was in response to the question: Is Leveled Literacy
Intervention effective and efficient for students in Central New York?
To assist with determining what is effective and efficient, we referred to the research
study conducted by Elizabeth Ward Ph.D., “Leveled Literacy Intervention Research and
Data Collection Project 2009-2010” (Executive summary is posted on OCM Literacy
Programs Web-page). Her findings are referenced as benchmarks to which our results
are compared.
Was student progress accelerated during Leveled Literacy Intervention?
Using the research developed within Elizabeth Ward’s study, text levels were converted
to time equivalent scores. “These scores represent the number of months students, with
typical progression, should have completed when they demonstrate the reading
behaviors associated with the Fountas and Pinnell levels”. Gain is determined by
calculating the difference between the entry and exit time equivalent score. This
provides an indication of progress in units of months which is then compared to amount
of time students receive instruction in Leveled Literacy.
18
Student Growth
Student growth ranged from zero months to 18.5 months. The mean reading gain was
8.0 months. This compares to 8.0 months in the national study.
19
Growth Acceleration:
Student growth in time equivalent was compared to time enrolled in Leveled Literacy
Intervention to determine if student growth was accelerated. Similar to the national
study, our results indicate that student growth was accelerated. While for the purpose of
this report, the results from students receiving special education were not
disaggregated, districts are encouraged to look at this data. It was noticed that for
many students receiving special education supports, their progress was accelerated even
though grade expected benchmarks were not achieved. Sixty-eight percent of students
demonstrated accelerated growth. Students that did not demonstrate growth were, in
many cases, provided with alternative instruction.
20
What degree of reading gain was achieved?
To answer this question, text level gains were examined. First we looked at students
making at least three text level gains. Time as a variable was not considered. Ninetyone percent of the students made minimum three text level gains. This compares to
sixty-eight percent in the national study and 2010-2011 OCM’s results were Seventy-five
percent of the students made minimum three text level gains.
Gains of a minimum of seven text level gains were also looked at. The OCM BOCES
region had 22% of students achieving this significant gain. National study reports 12.5
percent with seven or more text level gains and in 2010-2011, the OCM BOCES region
had 18% of students achieving this significant gain.
21
Are students exiting Leveled Literacy Intervention meeting grade level expectations?
Within the OCM BOCES region, 36% of the enrolled students exited meeting grade level
benchmark whereas last year reports 51% met grade level benchmark.
Student’s with Special Needs in the OCM BOCES region, a total of 103 number of
students represented in this data, 36% of the enrolled students exited meeting grade
level benchmark. It should be noted that this data includes a small percentage of
Students with Disabilities received LLI as their core program during 2011-2012.
22
Comparable to the national study, we also looked at students who were “close” to
meeting grade level expectations or benchmarks. Our data shows that greater
percentages obtained desired outcomes when including students who are within one text
level of benchmark. Fifty-five percent are within one text level of benchmark. This
compares to 64.8% in the LLI subset of the national study and slightly less to last year’s
report which was 59% were within one text level of benchmark.
Of the 103 Students with Special Needs, 24 % were “close” to meeting grade level
expectations within One Text Reading Level.
23
In the subset of the national study, 79.2% were at or within two text reading levels for
grade expectation. In the OCM BOCES region, our data shows 70% at this level of
achievement.
Students Making Benchmark or within Two Text
Reading Levels
2011-2012
30%
70%
Students with Special Needs 38% of 103 were two text levels away from reaching
grade level expectations (benchmark).
24
Conclusions
OCM’s data results seem to indicate that, even if students don’t make grade level
expectations or benchmark, 79% of all student’s progress accelerated and 54% of the
Students with Special Needs progress accelerated. The data also indicates that students
with special needs progressed at the same rate as students in the general education
category. Another interesting piece of data captured was that 55% of students were one
text level away and a tier one intervention can be implemented to continue their
progress while 34% of students were two text reading levels close to reaching
benchmark will need further support such as a small group intervention to be certain the
gap continues to close. It is recommended that, as a group, we continue to collect and
study our results.
Report Prepared by:
Frances Malave, Lisa Schlegel and Lynn Radicello
July 2012
25
Appendix A
The following are informal responses made about the LLI program from colleagues,
administrators, parents and students:

























No informal responses were shared from teachers, administrators, parents or students.
K "I have seen some great shifts in his learning."
"It's nice to have a benchmark system that every other teacher in the school has been trained to use
so that we're all on the same page."
The structured lessons and pace was just what my students needed.
Very excited to see all their student could do when they observed a lesson. The teacher then had
new expectations in the classroom and made some group changes that held the child more
accountable. They were very excited to see all students were able to do.
Parents liked the take-home component. Teachers shared that they were seeing students who
perceived themselves as readers.
The flow of the lessons lends nicely to best practices in the classroom for literacy centers.
Students loved the books and were excited to read! Enthusiasm for reading increased!
I did not receive any feedback from parents, teachers, or children in response to the use of this
program.
I had one out of the six students I had in LLI no longer receive AIS services, but the other 5
students made solid gains.
My feeling is many of these students will need to be tested for a learning disability. One student
who did quite well is being retained due to his immaturity. The two I picked up later in the year
may have gotten to grade level if there was more time. State testing/Aimsweb testing took away a
lot of teaching time from these kids.
Students enjoyed rereading book. Non-readers also enjoyed looking at the pictures.
I have several students express that they sound better when they read.
Noted growth of students across the program time period
My child really enjoys the take home books. The parent sheets are helpful.
I love to read the take home books to my mom and dad
Teachers are pleased with the program and the quality of the literature.
I was worried about Jalynn's reading, but she is really getting much better with her reading. (the
student's mother said this during a phone conversation)
parents love the take home text
I can't think of anything specific, at the moment, sorry.
liked the take- home books, very helpful
A new 2nd grade student reading at D, who I thought needed to be sent for a CSE evaluation, is
going to 3rd grade reading on grade level after spending all of 2nd grade in LLI.
It was wonderful to see special needs students who were considered non-readers read.
Parents liked the take home books and info sheet with info about what to review with their child.
I am concerned that my LLI groups will occur in the classroom next year.
26


















LLI is a great program and most of our students make great growth. Parents and students loved the
Take-Home Books.
"We love having the books at home to read with our child"
I hope our district can afford to buy the Grade 3 LLI kit.
It's nice to see the kids reading books.
Parents are generally very happy with the intervention for its home books and their child's increase
in interest for reading.
This was my first year doing LLI, most students made reading gains.
Students made slow consistent progress. Strategies were easy to carry over into classroom
activities.
"I can't believe how much progress they have made in such a short period of time."
It's hard to service only 3 students 30-40 mins a day with 24 students in the classroom
I love Orson and Taco books- from a student
"I love the successes the students are making with LLI."
Teachers were very happy with the progress they saw in their students.
We can't wait for the 3rd grade boxes!!!!!
Most of the gen ed classroom teachers felt that it was too time consuming to be responsible for
both the regular reading group for the student, and the AIS LLI groups.
students coming into first grade made great progress towards their IEP goals
very time consuming when I have a class of 24 to spend 35 minutes with 2or 3 students, though I
do believe the program to be of quality
Parents were pleased with progress. Students have significant disabilities and made gains at their
developmental level.
While I have only just started the program and have not yet been able to complete for the full 18
weeks I have observed some nice growth in my students and they really enjoy the program!
27
Appendix B
The following are the questions from the surveymonkey link:
LLI End of the Year 2011-2012
1. LLI End of Year 2011-2012 Data
OCM BOCES Mission Statement - The mission of the Onondaga-Cortland-Madison BOCES is to
develop and provide educational programs and services of the highest quality for school districts
and the community.
OCM BOCES Slogan - "Committed To Your Success"
*1. In what district were LLI students served?
In what district were LLI students served?
*2. In which building were LLI students served?
In which building were LLI students served?
OCM Cedar Street OCM CTC OCM Homer Elementary
OCM Morgan Road OCM McEvoy OCM Onondaga Road OCM Other
Barry Elementary Burton Street Elementary Bridgeport Elementary
Chestnut Hill Elementary Donlin Drive Elementary
East Hill Elementary Elbridge Elementary Elmcrest Elementary
Jamesville Elementary Kathryn C. Heffernan Elementary Lake Street Elementary
Liverpool Elementary Lyncourt Elementary Moses Dewitt Elementary
Onondaga Road Elementary Parker Elementary Ramsdell Elementary
Randall Elementary Smith Elementary Solvay Elementary
Split Rock Elementary Stonehedge Blue Elementary Stonehedge Gold Elementary
Tecumseh Elementary Tully Elementary Virgil Elementary
Willow Field Elementary Rockwell Elementary Other
*3. What is your title in your building?
What is your title in your building?
*4. In what other role(s) (if any) do you primarily work in the school ?
In what other role(s) (if any) do you primarily work in the school ?
have no other role
are a Title I or reading teacher
a classroom teacher
a staff developer
28
a special education teacher
an ELL teacher
a school or district administrator
Reading Recovery
*5. How many groups were you able to serve this year?
How many groups were you able to serve this year?
*6. How many LLI students were able to receive the service?
How many LLI students were able to receive the service?
*7. How many LLI students accelerated and reached benchmark?
How many LLI students accelerated and reached benchmark?
*8. How many LLI students received the full 18 weeks and were recommended for further
support? (not including the students who were provided additional weeks in LLI)
How many LLI students received the full 18 weeks and were recommended for further support?
(not including the students who were provided additional weeks in LLI)
*9. How many LLI students received longer than 18 weeks of LLI service?
How many LLI students received longer than 18 weeks of LLI service?
*10. How many LLI students moved out of district while being served?
How many LLI students moved out of district while being served?
*11. How many LLI students were removed from LLI under unusual circumstances with fewer
than 18 weeks of instruction?
How many LLI students were removed from LLI under unusual circumstances with fewer than 18
weeks of instruction?
Please specify reason
*12. If a vacancy occurred in a LLI group was the resulting open slot filled by another child?
If a vacancy occurred in a LLI group was the resulting open slot filled by another child?
Yes
No
Sometimes
(use provided space to clarify a No or Sometimes response)
*13. How many LLI students were receiving the intervention and didn't receive the full number of
weeks because of insufficient time to complete the intervention?
29
How many LLI students were receiving the intervention and didn't receive the full number of
weeks because of insufficient time to complete the intervention?
*14. How many LLI students were considered for retention?
How many LLI students were considered for retention?
*15. Which assessment system do you use to determine instructional text reading level and exit
text reading levels?
Which assessment system do you use to determine instructional text reading level and exit text
reading levels?
DIBELS
DRA
Fountas and Pinnell Benchmark Assessment System
Rigby PM
Other
*16. Did each student receive a benchmark assessment to determine their entry text level before
the LLI placement group?
Did each student receive a benchmark assessment to determine their entry text level before the LLI
placement group? Yes No
*17. Did each student receive a benchmark assessment to determine exit text level after receiving
the LLI services?
Did each student receive a benchmark assessment to determine exit text level after receiving the
LLI services? Yes No
*18. What informal responses to LLI did teachers, administrators, parents and students make?
For example:
The following was shared by a Leveled Literacy Intervention teacher:
"This is the very first year I will be able to send all students on grade level!"
What informal responses to LLI did teachers, administrators, parents and students make? For
example: The following was shared by a Leveled Literacy Intervention teacher: "This is the very
first year I will be able to send all students on grade level!"
30