1
Fountas and Pinnell
Leveled Literacy Intervention
Spring 2011
Program Evaluation
Leveled Literacy Intervention
2
Fountas and Pinnell Leveled Literacy Intervention Evaluation Report
Table of Contents
I.
Executive Summary
pg. 3
II.
Introduction to the Report
pg. 7
III.
a. Purpose of the Evaluation
pg. 7
b. Audiences for the evaluation report
pg. 7
c. Limitations of the evaluation
pg. 8
d. Overview of report contents
pg. 8
Focus of the Evaluation
pg. 8
a. Description of the evaluation objects
pg. 8
b. Evaluative questions used to focus the study
pg. 9
c. Information needed to complete the evaluation
pg. 9
IV.
Brief Overview of evaluation plan and procedures
pg. 10
V.
Presentation of evaluation results
pg. 12
VI.
VII.
a. Summary of evaluation findings
pg. 13
b. Interpretation of evaluation findings
pg. 13
Conclusions and recommendations
pg. 14
Appendices
pg. 15
Leveled Literacy Intervention
2
3
Executive Summary
Purpose
The Fountas and Pinnell Leveled Literacy Program began at Lyman Hall Elementary
School three years ago. The LLI Program is available for students in kindergarten through
fifth grade. The Leveled Literacy Intervention System is a small-group, supplementary
intervention designed for children who find reading and writing difficult. LLI is designed to
bring children quickly up to grade-level competency—in 14 to 18 weeks on average. LLI
serves those students who need intensive support to achieve grade-level competency.
These children are the lowest achieving children in the classroom who are not receiving
another supplementary intervention. Each lesson in the LLI system also provides specific
suggestions for supporting English language learners who are selected for the program.
An evaluation of the Leveled Literacy Intervention Program was conducted for a few
reasons. The program has been in place for three years and has never been evaluated. Doing
an evaluation was a good way to decide what is working with the program, what may need
to be changed, or what is not working with the program. Several questions were addressed
in the evaluation. The main questions addressed and researched for this program
evaluation is as follows: How many students recover before 14 weeks, during the 14-18
week period or do not recover at the completion of 18 weeks. What are the parent, teacher
and staff perceptions about the Leveled Literacy System?
Information Collection
When completing the LLI Program evaluation, vital information was gathered. The
primary source of collecting data for this evaluation was through formal DRA’s (directed
reading assessments) and informal DRA’s known as cold reads. The information gathered
through these assessments were then used to answer evaluation questions related to
Leveled Literacy Intervention
4
whether students are reading below grade level or reading on grade level. Interviews were
also conducted with parents, teachers, and staff members to better understand the
perceptions about the LLI Program.
Students had to answer the following questions when completing the DRA and cold
reads. As well as answering these questions students receive a percentage on their reading
ability. (see appendices DRA)

Look at the title and the pictures and tell me what you think is going to happen in
the book? (Prediction)

Start at the beginning and tell me what happened in the book? (Retelling)

What was your favorite part and why?

What connection did you make while reading the book?
By having students answer these questions many conclusions were made. Below is a list of
findings. A more detailed list with explanations can be found in the full evaluation report.
Findings

Out of 5 students evaluated for the LLI Program 0 of the students were on grade
level before 14 weeks.

Out of the 5 students evaluated for the LLI Program 0 of the students were on grade
at 14 weeks.

Out of the 5 students evaluated for the LLI Program 0 of the students were on grade
level after the 18 week completion of the program.
The findings from the parent, teacher and staff surveys were unanimous. The main concern
expressed through the survey was that the program benefits middle class society as well as
those students with a good English vocabulary. The students within this program are
students who speak Spanish as their first language and English as their second. Their
Leveled Literacy Intervention
4
5
families are below the poverty line, they are considered migrant students who move for
work and they do not receive help from home. The teachers and staff suggested piloting a
program that is specifically geared toward low end, second language learners.
Judgments
Overall the students and staff were very willing to help in data collection. The
evaluation indicates that the program is not recovering students to grade level reading
within the specified time, 14-18 weeks. The evaluation also shows that students are
becoming frustrated with their reading skills/abilities and are then choosing not to read.
Recommendations
After viewing all of the data, a list of recommendations has been made. Please refer
to the entire report for more information and explanations.

Continue to use the LLI structure of even days reading books and odd days
writing about the books, 5 minutes reread, 5 minutes phonics work, 15
minutes new book or writing depending on day and 5 minutes word work.
Seeing Star boxes can add to classroom word work to allow for
differentiation, more exposure and use of nonsense words (more
explanation is given under recommendations later in the report).

Instead of using LLI books use Lyman Hall Leveled Books developed for English
Language Learners on their independent reading level and instructional reading
level.

Before taking a warm read or formal DRA allow the student to read a warm
read book (independent level) to get their confidence built and then
administer the cold read or formal DRA.
Leveled Literacy Intervention
6
The LLI Program at Lyman Hall Elementary School is not working well and needs to
be improved. From the feedback and data collected it is obvious that the program is not
recovering students to grade level within the 14-18 week time line. With the
recommendations provided, the LLI Program can in turn reach below grade level students
and have them continue to grow in their reading levels.
Please review the full report for more information and data that reflects the information
stated in the Executive Summary.
Leveled Literacy Intervention
6
7
Introduction
Purpose of Evaluation
The main purpose of the Leveled Literacy System evaluation was to determine whether
the program was meeting the objectives that had been outlined in the description of the
program. This evaluation evaluates if the students receive the amount of support needed in
order to benefit from the system’s objectives. The main question that arises during this
evaluation was, are the students being able to close the reading gap after completing the
program? The evaluation of the Leveled Literacy System was conducted to research this
question to determine if this program was benefiting the student’s reading level. Data was
collected from administration and staff members who have first-hand experience with this
literacy program. This data gave us a picture of how this program was working within the real
world setting.
Audiences of the Report
This evaluation can help all staff members that are involved with the Leveled Literacy
System. Teachers who use this intervention system to improve their student’s reading level are
the targeted audience who will benefit the most from the outcome of this evaluation. Other
groups of people like parents and other school systems can use this evaluation as an insight as
to how the program works and how effective the program is. Each person who reviews this
evaluation will be able to see the positives and negatives, as well as the areas in which
improvements can be made.
Leveled Literacy Intervention
8
Limitations of the Evaluation
This evaluation only had several limitations. The main limitation was the teachers did
not have an universal recording system where all administrators could see the process of the
students who participated in the leveled literacy intervention. The teachers track their students
on their own and keep their recordings for their records. If there was a progress monitoring
sheet for each teacher to record the students’ progress then the administrators and other
teachers could easily keep track of each student and his progress within the system. This
progress monitoring sheet would be saved on the school drive and would be created to use for
the recording of the Level Literacy System’s outcomes.
Overview of the Report Contents
This report includes a description of the Fountas and Pinnell Leveled Literacy System
and how it is used to increase the reading levels of struggling students. The audience of this
evaluation will learn what questions were important when evaluating the system. Included is
also a description of the evaluation plan and procedures which were used to evaluate the
system. A summary and interpretation of the findings from the research of the level literacy
system are also included as a part of the evaluation. In the final section, recommendations and
conclusions are made for this intervention program.
FOCUS OF THE ELVAUATION
Description of the evaluation object:
The evaluation focuses on the Fountas and Pinnell Leveled Literacy Intervention System.
This intervention program is designed for a small group setup. The system is not a reading
program, but a supplementary program to be used in addition to the reading program that is
already in place for that school system. The small group who received the supplementary
program are those students who need intense support to reach grade level competency.
Leveled Literacy Intervention
8
9
Therefore, the students who receive the leveled literacy intervention are low-achieving children
who are not receiving other forms of supplementary intervention.
The leveled literacy intervention is a short term program that has been proven to bring
students up to grade level competency in reading in fourteen to eighteen weeks. It is broken
into three systems that support different levels of learners. Students are tested using the
running record testing program to determine which system (orange, green, or blue) that the
child needs to begin the appropriate support instruction.
Each system contains a collection of at least seventy books. For each book, there is a
lesson for the teacher to follow. Within each lesson, there is an emphasis on reading skills as
well as writing skills. The specific components of each lesson are listed below:

Comprehension strategies

Attention to fiction and nonfiction texts

Specific work on sounds, letters, and words (and how words “work”)

Vocabulary practice

Fluency and phrased reading

Writing about reading to practice writing strategies
Evaluative questions used to focus the study:

How many students recover before 14-18 weeks, during the 14-18 week period
or not recover at the completion of the 18 week?

What are the parent, teacher, and staff perceptions about the leveled literacy
system?
Information needed to complete the evaluation:
Before beginning the evaluation information had to be collected on the Leveled Literacy
Intervention Program. Individual student’s formal and informal DRA assessments were collected
Leveled Literacy Intervention
10
to evaluate their progress throughout the program. This is the greatest source of information for
us to use as we measured the effectiveness of the program. The formal DRA assessment was
given twice during the program and the informal DRA assessment was given every two weeks
with a total of sixteen assessments. Depending upon the score on these assessments
determined whether the student would then revisit the current level or progress to a new level
and which level they would progress to.
Brief Overview of Evaluation Plan and Procedures
Before this evaluation began, there were extensive steps to complete the background
research for the Leveled Literacy System. Research was completed to gain information
about the Leveled Literacy System. There were documents that were collected to
understand the objectives and goals of the program. Documented outcomes of students
who completed the program were analyzed to determine exactly how the program is
working in a particular school system. The main focus of the evaluation was discussed
based on items collected. The evaluation questions were formed after these items had been
analyzed.
The second step of the evaluation was to complete an evaluability assessment. This
assessment is a tool that serves a communication between the stakeholders and evaluators
to make sure the program is worthy of the evaluation. Everyone that was involved in this
evaluation was a part of the evaluability assessment. Teachers and parents were able to
share concerns that were taken into consideration for the assessment. Including all persons
involved allowed the evaluators to see if the program was meeting the needs of all.
After reviewing the evaluability assessment, the decision was made that the
evaluation would be completed. The idea was discussed that the outcome of this evaluation
would help school systems across the state to determine if the program would work based
Leveled Literacy Intervention
10
11
on the specific needs. Documentation has been withheld from testing, parent and teacher
surveys, and student achievement to provide the information needed throughout the
process.
After the evaulability assessment was complete, the next step of the evaluation was
to create the standards and criteria. Each standard was used to evaluate the success of each
question in the evaluation. The standards and criteria that were produced followed along
with the Georgia Performance Standards for the specific grade level that it was being used
in. The standards also align with the evaluation questions that were produced at the
beginning of the evaluation.
A spreadsheet was created to organize all information that summarized the
evaluation plan. Within this spreadsheet, one could determine the evaluation questions,
information required, design, information source, method of collecting information,
sampling, information collection procedures, schedule, and analysis procedures. It was
determined that the data collected would come from teacher information about the success
of students within a specific grade level.
Finally, the plan was placed into action when the data collection process began.
Students were monitored as they completed the Leveled Literacy System with their
homeroom teacher. This tool was used to generate a specific percentage of students who
were closing the reading gap before the 14 week period, during the 14 – 18 week period, or
not closing the gap at all. We compared these results with the results of the teacher surveys
to obtain any relation between the program outcomes and the administers.
Leveled Literacy Intervention
12
Presentation of Results
Summary of Findings
Conclusions were drawn based on test scores from Running Records and Developmental
Reading Assessments. Feedback from parents and teachers were also used to make
conclusions. Zero percent of students between the 14-18 weeks of intervention, closed the
reading gap to meet grade level peers and grade level standards according to the test results.
When conducting the online survey, we had lack of parent response, and ninety-five
percent of teachers responded. Ninety-five percent of teachers felt that the Fountas and Pinnell
program was not an efficient reading intervention program. As mentioned earlier the report the
program is too repetitive and students lack motivation for moving up levels. It was also stated
that students were frustrated and bored with the Leveled Literacy Intervention program causing
lack of motivation to build fluency and comprehension. A first grade teacher stated that
students need hands-on learning manipulatives and visuals to manipulate diphthongs, diagraphs
and letter sounds to build fluency. Another first grade teacher, expressed concerns with the lack
of background knowledge and vocabulary for second grade content due to lack of parent
involvement. Zero percent of parents participated in the online survey of the Fountas and
Pinnell Literacy Program.
Interpretations of the Findings
The evaluation indicates that the program is not helping students close gaps in order to
meet grade level reading competency. Because students are bored with the repetitive nature of
the program, they are unable to view themselves as successful grade level readers. It is also
mentioned that the program is designed for middle class students who speak only English. The
main concern expressed through the survey was that the program benefits middle class
Leveled Literacy Intervention
12
13
society as well as those students with a good English vocabulary. The students within this
program are students who speak Spanish as their first language and English as their second.
Another concern is that when using the program, teachers do not have the opportunity
to make the learning meaningful. Research states that when students internalize and make
connections to their reading they are more likely to build fluency and comprehension. Due to
the students’ lack of exposure to reading content and vocabulary, they are unable to make
connections. These students’ first need to understand why it’s important to be able to read,
then they will be more likely to internalize and connect to the content; lack of parent
involvement creates a huge barrier for these students.
Conclusions and Recommendations
After reading the data, a list of recommendations with explanations has been made. Since
teachers believe the root of the problem is the focus on students who speak one language and
the repetitive nature of the program, suggestions have been made to incorporate more hands
on, visual learning.
1.
Teachers can incorporate graphic organizers to aid in teaching comprehension
strategies. Use Lyman Hall Leveled Books developed for English Language Learners on their
independent reading level and instructional reading level; then have students create graphic
organizers in small groups to connect to the content.
Comprehension strategies can be taught through making hands on material that can
help students visualize and connect to the content. For students to find the plot of the story,
they could use paper to create a house. At the top of the house, they could write or draw about
what happened in the beginning of the story, in the middle of the house they could write or
draw about the middle of the story and at the bottom of the house they could write or draw
about what happened at the end of the story; students have now created a graphic organizer.
Leveled Literacy Intervention
14
Before assessing comprehension by using a cold read or formal DRA allow the student
to read a warm read book (independent level) to get their confidence built and then
administer the cold read or formal DRA.
2. Teachers could incorporate Seeing Stars boxes into daily reading rotations. The teacher
holds up a nonsense word (not a real word) from the box. She then has students use letter tiles
to make the word. A student reads the word and then the teacher asks the student to delete or
add letters to create a new word. At first, the goal is for the teacher to have the students
create words they can easily recognize to build self-esteem then, the teacher has the students
build more difficult words as they become more comfortable. Seeing star boxes allow teachers
to individualize instruction for phonemic awareness, sight words and spelling, which also builds
grade level fluency. On Monday, the teacher would get a baseline and on Friday, teachers will
assess the phonemic awareness skill worked on for the week. For example if the teacher worked
on long vowels she would get a baseline on Monday to see what the students already know.
Throughout the week, the teacher would focus on long vowels using Seeing Star Boxes during
reading rotations. On Fridays, she would assess words the students have never seen to get the
results and help guide her instruction for the next week. If the students needed more work on
long vowels, she would continue to work long vowels and create new words for the following
week to assess.
Leveled Literacy Intervention
14
15
Appendices
DRA short form for recording reading accuracy and comprehension.
Leveled Literacy Intervention
16
Leveled Literacy Intervention
16