Using Keystone Books
for Instruction and
Assessment
Keystone Assessment Books?
Original Goal
•To develop a list of high-quality leveled texts
from multiple publishers to use for frequent
progress monitoring.
•To prevent overuse of published assessment
‘kits’ (DRA, Benchmark Reading
Assessment and Dominie Reading Assessment)
that are integral to school district assessments.
Keystone Assessment Books?
In an effort to differentiate between the
numerous associations for
“benchmark books” in education today
we chose
Keystone
as the descriptor
for the UALR assessment collection.
The associations conveyed by
the term
“Keystone” are:
A central supporting element
of a larger structure
such as a theory or organization,
without which the whole
structure would collapse,
a central principle on which all
depend.
(The definition)
Purpose of Keystone Book Collection
A Keystone book
is a reliable exemplar selected for each
level on the gradient.
Keystone books
are used to document student reading
progress over time or to place students
in initial instructional groups.
Using books that allow us to
observe a student’s reading
behaviors at the
instructional level is a
central supporting element
necessary for teachers
in the on-going
development of the literacy
processing system
(the larger structure).
Process
Identify a large variety of books at each
level of the gradient from a variety of
publishers.
The original list was compiled by
Reading Recovery teachers and
Literacy Coaches
from Conway School District in Arkansas.
It represented a limited number of
publishers in the collection.
Process
2008
More books were added to the list
with a larger variety of publishers
represented.
Reading Recovery Teachers and
Literacy coaches from the
Russellville, AR School District
reviewed the existing book
introductions and suggested new
books to be added.
An On-going Process
Over a Four Year Period
Beginning
Spring 2010
In collaboration with intervention
teachers, literacy coaches, and
university faculty
the large collection of
Keystone Books were field-tested
and the list was refined based on
teacher feedback.
An On-going Process
Over a Four Year Period
Beginning
Spring 2010
At the end of Year Three, a team of
Comprehensive Intervention Model
(CIM) trainers from five
universities reviewed the teachers’
feedback,
selected the most suitable books,
and created standardized book
introductions.
Validating the Collection
This list became the testing packet* that
was used in the research study.
Participants in the study included
47 reading teachers
24 schools
9 states
806 K-2 students
* See collection available from Song Lake books.
Validating the Collection
“The findings provide strong evidence
that the Keystone Collection
(Levels A-M)
represents a valid and reliable method
of estimating
children’s text-reading level
through an authentic,
teacher-administered reading task.”
Dorn, L.J., Forbes, S., Poparad, M.A. & Schubert, B. (Eds). (2015). Changing
Minds, Changing Schools, Changing Systems: A Comprehensive Literacy
Design for School Improvement. Hameray Publishing Group.
Validating the Collection
You can access more information
about the validation study in Chapter 3
of Changing Minds, Changing
Schools, Changing Systems: A
Comprehensive Literacy Design for
School Improvement
Dorn, L.J., Forbes, S., Poparad, M.A. & Schubert, B. (Eds).
(2015). Changing Minds, Changing Schools, Changing
Systems: A Comprehensive Literacy Design for School
Improvement. Hameray Publishing Group.
The Keystone Collections*
A complete list of the books with book
introductions and directions can be
downloaded from
www.ualr.edu/literacy
Choose “Teaching Resources”
To order the set of books (70 titles from
11 publishers) used in the study download
the order form from
Song Lake books available at the above
site.
Original sources for text levels.
Elements of a good book orientation
Literacy Lessons II page 91 M.M. Clay (2005)
“Having carefully selected a book for a particular child
the teacher reads it herself, thinking about the best ways
to orient this child to this book.
Make the child familiar
• with the story
• with the plot
• with the phrases of language that he might never have
heard,
• with unusual names and new words,
• and with old words used in an unusual way”
Create short, standardized introductions for
each Keystone book
two or three sentences that focus on the
main idea of the book including genre or
story structure
use some of the language, which
might include names of characters
use specific or technical vocabulary,
or other unusual language structures
An Observation Survey of Early Literacy Achievement
M.M. Clay 2013 p.2
“To use systematic observation the
teacher has to set time aside
from teaching to become a
neutral observer.”
Use running records.
There are three important reasons why the
Keystone Books can be useful for progress
monitoring:
1) represent a wide variety of publishers,
authors, text types, genres, formats, and
writing styles;
2) are not aligned with any particular
publisher; and
3) can be used interchangeably for guided
reading groups and progress monitoring.
A set of directions
provides for
systematic observation,
a standard task,
a standard way of setting up the
task (administration).
Attach introductions to back cover
and include the following:
Book title
Level
Number of running words
Number of errors a reader can
make before the score goes
below 90 percent accuracy
Protocol for Using Level 1-16
Benchmark Books
• Teacher will read the introduction
aloud and then have the student read
the story aloud
• 90% accuracy or above with good
processing is probably a good
instructional level
• Comprehension Guides and Fluency
rubric will be used to determine the
“just right” level.
Protocol for Using Benchmark Books
Higher Than Level 16
• Teacher will read the introduction aloud
and then have the student read the text
aloud
• Teacher takes running record on a
selected passage. 90% accuracy
or above with good processing
is probably a good instructional
level
• Check comprehension
Protocol for Using Benchmark Books
Higher Than Level 16
• Teacher then asks the student to
read a short section silently.
• Teacher checks comprehension
• Comprehension Guides and
Fluency rubric will be used to
determine the “just right” level
Comprehension Guides and
Fluency rubrics
are available in
Interventions that Work
Dorn, L.J. & Soffos, C (2012). Interventions that Work.
Boston,MA: Pearson
Why are you assessing the student?
 to determine, in general, the instructional
level?
 to determine the level the
student can read different genres?
What makes a book hard?
 Structure
 Vocabulary
 Sentence length
 A reader’s prior knowledge
 Narrative or nonfiction
Mondo Publishing
Book Shop Series
GRL B
Interv. Level 3
Introduction: The boy in this book hits the
ball but it went into the wrong place. Will
he get a home run?
Introduction: This book is about living things
and how many legs each one has.
Mondo
Book Shop
Series
GRL C
Interv.
Level 6
Title: Why Caterpillars Become Butterflies
Level: GRL G
Word Count: 198 Errors: 21
Genre: Animal Fantasy Fiction
Introduction: Once there was a lonely caterpillar.
He called out to see if there were others who could
come out. Read to find out how these caterpillars
changed so they could get together.
Item #1404536930 Wright Group/Gear Up
Pioneer Valley
Educational Press
Literacy Footprints
Nonfiction Set 7
GRL G
Interv. Level 12
Title: Butterflies
Level: GRL G Word Count: 213 Errors: 23
Genre: Non Fiction
Introduction: This books tells us all about butterflies, the
parts of their body, how they eat and how they go through
metamorphosis. That means changing from an egg to a
caterpillar to a chrysalis and finally a butterfly. Read to
find out how that happens.
Item #NFS72 Pioneer Valley Educational Press
Where are we now in the process?
 A data base for the collection is complete.
 A smaller set of books identified for
research purposes.
Directions for administration is complete
Comprehension guides are complete
In the process of writing prompts for
writing about reading.
 In the process of adding books from new
publishers to the collection.
Finalizing the Process
This is a PILOT Project
We expect anyone using these books as
presented for assessment will provide us
with feedback.
The UALR committee will use this
feedback to revise the documents into
their final form for distribution.
This will continue to be
a work in progress.