Identifying Research-based
Practices for RtI: Scientificallybased Reading Instruction
Janet S. Twyman, Ph.D., BCBA
Presented at the 2nd Annual
Evidenced-Based Practices Summit
of the Wing Institute
Berkeley, CA
April 26, 2007
Historical Perspective:
Reading in America
 After a century and a half of universal public education,
and despite the highest per-pupil expenditure on public
elementary and secondary education in the world,
40 percent of U.S. fourth-graders are reading below the
minimally acceptable level (NAEP 2006).
 For minority students in inner-city schools, the reading
failure rate is 65 percent.
 Children who don’t read by fourth grade almost always
fall behind in all other subjects, frequently require special
education programs, and, as adults, have higher rates of
drug addiction, incarceration, and welfare dependency.
Legislation to Avert our
National Reading Crisis
The No Child Left Behind Act of 2001 commits the
nation to ensuring that every child in the country will learn
to read by the end of third grade.
Within NCLB, Reading First was first authorized at more
than $6 billion to apply scientifically based reading
research to strengthen children’s reading skills from
kindergarten through third grade.
Yet, at $1 billion per year it accounts for only 2 percent of federal education spending.
Reading First – An Overview
 As part of NCLB, it mandates that the best scientific
research should guide the teaching of reading.
 It supports research-based methods to ensure that
every child in public schools reads at or above grade
level by third grade.
 The essential components of reading instruction are
defined in the bill as explicit and systematic
instruction in phonemic awareness; phonics; vocabulary
development; reading fluency, including oral reading
skills; and reading comprehension strategies.
Reading First –
What it Means to Schools
 Preparing teachers through professional development and other support,
so they can identify specific reading barriers facing their students and
have the tools to effectively help their student to learn to read.
 Selecting or developing rigorous diagnostic reading assessments that
document (1) effectiveness in improving students' reading and (2) the
schools accountability for their results.
 Selecting or developing effective instructional materials, programs, and
strategies to implement methods that have been proven to prevent or
remediate reading failure (including newly identified Response to
Intervention Initiatives).
 Establishing reading programs for students in grades kindergarten
through 3 that are grounded in scientifically-based reading research
(SBRR), in order to ensure that every student can read at grade level or
above by the end of the third grade.
What the Law says is ScientificallyBased Reading Research
The term `scientifically based reading research' means research that-(A) applies rigorous, systematic, and objective procedures to obtain valid knowledge
relevant to reading development, reading instruction, and reading difficulties;
and
(B) includes research that-(i) employs systematic, empirical methods that draw on observation or experiment;
(ii) involves rigorous data analyses that are adequate to test the stated hypotheses
and justify the general conclusions drawn;
(iii) relies on measurements or observational methods that provide valid data across
evaluators and observers and across multiple measurements and observations;
and
(iv) has been accepted by a peer-reviewed journal or approved by a panel of
independent experts through a comparably rigorous, objective, and scientific
review.
Source: H.R.1. 2001
“Scientifically-based”
- Controversies Abound
There little agreement about what constitutes “evidence.”
 NCLB permits both quantitative and qualitative evidence (without clearly
specifying the types of questions best answered by each approach).
 Only small number of interventions used today have been empirically
evaluated by some peer-reviewed standard.
 Evidence-based interventions are less likely to be used than interventions
for which there is no evidence or evidence shows
a lack of impact (Kazdin, 2000).
 In many instances educators/practitioners are not aware of evidencebased interventions (Kratochwill, Albers & Shernoff, 2004).
Source: Detrich, Keyworth, & States (2007). Evidence-based Education:
It Isn’t as Simple as You Might Think.
What the education field refers to
as “Research-Based”
Current uses of the term research–based in federal documents
and journal articles (referred to as “evidence” when
applied to early reading programs):
(1) the program contains elements that research suggests
are effective (see for example Simmons & Kame’enui, 2003)
(2) pretest vs. posttest or simple comparison studies have provided some
evidence of effectiveness for an instructional program, or
(3) the program has undergone some form of scientifically controlled
study, often involving randomized control groups (Coalition for
Evidence–Based Policy, 2003)
However,
…these uses fail to distinguish between the
scientific development of a program, and the
scientific evaluation of outcomes after a program
has been developed.
Types of “Research” to consider
when evaluating programs:

Research-Informed
A program based upon best practices identified by research or contain elements
that other research suggests are effective.
-No requirement that the program itself be scientifically designed.

Research-Filtered (Summative Evaluation)
A program, regardless of how it was designed, measured against an alternative
form of instruction, or at times, no instruction at all.
-No requirement that the program itself be scientifically designed.

Research-Guided (Formative Evaluation)
A program that has been scientifically designed and tested during its
development (most often guided by previous research results).
In the research–guided approach, formative evaluation is intertwined into the
instructional design protocols, and at its most thorough influences program
development through iterations of testing, revising, and testing again.
How does one know what “evidence”
to use?
 Evidence is more than ONE study showing support.
 Involves a body of knowledge, specifying:
 the quality of evidence
 the quantity and type of replication of effects
 Level or rigor of evidence falls along a continuum.
 Sufficient Threshold:
 Evidence must be of a specific quantity and quality before a
program should be considered evidence-based.
 Hierarchy of evidence approach:
 Rely on the best available evidence to date.
Source: Detrich, Keyworth, & States (2007). Evidence-based Education:
It Isn’t as Simple as You Might Think.
Continuums of Evidence
Quantity of the Evidence
Meta-analysis
(systematic review)
Repeated Systematic
Measures
Single Case Replication
(Direct and Parametric)
Threshold
of Convergent Evidence
Evidence
Various Investigations
Single Study
Quality of the Evidence
Program Development
Current “Gold Standard”
High Quality
Randomized Controlled Trial
Research-Guided
and Informed
Single Case Designs
Research-Guided
Semi-Randomized Trials
Well-conducted
Clinical Studies
Research-Filtered
Uncontrolled Studies
Research-Informed
Expert Opinion
Consensus of Experts
General Consensus
Personal Observation
Common Wisdom or Lore
An Example:
How states determined their Reading First programs:
States Report Difficulty in Determining
Evidence-based Reading Programs
Are we preparing our teachers?
How can we expect our
educators to understand
evidence-based instruction
in reading, if our teacher
preparation institutions aren’t
teaching reading science?
85% earned a “failing” grade,
even though a passing grade was
possible if a professor devoted
less than 20 percent of the
lectures to the science of reading.
All 5
components
4/5
Source: National Council on Teacher Quality
3/5
2/5
1/5
None of
the components
When faced with “evidence”
…ask at least three questions:
 Is the evidence valid?
Where does the evidence fit within the continuum?
Could it be biased?
 Is it important?
Are the findings to relevant your situation? What is the
significance of the outcome compared to other options?
 Is it applicable to the learner in front of me?
Are the participants and procedures in the original
research comparable to your learner and setting?
Sources for “Evidence”
http://www.whatworks.ed.gov
Assistance in determining evidence:
Florida Center for Reading Research
www.fcrr.org/
FCRRreports
Provides one of
the most
comprehensive
lists of reviews
of reading
programs
http://www.fcrr.org
Oregon Reading First
“Curriculum Review”
leads to the Consumer's
Guide to Evaluating
Reading Programs an
evaluation of various
reading programs.
http://oregonreadingfirst.uoregon.edu
Big Ideas in Beginning Reading
http://reading.uoregon.edu/curricula/
Promising Practices Network
http://www.promisingpractices.net/programs.asp
National Institute for Literacy
http://www.nifl.gov/cgi-bin/pfr/search.cgi
California
Learning
Resource
Network
Logging in
provides
access to
evaluations of
reading
programs
http://www.clrn.org
Interventions in School Psych.
“Projects” leads to:
Procedural and
Coding Manual for
Identification of
Evidence-Based
Interventions
http://www.sp-ebi.org
Other Interventions
http://nrepp.samhsa.gov/index.htm
IRIS Center – Star Legacy Modules
Interactive Tutorials on:
ACCOMMODATIONS
BEHAVIOR
COLLABORATION
DIFFERENTIATED INSTRUCTION
Response to Intervention
Reading Strategies
DISABILITY
DIVERSITY
Scientifically Based Reading Research (SBRR)
http://www.headsprout.com/aol/teachers.cfm
Reading First
Technical Assistance Centers
Western Regional Reading
First Technical Assistance
Center (WRRFTAC)
Center on Teaching & Learning
University of Oregon (Eugene)
(541) 346-1608
Central Regional Reading First
Technical Assistance Center
(CRRFTAC)
Center for Reading & Language
Arts
The University of Texas at Austin
(512) 232-1901
Eastern Regional Reading First
Technical Assistance Center
(ERRFTAC)
Florida Center for Reading
Research
Florida State University
(Tallahassee)
(850) 644-9352
Wing Institute Home Page
What does this mean for real educators,
real students, real classrooms?
 Find and use programs with rigorous levels of research
 Programs furthest along the continuum of empirical evidence of
effectiveness (quantity, quality, and type of program development)
 Access available public resources reviewing “evidence”
 Learn from the community of available tools and tutorials
 Use and be responsive to data from your own students, classroom,
school
 CCCROD
 CBM
 Progress Monitoring
 DIBELS*
 AIMSweb*
Need for Research into Practice
 A key outcome of evidence-based education is for
scientifically proven interventions to be adopted and
successfully implemented at the “practice” level.
 In order for research to translate into practice,
interventions must be efficacious (have a foundation
in rigorous scientific research) as well as effective
(address the necessary social influence factors to
ensure the desired outcomes).
Source:
RtI facilitates the use of evidence
and research-based programs
 RtI framework encourages a system for identifying:
need, treatments, measuring progress and
outcomes, evaluating teaching practices, and
revising whenever the data indicate to do so.
 RtI is integral to knowing what is working and what
is not, for each individual child.
Relevance for
Special Education students
 For some students require the modification of the
general education reading curriculum, in terms of
content or instructional approach.
 Other students may require general education
classroom reading instruction to be supplemented with
an alternative, more intensive program.
 Yet other students may require a complete replacement
reading program, including other publicly available
materials, or “teacher-devised” curricula.
Suggested Principles Regarding Reading
Instruction for Special Education Students








Address all areas of the reading process as identified by reading research.
Identify, early, students demonstrating difficulty learning to read.
Design a tier (continuum) of intervention services prior to referral to special education.
Include in each special education student’s individual plan of instruction carefully
selected curriculum and instructional techniques matched to student needs.
Base instructional plans on initial and ongoing assessment that reflects research-based
best practice, including a historical analysis of the student’s reading instruction,
consideration of the student’s specific characteristics and knowledge of the student’s
deficits and strengths.
Use assessment data to guide and inform instruction.
Vary the intensity of intervention (amount of time for reading instruction, the size of the
instructional group, means of service delivery, and the nature of the selected reading
curriculum) provided to individual special education students based on the level of
need and progress made (or lack thereof).
Use technology, as appropriate, to enhance the development of reading skills and to
ameliorate the effects of deficiencies in reading.
Handout available in resource packet.
IMPORTANT CAVEAT
 Don’t become formulaic;
even with SBRR or RTI
 Never lose sight of your own ongoing
data collection and analysis