An Investigative Analysis of
Evidence-Based Practices
Changnam Lee, Ph.D.
Kennesaw State University
Kennesaw, GA
Where are we in Using Evidence-Based
Practices (EBPs) of Teaching?
• Criteria for EBPs are defined (Gersten et al.,
2005; Horner et al., 2005 )
• EBPs are identified (e.g., What Works
Clearinghouse, National Center on Intensive
Intervention).
• Using them in classrooms is uncertain (Cook &
Tankersley, 2013;Darling-Hammond, 2009).
We have EBPs, but how can we Use
them?
• “America has the most effective instructional
methods in the world—yet we are not using
them” (Binder & Watkins, 1989, p. 33).
A Proposed Model: ADAPT
(Lee & Picanco, 2013)
Independent
performance
Intensity of structure
Acquisition
Fluency
Maintenance
Generalization
Direct Instruction
Behavior analysis
Mnemonic Instruction
Peer tutoring
Constructivist instruction
Cooperative learning
Learning strategies and study skills instruction
Mastery learning
Research Questions for an
Investigative Analysis
1. Can evidence-based teaching practices be
reliably classified according to the phases of
learning: acquisition, proficiency (fluency),
maintenance, and generalization?
2. How many EBPs are found in relation to
academic subject areas and grade levels?
3. When classified, what EBPs can be used for
each phase of learning?
Sources of EBP
• Evidence-Based Intervention Network:
http://ebi.missouri.edu/
• Intervention Central: http://www.interventioncentral.org/
• National Institute for Direct Instruction:
http://www.nifdi.org/15/
• National Center on Response to Intervention
(http://www.rti4success.org/) renamed to NC on
Intensive Intervention
(http://www.intensiveintervention.org/)
• U.S. Department of Education web site, What Works
Clearinghouse (WWC): http://ies.ed.gov/ncee/wwc/
• OSEP Technical Assistance Center on Positive Behavior
Interventions and Supports (PBIS): http://www.pbis.org/
• IRIS Center (http://iris.peabody.vanderbilt.edu/ebp/)
The Study Procedure
• Participants: Two individuals with Ph.D.
degrees participated in this study. Their
responsibilities included being trained to
review and rate EBPs.
Selecting and Rating EBPs
The two raters independently:
1. Logged on to What Works Clearinghouse (WWC)
at http://ies.ed.gov/ncee/wwc/.
2. Clicked on the Find What Works tab.
3. Selected the Subject Area (math, literacy, or
science).
4. Selected the Positive or Potentially Positive
option.
5. Summarized and rated the items by using the
rubric and the data collection form.
Training and Developing a Rubric
1. Both raters read the Lee and Picanco (2013) article for
conceptual understanding of the task.
2. Each rater did an initial round of ratings, using a
preliminary version of a rubric with a five-point scale.
3. The two raters met weekly to discuss the ratings for two
weeks before beginning independent ratings.
4. Together, they developed a comprehensive rubric with
commonly found descriptors for each phase of learning.
5. With the comprehensive rubric with a five-point scale, the
raters rerated the WWC EBPs that they rated prior to the
rubric.
6. Each rater rated other EBP items independently.
Results: EBPs Selected and Rated
Mathematics
(13 EBPs)
Literacy (48
EBPs)
Science (3
EBPs)
Pre-Elementary
and Early
Elementary
(PreK to Grade
2)
Late Elementary High School
and Middle
(Grades 9-12)
Grades (Grades
3-8)
7 (53.8%)
7 (53.8%)
3 (23.1%)
34 (70.8%)
19 (39.6%)
5 (10.4%)
0 (0%)
3 (100%)
1 (33.3%)
Reliability Percentages in Subject Areas
of WWC Rating
Types of
Reliability
Mathematics
(52 ratings)
Literacy (192 Science (12
ratings)
ratings)
Perfect
Agreement
33 (63.5%)
121 (63.0%) 6 (50.0%)
Agreement
within One
Point
Difference
46 (88.5%)
167 (87.0%) 12 (100%)
Summary of EBPs Rated High (4 and 5)
by Phases of Learning
Summary and Discussion
1. A relatively large number of EBPs in Literacy have been
identified from research whereas small numbers of them
are found in mathematics and science.
2. In the area of Literacy, a predominantly large percentage
(70.8%) of EBPs have been identified for pre-elementary
or early elementary grades with a small percentage for
high school (10.4%).
3. Mathematics EBPs largely focused on the Acquisition
phase whereas the majority of Literacy EBPs focused on
Proficiency, followed by Acquisition.
4. The Generalization (applying to real-world situations)
emphasis is weak for literacy although the Common Core
Standards emphasize application and generalization.