Evidence-Based Practice: Yes,
but with Cautions &
Recommendations
Steve Graham
Vanderbilt University
• I will use Writing, my area of expertise to
illustrate the 5 points that I plan to make.
POINT 1
• The use of Evidence-Based practices is
more trustworthy than the use of practices
based on professional (expert) judgment
or teacher experience
2 Sources for Effective Practice
• Professional Writers – many draw on their
experience to give advice on either how to
teach writing or facilitate writing
development
• This advice ranges from useful to
simplistic to ridiculous
Don’t use words too big for the subject.
C.S. Lewis
When you catch an adjective kill it.
Mark Twain
•
•
•
•
Have something to say.
Say it.
Stop when you have said it.
Give it an accurate title.
Rules for Writing a Paragraph
Set type for as long as you can hold your
breath without getting blue in the face
Then put in a comma
When you yawn put in a semicolon, and
when you sneeze, that’s time for a
paragraph.
Sources for Effective Practice
• Experienced Writing Teachers
This can involve the insights they have
acquired from teaching or insights
acquired from others who study them
LIMITATIONS
Difficult to separate “wheat from shaft”
Often no direct evidence on effectiveness (validity
-- testimonials)
Evidence is often selective
(generalizability -- possible bias)
Evidence often based on 1 or a few teachers
(reliability – can’t predict its effectiveness)
Sources for Effective Practice
• Scientific Studies -- Should be more trustworthy than
insight and experience
Collect evidence
Present findings for all participants
Replicability
Strength of Impact
Point 2
• Make the best of the available evidence.
The data-base is currently very thin for
some areas.
I illustrate this with writing instruction for
students with LD – This is based on a
recent chapter by myself, Natalie
Olinghouse, and Karen Harris
FINDINGS – 4 or more studies
• 1. Teach students with LD strategies for
planning, revising, and editing their
compositions (strong positive impact).
• 2. Use direct instruction to teach grammar
skills to students with LD (moderate
positive impact).
FINDINGS – 2 to 4 studies
3. Have students with LD work cooperatively
with other struggling writers to plan, draft,
revise, and edit their compositions (strong
positive impact).
4. Explicitly teach students with LD
strategies for producing written summaries
of reading material (strong positive
impact).
FINDINGS – 2 to 4 studies
5. Make the process writing approach more
effective for students with LD by explicitly
teaching them strategies for carrying out
the processes of planning, revising, and
editing (strong positive impact).
6. Set clear and specific goals for what
students with LD are to accomplish in their
writing (moderate to strong positive
impact).
This is IT!!!
We can only make 6 recommendations on
writing instruction for students with LD that
have some scientific backing
POINT 3 – The What Works
Clearinghouse Syndrome
In other words – only certain types of studies
will be included in determining what is or is
not evidence-based
They have adopted a best evidence
synthesis approach
As a result, they are often referred to as:
THE WHAT DOESN’T WORK
CLEARINGHOUSE
RECOMMENDATION
• We need to take a careful look at our definitions
of evidence-based practice
• Are they too stringent are they too lax?
• More pointedly, we need to systematically study
the process and criteria used to identify
evidence-based practices.
POINT 4
• Until our data base is more sufficiently
developed, we will need to supplement our
evidence-based recommendations with
recommendations taken from areas other
than experimental and single-subject
design research
We will need to continue to draw on the
practices of effective teachers.
For example, we located 5 qualitative
studies that looked at the writing
instructional practices of effective teachers
and schools. We identified those practices
employed by a majority of teachers as
promising
Some Examples
•
•
•
•
Dedicate time to writing, including writing
across the curriculum.
Involve students in various forms of writing
over time.
Have students plan, draft, revise, edit, and
share their work.
Keep students engaged and on-task by
involving them in thoughtful activities (such as
planning a composition) versus activities that
require little thoughtfulness (such as
completing a workbook page that can be
finished quickly).
Some More Examples
•
•
•
•
Model, explain, and provide guided assistance when
teaching.
Provide just enough support so that students can
make progress or carry out writing tasks and
processes, but encourage students to act in a selfregulated fashion, doing as much as they can on their
own.
Be enthusiastic about writing and create a positive
environment, where students are constantly
encouraged to try hard, believe that what they are
learning will permit them to write well, and attribute
success to effort and what they learned.
Set high expectations for students, encouraging them
to surpass their previous efforts or accomplishments.
POINT 5
Be Cautious About What Is Promised, as
there is no guarantee that evidence-based
practices travel easily or as expected
RTI Is A Case In Point
Based in part on the premise that the
implementation of effective practices
(evidence-based) in the regular classroom
will reduce the number of students who
are misidentified as LD (in other words,
eliminate educational casualties)
While this may be the case, this is a huge
burden for Evidence-Based practices, as
we have some schools where the writing
of three quarter or more of the students is
not strong enough to meet classroom
demands.
In addition, much can go wrong in the
implementation of any educational
approach
Case in Point 2
The National Reading Panel
Recommendations
MAY BE A CASE OF OVER-SELL THAT
WILL COME BACK AND LITERALY BITE
US IN THE BUTT.
WE MUST BE REASONABLE AND
CAUTIOUS IN HOW WE SELL
EVIDENCE-BASED PRACTICES TO THE
EDUCATIONAL COMMUNITY