Implementing Evidence-Based Practice:
An Overview of the BEST Training with
Practitioner Responses
Presented by:
Sarah E. Bledsoe, Ph.D., M.Phil., M.S.W.
School of Social Work
University of North Carolina at Chapel Hill
Jennifer L. Bellamy, Ph.D., L.M.S.W.
George Warren Brown School of Social Work
Washington University
Implementing Evidence-Based Practice:
An Overview of the BEST Training
with Practitioner Responses
Sarah E. Bledsoe1
Jennifer L. Bellamy2
Lin Fang3
Jennifer I. Manuel3
Edward J. Mullen3
for the BEST Team
1School
of Social Work, University of North Carolina at Chapel Hill
2George Warrant Brown School of Social Work, Washington University
3Columbia University School of Social Work
Supported in part by the Willma & Albert Musher Program at Columbia University
Introduction

Underutilization of scientific knowledge

Evidence-practice gap
2 Challenges:

Lack of knowledge and training in EBP & ESI

Discrepancies in the definition of EBP & ESI
Consequences:

Practice outcomes

Translational research and knowledge generation
BEST Project


Bringing Evidence for Social Work Training

Multi-phase exploratory study

Training-based collaborative agency-university partnership

3 social agencies
Response to:

Limited EBP/ESI implementation research

Push for EBP/ESI in practice and education
Evidence-based practice is the integration of best
research evidence with clinical expertise & patient values
From: Shlonsky, A., & Gibbs, L. (2004). Will the real evidence-based practice please stand up? Teaching the process of evidence-based
practice to the helping professions. Brief Treatment and Crisis Intervention, 4, 137-153.
Empirically Supported Interventions

Interventions for which there is consistent
scientific evidence showing that they
improve client outcome

Interpersonal psychotherapy for depression

Therapuetic foster care

Assertive community treatment teams
(Drake, Goldman, Leff, et al., 2001).
EBP in the BEST Training

Conceptualized as a professional model
of practice:
1.
2.
3.
4.
5.
6.
7.
Motivation
Question development
Search for relevant research
Research appraisal
Application
Evaluation
Dissemination
(Gibbs, 2003)
BEST
Training
BEST Training

10 training modules

Designed to:

Increase knowledge of EBP

Improve EBP research skills

Foster positive attitudes toward EBP

Identify factors that promote the use of EBP in
social service organizations
Overview of the BEST Training with
Practitioner Responses

Today we report:

10 training modules

Agency practitioner perceptions of the training

Trainer experiences of implementation
Methods

Convenience sample of 3 agencies (n=16)

Process notes and post-training focus groups





Tape recorded and transcribed verbatim
Interviewer notes of emerging themes
NVivo
Pre-identified and emergent themes
Data analytic strategies:



Triangulation of data sources
Multiple coders
Member checking
Sample
Age
34.9 years (mean)
Female
87.5%
White/Caucasian
37.5%
Asian
37.5%
35K-50K
56.3%
Masters Degree
50.0%
Social Work
81.3%
Social Work License
50.0%
Time at current agency
6.3 years (mean)
Intervention

Cooperative agency-university partnership

Earliest stages included partner agencies

Pretraining meetings and focus groups

Guided by agency teams & administrators

Practice-relevant topic selected by agency team

Hands-on training using problem-based learning strategy

Research team provided technical assistance & guided the
group process
Modules of the BEST Training
EBP Introduction and Overview
2. Question Selection
3. Overview of Research Evidence
4. Search Tools
5. Search Demonstration
6. Troubleshooting the Search
7. Evaluating the Evidence
8. General Findings and Observations
9. Synthesizing Evidence Found
10. Action Plan
1.
Module One:
EBP Introduction and Overview
I. Definition of EBP
a. Brief history of EBP
b. Major motivations to use EBP
c. Brief overview of different models
d. Highlight Gibbs (2003) model: Steps as a general
guide
e. What we mean when we say “evidence”
-research evidence
-practitioner experience
-client preference
-agency mission and values
-ethics
f. Disconfirming and confirming evidence
g. Feedback from the group/questions/any tweaking
necessary?
Practitioners Reactions Module One
“I think we have a fuller view of EBP…[the
training] definitely deepened our
understanding about the process and let us
have a chance to try it out.”
“It feels much more tailored to where we’re
coming from and what our interests and
values are and not like there’s going to be
a protocol that is going to be put onto
our practice that we would have to somehow
manage to squeeze ourselves into and that
wouldn’t be desirable.”
Module Two:
Question Selection
I. Question selection
a. List project focus area for the question
b. Discuss elements of “researchable” questions
-Relevance to practice
-Not something you already have the answer to
-Specific
c. Practice making up a few questions
d. Discussion, refinement & selection of question
II. Background Information for the Question
a. Why this question in particular?
b. Population
-Age, race/ethnicity
-Resources
-Key mental health/substance abuse issues
c. Agency
-Referral or recruitment process
-Resources available
-Current available services
Practitioners Reactions Module Two
“When you start to research [the practice problem], it
can become so overwhelming and this is a way to
narrow it so that you get what you want, so that it’s
more practical. Its a little bit more time
consuming and taxing, but on the other hand, its
a time saver, more effective.”
“Learning under the discipline of coming up with a
question, we struggled with that for weeks…If we
ask about it one way, we’re asking about
effectiveness. We ask it another way, we could be
asking a comparison question. So all the ways there
are to ask a question and the importance of how you
ask [is related] to what you get.”
Module Three:
Overview of Research Evidence
I. Defining different sources of research
evidence
a. Review of different types of “evidence”
from module two
b. Review of different types of research
evidence:
Outcome studies, Meta Analyses,
Program Evaluations, etc.
Practitioners Reactions Module Three
“There are certain more theoretical readings
that aren’t particularly related to empirical
demonstrations of different techniques…and I
know they have a place in this…I didn’t
emerge with a lot of clarity about where they
fit in to the process, but sometimes I think it
would be useful for us to dedicate some
time to thinking about [theory].”
Module Four:
Search Tools
I. Search Engines that are Electronically Available
a. Types of search engines & overlap between them
b. Starting from meta-analyses & systematic
reviews to more specifics
II. Using Search Terms
a. Example of search terms
b. Group brainstorm about search terms for
question
c. Group search “plan”
III. Using MOLES
(Methodology Orienting Locators for Evidence Search)
Practitioners Reactions Module Four
In reference to handouts listing available search
engines identified as free or fee based:
“That is awesome, that’s great. If we got nothing at
all but that, this would have been a bonus.”
“We found that the tools you gave us, to streamline
the search…they were great. My problem is, really,
not having the time. It’s a really big problem. And
also, some of the engines you gave us, we’re not
going to be able to access because we would have to
pay and it would be nice to have access to
everything.”
“I need more time to use the tools because I don’t
know how to. I need to actually be practicing. I
don’t have a real grasp.”
Module Five:
Search Demonstration
I. Electronic Demonstration of Search Using a Database
a. Getting to a database
b. Starting a search
c. Keeping track of searched terms
d. Using abstracts
II. Introduction to Other Potentially Helpful Websites
III. Review of Search Plan
a. How does the agency see their ability to search?
b. Will there be a single person who is a “searcher”?
Several people? Will everyone search?
Is there no good candidate?
c. Search worksheet
Practitioners Reactions Module Five
“The organization of [the search] helped…using a table to
organize the [search]…I think [providing tools] like these would
make people more willing to do this.”
“It was really good to have a concrete example so we could do
an actual practice run through the process. Just to do it, try
out our own key words in session on our computers. I thought
that was extremely helpful.”
“I think the major issue probably is access to the actual
literature…that’s the only real difficult part. So, other than that,
the process is fine.”
“I don’t think people are going to take the time to search for a
list of sites and to search for different key words.”
Module Six:
Trouble Shooting the Search
I.
Review of Search Experience
a. Amount of material found
b. Where material was found
c. What was easy
d. What was difficult
e. Address general questions and issues
Practitioners Reactions Module Six
“To base practice on the evidence that we
were [able to access] for this question, I
mean we were struggling. We were
struggling.”
“When I come to articles on certain
populations…I’m not confident enough to use
[them on my own] because there aren’t
people available for consultation.”
Module Seven:
Evaluating the Evidence
I.
Examining Evidence Found
a. Review of different types of evidence
b. Example of quality review
c. Examples of quality rating scales
d. Identify the types of research evidence found by team
members
e. Value of meta-analyses & systematic reviews
II. Evidence Quality
a. Review of the different philosophies/ratings of evidence
b. Match of evidence/research type with question- the same
type of research is not appropriate for all questions
c. General indicators of quality
d. Ratings worksheet
e. In vivo practice using worksheet via discussion of evidence
found
III. Refinement of Search Plan Based on Findings To-Date
a. Any search engines not covered?
b. Any types of evidence missing?
c. Continue collecting and assigning evidence
Practitioners Reactions Module Seven
“I think the tools were wonderful, but extremely rigorous
and I think we would probably do it a little more
impressionistically…I think if practitioners could read
[evidence] together, almost like a book review, talk about
it, that would feel almost in sync with how we approach
issues and challenges.”
“I think it’s really important if you actually were able
to wrap your head around the math…I still don’t know
which study’s data is really that clean…or if the methods
are really that good…having a [statistic] helps you decide
that but I can’t see myself going through 15 articles and
calculating [a statistic].”
“I can find research articles…but I don’t have confidence
interpreting the findings.”
“I start to zone out.”
Practitioners Reactions Module Seven
“One question is, how much research is out there that is
relevant?”
“Sometimes I have difficulty [evaluating evidence] because a lot
of research is done with…other populations and…I have to
think, can I really generalize these techniques and would
it be applicable to [the] specific population I’m working
with? A lot of times I find [the empirical evidence] seems really
good but I wonder if it’s helpful or good [for my clients]. This
evaluation would be challenging to me.”
“In terms of fit, so it almost would require another piece of
training or knowledge around how you judge evidence that’s
been produced out of a completely different population. In this
case, we had difficulty finding research dedicated to the
population that looks like [our clients].”
Module Eight:
General Findings and Observations
I. Review of General Findings
a. Overall, does the research answer our question of interest?
b. Is there a great deal of research related to our question of
interest?
c. Which types of research are missing?
II. Review of Research Found
a. Start with research syntheses
b. Move to individual research articles and other pieces
III. Research synthesis and summary handouts
a. General overview of the research question or hypothesis
b. How recent is the evidence?
c. Main findings
d. What is the general quality of the evidence?
e. How well does it fit?
Practitioners Reactions Module Eight
“I really think the good research can
incorporate the practice perspective.”
“I just want to say, I did get a sinking
feeling when we were…actually evaluating
the articles we all found…like, is this the
[best available evidence]?”
Module Nine:
Synthesizing the Evidence Found
I.
Research Synthesis
a. Title (Year)
b. Source Type
c. Major Findings
d. Quality and Fit Notes
II. Summary Statement
a. Body of evidence currently available
b. Summary of findings
c. Recommended plan of action (if any)
III. Refinement of Group Search Plan Based on Findings To-Date
a. Any search engines not covered?
b. Any types of evidence missing?
c. Continue collecting and assigning evidence
Practitioners Reactions Module Nine
“What we’ve learned about [EBP]..it’s helping us
understand how to find the areas of research that are
missing and figure out places where other
contributions need to be made.”
“There’s a gap between what has been done on
the research level and what’s being done
everyday on the frontline. It would be great to
break that and its hard to do. I know we came up
with some ideas about trying to implement some of
this research knowledge in more day to day functions
here. I think it would be interesting and helpful to do
that.”
Module Ten:
Action Plan
I. Possible Next Steps
a. Summary Statement can form the basis for:
-Letter to administration
-Training to share findings
-Seek training for an identified intervention
-Use or development of new instruments or tools
-Resource manual
-Procedural changes
-Request for additional resources or funds
-Research at the agency
b. Team Plan
-What will we do with our findings?
-Who will carry out which steps?
-When will these steps be carried out?
c. EBP Next Steps
-How will you plan to continue using EBP?
-Who will carry out these steps?
-When will these steps be carried out?
Practitioners Reactions Module Ten
“We need to incorporate it into our daily practice.
Our regular team meetings will focus on [EBP]…and
incorporate [EBP] into agency structure and promote
that type of agency structure.”
“Training is important so people can read the
research and…apply it to the situation. I believe we
have the responsibility, the agency has the
responsibility. I think it is a really challenging
process.”
“As problems come up, as puzzles arise. I mean, I
have a plan to use this. I’m just waiting for the
time.”
Practitioners Reactions Module Ten
“You’re seeing how hard it is to take the little
attention that we have, compared to everything
else we have to do here, to think this way.”
“We don’t generally have someone like [the
research team] to call up and say, ‘We really want
to get an article on x or y, could you find 6 things or
help us find 6 things?’, ‘What do you think about the
key [search terms]?’, or ‘Where would you start?”
“The only drawback is time…But it could never be a
drawback to using it. It’s valuable.”
“We’re willing to do what we can.”
Limitations
Exploratory study
 Sample size
 Purposive sample
 New York City agencies

Recommendations
Participants highlighted need for:
 Additional training/supervision in EBP & ESI
 Continued consultation & technical support
 More evidence addressing real world practice
problems & populations
 Increased technical & statistical skills to evaluate
the evidence
 Addressing real barriers




Limited time & competing demands
Limited funding to support EBP
Limited access to sources of empirical evidence
Agency/organizational culture
Recommendations for Social Work
& Social Work Education

Training Programs (classroom and field)





Research



Process of EBP
Specific ESI
Research Skills
Statistical Skills
Increase partnerships with practitioners
Increase practice relevant research
Continuing Education


Library access for Alumni & Field Agencies
Quality continuing education (EBP & ESI)
Evidence Based Practice & Policy
Online Resource Training Center
http://www.columbia.edu/cu/musher/
Website/Website/index.htm