Resources for
Evidence-Based School
Social Work Practice
Stephanie Baus
Tulane University
Workshop Outline

Overview of Evidence-Based Practice
–
–
–
–

EBP definition and model
EBP process (steps)
Range of practice questions
Resources for learning EBP skills
EBP in School Social Work
–
–
–
–
Relationship to RTI and PBS
Assessment
Interventions
Your practice questions
Workshop Outline (cont.)

Formulating practice questions
– Effectiveness/prevention
– Risk/Assessment
– Descriptive

Resources for locating evidence
–
–
–
–
–
Systematic reviews of studies
Electronic databases
Search strategies
Specialized sites
Scholarly books
Workshop Outline (cont.)

Evidence-informed Assessment in SSW
–
–
–
–

Assessment questions
Locating evidence (searching, resources)
Critical appraisal
Application
Evidence-Informed Intervention in SSW
–
–
–
–
–
Effectiveness/prevention questions
Locating evidence (searching, resources)
Critical appraisal
Application
Iatrogenic interventions
Workshop Outline (cont.)

Evaluating Outcomes
– Group designs
– Single subject designs
Overview of
Evidence-Based Practice
Definition of
Evidence-Based Practice
“Placing the client’s benefits first, evidencebased practitioners adopt a process of lifelong
learning that involves continually posing specific
questions of direct practical importance to
clients, searching objectively and efficiently for
the current best evidence relative to each
question, and taking appropriate action guided
by evidence” (Gibbs, 2003, p 6).
EBP Model
Clinical state
and
circumstances
Practitioner's
Expertise
Client
Preferences
and actions
Research
Evidence
Haynes, Devereaux, and Guyatt, 2002
Steps of EBP*

Step 1: Convert an information need into an
answerable practice question.

Step 2: Efficiently locate the best evidence to
answer the question.

Step 3: Critically appraise the evidence for its
validity and usefulness.
Steps of EBP (cont.)*

Step 4: Using practice expertise to integrate
evidence with student characteristics
and school context, apply the results of
the evidence appraisal to practice.

Step 5: Evaluate the outcome of evidence-based
action.

Step 6: Teach others: challenges and obstacles
*Based on Sackett, et al., 1997, and Gibbs, 2003
Range of Practice Questions
Effectiveness/Prevention
 Assessment/Risk
 Descriptive

Terminology
Evidence-Based Practice (EBP)
 Evidence-Informed Practice
 Empirically-Supported Interventions/Treatments

Resources for Learning EBP Skills




Gibbs, L. E. (2003). Evidence-based practice for the
helping professions. Pacific Grove, CA: Brooks/ColeThomson Learning.
Kelly, M. S., Raines, J. C, Stone, S. & Frey, A. (2010).
School social work: An evidence-informed framework for
practice. New York: Oxford University Press.
Raines, J. C. (2008). Evidence-based practice in school
mental health. New York: Oxford university Press.
Rubin, A. (2008). Practitioner’s guide to using research
for evidence-based practice. Hoboken, NJ: John Wiley &
Sons.
Evidence-Based Practice in
School Social Work
EBP in Response to Intervention
and Positive Behavior Support Models

Efficient and Reliable Assessment
Tools/Procedures
– Screening to identify prevalence of problems
– Diagnostic evaluation of individual students
– Monitoring of treatment fidelity and student progress

Empirically-Supported Interventions
– Primary Tier: core curriculum to all students/
preventive
– Secondary Tier: additional support to at-risk students
– Tertiary Tier: intensive support to individual students
Evidence-Informed School
Social Work Practice

Recent survey of over 1600 school social
workers indicates
– Few use online databases, journals, or scholarly
books to inform practice.
– Primary focus on interventions targeting individual
change and risk factors rather than primary
prevention.
(Kelly, et al., 2008).

Your practice questions?
Resources for Formulating
Practice Questions
COPES Questions
Client-Oriented
 Practical
 Evidence-Search

Gibbs, L. E. (2003). Evidence-based practice for the helping
professions: A practical guide with integrated multimedia. Pacific
Grove, CA: Brooks/Cole.
COPES Questions
Client-Oriented
Practical
Evidence-Search
Four Features of a Well-Built Question
Client Type and
Problem:
What You Might
Do:
Alternate Course
of Action:
What You Want
to Accomplish:
How would I describe
a group of clients of
similar type?
Apply a treatment;
act to prevent a
problem; survey or
interview clients;
measure to assess a
problem; screen
clients to assess risk.
What is the main
alternative other than
in the box to the left, if
any?
Outcome of treatment
or prevention;
accurate description
of client; valid
measure; accurate
risk estimation.
Gibbs (2003)
Research Question
COPES Question
Identifies specific sample Describes type of client
and problem
Identifies particular
instance of intervention,
including what, where,
when, how
May describe particular
type(s) of interventions
or ask about “best
intervention”
Identifies specific
measurement
instruments
Describes intended
outcomes, not measures
Formulating Effectiveness/
Prevention Questions:
a) Comparing an intervention to no intervention
b) Comparing two interventions
c) What is the best intervention?
Examples
Client/Problem Action
Alternate
Outcome
Rick Ager’s
question:
If African
American
adolescents
receive
Motivational
Interviewing
or no
intervention
are the former more
likely to decrease their
use of alcohol and other
drugs?
Loretta
Pyles’
question:
For clients of
social service
agencies
do hierarchical
or collectively
models of agency structured
management
management
models
Lynn Pearlmutter’s
question:
For couples who
have experienced
one partner’s
marital affair
what is the best therapeutic
approach
result in greater client
empowerment?
to rebuild trust?
Formulating Assessment/Risk Questions:
a) Comparing an assessment instrument/procedure to
no assessment
b) Comparing two assessment instruments/procedures
c) What is the best assessment instrument/procedure?
Examples
Client/Problem
Action
Alternate
Outcome
Loretta Pyles’
question:
Among battered
women
will the Danger
Assessment
Scale
as opposed to
no assessment
accurately predict
lethality?
Judy Lewis’
question
In an urban
community
devastated by a
natural disaster
does door-todoor contact
with residents
or outreach to
existing
institutions in
the area
a better means of
predicting the
potential viability
of a given
community?
Eileen Ihrig’s’
question:
For children in
cross-cultural
settings
what is the best assessment tool?
to assess
psychosocial wellbeing?
Formulating Descriptive (Quantitative)
Questions:
a) Summarizing characteristics (how much, how
many, what percentage, what is the average?)
b) Asking about relationships between variables
Examples
Client/Problem Action
Cyleste Collins’
question
[If] social work
students
[are surveyed]
Rick Ager’s
question
[If] people who
have been
affected by a
natural disaster
[are surveyed]
Alternate
__________
__________
Outcome
what percentage
endorse
stereotypical
attitudes about
victims of domestic
violence?
do they report
increased alcohol
and other drug use
compared to predisaster levels?
Formulating Descriptive (Qualitative)
Questions:
a) Require narrative responses
b) Suggest in-depth explorations, little known
phenomena (how would they describe, what is
the process, how do they experience?)
Examples
Client/Problem
Action
Loretta Pyles’
question:
[If] women who
have been
battered
[are interviewed
in depth]
Eileen Ihrig’s
question:
If children in
conflict situations
who were
abducted and
became soldiers
[are interviewed
in depth]
Alternate
__________
__________
Outcome
how do they
describe what
safety means to
them?
how do they
describe their
experiences of
reintegrating into
the community?
Practice Exercise
Describe a practice situation:
Ask a practice question:
Identify question type:
Formulate a COPES question:
Client/problem
Action
Alternative
Outcome
Resources for Locating
Evidence
Systematic Reviews and
Meta-analyses

Campbell Collaboration (Social Work, Education
& Criminal Justice
http://www.campbellcollaboration.org/

National Registry of Evidence-Based Programs
and Practices, SAMHSA
http://www.nrepp.samhsa.gov/index.htm

What Works Clearinghouse
http://www.ies.ed.gov/ncee/wwc/
On-line Databases

Subscription databases: Cochrane collaboration,
SWAB, SSAB, PsycInfo, Criminal Justice Ab.,
Sociological Ab., Medline, CINHAIL, etc.

Free access databases: Campbell Collaboration,
ERIC, Pubmed, Google Scholar, PILOTS

Government and professional sites: National
Center for PTSD at http://www.ptsd.va.gov/,
Information for Practice at http://ifp.nyu.edu/
Search Planning Worksheet
 COPES
question
 Synonyms
 Thesaurus
 Search terms
 Boolean logic
– AND, OR, NOT
– “wildcards” (truncation)
Client/Problem Action
Alternate
Outcome
MOLES
COPES
Question
Synonyms
Controlled
Language/
Thesaurus
Search
Terms
(adapted
from Gibbs,
2003, p.
140)
Client/Problem Action
COPES
Question
If adjudicated
delinquent
juvenile males
Alternate
complete an as opposed
Outward
to no
Bound type program
of program
Outcome
MOLES
will the
former have
lower rates
of
recidivism?
Synonyms
Controlled
Language/
Thesaurus
Search
Terms
(adapted
from Gibbs,
2003, p.
140)
Client/Problem Action
COPES
Question
If adjudicated
delinquent
juvenile males
Synonyms mandated
court ordered
youth
adolescent
Alternate
Outcome
complete an as opposed
Outward
to no
Bound type program
of program
will the
former have
lower rates
of
recidivism?
wilderness
program
adventure/
challenge
program
re-arrest
reoffend
MOLES
Controlled
Language/
Thesaurus
Search
Terms
(adapted
from Gibbs,
2003, p.
140)
Client/Problem Action
COPES
Question
If adjudicated
delinquent
juvenile males
complete an
Outward
Bound type
of program
Synonyms mandated
wilderness
program
adventure/
challenge
program
Controlled delinquency
Language/
Thesaurus
wilderness
therapy
outdoor
education
court ordered
youth
adolescent
Search
Terms
Alternate
as opposed
to no
program
Outcome
MOLES
will the
former have
lower rates
of
recidivism?
re-arrest
reoffend
(adapted
from Gibbs,
2003, p.
140)
Client/Problem Action
COPES
Question
If adjudicated
delinquent
juvenile males
complete an
Outward
Bound type
of program
Synonyms mandated
wilderness
program
adventure/
challenge
program
Controlled delinquency
Language/
Thesaurus
wilderness
therapy
outdoor
education
Search
Terms
“Outward
Bound” or
wilderness or
“adventure
program*”
or “challenge
program*”
court ordered
youth
adolescent
(delinquen* or
adjudicated or
mandated)
(juvenile* or
youth* or
adolescen*)
Alternate
as opposed
to no
program
Outcome
MOLES
will the
former have
lower rates
of
recidivism?
re-arrest
reoffend
recidivism
or reoffend
or re-offend
or rearrest
or re-arrest
(adapted
from Gibbs,
2003, p.
140)
MOLES
Methodology
Orienting
Locators for
Evidence
Searches
(Gibbs, 2003)
Search Summary
#1: meta-anal* or metaanal* or meta
anal* or systematic review* or synthesis
of studies or study synthesis (6,807)
#2: delinquency prevention
#3: #1 AND #2
(250)
(5)
meta MOLES AND delinquency prevention
250
5
6807
Effectiveness
Questions
Prevention
Questions
Risk/Prognosis
Questions
Assessment
Questions
Description
Questions
(With
Qualitative
Studies a
Subset)
Syntheses of
Studies
Random*
OR
Controlled
Clinical
trial*
OR
Control group*
OR
Evaluation stud*
OR
Study design
OR
Statistical*
Significan*
OR
Double-blind
OR
Placebo
(Random*
OR
Controlled
Clinical
trial*
OR
Control Group*
OR
Evaluation Stud*
OR
Study Design
OR
Statistical*
Significan*
OR
Double Blind
OR
Placebo)
AND
Prevent*
(Risk Assessment
OR
Predictive Validity
OR
Predictive Value
OR
Receiver Operat*
OR
ROC
OR
Sensitivity
OR
Specificity
OR
False positive*
OR
False negative*
OR
Prognos*)
AND
Predict*
(inter-rater
OR
Inter-observer
OR
True positive*
OR
Specificity
OR
False Positive*
OR
False negative*
OR
Sensitivtiy
OR
predict*
OR
Receiver operat*
OR
ROC)
AND
(assess*
OR
diagnos*)
(Random* Select*
OR
Survey
OR
Representative
Sample)
meta-anal*
OR
meta anal*
OR
metaanal*
OR
Systematic
Review*
OR
synthesis of
studies
OR
Study Synthesis
Gibbs, 2003, p.100
Qualitative
Studies
Qualitative
Study OR
Qualitative
Analysis OR
Content Analysis
OR In Depth
Interview* OR InDepth
Interview* OR
Participant
Observation OR
Focus Group*
Client/Problem Action
COPES
Question
If adjudicated
delinquent
juvenile males
complete an
Outward
Bound type
of program
Alternate
Outcome
MOLES
as
opposed
to no
program
will the
former have
lower rates
of
recidivism?
Random* or
controlled
clinical trial*
or control
group* or
evaluation
stud* or
study
design or
statistical*
significan*
or double
blind or
placebo
Synonyms mandated
wilderness
program
adventure/
challenge
program
Controlled delinquency
Language/
Thesaurus
wilderness
therapy
outdoor
education
Search
Terms
“Outward
Bound” or
wilderness or
“adventure
or
program*”
or “challenge “outdoor
education”
program*”
court ordered
youth
adolescent
(delinquen* or
adjudicated or
mandated)
(juvenile* or
youth* or
adolescen*)
re-arrest
reoffend
recidivism
or reoffend
or re-offend
or rearrest
or re-arrest
(adapted
from Gibbs,
2003, p.
140)
Client/Problem Action
COPES
Question
If adjudicated
delinquent
juvenile males
complete an
Outward
Bound type
of program
Alternate
Outcome
MOLES
as
opposed
to no
program
will the
former have
lower rates
of
recidivism?
meta-anal*
or
metaanal*
or meta
anal or
systematic
review* or
synthesis of
studies or
study
synthesis
Synonyms mandated
wilderness
program
adventure/
challenge
program
Controlled delinquency
Language/
Thesaurus
wilderness
therapy
outdoor
education
Search
Terms
“Outward
Bound” or
wilderness or
“adventure
or
program*”
or “challenge “outdoor
education”
program*”
court ordered
youth
adolescent
(delinquen* or
adjudicated or
mandated)
(juvenile* or
youth* or
adolescen*)
re-arrest
reoffend
recidivism
or reoffend
or re-offend
or rearrest
or re-arrest
(adapted
from Gibbs,
2003, p.
140)
Descriptive (quantitative) MOLES
random* select*”
OR
“random* sampl*”
OR
survey
OR
questionnaire*
OR
“representative sample”
OR
“national sample”
OR
“interview schedule”
OR
“correlat*”
(Baus)
Revising the Search

Keep a search history (databases, terms, hits,
relevance)

Too few hits: Broaden search, include all
synonyms and combinations, use controlled
language, check spelling, check Boolean
structure, change databases. (Removing MOLES
is last resort.)

Too many hits: Narrow search, more specific
terms, add categories, check Boolean structure.

Hits not relevant: Consider multiple meanings
and edit terms, use controlled language, use
“not.”
Evidence-Informed Assessment
in School Social Work
Evidence-Informed Assessment

Assessment questions

Resources
– Kelley. M. L., Noell, G. & Reitman, David. (2003). Practitioner’s
guide to empirically based measures of school behavior. New
York: Kluwer Academic Publishers.
– Fischer, J & Corcoran, K. (2007). Measures for clinical practice
and research: A sourcebook (4th Ed.). New York: Oxford
University Press.
– www.ets.org/test_link/find_tests/

Searching
Step 3: Critically Appraising
the Evidence
Criteria for Evaluating
Effectiveness Studies

Relevance (importance to serving clients, common to
practice, feasible, may lead to change)

Evidence quality (control group, randomization, attrition,
subjects and raters “blind,” equivalent groups)

Statistical significance (p < .05 - unlikely due to chance)

Impact of intervention (effect size, absolute risk
reduction, number needed to treat, number needed to
harm)
(Armstrong, cited in Gibbs, 2003)
Meta-Analyses and
Systematic Reviews

Differences between meta-analyses,
systematic reviews and narrative reviews

Locating meta-analyses and systematic
reviews

Critically appraising meta-analyses and
systematic reviews
Criteria for Evaluating
Descriptive Studies

Relevance (provides important information about client
needs, characteristics, and perceptions)

Evidence quality (random selection or representative
sample, sample size, clearly written pretested questions,
response rate, appropriate generalization)

Statistical significance for relationships (p < .05 unlikely due to chance)

Strength of relationships/effect size
Criteria for Evaluating
Assessment/Risk Studies

Relevance (importance to clients, common to practice,
feasible, may lead to change)

Ease of use (administration, scoring)

Evidence quality
– Reliability: interrater agreement and/or Cronbach’s
alpha > .70
– Validity: comparison with gold standard > .70,
positive predictive value, negative predictive value
(Armstrong, cited in Gibbs, 2003)
Criteria for Evaluating
Qualitative Studies

Relevance (provides important understanding of client
experiences, perspectives, processes)

Evidence quality
– appropriate, clearly described methods
– theoretical consistency
– procedures to enhance trustworthiness* (credibility,
dependability, confirmability, transferability)
– systematic analysis
– no unfounded generalization
*Lincoln & Guba, 1985
Use of Checklists/Rating Forms

Rate study quality

Applying explicit criteria increases
reliability

Use summary score or not?
Step 4: Applying the
Evidence to Practice
Application to Practice

Applicability to client and practice context

No evidence is an answer – role of theory,
informed consent

Practice wisdom – integrating art and
science, experience, relationship, intuition,
and evidence
Application to Specific Clients
and Agency Contexts
(Slide from Jim Barber, based on Sackett, et al., 2000)
Application Questions
Decision Aids & Prompts
1. Is this case so different from those in the
literature that you cannot apply their
results?
Differences can be of:
Degree (quantitative) and/or
Type (qualitative)
2. Is the intervention feasible in our setting?
Potential obstacles include:
Agency policy, ethics or mandate
Cost (including time)
Agency skill
3. What are the potential benefits and harms
to our client?
Taking account of cost, ‘outcome valence’ and NNT/H:
Weigh the seriousness if you do
Weigh the seriousness if you don’t
4. What are our client’s values & expectations for
both the outcome we are trying to achieve
and the intervention we are offering?
Assist the client to weigh the intervention’s:
Acceptability
Desirability
5. Taking account of the above, does the
intervention need to be modified?
Modifications can be of:
Degree (length or intensity)
Type
Step 5: Teaching Others
Challenges and Obstacles

Misconceptions about EBP

Agency/colleague support

Availability of databases/articles

Potential consequences of knowledge
References
Gibbs, L. E. (2003). Evidence-based practice for the helping professions: A
practical guide with integrated multimedia. Pacific Grove, CA: Thompson
Brooks/Cole.
Haynes, R. B., Devereaux, P. J. & Guyatt, G. H. (2002). Clinical expertise in the
era of evidence-based medicine and patient choice. Vox Sanguinis, 83, Suppl 1,
383-386.
Haynes, R. B., Sackett, D. L., Gray, J. M., Cook, D. J., & Guyatt, G. H. (1996).
Transferring evidence from research into practice: 1. The role of clinical care
research evidence in clinical decisions. ACP Journal Club, 125 (3), A14-A16.
Lincoln, I. & Guba, G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.
Sackett, D. L., Richardson, W. S., Rosenberg, W. & Haynes, R. B. (1997).
Evidence-based medicine: How to practice and teach EBM. New York:
Churchill Livingstone.
Sackett, D. L., Rosenberg, W., Gray, J. M., Haynes, R. B., & Richardson, W. S.,
(1996), British Medical Journal, 312, 71-72.
Shlonsky, A., & Gibbs, L. (2004). Will the real evidence-based practice please stand
up? Teaching the process of evidence-based practice to the helping professions.
Brief Treatment and Crisis Intervention, 4 (2), 137-153.
Stephanie Baus
sbaus@tulane.edu