Using Evidence to Make Everyday
Decisions
ROBIN STEED, PHD, LOTR
LOTA CONFERENCE
2014
Objectives and Agenda
Part One
• Intro: What? Why?
• Learn the 5 easy steps:
1. Identify the problem
2. Ask a question
3. Find some answers
4. Evaluate the answers
5. Try the best one out
Part Two
 Practice EBP Skills
 Easy
Checklist
 Getting Started
Part One
Introduction to Evidence Based
Practice
WHAT DID YOU SAY?
NOW WHY WOULD I WANT TO DO THAT?
What is EBP?
knowledge
Research
Client
Factors
Clinical
Reasoning
Best Practice
(Sackett ,Straus, Richardson, Rosenberg, & Haynes, 2000).
wisdom
Why? Client education
Why? Client trust
Why? Do no harm!
 NSAIDs do not reduce fibromyalgia pain
 No evidence to support psychoanalytic (Freudian) treatment
 Stretching does not prevent or improve contracture
 Sheltered workshops do not increase employment
 Cognitive remediation does not improve function
Why? Your own peace of mind
Why? Our clients deserve the BEST!
The Process of
Evidence Based Practice
Yes, you too can be
an evidenced based
practitioner in just 5
easy steps!
Identify the clinical problem
1
I want my clients to:
 be more functional
 have a higher quality of life
 have better outcomes faster
 transfer skills to home environment
 have maintained gains at follow-up
 have better adherence to treatment plan
Types of problems and their questions
1
 Assessment/Diagnostic (What is the best assessment to
use to identify performance deficits?)
 Intervention efficacy (What is the best treatment?)
 Intervention cost-effectiveness (Which treatment gives
me the most return for my money?)
Identify your problem
1
Complete the problem identification worksheet.
Examples:
 My clients don’t like constraint induced therapy, would
a modified approach work just as well?
 My clients have poor sensory motor skills and have
trouble with handwriting.
 We send instructions home with the clients but we
aren’t sure they understand them.
A good problem
1
Ask a PICO question
2
Patient or Problem
Intervention
Comparison Intervention
Outcomes
PICO Examples
2
 In children with autism, evidence on vigorous vs. mild
exercise, on stereotyped behaviors?
 In clients with schizophrenia, evidence on environmental
supports on ability to transition to community?
 In children with hemiplegia, evidence on constraint
induced therapy on motor function?
Work on step two worksheet
Good Questions
2
 Within occupational therapy domain
 “What” questions, not ‘why’
 Just right focus, not too wide, not too narrow
 Measurable in some way
 Clinically important
 Not already answered!
Take a break and relax!
Looking for some answers
3
 P: serious or chronic mental illness, psychosis,
schizophrenia, mood disorder, anxiety disorder
 I: child care, meal preparation, home management,
shopping, time management, safety, social participation,
education exploration, retirement exploration,
employment seeking
 O: employment and education
(Arbesman & Logsdon 2011)
Types of evidence
3
 Meta-analysis
 Systematic review
 Critically appraised papers (CAPs)
 Critically appraised topics (CATs)
 Individual journal articles
wikipedia
Where do we find the (free) evidence?
3
AOTA Evidence Exchange
Centre for Evidence-based Mental Health
Cochrane Consumer Network
McMaster Occupational Therapy EBR Reviews
National Guideline Clearinghouse™ (NGC)
OT Seeker - The Occupational Therapy Evidence Database
OT Evidence at www.otevidence.info
Everyday Evidence Podcast from AOTA
Centre for Reviews and Dissemination
Where do we find the articles?
3
 Pub Med ncbi.nlm.nih.gov/pubmed
 Pub Med Central: pubmedcentral.gov
 Directory of Open Access Journals: www.doaj.org
 NBCOT if registered (ProQuest)
 Google Scholar
PubMed Central
3
Open Access:
• Arthritis
• Autism Research
• Journal of Pain
Research
NBCOT
3
ProQuest
RefWorks
 AJOT
 An easy way to keep
 OTJR
track of your articles
 OT International
 OT Canada, NZ
Google Scholar
3
Critically evaluate the evidence
4
 Read the article
 Fill out a form
 Critical
Review from McMaster’s University
 CAP from AOTA
 CASP in the UK
Critically evaluate the evidence
4
 Study Design
 randomization
 control for bias
 ethical
 control group
 Sample
 size
 described
 Assessment Properties
 Intervention
 detailed/replicable
 Statistical Analysis
 Threats to Validity
internal
 external

Make a decision
Given the quality of the
evidence, what are the
implications for clinical
practice?
Implement and evaluate
5
Implement
Evaluate
 What is the risk?
 Choose outcome
 What is the cost?
measures
 Keep records of
outcomes and adverse
events
 Do a ROI assessment
 Do you need special
training?
 Getting client consent
 Facility support
Take a longer break!
Part Two
Evidence Based Practice Practice
1. Easy Checklist
2. Getting Started
Checklist
 Are the results valid?
 Subject
selection
 Research design
 Data analysis & results
 Are the results meaningful?
 Are the results applicable to my client?
Source: Portney & Watkins (2009)
How were the subjects selected?
1
 Random sampling
 Random assignment
 Convenience
Sample Size?
2
not that great <30> much better
Sample Size?
3
Inclusion: who we let in
Exclusion: of the ones we let in,
who do we kick out
How were participants assigned?
4
Did it result in two equal groups at the start?
Was the design reasonable?
5
• Exercise 2x week
• Constraint Induced Therapy 20 hours/day
• A 4 hour workshop on cultural competence
Was the intervention the reason for results?
6
•
•
•
•
•
•
•
•
•
History
Measurement Instrument
Statistical Regression
Maturation
Unequal Groups
Attrition
Multiple Treatments
Treatment Diffusion
Participant Responses
History
Instrumentation
Statistical Regression
Maturation
Unequal Groups
Mortality/Attrition
Multiple Treatment
Treatment Diffusion
Participant’s Reactions
 Artificial environment in effort to control
extraneous variables (can’t generalize to real
world)
 Hawthorne effect- subjects aware of being
studied
 John Henry effect- subjects get competitive
between groups
 Novelty effect- increase motivation because tx is
new
Was everyone blind to treatment?
7
Did the researchers pick the right assessment?
8
• Toglia Categorization Assessment or the Allen
Cognitive Levels to measure change over time.
• Using the O’Connor as a measure of functional
ability
Observation versus self-report
Simulation versus natural setting
Standardized versus made up test
Were the measures valid and reliable?
9
Validity
Reliability
 Content
 Re-test
 Face
 Tester
 Criterion
 Internal Consistency
 Concurrent
 Pearson’s
 Alpha
r
 ICC
.7 and above
Cronbach
How many people dropped out of the study?
10
 Why?
 Percentage
http://www.englishblog.com/2013/10/cartoon-oreos-more-addictive-than-cocaine.html#.UuqugLTkroI
Were the appropriate statistics used?
11
Variables
Levels
Groups
n > 30
n < 30
1
2
1
Paired t test
Wilcoxon
2
Unpaired t (independent t)
Mann-Whitney U
1
One way repeated measures ANOVA
Friedman
2
One way ANOVA
Kruskal Wallis
1
2 way repeated measures ANOVA
2+
Two way ANOVA
3+
2+
Source: Portney & Watkins (2009)
Were the results significant?
12
Increased shoulder ROM 10° = function?
Power- the ability of a study design to find an effect if there
is one, depends on:
 Effect size- the amount of impact the independent
variable has on the dependent variable (Cohen’s d)
small= .20 medium= .50
large = .80
 Sample size- increases the power to find effect
Source: Portney & Watkins (2009)
Take another break
Are the results meaningful?
13-16
 Is the intervention feasible, cost effective?
 Is there any risk?
 Is this within the scope of OT?
 What is the ‘take home’ message?
www.haven.com
www.bioness.com/Healthcare_Professionals/H200_for_Hand_Paralysis.php
Are the results applicable?
17-19
 Given the quality of the study, should I apply this to my
practice?
 Do I need extra training?
 Do I have the resources to replicate intervention?
 Are subjects similar to my clients?
Get Real!
LIKE I AM REALLY
GOING TO HAVE TIME
FOR ALL THIS!
Why not? Therapists said . . .
 Lack of time (87%)
 Large workload/caseload (67%)
 Limited searching skills (50%)
 Limited critical appraisal skills (44%)
McCluskey (2003)
Expert Solutions
 Prioritizing daily tasks
 Making plans & goals
 Delegating tasks
 Setting aside blocks of time
 Dividing into manageable chunks
Influences on Success
 Readiness for change
 Personal and organizational expectations
 Presence of deadlines
 Availability of support
EBP in just 10 minutes a day!
 Join AOTA (Journal Club Toolkit)
 Listen to AOTA evidence podcast
 Read one or two journal articles a month
 Look up the quick summaries on EBP web sites
 Sign up for evidence alerts at McMaster’s
 Start an EBP file at work
 Join/start a journal club
 Keep records
EBP Books on Evidence
photos: amazon.com
EBP Books on Implementing EBP
Reasons to Create Evidence
 Referrals
 Client self-efficacy
 Funding for program
 Third party payment
 Resource allocation
 Keep your job !
References







Arbesman, M., & Logsdon, D. (2011). Occupational therapy interventions for employment and
education for adults with serious mental illness: A systematic review. American Journal of
Occupational Therapy, 65, 238-246.
Dunn, (2008). Bringing evidence into everyday practice. Thoroughfare, NJ: Slack
McCluskey, A. (2003), Occupational therapists report a low level of knowledge, skill and
involvement in evidence-based practice. Australian Occupational Therapy Journal, 50, 3–12.
doi: 10.1046/j.1440-1630.2003.00303.x
Page, S. J., Levine, P., & Hill, V. (2007). Mental practice as a gateway to modified constraint
induced movement therapy: A promising combination to improve function. American Journal of
Occupational Therapy, 61(3), 321–327. http://dx.doi.org/10.5014/ajot.61.3.321
Portney, L. & Watkins, M. (2009). Foundations of clinical research (3rd ed.).Upper Saddle River NJ:
Pearson.
Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (2000). Evidence
based medicine: How to practice and teach EBM (2nd ed.). London: Churchill Livingstone.
Gladstone, D. J., Danells, C. J., & Black, S. E. (2002). The Fugl-Meyer Assessment of Motor
Recovery after Stroke: A Critical Review of Its Measurement Properties. Neurorehabilitation and
Neural Repair, 16(3), 232-240. doi: 10.1177/154596802401105171
Resources at LSUHSC Shreveport
www.robinsteed.pbworks.com