Evidence-Based Practice in Augmentative &
Alternative Communication: How do you do it?
What does it mean for Individuals who use AAC?
Part II- Steps 4, 5, 6 & 7
Facilitators & Barriers to EBP
Pammi Raghavendra. Ph.D.
Senior Lecturer, Disability & Community Inclusion, School of Health Sciences
Flinders University, Australia
Parimala.raghavendra@flinders.edu.au
ISAAC-Israel Annual National AAC Conference, Tel Aviv
8 June 2014
All day workshop
Part II
1.
2.
3.
What is EBP?
What does EBP mean for individuals who use AAC and other
stakeholders?
Steps involved in EBP – 7 steps
1.
2.
3.
4.
5.
6.
7.
4.
5.
6.
7.
Asking a clinically relevant & answerable question (Group work)
Searching for the evidence
Critically appraising the evidence (Group work)
Collating & synthesising the evidence – Developing a CAT
Implementing the evidence into practice
Evaluating the use of the evidence
Disseminating the EBP process & findings
Facilitators and barriers to EBP (Group work)
Practical Suggestions to implement EBP
Nature and extent of evidence in AAC
What can you do to add evidence to the AAC field?
Step 4. Collating &
Synthesising the evidence
What types of
interventions
enhance the
participation of
AAC users in
school settings?”
“What factors
influence the
participation of
AAC users in
school settings
Overview of Articles Appraised
10 Experimental - Single
Subject (SSED)
8 Experimental - Group
2
1 Meta-Analysis of SSED
3
10
5 Surveys
5
7 Reviews
5 Case Studies
7
8
5
1
2 Observational Studies
3 Qualitative Studies
Aims Stated
Topic Defined
Broad Literature Review
Variables Stated Clearly
Study Design Described
Pilot Study
Ethical Practices
Source of Population
Random Sampling
Random Assignment
Arthur, M, Butterfield, N. &
McKinnon, D.
1998
1
1
1
1
1
0
1
1
NA
NA
Beck, A. & Dennis, M.
1996
1
1
1
1
0
0
1
1
0
1
Beck, A., Fritz, H., Keller, A.
& Dennis, M.
Beck, A., Kingsbury, K., Neff,
A. & Dennis, M.
2000
1
1
1
1
1
0
0
1
0
0
2000
1
1
1
1
1
0
1
1
1
1
Author
Year
Step 4 - Summarising the Evidence
Step 4 – Summarising the Evidence
Author(s)
Carter, M. &
Maxwell, K.
Year
Title
1998 Promoting interaction with
children using augmentative
communication through a
peer-directed intervention
Hughes, C.,
Rung, L.,
Wehmeyer, M.,
Agran, M.,
Copeland, S. &
Hwang, B.
Design
Participants
To examine the effects of a Experimental - 4 AAC users with
partner-directed
Group
CP (3 boys, 1 girl);
intervention program on
30 typically
the social interaction
developing
between regular peers &
children
young AAC users within a
classroom setting
Intervention(s)
Results
Peer training resulted in
increase in social
interactions by peers &
by AAC users
Peers attended 6
lessons covering
mutual attention to
play activity,
commenting on
activities,
acknowledgement
of partner's
behaviours
1993 Using the milieu approach to To determine whether
SSED 15 year old boy
Expectant look,
increase spontaneous
milieu techniques would be multiple
with autism &
question, mand &
communication book use
effective in teaching
probe across mental retardation model;
across environments by an
communication book use
settings
reinforcement of
adolescent with autism
communication
book use &
signing
Direct observation
coding of 11
communicative
behaviours (verbal &
nonverbal)
Peer intervention
resulted in improved
rates of social
interaction during play
for 4/5 autistic children
2000 Self-prompted
communication book use to
increase social interaction
among high school students
Observation of
initiations by
participants when
interacting with
communication
partners (5
minutes/day)
To determine whether
SSED peers without disabilities
ABCB
can be taught to use
reversal
strategies to facilitate
communicative interactions
with autistic preschoolers
To examine the effects of
self-prompted
communication book
training provided by peers
on conversational
initiations
Instruction of
peers in how to
interact with AAC
users (eg.
strategies such as:
look, ask, wait,
respond)
Evaluation
tools/processes
Recording of
intervals with
interaction by
subjects & intervals
with interaction by
peers, use of
intervention
strategies
Goldstein, H.,
1992 Peer-mediated intervention:
Kaczmarek, L.,
Attending to, commenting
Pennington, R. &
on, & acknowledging the
Shafer, K.
behavior of preschoolers
with autism
Hamilton, B. &
Snell, M.
Aim
5 boys with
language, social &
cognitive deficits
+ 10 typically
developing
children (5 boys &
5 girls)
SSED 5 high school
multiple
students (4 male, 1
baseline
female)
across
participants
with multiple
probe
component
Self-prompted
communication
book training
provided by peers
Recording of
Mileu training was
spontaneous
effective
responses, expectant
look prompts, mand
prompts & incorrect
responses
Training led to increase
in appropriate initiations
& decrease in
inappropriate initiations
Develop a CAT- Critically
Appraised Topic
What is a CAT or Critically Appraised
Topic?
A CAT is a short summary of evidence on
a topic of interest, usually focussed
around a clinical question.
www.otcats.com, McCluskey, A.,
Characteristics of a CAT
• A CAT is a shorter and less rigorous
version of a systematic review
• Summarises the best available research
evidence on a topic
• Usually more than one study is included
in a CAT
www.otcats.com, McCluskey, A.,
CAP (Critically Appraised
Paper)
When a single study/paper is critically
appraised and summarised, it is called a
CAP.
• Can become part of a CAT or can stand
alone
www.otcats.com, McCluskey, A.,
How do you develop a CAT?
Follow the same EBP Process
1. Asking a clinically relevant & answerable
question-PICO/PESICO
2. Searching for the evidence
How do you develop a CAT?
Strategies for Searching
Search for current best evidence
1) look for pre-filtered evidence, e.g., Cochrane
2) look for reviews before individual studies
3) look for peer-reviewed before non-peer reviewed
Include and appraise only those articles that are
relevant to your question
(Schlosser, Wendt, Angermeier & Shetty, 2005)
How do you develop a
CAT?
3. Critically appraise the evidence,
• Use appraisal checklists where applicable
– Internal validity
– Social validity
– External validity
• Apply the level of hierarchy of evidence
(Joanna Briggs & Schlosser & Raghavendra (2004)
How do you develop a CAT?
• Examine the evidence for certainty/Strength of evidence
( Millar, Light & Schlosser, 2006, based on NL Smith & Simeonsson & Bailey, 1995)
– Conclusive (design provided tight experimental control,
adequate or better dependent variable reliability, treatment
integrity)
– Preponderant (study had minor flaws with respect to design
with adequate or better dependent variable reliability, treatment
integrity)
– Suggestive (several minor flaws, inadequate DV &TI leading to
conclusions that the outcome was plausible but not certain)
– Inconclusive evidence (significant flaws in design, fails to
establish experimental control)
How do you develop a CAT?
• Tabulate the articles
• WRITE the CAT
– 1 page summary with clinical bottom line
– Keep it in an accessible place at work so that
you and your colleagues can access it easily
– Put it on a website, CAT bank or submit to a
journal
Examples of a CAT
Examples resources
• www.otcats.com
• http://www.nswspeechpathologye
bp.com.au/critically-appraisedpapers-and-topics
Advantages of a CAT (Wendt, 2006)
• Based on everyday clinical scenarios, so highly
motivating for clinicians and students
• By focusing on the research evidence, one gains
experience in several steps in EBP process
• Writing CATs can be part of university educational
programs
• For established practitioners & researchers,
provides a framework for writing up discussions and
outcomes form journal clubs & other forums
Limitations of CATs (Wendt, 2006)
• CAT can be inaccurate or wrong
– No peer review
– Done in short time, may have missed key evidence, poor appraisal
• Short Shelf life
– New evidence replaces older evidence
– Constant updating is needed
Challenges in developing a CAT
– Lack of time and skills
Step 4. Synthesising the Evidence:
Interpreting study findings
What is the evidence telling me?
Validity (truth) and usefulness (clinical relevance)
How does it match with my client and their
environment?
Who are the stakeholders?
What are their beliefs and expectations?
Step 5: Implementing the evidence into
practice
• Integrate the research evidence with
clinical expertise and patient/client
preferences to make clinical decisions
• Development of clinical/practice
guidelines
How can we promote “clinical
expertise”?
•
•
•
•
•
Continuing education
Peer support and review
Mentoring
Discussion forums/Journal clubs
Incentives / rewards
How can we promote the integration
of clinical expertise, client’s
perspectives & research evidence?
• Role model integration
• Explicitly articulate & share examples of
integration and ways to communicate with
clients
• Involvement in research activities
• Mentoring
What are Practice/Clinical
guidelines?
Research to Guidelines Pathway: the journey from research to clinical practice
Judi Strid - Women's Health Action, www.nzgg.org.nz, accessed 30.6.07
Guidelines: Can be recommendations based on
consensus agreement, expert opinion or experience.
Some forms of evidence may also be included. The
guideline provides the recommended approach but not
the practical “how to” details specified in a protocol or
pathway.
Best Practice guidelines: Based on expert opinion generally used when evidence is limited, of poor quality
or conflicting.
What are Evidence-based guidelines?
Research to Guidelines Pathway: the journey from research to clinical practice Judi Strid - Women's
Health Action, www.nzgg.org.nz, accessed 30.6.07
Evidence-Based Guideline
• Based on evidence which is explicitly stated and includes levels of evidence
• The guideline is a tool for transferring research findings into a guide for clinical
practice
•
•
•
Follows a clear format and process which is described within the guideline so it
can be readily replicated.
An algorithm or flowchart can be included to provide easy to follow steps that
reflect the recommendations outlined in detail within the guideline
Should be developed in a collaborative manner (to avoid duplication) and
involve a multidisciplinary approach with consumer input
Where to find evidence-based practice
guidelines?
(ASHA website, 2009)
• http://www.asha.org/
• http://www.asha.org/members/compendiumSearchResults.aspx
?type=0&searchtext=Augmentative%20and%20Alternative%20
Communication
Step 6. Evaluating the
implementation of EBP
1. Has the EB intervention/assessment with clients’ input changed
outcomes for clients?
1. SSED, systematic collection of data from clients
2. Are the guidelines appropriate & effective?
3. Monitoring your clinical expertise, your ability to communicate with
clients,
4. Do we need to make any changes to our practice? If yes, what
are they?
5. Are there questions that need further investigation?
6. What new information is known? Updating the evidence
Step 7. Dissemination of EBP process &
results
• Share the process & outcomes of EBP
with colleagues, work site seminars, at
state, national and international
conferences
• Submit CATs to a CAT Bank
• Submit SR and other research to journals
• Use the EBP process when you plan
your own research
• Aim to conduct rigorous research and
thus contribute to high quality evidence
The Change to EBP
The shift to EBP requires major change in
attitudes, behaviors and routines
Barriers to implement EBP
Two surveys show limited use of research/theory in practice in OT (Dobson 2001;
Forsyth & Hamliton, in press;cited in Forsyth et al., 2005)
1.
2.
3.
4.
5.
6.
Information mismatch between research/theory- and practice
Lack of time to read research evidence
Lack of time to reflect
Willingness to change intervention approach based on evidence
Knowledge and skills needed to understand research articles
written for other researchers
Lack of organizational and/or professional support
Group Work Activity 5 – Barriers &
Facilitators
Factor
Patient/Client
EBP process—7 steps
Team Issues
Care/service process
Management Support
Time/facilities/cost
Health System
Other
Potential barrier(s)/Facilitators
Examples of Positive
Developments/Strategies
• Two New Secondary Journals that reviews and appraise research with
bottom-line messages
– Evidence-Based Communication Assessment &
intervention
– Communication Disorders Review
• Increase in the # of Systematic Reviews
• Journals have a summary at the end of articles on what it means
clinically –take home/key messages (e.g., Physiotherapy, AJOT, AJSLP)
• Critically Appraised Topics & Papers (CATs & CAPs- e.g.,AJOT)
• Development of Clinical/Practice guidelines
How does a busy clinician incorporate
evidence into everyday practice?
(Corcoran,2006)
1) To make positive changes, requires attitudes, knowledge
and skills
1) Attitudes: Ethical & professional responsibility
2) An hour to get started
3) Get a bird’s eye view of research available e.g., new journal
JEBCAI; look for systematic reviews.
• Bookmark most useful websites
• Learn to search efficiently & effectively by using keywords to search
favorite databases
• Find services that’ll find info. for you, e.g., send alerts of new articles
What our experiences have revealed
• Training is essential
• Appraisal of articles takes time and resources
• Accuracy of responses in appraising varies (inter-rater
reliability)
• Compiling results of EBP reviews are a challenge
• Integration of findings to re-evaluate current practice
and potentially alter discipline practice takes time
• Good library and information support
• Sustainability needs to be factored in to the process
Summary
1. Nature and extent of evidence in
AAC
1. SR’s increasing
2. 2 new journals
3. Increased awareness of more rigorous
research
Summary
• What can you do to add evidence to the
AAC field?
–
–
–
–
–
Collect outcomes data systematically
Look at that data; analyse; discuss
Read, Read, Read; Ask, Ask, Ask
Share your findings
Collaborate with researchers and
Practitioners
As opposed to tradition or authority,
EBP is a practice based on the
integration of scientifically
sound knowledge
which incorporates both client values
and perspectives
and clinical expertise
Workshop EVALUATION
THANK YOU!!!
References
Robey, R., Apel, k., Dollaghan, C., Ellmo, W., Hall, N., Helfer, T., Moeller, M., Threats, t., Hooper,
C. Kent, R., Drown, J., & Lonsbury-Martin, B. (2004). Report of the Joint coordinating
committee on EBP. American Speech-Language-Hearing Association: Washington DC.
Sackett, D. L., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (1997). Evidence-based
medicine: How to practice and teach EBM. New York: Churchill Livingstone.
Schlosser, R. W. (2004, June 22). Evidence-based practice in AAC: 10 points to consider. The
ASHA Leader, pp. 6-7, 10.
Schlosser, R. W., & Raghavendra, P. (2003). Towards evidence-based practice in augmentative
and alternative communication. In R. W. Schlosser, The efficacy of augmentative and
alternative communication: Towards evidence-based practice (pp. 259-297). New York:
Academic Press.
Schlosser, R. W., & Raghavendra, P. (2004). Evidence-based practice in augmentative and
alternative communication. Augmentative and Alternative Communication, 20(1), 1- 21.
Tickle-Degnen, L. (2000). Gathering current research evidence to enhance clinical reasoning.
American Journal of Occupational Therapy, 54(1),102-105.