Juliet Goldbart
Research Institute for Health & Social Change
Manchester Metropolitan University
1.
Brief consideration of Evidence-Based
Practice.
2.
Hierarchy of research evidence.
3.
Where there isn’t sufficient evidence…
4.
Options for practitioners to add their own
evidence.
Research
evidence
Service user
values
Professional
expertise





To provide the best possible education/ therapy/
intervention for a particular individual.
To meet increasing demands for accountability.
To avoid unethical use of ineffective education/
therapy/ intervention.
To provide a common language for use among
multidisciplinary teams.
To promote transparency for service users and
other stakeholders.

Whose evidence?

Publication bias (Dickersin, 1990; Chalmers et al.,
1990; Fanelli, 2011; Torgerson, 2006).

Non-reporting of negative evidence.

What research gets funded?

Remember:
An absence of evidence does not mean
negative evidence.

Who are the experts? Us?

Are the experts right?

What factors inform and constrain our
professional decision-making?

Eclecticism versus protocol driven or
centrally determined approaches?
Which is Intensive Interaction (II)?

Universal vs targeted vs individual(ised)
approaches? Where does II fit?




In the era of “consumer choice” is there a
tension between choice and evidence?
What role do learners/clients/family members
want?
How are comfortable are we with involving
learners/clients/family members in decisionmaking?
(How) can we ensure that learners/clients with
significant cognitive impairments have the
knowledge and skills for active choice?
Level
1a
1b
2a
2b
3a
3b
4
5
Type of Evidence
Systematic Review or Meta-Analysis of RCTs
A single Randomised Controlled Trial (RCT)
Systematic Review of Cohort Studies
A single Cohort Study
Systematic Review of Case Control studies or
Quasi Experimental studies
A single Case Control Study or Multiple Baseline
SCED design
Non experimental descriptive studies eg correlation
studies, other single case experimental designs
Expert opinion, textbooks, “first principles” research

Government guidance and policy?

Initial professional education?

In-service courses?

Textbooks?

Professional magazines?

Academic journals?

Internet: Wikipedia to SpeechBITE?
Current knowledge & experience.
 Learners’/clients’/other stakeholders’ apparent
preferences.
 Context e.g. school vs. adult service.
 Aims: individual, group, service.
 Assessment data.
 Resources.
 Access to research literature.
 Availability of relevant evidence.

Production of knowledge
Is the evidence base in intellectual disability,
speech path and special education sufficient?

Research – Practice Gap
Are teachers, therapists and other staff using
the available evidence?
If not, why not?

(e.g. Burton & Chapman, 2007; Odom et al., 2005)

Challenges of controlled research studies in
education and social care settings, e.g.
•
•
•
•

Controlling social variables,
Frequent staff changes,
Service emphasis on individualisation,
Complex and multiple interventions.
Low prevalence and heterogeneity of
potential participants.


Many interventions in learning disabilities are
“of uncertain value and … have never been
tested” (Parmenter, 2001, p.191).
In relation to PMLD, “researchers have shown a
limited interest in providing an empirical base
for these interventions” (Vlaskamp & Nakken, 2008,
p.334).

“randomized trials are rarely applicable for
students from a low incidence population” (Snell,
2003, p.143).
(Burton & Chapman, 2007; McDonnell & O’Neill, 2003;
Snell, 2003)
Available research is not relevant to practice.
 Practitioners may have limited access to
research findings.
 Research findings may contradict long held
beliefs and practices.
 Family, legal and service pressures may affect
decision-making.


Take part in conferences & training!
Collaborate with researchers to obtain
research funding.
 Make the best use of what there is:
o See tomorrow’s session
 Contribute to the evidence base yourselves:
o Well described Case Studies; SCEDs.


Establish a track record through small scale
research e.g. Masters degree.

Collaborative bids between schools, health service
& universities.

NHMRC: e.g. Translating Research Into Practice
(TRIP) Fellowships

Using NHMRC’s Research Help Centre and other
supports.

ARC’s National Competitive Grants Program, but
less clear for education and social care than health.

Resources may not be available to undertake
large scale studies.

Teachers, therapists & other staff have great
data or the potential to collect such data.

Single Case Experimental Designs (e.g. Kazdin,
2011; Romeiser-Logan et al., 2008) and Case Study
designs allow practitioner research to contribute
meaningfully to the evidence base.





An empirical inquiry
“that investigates a contemporary
phenomenon
within its real-life context;
when the boundaries between phenomenon
and context are not clearly evident;
and in which multiple sources of evidence are
used.”
(Yin, 1984, p. 23)








A “how” or “why” research question.
Choose your “cases.”
Describe the context.
What data do you need?
Often quantitative AND qualitative data.
Use a table or database to organise the data.
Collect data systematically but flexibly over time.
Look for patterns and links but also contradictions
and disagreements.




Case study template to support experimental
design in AAC and AT (Murray et al., 2013)
Origins in Cochrane review (Pennington et al)
showing insufficient reporting of study details in
SLT for children with cerebral palsy.
Pennington, Marshall & Goldbart (2007): a
detailed, ICF-based reporting framework.
Murray et al., using an expert reference group,
refined the framework for use in practice.
Template allows 24 pages of consistent detailed
information on participants, communication
partners, the intervention and outcomes to be
uploaded to a database .
 Database can be interrogated by practitioners and
researchers on submission of ethically approved
request.
 www.communicationmatters.org.uk/aac-evidencebase
 Since May 2013, 35 case studies uploaded.


Similar model could be used in other areas.

What would be needed to set up an Intensive
Interaction Case Study Framework/

Who would manage it?

Would you contribute to it?

Would you use the findings?

One type of “Time Series” design

Data are collected on one or more participants
during a baseline phase then during a
treatment/intervention phase.

Not a robust design because few threats to
validity are controlled for.

BUT by aggregating detailed case study data we
can build up meaningful evidence.
Increasing Classroom Attending
Percent Attending
Behavior
110
90
70
Baseline
Intervention phase
50
30
10
5
9
15
20
No. of 10-min. Observation Sessions
Walker, H.M., & Buckley, N.K. The use of positive reinforcement in conditioning attending behavior. Journal of Applied Behavior
Analysis, 1968, 1, 245-250
 Types of “Time Series” designs.
 Looking at change over time in response to
treatment/intervention.
 Each participant serves as one trial and as their
own control.
 Avoid the ethical problem of no-treatment
controls.
 Practitioners can contribute to the evidence base
(Cakiroglu, 2012; Horner et al., 2005)

Usually require the establishment of a stable (or
decreasing) baseline.

Use of repeated measurements during baseline
and treatment sessions or frequent “probes.”

Analysis can be statistical, but more usually
visual: variability, slope, overlapping data points.

Use multiple iterations/replications to
demonstrate effect.
Various design options:

Multiple Baseline designs - (across participants,
behaviours or settings).

Withdrawal, Reversal or ABAB designs – assume
behaviour can be reversed - relevance?

Alternating treatment designs - the rapid
alternating between two or more conditions. No
baseline needed.





Evaluation of Intensive Interaction in a hospital
setting.
6 participants: adults “who experienced severe
difficulties in learning and in relating to others.”
Multiple baseline across participants.
Baseline observational data on initiation and
maintenance of interaction plus PVCS.
Findings: Increases in social and communicative
abilities.

Quality appraisal through Tate et al (2008) or
more recently RoBiNT (Tate et al, 2013) – used
by SpeechBITE.

Multiple baseline designs more robust.

Nine replications across studies: “well
established” (Parker et al., 2008, p.545).

Three replications across studies: “probably
efficacious” (Parker et al., 2008, p.545).



Evidence based practice is important for
service delivery and for clients (inter alia).
The evidence base can be expanded by
researchers and practitioners.
We need to be creative in working out ways
to use the evidence we have, generate new
evidence and influence the evidence that is
sought.
Thanks for your interest
Any questions?
Juliet Goldbart
j.goldbart@mmu.ac.uk