Using Research to Inform
Professional Decision
Making in a Context of Risk
Brian J Taylor
Department of
Social Work
Social Work takes place in a context of
uncertainty or ‘risk’

Assessing & managing ‘risk’ is a
central feature of practice, involving
decision making in situations of
uncertainty*
*Skills for Care and Development (2005) “National
Occupational Standards for Social Work”
London: SCD (paraphrased)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
2
Assessing and Managing Risk






What do we do when we “assess risk”?
We assess ‘Risk’, BUT what do we do next?
What do we mean by “manage risk”?
What are ‘reasonable steps to minimise risk’?
Is it all about Health & Safety at Work
legislation?
What are the issues for organisations,
professions, policy makers, managers, trainers
and practitioners?
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
3
Organisational assessment of risk
CONSEQUENCE
1
Insignific
ant
LIKELIHOOD
2
Minor
3
Moderate
4
Major
5
Catastro
phic
5
Almost
certain
Low
5
Medium
10
Medium
15
High
20
High
25
4
Likely
Low
4
Medium
8
Medium
12
High
16
High
20
3
Possible
Very Low
3
Low
6
Medium
9
Medium
12
Medium
15
2
Unlikely
Very low
2
Low
4
Low
6
Medium
8
Medium
10
1
Rare
Very low
1
Very low
2
Very Low
3
Low
4
Low
5
Department of Health (2003) Controls Assurance Standards January 2003
London: Department of Health
Department of Health, Social Services & Public Safety (2003) Risk
Management: Core Standard Belfast: DHSSPS
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
4
Organisational management of risk

Publicly accountable members of the Board
must have in place systems so that they
know what the major risks are in their health
and social care organisation



Financial
Legal
Professional care and treatment (statutory duty
of quality)
Department of Health, Social Services And Public Safety (2001) Best
Practice – Best Care: A Framework for Setting Standards, Delivering
Services and Improving Monitoring and Regulation in the HPSS: A
Consultation Paper Belfast: DHSSPS
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
5
Professional management of risk



Accountability of professions for managing
risk as part of corporate risk management
“… focus on safety, accountability and
improvement of services”*
“… culture of self-evaluation and
continuous improvement.”*
Department of Health, Social Services and Public Safety (2002)
Departmental Guidance HSS(PPM) 10/2002 Governance in the
HPSS – Clinical and Social Care Governance: Guidelines for
Implementation Belfast: DHSSPS
*Simmons L (2007) Social Care Governance Workbook Belfast:
DHSSPS and London: SCIE for Clinical and Social Care
Governance Support Team
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
6
By now we
are all
familiar with
undertaking
a ‘risk
assessment’
on activities
– even
Dilbert is
gradually
getting used
to the idea!
SSRG 09 April 2008
copyright©1997
United Feature
Syndicate, Inc.
Reproduced by
permission
bj.taylor@ulster.ac.uk
7
How
meaningful is
this risk
assessment?
What value
does it have?
How does
risk
assessment
fit with
professional
values, client
values, & the
professional
task?
SSRG 09 April 2008
copyright©1997
United Feature
Syndicate, Inc.
Reproduced by
permission
bj.taylor@ulster.ac.uk
8
Do we
all see
“risks”
the
same
way ?!
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
9
Three examples of using research in
relation to risk and decision making



Qualitative research
 in work with Older People
Experimental research
 in Mental Health
‘Theoretical research’
 in Family and Child Care
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
10
EXAMPLE 1: Professional Perspectives
on Risk and Decision Making


To make organisational &
professional approaches to
assessing & managing risk
effective, we need to
understand how practitioners
‘do the risk business’
What tools and concepts do
professionals use to ‘do risk’?
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
11
Perceptions of health & social care staff on
decision-making on long-term care of older
people
 Grounded theory approach
 Sample: 4 of 11 HSS Trusts in NI that
deliver community health & social services
 one in each Board area (policy effects)
 at least one providing only community
health & social services
 at least one ‘integrated Trust’ (with acute
services)
(also a proxy for rural and urban)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
12
Data gathering method & sample



Focus groups with social workers,
community nurses, occupational
therapists, hospital discharge support
teams, home care managers
Interviews with consultant geriatricians,
general medical practitioners, older
people (with their carers, where
appropriate)
99 staff in total; 9 older people
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
13
Identified Risk Factors for Admission
to Institutional Care (fits the evidence!)








mental impairment
including dementia
falls & fractures
ADL limitations
managing medication
incontinence
health-related needs
sleep problems
nutrition
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
14
What evidence do you use in
assessment & decision making?



We only predict from our experience and
knowledge (Care Manager)
We know the number of people who end up
in institutional care after a fracture … most
of that [predictive data on the probability of
harm] would be helpful (Geriatrician)
BUT generally little explicit use of research
or synthesised data in relation to individuals
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
15
Use of theory (based on evidence?) (1)



I don’t think we have put <risk> on the
training agenda explicitly (Geriatrician)
There is no tick chart or formula that I
work to. It’s just what is staring me in
the face when I go out there (GP)
I’m used with the Brearley model to
make my mind focus on issues … You
know when people are shouting at you
“they are at risk out there” (SW)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
16
Building A Model of Admission Decisions


Some findings led to
a new
conceptualisation of
decision making
about admission of
an older person to a
residential or
nursing home
We illustrate using
one factor
>
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
17
Rated very seriously

And then there is so much robbery now and
beating old people up. If she <elderly aunt>
falls she would always get better again but if
somebody came in and beat her up or robbed
her, she would never get over it. It would kill
her. (Family carer interviewed with her aunt,
an older woman living at home against
professional advice)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
18
Client fears – even refusing services


Everybody knows who you are. I’ve got my
pension and I suppose they know how much
you are worth every week and are liable to
break in. Sure it’s happening all the time.
(Older Man)
She needed a handrail at the front door but
<refused it> because there were so many
break-ins recently she thought that it might
make her look more vulnerable if she had a
rail. (Occupational Therapist)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
19
But not regarded as a legitimate or
effective reason for admission



Researcher: Is this a factor in admission to
institutional care, this vulnerability to intruders
or burglary that you mentioned?
Social Worker: It would not be of course
officially in an assessment
This and other elements led to a new model:
Taylor BJ & Donnelly M (2006) ‘Professional perspectives
on decision making about the long-term care of older
people’ British Journal of Social Work 36 (5): 807-26.
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
20
Elements of Admission Process
Person living
alone loses
confidence
Person
living with family carer
who is no longer able or
willing to meet
needs
Availability
of services
incl, funding &
people
SSRG 09 April 2008
Professionals
assessing risk
considering law,
policy, theory, etc
bj.taylor@ulster.ac.uk
21
APPLYING THIS EVIDENCE IN:
Development of a Single Assessment
Tool for the Health and Social Care
Needs of Older People
– Jan 2006 to Jun 2008 (DHSSPS)
http://www.dhsspsni.gov.uk/ec-single-assessment-tool
PROJECT TEAM – University of Ulster
Professor Brendan McCormack, Institute of Nursing Research
Dr Brian Taylor, Social Work
Mrs Bridget Murray, Rehabilitation Sciences
Mrs Joanne McConville, Project Officer
Dr Paul Slater, Research Fellow
http://www.science.ulster.ac.uk/sat/
Elements of the Tool – developed with
350+ stakeholders

Contact Screening

Core Assessment


GP and Medical Practitioner Report
Specialist Referral
Specialist Summary & Recommendations
Carers’ Assessment

Complex Assessment


SSRG 09 April 2008
bj.taylor@ulster.ac.uk
>1
>2
23
Core Assessment – informed by a range
of health & social care research evidence
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Physical Health
Mental Health & Emotional Well-being
Awareness & Decision Making
Medication Management
Communication & Sensory Functioning
Walking & Movement
Personal Care & Daily Tasks
Living Arrangements & Accommodation
Relationships
Work, Finance & Leisure
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
24
Complex Assessment



Focus on risks to independent living in
relation to each domain in Core Component of the Tool
Recent events, crises and their impact on the older
person, including their confidence to live at home
Disruption or distress due to identified needs
(concerns, risks)





ability to cope with needs
rehabilitation and learning potential
health promotion possibilities
additional care or support required
How do identified difficulties impact on ability to make
choices and decisions about daily living?
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
25
Example 2: Applying experimental evidence
(knowledge) to manage uncertainty
Basic ethical principle
 ‘primum non nocere’ = first, do no harm
 “Counselling and psychotherapy* is probably pretty
useless and therefore mostly harmless”
*‘psychotherapy’ = “A generic term for the treatment of
mental illness or emotional disturbances primarily by
verbal or nonverbal communication” [Medline
database] ie includes art therapy, drama therapy as
well as most varieties of ‘counselling’
 BUT let’s look at an example of emerging evidence:

SSRG 09 April 2008
bj.taylor@ulster.ac.uk
26
Critical Incident Stress Debriefing Study
Debrief (Hi)
Control (HI)
40
Impact of Event Scale
35
30
25
20
15
10
5
Baseline
4 months
3 years
Richard Mayou et al, 2000
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
27
Risks in inappropriate counselling?





MAY make people worse than without treatment
In depression – behavioural activation is more
effective than just talking about problems
After the Omagh bomb in Aug 1998 the local Health
and Social Care Trust activated its emergency plan
& set up a Community Trauma and Recovery Team
(CTRT)
Of people coming because of Post Traumatic Stress
Disorder (PTSD) 50% had previous unsuccessful
psychological therapy or counselling which generally
added to their sense of hopelessness
WHY MIGHT THIS BE? (theory)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
28
Omagh bombing study - factors most highly
associated with Chronic PTSD
Injured
 Beliefs: ‘Unsafe World’
 In Market Street
 Thought would die
 Thought /Emotion Suppression
 Rumination
 Negative View of Symptoms/Self

.38
.38
.40
.42
.43
.54
.72
Duffy M, Gillespie K & Clark DM (2007) ‘Posttraumatic Stress Disorder in
the Context of Terrorism and Other Civil Conflict in Northern Ireland: A
Randomized Controlled Treatment Trial’ British Medical Journal
334(7604):1121-2.
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
29
What will help? How do we know?




The Omagh CTRTeam (included Social Work,
Psychiatric Nursing, Psychology, Psychiatry,
Occupational Therapy) had to face the question:
What helping approach shall we use?
How will we know if it is effective?
The team leader Michael Duffy (SW) and colleagues
developed a randomised controlled trial of the
effectiveness of Cognitive Behavioural Therapy (the
main treatment used) for the treatment of Post
Traumatic Stress Disorder in the context of civil
conflict – a previously untested application of CBT
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
30
CBT for common problems
Major depressive disorder
Panic disorder
Posttraumatic stress disorder
Social phobia
Generalised anxiety disorder
Obsessive compulsive disorder
Specific phobias
recovery
rate
50%
75%
80%
75%
50%
45%
75%
Recovery rate is the approximate proportion of individuals
who do not have the problem at the end of intervention
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
31
The end of psychosocial treatments?





ABSOLUTELY NOT!!
BUT
We need to note & appraise the evidence
which is much stronger for
STRUCTURED counselling models
E.g. “…problem-solving therapy, interpersonal
psychotherapy, and pharmacotherapy would be
considered efficacious interventions for major
depression, with cognitive-behavioral and
cognitive therapy considered possibly
efficacious.”
Wolf NJ & Honko DR (2008) ‘Psychosocial and pharmacological
interventions for depressed adults in primary care: A critical review’
Clinical Psychology Review, 28(1):131-61
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
32
Report by Lord Richard Layard
Mental illness is the UK’s greatest social problem £3 million to be set aside for training in evidence
based psychotherapies 2008-2010 (in England)
Layard R (2004) ‘Mental Health: Britain’s Biggest Social Problem?’ Paper
presented to Government Strategy Unit, London
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
33
Social Work & Psychosocial Interventions



If we in Social Work lose our role in
relation to psychosocial interventions
it will be because of our entrenched
attitudes and systems
not because such treatments do not work
or could not be shown to work or because
of a lack of public funding
cf Layard R (2006) ‘The case for psychological treatment centres’
British Medical Journal 332 (7548) 1030-2
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
34
Using Evidence:
in commissioning
new services
 in commissioning
training
 in decisions
about
individuals

SSRG 09 April 2008
bj.taylor@ulster.ac.uk
35
Applying evidence to decisions about care
of individuals





Going back 5 slides to the one on recovery
using CBT
CBT was generally shown to be effective for
a range of conditions, BUT
Between 20% and 55% of people were NOT
helped sufficiently to no longer ‘have that
problem’
What approach for THIS individual?
Can we refine our decision making?
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
36
Example 3: Slated by (ab)use of
research?!

SSRG 09 April 2008
“Now tell me Miss
Social Worker, in
relation to my client Mrs
Whitehouse and her
beloved child Amy,
have you read this
piece of research by
Sniff and Jones about
successful treatment of
addictions …?”
bj.taylor@ulster.ac.uk
37
Research Evidence for Court Decisions







What value is one research study?
How will we judge the quality of a study?
What value is a theory?
 E.g. attachment and bonding; stigma; loss;
rehabilitation; nurturing environment; discipline
Can we put more weight on a synthesis of
studies than a single study?
What criteria do we set for a good synthesis?
How does this study apply to Mrs Whitehouse?
How will we train Social Workers for this work?
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
38
Research-based reasoning in court?

“Effective clinical reasoning requires skill in developing
arguments, establishing the relevance of
information to an argument, and evaluating the
plausibility of assertions or claims…” Osmo R & Landau R
(2001) ‘The need for explicit argumentation in ethical decision–making in
social work’ Social Work Education 20(4) 483-492 (page 489)


“Expressions of opinion must be supported by detailed
evidence and articulated reasoning” (Munby J in Re M
(Care Proceedings: Judicial Review) (2003) 2FLR 171 p.183)
“Mrs R’s Article 8 rights required that her child should not be
taken from her unless every feasible alternative was
thoroughly explored and rejected for good reason.” (AR v
Homefirst Community Health and Social Services Trust
(2005) p.18 para 101)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
39
One model of explicit argumentation
1.
2.
3.
4.
5.
6.
7.
CLAIM Make a claim or conclusion
WARRANT Explain relationship between claim & data
DATA Provide grounds, data & evidence
JUSTIFY relationship between evidence and claim
QUALIFY the claim (degree of confidence)
Highlight LIMITATIONS & reservations to the claim
(including conditions where it might be weakened)
JUSTIFY (2) with additional evidence in relation to
qualifiers and limitations
Stephen Edelston Toulmin (1958) ‘The Uses of Argument’ Cambridge:
Cambridge University Press (adapted)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
40
A Child Care example




Ms G has three children who are in care –
problems of alcohol abuse & neglect
Child A (12) & Child B (14) are with different
foster families, Child C (15) is in residential
care
Ms G has been erratic regarding contact and
there have been problems of aggression
Decision required about contact – considered
detrimental to the children
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
41
Evidence in child protection (1)?
CLAIM - Rights under Article 8 HRA should
be over-ridden in the interests of the children
WARRANT – outline issues & relationship of
these
1.
2.



3.
4.
To whose benefit is continued contact?
Emotional well-being of children: identity, distress,
future prospects, stability
Impact of Ms G’s addiction on children
DATA on visits, views of children etc, &
research evidence related to these issues
to inform professional judgment
JUSTIFY connection: evidence and concerns
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
42
Evidence in child protection (2)?
5.
6.
7.
QUALIFY – research relating to severity &
likelihood of harm and benefit for continuing
contact and for ceasing it
LIMITATIONS – different situation of each
child, possible change in problematic
maternal behaviour
JUSTIFY (2) – research on resilience,
capacity to change once addicted etc
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
43
What is the research process to test &
bring this model to practice, if it proves
suitable at each stage?
1.
2.
3.
4.
5.
Consider application of theoretical model
Apply to historical or hypothetical cases?
Seek professional views?
Pilot studies?
Controlled trial of effectiveness, usability
& acceptability?!
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
44
Key Issue: Prediction of Harm





Is it ‘safe’ to return this child home?
Has this parent recovered sufficiently from
mental illness or addiction?
Is it ‘safe’ to discharge this person from
psychiatric hospital in terms of possible
violence or suicide?
Is it ‘safe’ for elderly Mrs Brown to return
home from hospital after her stroke?
Is it ‘safe’ for ‘us’ to support this step towards
independent living of young Mr O’Kane with a
disability?
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
45
Predicting (Screening for) ‘problems’



Suicide rate for Northern Ireland 2006
 227 male; 64 female
Treat as approximately 300 in a population
of about 1.5 million
 i.e. 2 in 10,000
Could we use a screening tool to identify
these people so as to target services?
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
46
Predicting rare events




If we had a test that was 90% accurate
and we are trying to predict a rare event such
as abuse, violence or suicide that occurs say
2 in 10, 000 people per year (as in this
example)
amongst e.g. a population of 100,000 people
EXERCISE: IDENTIFY ISSUES IN USING
THE TOOL BY CALCULATING HOW MANY
ARE CORRECTLY AND INCORRECTLY
IDENTIFIED
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
47
Effectiveness of a tool to predict
harm
TOOL:
REALITY
yes
no
yes
<1>
TP
<2>
FN
No
<4>
FP
<3>
TN
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
48
Predicting rare events



The real incidence of abuse or suicide or
other risk would be 20 cases per year in the
population of 100,000 [2 in 10,000], and so
there would be 99,980 cases that year where
this harm did not occur.
[total = 100,000]
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
49
False Negatives



With a 90% accurate tool, we would
detect
18 of the 20 cases of harm correctly
(item 1, TP),
but would miss the other 2 “false
negatives” (2, FN) because the test
wrongly said there was no risk
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
50
False Positives



OF the 99,980 cases that were not a risk,
the test would correctly show 89,982
[90%] as being no risk situations (3, TN)
but would incorrectly pick up 9,998 (10%)
of the no-risk situations as being risky
“false positives” (4, FP) because the test
incorrectly labels them as risky when they
are not.
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
51
Issues in using a tool to predict harm
TOOL:
REALITY
yes
yes
no
total
18
2
20
No
9,998
89,982
99,980
Total
10,016
89,984
100,000
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
52
Using a 90% accurate tool to predict NI
suicides (pop 1.5M)
TOOL:
REALITY
yes
yes
No
149,970 1,349,730 1,499,700
(FP)
(TN)
150,240 1,349,760 1,500,000
Total
SSRG 09 April 2008
no
270 (TP) 30 (FN)
bj.taylor@ulster.ac.uk
total
300
53
Problem of False Positives




Trying to predict rare events such as abuse,
violence or suicide in a much larger population
presents particular problems
Many non-risky situations will be identified as
being “risky” with consequences for individuals
and for workloads of professions, teams &
organisations
How can we know for certain whether or not this
individual is the one where the harm will occur?
IMPOSSIBLE!
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
54
The Best Prediction Tools



The very best prediction tools in social work
and criminal justice are achieving rather
lower accuracy than the example above
(perhaps 70% - 80%)
e.g. predicting the likelihood of a convicted
offender being re-convicted by committing a
similar type of serious crime
False positives & false negatives are an even
greater problem with a less accurate tool
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
55
“A Continuum of Risk”


“Research therefore cautions us that [in
mental health] as in other fields such as
medicine and child protection there is no
such thing as a ‘risk free’ assessment….
… There are no criteria which enable us to
place individuals into sharply-defined, onceand-for all categories of ‘dangerous’ or ‘not
dangerous’. Rather there is a continuum of
statistical risk with uncomfortably limited
predictive capacity.”
Perry, J and Sheldon, B (1995) Richard Phillips Inquiry Report,
London: City of Westminster, and Kensington & Chelsea &
Westminster District Health Authority (p.18)
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
56
Prospects for Actuarial Risk Assessment






Tools continually improving for predicting probability
of harm esp. criminal justice & mental health
Actuarial prediction often discriminates better
between risk levels than clinical judgment
Might inform priority for services
Recognise limitations regarding predicting harm
Take a broader view of ‘assessment’
Accept more diffuse approaches to targeting
preventive measures


E.g. postcode vs identifying individual ‘children in need’
Study how we make professional decisions
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
57
Conclusions (1)



To make organisational & professional
approaches to assessing & managing risk
effective, we need to understand how
practitioners ‘do the risk business’
Assessment tools to support decision making
can be valuably informed by research and can
support professional clinical judgment but we
cannot predict rare events (harm) accurately
Be confident in our SW skills in psycho-social
interventions – but we need to focus on
treatments based on robust evidence
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
58
Conclusions (2)


Evidence can be applied more readily to
commissioning services and training than to
decisions about care of individuals
We need a ‘network’:



to energise research on risk, evidence and
decision-making in Social Work
to build theoretical models of decision making
To develop models for bringing research from
theoretical concepts through empirical studies
and into practice
SSRG 09 April 2008
bj.taylor@ulster.ac.uk
59
Using Research to Inform
Professional Decision
Making in a Context of Risk
THE END
bj.taylor@ulster.ac.uk