Randomised Trials in UK
Social Work: A Discussion
of Possible Approaches
Donald Forrester
Professor of Social Work Research
Director of the Tilda Goldberg Centre
Proposed RCT
• Early stages of development (2012-14)
• Want to openly discuss issues and challenges
– Not got all answers and would rather have errors pointed
out now...
• Focus: whether Motivational Interviewing improves
engagement of parents who misuse drugs or alcohol
• But first a bit of history, theory and background
Social Work’s Evidence Base:
Emperor’s New Clothes?
RCTs are one of the
greatest intellectual
th
achievements of the 20
century
What is an “RCT”?
Participants
RANDOMLY divided
Desired outcomes
measured
Desired outcomes
measured
Intervention
Nothing / Normal Service
Desired outcomes
measured
Desired outcomes
measured
What are advantages?
• The best way of ruling out bias
• Bias defined as explanations other than the
intervention that might explain results
• Key types of bias:
–
–
–
–
Self selection bias
People are active and resolve problems
Social work is about maintenance not cure...
Self-report not good indicator of impact – people
usually appreciate services
History of RCTs
• First comparative trial – in the Bible. Daniel compares
vegetarian diet with meat and drink.
• RCTs in education and social work 1930s
• Medicine from the late 1940s
• Now dominant approach in medicine
Selective history of RCTs
Cambridge and Sommerville Study
• USA 1930s
• Intensive, long-term befriending from social
worker for boys considered at risk of offending
• Much appreciated by young men
• Increased offending
From Oakley, 2002
Social work and RCTs
• Dominant social work approach in 1960s,
psycho-dynamic casework
• Reid and Shyne tested and found:
– Long-term work LESS effective than short term
– Developed task-centred approaches based on
approach
Social work and RCTs
• Homebuilders Intensive Family Preservation
Service
• Aimed at preventing children entering care
• Success rates from 70 – 100%
• Large-scale RCT – no impact
Field social work RCTs in the UK
1970-2010
Problems with RCTs
Bias
Generali
sability
Context
Context
Pawson and Tilley – Realist critique – context crucial for
understanding nature of intervention and its impact
Orford (2008) from within the RCT field – reflecting on the
“Dodo Bird” effect
• Whatever is happening across interventions more
important than differences between interventions
• Context of service, client or research
• Requires emphasis on understanding context, process as
well as outcomes
Generalisability
• RCTs on drugs establish general knowledge
• This isn’t true for psychosocial interventions:
– Implementation fidelity and innovation effects
– What the control group receives eg “nothing works
in Sweden…”
– Or Dodo effects – everything works equally…
• This is a fair criticism of naïve positivism
• But ALL research makes claims to generalisability – and
all should be considered critically
Outcomes and social work:
the politics of research
Society’s
View
Client’s
View
Expert
View
Reimagining RCTs
• Starting point to develop a critical form of
RCTs that addresses these concerns – a
characteristically and perhaps distinctively
social work approach to RCTs
• Many of problems with RCTs are of RCTs as
done not as should or could be
• So how should we do RCTs...
Early social work RCTs
• Reid and Shyne (1969) – brief vs open-ended social
work with middle class couples with relationship
problems
– Randomization purely on length of intervention
– Taped interviews and found this led to more focussed
approach
– Extensive outline of context of intervention
– Both women and (particularly) men preferred this
– Social work, service user and standardised measures of
outcome all agreed ST work produced better outcomes at
12 months
Goldberg (1970): Helping the Aged
• Randomised older people to “normal service” (with
unqualified workers) or service from qualified social
workers
• Detailed description of needs, services and outcomes
and combination of qualitative and quantitative
methods
• High levels of need, improvements in both groups
• Major problem: the “intervention” differed in
multiple ways
Evaluating Helping the Aged
• The descriptive work was fantastic
• The research is transparent and self-critical
• Contributed to the programme of work that
culminated in “Community Care”
• Should have been the start for RCTs in social
work...
Field social work RCTs in the UK
1970-2010
Not the research questions
• Can we show that any of the approaches
taught to social workers makes a positive
difference – compared to “normal practice”?
• Can we do this in a “normal” local authority
setting? (chosen child and family work)
• Can we make an RCT “work” in social work?
Key Considerations: Outcomes
• What outcome/s to measure?
• Legal basis (Children Act 1989)
– Protection from harm
– Meeting needs
• Every Child Matters outcomes (2004)
–
–
–
–
–
Health
Safety (eg from abuse)
Education
Making positive contribution
Achieving economic wellbeing
Key Considerations: Outcomes
• Complex factors influencing outcome/s
–
–
–
–
Social situation (eg poverty, racism etc)
Personal problems of parents (eg addiction)
Difficulties in child (eg challenging behaviour)
Patterns of family functioning and parenting
• Complexity of social work “intervention”
–
–
–
–
Direct work
Interagency linkage
Risk assessment and decisions
Not voluntary clients seeking help
Key Considerations: Outcomes
Single biggest problem likely to be
family participation rate in research
Commonly as low as 5% - we need 60-70%
Or a good proxy...
Key Considerations: Intervention
• Chose Motivational Interviewing as:
– Strong evidence base in substance misuse field
– Very flexible style of working
– General style of communication aimed at
engaging resistant clients
– Consistent with social work values
– Initial piloting study undertaken – which
highlighted challenges
Simple Overview of Intervention
Process
Improved
Communication
Skills
Increased
Engagement
Positive Impact
on Parent and
Family
Child Protected
and Thriving
Piloting approach to outcomes
Improve
Engagement of
Parents
• Tapes– parent response (MISC and bespoke)
• Parent views of SW (simple ratings)
• Number and length of visits (worker and parent)
• Engagement with other services (worker and parent)
Improve Parent
and Family
Wellbeing
• Reduced family stress (PSI)
• Reduced alcohol and drug use (MAP)
• Reduced parental stress (GHQ)
Protect child and
improve welfare
• Whether abused/neglected (problematic) – using
parent and social worker measures and one SI
• Child’s emotional and behavioural welfare (SDQ)
• NOT expecting to show an impact...?
Current Pilot
• Combining small-scale RCT of MI with quasiexperimental study of “Hackney Model”
• 40 workers in 10 teams/units in 2 contrasting LAs
receive intensive MI input
• 40 workers do not
• Piloting MI input, types of data collection and
success of different approaches to families
Elements of Data Collection Piloted
• Direct tapes of practice
– Measured for MI-ness
– Social work skill ratings being developed with group of experts by
experience through iterative process
• Brief questionnaires to all families allocated
– With follow-up from administrators
• Research interviews with families using various
standardised instruments
– Key problem likely to be low response rate
• Social worker questionnaires on families
– Key problem validity
– To be tested by comparing with research intvws
• Exploring using LA data measures
Concluding Thoughts
An RCT is the most robust test of effectiveness,
because it minimises bias
Social work needs to engage with the complexity
of RCTs in order to evaluate whether what we
do works
A critical social work perspective is also likely to
create better RCTs
Concluding Thoughts
Developing RCTs fit for use in social work
settings is not straightforward or easy
Doing so will not provide all of the answers
However, hopefully it will provide some
evidence about what skilful social work is and
the difference it can make...
“If we knew what it was we
were doing, it would not be
called research, would it?”
Einstein
References
Forrester, D. (2010) Playing with fire or rediscovering fire? The perils and Potential for Evidence
Based Practice in Child and Family Social Work”, in Ayre, P. and Preston-Shoot, M. (Eds)
Children’s Services at the Crossroads. A Critical Evaluation of Contemporary Policy for
Practice, Russell House Publishing; Dorset
Goldberg, E.M., with Mortimer, A. and Williams, B.T. (1970) Helping The Aged—A Field
Experiment In Social Work National Institute for Social Work Training, No. 19, George Allen &
Unwin: London. 1970.
Holosko, M. J. (2010) “What Types of Designs are We Using in Social Work Research and
Evaluation?”Research on Social Work Practice; Nov2010, Vol. 20 Issue 6, p665-673
Oakley, A. (2000) Experiments in Knowing: Gender and Method in the Social Sciences, Polity Press
Orford, J. (2008) “Asking the right questions in the right way: the need for a shift in research on
psychological treatments for addiction”, Addiction, Volume 103, Issue 6, pages 875–885, June
2008
Reid, W.J. and Shyne, A.W. (1969) Brief and Extended Casework, Columbia University Press
Torgerson, D.J. and Torgerson, C.J. (2008) Designing Randomised Trials in Health, Education and
the Social Sciences, An Introduction, Palgrave Macmillan; Basingstoke