Overcoming problems associated
with the utilisation of RCTs in
forensic settings
Jane Clarbour, Cynthia McDougall and
Amanda Perry for the Treatment
Change Design Team (TCDT).
RCT in the Social Sciences:
Challenges and Prospects
1
Acknowledgements
Treatment Change Project: Evaluating cognitive behavioural
programs in prisons
Treatment Change Design Team
Cynthia McDougall
Roger Bowles
Jane Clarbour
Amanda Perry
David Richardson
Jeremy Miles
Catherine Hewitt
Padraic Monaghan
Ben Cross
RCT in the Social Sciences:
Challenges and Prospects
2
RCTs in the UK Criminal
Justice System
• Review of RCT literature in UK CJS
(Farrington & Welsh, 2003)
– RCTs at height of popularity in 1960s in the UK CJS
– Clarke and Cornish (1970’s) disillusionment with RCTs
in the CJS following their study of the Kingswood
training school RCT
• Therapeutic community
• Traditional regime
 2 year follow up = no difference in reconviction rates
RCT in the Social Sciences:
Challenges and Prospects
3
RCTs in the UK Criminal
Justice System
Martinson (1974) doctrine that Nothing Works:
-
Led to policy change in CJS and reduction of RCTs to evaluate CBT
programmes
What Works literature of 1980’s
-
Led to increasing use of accredited CBT programmes
Increasing number of evaluations using pre-to-post test design
Department of Health/Home Office 2000 White Paper Part II Reforming
the Mental Health Act suggested:
“There is a need to undertake long term randomised trials with long term
follow up” (section 6.53)
RCT in the Social Sciences:
Challenges and Prospects
4
Problems encountered
• Feasibility studies
(Farrington & Jolliffe, 2002; Farrington et al., 2001).
– Young offenders had education disrupted
– Institutional staff
– Insufficient case flow
– Few people assessed
RCT in the Social Sciences:
Challenges and Prospects
5
Background to current RCT
Research on Cognitive Behavioural
Programmes in 1990s
– Positive results on reconviction rates
Research in 2000-2004
– Mixed results unsure whether ETS and R&R
courses are effective for all.
RCT in the Social Sciences:
Challenges and Prospects
6
Aims of the current project
• To evaluate ETS courses.
• Which offenders benefit from treatment
and under what conditions?
• What effect do programmes have on
prison behaviour of those who complete &
dropout from participation?
RCT in the Social Sciences:
Challenges and Prospects
7
The study
Evaluation of ETS treatment programme
undertaken within H.M. Prison Service
Cognitive-behavioural programme
10 prisons
Target group
Young & Adult male offenders
Normal IQ
On remand/sentenced
Eligible to complete Enhanced Thinking Skills (ETS)
RCT in the Social Sciences:
Challenges and Prospects
8
Waiting-list control design
Pre-test
Group 1
Time 1
Group 2
(Waitinglist control)
4-week
intervention
Post-test 1
Post-test 2
Time 2
Time 3
Pre-test
Post-test 1
Post-test 2
Time 2
Time 3
Time 1
4-week
intervention
(Time)
RCT in the Social Sciences:
Challenges and Prospects
9
Waiting-list control design
Pre-test
Group 1
Time 1
Group 2
(Waitinglist control)
*PrePreTest
4-week
intervention
(Test – Retest
Reliability)
Post-test 1
Post-test 2
Time 2
Time 3
Pre-test
Post-test 1
Post-test 2
Time 2
Time 3
Time 1
4-week
intervention
(Time)
RCT in the Social Sciences:
Challenges and Prospects
10
Ethical considerations
• Unique identification number
• Waiting list control avoids withholding
treatment
– BUT random allocation considered unethical if
release date is before start of treatment
hence cohort design
RCT in the Social Sciences:
Challenges and Prospects
11
Random Allocation Model
Prison population at each
establishment
Treatment Managers
assess referrals to
obtain eligbility for
the course
Prioritized offenders
are removed from
the random
allocation
Unsuitable referrals
are disgarded from
the list.
Random allocation
occurs
Randomly allocated
to Course (I)
RCT in the Social Sciences:
Challenges and Prospects
Randomly allocated
to Waiting List
Control (WLC)
12
Design
Randomised control trial
Experimental group
– Individuals randomly allocated to start course immediately
Waiting list control group
– Individuals randomly allocated to start the next course
PLUS:
Additional cohort group
– Individuals prioritised by treatment managers as requiring
treatment immediately
RCT in the Social Sciences:
Challenges and Prospects
13
Example: Joe Bloggs
• 25 years old
• Convicted of Burglary and Theft (prolific offender)
• Referral to psychology department from
sentence planning
• Highly motivated to attend ETS course
• Release Date
(4 weeks after completion of course)
RCT in the Social Sciences:
Challenges and Prospects
14
Problems in random allocation
• Formation of the group
– Group Dynamics (e.g., motivational factors)
– Operational decisions (e.g., security issues)
– Offender characteristics (e.g., release dates)
– Static factors (e.g., prison population)
RCT in the Social Sciences:
Challenges and Prospects
15
Reasons for non-completion
• Change in prisoner status (e.g., mains to VP).
• Removed from course due to behaviour
• Transferred to another prison
• Released
• Deselected from course
RCT in the Social Sciences:
Challenges and Prospects
16
Summary
• Despite advantages of RCT, few have been
conducted.
• Acknowledgement of the practical and ethical
implications in a forensic setting
• BUT  333 individuals have been randomly
allocated so far…. one of the largest trials to be
conducted
RCT in the Social Sciences:
Challenges and Prospects
17
Centre for Criminal Justice
Economics & Psychology
Wentworth College
Heslington
York, YO1O 5DD
E-mail:criminaljustice@cj.york.ac.uk
RCT in the Social Sciences:
Tel: +44 (0) 1904 434880
Challenges and Prospects
18