Nudge Meets RCT:
Evidence-based
Manipulation for the Public
Good?
Martyn Hammersley
The Open University
Workshop on:
Bio-Social Methods for a Vitalist Social Science
July 2013
The Nudge Unit
‘The Behavioural Insights Team, often called
the “Nudge Unit”, applies insights from
academic research in behavioural economics
and psychology to public policy and services.’
One of its ‘responsibilities’ is listed as:
‘championing scientific methodology to bring
greater rigour to policy evaluation.’
This, of course, is where randomised controlled
trials (RCTs) come in.
(https://www.gov.uk/government/organisations/
behavioural-insights-team)
Nudge Nudge, Wink Wink
‘The team's work has already led to an extra
£200m in income tax being collected by
telling late payers on tax forms that most
people in their towns had already paid their
tax. This increased payment rates by 15
percentage points. Now the team is
applying lessons learned from increasing
voter turnout in US elections to get Essex
jobseekers into work 15%-20% quicker than
traditional methods’ (Benjamin 5.2.13)
The appliance of behavioural
science?
• What is proposed here is the application of
academic knowledge, both theoretical and
methodological, to policymaking.
• But what is being applied is only a narrow range
of academic knowledge. And it aims at
challenging the already established approaches
of other disciplines: political science, sociology,
social policy research, and evaluation research.
• It forms part of a broader process of territorial
colonisation by psychology and economics.
Assumptions about the relationship
between research and policymaking
• ‘Libertarian paternalism’: A focus on nudging
policies, rather than on coercion, rational
persuasion, rhetoric and spin, or laissez-faire.
• Policymaking is or should be ‘evidence-based’
• The task of research is to determine ‘what
works’, this indicating relatively conclusively
what policies should, and should not, be
pursued.
• It is claimed that the only reliable means for
determining this is the RCT.
The opening pitch of
‘Test, Learn, Adapt’
• ‘Randomised controlled trials (RCTs)
are the best way of determining whether
a policy is working. […]’
• ‘RCTs are not routinely used to test the
effectiveness of public policy
interventions in the UK. We think that
they should be’ (Haynes et al 2012:4)
Focus on ‘What Works?’
Effectiveness and efficiency are important, but
they are not the whole of policymaking.
Equally, if not more, important, both in
principle and in practice, are:
• The identification, prioritising, and diagnosis
of policy problems
• Evaluating policy proposals as regards ethics,
side-effects, and the distribution of benefits
and costs among those involved.
• Politics in the narrow sense: how will this play
with the Daily Mail? Will this wrong-foot
(nudge?) the Opposition?
Nudging people’s choices
• Origins in techniques of ‘demand management’
and HR employed by commercial organisations,
often informed by psychology.
• ‘We argue for self-conscious efforts, by
institutions in the private sector and also by
government, to steer people’s choices in
directions that will improve their lives’. The aim
is ‘to influence choices in a way that will make
choosers better off, as judged by themselves’
(Thaler and Sunstein 2009:5)
• Some Nudge Unit projects aim at the common
good, rather than that of the nudged individuals.
Problems with ‘the common
good’
• Is there any such thing: common to whom,
and good by what criteria? Some sense can
be made of this notion, but it is always going
to be a contentious matter.
• Does the state currently operate so as to
serve the ‘common good’? Are there
preconditions that must be met if it is even to
approximate this mode of functioning?
• Can democracy deliver this? Do we live in a
democracy?
Research-Based Policymaking?
• Evidence is not all that is required in
policymaking: it needs to be a process of
phronesis, in other words wise judgment.
• Research is not the only source of relevant
evidence.
• RCTs are not the only source of valuable
research evidence
• The provision of evidence about effectiveness
or efficiency is not the only contribution that
social science can make to policymaking
(Hammersley 2013)
Making the case for RCTs
‘What makes RCTs different from other types
of evaluation is the introduction of a
randomly assigned control group, which
enables you to compare the effectiveness of
a new intervention against what would have
happened if you had changed nothing’
(Haynes et al 2012:4).
As I will show, this is at best only partly true.
Moreover, RCTs are not applicable to all
policies: these must involve specific and
standardisable ‘treatments’.
A germ of truth
‘The introduction of a control group eliminates
[…] biases that normally complicate the
evaluation process […]: if you introduce a new
“back to work” scheme, how will you know
whether those receiving […] extra support
might not have found a job anyway?’ (p4)
By randomly allocating units to treatment and
control groups, the chances of background
differences between the groups affecting the
outcome are minimised, and we can estimate
statistically the chances that these background
factors produced any outcome difference.
But
• Randomisation does not eliminate all possibility
of background factors affecting the outcome,
nor does it deal with all potential sources of
error: measurement errors, expectation effects.
• While the internal validity of RCTs is generally
relatively high, there are problems with external
validity: the ‘treatment’ rolled out is not always
the same as in the trial, and the people and
situations often differ in causally significant
ways (Cartwright and Hardie 2012).
• The results of RCTs will often have a short
shelf-life, as may the effects of policy nudges.
The positioning of the ‘subject’
• An(other) attack on the rational model: homo
economicus is a myth. Perhaps we knew
that?
• The behaviour of subjects is largely a product
of ‘the automatic system’ (Thaler and
Sunstein 2009): they know what is good for
them but routinely fail to act in ways that are
effective in achieving it.
• The task is to nudge the defective citizen into
behaving in ways that are best for everyone.
The positioning of policymakers
• They know what is in the best interests
of ‘subjects’.
• They are motivated to ‘nudge’ subjects
in ways that serve those interests.
• They can be provided with technical
means that have a demonstrable
capacity to achieve this.
The positioning of the behavioural
scientist
‘The unit tries to nudge people into leading
better lives, and save the rest of us a
fortune. It is politics done like science’
(Benedictus 2013)
Ellieban: ‘I confess to being a bit of a fangirl,
but the behavioural insights team is nonideological and non-partisan, and exists
purely to help government (any
government) make policy that actually
works. […] They are scientists, not
politicians or propagandists.’
Conclusion
• The Behavioural Insights Unit combines
commitment to government-by-nudge with a
commitment to evidence-based
policymaking. It promotes itself by
missionary rhetoric that exaggerates what
both nudging and RCTs can do.
• It misleadingly positions both itself and
policymakers as politically neutral, and it
promotes a narrow and overly direct model
of the general relationship between
research and policymaking.
Conclusion concluded
• There is undoubtedly a role for nudging
policies, and for RCTs to test them, even if
this is quite limited.
• Some of us may want to believe that there
is a clearly defined and easily identifiable
common good, and that we are ruled by a
benevolent and competent regime that can
realise this good simply by nudging us to do
what we know is in our own best interests.
However, unfortunately, this fairy tale, like
others, is false.
References
• Benedictus, L. (2013) ‘The nudge unit: has it worked so far?’, The
Guardian, Thurs 2 May. Available at:
http://www.guardian.co.uk/politics/2013/may/02/nudge-unit-has-itworked
• Benjamin, A. (2013) ‘David Halpern: We try to avoid legislation and
ordering’, The Guardian, 5.2.13. Available at:
http://www.guardian.co.uk/society/2013/feb/05/david-halperngovernment-nudge-unit
• Ellieban ‘Comment on Benjamin’, see above.
• Cartwright, N. and Hardie, J. (2012) Evidence-Based Policy, Oxford,
Oxford University Press.
• Hammersley, M. (2013) The Myth of Research-Based Policy and
Practice, London, Sage.
• Haynes, L., Service, O., Goldacre, B., and Torgerson, D. (2012) Test,
Learn, Adapt: Developing Public Policy with Randomised Controlled
Trials, London, Behavioural Insights Team, Cabinet Office, UK
Government.