Myths, Misunderstandings & Mistakes
of Jury Research
UCL Jury Project:
2007: Diversity & Fairness in the Jury System
Jury summoning system – who is summoned, who serves,
are jurors representative, does system discriminate?
2010: Are Juries Fair?
Jury decision-making - Do juries discriminate against
minorities? Jury conviction rates. Do jurors understand
judicial directions? Juror internet use and impact of
media coverage of trials.
3 Main Research Methods
(1) Case simulation
(2) Large-scale data analysis of verdicts
(2) Surveys/interviews with jurors
Best research uses all 3 – triangulation
Case simulation research
Study run only with real juries at Crown Courts
Large number of juries see an identical case
Defendant charged with violent crime (assault)
Only difference is race of defendant
Large-scale verdict analysis
Correlational studies - analysing all jury verdicts in all courts
in England & Wales over 2 year period (>½ million charges)
Jury conviction rate
67%
63%
White
Black
63%
64%
65%
Asian
Other
Not Known
Defendant ethnic group
Juror surveys & interviews
Immediate post-verdict jury studies at court:
• Juror awareness of media coverage of cases
• Juror use of internet during trial
• Juror views of deliberation process
Conducting Reliable
Jury Research
Reliable research about jury system in UK has been
seriously hampered by a number of
• Myths
• Misunderstandings
• Mistakes
Jury Research Myths
Myths about what can and cannot be explored
with real juries have affected both the jury policy
and research agenda
Jury Research
Misunderstandings
Misunderstandings about how to address
specific jury questions have led to misguided
“jury research”
Jury Research Mistakes
Mistakes about how to properly use jury research
methodologies have produced questionable if
not dangerous research findings
Myth of Section 8
S8, Contempt of Court Act 1981: it is a criminal offence to:
“Obtain, disclose or solicit any particulars of statements made,
opinions expressed, arguments advanced or votes cast by members
of a jury in the course of their deliberations”
Impact of s.8 Myth
One empirical study with juries in England & Wales
1981- 2007
What filled the black hole?
Research from other
jurisdictions
Assumed our juries behave in same way
Fundamental error
• Crucial differences in jury & social systems
• Research on race and juries revealed distinct differences
between US juries and juries in England and Wales
• Research from other jurisdictions can be valuable – but mainly
in terms of research design and methodology
• Each jurisdiction needs to conduct its own research
High profile cases
In absence of reliable evidence here,
policy agenda for reform of jury trials
dominated by two elements.
(1) High profile jury trials where
something went fundamentally wrong
Create demand for reform … but
invariably without answering 2 key
questions with empirical evidence:
1. How widespread is the problem?
2. What is most effective method of addressing the
problem?
Professional Anecdote
Also setting policy agenda and
dominating beliefs about juries
• Those who work in the criminal courts can develop
strong views about juries
• Professionals’ personal experiences can be
extremely helpful in background research
• But they cannot provide reliable empirical
evidence
• Their perceptions need to be tested objectively.
What is the state of jury research?
Hardcore approach to jury research
At best misconceivedat worst dangerous
Most “jury research” is in fact:
• not done with actual jurors
• not done with authentic and complete case
materials
• not conducted at the jury verdict level
• not conducted with large enough or
representative sample sizes to generate reliable
conclusions
Common Methodological Errors
Actual case analysis
• Insufficient sample sizes
• esp multi-variable analysis
Surveys/Interviews
• Unrepresentative samples
• Low response rates
• Over-reliance on volunteers
• On-line surveys
The self-selection problem
Temptation of case simulation
Relatively easy now to do some version of “mock jury”
research
But not easy to do this research so it produces reliable
conclusions about how juries decide cases
Common methodological errors
in case simulations
Participants:
Myth of s.8 led to over-reliance on
“proxy jurors”
• Students ≠ Jurors
• Volunteers ≠ Jurors
Failure to consider unique jury environment
Materials must be:
• Authentic
• Complete
• Capable of precise controlled testing of variables
• Capable of large-scale repetition
The “acting out” problem
Juror decisions ≠ Jury verdicts
Most case simulation research looks only at individual juror
decisions not jury verdicts
Not surprising – jury verdict research is much more time
consuming and complicated
But it is not a case of 12 individual decisions = jury verdict
Dangerous to extrapolate what verdicts will be from
individual decisions
In real world of criminal trial only verdict counts
Sample sizes
There is no set sample size for case simulation studies
• Sample size will depend on the number of variables
being examined and case variations.
• But substantial numbers of full juries are necessary
Some research has drawn major conclusions based
on only 1 jury verdict per variable/variation.
• This is highly unreliable and can be dangerously
misleading.
• Especially important when issues addressed in the
research involve important policy issues.
Most researchers looking
only to discover problems
No attempt to test solutions
“Merely to explain and understand is to
fiddle while Rome burns.” Ronald V. Clarke
Responsible jury research
Responsibility in carrying out jury research to insure:
1. Findings are not over-estimated
2. Original research design includes plans to test
solutions to problems if found
Danger of being in
the dark about juries
These are not just narrow academic points about jury
research methodology – real world impact
• Significant challenges facing trial by jury in 21st century
… including major criminal law reforms based on little
more than assumptions about how juries work
• In the existing information vacuum about juries, it is easy
for any research which purports to be about juries to be
seized upon to suit different policy agendas.
• It will also almost inevitably attract media attention.
Important questions in need
of hardcore jury research
Evidence: complex, forensic, fraud, remote, virtual
• Do juries understand complex evidence?
• Do juries really defer to experts?
• What is best way of presenting complex evidence to jury to
ensure it is understood?
• Is there a CSI effect?
• How does visual presentation affect jurors?
• How do jurors perceive “virtual” evidence?
Future of trial by jury?
Should be determined by properly conducted research
with real jurors that produces reliable evidence.
UCL Jury Project current research projects
Preventing improper internet use (ESRC)
Improving jury deliberations (ESRC)
Impact of special measures (Nuffield)
Judicial directions on law (Nuffield)
Insanity defence & fitness to plead (Law Comm)
Questions?