Do Judges Really Think
That Way?
Chris Guthrie
Dean
John Wade-Kent Syverud Professor of Law
Vanderbilt Law School
chris.guthrie@vanderbilt.edu
Project

Collaboration involving:
 Jeff
Rachlinski, Cornell Law School; and
 Andrew Wistrich, Central District of California

Goal:
 To

explore how trial judges judge
Method:
 Experimental

– mostly “between-group” studies
Data:
 responses
to hypotheticals from thousands of federal
and state trial judges, bankruptcy judges, and ALJs
Judges and Judging

Ordinary courts in U.S. resolve 35 million cases
a year
 98%

by trial courts
Judges (30,000+) are the key actors
 Judges
preside
 Many more bench trials than jury trials
 Many more decisions on motion than trials
 Judges play a role in many settlements

Dominant academic accounts of judging:
 Formalist
 Realist
“Pure” Formalist Model
“Pure formalists view the judicial system as if it
were a giant syllogism machine with a
determinate, externally-mandated legal rule
supplying the major premise, and objectively
‘true’ pre-existing facts providing the minor
premise. The judge’s job is to act as a highly
skilled mechanic….”
- Burt Neuborne (1992)
“Pure” Realist Model
“[T]he judge really decides by feeling, and not by
judgment; by ‘hunching’ and not by [reason] . . .
[T]he astute judge, having so decided, enlists his
every faculty and belabors his laggard mind, not
only to justify that intuition to himself, but to
make it pass muster . . .”
- Joseph Hutcheson (1929)
The Intuitive-Override Model

Dual Process Model
 Intuitive

& Deliberative
Trial judges rely heavily on intuition but can, and
sometimes do, override with deliberation
 Stimulus

 Intuitive Response  Override?
Evidence:
 Cognitive
Reflection Test
 Judicial Decision Making studies


“Heuristics & Biases”
Deliberately Disregarding
Cognitive Reflection
Test
Cognitive Reflection Test

Three items
 Each
has intuitive, but inaccurate, answer
 Correct answer is easy, if respondent uses
deliberative, rational process

Thus, CRT designed to compare “intuitive” and
“deliberative” thought processes
CRT Item #1

A bat and a ball cost $1.10 in total. The bat
costs $1.00 more than the ball. How much does
the ball cost?
Item #1 Answer

A bat and a ball cost $1.10 in total. The bat
costs $1.00 more than the ball. How much does
the ball cost?
 Wrong,

But if the ball costs 10 cents, this means the bat costs $1.10,
and the total cost would equal $1.20, not $1.10 as specified
 Correct



but intuitive, answer = 10 cents
answer = 5 cents
i.e., Ball = 5 cents
i.e., Bat = $1.05
Total = $1.10
CRT Item #2

If it takes 5 machines 5 minutes to make 5
widgets, how long would it take 100 machines to
make 100 widgets?
Item #2 Answer

If it takes 5 machines 5 minutes to make 5
widgets, how long would it take 100 machines to
make 100 widgets?
 Wrong,

but intuitive, answer = 100 minutes
But if there are 20 times as many machines (i.e., 100), they
should produce 20 times as many widgets (i.e., 100) in the
same period of time (i.e., 1 per every five minutes)
 Correct
answer = 5 minutes
CRT Item #3

In a lake, there is a patch of lily pads. Every
day, the patch doubles in size. If it takes 48
days for the patch to cover the entire lake, how
long would it take for the patch to cover half of
the lake?
Item #3 Answer

In a lake, there is a patch of lily pads. Every
day, the patch doubles in size. If it takes 48
days for the patch to cover the entire lake, how
long would it take for the patch to cover half of
the lake?
 Wrong,

but intuitive, answer = 24 days
But if the lake is half covered in 24 days, and then doubles, it
will be fully covered in 25 (rather than 48) days.
 Correct
answer = 47 days
CRT Results
Average Scores
MIT students
Harvard students
ALJs
Florida state judges
Michigan students
Web-based participants
Michigan State students
Toledo students
2.18
1.43
1.33
1.23
1.13
1.10
0.79
0.57
CRT Results
Judges
Sample size
Average score
%age 0 correct
%age 1 correct
%age 2 correct
%age 3 correct
252
1.23
30.6
31.1
23.8
14.7
CRT Results
Judges (cont’d)
Question
#1
#2
#3
% right
28
44
50
If wrong,
% selecting intuitive
97
57
70
CRT:
Basic Findings

Even though the questions are easy, most
judges get most of them wrong

The wrong answer most commonly chosen
tends to be the intuitive one

Those who get a problem wrong tend to think it
is an easier problem than those who get it right

But some people (including 1/7th of the judges)
overcome their intuitive responses
Judicial Decision
Making Studies
Heuristics & Biases

Cognitive shortcuts or intuitive “rules of thumb”
 Operate
quickly and unconsciously
 Often produce good outcomes, but can lead to
systematic and predictable errors (“cognitive
illusions”)
 e.g., anchoring, framing, hindsight bias
Anchoring
Estimates tend to be biased toward the first
number we encounter
Anchoring Problem #1
Facts






P is a 31-year-old teacher and father
D is a package-delivery company
P loses his arm and breaks three ribs in auto accident –
pain, work problems, recreational limitations
D concedes liability and economic damages, but
contests pain and suffering damages
Unsuccessful settlement conference
Experimental group judges only:


Plaintiff’s lawyer demands $10 million to settle
Question – What pain and suffering damages will you
award?
Anchoring Problem #1
Results
Mean
Award
Median
Award
Control
$808,000
$700,000
Anchor
$2,210,000 $1,000,000
Anchoring Problem #2
Facts







Employment discrimination case
P was subjected to racially/ethnically offensive conduct;
complained; and was fired
P provides credible evidence of “anxiety, sleeplessness,
and bad dreams”
D concedes liability but contests compensatory damages
for “mental anguish”
Control Group – P saw a Court TV program where
victim received award
Anchor Group – P saw a Court TV program where
victim received $215,000 award
Question – What mental anguish damages will you
award?
Anchoring Problem #2
Results
Mean
Award
Median
Award
Control
$35,000
$6,000
Anchor
$58,000
$50,000
Framing/Loss Aversion



We tend to perceive numeric options as “gains”
or “losses”
Losses are more unattractive than gains are
attractive
The same outcome seems much more unfair if it
appears to be a loss rather than a gain
Profit and Pay
Kahneman, Knetsch & Thaler (1986)
Facts & Results

A company is making a small profit during a
recession and period of high unemployment.
 “Gain”
version – The company increases wages 5%
in a 12% inflation environment
 “Loss” version – Company decreases wages 7% in a
no-inflation environment

Is this fair?
 “Gain”
version – 78% fair
 “Loss” version – 38% fair
Landlord-Tenant Case (ALJs)
Facts & Results

Landlord files claim against tenant for failure to
pay rent
 “Gain”/Discount
– $2100 rent and $50 discount for
tenant if he pays cash (from $2100 to $2050)
 “Loss”/Surcharge – $2000 rent and $50 surcharge for
tenant if he pays credit card (from $2000 to $2050)

Is this fair?
 “Gain”/Discount
version – 95% fair
 “Loss”/Surcharge version – 52% fair
Hindsight Bias
People overestimate the predictability of events
that have already occurred
 a.k.a.
“20/20 hindsight” or “Monday morning
quarterbacking”
Hindsight Bias/Juries
Kamin & Rachlinski (1995)
Subjects receive identical information about the
likelihood of a flood causing damage
group (i.e., pre-accident) – Should D take
a precaution? (24% said “yes”)
 Hindsight group (i.e., post-accident) – Was D
negligent for not taking the precaution? (57% said
“yes”)
 Foresight
Hindsight Bias & Judging Problem
(ALJs)
Facts & Results




Five kids making trouble in a toy store
Guard tracks one African-American kid and thinks he
sees him shoplift
Guard arrests the kid, perhaps using racial slurs
Two versions:



Innocent version – No shoplifting
Guilty version – Shoplifting
Was the arrest racially motivated?


Innocent version – 68% yes
Guilty version – 29% yes
“Difficulty Disregarding”

Difficult to ignore relevant but inadmissible
evidence
 Many
mock jury studies support this
Difficulty Disregarding Problem #1
Anchoring Problem #1 from Above

Pain and suffering award influenced by $10
million demand made during settlement
conference
 Under
FRE 408, this is inadmissible
Difficulty Disregarding Problem #2
Landsman & Rakos (1994)
Facts




While attempting to burn leaves, P is burned due to
“flashback” from gasoline container.
P sues D manufacturer, claiming that the container was
defective because it did not have a “flame arrester”
D claims that “flame arrester” would not have prevented
injury
Experimental group judges only:


P seeks to introduce evidence of a “subsequent remedial
measure” taken by D – a warning and recall letter re: the
possibility of fire flashback. The prior judge who heard the
motion suppressed the evidence under Rule 407.
Question – Will you rule for P or D?
Difficulty Disregarding Problem #2
Landsman & Rakos (1994)
Results
%age ruling for P
Control (28)
0%
Experimental (28) 25%
Difficulty Disregarding Problem #3
Facts


Criminal prosecution of date rape at fraternity
party; jury trial waived
Only issue is whether victim consented




Defendant testified she did, but victim denied it
Evidence corroborating lack of consent (bruising,
rapid reporting, emotional distress)
Experimental group only – Defendant seeks to
introduce testimony about victim’s sexual
promiscuity. Will you admit evidence?
Question – Will you convict?
Difficulty Disregarding Problem #3
Results
%age convicting
Control
49%
Experimental (sexual
history but excluded)
20%
Judicial Decision Making Problems:
Basic Findings

Most judges are influenced by heuristics that
induce intuitive responses and are unable to
disregard relevant but inadmissible evidence
Litigators?






If you want intuitive thinking, seek rapid
decisions; if you want deliberative thinking, seek
the opposite
Use anchors
To make desired options more appealing, frame
Think carefully about managing settlement
negotiations in front of a decision-making judge
Try to introduce probative but potentially
inadmissible evidence
Consider potential advantages of juries
Reforms?


Accuracy critical, but benefits of enhanced
accuracy must be balanced against costs
Time?
 e.g.,
Docket reductions
 e.g., motions in limine versus at trial



Training/feedback?
More frequent opinion writing?
Divided decision making?
 e.g.,

“managerial” judge vs. “trial” judge
Group decision making?
 e.g.,
juries vs. judges; panels vs. individuals