Judgments and Decisions
Psych 253
• Using Decision Analysis to Answer Questions
about the Value of Getting Information
• Perfect vs. Imperfect Information
• Signal Detection Theory
• Medical Example
• Jury Decision Making
You can select A, B, or C. Below are the payoffs for each
depending on the throw of a die. Which one do you
want to have?
A
1
$1
2
$2
Die Roll
3
4
$3
$4
B
$6
$2
$5
$2
$2
$2
C
$7
$5
$4
$4
$2
$1
What should you choose?
5
$5
6
$6
A
1
$1
2
$2
Die Roll
3
4
$3 $4
B
$6
$2
$5
$2
$2
$2
$3.17
C
$7
$5
$4
$4
$2
$1
$3.83
5
$5
6
$6
EV
$3.50
Which one would you select if you knew what
number would come up on the die?
A
1
$1
2
$2
Die Roll
3
4
$3 $4
B
$6
$2
$5
$2
$2
$2
C
$7
$5
$4
$4
$2
$1
5
$5
6
$6
What is the expected value of knowing the number
that would come up?
1/6*[ $7 + $5 + $5 + $4 + $5 + $6 ] = $5.33
A
1
$1
2
$2
Die Roll
3
4
$3
$4
B
$6
$2
$5
$2
$2
$2
C
$7
$5
$4
$4
$2
$1
5
$5
6
$6
The value of the information is
$5.33 - $3.83 = $1.50
Value of the
decision
WITH
perfect
information
Value of the
decision
WITHOUT
perfect
information
Miracle Movers rents out trucks with a crew of 2
people. One day, Miracle discovers it is a truck
short. When this happens, Miracle rents a truck
from a local firm. Small trucks cost $130 per day,
and large ones cost $200. The advantage of the
small truck may vanish if the crew has to make 2
trips. The extra cost of a second trip is $150, and the
probability of 2 trips is 40%.
Large Truck
$200 $200
P(NL) =.40
Small Truck
Need Large
Truck $280$280
P(NS) =.60 Need Small
Truck $130$130
What would it be worth to Michelle to know for
sure whether she would need a small truck?
Large Truck
$200 $200
P(NL) =.40
$280$280
Small Truck
P(NS) =.60 $130$130
What would it be worth to Michelle to know for
sure whether she would need a small truck?
Large Truck
$200 $200
Small Truck
EV= $190280
What would it be worth to Michelle to know for
sure whether she would need a small truck?
Don’t Collect
Information
Small Truck $190
“NL” P(NL) = .40
Collect
Information “NS”
(that happens
to be perfect)
P(NS) =.60
Need Large Truck
Need Small Truck
Need Small Truck
Need Large Truck
What does Perfect Information Mean?
Collect Perfect
Information
“NL”
P(NL)=.40
Need Large
Truck $200
“NS”
P(NS) =.60
P (“NL”|NL) = 1 and P (“NL”|NS) = 0
P(“NS”|NS) = 1 and P (“NS”|NL ) = 0
Need Small
Truck $130
Value of the decision with perfect information
= p(“NL” and NL)*$200 + p(“NS” and NS)*$130
+ p(“NL” and NS)*$200 + p(“NS” and NL)*$280
But because the information is perfect,
p(“NL” and NS) and p(“NS” and NL) never occur.
So the value of the decision with perfect
information is
= p(“NL” and NL)*$200 + p(“NS” and NS)*$130
What is P(“NL” and NL) and P(“NS” and NS)?
P(“NL” and NL) = P(“L”|NL)*P(NL) = 1.0*(.4) =.4
P(“NS” and NS) = P(“S”|NS)*P(NS) = 1.0*(.6)=.6
Value of the decision with perfect info
= p(“NL” and NL)*$200 + p(“NS” and NS)*$130
= .4*$200 + .6*$130 = $158
Value of the Decision – Value of the Decision
with perfect info = Value of the Information
Value of the information = $190 - $158 = $32
Perfect information means this…
Information
“S”
“L”
NS P(“S”|NS) =1 P(“L”|NS)=0
Actual
NL P(“S”|NL) =0 P(“L”|NL) =1
If the information is imperfect, we need to
quantify that uncertainty.
Information
“S”
“L”
NS P(“S”|NS) =.9 P(“L”|NS)=.1
Actual
NL P(“S”|NL)=.2 P(“L”|NL) =.8
Signal Detection Theory:
A Theory of Repeated Decision Making
when Information is Fuzzy
-deciding which customers would default on
a loan
-deciding whether to hire an employee
-determining whether someone is lying or
not
-and any other decision that is made
repeatedly using ambiguous evidence
Imagine a doctor who treats patients for a
disease. A test for the number of goodies in the
patient’s blood is the only predictor or clue he
has to predict whether or not the patient has
the disease.
From past research, the doctor knows that
goodies are normally distributed in the blood
of Healthy and Sick people. The average # of
goodies in the blood of Healthy people is 100,
and the average number of goodies in the
blood of Sick people is 115.
0.45
Healthy
0.4
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability of Disease
0.35
We can analyze the doctor’s ability to diagnose a
patients with Signal Detection Theory. This theory
has two parameters:
d´ = d prime - the ability to distinguish Healthy people
from Sick people with the blood test
B = beta (also called the threshold or cutoff) - the
doctor’s tendency to call a patient “Healthy” or
“Sick”
0.45
d’
0.4
Healthy
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability of Disease
0.35
“Healthy”
B
“Sick”
0.45
0.4
Healthy
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability of Disease
0.35
No matter where the doctor sets his cutoff,
there are two errors and two correct decisions.
Decision
“Healthy” “Sick”
State of Healthy
P(“H”|H) P(“S”|H)
the World
Sick P(“H”|S) P(“S”|S)
Each outcome has a name.
Decision
“Healthy” “Sick”
Correct
State of Healthy Rejection
the World
Sick
Miss
False
Alarm
Hit
“Healthy”
B
“Sick”
0.45
0.4
Healthy
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability of Disease
0.35
Which error is worse? It depends.
If the decision is whether to treat a patient
who may have cancer, a miss may be worse
than a false alarm.
If the decision is whether to attack an
unidentified plane, a false alarm may be
worse than a miss.
Where should the doctor set his threshold?
B
“Healthy”
“Sick”
0.45
0.45
0.4
0.4
0.35
0.35
0.2
0.15
# Goodies in the Blood
14
5
55
14
5
13
0
12
5
12
0
11
5
0
10
0
0
85
0.05
70
0.05
13
0
0.1
12
5
0.1
0.25
12
0
0.15
Sick
11
5
0.2
“Sick”
Healthy
10
0
0.25
0.3
85
Sick
70
Healthy
Probability of Disease
0.3
55
Probability of Disease
“Healthy”
B
# Goodies in the Blood
As we move B from the left to the right, we decrease P(“S”|H)
(false alarms), but increase P(“H”|S) (misses)
Where shouldthe doctor to put his cutoff (B)?
SDT tells the decision maker where to locate B,
the cutoff, for a given objective. In this sense,
it is a normative theory.
But SDT can also be used to estimate where
someone puts his or her B. In this sense, it is
a descriptive theory.
The optimal location for B depends on the base
rate for the event (in this case, the disease)
and the utilities associated with the four
possible outcomes.
d’ d’
Healthy
Sick
70
55
85
10
0
11
5
13
0
14
5
16
0
17
5
Sick
Healthy
40
25
Probability
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
Values of Y hat
Increasing accuracy requires one to find
variables that strengthen the predictability
of the criterion.
Using those variables in a consistent, statistical
fashion improves predictability.
Using a fixed (and optimal) response threshold
reduces noise further improves
predictability.
Another Example: Jury Decision Making
Probability
“Acquit”
“Convict”
Guilty
Innocent
Perceived Culpability
What are acceptable probabilities of errors?
Convicting the innocent - False Alarm
Acquitting the guilty - Miss
Within this framework, how good must the
juror be to achieve reasonable rates of error ?
Assuming normal distributions and equal variances,
If false alarms and misses are 1%, d’ = 4.7.
If false alarm and miss rates are 5%, d’ = 3.3.
If false alarm and miss rates are 10%, d’= 2.6.
What are common values of d’?
Legal Settings
d’
Detecting liars in a mock crime
Detecting liars with polygraphs
0
0.5 - 1.0
Personnel Selection
d’
Job placement with the Armed
Forces Qualification Test
0.6 - 0.8
Weather Forecasting
d’
Fog-risk in Canberra, Australia
24 hours earlier
18 hours earlier
12 hours earlier
Tornados near Kansas City
Rain in Chicago
Minimum temp in Albuquerque
0.8
1.0
1.2
1.0
1.5
1.7
Medical Settings
d’
Detecting breast cancer w mammograms
Detecting brain lesions with RN Scans
Experts detecting cervical cancer
Detecting cervical cancer with computer
based systems
Detecting prostate cancer with PSA tests
Detecting brain lesions with CT Scans
1.3
1.4 - 1.7
1.6
1.8
2.0
2.4 - 2.9
Signal detection theory and empirical estimates
of d’ (with all possible technological advances)
tells us we are not achieving errors that are even
remotely desirable.
What might this imply for policy?