Finish: History of JDM
Begin: Linear Judgment Models
Psychology 466: Judgment & Decision Making
Instructor: John Miyamoto
10/06/2015: Lecture 02-2
Note: This Powerpoint presentation may contain macros that I wrote to help me
create the slides. The macros aren’t needed to view the slides. You can disable or
delete the macros without any change to the presentation.
Outline
• Brief history of the psychology of decision making
• Capacity limitations in human cognitive processes
• Four linear judgment models
♦
Why are psychologists interested in linear judgment models?
♦
How can we make decisions based on a linear judgment model?
Psych 466, Miyamoto, Aut '15
History of Psych of Decision Making
2
History of the Psychology of Decision Making
• Victorian rationality, Freudian irrationality,
behaviorist arationality.
• Expected utility theory (Von Neumann & Morgenstern, 1944)
Rational agent model of economic behavior
• Heuristics and biases movement, 1970 – 1990 (approx.)
• Reactions to heuristics and biases movement
Evolutionary psychology, ecological psychology, naturalistic decision making,
Bayesian models of psychological processes
• Psychology of happiness
• Separate development – neuroscience of decision making
(current hot topic!)
Psych 466, Miyamoto, Aut '15
The Cognitive Approach to Judgment & Decision Making
3
The Cognitive Approach to Judgment & Decision Making
• Cognitive limitations – limitations on human cognitive capacity
affect judgment and decision making
• Heuristics and biases movement: 1970 – 1990 (approx.)
• Reactions to heuristics and biases movement
♦
Evolutionary psychology
♦
Ecological psychology
♦
Naturalistic decision making
♦
Bayesian models of psychological processes
♦
Emotion in decision processes
Psych 466, Miyamoto, Aut '15
The Standard Memory Model
4
The Standard Cognitive Model of Human Memory
H&D
Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological
Buffer
Goal
Stack
Visuospatial
Buffer
Long-Term Memory
Psych 466, Miyamoto, Aut '15
Sensory Registers
5
H&D
Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological
Buffer
Goal
Stack
Visuospatial
Buffer
Long-Term Memory
• Sensory registers retain the sensory information for very brief
periods of time.
Psych 466, Miyamoto, Aut '15
Working Memory
6
H&D
Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological
Buffer
Goal
Stack
Visuospatial
Buffer
Long-Term Memory
• Working memory (WM) holds a limited amount of information
for 10 – 20 seconds. Thoughts are actively manipulated in WM.
Psych 466, Miyamoto, Aut '15
Long-Term Memory
7
H&D
Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological
Buffer
Goal
Stack
Visuospatial
Buffer
Long-Term Memory
• Long-term memory (LTM) retains information over longer periods
of time. LTM interacts with WM.
Psych 466, Miyamoto, Aut '15
General Hypothesis of Cognitive Research
8
H&D
Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological
Buffer
Goal
Stack
Visuospatial
Buffer
Long-Term Memory
General Hypothesis of Cognitive Research
♦
Limitations in working memory impose limitations on human ability
to engage in complex reasoning.
♦
Decision making requires complex reasoning.
Psych 466, Miyamoto, Aut '15
Basic Message: Cognitive Limitations Produce Simplifications
9
Working Memory (WM) Has Severe Capacity Limitations
• WM can only hold a limited number of "chunks" of information.
Information is lost from WM fairly quickly (within ~20 sec.)
• When information is complex, people are forced to simplify
the reasoning process.
• Simplifications can lead to distortions.
• EXCEPTION: Experience can teach one to integrate specific
types of complex information but only in some cases.
♦
Example: Expert chess players can reason about complicated chess
problems.
♦
Example: Experienced drivers can understand traffic situations that are
actually very complex.
Psych 466, Miyamoto, Aut '15
Same Slide but with Additional Comment re the Role of Attention
10
Working Memory (WM) Has Severe Capacity Limitations
• WM can only hold a limited number of "chunks" of information.
Information is lost from WM fairly quickly (within ~20 sec.)
• When information is complex, people are forced to simplify
the reasoning process.
• Simplifications can lead to distortions.
• Limited WM implies that attention plays a central role in
cognition.
♦
Misallocation of attention can lead one to overlook important
relevant info.
♦
Complex information processes have many points at which
cognitive limitations can exert an influence.
Psych 466, Miyamoto, Aut '15
Where Are We in this Lecture?
11
Where Are We in the Lecture?
• Normative and prescriptive decision models
require complex representations and processing
• Cognitive limitations cause us to simplify decisions,
and this can produce errors
NEXT: How to Deal with Cognitive Complexity
• Intuitive clinical judgment versus statistical Models
• Brunswik’s Lens Model of Human Judgment
• Linear models applied to making better choices
• Applications to clinical judgment
Psych 466, Miyamoto, Aut '15
Clinical vs Actuarial Jdmt
12
Intuitive Judgment versus Acturial Judgment
• Intuitive judgment
♦
♦
Combine complex information in your head
Make decision based on gut feeling
• Actuarial judgment (a.k.a. statistical model or linear model)
♦
Base decisions on a statistical decision rule.
• Intellectual warfare between cognitive psychology and
clinical psychology. (Especially in 1950's - 1970's).
Psych 466, Miyamoto, Aut '15
Statistical Models Outperform Human Judges
13
Examples of Judgment Problems
• We will only consider decisions to which intuitive judgment and
actuarial judgment (statistical methods) both apply.
-------------------------------------------------------------------------E.g., Clinicians attempt to identify patients with progressive
brain dysfunction.
♦
♦
♦
Data = intellectual test results
Experienced clinicians achieved 58% correct detection of new cases.
Statistical model achieved 83% correct detection of new cases.
• E.g., Bank loan officer must decide which loan applications are
“good risks” and which are “bad risks.”
• E.g., Professors must decide which applicants will do well in
grad school and which will not do well.
Psych 466, Miyamoto, Aut '15
Critique of Clinical Judgment – What Is It?
14
Critique of Clinical Judgment
• Clinical insight – does it exist?
• Clinical judgment – what is it good for?
• Clinical judgment – what are its weaknesses?
• Accusation: Belief in the efficacy of intuitive clinical judgment
is a cognitive conceit.
Psych 466, Miyamoto, Aut '15
General Finding: Stat Models Outperform Human Judges
15
General Finding: Stat Models Outperform Human Judges
• Statistical models almost always outperform the human judges
on clearly defined decision tasks.
• Human cognitive processes are good at noticing particular pieces
of information.
♦
Does my friend look happy? Sad? Stressed? Irritated?
♦
Is the patient nervous? Defensive? Exhibitionistic?
• Human cognitive processes are not good at integrating
multiple pieces of information.
♦
Can I predict how my friend will feel about a surprise party?
♦
Can the clinician predict how the patient will progress after 4 months
of therapy?
Psych 466, Miyamoto, Aut '15
Implications of this Lecture / END
16
Implications of this Topic
We can improve human decisions by stressing what humans
are good at:
... noticing what are important issues that are relevant to
a decision;
... evaluating how good or bad is an outcome on a specific
dimension;
while avoiding what we are not good at:
... combining complex information in our heads.
• Know thyself → Make better decisions
Psych 466, Miyamoto, Aut '15
Brunswik's Lens Model
17
The Lens Model of Egon Brunswik
To-be-judged
criterion = the thing
you are trying
to predict.
Figure 3.1 of Hastie & Dawes.
Judgment = the
“judge’s” prediction
(you are the judge)
Cues = things you can
observe about the
criterion
Psych 466, Miyamoto, Aut '15
The lens model is a conceptualization of
the structure of typical judgment problems.
Examples of Judgment Problems: Explain Idea of Criterion, Cues & Judgment
18
Examples of Judgment Problems
Psych 466, Miyamoto, Aut '15
Brief Digression: Definition of Holistic Judgment
19
Holistic Judgment
• Holistic judgment - judgment of a complex trait from multiple
cues by means of a single intuitive judgment as opposed to a
calculation based on a formula.
Psych 466, Miyamoto, Aut '15
Example of Holistic Judgment
20
Example of Holistic Judgment
• Suppose a clinician has to decide whether a patient is suicidal
and should be hospitalized. He considers patient’s appearance,
what patient says, the patient’s background and record, previous
interactions, etc. Eventually clinician makes a decision based
on an intuitive integration of all this information.
------------------------------------------------------------------------• Note: Holistic judgment may include stages in which the
judge considers component features of the decision.
• What makes it a holistic judgment is that the ultimate
evaluation is made by the judge through an intuitive
integration of all of the information about the decision.
Psych 466, Miyamoto, Aut '15
Digression on the Meaning of “Holism”
21
Holism
• Holistic judgments are sometimes called global judgments to
distinguish them from judgments that evaluate the separate
cues.
“Holism” is sometimes written as “wholism.”
• Holistic judgment strategies are contrasted with
analytical judgment strategies.
Analytical judgment strategy: Break the decision into
component parts; use an explicit calculation to combine
these parts.
Psych 466, Miyamoto, Aut '15
Claims Regarding Holistic & Analytical Judgment Strategies
22
Claims Regarding Holistic & Analytical Judgment Strategies
• Claim: An analytical judgment strategy will generally produce
better predictions than a holistic judgment strategy.
♦
E.g., clinical judgments are more often correct if they are made analytically
than if they are made holistically.
♦
E.g., your own predictions of what will happen in sports events, on the
stock market, and predictions about social behavior will be more often
correct if they are made analytically than if they are made holistically.
• Related Claim: An analytic judgment strategy is better than an
intuitive judgment strategy because it provides a better way
to combine complex information.
Psych 466, Miyamoto, Aut '15
Analytical versus Intuitive Judgment Strategy (cont.)
23
Do Analytical Judgment Strategies
Exclude Human Intuition?
NO!
• A human judge is still needed to ...
... decide what are the important issues in a decision;
... judge how good or bad an outcome would be on a
particular dimension,
♦
e.g., social impact of a decision, health impact of a decision,
political impact of a decision, etc.
Psych 466, Miyamoto, Aut '15
Reminder: Standard Model of Human Memory
24
H&D
Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological
Buffer
Goal
Stack
Visuospatial
Buffer
Long-Term Memory
(Reminder) General Hypothesis of Cognitive Research
♦
Limitations in working memory impose limitations on human ability to
engage in complex reasoning.
♦
Decision making requires complex reasoning.
Psych 466, Miyamoto, Aut '15
Four Linear Judgment Models
25
Four Linear Judgment Models
1. Using multiple regression on objective data for which
the true state is known.
2. Using multiple regression on judgment data where
the true state is not known.
3. SMART (Simple Multi-Attribute Rating Technique)
4. Unit Weighting Model
-------------------------------------------♦
Except for Model 4 (unit weighting), the models do NOT describe
everyday judgment.
♦
Some natural models are similar to these models.
♦
Using any of these models can improve human performance.
Psych 466, Miyamoto, Aut '15
Same Slide with Expanded Comments about Each Model
26
Four Linear Judgment Models
1. Using multiple regression on objective data for which
the true state is known.
• This
method produces a proper linear model that is statistically
the best way to generate accurate predictions.
2. Using multiple regression on judgment data where
the true state is not known.
• This
method produces a model of the judge
(abbreviated as MUD or Model of the jUDge)
3. SMART (Simple Multi-Attribute Rating Technique)
♦
This method produces an improper linear model.
4. Unit Weighting Model
•
This method produces an improper linear model.
Psych 466, Miyamoto, Aut '15
Relevance of these Four Models to Assignment 1
27
Relevance of these Four Methods to Assignment 1
Four Linear Judgment Models
1.
Proper Linear Model
2.
Model of the Judge (MUD)
3.
SMART (Simple Multi-Attribute Rating Technique)
(a type of improper linear model)
4.
Unit Weighting (a type of improper linear model)
• Assignment 1: Explain how to use a linear judgment
procedure to choose 1 UW course to take.
♦
♦
♦
Minimize issues of major or minor requirements.
Compare intuitive judgment and linear judgment models.
On Assignment 1, you only need to explain 1 of 4 methods.
Psych 466, Miyamoto, Aut '15
Introduction to Baron’s College Admission Decision Problem
28
Next: Explain These Four Methods
on a Concrete Judgment Problem
• Concrete Problem: Predicting college performance (GPA)
based on information in a college application.
♦
Predictions based on a linear model – how to produce them
♦
Predictions based on a model of the judge – how to produce them
♦
Predictions based on SMART method
♦
Predictions based on unit weights – how to produce them
Assignment 1: Pick one of these four methods. Describe this
method and then discuss its strengths and weaknesses relative to
other methods (including intuitive judgment.
Psych 466, Miyamoto, Aut '15
Continue the Intro to Baron's College Admission Problem
29
Baron's College GPA Judgment Problem
• Using the terminology of the lens model, the GPA judgment
problem looks like this:
Terminology
Example
Criterion
(what we want to predict)
Cues
(these are things we know)
Judgment
(this is our prediction)
Psych 466, Miyamoto, Aut '15
COL = College GPA of high school student
GPA = High school GPA
SAT = SAT test scores
REC = recommendations (converted to ratings)
ESS = essay quality (converted to a rating)
PRE = Estimate (guess) of student’s future
college GPA
Table Showing Data & Variables for the College Admission Example
30
Judgment Data Used in Baron's Chapter 20
COL = college GPA. This is the criterion. This is what the judge wants to predict.
SAT = SAT score; REC = judge's rating of the recommendation; ESS = judge's rating
of the student's essay; GPA = high school GPA. These are the cues.
Psych 466, Miyamoto, Aut '15
Comment re Qualitative Variables
31
Qualitative Variables Must Be Converted
To Quantitative Variables
SAT, GPA are already quantitative.
REC = strength of recommending letters is qualitative; convert to quant
measure by having humans rate the letters for how positive they are.
ESS = quality of applicant’s essay is qualitative; convert to quant
measure by having humans rate the essay for how good it is.
Psych 466, Miyamoto, Aut '15
Four Methods for Predicting Future Cases
32
Four Ways to Compute a Statistical Prediction Model
Method 1: Multiple regression applied to existing data.
Called a “proper linear model”
Next
Method 2: Multiple regression applied to a judge’s predictions
Called a “model of the judge”
Method 3: SMART Method with "importance" weights
Called the SMART method or importance weighting method
Method 4: Unit weighting model
Called the “unit weighting model or unit weighting method”
Psych 466, Miyamoto, Aut '15
Multiple Linear Regression
33
Method 1: Multiple Linear Regression
• Multiple linear regression is a statistical method for finding a
formula that predicts the criterion from a set of data.
COL is the
criterion
Psych 466, Miyamoto, Aut '15
These are the cues
PRE = statistical prediction
(predicted college GPA)
Prediction Equation from the Multiple Regression
34
Prediction Equation with 4 Predictor Variables
COL = the criterion
= college GPA
= to-be-predicted quantity
SAT, REC, ESS, GPA are the
cues (predictor variables)
Prediction Equation
PRE = 0.000175·SAT + 0.092·REC + 0.217·ESS + 1.893·GPA  5.161
The regression weights for the prediction equation are underlined above:
A statistics program can compute the regression weights based on the data
in the table.
Psych 466, Miyamoto, Aut '15
Example of a Regression Equation
35
Prediction Equation with 4 Predictor Variables
COL = the criterion
= to-be-predicted quantity
SAT, REC, ESS, GPA are the
cues (predictor variables)
Prediction Equation
PRE = 0.000175·SAT + 0.092·REC + 0.217·ESS + 1.893·GPA  5.161
Example: If a high school student has
SAT = 1200, REC = 3.7, ESS = 3.9, GPA = 3.2, then we predict
PRE = 0.000175·(1200) + 0.092·(3.7) + 0.217·(3.9) + 1.893·(3.2) – 5.161
= 2.2933
Psych 466, Miyamoto, Aut '15
How to Use the Multiple Regression Model to Predict New Cases
36
How We Use the Linear Regression Model
• Step 1: Apply multiple regression calculation to data for
which the value of the criterion (COL) is known.
This produces the prediction equation.
• Step 2: Use the prediction equation to predict the criterion
(COL) for new cases where the value of the criterion is NOT
known.
Psych 466, Miyamoto, Aut '15
Linear Model Outperforms the Human Judge
37
Linear Model Outperforms Human Judges
• In Baron’s college GPA example, the statistical model makes
more accurate predictions of college GPA than do expert human
judges.
• Same finding on many other examples
(see Dawes, Faust & Meehl paper)
• Is this surprising?
Is it interesting from the standpoint of psychology?
Who cares? Or who might care?
♦
Judges think that they make their judgments by means of a complex,
interactive, nonlinear evaluation of the cues.
♦
Whether or not the judge's introspections are veridical, the results of
these studies show that a much simpler combination of the cues can
produce better predictions than the processes used by human judges.
Psych 466, Miyamoto, Aut '15
Return to Outline of Models
38
Four Ways to Compute a Statistical Prediction Model
Method 1: Multiple regression applied to existing data.
Called a “proper linear model”
Method 2: Multiple regression applied to a judge’s predictions
Next
Called a “model of the judge”
Method 3: SMART Method with "importance" weights
Called the SMART method or importance weighting method
Method 4: Unit weighting model
Called the “unit weighting model or unit weighting method”
Psych 466, Miyamoto, Aut '15
Model of the Judge
39
• The values of the
criterion (COL) are
NOT available.
Not Available
• Researcher has available
the scores for SAT,
REC, ESS and GPA.
Not Available
Method 2: Model of the Judge
• Researcher asks the judge to make intuitive, global predictions
for these cases. This produces the column labeled "JUD”
(next slide).
Psych 466, Miyamoto, Aut '15
Same Slide Except JUD Column Added to Table
40
• The values of the
criterion (COL) are
NOT available.
Not Available
• Researcher has available
the scores for SAT,
REC, ESS and GPA.
Not Available
Method 2: Model of the Judge
• Researcher asks the judge to make intuitive, global predictions
for these cases. This produces the column labeled "JUD."
Psych 466, Miyamoto, Aut '15
Same Slide Except Column Labeled “MUD” Added to Table
41
• “Policy Capturing”.
Not Available
• Compute a regression
model that predicts JUD
(Model of the jUDge or
MUD).
Not Available
Method 2: Model of the Judge
It is an accident that in this example,
JUD and MUD are identical
• Example:
MUD = (–5x10–18)·SAT + 0.1·REC + 0.1·ESS + 2.0·GPA  4.8
• Use the MUD model to predict college GPA (COL) for these
cases or future cases.
Psych 466, Miyamoto, Aut '15
Discussion of Model of the Judge
42
Tuesday, October 06, 2015: The Lecture Ended Here
Psych 466, Miyamoto, Aut '15
43
Discussion of Case 2: Model of the Judge (MUD)
• Empirical findings for Case 2 are the same as for Case 1
♦
MUD more accurate than the intuitive judgments of the judge.
♦
Judges think that they make their judgments by means of a complex,
interactive, nonlinear evaluation of the cues.
♦
Whether or not this is really true, studies show that a much simpler
combination of the cues produces better predictions.
• We don't need to know the value of the criterion (COL) in order
to find a statistical formula (prediction equation) that can
outperform the judge. .
Psych 466, Miyamoto, Aut '15
Why Does MUD Outperform the Judge?
44
Why Does MUD Outperform the Judge?
The model of the judge (MUD) is a model of the judge’s
decisions, not of the criterion (reality).
•
The MUD outperforms the judge because the statistical
formula is consistent – it treats each case by the same
formula.
• A human judge has all sorts of random variations in his or her
judgment. These random variations simply increase the
inaccuracy (error) of the judge's predictions.
Psych 466, Miyamoto, Aut '15
Four Methods – SMART Method is Next
45