Pictures and Words
Elinor Amit
Behavioral Decision Making: Building Approaches from
Laboratory Insights
October 31th 2011
How do we think about things?...
The 3 main approaches:
• Words (inner speech) (e.g., Vygotsky, 1934; Oppenheim & Dell, 2010)
• Pictures (mental imagery) (e.g., Kosslyn et al)
• Abstract, a-modal representations (e.g., Caramazza, Hillis,
Rapp, & Romani, 1990; Lambon Ralph, Graham, Patterson, & Hodges, 1999; Rapp, Hillis, & Caramazza,
1993; Tyler & Moss, 2001; Mahon & Caramazza, 2008)
How do we think about things?...
The 3 main approaches:
• Words (inner speech) (e.g., Vygotsky, 1934; Oppenheim & Dell, 2010)
• Pictures (mental imagery) (e.g., Kosslyn et al)
• Abstract, a-modal representations (e.g., Caramazza, Hillis,
Rapp, & Romani, 1990; Lambon Ralph, Graham, Patterson, & Hodges, 1999; Rapp, Hillis, & Caramazza,
1993; Tyler & Moss, 2001; Mahon & Caramazza, 2008)
“the two most vivid forms of working memory
are mental images... and snatches of inner
speech”
Pinker, 2008
Apple
Red apple
Apple
Red apple
Differences between words and pictures
Medium
words
pictures
Dissimilar to the object: Similar to the object:
Red apple
Cognized
Arbitrary
Abstract
Gist
Simple
Perceived
Not arbitrary
Concrete
Peripheral
Complex
We will discuss today…
- The association between medium and
psychological distance
- The implications of the medium of representation
to moral judgment
We will discuss today…
- The association between medium and
psychological distance
- The implications of the medium of representation
to moral judgment
Thedo
medium/morality
How
we think abouthypothesis
things?...
It depends…
Proximal event
Distal event




Visual
representation
Verbal
representation
Amit, Algom, & Trope, 2009
What is psychological distance?
The distance between an individual and a target.
• Subjective
• Egocentric
What is psychological distance?
• Temporal: How much time separates
between the individual’s present and the
target event
• Spatial: How far in space is the event
from the individual
• Social: How different is the social
target from the individual
Bar Anan, Liberman & Trope, 2006
Why is medium associated with
distance?
• Pictures and words serves different cognitive
functions:
• Words  preserve the invariant & essential
properties of the item for a distal use
• Pictures  preserve the stimulus in details for
an immediate use.
Construal Level Theory (CLT)
Mental construal processes serve to traverse psychological distances
and switch between perspectives.
Distal events  not so much information =>
abstract representation, entailing the essence, invariant
features of the referent.
Proximal events  there is information =>
concrete, subordinate representation.
Trope & Liberman, 2003; 2010
Main hypothesis
Pictures and words are associated with psychological
distance:
• People elect to represent close targets in pictures and
distal targets in words.
• Pictures impart a feeling of proximity, whereas words
impart a feeling of distance.
• cognitive processing:
- Identification
- Categorization
- Selective attention
- Memory
• Social cognition:
- Interpersonal communication
- Moral judgment
• Neural correlates
• cognitive processing:
- Identification
- Categorization
- Selective attention
- Memory
• Social cognition:
- Interpersonal communication
- Moral judgment
• Neural correlates
Speeded Identification: Social distance
football
Medium: pictures, words
Distance: socially near, far
Task: speeded identification
soccer
Socially proximal
Socially distal
Amit, Algom, & Trope, 2009, Jep: General
Speeded Identification: Social distance
Reaction Time (ms)
650
proximal
distal
630
610
590
570
picture
word
F(1,13)=7.63, p < .05
Amit, Algom, & Trope, 2009, Jep: General
Speeded Identification: Spatial distance
Medium: pictures, words
Distance: spatially near, far
Task: speeded identification
Amit, Algom, & Trope, 2009, Jep: General
Speeded Identification: Spatial distance
700
proximal
distal
Reaction Time (ms)
680
660
640
620
600
picture
word
F(1,15)=6.3, p < .05
Amit, Algom, & Trope, 2009, Jep: General
Mental travel
Exp.1: event related design, 11 subjects
Exp.2: block design, 10 subjects
+
Tomorrow
CHAIR
+
10 years
APPLE
 20 sec
+
 1 sec
Amit, Rim, Greene, & Trope, in prep
Mental travel (Harvard sample)
far>near
near>far
Language regions
Visual regions
precuneus
Fusiform R
calcarine
BA47L
Fusiform L
BA 21 L
N=10, p =.001
Interim conclusion
Pictures are associated with proximity, whereas
words are associated with distance.
- This association influence performance in
various cognitive tasks (e.g., identification)
- People spontaneously elect to represent
proximal things visually (embodied cognition),
and distal things verbally.
Implications of the medium/distance
association for public policy
pressure
from the
government
convenience for
costumers
existing routes
paths
location of
main public
services
air pollution
cost
location of
businesses
bus driver’s
union demands
Walk a lot
Wait a little
?
Walk a little
Wait a lot
We will discuss today…
- The association between medium and
psychological distance
- The implications of the medium of representation
to moral judgment
We will discuss today…
- The association between medium and
psychological distance
- The implications of the medium of representation
to moral judgment
The
medium/morality hypothesis
hypothesis
The
medium/distance
Proximal event
Distal event




Visual
representation
Verbal
representation
The
medium/morality hypothesis
hypothesis
The
medium/distance
Visual
representation
Verbal
representation




Proximal event
Distal event
The
medium/morality hypothesis
hypothesis
The
medium/distance
Implications for behavior…
Visual
representation
Verbal
representation




Proximal event
Distal event
If you see something
Do something!
• Pictures  “emergency”
reaction
• Words  not so much
Visual
processing
Verbal
processing
 Psychologically 
proximal
High emotional
reaction
 Psychologically
distal
Low emotional
reaction

5 >1
Visual
processing
Verbal
processing
 Psychologically 
proximal
High emotional  Emotionally-driven
judgments
reaction
 Psychologically
distal
Low emotional  “cognitive”-driven
judgments
reaction

5 >1
Dual-Process Moral Cognition
5 >1
Yes
?
Dual-Process Moral Cognition
5 >1
Yes
No
X
Greene et al, Cognition, 2009
?
Predictions
Visual processing  deontological judgments
(rely on emotional reactions)
Verbal processing  utilitarian judgments
(rely on “controlled”
reaction)
Another rationale for the medium/moral judgment prediction
Pictures  thinking about means (consistent with
deontological moral reasoning)
Words  thinking about end-goals (consistent with
utilitarian moral reasoning)
Exp 1: How media preferences are related to moral judgments?
L.A. Times
Wall Street journal
Exp 1: How media preferences are related to moral judgments?
10
9
8
More utilitarian
7
6
5
4
3
2
1
0
0
1
2
3
4
r(108) = 0.233, p = 0.011*
After controlling for level of education: r(107) = .22, p = 0.018*
5
6
7
Exp 1: How media preferences are related to moral judgments?
10
9
8
7
More utilitarian 6
5
4
3
2
1
0
0
1
2
3
4
Hours/day of TV watching
R(170) = -0.18, p = 0.017*
After controlling for education: r(169) = -0.18, p = 0.01**
5
6
7
Conclusions from Experiment 1:
Media consumption preferences are associated
with moral judgments tendencies:
Visual  deontological style
Verbal utilitarian style
* Cannot be accounted for by education
Exp 2: How cognitive style is related to moral judgments?
Amit & Greene, under revision
Exp 2: How cognitive style is related to moral judgments?
p < .01
N =R(50)=.346,
50
r = .35
p = .01
utilitarian
More utilitarian
Verbal - Visual Accuracy
Conclusions from Experiment 2:
Cognitive style is associated with moral
judgments tendencies:
Visual style  deontological
Verbal style  utilitarian
Amit & Greene, under revision
Exp 3: How visual and verbal interference affect moral judgments?
Condition 1
Condition 2
Enemy soldiers
have taken over
your village….
You are standing
near a
footbridge…
5 sec
Is it appropriate
to smother your
baby?
Is it appropriate
to smother your
baby?
5 sec
no…..yes
no…..yes
 2.5 sec
Amit & Greene, under revision
Prediction:
• pictorial interference prevents representing
the dilemma visually, thus leads to more
utilitarian judgment.
• Verbal interference prevents representing the
dilemma verbally, thus leads to more
deontological judgment.
Amit & Greene, under revision
Exp 3: How visual and verbal interference affect moral judgments?
0.58
0.56
*
0.54
n.s
More utilitarian 0.52
0.5
0.48
0.46
0.44
0.42
0.4
picture
word
control
Interference type
Amit & Greene, under revision
Conclusions from Experiment 3:
Visual imagery creates more deontological
judgments.
No effect for words (compared to control) –
suggests that the default mode of thinking
about moral dilemmas is visual.
Amit & Greene, under revision
Exp. 4
• Subjects read mean vs. side dilemma.
• Than made moral permissibility judgment
• And reported whether they imagined the one
to be killed or the five to be saved.
What did you imagine more?
The 5
5
4
3
2
1
The one
0
side
F(1,331) = 15.1, p < .0001
mean
imagery
P = .000***
Dilemma
(mean, side)
P = .001**
P = .000***
Sobel = -2.04, p = .004
Moral judgment
Take-home message
Intriguing implications for the differences between
pictures and words…
- People think about proximal things in pictures
and about distal things in words
- They think about “means” in pictures and about
“end-goals” in words
- Thinking in pictures lead to deontological moral
judgments, and thinking in words lead to
utilitarian moral judgments.
Thanks
•
•
•
•
•
•
•
Joshua Greene
Yaacov Trope
Daniel Algom
Galit Yovel
Evelina Fedorenko
SoYon Rim
Georg Halbeisen
•Nobuhito Abe
•Alek Chakroff
•Fiery Cushman
•Joe Paxton
•Steven Frenkland
•David Rand
•Regan Bernhard
•Ryan Halprin
•Sara Gottlieb
•Allison Gofman
•Rebecca Fine
•Warren Winter
•Paul Lively
Thanks!
Download

Amit/Harvard_Kennedy_talk_103111_for students