Theoretical REsearch in Neuroeconomic Decision-making
(www.neuroeconomictheory.org)
From perception to action:
an economic model of brain processes
Isabelle Brocas
USC and CEPR
Juan D. Carrillo
USC and CEPR
What is “Neuroeconomic Theory”?
Use evidence from neuroscience to revisit economic theories of
decision-making
Neuroscience evidence includes:
Multiple systems in
the brain
Interactions between
systems
Physiological
constraints
What is “Neuroeconomic Theory”?
Revisiting theories of decision-making includes:
• Revisit the individual decision-making paradigm:
not decision theory but game-theory approach
• Provide “micro-microfoundations” for preferences, i.e. elements
traditionally considered as exogenous (discounting, risk-aversion, etc.)
• Provide foundations for processes traditionally taken for granted
(learning, information processing, etc.)
• Understand intra-personal conflicts and behavioral “biases”
This paper
Build a brain-based model of information processing using evidence from
Neurobiology:
i.
Neurons carry information from sensory circuitry to decision-making
circuitry through a cell firing process
ii. There is stochastic variability in neuronal cell firing
iii. Thresholds of neuronal activity trigger actions (economical
information processing technology)
iv. Thresholds can be modified (learning, adaptation)
A “simple” problem of information processing!
An illustration
• Today is a dangerous day (state A) or a safe day (state B)
• Individual collects some imperfect information before making decision:
he takes a look out of the cave
• Individual decides to stay in the cave (action a) or go hunting (action b)
• Individual obtains a payoff (catches animal, killed by predator, starves)
sensory system  decision system  motor system
OVERVIEW
•
•
•
•
•
A two-actions model
More complex environments
Behavior implications
Relation to neuroscience theories
Conclusions
A two-actions model
(most interesting for neuroeconomics, least interesting for economic theory)
The sensory system
There are three main elements: environment, preferences, information
1. Environment.
- Set of states of the world: S={A,B} with element s
- Set of possible actions:
={a,b} with element 
2. Preferences over outcomes.
Represented by loss/utility function l(|s).
Payoff maximized if a in A and if b in B.
The complete description is: L = {l(a|A),l(b|A),l(a|B),l(b|B)}
3. Information structure. It consists of:
- Prior belief : assessment of likelihood of states without information
(memories, encoded values in cells).
Pr(A) = p0 and Pr(B) = 1 - p0
- Information from outside world: encoded and translated into
neuronal activity c  [0,1] which is an indicator of the state from
c = 0 (no perceived danger) to c = 1 (highest perceived danger)
There is stochastic variability in neuronal cell firing (cell activity
varies, competition between neurons, metabolic costs, noise,
circumstances)
Perception is correlated with true state
Pr(c | A)  f A (c) and Pr(c | B)  f B (c) with
f A (c)  f B (1  c) (symmetry)
Pr(c)
fB(c)
(stochastic) low
cell firing if S=B
0
Example:
d  f B (c ) 

  0 (MLRP)
dc  f A (c) 
fA(c)
1/2
(stochastic) high
cell firing if S=A
1
c
c = proportion of neurons detecting a danger
c is high in “dangerous” days, c is low in “safe” days
The decision system
A decision is a mapping from (L, p0, c) into 
Objective of the brain: conditional on L and p0, determine under which
circumstances c should trigger action a or action b.
Premise of the model: process must be economical and compatible with
existing evidence of neuronal functioning.
• Changes in beliefs and magnitude of payoffs are correlated with activity
and choice  support for maximization of expected payoff
• Neurons perform statistical inferences closely based on information
received  support for Bayesian updating
• Neuronal activity: neuronal thresholds (high or low), synaptic
connections (weak or strong). Information is interpreted to trigger
decision  support for decision-threshold, i.e. a mechanism like
threshold x such that one action if c < x and another action if c > x
Note. This process is “economical” (some information is filtered out)
• Thresholds are modulated  support for (as if) optimization process
(see paper for discussion of neuroscience and neurobiology literature)
Summary of timing
Two-alternative
task (6)
prior p0
updated p1
S  {A,B}
threshold
c >
<x
action
nature
x
cell firing
  {a,b}
l (|S)
payoff
Assumption: decision threshold is set optimally
Decision threshold x maximizes expected payoff given Bayesian
updating and neuronal constraints (stochastic variability + inability to
process all information)
Optimal threshold
• At stage 2, the posterior is pH1  Pr(A | p0, c > x) or pL1  Pr(A | p0, c < x)
• Given Bayesian updating, it is immediate that for any x :
pH1 (x) > p0 > pL1 (x)
that is, c > x is evidence of state A and c < x is evidence of state B
• Therefore, optimal threshold x* such that  = a if c > x and  = b if c < x.
It solves:
max V(x) = Pr(c > x )U(a, pH1(x)) + Pr(c < x )U(b, pL1(x))
• Let l(a|A) = A, l(b|B) = B, l(a|B) = l(b|A) = 0, and  = A / B
Proposition 1. dx*/dp0 < 0 and dx*/d < 0
dx*/dp0 < 0
Tradeoff between likelihood and impact of information
Suppose that prior strongly favors state B (p0 is “small”):
 Only strong evidence of A convinces subject to choose  = a.
 Strong evidence means a high threshold must be surpassed.
 A high threshold means a low prob. (whether true state is A or B).
(there is an analogous result in Theory of Organizations literature)
dx*/d < 0
Tradeoff between likelihood and impact of mistakes
Suppose that correct choice in B has higher potential (B is large):
 Only strong evidence of A convinces subject to choose  = a.
… same logic as before
Property of the optimal threshold
Let p(̃ c) = Pr(A|c). The optimal threshold x* is such that:
U(a, p(̃ x*)) = U(b, p(̃ x*))
Therefore: U(a, p(̃ c)) > U(b, p(̃ c)) for all c > x*
U(a, p(̃ c)) < U(b, p(̃ c)) for all c < x*
 For the purpose of choice, it is equivalent to observe exact c or
only whether c >
< x*
 Threshold is economical and fully efficient when there are only
two possible actions (not robust to more complex situations)
More complex environments
(robustness analysis)
Definitions
Note: A decision threshold discriminates between two actions.
We make a distinction between
• Cognitive processes: discriminates between all relevant actions
(involves several thresholds if more than two actions)
• Affective processes : neglects some relevant actions (involves less
thresholds than necessary)
Continuum of actions
Loss function l(s-1) if S=A and l(s) if S=B
Example: A = danger, B = no danger
Choice = hunt at distance 1-s from the cave
a. l(z) linear or convex. Optimal solution is corner: * = 0 or * =1
 identical analysis as before
b. l(z) quadratic. Departures are increasingly costly. There is one optimal
action *(p1 ) for each posterior.
 Cognitive process: requires a continuum of thresholds.
 Affective process setting one threshold: x affects posteriors (pH1, pL1).
 Optimal x* minimizes expected error.
Proposition 2. Under regularity conditions:
(i) x* has same qualitative properties: dx*/dp0 < 0 and dx*/d < 0
(ii) There is a utility loss of not observing the exact cell firing c.
Dynamic information acquisition
Individual takes a second look before selecting an action:
- Neuronal activation threshold x
Brain learns if x is surpassed or not. Beliefs updated to p1
- Neuronal activation threshold y
Brain learns if y is surpassed or not. Beliefs updated to p2
Proposition 3. Under regularity conditions, beliefs are again more likely
to be reinforced. For linear and symmetric densities functions:
y*(p) < x*(p) <1/2 if p > 1/2
1/2 < x*(p) < y*(p) if p < 1/2
Intuition.
At stage 1, information acquisition is more important than knowing if p1 is
greater or smaller than 1/2. Thus, weaker 1st period modulation
 Snowball effect: threshold modulation exacerbated in dynamic settings
 There is a utility loss of not observing the exact cell firing c.
Continuum of states
State S  [0,1] with Pr(S) = pS
2 actions,   {a,b}
Payoff is l(1 - S) if s=a and l(- S) if s=b
Example:
S = intensity of danger
a = stay , b = hunt
(assume continuous version of MLRP)
Proposition 4. Same qualitative properties as before:
if weight on dangerous (high) states increases, threshold decreases;
undesirable outcomes are likely to be avoided.
Conclusions regarding threshold modulation are robust to:
• Dynamic choices
• Increased action space
• Increased state space
However, there is a utility loss in using this economical mechanism
Behavioral implications
Implication 1. Belief anchoring and first impressions
Existing beliefs are more likely to be endorsed and less likely to be
refuted than opposite beliefs.
If state A becomes more likely (p0 increases), threshold x* decreases, so
more likely to be surpassed.
The sequence in which signals are received affects beliefs and actions
Rational Stubbornness:
- Why people develop habits that are difficult to change
- Why individual are less likely to change their mind with age
Implication 2. Polarization of opinions
Individuals with different priors who are exposed to the same evidence and
are subject to the same cell firing may update beliefs in opposite directions
Suppose pi0 > pj0. By proposition 1, xi*< xj*. If c  [xi*, xj*], then
pi1 > pi0 and pj1 < pj0.
The individual with stronger conviction that predators are present will
interpret a mixed signal as evidence of danger whereas the other will
interpret the same signal as evidence of no danger.
Implication 3. Preferences shape beliefs
Individuals with identical priors, exposed to the same evidence, and
subject to the same cell firing may end up with different posteriors due
exclusively to differences in their marginal preferences over outcomes
Suppose iA / iB > jA / jB. By proposition 1, xi*< xj*. If c  [xi*, xj*],
then (i*=a, j*=b) and pi1 > p0 > pj1
The individual with higher utility of catching a prey may go hunting while
the other stays in the cave. In that case, the former will also believe that
there is less danger than the latter.
Implication 4. Probability functions
• An optimal decision threshold mechanism generates payoff dependent
posterior beliefs.
• The individual is best represented as one entity with utility function
∑ Ps(πA, πB) πsl(γ|s)
Implication 5. Elimination
• The number of relevant actions determine the number of thresholds
necessary to achieve efficient decision making
• Affective and cognitive channels:
- 3 states: S  {A,B,C}, and 3 actions:   {a,b,c}
- Payoff 1 if a when A or if b when B or if c when C
Affective process: sets one threshold x
Cognitive process: sets two thresholds x1 and x2
The affective channel optimally discriminates between the two actions
that are most likely to occur and ignores the third one. The loss is
therefore smallest when one state is highly unlikely
Relation to neuroscience theories
The Somatic Marker Hypothesis
(Damasio ’94)
a. “Pre-existing somatic states influence the threshold of neuronal cell
firing [...] Pre-existing positive states reinforce positive states, but
they may impede negative ones” (Bechara-Damasio, GEB 2005)
Emotions regulate neuronal activity by affecting thresholds in a
precise way: existing beliefs are likely to be supported
b. “These somatic states are indeed beneficial, because they consciously
or non-consciously bias the decision in an advantageous manner”
This threshold modulation improves decision-making
SMH provides no explanation why such a “bias” is “advantageous”
Our paper formalizes and proves their claim: if emotions are responsible
for threshold modulation in that particular way, then it is true that
emotions help decision-making.
Cognitive control
(Miller and Cohen ’01)
•
•
•
The function of the PFC is to override automatic responses to shape
behavior on the basis of plans or intentions (e.g. take best action
given preferences and information)
The flow of neural activity is guided along pathways that establish
correct mappings between inputs, internal states and eventual actions
(e.g. represented by decision threshold)
Some features are retained, others are neglected (e.g. threshold acts
as a filter of information).
Conclusions
•
A theory of the brain as a (constrained) optimal processor of information
(needless to say, it is an “as if” approach).
•
The theory relates the ability to make correct choices to primitives. It
determines why and when this economical process implies an inefficiency.
•
The theory has behavioral implications:
The existence of a confirmatory bias and belief anchoring mechanism
The possibility to generate polarization of opinions
The active role of preferences on formation of beliefs (agree to
disagree)
The relationship to non expected utility theory
The fact that options in large choice sets are eliminated
•
Implications of the theory are consistent with findings in neuroscience.
Some testable predictions can help design new experiments.