Computational Approaches to
Economic Valuation & Strategy Choice
Colin Camerer
Antonio Rangel
Caltech
Outline
•
•
•
•
Brief history of the role of computation in Economics
Models of valuation and simple choice (Rangel)
Models of strategic choice and learning in games (Camerer)
Computational issues at different levels: individuals, firms, markets
(Camerer)
• Future directions of research
I
Brief history of the role of computation in Economics
Computation is at the heart of economic problems
Consider some typical problems:
•Individual: What snack should I pick out of the buffet table?
•Individual: Optimal investment portfolio?
•Firm: Price setting and production selection problem
•Market system: price formation
Thus, one would expect computational based models of decisionmaking to be common in Economics
This is not the case:
Traditional deliberate ignorance of computational detail
• The triumph of “as if” modelling
(economic behaviorism)
-- Pareto (1987):
“Pure political economy has therefore a great interest in
relying as little as possible on the domain of psychology”
-- Friedman (1953)
Test predictions of theory rather than realism of
assumptionsà can ignore computational detail
• Fictional stand-ins for computation
- Walrasian auctioneer
- equilibrium in games
• Underlying computational processes are modeled in
REDUCED FORM
Traditional view (cte.)
• Axioms are considered primitives
(logic vs biology as constraint on choices)
• A developed preference for general mathematical proof over
simulation
– Study of “procedural rationality” algorithms (Simon) did
not gain traction
– Distrust of complicated many-equation macroeconomic
models & simulations
– Little post-1990 taste for SFI agent-based modeling
Neuroeconomics: neurobiologically based
computational models of decision-making
Goals of Neuroeconomics:
1.
What computations are carried out by the brain to make
different types of economics decisions?
2.
How are these computations implemented by the brain?
3.
What are the implications of this knowledge for economics,
finance, education, AI, marketing, … ?
Computation is at the core of Neuroeconomics
BUSINESS
APPLICATIONS
JUGDMENT
& DM
THERAPEUTIC
APPLICATIONS
ECONOMIC
APPLICATIONS
COMPUTAT.
MODELS
NEUROSCIENCE
A.I.
PSYCHOLOGY
II
Neuroeconomic Models of Valuation and Simple Choice
Example I: Reward Prediction Learning
5
8
9
10
11
Reward
wait
7
…
Cue
6
Reward
wait
Reward
3 4
trial 3
Cue
Time: 1 2
trial 2
wait
Cue
Event: trial 1
12 ….
• Brain’s problem: learn to predict size & timing of rewards that follow
each type of cue
• Temporal-difference learning algorithms have been designed in CS
to solve this problem (Sutton & Barto (1998))
How can the brain learn the reward function?
Notation:
- True value of state s: mean r(s)
- pt(s) = computed predicted value at beginning of triat t
(= brain’s best guess about the state’s true value)
- t(s) = r(s) - pt(s)
= error signal in trial t
This error term is extremely important: it serves as THE
teaching signal!
Learning Algorithm
Step 1. Arbitrarily initialize the decision values p1(s) for all s
Step 2. Every trial t:
-- begin with pt(s)
-- measure actual reward
-- Compute error (t)
-- Update the DV for a and c active in trial as follows:
pt+1(s) = pt(s) +  (t)
where
-> (0,1) is a learning rate
• Under very general conditions, E(pt(a|c)) -> E(r(s)) for all s
How well do TD algorithms describe brain’s
reward learning?
Cue
TD-Errors
Before Learning
TD-Errors
During Learning
TD-Errors After
Learning if Unexpected
Omission of reward
Reward
Can we find evidence of TD-error signals in monkeys’ brains?
Single unit recordings from VTA dopamine neurons revealed that these neurons
produce responses consistent with TD - learning:
Schultz [1998]
What brain areas show activation that
correlates w/ TD-error signals in humans?
p<<0.001
+3
R
-3
-6
-30
+54
From O’Doherty et.al. [2003]
 at time of CS
+6
CS+ trials
Example II: Role of Visual Attention in Simple Choice
?
Model: Three Parallel Processes
• e=time elapsed since beginning of choice trial
Visual
attention
DVs
computation
Comparator
g(e)= L,R
dL(e), dR(e)
Choose L,R,or wait
Visual
attention
switch
g(e)
g(e)
DV
Computation
Comparator
dL(e), dR(e)
choose g(e) or
switch
Visual attention process
First fixation: Stochastic bottom-up process
•P0 = Prob first fixation to L
•Exponential latency: Pr(First fixation begins at t)= 1- B.e- t
Subsequent fixations: top-down process
•Follow the commands of the comparator process
Value construction process
v+
0
vt
Comparator process:
•
During each fixation, the comparator either chooses g(e) or sends a signal to the visual
system to switch gaze
•
Length of each fixation stochastic:
-- d = duration of current fixation
-- Pr(comparator evaluates at d)= 1- A.e- d
•
Decision made as follows:
-- rx(e) = d(tx(e)) - d(ty(e))
•
-- Choose g(e) with probability
-- Wait (and switch fixation) with prob
•
Always switch after first fixation
Model predictions
• Behavioral: S-shaped choice probabilities
• Process: RTs and #saccades increase with choice difficulty
• Performance:
- Importance of first fixation:
P(choice=best|fist-fixation=best)> P(choice=best|fistfixation=worse)
- First look bias: for items with similar value
P(choice=L|fist-fixation=L)> P(choice=L|fist-fixation=R)
-…
Test
Enforce
2000 ms
fixation
+
Present until a choice
is made
1000 ms
+
+
Collect eye-fixations @ 50 Hz
Results
QuickTime™ and a
TIFF (PackBits) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Summary
•
Computation is at the core of the nascent field of Neuroeconomics
•
Goal is to (1) describe the computation and processes that the brain
uses to make decisions and (2) establish their neural basis
•
Test the computational processes directly using modern neuroscience
and psychology tools -- from fMRI to eye tracking
•
Feasibility of the research agenda has already been proven
•
Novel insights into DM are already being generated by this class of
models.
III
Models of Strategic Choice & Learning in Games
Some theoretical interest in computational models
• Finite-state automata (Rubinstein, Neyman, et. al.)
• Computational complexity (Gilboa-Zemel on NP-hard games)
• Not linked to data or practical problems
Cognitive hierarchy models of limited strategic thinking
• Selten (1998):
– “The natural way of looking at game situations…is not based on
circular concepts, but rather on a step-by-step reasoning
procedure”
• Cognitive hierarchy
– “Level 0” use a heuristic (e.g. randomize)
– “Level k” best-respond to choices of level 0-(k-1)
– Axiom f(k)/f(k-1)  1/k (k-th step increasingly difficult)
 f(k)=e-ttk/k! (Poisson)
– Limit as t   often converges to equilibrium
– Simpler than equilibrium in some ways
easier to compute predictions
no problem of multiple equilibria
Limited planning ahead in bargaining
(Science, 03)
3-stage bargaining
1:
$5
p1 offers
2: $2.50 p2 offers
3: $1.25 p1 offers
(0,0) if rejected
E.g. “P-beauty contest” (Ho et al AER 98)
pick x in [0,100], x closest to (2/3) of average wins
1
0.8
equilibrium=0
0.7
data
0.6
CH prediction (τ=1.5)
0.5
0.4
0.3
0.2
0.1
number choice
95
85
75
65
55
45
35
25
15
0
5
relative frequency
0.9
“Choosing” computations are different than “belief
formation” computations
Bhatt-Camerer GEB 2005
Field application: “Cold opening” of movies
(unavailable to critics for Friday review)
Studios do not let worst movies get reviewed…
”cold” opening increases box office
EWA learning in games: Generalized reinforcement
• Reinforcement, fictitious play linked (Econometrica 99)
• Update attractions to strategy j from payoff
A ij (t) - A ij (t-1) = [*π(s ij,s-i (t)) -A ij (t-1)]/(ϕN(t-1)+1)
= prediction error/increasing weight
 is “imagination” of counterfactual payoffs
ϕ is recency weight
Typical values: N(0)=1, ϕ=.8, weights go from .56  .20
• Can replace , ϕ with “self-tuning” functions (JET ’07)
• Can add “sophistication”– players know others are
learning (JET 02, GEB 06)
Example: Price matching with loyalty rewards
(Capra, Goeree, Gomez, Holt AER ‘99)
• Players 1, 2 pick prices [80,200] ¢
Price is P=min(P1,,P2)
Low price firm earns P+50
High price firm earns P-50
• What happens?
– Theory: competition drives prices to 80
191~200
181~190
171~180
161~170
151~160
141~150
131~140
121~130
111~120
101~110
91~100
81~90
80
1
3
5
Period
7
9
5
9
191~200
181~190
171~180
161~170
151~160
141~150
131~140
121~130
111~120
101~110
91~100
81~90
1
80
3
Period
7
Empirical Frequency
0.9
0.8
0.6
0.7
0.5
0.4
Prob
0.3
0.2
0
0.1
Thinking fEWA
Strategy
0.9
0.8
0.7
0.6
0.5
Prob
0.4
0.3
0.2
0.1
0
Strategy
IV
Computational Issues at different levels:
individuals, firms, markets
Levels of computational modelling in economics
•
•
Individuals (what you’ve seen)
Firms
– Firms as hierarchies of imperfectly informed individuals (Radner-Van Zandt)
Optimal hierarchies for aggregating formation
•
Mechanism design
– Computability as an individual rationality constraint (Ledyard)
•
Markets
– Markets as computational mechanisms
• Computing equilibria (Judd, Kearns et al)
– Smart markets: Hybrids of bids and optimal combination (e.g. combinatorial
“package auctions” e.g. PCS spectrum)
– Information aggregation
• Markets ‘compute’ probabilities of events (e.g. prediction markets)
Prediction markets
•
•
•
•
•
•
•
•
•
•
Began with basic research: 20 yrs to wide use
Plott and Sunder (1982 Econometrica):
Markets for “contingent claims”
Pay $1 if an event occurs. Prices reveal probabilities
Markets are $-weighted opinion polls of self-selected respondents
Iowa Political Markets 1988 (http://www.biz.uiowa.edu/iem/)
Markets for political events predict surprisingly accurately
Tradesports 2002 (http://www.tradesports.com/) et al
Used by some companies, policy markets
See Wolfers & Zitzewitz J Econ Perspectives 04
Six hours earlier (9pm EST Oct 26 ‘06): Guess about Karl Rove nonindictment appears in Intrade price drop…36 hrs before Oct 28 Libby
indictment
ROVE.INDICTED.31DEC
ASK
BID
Price
Price
1
25.1
29.9
1
10
25.0
30.0
3
1
24.6
31.9
1
1
24.4
32.0
2
1
24.2
32.7
2
1
23.7
33.0
10
1
23.6
34.9
4
3
23.4
35.0
20
11
23.3
39.0
5
27
23.2
40.0
11
16
23.0
50.0
10
13
22.5
68.8
5
10
22.0
70.0
99
20
21.0
72.0
4
74.9
1
Qty
11
20.0
Qty
Google news at 1:46am EST Oct 27: Will Karl Rove be indicted?
•Rove critics again turn up the volume
New York Daily News - Oct 27 1:18 AM
With rampant rumors of a soon-to-drop indictment in Special Counsel Patrick
Fitzgerald 's CIA leak investigation, the Karl Rove literary business is booming.
•Rove's Last Campaign
Washington Post - Oct 26 11:31 AM
Will Karl Rove, architect of President Bush's improbable political career, snatch
one last victory from the jaws of defeat? (Or at least avoid getting indicted?)
Something appears to have provoked special prosecutor Patrick J. Fitzgerald
into a last-minute flurry of activity centered............
•Leak Counsel Is Said to Press on Rove's Role
New York Times - Oct 25 7:25 PM
Three days before the grand jury is set to expire, Patrick Fitzgerald appeared to
be trying to determine Karl Rove's role in the outing of a C.I.A.'s officer's identity.
•Libby, Rove Await Indictment Decisions By Martin Sieff, UPI Senior News
Analyst Washington DC (UPI) Oct 25, 2005
Space War - Oct 26 9:53 PM
Washington seethed with rumors and speculation Tuesday night on the eve of
the expected announcement of possible indictments in the Valerie Plame CIA
leak probe.
Current (3/15) “prices” of
Scooter Libby pardon
Legal - I. Lewis Libby
Contract
Bid
Ask
Last
Vol
Chge
I. Lewis (Scooter) Libby Pardon
LIBBY.DEC
07.PARDO
N
I.Lewis
Libby to be
pardoned
by 31 Dec
2007
M
10.0
17.0
9.5
1012
0
LIBBY.EOT
.PARDON
I.Lewis
Libby to be
pardoned
by the end
Mar 16 - 3:18AM GMT
of
President
Bush's
term in
office
M
62.0
63.2
63.2
948
-0.2
V
Future of computational models of decision in Economics
@ the Individual level
• It will look like theoretical neuroscience
• Focus on modeling the neural and psychological processes
involved in decision-making
• Modeling constraints provided by neural, psychological, and
behavioral data
• Models will be tested with techniques such:
- fMRI
- electrophysiology
- TMS
- eyetracing
- behavioral predictions
@ the firm and market levels
• Will build on the properties of the individual level models
• Model the interactions of many agents
• Goal will be to improve our understanding of:
- auctions
- price formation in markets
- financial markets dynamics
- macroeconomic performance and policy