CS 182
Lecture 28: Neuroeconomics
J.G. Makin
April 27, 2006
Decisions, Uncertainty, and the Brain
Paul Glimcher (2003); MIT Press
• Thesis: neuroscience has been
dominated by the reflex paradigm
• Alternative: investigations rooted in
economics, evolution, game theory,
and probability
Reflex Theory
• Model: Input-Association-Output
– (think of trying to explain language this way)
– Even ANNs?
• Methodology: thoroughly constrain the environment
– “Isn’t this how science is done?”
– Obscures a system-level view
• Has this really led researchers astray?
• Why are there so many questions on this slide?
Glimcher 2003
Reflex Theory (con’t)
Challenges to “naïve” reflex theory
• T. Graham Brown and the Half-Center Oscillators
[This is not the name of a band, as far as I know,
though it should be]
– Sherrington: stimulus for walking from enteroceptive
or interoceptive sources only
• Reafference and Efference Copy (Von Holst and
Mittelstaedt)
– [Glimcher actually has these confused]
Reflex Theory (con’t)
Challenges to “naïve” reflex theory
• T. Graham Brown and the Half-Center Oscillators
[This is not the name of a band, as far as I know,
though it should be]
– Sherrington: stimulus for walking from enteroceptive
or interoceptive sources only
• Reafference and Efference Copy (Von Holst and
Mittelstaedt)
– [Glimcher actually has these confused]
An Alternative
Behavior is structured
– by goals (cf. shoulder reflex)
– by optimization strategies in the face of uncertainty
– Specification of the problem on the basis of function
rather than implementation (Marr)
– In particular, the problem is an optimization problem
– Conclusion: Neuroscience needs probability theory,
economics, evolutionary theory, and game theory
Reflex Theory (con’t)
What reflex theory doesn’t address
– the shoulder “reflex” (Paul Weiss)
– foraging
– mate selection
– exploratory behaviors
– Language & thought
An Alternative
Behavior is structured
– by goals (cf. shoulder reflex)
– by optimization strategies in the face of uncertainty
– Specification of the problem on the basis of function
rather than implementation (Marr)
– In particular, the problem is an optimization problem
– Conclusion: Neuroscience needs probability theory,
economics, evolutionary theory, and game theory
I: Optimization
• Q: Optimization with respect to what?
• A: Inclusive fitness but modularized. Evolution
provides the goals, economics the optimization
techniques
• Do we have a prayer at specifying the optimum?
– Phototransduction near the quantum limit
– Hair cells can detect individual fluid molecule collisions
– Convergent Evolution: Cichlid fish of Tanzania
II: Uncertainty: Epistemological
• Reflex theory dominated by deterministic responses
to input (from a highly constrained set)
• Alternative: in general, we suffer from
epistemological uncertainty, so we have to
optimized in an indeterminate world
Uncertainty (con’t)
• An empirical test of foraging economics:
the prey model, Parus major
• View foraging as an optimization problem: choose the
probability p_i of attacking the prey i that maximizes
the rate at which energy is gained
• Solution:
– “zero-one” rule
– “independence from encounter inclusion rate” principle
Uncertainty (con’t)
• Frequencies of large and small mealworms were varied
• Small mealworms always had larger handling time
• Prediction (from optimal sol’n):
– Preference for large worms as their freq. increases,
regardless of small worm freq. (by IEIR principle)
– If the bird couldn’t get all the worms, it should give up
entirely on the small ones (by the zero-one rule)
• Result: yes and no (only 85% selective)
• Maybe this is an optimal strategy after all…
Epistemological Uncertainty & the Brain:
A Series of Studies
• Input-association-output model: sensory-parietalmotor
• Lateral intraparietal area (LIP) and monkey
saccades:
– Monkeys trained to perform task w/juice reward
– Invariant to input stimulus (light or button or
whatever)
– Position-encoding
– Conclusion: command signal (Mountcastle)
Epistemological Uncertainty & the Brain
(con’t)
• Lateral intraparietal area (LIP) and monkey
saccades:
– Fixation and saccade tasks w/eccentric light
– Weak activation on fixation, but increasingly active
over trials of saccade task
– Conclusion: attentional enhancement (Goldberg)
Epistemological Uncertainty & the Brain
(con’t)
• Lateral intraparietal area (LIP) and monkey
saccades:
– Memory saccade task: target is extinguished but LIP
neuron still fires—until the motor command is
executed
– Conclusion: motor intention (Gnadt & Anderson)
Epistemological Uncertainty & the Brain
(con’t)
Platt & Glimcher: encoding the probability of pay-off
Epistemological Uncertainty & the Brain
(con’t)
Probability experiment
Epistemological Uncertainty & the Brain
(con’t)
Value experiment
III: “Irreducible” Uncertainty & Game Theory
• Static environment  Dynamic competition with
other agents
• Then the optimal approach is given by gametheoretic approaches
• In these cases, the optimum often involves
(purposefully) random behavior
Uncertainty & Game Theory (con’t)
• Example 1: Chicken
Uncertainty & Game Theory (con’t)
• Conclusion: Smith is best served by behaving nondeterministically, but with probability 0.647 of being
a chicken. (Ditto for Jones.)
• If Jones finds non-randomness in the distribution of
Smith’s choices, he can predict above chance which
option Smith will pick—and win.
• Random behavior is the optimal solution, so: we
shouldn’t expect behavior to look deterministic
(contrast w/reflex theory).
Intermezzo: How Random Are We?
• Paper, scissors, rocks
• Dice, viscera divination, etc.: technological
breakthrough (Jaynes)
• Unconscious vs. conscious behaviors; natural
selection vs. “rational actors”
• Pigeons, babies, and adults: the matching rule and
cognitive load (and reward)
Game Theory and Ethology
• Duck foraging
– Two feeders at opposite ends
– 33 ducks
– Rate of food depends on feeder, but the more ducks
in an area the worse it is
– Where to sit?
Game Theory & Ethology (con’t)
Game Theory & Ethology (con’t)
• Person 1: 2-gram bread ball every 5 sec
• Person 2: 2-gram bread ball every 10 sec
Game Theory & Human Behavior:
Work or Shirk
Game Theory & Human Behavior:
Work or Shirk (con’t)
Insp = -50
Insp = -5
Game Theory & Human Behavior:
Work or Shirk (con’t)
• Experiment: subjects play against a computer
program which looks for statistical regularities in its
opponent’s plays and tries to exploit them
• Subjects are only told that they can make money by
playing
• 150 trials, then the pay-off matrix switches
(unannounced)
• Guess how human beings played….
Game Theory & Human Behavior:
Work or Shirk (con’t)
• 150 trials, one pay-off matrix, vis-à-vis the Nash
equilibrium?
Game Theory & Human Behavior:
Work or Shirk (con’t)
Game Theory & Human Behavior:
Work or Shirk (con’t)
• Work-shirk-work-shirk yields 50% behavior. Shannon
entropy of choices?
Game Theory & Human Behavior:
Work or Shirk (con’t)
Game Theory & Human Behavior:
Work or Shirk (con’t)
• Switching between pay-off matrices?
Game Theory & Human Behavior:
Work or Shirk (con’t)
Game Theory & the Brain
• Repeat the game, this time with monkeys instead of
humans
• Simultaneously record from parietal area LIP
• Prediction: if these neurons encode expected utility,
then they will fire at constant rates over various
movements and various rewards (contrast Platt &
Glimcher 1999)
• Now we have an experiment that yields nondeterministic behavior but about which predictions
of lawful actions can nevertheless be made
Game Theory & the Brain (con’t)
Game Theory & the Brain (con’t)
• Across trials:
– Monkeys behave (near?) optimally: their behaviors
track the Nash equilibrium
– LIP neurons do not track the Nash equilibrium
suggesting that they are, in fact, encoding (relative)
expected utility
• Play-by-play:
– The relative expected value on any given play does
vary slightly, given the randomness of play
– Positive correlation b/n this fluctuating expected
value and fluctuations in LIP neurons
Neuroeconomics & Language
• Skinner’s Verbal Behavior
• Programs that are more than input/output
• Bayes Nets for utility as well as beliefs
• Minimum description length: grammar
• Minimum description length: evolution
Neuroeconomics & Language
• “The paradox disappears only if we make a radical
break with the idea that language always functions
in one way, always serves the same purpose: to
convey thoughts—which may be about houses, pains,
good and evil, or anything else you please.” (Sec.
304)