Describe the coding of taste and smell signals. Compare both with audition, vision, proprioception, and touch.
Describe how attention affects perception. How does attention affect neural responses?
What kinds of information are conveyed by the somatosensory system? How is that information
encoded in the nervous system?
Describe the various ways that optic flow can be used in vision?
What is known about sensory prediction? Why is it important?
•
•
What is Bayesian inference?
How does it apply to perception?
x+y=9
What are x and y?
This is an example of an ill-posed problem
• problem that has no unique solution
Perception is also an ill-posed problem
Example:
Spectrum of
Illuminant
×
Reflectance function
of surface
=
Light Hitting
Eye
Question we want to answer: what are the surface
properties (i.e., color) of the surface?
Equivalently:
X × Y = some cone responses R
Given R, was Y?
(you’d have to know X to make it well-posed)
Comparison patch
Same light hits the eye from both patches
Perception is an ill-posed problem
Example #2:
3D world
⇒
2D retinal image
Question: what’s out there in the 3D world?
Ill-posed because there are infinitely many 3D worlds that
give rise to the same 2D retinal image
• need some additional info to make it a well-posed problem
Figure 1. (a) The Necker Cube
induces a bi-stable percept. (b)
Disambiguation of
the bi-stable Necker Cube percept by
introducing an occlusion cue and a
shadow.
(c) An infinite number of 3D
configurations could produce the
same projection
image. Here this fact is illustrated by
the cast shadow on the tabletop, but
the
same projected images would be
formed on the eye’s retina.
Luckily, having some probabilistic information can help:
x+y=9
Tables showing past
values of y:
x
y
2 1 2
7 7 7
2 2 2
7 7 7
2 3 1
5 7 7
3 1 2
7 6 7
2 2 2
7 7 7
1 2 2
8 7 8
3 2 2
7 7 7
2 2 2
7 7 7
Given this information
about past values,
what would you guess to
be the values of x?
How confident are you in
your answer?
Bayes’ rule
• very simple formula for manipulating probabilities
P(A | B) P(B)
P(B | A) =
P(A)
conditional probability
“probability of B given that A occurred”
Formula for computing:
P(what’s in the world | sensory data)
(This is what our brain wants to know!)
from
P(sensory data | what’s in the world) = “likelihood”
(given by laws of physics;
&
ambiguous because many world states
could give rise to same sense data)
P(what’s in the world) = “prior”
(given by past experience)
Examples:
Using Bayes’ rule to understand how the brain
resolves ambiguous stimuli
Many different 3D worlds can give rise to the same 2D retinal image
The Ames Room
A
B
How does our brain go about deciding which interpretation?
P(image | A) and P(image | B) are equal! (both A and B could have generated this image)
Let’s use Bayes’ rule:
P(A | image) = P(image | A) P(A)
P(B | image) = P(image | B) P(B)
Which of these is greater?
Which dimples are popping out and which popping in?
P( image | OUT & light is above) = A
P( image | OUT & light is below) = 0
P(image | IN & Light is above) = 0
P(image | IN & Light is below) = A
• Image equally likely to be OUT or IN given sensory data
alone
What we want to know: P(OUT | image) vs. P(IN | image)
Apply Bayes’ rule:
prior
P(OUT | image) = P(image | OUT & light above) × P(OUT) × P(light
above)
P(IN | image) = P(image | IN & light below ) × P(IN) × P(light
below)
Which of these is greater?
P( image | OUT & light is above) = A
P( image | OUT & light is below) = 0
P(image | IN & Light is above) = 0
P(image | IN & Light is below) = A
P(OUT | image) = P(image | OUT & light above) × P(OUT) × P(light above)
P(IN | image) = P(image | IN & light below ) × P(IN) × P(light below)
P(OUT | image) = A × 0.5 × P(light above)
P(IN | image) = A × 0.5 × P(light below)
Let’s say:
“Light above” is 10 times more likely than “light below”
P( image | OUT & light is above) = A
P( image | OUT & light is below) = 0
P(image | IN & Light is above) = 0
P(image | IN & Light is below) = A
P(OUT | image) = P(image | OUT & light above) × P(OUT) × P(light above)
P(IN | image) = P(image | IN & light below ) × P(IN) × P(light below)
P(OUT | image) = A × 0.5 × P(light above) = 5 × A
P(IN | image) = A × 0.5 × P(light below) = 0.5 × A
Let’s say:
“Light above” is 10 times more likely than “light below”
Bayesian account: “Out” is 10 times more likely!
Summary
•
•
•
•
Perception is an ill-posed problem
equivalently: the world is still ambiguous even given all our
sensory information
Probabilistic information can be used to solve ill-posed problems
(via Bayes’ theorem)
Bayes’ theorem:
posterior
P(world | sense data)
•
likelihood
prior
∝ P(sense data | world ) P(world)
The brain takes into account “prior knowledge” to figure out
what’s in the world given our sensory information
(Note that the “posterior” can be considered the prior for the next time
step in an ongoing learning process)
Two take-home facts about “what it means to be Bayesian”
in psychology / neuroscience:
1. Use priors
• we recognize that perception is ambiguous (“ill-posed”), and that the only /
best way to deal with that is to use prior information
(built up over last 2 minutes, last 2 days, last 2 decades, last 2M years, etc.)
ref: • Weiss, Simoncelli & Adelson, “Motion illusions as optimal
percepts.” Nature Neuroscience (2002)
2. Keep around a “full posterior distribution”
• use probabilistic information (don’t just store the “most likely answer” —
also store an estimate of our uncertainty)
refs:
(cue combination)
• Ernst, M. & Banks, M. “Humans integrate visual and haptic
information in a statistically optimal fashion” Nature, 2002
• Körding, K. & Wolpert, D. “Bayesian Integration in Sensorimotor
Learning” Nature, 2004
•
•
•
What does it mean to perceive optimally?
What is a Bayesian model of human perception?
How can we formulate / fit Bayesian models?
Visual capture
Vision and haptics – view object through a cylindrical lens: vision dominates, but a small effect
Of haptic info.
Vision and audition – sounds are localized to visual source eg speakers mouth
In other instances, senses complement each other – eg feeling an object
where there is no conflict, info from back is given by haptics, front by vision.
Auditory capture
Number of beeps determines whether single or multiple flashes are seen
Dominance determined by reliability of the estimate.
Figure 3. Visual–haptic size-discrimination performance determined with a 2-interval forced-choice task [29]. The relative reliabilities of
the individual signals feeding into the combined percept were manipulated by adding noise to the visual display.
With these different relative reliabilities four discrimination curves were measured. As the
relative visual reliability decreased, the perceived size as indicated by the point of subjective equality (PSE) was increasingly
determined by the haptic size estimate (haptic
standard, SH) and less by the visual size estimate (visual standard, SV). This demonstrates the weighting behaviour the brain adopts
and the smooth change from visual
dominance (red circles) to haptic dominance (orange triangles). As shown, the PSEs predicted from the individual visual and haptic
discrimination performance (larger
symbols with black outline) correspond closely to the empirically determined PSEs in the combined visual–haptic discrimination task.
(JND . just noticeable difference.)
What does it mean to perceive optimally?
(better to say “act”)
General Goal: Decide: “what stimulus is present?”
Assumptions:
1. Our senses provide noisy measurements of the
environment (e.g., “something moving in the grass”)
2. We have some well-defined (deterministic) cost
function describing the cost of making different errors
(e.g., cost saying “wildebeast” when the true stimulus was “lion”)
3. We have some beliefs about the underlying probability
of different stimuli (e.g., “wildebeast” more common than “lion”)
What does it mean to perceive optimally?
(better to say “act”)
Solution: Given a noisy measurement, pick stimulus that
minimizes the expected cost.
Assumptions:
1. Our senses provide noisy measurements of the
environment (e.g., “something moving in the grass”)
2. We have some well-defined (deterministic) cost
function describing the cost of making different errors
(e.g., cost saying “wildebeast” when the true stimulus was “lion”)
3. We have some beliefs about the underlying probability
of different stimuli (e.g., “wildebeast” more common than “lion”)
What does it mean to perceive optimally?
(better to say “act”)
“Bayesian Decision Theory”
Solution: Given a noisy measurement, pick stimulus that
minimizes the expected cost.
Assumptions:
noise distribution / likelihood
1. Our senses provide noisy measurements of the
environment (e.g., “something moving in the grass”)
cost function
2. We have some well-defined (deterministic) cost
function describing the cost of making different errors
(e.g., cost saying “wildebeest” when the true stimulus was “lion”)
prior
3. We have some beliefs about the underlying probability
of different stimuli (e.g., “wildebeest” more common than “lion”)
Two basic approaches:
1. “Objective” approach: measure these three ingredients
in the real world. Determine if observers are in fact optimal.
(“Bayesian ideal observer analysis”)
What is the true sensory noise?
(Or impose noise that swamps internal
noise.)
noise distribution / likelihood
What is the true cost of saying “wildebeest” when it’s
a lion, vs. saying “lion” when it’s a wildebeest?
(Or impose costs: e.g., pay subjects)
Measure the relative population size of wildebeests and lions
(Or impose prior in an artificial task).
• Definitive answer: “yes” or “no”.
cost function
prior
Can quantify how close observers are to optimal
Two basic approaches:
2. “Subjective” approach: measure stimuli and observer
responses. Is there some setting of these three under which
observer can be said to be Bayes optimal?
Under-constrained problem in general - have to make some assumptions
Assume to be Gaussian
(estimate variance)
noise distribution / likelihood
Assume some standard choice
(e.g., mean-squared error)
Estimate this!
cost function
prior
• Not clear if the answer is ever “no”. (Would anyone publish a null result?)
• However: do the noise, cost and prior make sense? Is it reasonable to think that
this is what subjects are really doing? (Offers parsimonious explanation of percepts)