Motion Illusions As Optimal Percepts
What’s Special About Perception?



Arguably, visual perception is better optimized by
evolution than other cognitive abilities.
Human visual perception outperforms all modern
computer vision systems.
Understanding human vision will be helpful for
building AI systems
Ambiguity of Perception


One-to-many mapping of retinal image to objects in
the world
Same issue with 2D retina and 3D images
Hermann von Helmholtz
(1821-1894)


German physician/physicist who made
significant contributions to theories of
vision
Perception as unconscious inference
 Recover the most likely objects in the world based
on the ambiguous visual evidence

Percept is a hypothesis about what the brain thinks is
out there in the world.
Additional Knowledge
Is Required To Perceive
•
Innate knowledge
– E.g., any point in the image has only one interpretation
– E.g., surfaces of an object tend to
be a homogeneous color
– Gestalt grouping principles
•
Specific experience
– E.g., SQT is an unlikely letter
combination in English
– E.g., bananas are yellow or
green, not purple
Illusions
•
•
Most of the time, knowledge helps constrain
perception to produce the correct interpretation of
perceptual data.
Illusions are the rare cases where knowledge misleads
– E.g., hollow face illusion
– http://www.michaelbach.de/ot/fcs_hollow-face/
– Constraints: light source, shading cues, knowledge of
faces
The Aperture Problem
Some slides adapted from Alex Pouget, Rochester
The Aperture Problem
The Aperture Problem
(deg/s)
Vertical velocity
velocity
vertical
The Aperture Problem
horizontal
velocity
Horizontal velocity
(deg/s)
The Aperture Problem: Plaid
Vertical velocity (deg/s)
The Aperture Problem: Plaid
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
The Aperture Problem: Rhombus
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
The Aperture Problem
Horizontal velocity (deg/s)
Actual motion in blue
Standard Models of Motion Perception



IOC: intercept of constraints
VA: vector average
Feature tracking: focus on some distinguishing feature
of display (e.g., max luminance)
Which model best fits data depends on speed,
contrast, presentation time, retinal location, etc.
Maybe perception is an ad hoc combination of models,
but that’s neither elegant nor parsimonious.
Standard Models of Motion Perception
IOC
Vertical velocity (deg/s)
VA
Horizontal velocity (deg/s)
Standard Models of Motion Perception
IOC
Vertical velocity (deg/s)
VA
Horizontal velocity (deg/s)
Standard Models of Motion Perception

Problem: perceived motion is close to either IOC or VA
depending on stimulus duration, eccentricity, contrast
and other factors.
Standard Models of Motion Perception
Example: Rhombus With Corners Occluded
Actual motion
Percept: IOC
VA
Percept: VA
IOC
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
Vertical velocity (deg/s)

VA
IOC
Horizontal velocity (deg/s)
Rhombus Thickness Influences Perception
Bayesian Model of Motion Perception

Perceived motion correspond to the Maximum a
Posteriori (MAP) estimate
( )
P( I | v) P( v)
P( v | I ) =
µ P( I | v) P( v)
P( I )
= P( v)Õ P( I ( x , y ) | v)
v = arg max P v | I
*
v
i
i
i
Independence of observations
Prior
Weiss and Adelson:
Human observers favor slow motions
50
Vertical Velocity

0
-50
-50
( )
0
50
Horizontal Velocity
(
2
P v µ exp - v / 2s 2p
)
Likelihood
Weiss and Adelson
50
Vertical Velocity

0
-50
-50
0
50
Horizontal Velocity
Likelihood
( ) (
)
I ( x, y,t ) - I ( x - v Dt, y - v Dt,t - Dt ) = h
I x, y,t = I x - v x Dt, y - v y Dt,t - Dt + h
x
y
(
(
æ
I ( xi , yi ,t ) - I x - v x Dt, y - v y Dt,t - Dt
ç
P I ( xi , yi ,t ) | v µ exp 2
ç
2
s
çè
(
)
(
) ( )
I ( x , y ,t ) - I ( x - v Dt, y - v Dt,t - Dt ) = I v Dt + I v Dt + I Dt
= ( I v + I v + I ) Dt
)) ö÷
2
÷
÷ø
I xi - v x Dt, yi - v y Dt,t - Dt » I xi , yi ,t - I x v x Dt - I y v y Dt - I t Dt
i
i
i
x
i
y
x
x
x
y
x
y
(
y
y
t
æ I v +I v +I
x x
y y
t
P I xi , yi ,t | v µ exp ç ç
2s 2
è
((
) )
First-order
Taylor series
expansion
t
) ö÷
2
÷
ø
Likelihood
((
) )
((
) )
P I x, y,t | v = Õ P I xi , yi ,t | v
i
(
)
2ö
æ 1
µ Õ exp ç - 2 I x v x + I y v y + I t ÷
è 2s
ø
i
æ 1
µ exp ç - 2 å I x v x + I y v y + I t
è 2s
x, y
(
)
2
ö
dxdy ÷
ø
Posterior
( ( (
log P v | I x, y,t
))) = log ( P ( I ( x, y,t ) | v )) + log ( P ( v )) + C
2ö
æ
1
1
2
2
= ç å - 2 I x v x + I y v y + It ÷ v
+
v
+C
x
y
2
è x,y 2s
ø 2s p
(
)
(
)
Bayesian Model of Motion Perception

Perceived motion corresponds to the MAP estimate
Posterior is Gaussian → MAP is mean
(
) (
)
( )Õ P( I ( x , y ) | v)
v = arg max P v | I , P v | I = P v
*
v
æ
ç
ç
ç
ç
ç
è
2
s
2
I
å x +s2
p
åI I
åI I
s
åI + s2
p
x y
x y
2
2
y
i
ö
÷
æ
÷ *
ç
v
=
÷
ç
è
÷
÷
ø
i
i
ö
I
I
åxt ÷
å I y It ÷ø
Only one free parameter
Likelihood
- L(v) =
1
2s 2
å(
I x v x + I y v y + It
x,y
)
2
(
(
+
1
2s 2p
å(
x, y
)
)
æ 2
I x v x + I y v y + It I x
å
¶L(v)
1 ç x,y
=- 2ç
¶v
2s ç 2å I v + I v + I I
y y
t
y
çè x, y x x
ææ
çç
1 çç
= - 2 çç
s ç
ç
çç
èè
2
s
2
I
å x +s2
x,y
p
åI
x,y
I
y x
åI
I
y x
x,y
2
s
2
I
å y +s2
x, y
p
v 2x + v 2y
ö
æ
÷
ç
÷
÷ v+ç
ç
÷
è
÷
ø
)
ö
æ 2 v
å
x
÷
1 ç x,y
÷2 ç
2
s
÷
p ç 2å v y
÷ø
è x, y
åI I
t x
x, y
åI I
t y
x,y
ö
ö÷
÷÷
÷÷
÷÷
ø÷
ø
ö
÷
÷
÷
ø
Motion Through An Aperture
Vertical Velocity
Likelihood
50
0
-50
0
50
Horizontal Velocity
50
0
-50
Prior
ML
Vertical Velocity
Vertical Velocity
-50
50
0
MAP
-50
-50
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Motion And Contrast

Driving in the fog
 Drivers tend to speed up

In low contrast situations, the prior dominates
 In fog, poor quality visual information about speed
 Priors biased toward slow speeds
Motion And Constrast

Individuals tend to underestimate velocity in low
contrast situations
 perceived speed of lower-contrast grating relative to
higher-contrast grating
Perceived Velocity And Contrast
Vertical Velocity
Likelihood
50
0
-50
High
Contrast
0
50
Horizontal Velocity
50
0
-50
Prior
ML
Vertical Velocity
Vertical Velocity
-50
50
0
MAP
-50
-50
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Perceived Velocity And Contrast
Vertical Velocity
Likelihood
50
0
-50
Low
Contrast
0
50
Horizontal Velocity
50
0
-50
Prior
ML
Vertical Velocity
Vertical Velocity
-50
50
MAP
0
-50
-50
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Perceived Direction And Contrast
Likelihood
50
Vertical Velocity
Vertical Velocity
50
0
-50
High
Contrast
-50
-50
0
50
Horizontal Velocity
-50
Vertical Velocity
Vertical Velocity
50
0
-50
0
50
Horizontal Velocity
50
IOC
MAP
0
-50
-50
Prior
0
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Influence Of Contrast On Perceived Direction
Likelihood
50
Vertical Velocity
Vertical Velocity
50
0
-50
Low
Contrast
-50
-50
0
50
Horizontal Velocity
-50
Vertical Velocity
Vertical Velocity
50
0
-50
0
50
Horizontal Velocity
IOC
50
MAP
0
-50
-50
Prior
0
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Perceived Direction And Contrast


Low contrast -> greater uncertainty in motion direction
Blurred information from two edges can combine if edges have similar angles
Influence Of Edge Angles
On Perceived Direction Of Motion
Example: Rhombus
Actual motion
Percept: IOC
VA
Percept: VA
IOC
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
Vertical velocity (deg/s)

VA
IOC
Horizontal velocity (deg/s)
Influence Of Edge Angles
On Perceived Direction Of Motion

Type II plaids
True velocity is not between the two surface normals

Vary angle between plaid components
Analogous to varying shape of rhombus

Greater alignment of edges -> less benefit of
combining information from the two edges
Interaction of Edge Angle With Contrast

More alignment with acute angle
-> Union vs. intersection of edge information at low
contrast with acute angle
Actual motion
VA
IOC
Horizontal velocity (deg/s)
Vertical velocity (deg/s)

More uncertainty with low contrast
Vertical velocity (deg/s)

VA
IOC
Horizontal velocity (deg/s)
Barberpole Illusion (Weiss thesis)
Actual motion
Perceived motion
Plaid Motion: Type I and II


Type I: true velocity
lies between two
normals
Type II: true
velocity lies outside
two normals
Plaids and Relative Contrast
Lower
contrast
Plaids and Speed

Perceived direction of type II plaids depends on
relative speed of components
Plaids and Time

Viewing time reduces uncertainty
Motion Illusions As Optimal Percepts


Mistakes of perception are the result of a rational
system designed to operate in the presence of
uncertainty.
A proper rational model incorporates actual statistics
of the environment
 Here, authors assume without direct evidence:
(1) preference for slow speeds
(2) noisy local image measurements
(3) velocity estimate is the mean/mode of posterior
distribution

“Optimal Bayesian estimator” or “ideal observer” is
relative to these assumptions