Motion Illusions As Optimal Percepts
What’s Special About Perception?

Visual perception important for survival
Likely optimized by evolution
 at least more so than other cognitive abilities

Human visual perception outperforms all modern
computer vision systems.
 Understanding human vision should be helpful for
building AI systems
Ambiguity of Perception


One-to-many mapping of retinal image to objects in
the world
Same issue with 2D retina and 3D images
Hermann von Helmholtz
(1821-1894)


German physician/physicist who made
significant contributions to theories of
vision
Perception as unconscious inference
 Recover the most likely objects in the world based on
the ambiguous visual evidence

Percept is a hypothesis about what the brain thinks is
out there in the world.
Additional Knowledge
Is Required To Perceive
•
Innate knowledge
– E.g., any point in the image has only one interpretation
– E.g., surfaces of an object tend to
be a homogeneous color
– Gestalt grouping principles
•
Specific experience
– E.g., SQT is an unlikely letter
combination in English
– E.g., bananas are yellow or
green, not purple
Illusions
•
•
Most of the time, knowledge helps constrain
perception to produce the correct interpretation of
perceptual data.
Illusions are the rare cases where knowledge misleads
– E.g., hollow face illusion
– http://www.michaelbach.de/ot/fcs_hollow-face/
– Constraints: light source, shading cues, knowledge of
faces
The Aperture Problem
Some slides adapted from Alex Pouget, Rochester
The Aperture Problem
(deg/s)
Vertical velocity
velocity
vertical
The Aperture Problem
horizontal
velocity
Horizontal velocity
(deg/s)
The Aperture Problem: Plaid
Vertical velocity (deg/s)
The Aperture Problem: Plaid
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
The Aperture Problem: Rhombus
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
The Aperture Problem
Horizontal velocity (deg/s)
Actual motion in blue
Standard Models of Motion Perception

Feature tracking
 focus on distinguishing features

IOC
 intercept of constraints

VA
 vector average
Standard Models of Motion Perception
IOC
Vertical velocity (deg/s)
VA
Horizontal velocity (deg/s)
Standard Models of Motion Perception
IOC
Vertical velocity (deg/s)
VA
Horizontal velocity (deg/s)
Standard Models of Motion Perception

Problem
 Perceived motion is close to either IOC or VA
depending on stimulus duration, retinal eccentricity,
contrast, speed, and other factors.
Maybe perception is an ad hoc combination of models,
but that’s neither elegant nor parsimonious.
Standard Models of Motion Perception
Example: Rhombus With Corners Occluded
Actual motion
Percept: VA
Actual motion
Percept: IOC
VA
IOC
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
Vertical velocity (deg/s)

VA
IOC
Horizontal velocity (deg/s)
Rhombus Thickness Influences Perception

rhombus demo
Bayesian Model of Motion Perception

Perceived motion correspond to the Maximum a
Posteriori (MAP) estimate
(
v * = arg max P v | I
v
)
 v: velocity vector
 I: snapshot of image at 2 consecutive moments in time
* Digression *

Maximum a posteriori
( )
P( I | v) P( v)
= arg max
P( I )
= arg max P ( I | v ) P ( v )
v = arg max P v | I
*
v
v
v

Maximum likelihood
(
v * = arg max P I | v
v
)
Bayesian Model of Motion Perception

Perceived motion corresponds to the Maximum a
Posteriori (MAP) estimate
(
v * = arg max P v | I
v
)
(
)
(
)
P( v | I ) =
P( I )
µ P( v) P( I | v)
µ P( v)Õ P( I ( x , y ) | v)
P I |v P v
Conditional independence
of observations
i
i
i
Prior
Weiss and Adelson:
Human observers favor slow motions
50
Vertical Velocity

0
-50
-50
( )
0
50
Horizontal Velocity
(
2
P v µ exp - v / 2s 2p
)
Likelihood
Weiss and Adelson
50
Vertical Velocity

0
-50
-50
0
50
Horizontal Velocity
Likelihood
( ) (
)
I ( x, y,t ) - I ( x - v Dt, y - v Dt,t - Dt ) = h
I x, y,t = I x - v x Dt, y - v y Dt,t - Dt + h
x
y
(
(
æ
I ( xi , yi ,t ) - I x - v x Dt, y - v y Dt,t - Dt
ç
P I ( xi , yi ,t ) | v µ exp 2
ç
2
s
çè
(
)
(
) ( )
I ( x , y ,t ) - I ( x - v Dt, y - v Dt,t - Dt ) = I v Dt + I v Dt + I Dt
= ( I v + I v + I ) Dt
)) ö÷
2
÷
÷ø
I xi - v x Dt, yi - v y Dt,t - Dt » I xi , yi ,t - I x v x Dt - I y v y Dt - I t Dt
i
i
i
x
i
y
x
x
x
y
x
y
(
y
y
t
æ I v +I v +I
x x
y y
t
P I xi , yi ,t | v µ exp ç ç
2s 2
è
((
) )
First-order
Taylor series
expansion
t
) ö÷
2
÷
ø
Likelihood
((
) )
((
) )
P I x, y,t | v = Õ P I xi , yi ,t | v
i
(
)
2ö
æ 1
µ Õ exp ç - 2 I x v x + I y v y + I t ÷
è 2s
ø
i
æ 1
µ exp ç - 2 å I x v x + I y v y + I t
è 2s
x, y
(
)
2
ö
dxdy ÷
ø
Posterior
( ( ( )))
= log ( P ( I ( x, y,t ) | v )) + log ( P ( v )) + C
log P v | I x, y,t
2ö
æ
1
1
2
2
= ç å - 2 I x v x + I y v y + It ÷ - 2 v x + v y + C
è x,y 2s
ø 2s p
(
)
(
)
Bayesian Model of Motion Perception

Perceived motion corresponds to the MAP estimate
(( ) )
v * = arg max P ( v | I ) = arg max P ( v ) Õ P I xi , y | v
v
æ
ç
ç
ç
ç
çè
v
i
Gaussian prior, Gaussian likelihood
→ Gaussian posterior
→ MAP is mean of Gaussian
2
s
2
I
å x +s2
p
åI I
åI I
s
åI +s2
p
x y
i
x y
2
2
y
ö
÷
æ
ö
I
I
å
x t
÷ *
ç
÷
÷ v = -ç
I
I
å
y
t ÷
è
ø
÷
÷ø
Only one free parameter
Likelihood
- L(v) =
1
2s
2
å( I v
x
+ I y v y + It
x
x,y
)
2
(
(
+
1
2s
å( v
2
p x, y
)
)
æ 2
I x v x + I y v y + It I x
å
¶L(v)
1 ç x,y
=- 2ç
¶v
2s ç 2å I v + I v + I I
y y
t
y
çè x, y x x
ææ
çç
1 çç
= - 2 çç
s ç
ç
çç
èè
=0
2
s
2
I
å x +s2
x,y
p
åI
x,y
I
y x
åI
I
y x
x,y
2
s
2
I
å y +s2
x, y
p
ö
æ
÷
ç
÷
÷ v+ç
ç
÷
è
÷
ø
2
x
+ v 2y
)
ö
æ 2 v
å
x
÷
ç
1
x,y
÷2 ç
÷ 2s p ç 2å v y
÷ø
è x, y
åI I
t x
x, y
åI I
t y
x,y
ö
ö÷
÷÷
÷÷
÷÷
ø÷
ø
ö
÷
÷
÷
ø
Motion Through An Aperture
Vertical Velocity
Likelihood
50
0
-50
0
50
Horizontal Velocity
50
0
-50
Prior
ML
Vertical Velocity
Vertical Velocity
-50
50
0
MAP
-50
-50
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Driving In The Fog

Drivers in the fog tend to speed up
 underestimation of velocity

Explanation
 Fog results in low contrast visual information
 In low contrast situations, poor quality visual information
about speed
 Priors biased toward slow speeds
 Prior dominates
Influence Of Contrast On Perceived Velocity
Vertical Velocity
Likelihood
50
0
-50
High
Contrast
0
50
Horizontal Velocity
50
0
-50
Prior
ML
Vertical Velocity
Vertical Velocity
-50
50
0
MAP
-50
-50
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Influence Of Contrast On Perceived Velocity
Vertical Velocity
Likelihood
50
0
-50
Low
Contrast
0
50
Horizontal Velocity
50
0
-50
Prior
ML
Vertical Velocity
Vertical Velocity
-50
50
MAP
0
-50
-50
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Influence Of Contrast On Perceived Direction

high vs. low contrast rhombus
Influence Of Contrast On Perceived Direction


Low contrast -> greater uncertainty in motion direction
Blurred information from two edges can combine if edges have similar angles
Influence Of Contrast On Perceived Direction
Likelihood
50
Vertical Velocity
Vertical Velocity
50
0
-50
High
Contrast
-50
-50
0
50
Horizontal Velocity
-50
Vertical Velocity
Vertical Velocity
50
0
-50
0
50
Horizontal Velocity
50
IOC
MAP
0
-50
-50
Prior
0
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Influence Of Contrast On Perceived Direction
Likelihood
50
Vertical Velocity
Vertical Velocity
50
0
-50
Low
Contrast
-50
-50
0
50
Horizontal Velocity
-50
Vertical Velocity
Vertical Velocity
50
0
-50
0
50
Horizontal Velocity
IOC
50
MAP
0
-50
-50
Prior
0
0
50
Horizontal Velocity
-50
0
50
Horizontal Velocity
Posterior
Influence Of Edge Angles
On Perceived Direction Of Motion
Example: Rhombus
Actual motion
Percept: IOC
VA
Percept: VA
IOC
Horizontal velocity (deg/s)
Vertical velocity (deg/s)
Vertical velocity (deg/s)

VA
IOC
Horizontal velocity (deg/s)

Greater alignment of edges -> less benefit of
combining information from the two edges
Barberpole Illusion (Weiss thesis)
Actual motion
Perceived motion
Motion Illusions As Optimal Percepts


Mistakes of perception are the result of a rational
system designed to operate in the presence of
uncertainty.
A proper rational model incorporates actual statistics
of the environment
 Here, authors assume without direct evidence:
(1) preference for slow speeds
(2) noisy local image measurements
(3) velocity estimate is the mean/mode of posterior
distribution

“Optimal Bayesian estimator” or “ideal observer” is
relative to these assumptions
Bonus

More demos
Motion And Constrast

Individuals tend to underestimate velocity in low
contrast situations
 perceived speed of lower-contrast grating relative to
higher-contrast grating
Influence Of Edge Angles
On Perceived Direction Of Motion

Type II plaids
True velocity is not between the two surface normals

Vary angle between plaid components
Analogous to varying shape of rhombus
Interaction of Edge Angle With Contrast

More alignment with acute angle
-> Union vs. intersection of edge information at low
contrast with acute angle
Actual motion
VA
IOC
Horizontal velocity (deg/s)
Vertical velocity (deg/s)

More uncertainty with low contrast
Vertical velocity (deg/s)

VA
IOC
Horizontal velocity (deg/s)
Plaid Motion: Type I and II


Type I: true velocity
lies between two
normals
Type II: true
velocity lies outside
two normals
Plaids and Relative Contrast
Lower
contrast
Plaids and Speed

Perceived direction of type II plaids depends on
relative speed of components
Plaids and Time

Viewing time reduces uncertainty