Effects of Viewing Geometry on
Combination of Disparity and
Texture Gradient Information
Michael S. Landy
James M. Hillis
Martin S. Banks
Outline
•
•
•
•
•
•
Background: Optimal cue combination
Methods: slant discrimination
Single-cue results
Two-cue results: perceived slant
Two-cue results: JNDs
Conclusions
Outline
•
•
•
•
•
•
Background: Optimal cue combination
Methods: slant discrimination
Single-cue results
Two-cue results: perceived slant
Two-cue results: JNDs
Conclusions
Sources of Depth Information
•
•
•
•
•
•
•
Motion Parallax
Occlusion
Stereo Disparity
Shading
Texture
Linear Perspective
Etc.
Depth Cues
•
•
•
•
•
•
•
Motion Parallax
Occlusion
Stereo Disparity
Shading
Texture
Linear Perspective
Etc.
Optimal Cue Combination:
Statistical Approach
If the goal is to produce an estimate with
minimal variance, and the cues are
uncorrelated, then the optimal estimate is a
weighted average
Sˆ  wt Sˆt  wd Sˆd ,
where
1/ 
wt 
1/  t2  1/  d2
2
t
1/ 
and wd 
.
2
2
1/  t  1/  d
2
d
Optimal Cue Combination:
Bayesian Inference Approach
From the Bayesian standpoint, the
measurements D and T each result in a
likelihood function
p(T | S ) and
p( D | S ).
These are combined with a prior distribution
p( S ).
Optimal Cue Combination:
Bayesian Inference Approach
From Bayes rule, and assuming conditional
independence of the cues, the posterior
distribution satisfies:
p(S | T , D)  p(T | S ) p( D | S ) p( S ).
Optimal Cue Combination:
Bayesian Inference Approach
Finally, assuming Gaussian likelihoods and
prior, it turns out that the maximum a
posteriori (MAP) estimate satisfies:
Sˆ  wt Sˆt  wd Sˆd  wp S p ,
where p stands for the prior which acts as if it
were an additional cue, and the weights are
again proportional to inverse variance.
Previous Qualitative Tests that Cue
Weights Depend on Reliability
•
•
•
•
•
•
Young, Landy & Maloney (1993)
Johnston, Cumming & Landy (1994)
Rogers and Bradshaw (1995)
Frisby, Buckley & Horsman (1995)
Backus and Banks (1999)
etc. etc.
Previous Quantitative Tests that Cue
Weights Depend on Reliability
• Landy & Kojima (2001) – texture cues to
location
• Ernst & Banks (2002) – visual and haptic
cues to size
• Gepshtein & Banks (2003) – visual and
haptic cues to size
• Knill & Saunders (2003) – texture and
disparity cues to slant
The Current Study
• Texture and disparity cues to slant
• Vary reliability by varying base slant (as in
Knill & Saunders, 2003) and distance
• Measure single-cue reliability
• Compare two-cue weights to predictions
• Compare two-cue reliability to predictions
Outline
•
•
•
•
•
•
Background: Optimal cue combination
Methods: slant discrimination
Single-cue results
Two-cue results: perceived slant
Two-cue results: JNDs
Conclusions
Types of Stimuli
• Disparity-only: sparse random dots
• Texture: Voronoi textures viewed
monocularly
• Two-cue stimuli: Voronoi texture
stereograms, both conflict and no-conflict
Stimuli – Disparity-only
Stimuli – Voronoi textures
Cue Conflict Stimuli
Methods
• Task: 2IFC slant discrimination
• Single-cue and two-cue blocks
• Opposite-sign slants mixed across trials in a
block to avoid slant adaptation
• One stimulus fixed, other varied by
staircase; several interleaved staircases
• Analysis: fit psychometric function to
estimate PSE and JND
Outline
•
•
•
•
•
•
Background: Optimal cue combination
Methods: slant discrimination
Single-cue results
Two-cue results: perceived slant
Two-cue results: JNDs
Conclusions
Single-cue JNDs: Texture
Single-cue JNDs: Disparity
Single-cue JNDs: Disparity
Predicted Cue Weights
Outline
•
•
•
•
•
•
Background: Optimal cue combination
Methods: slant discrimination
Single-cue results
Two-cue results: perceived slant
Two-cue results: JNDs
Conclusions
Cue Conflict Paradigm
Determination of PSEs
Determination of Weights
Full Two-Cue Dataset
ACH
JMH
Effect of Viewing Distance
Effect of Base Slant
Outline
•
•
•
•
•
•
Background: Optimal cue combination
Methods: slant discrimination
Single-cue results
Two-cue results: perceived slant
Two-cue results: JNDs
Conclusions
Improvement in Reliability with Cue
Combination
If the optimal weights are used:
1/  t2
wt 
2
2
1/  t  1/  d
1/  d2
and wd 
2
2
1/  t  1/  d
then the resulting variance
 
2
2
t d
2
t
2
d
is lower than that achieved by either cue alone.
Improvement in JND with 2 Cues
Conclusion
• The data are consistent with optimal cue
combination
• Texture weight is increased with increasing
distance and increasing base slant, as predicted
• Two cue JNDs are generally lower than the
constituent single-cue JNDs
• Thus, weights are determined trial-by-trial, based
on the current stimulus information and, in
particular, the two single-cue slant estimates
Are Cue Weights Chosen Locally?
Are Cue Weights Chosen Locally?