Why is Spatial Stereoacuity so Poor?
Martin S. Banks
Sergei Gepshtein
School of Optometry, Dept. of Psychology
UC Berkeley
Vision Science Program
UC Berkeley
Michael S. Landy
Dept. of Psychology, Center for Neural Science
NYU
Supported by NIH
Depth Perception
Depth Perception
How precise is the depth map generated from disparity?
Precision of Stereopsis
from Tyler (1977)
Stereo precision measured in various ways
A: Precision of detecting depth change on line of sight
D: Precision of detecting spatial variation in depth
Precision of Stereopsis
from Tyler (1977)
Stereo precision measured in various ways
A: Detect depth change on line of sight
D: Precision of detecting spatial variation in depth
Precision of Stereopsis
from Tyler (1977)
Stereo precision measured in various ways
A: Detect depth change on line of sight
D: Detect spatial variation in depth
Spatial Stereoacuity
• Modulate disparity
sinusoidally creating
corrugations in depth.
• Least disparity
required for detection
as a function of
spatial frequency of
corrugations:
“Disparity MTF”.
• Index of precision of
depth map.
Disparity MTF
•Disparity modulation
threshold as a
function of spatial
frequency of
corrugations.
•Bradshaw & Rogers
(1999).
•Horizontal & vertical
corrugations.
•Disparity MTF:
acuity = 2-3 cpd;
peak at 0.3 cpd.
Luminance Contrast Sensitivity & Acuity
• Luminance contrast sensitivity
function (CSF): contrast for
detection as function of spatial
frequency.
• Proven useful for characterizing
limits of visual performance and for
understanding optical, retinal, &
post-retinal processing.
• Highest detectable spatial
frequency (grating acuity): 40-50
c/deg.
Disparity MTF
Spatial stereoacuity
more than 1 log unit
lower than
luminance acuity.
Disparity MTF
Spatial stereoacuity
more than 1 log unit
lower than
luminance acuity.
Why is spatial
stereoacuity so
low?
Likely Constraints to Spatial Stereoacuity
1. Sampling constraints in the stimulus: Stereoacuity
measured using random-element stereograms. Discrete
sampling limits the highest spatial frequency one can
reconstruct.
2. Disparity gradient limit: With increasing spatial frequency,
the disparity gradient increases. If gradient approaches 1.0,
binocular fusion fails.
3. Spatial filtering at the front end: Optical quality & retinal
sampling limit acuity in other tasks, so probably limits
spatial stereoacuity as well.
4. The correspondence problem: Manner in which binocular
matching occurs presumably affects spatial stereoacuity.
Likely Constraints to Spatial Stereoacuity
1. Sampling constraints in the stimulus: Stereoacuity
measured using random-element stereograms. Discrete
sampling limits the highest spatial frequency one can
reconstruct.
2. Disparity gradient limit: With increasing spatial frequency,
the disparity gradient increases. If gradient approaches 1.0,
binocular fusion fails.
3. Spatial filtering at the front end: Optical quality & retinal
sampling limit acuity in other tasks, so probably limits
spatial stereoacuity as well.
4. The correspondence problem: Manner in which binocular
matching occurs presumably affects spatial stereoacuity.
Spatial Sampling Limit: Nyquist Frequency
• Signal reconstruction from
discrete samples.
• At least 2 samples
required per cycle.
• In 1d, highest recoverable
spatial frequency is
Nyquist frequency:
1
fN 
2N
where N is number of
samples per unit distance.
Spatial Sampling Limit: Nyquist Frequency
• Signal reconstruction from 2d discrete samples.
• In 2d, Nyquist frequency is:
1
fN 
2
N
A
where N is number of samples in area A.
Methodology
• Random-dot stereograms with sinusoidal disparity
corrugations.
• Corrugation orientations: +/-20 deg (near horizontal).
• Observers identified orientation in 2-IFC psychophysical
procedure; phase randomized.
• Spatial frequency of corrugations varied according to adaptive
staircase procedure.
• Spatial stereoacuity threshold obtained for wide range of dot
densities.
• Duration = 600 msec; disparity amplitude = 16 minarc.
Stimuli
Spatial Stereoacuity as a function of Dot Density
•Acuity
proportional
to dot
density
squared.
•Scale
invariance!
•Asymptote
at high
density.
Spatial Stereoacuity (c/deg)
JMA
MSB
a d
1.0
1
Modulation
Amplitude
Viewing
Distance
16 min
39 cm
0.1
0.1
DMV
TMG
1.0
1
0.1
0.1
0.1
0.1
1.0
1
10
10
100
100
0.1
0.1
1.0
1
Dot Density (dots/deg2)
10
10
100
100
Spatial Stereoacuity & Nyquist Limit
• Calculated
Nyquist
frequency
for our
displays.
Spatial Stereoacuity (c/deg)
JMA
MSB
1.0
1
Modulation
Amplitude
Viewing
Distance
16 min
39 cm
0.1
0.1
DMV
TMG
1.0
1
0.1
0.1
0.1
0.1
1.0
1
10
10
100
100
0.1
0.1
1.0
1
Dot Density (dots/deg2)
10
10
100
100
Spatial Stereoacuity & Nyquist Limit
• Calculated
Nyquist
frequency
for our
displays.
• Acuity
approx.
equal to
Nyquist
frequency
except at
high
densities.
Spatial Stereoacuity (c/deg)
JMA
Nyquist frequency
MSB
1.0
1
Modulation
Amplitude
Viewing
Distance
16 min
39 cm
0.1
0.1
DMV
TMG
1.0
1
0.1
0.1
0.1
0.1
1.0
1
10
10
100
100
0.1
0.1
1.0
1
Dot Density (dots/deg2)
10
10
100
100
Types of Random-element Stereograms
Jittered-lattice: dots displaced randomly from regular lattice
Sparse random: dots positioned randomly
Spatial Sampling Limit: Nyquist Frequency
Same acuities with jittered-lattice and sparse random stereograms.
Both follow Nyquist limit at low densities.
Likely Constraints to Spatial Stereoacuity
1. Sampling constraints in the stimulus: Stereoacuity
measured using random-element stereograms. Discrete
sampling limits the highest spatial frequency one can
reconstruct.
2. Disparity gradient limit: With increasing spatial frequency,
the disparity gradient increases. If gradient approaches 1.0,
binocular fusion fails.
3. Spatial filtering at the front end: Optical quality & retinal
sampling limit acuity in other tasks, so probably limits
spatial stereoacuity as well.
4. The correspondence problem: Manner in which binocular
matching occurs presumably affects spatial stereoacuity.
Disparity Gradient
P1
P2
Disparity gradient = disparity / separation
= (aR – aL) / [(aL + aR)/2]
aL
aR
Disparity Gradient
P1
P1 & P2 on horopter
aR = aL, so disparity = 0
Disparity gradient = 0
P2
Disparity Gradient
P1
P1 & P2 on cyclopean line of sight
aR = -aL, so separation = 0
Disparity gradient =

P2
Disparity Gradient
Disparity gradient for
different directions.
P1
P2 (left & right
horizontal
disparity
eyes)
Disparity Gradient Limit
• Burt & Julesz (1980): fusion
as function of disparity,
separation, & direction (tilt).
P1
• Set direction & horizontal
disparity and found smallest
fusable separation.
direction
disparity
P2 (left & right
eyes)
Disparity Gradient Limit
• Fusion breaks when
disparity gradient
reaches constant
value.
• Critical gradient = ~1.
• “Disparity gradient
limit”.
• Limit same for all
directions.
Disparity Gradient Limit
Disparity Gradient Limit
• Panum’s fusion
area (hatched).
• Disparity gradient
limit means that
fusion area
affected by nearby
objects (A).
• Forbidden zone is
conical (isotropic).
Disparity Gradient & Spatial Frequency
• Disparity gradient for
sinusoid is indeterminant.
• We may have approached
disparity gradient limit.
• Tested by reducing disparity
amplitude from 16 to 4.8
minarc.
Disparity (deg)
• But for fixed amplitude,
gradient proportional to
spatial frequency.
highest gradient
peak-trough
gradient
Horizontal Position (deg)
Spatial Stereoacuity & Disparity Gradient Limit
• Reducing
disparity
amplitude
increases
acuity at high
dot densities
(where DG is
high).
• Lowers acuity
slightly at low
densities
(where DG is
low).
Spatial Stereoacuity (c/deg)
JMA
Nyquist frequency
MSB
1.0
1
Modulation
Amplitude
Viewing
Distance
16 min
39 cm
4.8 min
39 cm
0.1
0.1
DMV
TMG
1.0
1
0.1
0.1
0.1
0.1
1.0
1
10
10
100
100
0.1
0.1
1.0
1
Dot Density (dots/deg2)
10
10
100
100
Likely Constraints to Spatial Stereoacuity
1. Sampling constraints in the stimulus: Stereoacuity
measured using random-element stereograms. Discrete
sampling limits the highest spatial frequency one can
reconstruct.
2. Disparity gradient limit: With increasing spatial frequency,
the disparity gradient increases. If gradient approaches 1.0,
binocular fusion fails.
3. Spatial filtering at the front end: Optical quality & retinal
sampling limit acuity in other tasks, so probably limits
spatial stereoacuity as well.
4. The correspondence problem: Manner in which binocular
matching occurs presumably affects spatial stereoacuity.
Stereoacuity & Front-end Spatial Filtering
• Low-pass spatial filtering at front-end of visual system
determines high-frequency roll-off of luminance CSF.
• Tested similar effects on spatial stereoacuity by:
1. Decreasing retinal image size of dots by increasing
viewing distance.
2. Measuring stereoacuity as a function of retinal
eccentricity.
3. Measuring stereoacuity as a function of blur.
Stereoacuity & Front-end Spatial Filtering
• Low-pass spatial filtering at front-end of visual system
determines high-frequency roll-off of luminance CSF.
• Tested similar effects on spatial stereoacuity by:
1. Decreasing retinal image size of dots by increasing
viewing distance.
2. Measuring stereoacuity as a function of retinal
eccentricity.
3. Measuring stereoacuity as a function of blur.
Spatial Stereoacuity at Higher Densities
• Monocular
artifacts at
high dot
densities.
• Reduce dot
size to test
upper limit.
• Increase
viewing
distance from
39-154 cm.
• Acuity still
levels off, but
at higher
value.
SpatialStereoacuity
Stereoacuity (c/deg)
(c/deg)
Spatial
JMA
MSB
Nyquist frequency
1
1.0
Modulation Viewing
Amplitude Distance
4.8 min
39 cm
4.8 min
154 cm
0.1
0.1
DMV
TMG
1
1.0
0.1
0.1
0.1
0.1
1.0
1
10
10
100
100
0.1
0.1
1.0
1
2
Dot Dot
Density
Density(dots/deg
(dot/deg2) )
10
10
100
100
Stereoacuity & Front-end Spatial Filtering
• Low-pass spatial filtering at front-end of visual system
determines high-frequency roll-off of luminance CSF.
• Tested similar effects on spatial stereoacuity by:
1. Decreasing retinal image size of dots by increasing
viewing distance.
2. Measuring stereoacuity as a function of retinal
eccentricity.
3. Measuring stereoacuity as a function of blur.
Spatial Stereoacuity & Retinal Eccentricity
•Elliptical patch with
sinusoidal corrugation.
•Patch centered at one of
three eccentricities (subject
dependent).
•Eccentricity random;
duration = 250 ms.
fixation point
eccentricity
•Same task as before.
4 deg
•Again vary dot density.
8 deg
Spatial Stereoacuity & Retinal Eccentricity
Spatial Stereoacuity (c/deg)
Spatial Stereoacuity & Retinal Eccentricity
YHH
TMG
SSG
1.01
Retinal eccentricity
0 deg
6.2
12.4
0 deg
5.2
10.4
0.1
0.1
0.1
0.1
1
1.0
10
10
100
100
1
1.0
10
10
100
100
0 deg
6.8
13.6
1
1.0
10
10
Dot Density (dots/deg2)
•Same acuities at low dot densities; Nyquist.
•Asymptote varies significantly with retinal eccentricity.
100
100
Stereoacuity & Front-end Spatial Filtering
• Low-pass spatial filtering at front-end of visual system
determines high-frequency roll-off of luminance CSF.
• Tested similar effects on spatial stereoacuity by:
1. Decreasing retinal image size of dots by increasing
viewing distance.
2. Measuring stereoacuity as a function of retinal
eccentricity.
3. Measuring stereoacuity as a function of blur.
Spatial Stereoacuity & Blur
• We examined effect of blur on foveal spatial stereoacuity.
• Three levels of blur introduced with diffusion plate:
no blur (s = 0 deg)
low blur (s = 0.12)
high blur (s = 0.25)
Spatial Stereoacuity & Blur
• We examined effect of blur on foveal spatial stereoacuity.
• Three levels of blur introduced with diffusion plate:
no blur (s = 0 deg)
low blur (s = 0.12)
high blur (s = 0.25)
Spatial Stereoacuity & Blur
•Same acuities at low dot densities; Nyquist.
•Asymptote varies significantly with spatial lowpass filtering.
Likely Constraints to Spatial Stereoacuity
1. Sampling constraints in the stimulus: Stereoacuity
measured using random-element stereograms. Discrete
sampling limits the highest spatial frequency one can
reconstruct.
2. Disparity gradient limit: With increasing spatial frequency,
the disparity gradient increases. If gradient approaches 1.0,
binocular fusion fails.
3. Spatial filtering at the front end: Optical quality & retinal
sampling limit acuity in other tasks, so probably limits
spatial stereoacuity as well.
4. The correspondence problem: Manner in which binocular
matching occurs presumably affects spatial stereoacuity.
Binocular Matching by Correlation
1. Binocular matching by correlation: basic and well-studied
technique for obtaining depth map from binocular images.
Computer vision: Kanade & Okutomi (1994); Panton (1978)
Physiology: Ohzawa, DeAngelis, & Freeman (1990);
Cumming & Parker (1997)
2. We developed a cross-correlation algorithm for binocular
matching & compared its properties to the psychophysics.
Binocular Matching by Correlation
• Compute crosscorrelation
between eyes’
images.
• Window in left
eye’s image
moved
orthogonal to
signal.
• For each
position in left
eye, window in
right eye’s image
moved
horizontally &
cross-correlation
computed.
Left eye’s image
Right eye’s image
Binocular Matching by Correlation
• Compute crosscorrelation
between eyes’
images.
• Window in left
eye’s image
moved
orthogonal to
signal.
• For each
position in left
eye, window in
right eye’s image
moved
horizontally &
cross-correlation
computed.
Left eye’s image
Right eye’s image
Binocular Matching by Correlation
• Compute crosscorrelation
between eyes’
images.
• Window in left
eye’s image
moved
orthogonal to
signal.
• For each
position in left
eye, window in
right eye’s image
moved
horizontally &
cross-correlation
computed.
Left eye’s image
Right eye’s image
Binocular Matching by Correlation
Plot correlation as a
function of position in
left eye (red arrow) &
relative position in
right eye (blue arrow);
disparity.
Position in right eye’s image
Correlation (gray
value) high where
images similar & low
where dissimilar.
Y axis
Position in left eye’s image
Spatial Frequency
Examples of Output
1
0.1
0.1
1
10
100
Dot Density
Dot density: 16 dots/deg2
Spatial frequency: 1 c/deg
Window size: 0.2 deg
Disparity waveform evident
in output
Effect of Window Size & Dot Density
Effect of Window Size & Dot Density
Correlation window
must be large
enough to contain
sufficient luminance
variation to find
correct matches
w2 
0.7
d
Effect of Window Size & Spatial Frequency
Effect of Window Size & Spatial Frequency
When significant
depth variation is
present in a region,
window must be
small enough to
respond differentially
w
0.5
f
Window Size, # Samples, & Spatial Frequency
From two constraints:
w2 
w
0.7
0.5
d
f
then substitute for w,
take log:
1
log f  log d  0.22
2
Effect of Disparity Gradient
• Fix spatial frequency, dot
density, & window size.
• Increase disparity amplitude
(which also increases disparity
gradient: 0.21, 0.59, 1.77).
• As approach 1.0, disparity
estimation becomes poor.
Images too dissimilar in two
eyes.
• Matching by correlation yields
piecewise frontal estimates.
Effect of Disparity Gradient
• Fix spatial frequency, dot
density, & window size.
• Increase disparity amplitude
(which also increases disparity
gradient: 0.21, 0.59, 1.77).
• As approach 1.0, disparity
estimation becomes poor.
Images too dissimilar in two
eyes.
• Matching by correlation yields
piecewise frontal estimates.
Effect of Low-pass Spatial Filtering
• Amount of variation in image
dependent on spatialfrequency content.
• If s proportional to w and
inversely proportional to d ,
variation constant in
cycles/window.
• Algorithm yields similar
outputs for these images.
• For each s, there’s a window
just large enough to yield good
disparity estimates.
• Highest detectable spatial
frequency inversely
proportional to s.
Effect of Low-pass Spatial Filtering
• Spatial stereoacuity for different amounts of blur.
• s: all filtering elements: dots, optics, diffusion screen.
• Horizontal lines: predictions for asymptotic acuities.
• Asymptotic acuity limited by filtering before binocular combination.
Summary of Matching Effects
Correlation algorithm reveals two effects.
1. Disparity estimation is poor when there’s insufficient
intensity variation within correlation window.
w 2  0.7
a. when window too small for presented dot density
b. when spatial-frequency content is too low.
c. employ a larger window (or receptive field).
2. Disparity estimation is poor when correlation window
is too large in direction of maximum disparity
gradient.
a. when window width greater than half cycle of stimulus.
b. employ a smaller window (or receptive field).
w  0.5
d
f
Summary
1. Sampling constraints in the stimulus: Stereoacuity follows
Nyquist limit for all but highest densities. Occurs in
peripheral visual field and in fovea with blur.
2. Disparity gradient limit: Stereoacuity reduced at high
gradients.
3. Spatial filtering at the front end: Low-pass filtering before
binocular combination determines asymptotic acuity.
4. The correspondence problem: Binocular matching by
correlation requires sufficient information in correlation
window & thereby reduces highest attainable acuity. Visual
system measures disparity in piecewise frontal fashion.
Depth Map from Disparity
depth-varying scene
Disparity estimates are piecewise frontal.
Only one perceived depth per direction.