Introduction
Implementation
Results
Conclusion
Depth from Defocus
Daniel Kuntz
April 29, 2015
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Problem
Getting depth out of an image
Popular Approaches
Stereo Vision
Structured Light
A slightly stranger approach
Use image blur to estimate distance
Technique is called ”Depth from Defocus” (DfD)
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Why DfD?
DfD is attractive to estimating the depth of an image because
It avoids the occlusion problem that is common in wide
baseline techniques (stereo vision)
Has a circle of sample points instead of just two
In theory, it avoids the correspondence problem
Can be done with one camera (again, in theory).
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
What is Defocus?
1
1
Image: W.E. Crofts
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
Focus (continued)
The relationship between the focal length f and the distances u
and v is given by the thin lens equation
Thin Lens Equation
1
1
1
= +
f
u v
Daniel Kuntz
Depth from Defocus
(1)
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
Calculating Distance
Using geometry and the thin lens equation
u=
frv0
rv0 − f (r + σ)
(2)
Or with the lens’s ”F number”
u=
fv0
v0 − f − 2σF
Daniel Kuntz
Depth from Defocus
(3)
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
Point Spread Function
Light is distributed on the sensor via a Point Spread Function
(PSF):
PSF based on lens geometry
(
hu (x, y ) =
1
π(x 2 +y 2 )
x 2 + y 2 ≤ σ2
0
otherwise
(4)
A convenient approximation
1
hg (x, y ) =
exp
2πσ 2
Daniel Kuntz
−(x 2 + y 2 )
2σ 2
Depth from Defocus
(5)
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
Image Blur
We have seen that a blurry image contains information about
depth, how do we decode it? First, notice:
fb (x, y ) = h(x, y ) ∗ fc (x, y )
Where fc is the clear (un-blurred) image and fb is the image
blurred by the point spread function.
Daniel Kuntz
Depth from Defocus
(6)
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
Spatial Domain Analysis
0
Recall: u = v0 −ffv−2σF
So the important item to be calculated to get depth is the
diameter (or standard deviation) of the PSF. If we choose hg then:
hg (x, y ) ∗ fc (x, y ) = F {hg (x, y )} F {fc (x, y )}
= Hg (u, v )Fc (u, v )
= Fc (u, v ) exp −2π 2 (u 2 + v 2 )σ 2
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
Estimation of Blur
Consider:
Fc (u, v )Hg (u, v )
fb (x, y )
=
= Hg (u, v )
fc (x, y )
Fc (u, v )
(7)
The Fourier Transform of the clear image cancels out and the we
can calculate:
Blur Estimator 1
s
σ(u, v ) =
ln(Fc (u, v )) − ln(Fb (u, v ))
2π 2 (u 2 + v 2 )
Daniel Kuntz
Depth from Defocus
(8)
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
”S Transform” Approach
The S transform used by this paper is defined as:
SH {f (t)}(τ ) ,
N
X
(−1)k
k=0
k!
hk
dk
f (τ )
dτ k
(9)
Where hk is the kth moment of h(x, y ). It is used as the basis for
the deconvolution formula:
fc (x, y ) = fb (x, y ) −
Daniel Kuntz
σh2 2
∇ fb (x, y )
4
Depth from Defocus
(10)
Introduction
Implementation
Results
Conclusion
Spatial Method
S Transform Method
Estimation of Blur 2
Blur Estimator 2
σ(x, y ) = −
c(x, y )
b(x, y )
With:
rv0 1
1
β=√
−
2 f1 f2
1
b(x, y ) = β∇2 fb (x, y )
2
1
c(x, y ) = (fb (x, y ) − fc (x, y )) + β 2 ∇2 fb (x, y )
4
Daniel Kuntz
Depth from Defocus
(11)
Introduction
Implementation
Results
Conclusion
Results
Due to difficulties setting up the system (described later). No
results have yet been obtained. (Sorry!)
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Issues with DfD
DfD has a large number of practical problems that make it difficult
to implement.
Zoom must be normalized, or for accurate results, an
expensive telecentric camera must be used
Two cameras, with different parameters must take the same
picture at the same time for best results
The Laplacian operation used in the second formulation is
very sensitive to noise.
The blur circle is effectively a very small baseline, resulting in
high inaccuracy.
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Issues with DfD (continued)
Lens distortion effects make this method all but impossible to
implement with a cheaper camera
The thin lens and Gaussian PSF are not good approximations
of light behavior
Intrinsic parameters of the camera beyond the focal length
must be known in order to use the equations.
Very sensitive to changes in image intensity.
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Plan Forward
1
Generate simulated blurry test images using known camera
parameters and depth
2
Test Methods 1 and 2 outlined in this report on said
simulated images
3
Compare the results to the ground truth
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
DfD Applications
High end auto-focus systems on very expensive cameras
Measurement of malleable materials
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Conclusion
DfD is impractical for most applications.
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
References
W. E. Crofts, The Generation of Depth Maps via
Depth-from-Defocus, PhD Thesis, University of Warwick, 2007
M. Subbarao, G. Surya, Depth from Defocus: A spatial
Domain Approach, International Journal of Computer Vision,
December 1994
Y. Schechner, N. Kiryati, Depth from Defocus vs. Stereo: How
Different Really Are They?, International Journal of Computer
Vision, 2000
Y. Ding, J. Xiao, J. Yu, A Theory of Multi-Perspective
Defocusing, Computer Vision and Pattern Recognition, 2011
IEEE Conference on. pp 217-224
Daniel Kuntz
Depth from Defocus
Introduction
Implementation
Results
Conclusion
Questions
Daniel Kuntz
Depth from Defocus