Perspective
Structure from Motion
Orthographic  Perspective
• Last time, considered weak perspective
• With full perspective, can recover more
information (translation along optical axis)
• Result: can recover geometry and full
motion up to global scale factor
Perspective SFM Methods
• Bundle adjustment (full nonlinear
minimization)
• Methods based on factorization
• Methods based on fundamental matrices
• Methods based on vanishing points
Motion Field for Camera Motion
• Translation:
• Motion field lines converge (possibly at )
Motion Field for Camera Motion
• Rotation:
• Motion field lines do not converge
Motion Field for Camera Motion
• Combined rotation and translation:
motion field lines have component that
converges, and component that does not
• Algorithms can look for vanishing point,
then determine component of motion
around this point
• “Focus of expansion / contraction”
• “Instantaneous epipole”
SFM Algorithm
• Compute optical flow
• Find vanishing point (least squares
solution)
• Find direction of translation from epipole
• Find perpendicular component of motion
• Find velocity, axis of rotation
• Find depths of points (up to global scale)
Shape from Shading and Texture
Lambertian Reflectance Model
• Diffuse surfaces appear equally bright
from all directions
• For point illumination, brightness
proportional to cos q
Lambertian Reflectance Model
• Therefore, for a constant-colored object
with distant illumination, can write
E = L r ln
E = observed brightness
L = brightness of light source
r = reflectance (albedo) of surface
l = direction to light source
n = surface normal
Shape from Shading
• The above equation contains some
information about shape, and in some
cases is enough to recover shape
completely
(in theory) if Lr and l are known
• Similar to integration (surface normal is like
a derivative), but only know a part of
derivative
• Have to assume surface continuity
• No solution in the presence of noise
Variational Shape from Shading
• Approach: energy minimization
• Given observed E(x,y), find shape z(x,y)
that minimizes energy


E   E ( x, y )  Lr l  n( x, y)    px  p y  qx  q y
2
z
z
where p  , q 
x
y
2
2
2
2
 dx dy
Variational Shape from Shading
• Solve by techniques from calculus of
variations
• Use Euler-Lagrange equations to get a
PDE, solve numerically
– Unlike with snakes, “greedy” methods tend
not to work
Difficulties with Shape from Shading
• How to find L, r, l?
– Estimate based on scene statistics
• Shadows
• Non-Lambertian (e.g., specular) surfaces
• More than 1 light, or “diffuse illumination”
• Interreflections
Shape from Shading Results
[Trucco & Verri]
Shape from Shading Results
Active Shape from Shading
• Idea: several (user-controlled) light
sources
• More data
– Allows determining surface normal directly
– Allows spatially-varying reflectance
– Redundant measurements: discard shadows
and specular highlights
• Often called “photometric stereo”
Photometric Stereo Setup
[Rushmeier et al., 1997]
Photometric Stereo Math
• For each point p, can write
l1, x

r p l1, y
l1, z
l2 , x
l2 , y
l2 , z
 E p ,1 
l3, x   n p , x 




l3, y  n p , y   a  E p , 2 
 E p ,3 
l3, z   n p , z 


• Constant a incorporates light source
brightness, camera sensitivity, etc.
Photometric Stereo Math
• Solving above equation gives (r /a n
• n must be unit-length  uniquely
determined
• Determine r up to global constant
• With more than 3 light sources:
– Discard highest and lowest measurements
– If still more, solve by least squares
Photometric Stereo Results
Recovered normals (re-lit)
Input
images
Recovered color
[Rushmeier et al., 1997]
Texture
• Texture: repeated pattern on a surface
• Elements (“textons”) either identical or
come from some statistical distribution
• Shape from texture comes from looking at
deformation of individual textons or from
distribution of textons on a surface
Shape from Texture
• Much the same as shape from shading,
but have more information
– Foreshortening: gives surface normal (not
just one component, as in shape from
shading)
– Perspective distortion: gives information
about depth directly
• Sparse depth information (only at textons)
– About the same as shape from shading,
because of smoothness term in energy eqn.
Shape from Texture Results
[Forsyth]