Perception of
illumination and
shadows
Lavanya Sharan
February 14th, 2011
Studied indirectly
• Not a lot of studies examine illumination
or shading directly
• Role of illumination and shading in
perception of 3-D shape, reflectance,
object identity and space
Outline
• Shape from shading
• Illumination estimation
• Shadows
Shape from shading is underconstrained.
Fig 9.11, VPfaCGP
And yet, we perceive unique and stable shapes.
Theoretical cues for shape from
shading
•Reflectance map
•Isophotes
•Image orientation
(Horn, 1977)
(Koenderink & Van Doorn, 1980)
(Fleming, Torralba & Adelson, 2004)
Theoretical cues for shape from
shading
• Reflectance map
(Horn, 1977)
•
•
•
Representation of scene brightness
as a function of 3-D surface
orientation
Ignores shadows, inter-reflections,
vignetting, translucency etc.
Unclear whether this relationship
between image intensity & surface
orientation is used by visual system
Horn & Sjoberg, 1978
Theoretical cues for shape from
shading
• Isophotes
(Koenderink &
Van Doorn, 1980)
•
•
•
Curves of constant intensity,
depend on illumination and shape
Patterns of isophotes can reveal
shape (under assumptions of
lighting)
The visual system could use these
as a cue
Fig 9.12, VPfaCGP
Theoretical cues for shape from
shading
• Image orientation
(Fleming, Torralba & Adelson, 2004)
•
•
•
Orientation filters have strong
responses for strong curvature
regions.
By measuring these across a
surface can get local geometry
The visual system could use this
relationship between image
orientation and surface curvature.
Fig 9.13, VPfaCGP
Why is shape from shading hard?
Lots of ambiguities.
•Convex vs. concave?
•Surface orientation change vs. surface
reflectance change?
•Bas-relief ambiguity
Ambiguities in shape-from-shading
•Convex vs. concave?
(Ramachandran, 1988)
Ambiguities in shape-from-shading
•Reflectance vs. orientation change?
(Knill & Kersten, 1991)
Ambiguities in shape-from-shading
•Bas-relief ambiguity
(Belhumeur et al., 1999)
Gauge figure task to study shape
perception
(Koenderink et al., 1992)
What have we learnt from gauge figure
tasks?
• Subjects are consistent. Their (inferred) shapes
are related by affine transforms. (Koenderink et al., 1992)
• For
simple shapes, contours are often enough for
estimating shape, shading plays a lesser role.
(Mamassian & Kersten., 1996; Koenderink et al., 1996; Cole et al., 2009)
• Illumination
changes causes subtle distortions of
perceived shape. (Koenderink et al., 1996; Caniard & Fleming, 2007)
Intrinsic image analysis
Idea: Visual system
separates retinal
image into layers
that represent
distinct physical
causes. (Barrow &
Tenenbaum, 1978)
How? Proposals
include Retinex,
anchoring theory,
etc.
Fig 9.15, VPfaCGP
Mutual illumination affects reflectance
perception
We can distinguish black
and white rooms seen in
isolation based on interreflections. (Gilchrist & Jacobsen,
1984)
Mutual illumination
estimation is not perfect,
sometimes perceived as
surface color. (Bloj et al., 1999;
Doerschner et al., 2004)
Ruppertsberg & Bloj, 2007
Testing illumination perception directly
Subjects can tell direction (upto
convex/concave ambiguity.
Worse at estimating elevation of
light source (related to bas-relief
ambiguity)
Confirmed by other studies (Ho et
al., 2006; Pont & Koenderink 2003)
Koenderink et al., 2003
Testing illumination perception directly
Subjects can estimate diffuseness of a light source,
errors related to collimated/diffuse. (Pont & Koenderink,
2007)
Can account for two collimated light sources
(Doerschner et al., 2007)
Can to some extent account for changes across
space (Snyder et al., 2005) and time (Gerhard & Maloney, 2010)
Light from above prior
(Ramachandran, 1988)
A is usually seen as convex => light from above
Discs in B cannot be both convex or concave => one light source
Light from above and left prior
(Sun & Perona, 1998)
Time to search in display depended on the shading direction of distractors.
Should be lowest when shading matches prior.
Shadows help interpret scenes
Kersten et al., 1999
The spatial position of objects is influenced by cast shadows
Shadows help interpret scenes
(Let’s watch a movie...
http://vision.psych.umn.edu/users/kersten/kersten-lab/demos/Shadows.mov
)
Last word about shadows
Do these results hold for real-world images?
Surprisingly hard to see errors
(Ostrovsky et al., 2005)
Need to understand this better...
(Lopez-Moreno et al., 2010)
Summary
✓ We can estimate illumination, tested indirectly by
probing reflectance and shape, and also directly.
✓ We are not perfect, but the problem is also hard
(lots of ambiguities). We are consistent in our
estimates, e.g., light-from-above(-and-left) prior.
✓ Shading tells us about shape, shadows about
scene.
✓ Don’t know a lot about cues and effects in realworld scenes.