From: AAAI-82 Proceedings. Copyright ©1982, AAAI (www.aaai.org). All rights reserved.
LOCAL
COMPUTATION
OF SHAPE
Alex Paul Pentlsnd
Artificial
SRI International,
Intelligence
333 Ravenswood
Center
Ave, Menlo Park, CA. 94025
ABSTRACT
Local,
to infer
parallel
scene.
of shading
shape without
surface
viewed
analysis
The
able assumption
following
about
second
derivative
ponent
of surface
orientation
maximum-likelihood
estimate
of orientation
compute
surface
without
knowledge
been
to be determined
natural
comand a
exactly,
for each
image
shape
fashion
from local
image
then it should
information
take
possibility.
we examine
an image,
usually
in shading
(changes
a small neighborhood
in image intensity).
of a particular
is small changes
Finding
a contour
point
is unusual.
image
if we are to investigate
what we may learn about
point
from local examination
of an image,
images,
has
with shading.
and
surface
Thus
shape
Thus
shape
ourselves
the question:
from an unfamiliar
within
surface
we must concern
this paper examines
be recovered
local analysis
a point
around
all we find in the neighborhood
the neighborhood
of shape.
in
full advantage
to
This algorithm
and synthesized
calculate
bottom-up
When
of the
has been developed
in parallel,
that
a reason-
depth component
of scene characteristics.
on both
Given
direct,
the
allows the image-plane
An algorithm
a good estimate
about
local analysis
of the remaining
orientation,
evaluated
produces
smoothness,
of image intensity
to be made.
knowledge
has been proven:
surface
can possibly
in an image may be used
o priori
how may
image by using
of shading?
II. Previous Work
I. Introduction
Horn and his colleagues
A spatially
the
restricted
first stage
relatively
because
of any visual
simple,
quite complex.
for the remainder
of analysis
produces
remainder
of the visual
system
directly
with
Determining
as much about
is logically
analysis
might
that
of the system.
will be much
then
simpler
all of the ambiguities
than
the
if it
If a visual
known
in which the illumination
is known
Support
for the
Army
Research
05ce
contract
DAAG2479-C-0210.
portion
of this
research
was
substantial
Artificial
Intelligence
of Technology.
in
part
by
Department
stages
Laboratory
Support
the
of Defence
of this
research
was
accomplished
of the Massachusetts
for the laboratory’s
Advanced
N00014-75-C-0643
Grant
final
Research
under 05ce
research
Project
at the
techniques
information;
face whose
shape
of the
an initial
shape-from-shading
there
the image,
tech-
is su5cient
such as in a fac-
and the surface reflectance
theory
shape-from-shading
of the scene, it fails to answer the quesshape in an unfamiliar
satisfy
image.
our requirement
constraint
by smooth
from
occluding
is to be estimated.
Thus
only
techniques
boundary
contours)
Further,
of using
Horn’s shape-from-shading
by propagating
as provided
along
conditions
over the sur-
further
analysis
required.
is provided
Agency
Science
(such
Institute
of Naval Research
and in part by National
function
A
about
Horn’s
knowledge
tion of bow to determine
provided
function
orientation
beforehand.
because
sumes a priori
local image
by
a priori
of (1) the distribution
for which
information
none of these
l
in situations
to solve
reflectance
These
surface.
may be useful
However,
system
knowledge
and (3) the surface
on the object’s
several
for using image intensity
(2) the bidirectional
surface,
function
is therefore
schemes
tory setting
of the image.
the world as is possible
at this first stage of processing.
shape given D priori
niques
the
If the first stage
of the world,
for object
curve
to the
determines
integration
object’s
[l], [2]) have developed
(e.g.,
numerical
of illumination,
important
is available
and, therefore,
be
or it might be
is especially
a rich description
had to deal
important
This
the information
system,
image
image intensity,
This first stage of analysis
of the visual
requirements
of a single
system.
e.g., measuring
it determines
remainder
analysis
of
the
III. The Estimation
contract
Of Surface Orientation
Foundation
The
MCS77-07569.
mation
22
problem
of estimating
is, essentially,
surface
tbe problem
shape
from local
of determining
infor-
the unknown
surface
normal,
N, from image measurements
I, or its derivatives.
equations
Solving
requires
Because
having
the surface
require
of image intensity,
for the unknowns
more
normal
in any system
measurements
than
N has two degrees
of
unknowns.
of freedom,
we
at each image point to obtain
least two measurements
a
solution.
Therefore,
we cannot
image
intensity
alone,
ment
per image
point.
obtained
from
intensity;
ber of unknowns
for N simply
per point;
or higher
each
per point,
to solve
per point
-
does
Therefore,
to obtain
solve
more measurements
for N locally,
we must
make some
of unknowns
-_
.
_.._
Figure 1 The manner in whrch Image curvature %preads” indicates
the tilt of the surface.
This may he understood by imagining that
we could observe the lines of curvature on a surface directly. These
lines of curvature would look just like the lines drawn in this figure.
If we were looking straight down on a surface with no twist, the
lines of curvature would appear perpendicular,
as in (a). As we
tilted the surface off to one side, the lines of curvature would appear
progressively more spread, as in (b) and (c). Different directions of
tilt cause spreading in different directions.
give
we cannot
C
B
A
can be
of image
derivative
so that the number
larger that the number
using
measure-
derivatives
additional
into the equations.
assumptions
normal
one
it also brings an even larger num-
by using derivatives
in order
simplifying
second,
while
more measurements
the surface
it provides only
More measurements
the first,
however,
determine
because
is not
available to us.
of measurements
Thus one of the two components
A. The Tilt
tilt, may be determined
Of The Surface
tensity
When
we observe
impression
the
a smooth
surface
away from
slants
impression
be possible
we could
determine
component
the tilt
of freedom
orientation)
would
the
of the
(slant,
the lines of curvature
would
we were looking
straight
lines of curvature
(a).
surface
appear
(c). Different
down
appear
the surface
progressively
directions
This leaves
is a second-order
orientation,
tilt
directly.
exactly,
then
surface
only
surface.
in N.
on a surface
directly.
on a surface
with
perpendicular,
“twisting”
1. If
no twist,
the error
the
surface
1
as in Figure
of tilt would cause spreading
typically
causes
between
where
errors
on the order
points
across
on the
a surface,
the
is on the order of
The largest
errors occur for
ljpUVjj > > 0. For such surfaces,
those with
of l/lOOth
the assumption
surface
f(u, v)
of an arbitrary
observed
by this approximation
can he on the order
second-order
to one side, the lines of curvature
more spread,
surfaces,
[3]. Thus
we are observing
+ ve:! + f(u, v)ef,
l/lOOOth of tbe width of the surface.
These
as in Figure
uel
if ten p0int.s are observed
error incurred
in-
direction.
to he determined.
that the surface
6 is the spacing
Thus
maximum
that we
of the illuminant
+ ozv ‘. The approximation
a surface
by such
of 63, where
the
of image
that is, a surface that may be described
p =
patch
is of the form*01u2
of surface
the tilt? Imagine
assumes
surface,
by the Monge
If
knowledge
orientation,
derivative
only the slant of the surface
This proposition
a strong
look like the lines drawn in Figure
would
As we tilted
would
of surface
surface
be left undetermined
lines of curvature
we have
the depth-component
How we might go about estimating
could observe
a strong
direction
Because
to compute
one degree
-
us.
of the image-plane
it might
we obtain
that is, which
surface,
of the tilt of the surface
without
directly
of surface
from the second
of the width
that we are observing
does not introduce
of the
such a
much error.
1 (b) and
B. The
in different
Slant
Surface
Of The
directions.
We can not observe
of course,
lines of curvature
but we can observe
with the illuminant
The second
the interaction
in the second
derivative
of image
Pentland
on the surface directly,
of surface
derivatives
intensity
curvature
completely
of image intensity.
may
has three components:
spread
tilt.
of the lines of curvature,
term
is the greatest
The direction
direction
along
the direction
is also the direction
in which the spread
which
@I
in which
this
of
a smooth,
Lz#Iy,,
direction
sity,
81,
I zy #0,
Given
recond-order
rurface
then the tilt of the surface
in which the second derivative
estimate
surface,
with
thogonal
n,
direction,
of tbe
surface slant,
be
We
as
Lsplacian
of a smooth,
Of
Image
homogeneous
K2,z7;2 - tc?,
K,, is the surface
and
cannot
of the surface.
then
VI
-=I
direction
the tilt of the surface
of the surface
propositions.
an image
is also the
an image
while
(Normalized
Given
and so the following
of the Surface)
homogeneous
an unbiased
second-order
where
(Tilt
that
the slant
from the curvature
in the following
Proposition
proposition:
Proposition
make
Intensity)
of the surface
is the greatest
is the greatest,
still
disentangled
developed
Lz and I,,, the %urvaturen of image intensity along the z and
y axes, and Izv, the ‘spread* of those curvatures.
Just as with
the spread
[4] has proven
may be exacctly determined,
curvature
along the rurface
is the surface
curvature
t‘~ is the t component
tilt
in the or-
of the surface
is the image
of image inten-
*Linear
is greatest.
and
appropriately
23
constant
positioning
terms
in f
may
the coordinate
be accounted
axes.
for
by
normal
(which
is equal
to the arc cooie of the surface
slant)
What
this
proposition
shows
only of the squared
curvatures
the foreshortening.
Note
the surface
albedo
The range
and
the
V21/1
that
of the surface
curvature
say a uniform
distribution,
had a particular
uniform
face
likelihood
distribution,
value of V*1/1,
estimate
Of
of the foreshort
of ZN (the
equal
to the
Slant)
to zE2, we
Assuming
curvature
a
the msximum-
t component
arccosine
of the
of the
slant
our-
of the
is
/I V21
where
of
proposition.
of surface
estimate
surface)
a priori
is proportional
(Estimation
normal,
value
of the slant of the surface, i.e., COS-~(ZN).
distribution
likelihood
curvatures
observed
then for any observed
This leads to the following
Proposition
the surface
particular
the foreshortening
then have an eArnate
and
if we were given that the magnitude
we could make a maximum
Because
between
for any
Therefore,
nz and n?,, and
of the illuminant
in this quantity.
of relationships
is limited.
ening.
the effects
do not appear
foreshortening
V21/1 is a function
is that
of the surface,
a:
ia the
variance
I
xi
of the
distribution
of surface
curvatures.
Pentland
degree
[4] and Bruss
of freedom
information.
The
parameters
leaving
likelihood
estimate
ehading
the tilt estimate
together
one of the two
neither
knowledge
Figure 2 Evaluation on synthetic images.
(A) Artificial images of a
sphere and an ellipsoidal shape used to test the surface shape estimation algorithm, (B) side view of relief maps showing the true shape.
(C) Relief maps showing the estimate of surface shape made by the
shape algorithm for the sphere and ellipsoidal shape images. Compare
these relief maps to those of 4 (B).
exactly,
gives
by definition
Therefore,
constitute
one
shading
proposition
possible
Note that
require
This
estimate.
it is theoretically
information.
that
of the slant, which
unbiased
at least
by the
can be determined
undetermined.
and the tilt propositions
that
showed
orientation
the slant
that
undetermined
tilt proposition
the minimum-variance
of surface
[S] demonstrate
remain
of surface
only
maximum
will
a
is
&ant
the
the 6errt e&mate
to make
from
local
the slant estimate
of the illuminant
A.
nor
Synthetic
direction.
The
shape
generated
On the basis of this theory,
M.I.T.‘s
synt,hetic
LISP machines.*This
and natural
images.
the level of performance
under ideal conditions,
the performance
the more varied,
scenes.
of the shape
complex
Some examples
on both synthetic
Using
algorithm
images
algorithm
of the application
which
on both
allows
images
to be evaluated
was
appear
tested
timated
surface
quatcly
evaluat,e
the shape
allows
tegrated
under
maps
*The algorithm
is a straightforward
implementation
in the slant and tilt propositions
the
2 (A).
do not
the performance
algorithms’
to produce
were found
the
estimates
a relief
shape,
presented
The
of the cal-
algorithm’s
above.
24
relief
map
estimate
which
that
allow
of the
an observer
results
es-
to ade-
Therefore,
orientation
an adequate
were in-
These
relief
impression
are the output
in this paper even though
of surface
surface
of the algorithm,
map of the surface.
and so they
the
of the
of the algorithm.
of surface
2 (B)
showing
displays
performance
to give observers
surface
for the examples
here.
orientation
syntheticly
Figure
this image.
estimates
however,
of displaying
the estimated
of this shape algorithm
produces
found,
is not part of the shape algorithm
culations
using
in Figure
was used to generate
algorithm
it was
for purposes
found in natural
scenes are presented
which
shape
orientation;
to be checked
the use of natural
shape
The
the
on one of
has been tested
synthet,ic
and noisy conditions
and natural
for estimating
was implemented
of the shape
whereas
surface
an algorithm
algorithm
algorithm
images
shows the side view of a relief map for this surface,
IV. Evaluation
surface shape (the “shape algorithm”)
Images
of
shown
integration
per ue.
from
orientation
integrating
is shown
the shape
in Figure
2
(C).
Comparing
the estimate
the relief map which
of surface
orientation
2 (B) s h ows that the shape
map in Figure
a very high level of accuracy.
sphere,
81
using convolutions
It is important
knowledge
shape was recovered
map generated
orientation.
image
orientation.
(A) and 3 (B) correspond
shapes
of these
his impression
The
relief
maps
in Figure
as can be determined
two objects.
algorithm
on other natural
The reader should
3
to the
compare
images with the relief
has also been
images,
to the actual
parallel,
of surface
surface
local
does not require
successfully
and on electron
case the estimates
sufficient
of a rock,
3 (A) and 3 (B).
shape
the
of surface
image
from the shape algorithms
The
as closely
Figure 3
with the relief
estimates
of shape from the digitized
maps of Figure
which
algorithms
3 (B) s h ows the digitized
of surface
here.
of a log, together
wit.h the relief map generated
closely
in
any a priori
on several natural
will be presented
from the shape
Figure
estimates
that
without
has also been tested
(A) shows the digitized
each
0.01 %.
Images
and two such examples
actual
to within
that these shape were calculated
local image measures,
The shape algorithm
together
can attain
of scene characteristics.
B. Natural
images,
“truen relief
algorithm
X 200 pixel image of a
For a 200
to remember
from purely
from integrating
with 21 X 21 pixel masks to calculate
and V21, the correct
parallel
results
to the original
shape
shape.
images.
produced
These examples
computation
employed
microscope
described
any a priori
knowledge
to obt*ain a useful estimate
of surface
In
correspond
demonstrate
in this
paper,
of the scene,
is
1Figure 3 Evaluation on natural images.
(A) The digitized log image
__
_._
_
. .
and the relief map generated by the shape algorithm for that. image.
(El) The digitized rock image and the relief map generated by the
shape algorithm for that image.
shape in natural
images.
REFERENCES
[l] Horn, B.K.P.,“Shape
From Shading:
the Shape
of a Smooth
Technical
Report
of Technology,
[2] Ikeuchi,
Shading
K.
79, Project
Special
Horn,
E., *and Keller,
A.P.,“The
Massachusetts
B.K.P.,“NumericaI
Vision,
A.I.
Institute
Shape
Artificial
from
Intelligence,
15 (1981)
H. B.,
“Analysis
Visual Inference
From Local Features,”
Massachusetts
A.,
Thesis,
Institute
for Obtaining
from One View”,
Of
Numerical
(John Wiley and Sons, New York, New York, 1966)
[4] Pentland,
[S] Bruss,
MAC,
Boundaries,”
Issue on Computer
Methods,”
A Method
Object
(1970)
and
and Occluding
[3] Issacson,
Ph.D.
Opaque
Ph.D. Thesis,
Institute
“Shape
Dept.
of Technology
From Shading
Elec.
of Technology,
Engr.&
Of Shape: Computation
Psychology
Department,
(1982)
And Bounding
Comp.
Contour,”
Sci., Massachusetts
(1981).
25