COMPUTER PROCESSING OF STEREO IMAGES FOR THE
AUTOMATIC EXTRACTION OF RANGE
by
JEROME BARRY LERMAN
SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREES OF
BACHELOR OF SCIENCE
AND
MASTER OF SCIENCE
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June,
Signature of Author
Signature redacted
Df
Certified by
1970
rtment of Electricalt
gineering, May 21, 1970
Signature redacted
I'hesis Supervisg
Accepted by
Signature redacted
Chairlnan, iabe.iptl ta Committee on Graduate Students
ikrhives
gNSS. INST.
tEcH
JUL 3 0 1970
I
COMPUTER PROCESSING OF STEREO IMAGES FOR THE
AUTOMATIC EXTRACTION OF RANGE
by
JEROME BARRY LERMAN
Submitted to the Department of Electrical Engineering on
May 21, 1970 in partial fulfillment of the requirements
for the degrees of Bachelor of Science and Master of Science
ABSTRACT
A technique has been developed to generate a three-dimensional disparity
space from a pair of two-dimensional stereo images of a scene. The disparity
space will contain surfaces that correspond to the surfaces of objects in the viewed
scene. Spurious surfaces that do not represent real objects may also be present
and must be eliminated. Using computer-generated random patterns, the properties of disparity space are examined. The problems caused by occluding objects
and regions of uniform brightness are discussed. Based on these observations, a
computer program is presented that will produce a range map of a natural scene
from a pair of stereo images.
Adolfo Guzman-Arenas
THESIS SUPERVISOR:
TITLE: Assistant Professor of Electrical Engineering
ii
ACKNOWLEDGMENT
The work described in this thesis was sponsored by the Bioscience
Programs of NASA's Office of Space Science and Applications as DSR
Project 55-25700,
and performed at the M.I.T. Draper Laboratory.
Special thanks go to Louis Sutro, the project supervisor, for his encouragement and enthusiasm, and to Adolfo Guzman for his many helpful suggestions.
My sincere thanks to Catherine Griffin who typed the thesis.
*Contract NSR 22-009-138.
This report was prepared under DSR Project 55-25700, sponsored
by the National Aeronautics and Space Administration Headquarters,
Office of Space Science and Application, through Contract NSR-22-009-138
with the Massachusetts Institute of Technology.
The publication of this thesis does not constitute approval by the
Charles Stark Draper Laboratory or the National Aeronautics and Space
Administration of the findings or the conclusions contained therein.
is published only for the exchange and stimulation of ideas.
iii
It
TABLE OF CONTENTS
Chapter
1.
2.
3.
4.
5.
6.
Introduction to Depth Perception
1. 1
Depth Cues . . . . .
1. 2
Selecting Depth Cues for a Computer Algorithm
1. 3
Retinal Disparity
Disparity Space.
. . . . .
2.1
A Remapping of Visual Space.
2. 2
Generating a Disparity Space .
2. 3
Viewpoint of the Disparity Space
2. 4
STEREO - A Computer Program to Generate a
Disparity Space .
Random-dot Stereo Images
3. 1
Images of Plane Surfaces
3. 2
Ambiguities due to Occlusion.
3. 3
STROUT - A Simple Program to Extract Range.
3. 4
Random-dot Images of Arbitrary Objects
Images with Regions of Uniform Brightness
4. 1
Parallelograms of Matches
4. 2
Ambiguities Due to Occlusion
4. 3
EXPER - A Program to Extract Range
4. 4
FUSER - A Program that Implements Shadow-Checking .
Stereo Images of Natural Scenes
5.1
Noise . . . . . . .
5. 1
Registration of the Images
5. 3
Range Maps for Real Images .
Conclusions and Suggestions
References
Appendix A
The Relationship of Range to Disparity
Appendix B
A Limit on the Resolution of Range
Appendix C
Listings of Computer Programs
iv
CHAPTER
1
INTRODUCTION
The ability to perceive the world as three-dimensional is a talent developed
by man's precursors to cope with the rigors of arboreal existence 1.
Evolution flat-
tened our snouts and moved our eyes from the sides to the front of our heads. From
this position, they share similar but slightly different views of the same scene. Such
an arrangement is called a stereo imaging system.
Onto the retina of each eye is projected a two-dimensional representation of
the scene being observed.
Using these two retinal images, our brains can construct
the impression of solid objects lying in three-dimensional space.
The image thus
perceived is:
(a)
A single three-dimensional image
(b)
In front of the observer
(c)
Right side up (i. e. not upside down)
In contrast, the images on the retina are:
(a)
Two, two-dimensional images
(b)
In back of the eye
(c)
Upside down
Reference 2 gives a detailed explanation of depth perception.
1. 1
Depth Cues
The properties of the images that are useful for deducing depth are called
depth cues.
(1)
The most important are listed below:(3)
Interposition
-
an object that occludes part of another object is seen
-
the larger of two similar objects is seen as nearer.
as nearer.
(2)
Relative size
(3)
Relative height - the lower of two objects is seen as nearer.
4)
Relative clearness - hazy objects are seen as far away.
1
-
(5)
-
--
Linear perspective - the greater the convergence of lines, the greater
the impression of distance.
(6)
Light and shadow - shadow patterns favor the impression of depth.
(7)
Accommodation - the muscular tension on the lenses of the eye needed
to bring an object into focus is a function of distance.
(8)
Movement parallax - near objects tend to move faster than far objects.
(9)
Convergence - the angle between the optical axes of the two eyes de-
creases as the fixated object becomes more distant.
(10)
Retinal disparity - the amount an object is shifted when comparing
the views of both eyes is a function of distance.
Cues 1-8 are monocular since they require only one eye.
ocular cues that rely on the interaction of two eyes.
An important distinction among
the various depth cues may be made on the basis of complexity.
operate on recognized objects.
Cues 9-10 are binThe first six cues
In order for interposition to be effective, for ex-
ample, the observer must be able to recognize that some portion of a familiar object
is being blocked. Object recognition is a high level process that draws on past experience with the world.
Cues of this type are called content-dependent since they
must interpret the content of the image.*
Cues 7-10, on the other hand, are content-independent.
They require no in-
formation about the content of the scene, only a pattern of brightness.
tion of objects is necessary.
No organiza-
To decide whether an image is in focus does not de-
mand the recognition of objects in the scene.
Retinal disparity has recently been
shown to be content-independent for highly textured images such as random dot pat-
terns.(4)
1. 2
Selecting Depth Cues for a Computer Algorithm
It is the goal of this thesis to develop a content-independent algorithm for ex-
tracting depth information from images that are obtained through two-dimensional
sensors such as a photograph or a TV camera. Throughout the rest of this paper,
the word "image" will refer to a two-dimensional array of numbers obtained by sampling and quantizing a continuous image. Such an array is suitable for computer
processing.
By choosing content-independent cues the difficult problem of object recogni-
*It is of interest to note that the decomposition of a visual scene into the constituent
objects is possible, without having previous knowledge or familiarity with these
objects. "SEE" is a computer program (Guzman, 1968) that does this decomposition for scenes containing polyhedra (objects with flat faces).
2
a-w
W
P
zz
III
LENSES
ddL
LEFT V IEW
Fig. 1
dR
RIGHT VIEW
Stereo camera configuration with parallel
axes,
f = focal length
optical
of the lenses. a = interocular
distance.
3
nition may be avoided.
Of the cues available, we must eliminate motion parallax
since we are constrained to a static "snapshot" type imaging system. Accommodation
and convergence are useful for determining the range of a single fixated object,*
but do not account for the depth perception of an entire scene.
Retinal disparity is
capable of indicating the spatial arrangement of many objects.
It has been suggested
that accommodation and convergence provide a plane of reference for the information
obtained through retinal disparity.(5)
For this reason, throughout this paper we will
assume that the convergence of the stereo system is known and the images are in
focus.
Retinal disparity will be the only cue utilized by the algorithm to determine
depth.
A camera configuration such as in Fig. 1 will be used to obtain stereo images.
1. 3
Retinal Disparity
Two cameras (or eyes) that view the same scene from slightly different lo-
cations will produce similar images.
space may appear in both views.
In Fig. 2, points within the shaded region of
Their positions in the two images will be different.
This difference is known as retinal disparity.
To investigate retinal disparity, we must define some terms.
view will be the "true match" of a point in the other view if they are
same point in space.
A point in one
images of the
The "locus of possible matches" is the projection in one view
of all the points in space whose image is a given point in the other view.
In other
words, the locus of possible matches is the path of points that must be searched to
find the true match for a given point in the other view.
For each point in one view,
there is a locus of possible matches in the other view.
For a stereo system with flat image planes, the loci of possible matches are
always straight line segments that intersect at some point in the image plane.
In
particular, for a camera system whose optical axes are parallel (as in Fig.1), the
loci of possible matches will be parallel horizontal line segments.(6) By horizontal,
we mean parallel to the plane of the optical axes.
Thus, for such a system, an ob-
ject in the left view will be shifted laterally in the right view by an amount that depends on its distance from the camera (its range).
era systems.
We are assuming identical cam-
In particular, the magnification of the left image must be the same as
the right image.
If all the above conditions are met, there can never be any vertical
shifting of objects from one view to another.
If we restrict our study to stereo systems with parallel axes, the computational task will be simplified.
For digital images stored line by line, the locus of
possible matches of a point in one view will lie somewhere along the same line of the
* B.K. P. Horn has developed a computer program (M.I.T. thesis,
focussing to obtain depth information.
4
1970) that uses
eftfiw
----
I --
-1--
-A
_
-
LENSES
(
--
Fig. 2
The shaded region of space will be viewed by both
cameras.
y
I
YO
YO
Xr
LEFT VIEW
Fig. 3
I
RIGHT VIEW
For a point (xt, yo) in the left view of a stereo
pair, the locus of possible matches in the right
view is shown as a solid line.
5
other image.
Thus, to search for matches, only one line of each image must be
stored in the computer at any time.
For a system where the optical axes converge
at some finite point, several lines of the image would have to reside in computer
memory and the slope of the locus of possible matches would be a function of position
in the frame.
If images obtained from a converged stereo system could be trans-
formed so that the loci of possible matches become horizontal lines, then the algorithm to be described in subsequent chapters would be effective. This transformation
7
is possible and known.!
Thus, for the rest of this dissertation, all stereo systems
will be assumed to have parallel optical axes.
In Appendix A, the relationship between the range, z, of a point in space and
its retinal disparity, d, was found to be:
K
z=
where K
(1)
is a constant that depends on the camera configuration.
According to equa-
a point infinitely far from the camera will have zero disparity, i. e. it will
appear at the same coordinates in both views. As the point moves nearer, its disparity will increase. If a point in space a distance z0 from the plane of the camera
tion (1),
lenses casts an image at (xt,
right view will be at
x
-
yo) in the left view, by equation (1), its image in the
, yo).
A procedure for finding the range of a point at (x
y 0 )in the left view is as
follows (See Fig. 3):
yO) in the right view.
(1)
Find the position (xt,.
(2)
Look to the left along the horizontal line through (x
3 ,o)
for
a true match.
(3)
(d =
If a true match is encountered at (xr' YO), measure the disparity
xt - xr)
and compute the range by equation (1).
The most difficult step in the process described above is locating a "true
The only information directly available from the image are the quantized
match."
If an attempt is made to find true matches by
searching for points with the same brightness, it is very likely that several matches
will be discovered. By arranging these brightness matches at the position in space
numbers representing the brightness.
that their disparities imply, we would expect true matches to form surfaces, while
spurious matches will tend to form isolated clusters. A method for accomplishing
this is described in the next chapter.
6
CHAPTER 2
DISPARITY SPACE
The purpose of this section is to describe the theory and use of a threedimensional space that is easily generated from a pair of stereo images. Called disparity space, or d-space, it will contain a mapping of the objects viewed by both
cameras.
Many of the techniques discussed in this section were first presented by
Julesz in a paper on automatic depth perception. (8)
2. 1
A Remapping of Visual Space
The area of space viewed by both cameras (shaded region in Fig. 2) will be
called the visual space.
The surfaces in this space can be described as a functionof
three variables, e. g. for Cartesian coordinates:
Sv = f (x, y, z)
(2)
If we substitute for the range variable, z, according to equation (1), a new
mapping of space is obtained.
(3)
Sd = g (x, y, d)
In this new space, called disparity space (d-space),
tion of z will be mapped into a hyperbola.
any straight line that is a func-
Since the transformation is conformal,
we will expect continuous surfaces in visual space to be mapped into continuous surfaces in d-space.
Fig. 4 shows a plan view of a visual space containing a slanted
plane.
The plane's mapping into d-space is shown for several different viewing dis-
tances.
As the plane becomes more distant, its mapping in d-space becomes more
flattened.
This means that for distant objects, small variations in range will not be
easily detected.
2. 2
(See Appendix B for a further discussion of range resolution.)
Generating a Disparity Space
In Section 1. 3, the difficulties in identifying a true match by comparing
brightness were discussed.
If points with matching brightnesses were arranged in
a d-space, the true matches would form continuous surfaces, and might be distinguished from spurious matches.
It is necessary then to obtain matches as a function
7
T~< ~
d
ii
0.08
-
0.101
0.06 1
z
0.04
-
0 +6
112
z0 - 10
0 . 02
-
z
z0
0
z - 100
I
I
I
I
I
1
x
1
+
12
x
0
S
I I
x
I
I
I
I
1 +
0x x
+ 12
x
U
0
DISPARITIES FOR A PLANE WITH SLOPE 1/2
PLANE IN VISUAL SPACE
(a)
(b)
Fig. 4 (a)
A plane in visual space that is slanted.
(b) The same plane represented in disparity
space for three values of z
.
I
x
ll~
LEFT 20 7 15 10 32141 6 27
IMAGE 12 71 52 2 24115 61 73
1 1
1 1
0 0 0 011
0 0 0 0 1
0
1
6 27
RIGHT 20 7 13241171 2
IMAGE 12 71 124 15 72 121 61 73
LEFT 20 7 15 10 3241 6 27]
IMAGE 1271152 2 24 15161
0l
7 17 2 6
RIGHT 2013241
IMAGE 12 71
LEFT
IMAGE
0
0
0
00
22415
1 61173
7 6 27
207 151032 41
12 7115212 24 151 76
3
RIGHT 20J7 1321411171 2
J 6J127
IMAGE 121711 24115172 21161173
Fig. 5
Generation of the first three d-planes from a stereo
pair of two-line images
9
Di
of disparity.
Since disparity corresponds to a lateral shift, the following method will mark
all matches at a given disparity, n:
(1)
The left and right images are superimposed.
(2)
The right images is shifted n/2 picture elements (pixels) to the right
and the left image is shifted n/2 pixels to the left.
Elements that now overlap are compared.
(3)
If their difference is less than
a specified tolerance, they are considered a match and that position in the
array is replaced by a "1".
If they do not match, a "0" is inserted in the
array.
By following this procedure, an array of
1's and O's called a disparity plane
(d-plane) is formed.
The d-plane for a disparity of n pixels is represented as Dn'
Fig. 5 shows pictorially the generation of the first three d-planes for a two line image.
Mathematically the process is described by:
Dn (i,
j)
=
1
if IL(i - n/2,
0
j)- R(i + n/2, j) 1
(4)
otherwise
D (i,
n
L (i,
j) = ith column, jth row of d-plane at disparity of n units
R (i,
j)
j)
E
Dn (i, j)
= entry of
=
.th
left image array at i
column,
entry of right image array at it
.th
J row
column, jt
row
= tolerance of match (E is set to zero for computer-generated images)
is not defined for i, j <0 or i, j > frame size, e. g. 256 pixels
If a sequence of d-planes, starting with D 0 , is generated and stacked one
upon the other as in Fig. 6a, the three-dimensional d-space we have been seeking
will be produced.
The position of
1's (i. e. matches) is a function of x, y and d.
From Section 2. 1, we know that surfaces in visual space will map into surfaces in
d-space.
Thus, if continuous surfaces of matches could be identified in the gener-
ated d-space, the problem would be solved.
however,
The presence of spurious matches,
causes some non-trivial problems that will be discussed in subsequent
chapters.
The wedge-shaped nature of the d-space in Fig. 6a is caused by the shifting
of one image past the other.
For each shift, one less column will overlap,
and
therefore each successive d-plane will be one unit narrower than the plane that precedes it.
It should be noted, however, that the visual space in Fig. 2 is roughly
wedge-shaped also.
Because digital images are discrete and not continuous, disparity can take
10
(a) D-planes aligned to create
a d-space.
The scene is
mapped into this d-space
as if it were viewed threedimensionally from a point
midway between the two
cameras.
Do
(b) D-planes aligned to create a
d-space that maps the scene
as if it were viewed threedimensionally from the left
camera's viewpoint.
d
(c) D-planes aligned to create a
d-space that maps the scen
as if it were viewed threedimensionally from the righit
camera's viewpoint.
Fig. 6
11
D
The allowed values of range (by Eq. (1) will be z= K K , K
0 1 2
Fig. 7 shows the location of these ranges in visual space. The space nearest the
only integer values.
viewer is most finely divided.
better for near objects.
The accuracy of the computed range will therefore be
The resolution of range for distant objects will be poor. A
point that lies between the planes in Fig. 7, will be "rounded off" into one of the adjacent planes.
2. 3
An analytical expression for range resolution is derived in Appendix B.
Viewpoint of the Disparity Space
If an algorithm is developed that can produce the z
coordinate for each posi-
tion in the left or right view, we must decide which frame of reference to use - the
left view, the right view, or something else.
By holding your finger before your eyes, fixating a far off point and alternately
blinking each eye, you may convince yourself that the human frame of reference is
not the left eye or the right eye but "something else'.'
The position of your finger
with both eyes opened, will usually lie midway between its positions when viewed by
each eye alone.
The effect of ocular dominance (9
may complicate this observation
for some people, but most human observers see the world as if viewed by a single
three-dimensional sensor in the middle of their eyes.
Since there is no reason to
stress one eye over the other in a computer alogrithm, the inherent symmetry of
this central viewpoint is appealing.
The algorithm described in Section 2. 2 produces a d-space that is referenced
to the central viewpoint.
In subsequent sections, it will be useful to have disparity
spaces that are referenced to the left or right viewpoint.
stacking the d-planes differently.
These may be produced by
From Fig. 6a and Eq. (4), it can be seen that the
d-planes have been stacked so that their centers are aligned.
If the planes were
right-justified (as in Fig. 6b), the d-space would be referenced to the left view,
since the right view is being shifted past.
Similarly, if the d-planes where left
justified (see Fig. 6c), the d-space would be referenced to the right view.
2. 4
STEREO - A Program to Generate Disparity Spaces
A computer program has been written that will produce a d-space when given
a stereo pair of images as input.
rays stored on magnetic tape.
The input images are expected to be 256
X256 ar-
Because of memory limitations, the output d-space
contains only the first 36 d-planes.
This is not a very serious constraint,
since for
the camera arrangement used, 0-36 units of disparity includes ranges from infinity
up to 5' in front of the camera (See Appendix A).
columns, also due to memory limitations.
The width of the d-space is 128
In effect, the input images are treated
as 128 X 256.
A parameter,
SCOL, determines which column of the input images will be the
start of what is effectively a 128 column input image.
12
LSHIFT, RSHIFT, LOFSET,
Do SAT INFINITY
D
Z -k
LET k
Z
00
2
1.00
D
0
1.000
1
0.500
2
0.333
3
0.250
4
0.200
5
0.167
6
0.143
7
0.125
8
0.111
9
0.100
10
*
S
*
6
z
D3
0.050
20
0.028
36
04
05
D10
D36
PLANE OF CAMERA
Fig. 7
-
X
Location of the planes in space that are represented
by integer values of disparity.
13
and ROFSET are parameters provided to allow for registration of the two images,
EPS determines the tolerance of the match.
Since the locus of possible matches is a horizontal straight line, only one
line of each image need be in the computer at one time. If the stereo system converged at some finite point, several lines of each image would have to be resident in
the computer to produce one line of d-space
(of Section 1. 3).
Switches are provided to determine which viewpoint the generated d-space
will reference. See Appendix C for a program listing.
The program STEREO is used to generate all the d-spaces that are discussed
in the following pages. It may be regarded as a basic building block for all the procedures that are presented in this dissertation.
14
CHAPTER 3
RANDOM-DOT STEREO IMAGES
A pair of random-dot stereo images is shown in Fig. 8.
The method for
*
generating such a stereo pair is well known.10,11) In this case, the images are
256 X 256 arrays of randomly distributed numbers representing 64 shades of gray.
When viewed through a stereoscope,
Fig. 8 will appear to be a square floating be-
fore a background plane.
Computer-generated random-dot patterns have several properties that make
them a desirable beginning point.
(1)
They are highly structure, with few spurious matches.
(2)
They are noise free.
(3)
They cannot be mis-registered.
By studying these very artificial images, some insight into depth perception
will be gained that will be useful in processing real images.
3. 1
Images of Plane Surfaces
The images in Fig. 8 were created by shifting a 64 x 64 pixel region of one
image 5 pixels to the right.
The uncovered region is filled with more random dots.
This technique will produce a surface with a constant disparity, corresponding to a
plane in visual space that is perpendicular to the axes of the camera system.
If a d-space were produced from the images in Fig. 8, we would expect to
find a region of matches in D 0 corresponding to the background and a
region of
matches in D 5 corresponding to the square floating in space. All the other d-planes
will contain scattered spurious matches.
generated d-space,
Fig. 9 shows a cross-sectional view of the
i. e. a plane parallel to the x axis and the d axis. The detailed
area in Fig. 6a represents a similar section of a d-space.
Such sections are often
used to examine a disparity space since the entire three-dimensional structure is not
easily displayed.
Matches are represented as dots.
*A stereoscope suitable for viewing the stereo pairs in this thesis may be obtained
from Air Photo Supply, Yonkers, New York.
15
Fig. 8
A random-dot stereo pair depicting a square
floating before a background.
(Image quality
has been degraded by half-tone reproduction).
d--
x
Fig. 9
A cross-section of the d-space generated from the
stereo pair in Fig. 8.
df
d
X
(b)
(a)
Fig. 10 (a) A cross-section of d-space showing an occlusion
ambiguity for w > (ds
(b)
- db).
A cross-section of d-space showing an occlusion
ambiguity for w < (ds - d b)
16
For images of planes perpendicular to the observer, the continuous surfaces of true
matches will lie in the same d-planes, i. e. no surface will cut through several
d-planes.
Thus, a spurious match in a given-d-plane will not have very many neigh-
boring matches in that d-plane, while a true match will be surrounded by other
matches.
By imposing this kind of test on the d-space, spurious matches can be
eliminated.
3. 2
Ambiguities Due to Occlusion
If a section of the d-space of a scene containing a square floating before a
background is examined carefully, a region with no continuous surface is observed
on either side of the floating square.
This region, seen in Fig. 10a, is termed am-
biguous since no true match can be identified.
It is caused by the fact that some
parts of the background visible in one view will be hidden behind the floating square
in the other view (See Fig. 29).
For a square W pixels wide at a distance corresponding to a disparity
floating before a background at a distance corresponding to a disparity db,
ds'
there
will be two different types of occlusion ambiguity:
Case I:
W
! (d s- d b).
A section of d-space for this case is shown in Fig.IOa.
On either side of the nearer surface are regions of ambiguity that are (ds - db)/2
elements wide.
Case 2:
W < (ds - d b).
A section of d-space for this case is shown in Fig.10b.
If a small object is far enough in front of the background, two separate regions of
ambiguity will exist.
Further difficulty arises since the viewer can see both the ob-
ject and the region of space directly behind it.
For such cases, the nearer surface
will be chosen.
Tests on human observers have shown that "these regions of ambiguity are
always perceived as being the continuation of the adjacent area that seems to be farthest away." (12)
Continuing the occluded surface is a reasonable procedure that
should be incorporated into a depth algorithm.
able to decide which regions are ambiguous.
not contain a continuous surface is ambiguous.
x and y,
3. 3
In order to do this, we must first be
From Fig. 10, any region that does
This means that for a given value of
if none of the points in the d-planes, then that region is ambiguous.
STROUT - A Simple Program for Range Extraction
A program, STROUT, has been written that incorporates the techniques de-
scribed in Sections 3. 1 and 3. 2.
(See Section 2. 4).
Input to STROUT is a d-space generated by STEREO
Output will be an image array whose entries represent distance,
Such a range map may be displayed as an ordinary picture except
that the brightness of points in the image will indicate range. Since there are 36
not brightness.
17
d-planes in the disparity space, there can be only 36 values of range, and hence only
36 shades of gray in the range map.
The operation of STROUT may be divided into two steps.
First, continuous
surfaces of matches are found and ambiguous regions are marked.
matches should be rejected by this part of the program.
All spurious
In the second section, am-
biguities that have been previously marked are resolved.
The first section of the program will only find continuous surfaces of matches
that are contained in a d-plane, i. e.
surfaces that correspond to planes in visual
space. perpendicular to the observer.
A more complex program (and a more gen-
erally useful program) would search for arbitrary surfaces in the d-space (See
Section 3. 4).
For each value of x and y in the d-space, the d-plane with the maximum
number of matches in an N x N neighborhood centered at (x, y) is located (N is a
program parameter).
This value of disparity is associated with the point (x, y) and
the range follows directly from Eq. (1).
It is assumed that the point in d-space with
the most neighbors is part of a surface of matches.
Since spurious matches have
few neighbors, they will usually be rejected.
2LOWLIMV
If the maximum number of matches at (x, y) does not exceed N x
100
where LOWLIM is a program parameter, the point is marked as ambiguous. If
there are several d-planes with the same maximal number of matches, the program
arbitrarily chooses the nearest.
For a robot doing a navigational task, it would be
a good strategy to always pick the nearest of several possible choices.
Further ex-
amination of the scene may prove this guess incorrect, but at least no collision will
have occurred.
An improvement could conceivably be obtained by making our choice
in such a way that the resulting three-dimensional position is close to the threedimensional position of neighboring points.
After the above steps have been completed, the regions marked as ambiguous must be resolved.
A routine, RESOLV, searches the incomplete range line
by line to find marked regions.
When one is encountered, the first non-ambiguous
points to the left and right of the region are examined. Whichever point represents
a surface farthest from the camera is chosen to replace the marked ambiguities.
This completes the range map.
Fig. 11 is a stereo pair of a stick 64 pixels wide that slopes out toward the
viewer.
The top of the stick is farthest away.
by STROUT.
Fig. 12 is the range map computed
The background is dark (far) while the stick gradually brightens to-
ward the bottom of the image,
observer (which is true).
implying that the bottom of the stick is nearest the
There is a slight amount of irregularity along the lateral
edges of the stick due to chance variations.
18
The edges of objects are most suseep-
Fig. 11
A random-dot stereo pair of a plane that slopes
out toward the viewer.
Fig. 12
Range map produced by STROUT from the
stereo pair in Fig. 11.
In this image, bright-
ness is proportional to range, e. g. dark regions are far, lighter regions are nearer.
19
tible to errors by STROUT.
For this image (Fig. 12), N= 3, LOWLIM = 50 (i. e. if
less than 50% of the points in a neighborhood are matches,
the region is marked as
ambiguous.
Random Dot Images of Arbitrary Objects
3.4
Scenes composed of surfaces that are not planes pose additional problems.
It should be possible to search the d-space for the "best" arrangement of continuous
surfaces.
Because of the discrete nature of the d-space, the property of continuity
may not be obvious.
Fig. 13 shows a section of d-space generated from a scene containing a plane
surface that is skewed with respect to the cameras.
From Fig. 4, we would expect
such a surface in visual space to map into a hyperbola in d-space.
Because of the
discreteness of the input images and the d-space, the hyperbola is distorted by round.
off error.
If the surface in Fig. 13 were embedded in spurious matches, it would be
difficult to detect as a "continuous"
surface.
Unless round-off error is reduced by using images with more sample points
(e. g.
1000 x 1000), recognizing skewed surfaces near the observer will be difficult.
If we consider skewed surfaces far from the cameras, the hyperbola in d-space will
be flattened (as in Fig. 4) and not as susceptible to this type of noise.
As of this time, no program has been written (by me) to correctly process
arbitrarily shaped images. This failure is not as restricting as it appears, since a
program such as STROUT will produce reasonable results with arbitrary scenes as
input (See Section 6).
z
z, + 6
1/2
Z0
(b)
XX0
x + 12
0
PLANE IN VISUAL SPACE
(a)
Fig. 13 (a) Skewed plane in visual space.
(b) Cross-section of d-space that would be generated
Due to discreteness, the expected hyperbola does
not appear very continuous.
20
CHAPTER 4
IMAGES WITH REGIONS OF UNIFORM BRIGHTNESS
Scenes that contain surfaces of random dots are extremely rare in nature. In
most natural scenes, the probability of finding a particular brightness at a given
point in an image is conditional on the brightness of the surrounding points. It is not
unusual for connected regions of points to have the same or very similar brightness.
In a random-dot image such as Fig. 8 or Fig. 11, regions of uniform brightness are
very unlikely.
In order to move a step closer to real scenes, this chapter will consider
images that contain regions of uniform brightness.
to be used is the stereo pair in Fig. 14.
An example of the type of images
This stereo pair is similar to Fig. 8 except
that it is composed of random blocks, not random dots.
Each block is a 4 x 4 square
of picture elements that are assigned the same brightness.
When viewed stereo-
scopically, Fig. 14 depicts a square floating in front of a background (same scene as
in Fig. 8).
4. 1
Parallelograms of Matches
Before examing the stereo pair in Fig.14, let us first imagine a single square
of uniform brightness floating in front of a background of random dots. If a stereo
pair of images of this scene were obtained, the program STEREO could use them to
generate a d-space.
In this disparity space, we would expect to find a surface of
matches in the d-plane corresponding to the disparity of the background, and a
square of matches in the d-plane corresponding to the disparity of the floating square.
This will indeed be true, but there is more.
Fig. 15 is a cross-section of the d-space for the scene described above. Instead of just a line of matches at the position of the square (as in Fig. 9),
a parallelogram-shaped cluster of matches.
brightness of the square.
there is
This effect is caused by the uniform
If we imagine the two stereo pairs being shifted past one
another, an increasing area of the uniform square will overlap until the two squares
are exactly superimposed.
with additional shifts.
After this, the region of overlap will decrease to zero
Since the square has uniform brightness, any sections that
21
Fig. 14
Random-block stereo pair depicting a square
floating before a background.
d
x
Fig. 15
Cross-section of d-space generated from a
scene containing a square of uniform brightness.
22
overlap will generate matches in the appropriate d-planes.
In Fig. 16, this process
is illustrated with a one-line image containing a uniform region.
The expected par-
allelogram of matches is seen in the generated d-space.
Since the area of match increases (or decreases) by one unit for each unit of
shift, 1/2 unit must be added to (or subtracted from) each end of the line of matches
in successive d-planes.
This implies (and Fig. 16 verifies) that the slopes of the
sides of the parallelogram must be t 1/2.:Thus,the cluster of matches is really rectangular but appears to be a parallelogram due to differences in the scale of the display.
At this point, we have succeeded in locating the edges of the uniform region
in space, but there is no
information in the stereo pair that can be of help in deter-
mining the exact spatial location of the interior points of the uniform region.
Any
surface of matches that lies within the parallelogram of matches may be the true surface.
There is no texture on the viewed object to give a clue as to which surface to
choose.
Only at the edges of the uniform region where the parallelogram converges
to a single match can a non-arbitrary choice be made.
For human observers, the outline of the region of uniform brightness may
provide a suggestion of the three-dimensional shape of the object, e. g.
a region with
a circular outline might be seen as part of a sphere, while a region with a square
outline might be considered a flat surface.
These cues require object recognition
and will therefore not be used.
A reasonable compromise, then, would be to assume that the parallelogram
of matches
represents
some arbitrary surface in space.
We have located in three
dimensions the borders or boundary of the region, where is the rest of the surface
located? Since information is lacking, we could assume it to be a "minimum energy"
surface, obtained for instance by stretching a rubber
dimensional frame.
sheet
across
the three-
If we do this for the example discussed above, we will obtain a
plane.
In the output range map, these arbitrarily chosen surfaces could be marked
to indicate
their questionable nature.
This first guess (and it was a guess) might
then be modified by additional information from the scene.
Viewing the object from
a different position, or attempting to explore it with a feeler would
provide the
needed data.
If the surface of uniform brightness is not a plane perpendicular to the camera
axes, then the parallelogram of matches will be skewed.
For perpendicular surfaces,
the surface of minimum energy (the diagonal of the parallelogram) will lie entirely
within a single d-plane.
For surfaces that are tilted with respect to the camera
axes, the diagonal of the skewed parallelogram will cut through several d-planes.
23
Region of
Uniform Brightness
LEFT
I MAG E 201 7 15 10 132136 27
RI GH T 20 7 323217 2
I MAG E
6 27
Region of
Uniform Brightness
Do
20 7 15 10 32 32 6 27
20 7 3232 117
2
6 27
D,
D-SPACE
11 1o0lo oT1 1
_q%,
0D
I
21 10 732
32117 2
6
1 1 u I
0 0 11
0
27
0 1
10 10
0
0
0 101 0 101
D3
20 7 15 10 32 32 6 27
20
7 32 32 17 2
6 27
D4
i
[
207
15 110132 1321 6 1271
201 7 1321321171 2 16 127]
Fig. 16
D-space for a pair of one-line stereo images
containing a region of uniform brightness.
parallelogram of
A
1's is observed in the d-space.
24
Fig.17 shows a cross-section of d-space containing askewed parallelogramof matches.
The program EXPER to be described in 4. 3 will produce a surface of minimum
energy only for non-skewed parallelograms.
For skewed parallelograms,
an approx-
imation to a minimum energy surface is generated.
4. 2
Ambiguities Due to Occlusion
Lack of texture in a scene can make the resolution of occlusion ambiguities
difficult.
The simple technique described in Section 3. 2 for random-dot images will
no longer be effective.
Let us consider a scene in which a square of uniform brightness is floating
in front of a background of a different uniform brightness.
(If the two surfaces had
the same brightness, they would be seen as one surface).
In the d-space generated
from this scene, we would expect to find two parallelograms of matches in positions
corresponding to the disparities of the two surfaces.
The presence of a nearer oc-
cluding surface, however, will modify the parallelogram of the farther surface in an
interesting way.
Occlusion implies that part of an object is blocked from view by another
nearer object.
If the occluded object were of uniform brightness, then part of its
parallelogram of matches must be missing since part of the uniform surface is hidden
behind another object.
As the two stereo images are shifted past each other, the
blocked region will move 1/2 unit to the left (or right) in successive
d-planes.*
Figure 18 shows a cross-section of d-space generated from a scene in which an
occlusion occurs in the left view only.
Figure 19 shows a cross-section of d-space
generated from a scene in which surfaces are occluded in both views.
In this case,
the parallelogram due to the occluding surface lies within the parallelogram
of
matches corresponding to the farther, occluded surface.
The gaps in the parallelograms caused by an occluding object may be regarded
as "shadows" of an object.
lines with a slope of +
These shadows are projected by the occluding object along
Where the shadow of occlusion intersects a
1/2 and - 1/2 both.
or a surface of matches, there will be a region of ambiguity, i. e. no
matches will be present. If we re-examine Fig.10, the notion of shadows of occlusion
parallelogram
may be applied to textured surfaces also.
In general, any non-transparent physical
object in space must cast shadows of occlusion in a generated d-space.
This observation will prove useful in the next section for eliminating spurious surfaces of matches.
A general algorithm for removing the effects of occlusion ambiguities might be
implemented as
(1)
follows:
In d-space, identify surfaces of matches corresponding to
objects in vis-
(1 ual space.
* This assumes a d-space is being generated that references the center viewpoint
(cf. Section 2. 3).
25
'LA
x
Cross-section of d-space showing a skewed
Fig. 17
parallelogram of matches generated by a
tilted plane in visual space.
.
.
........ 1 11 1111, .....................
.................................
.........
..
..
..
..
..
..
..
.....
..........
..
..
..
..
..
..
..
......
....
...............
.......................................................
. ...........
.................
......................
..........................................................
..........
..........
..........
............. ........
.............
.............
.............
.............
.............
........
...............
..............
.............
...............
................
.............
...... . .........
........
....
............. .......
...........
............
.............
.............
.............
-
..........
.........
..........
.........
..........
.........
..........
................
.. ......
..........
..........
...............
................
.................
YO
............
......... ...
............
.............
............
...... ......
............
.............
.............
...... .....
...............
...............
.
..........
....
...............
...............
................
..........
.....
........................................................
.......
.............
................................................
............... .................
...........................................
.....................................................
............... :*" *............ ,........
...............
....
......
*
................................
left view
right view
(b)
(a)
Fig. 18 (a)
Stereo pair of a scene containing occluded
surfaces of uniform brightness in one view.
(b)
Cross-section of the generated d-space.
.....................................
...............
.........................
.
.....................
...................................................
...........
..................
I......................
.....................................................
.................
I...................................
..........
..........
..........................................
..........................
-........................
.............................
I................
I.......
............................................
......................................................
............................................. *............................................
-
*1*1111* ..............................
.....................................................
.............................................
.....................................................
1- 1.....................................................
..................................
I..........
..................................
11,11
.............................................
................
..............................................
11.......................
.......................
. ...............
..........
...............
................
.........
..............
................
*....
............
.............
YO
..........
.........
...............
...............
................
..........
..........
.........
..........
..........
...........
...
...............
...............
................
...............
...............
...............
...............
...............
................
.... I ..........
................
............
.....................................................
.....................................................
.....................................................
.....................................................
.....................................................
.....................................................
........................................
..
.............
............ 7
............
.............
.........
.........
........... VA
-.-.
......... ...
...........
*
.............
........
I ....
.............
...........
..............
'*
.............
...........
............
.............
............
.-.-.
.-.-.-..............
............
.............
............
.............
............
..............
...........
.............
............
..............
......................................................
.......................................................
......................................................
.......................................................
.......... ...........................................
....................
............................
......
.......... ............
.............................
.....................
.................................
........... ...................................
......
.......................................................
......................................................
....
I..... ............................................
I...........................................
.........
..................................................
.....................................................
............................................
.....................................................
.......................... - - ......................
.....................................................
.....................................................
.........................
..........................................
I........................
left view
x
(b)
right view
(a)
Fig. 19 (a)
Stereo pair of a scene containing occluded
surfaces of uniform brightness in both views.
(b)
Cross-section of the generated d-space.
26
(2)
Examine the regions that are in the shadows of occlusion cast by each
object - if a continuous surface or parallelogram of matches is cut by
a shadow, re-connect the region with a smooth curve or fill in the
parallelogram.
The method described above has not been incorporated into the programs to be
described
(although it certainly could be added).
As a result, the program EXPER
produces erroneous results when confronted with the situations in Figure 18 and
Fig. 19.
The disconnected parallelograms will be treated as separate surfaces,
yielding an incorrect range map. With additional programming (not possible due to
a lack of time), these cases could be properly handled.
4. 3
EXPER - A Program to Extract Range
The program STROUT described in Section 3. 3 looked for clusters of matches
in d-planes.
If a given position had several clusters in different d-planes, the one
implying the nearest surface was chosen.
matches will not occur in clusters.
This procedure assumes that spurious
This a good assumption for random-dot images,
but it will not be true for images with regions of uniform brightness.
Figure 20 is a section of the d-space generated from the stereo pair in Fig.14.
Since the images contain squares of uniform brightness, it is not surprising to find
many small parallelograms of matches.
They are organized into two continuous sur-
faces, one at a disparity corresponding to the background and the other at a disparity
corresponding to the floating square.
In addition, there are surfaces of matches re-
sulting from spruious parallelograms scattered throughout the space.
These spurious
matches must be rejected.
If a small neighborhood is examined in each d-plane, as STROUT does, then
incorrect choices will frequently be made by selecting the nearer of two possible
surfaces.
STROUT's performance could be improved by making the neighborhood
size larger than size of the uniform regions (i. e. N > 4 for Fig. 14), but that kind of
information about an image is not usually known.
Also, as the neighborhood size in-
creases, the resolution of the range map decreases.
A better strategy, then, would be to start by examining a small neighborhood.
If several possible surfaces are encountered, the neighborhood size is increased
until a single surface is found.
In this way, the largest organization of matches will
be chosen.
Small spurious surfaces of matches will be rejected in favor of larger
surfaces.
This is the procedure used by the program EXPER (See Appendix C for a
program listing).
In all other aspects, EXPER is identical to STROUT.
The type of
occlusion ambiguities described in the previous section will not be properly processed.
Figure 21 is a stereo pair of images representing a stick that slopes out to-
27
d
x
Fig. 20
Cross-section of d-space generated from the
stereo pair in Fig. 14.
Fig. 21
Random-block stereo pair of a plane that slopes
out toward the viewer.
28
ward the viewer.
It is similar to Fig. 11, except that it is composed of squares of
uniform brightness, not random dots.
variable.
The brightness of each square is a random
Given the d-space generated from these images as input, EXPER will pro-
duce the range map in Fig.22 (N = 3, LOWLIM = 50).
It is interesting to investigate
the errors, shown as bright dots along the edge of the stick.
Figure 23 is a cross-section of the d-space generated from Fig. 21 at a value
of y corresponding to one of the error points.
If we examine the regions on either
side of the near surface (the surface with a larger disparity),
find no surfaces of matches,
Section 3. 2.
we would expect to
since this is a region of ambiguity as described in
On the left side, however, a spurious parallelogram of matches occurs
in the ambiguous region.
EXPER will choose this surface since there are no larger
ones at that position, and it will erroneously appear in the output range map. Such
problems did not occur in random-dot images, since spurious matches did not form
surfaces.
For images with regions of uniform brightness, it is possible to find sur-
faces (actually parallelograms) of spurious matches.
We know from Section 4. 2,
however, that these spurious surfaces cannot be a real surface,
cast shadows of occlusion.
since they do not
The next section will describe a program that takes this
property into account, and properly eliminates surfaces that do not cast shadows of
occlusion.
4. 4
FUSER - A Program That Implements Shadow-Checking
As mentioned in Section 4. 2, any non-transparent physical object must cast
shadows of occlusion in d-space.
This property of surfaces of true matches will al-
low surfaces of spurious matches, as shown in Fig.23, to be identified and rejected.
By examining Fig. 23, we can see that the surfaces of matches that corrrespond to
real objects do cast the required occlusion shadows, while the spurious parallelograms of matches do not.
Thus, for every surface of matches in d-space, we must
check along lines of + 1/2 slope and
shadows.
- 1/2 slope to test for the presence of occlusion
This additional test for the validity of surfaces of matches is made nec-
essary by the fact that images with regions of uniform brightness are likely to contain spurious surfaces of matches.
For random-dot images, spurious surfaces of
matches are unlikely enough to be ignored.
In order to reduce the effort involved in writting a program to check for occlusion shadows, an interesting property of d-space in conjunction with an existing
program EXPER (See Section 4. 3) were utilized.
The method to be described is not
very efficient with respect to computer time, since it requires the formation of two
disparity spaces from each stereo pair.
A more straightforward method (but more
difficult to program) would utilize a single disparity space. The purpose of writing
this program is merely to demonstrate the effectiveness of occlusion shadow-checking
in removing spurious surfaces of matches.
29
Fig. 22
Range map produced by EXPER from the stereo
pair in Fig.21. Brighter points are nearer.
SPURIOUS SURFACE
di
x
Fig. 23
Cross-section of d-space generated from the stereo pair in
Fig. 21 showing spurious surfaces that appear in range map.
30
I
0 aw -- 66-60
In Section 2. 3, we discussed the different viewpoints that a d-space may
If the scene shown in Fig. 24 is viewed from the position of the left camera, the near object, S,
appears directly in front of the ambiguous
region, AL'
This region is ambiguous because it is seen only by the right camera and hence will
have no true match in d-space. Similarly, when the scene is viewed from the viewreference.
point of the right camera, the object
SI appears directly in front of the ambiguous
region AR.
Thus, by generating d-spaces from the left viewpoint and the right viewpoint, occluding objects will be vertically aligned with one of their occlusion shadows.
Figure 25 shows three cross-sections of the d-space generated from the scene in
Fig. 24.
The section of d-space in Fig. 25b is from the central viewpoint, and the
occluding object does not align vertically with either of its occlusion shadows. In
Fig. 25a and
25c, the occluding surface lies directly above one of its occlusion shadows. If EXPER were applied to the left viewpoint d-space (Fig. 25a) and the right
viewpoint d-space (Fig. 25c), two range maps would be produced.
Surfaces of
matches that did not align with occlusion shadows would not be present. Surfaces
that are present in both range maps therefore must have cast both the left (slope = +1/2)
and the right (slope = -1/2)
occlusion shadows.
The purpose of the program, FUSER,
is to transform the left viewpoint and right viewpoint range maps to the same point of
reference, the center viewpoint, and compare them. If points at the same position in
the two transformed range maps are seen at the same distance in space, then those
points cast the required occlusion shadows and no change is made. If points at the
same position are seen at different distances in the two range maps, then the point
is marked as ambiguous. After the entire frame has been examined the routine,
RESOLV, described in Section 3. 3, is used to resolve all points that have been
marked as ambiguous.
The effect of FUSER, then, is to determine which points in the final range
map resulted from identifying surfaces of matches in d-space that do not cast occlusion shadows. These points are labeled as ambiguous and the routine RESOLV
finds the surface
that they occlude.
Thus,
an "illegal"
surface
is stripped
away.
To transform from left viewpoint d-space to center viewpoint d-space, each
d-plane Dn is shifted n/2 units to the left. To transform from right viewpoint
d-space to center viewpoint each d-plane,
Dn, is shifted n/2 units to the right. The
alignment of d-planes for the different viewpoints is shown in Fig. 6.
Figure 26 is a schematic description of the entire process, starting with the
viewed scene. The surface S1 occludes parts of the background, B. In the d-space
generated from a stereo pair of images of the scene, the surfaces
served.
present.
S1 and B are ob-
Additional spurious surfaces which do not cast occlusion shadows are also
Neither of these spurious surfaces will be present in both the left and right
31
OCCLUDED REGIONS OF THE BACKGROUND
AR
AL
BACKGROUND, B
OCCLUDING OBJECT, Sl
z
x
LEFT CAMERA VIEWNT
Fig. 24
RI GHT CAMERA
Scene containing an object, S1, in
front of a background surface, B.
32
dI
X
(a)
dl
X
(b)
d
x
(c)
Fig. 25
The same cross-section of d-space,
but from different viewpoints.
(a)
left viewpoint
(b)
center viewpoint
(c)
right viewpoint
33
B
S1
z
STEREO CAMERA
t.X
a) VISUAL SPACE
STEREO
71- 7
d
R
SPURIOUS
SURFACES
B
b) d-space of the scene above
containing spurious surfaces
STEREO
SPURIOUS
SURFACES
~
Si -
STEREO
j
-7 B
B~
right viewpoint d-space
c)
left viewpoint d-space
EXPER
EXPER
Si-
Si
.-
B
d) d-spaces after EXPER has identified largest surfaces
FUSER
of matches and
resolved occlusion
ambiguities FUSER
e) d-spaces shifted to central VIEWPOINT by FUSER
B
f)
Fig. 26
S1
B
Final d-space after FUSER has resolved conflicts.
The spurious surfaces have been eliminated.
Elimination of spurious surfaces
by occlusion shadow -checking.
34
B-
viewpoint range map (See Fig. 26e), and they will be rejected. The final step(Fig. 26f)
shows a d-space in which the spurious surfaces have been eliminated, leaving the desired surfaces that represent physical objects in space.
If the method described above (using FUSER to check for occlusion shadows)
is applied to the random-block stereo pair in Fig. 21, the range map in Fig. 27 is
produced.
Note that the errors observed in Fig. 22 have been eliminated.
35
Fig. 27
Range map produced by FUSER
from stereo pair in Fig. 21. Errors have been eliminated.
36
CHAPTER 5
STEREO IMAGES OF NATURAL SCENES
Stereo images of a real scene may be obtained by digitizing photographs taken
with a camera (the Stereo Realist, a commercially available 35 mm camera,' exposes
two images simultaneously),
puter.
or directly through a TV camera connected to a com-
The problems encountered by either method are similar, and no distinction
between the two will be made.
Figure 29 is a stereo pair of images of a simulated
It was obtained by scanning the scene with the vidicon camera system
Mars scene.
described in Appendix C.
Each image consists of 256 x 256 points with 64 possible
gray levels.
5.1
Noise
In order for an image to be encoded into an array of numbers suitable for
computer processing, light levels must be measured and quantized.
Noise is intro-
duced into the image by shot noise in the light sensor and round-off error in the
quantizer. For a high quality image encoding system, noise present in the recorded
image usually cannot be detected by a human observer.
It is there, however, and
will cause difficulties for stereo processing.
The amount of noise in a digitized image may be measured by the following
scheme:
(1)
Two images of the same photograph or scene are recorded successively taking care to keep all parameters of the digitizing system
constant.
(2)
Corresponding points in the two digitized frames are subtracted to form
a difference frame.
(3)
A histogram of the entries in the difference frame will give the
probability distribution of the noise in the images.
*Available from the David White Instrument Division, Realist Inc., Menomonee Falls,
Wisconsin.
37
(4)
From the probability distribution, the mean and standard deviation
may be computed.
If no noise were introduced by the digitizing process, all the entries in the
difference frame would be zero. Figure 28 shows the probability distribution of the
noise that was measured using the equipment described in Appendix C.
The distri-
bution appears to be Gaussian with a mean of zero and a standard devition of 2 (out
of 64 possible gray levels). Thus, if a point in an image is digitized by this equipment, 68% of the time the value will be within 2 units of the true value. When two
digitized points are compared, they can be said to represent the same brightness
with a probability of .68 if they are within 4 units of each other. If the tolerance of
match, E in Eq.(4), is set equal to twice the standard deviation of the noise distribution, then 68% of the true matches in a stereo pair will be discovered. By increasing the tolerance of match, the percentage can be made larger but the number of
spurious matches will increase too.
The price that is paid for noise in the image is
an increase in the number of spurious matches that will be found in the d-space.
Figure 30 shows several cross-sections of the d-space generated from the stereo
pair in Fig. 29.
The tolerance of match was 4.
matches can be observed,
Many large parallelograms of
since the images have large regions that appear to have
uniform brightness due to the tolerance on matches.
If the tolerance was set to zero
(as it is for computer generated images which contain no noise), no recognizable surfaces would be observed in the d-space.
5. 2
Registration
For stereo images that are obtained through two different optical systems,
the registration of the images with respect to one another may be a problem. Since
the images will be quantized and stored as an array, vertical or horizontal misalignment, if it is detected, can be easily corrected by shifting the image several rows or
columns to define a new origin for the coordinate system. If the images are rotated
with respect to each other, correcting for this type of error is a time-consuming
procedure. Rotating a quantized image involves more than applying the usual transformation, x' = x cos 9 - y sin 0 and y' = x sin 0 + y cos 0,
used for the continuous
case. (13)
All of the above difficulties point to the desirability of including some kind of
fiducial marks in the images. A scanning algorithm could then use these marks to
compensate for vertical or horizontal misalignment, and perhaps warn if a rotational
misalignment has occurred. Unfortunately, fiducial marks were not included in the
images discussed in this chapter.
They were aligned by a trial and error procedure
which is not very accurate or reliable.
If the distance between picture elements in the digitized image is comparable
38
NOISE
PROBABILITY
Fig. 28
Probability distribution of noise in the image encoding system.
Fig. 29
Stereo pair of images of a simulated Mars scene.
d
Fig. 30
X
Cross-sections of the d-space generated from the
stereo pair in Fig. 29.
39
to the size of small objects or details in the continuous image, accurate reproduction
of the image is not possible.
Moreover, if the two stereo images are slightly mis-
aligned, small objects will be represented differently in the two views. This type of
distortion may be eliminated by using a camera with a sufficiently narrow angle of
view, or using more picture elements per frame.
If such distortion does occur, an
a posteriori procedure such as averaging the image over small neighborhoods will
improve the quality of matching but will further degrade the resolution of the image.
5. 3
Range Maps for Real Images
The programs discussed in the preceding pages were designed to work well
for images containing only plane surfaces perpendicular to the viewer.Natural scenes
consist of arbitrarily-shaped surfaces in visual space,
to apply these programs to such images.
and it may seem unreasonable
Because of the way real objects in visual
space are mapped into d-space, however, they may often be approximated as flattened surfaces contained in a single d-plane.
(1)
Two factors contribute to this effect:
As objects become more distant, small variations in range become less
noticeable in d-space (See Fig.4) and surfaces tend to appear flat
(a
blank wall and a wall with an ant on it will look the same).
(a)
The d-space is discrete, not continuous, and therefore only certain
values of range can be measured (See Fig.7).
Intermediate values of
range will be "rounded-off" to the nearest measurable value, also
tending to flatten surfaces.
Thus many arbitrarily-shaped objected in visual space, particularly those
that are distant from the cameras, will become flattened in d-space and can be approximated by plane surfaces.
As long as this approximation holds, the stereo pro-
gram will produce accurate results.
When it does not hold, a curved surface will be
approximated by several planes parallel to the retina, as in Fig.31.
By improving
the program EXPER as suggested in the next chapter, this type of approximate behavior can be replaced by more accurate results.
The stereo pair in Fig.29 were used by STEREO to generate a d-space
With this d-space, EXPER (N = 13, LOWLIM = 50) produced the range
map shown in Fig. 32. No occlusion shadow checking (described in Section 4. 4) was
performed, and several errors (black regions) can be seen near the base of the rock.
(EPS= 4).
No detailed structure is observable in the range map, but it clearly indicates that the
rock is in front o*f the backdrop (since it is brighter) and the stick slopes out toward
viewer (it gets brighter toward the. bottom of the frame).
The crater in the back-
ground is painted on flat surface and hence does not appear in the range map.
Figure 32 shows the output produced by applying EXPER to the left and right
40
d
-
-~
Fig. 31
An arbitrary curve in d-space approximated by planes parallel to
the cameras' retina.
Fig. 32
Range map of stereo pair in Fig. 29
with no occlusion shadow-checking
performed.
Fig. 33
Range map of stereo pair in Fig. 29
with occlusion shadow-checking performed.
41
viewpoint d-spaces, and using FUSER to generate a range map in which surfaces
that do not cast occlusion shadows are eliminated.
Notice that the errors around
the base of the rock have been removed.
Thus, in spite of the restriction of being able to identify only plane surfaces
perpendicular to the cameras, the programs described above are capable of producing a low resolution range map from a stereo pair of images of a natural scene.
This result is encouraging,
and with further tests, it may be possible to substan-
tially improved the quality of the range maps.
42
CHAPTER 6
CONCLUSIONS AND SUGGESTIONS
The previous chapters have described a method for extracting range information from a pair of stereo images. The procedure consists of two steps:
(1)
A three-dimensional disparity space is generated from a pair of stereo
images.
Surfaces corresponding to the surfaces of objects in the viewed
scene may be found in this space in addition to spurious surfaces that do
not represent real objects.
Ambiguities due to occlusion and lack of
texture will also appear.
(2)
The generated d-space is examined in order to reject spurious matches
and resolve ambiguities.
For each point
(x, y) in the d-space, an at-
tempt is made to find the largest surface (i. e. the surface containing
the most matches) that casts the required shadows of occlusion.
If
such a surface is found at a disparity d0 , for instance, it is assumed
to represent an object in space at coordinates
(x, y)
with a disparity
d , and hence a range of z = K (by Eq. (1)).
If no such surface is found,
0
do
the point (x, y) is considered ambiguous and assigned a range value according to the procedure described in Section 3. 2.
In this way, a range
map will be created in which a distance, z, is associated with each value
of
(x, y).
The procedure described above in Step 1 has been implemented in the program
STEREO. The goal of this program is to generate a d-space containing as few
spurious matches as possible in order to simplify the processing in step 2.
To im-
prove the performance of STEREO the following suggestions are made:
(1. 1)
Use color information when forming a d-space.
Many points that have
the same brightness in a black and white image will have different
colors.
If the points in the images were considered as three-element
vectors representing contributions from three primary colors, then a
match would occur if the magnitude of the vector difference between
two points were less than a given tolerance. It is expected that color in-
43
formation would significantly reduce the number of spurious matches in d-space. Reflection noise will still be present, however, since a point may look green in one
view and red-green in the other if it reflects the color of a nearby red object.
(1. 2)
Provide for automatic registration of the two stereo images (as described in 5. 2 for example).
For processing images of natural scenes,
it is necessary to ensure the proper alignment of the cameras, or the
recorded images.
(1. 3)
Study the effects of shot noise and reflection noise on the performance
of STEREO. A better signal-to-noise ratio may be obtained by using
a scanning system with more shades of gray (or color).
(1. 4)
An investigation should be made of the effect of the image resolution
on the generated d-space.
A higher resolution image (i. e. more
pixels per frame, or a narrower angle of view) should improve the
quality of the d-space, since the effects of discreteness (See Fig. 13)
will be diminished.
The procedure described in Step 2 has been partially implemented by the programs EXPER and FUSER.
EXPER will attempt to locate large surfaces of matches
in the left viewpoint d-space and the right viewpoint d-space, and FUSER checks these
proposed surfaces for the required shadows of occlusion.
In searching for surfaces
of matches, EXPER will commit two kinds of errors that may be easily corrected.
(2. 1)
EXPER, as it is presently written, will only recognize surfaces of
matches that are contained in a single d-space.
Surfaces that cut
through several d-planes may not be detected at all.
This is a serious
drawback unless the surfaces in the scene can be approximated by
planes that are perpendicular to the axes of the cameras. To modify
EXPER to handle the general case (arbitrary surfaces) will require
additional programming.
Instead of examining one d-plane at a time,
EXPER will have to examine several adjacent d-planes.
(2. 2)
As mentioned in Section 4. 2, EXPER does not properly process occluded surfaces of uniform brightness.
In real scenes, this failure
will often produce a distorted range map.
In order to overcome this
error, it may be necessary to find the nearest objects in d-space
first, compensate for their projected shadows of occlusion, and then
search for the next farther object.
If this method is applied recur-
sively until all surfaces have been examined, then the proper result
will be obtained.
This method deserves further study.
The programs that have been described were written for the purpose of exploration - flexibility was stressed, not speed.
44
Presently, the time required to pro-
duce a final range map (output from FUSER) is about 35 minutes of time of the PDP-9
computer described in Appendix C.
erations.
Most of this time is spent doing input/output op-
Because of the limited memory available,
can reside in the computer at a time.
only one of the three programs
Therefore, a program such as STEREO must
create an output tape of partial results to serve as input for the next program.
By
combining the programs so as to eliminate these redundant operations, operating
time might be reduced by a factor of 5.
Using a more sophisticated computer with
larger memory, or perhaps special purpose hardware,
range maps might be gener-
ated from stereo pairs in about a minute.
There are many possible applications for a computer program that can create
a three-dimensional view of a scene from a stereo pair of images.
Many scientific
experiments produce large numbers of stereo pairs that must be analyzed to extract
range information.
stereo pictures.
Oceanographers,
for example, measure ocean currents by taking
In one type of experiment, a dense pellet of dye is dropped from a
ship producing a vertical column of colored water.
When photographed some time
later, the deformation of the column is a measure of the direction and velocity of
currents at different depths.
produce 500 stereo pairs.
A small submarine photographing the dyed column may
The ability to automatically process these images to ex-
tract the desired information, would be of great aid to the experimenter.
Stereo processing of images would also be of use to a robot attempting to
navigate through an environment.
Since the algorithm presented does not depend on
knowledge of the type of scenes to be viewed,
a robot could maneuver through un-
known terrain avoiding obstacles in its patch such as boulders and holes in the ground.
45
REFERENCES
(1)
Gregory, R. L., Eye and Brain, McGraw-Hill, New York 1966, pg. 50.
(2)
Davson, H. ed., The Eye vol. 4
Visual Optics and The Optical Space Sense,
Academic Press, New York, 1962.
(3)
Krech,
New York, 1967,
(4)
Elements of Psychology, Alfred A. Knopf,
D. & Crutchfield, R. S.,
pp. 134-137.
Julesz, B., Depth Perception of Computer-Generated Patterns, Bell System
Technical Journal,
39 (1960)
pp. 1125-1161.
(5)
Gregory, R. L., op. cit. pg. 56.
(6)
Guzman, Adolfo,
Computer Recognition of Three-Dimensional Objects in a
Visual Scene, Ph.D. Thesis, M.I.T., E. E. Department,
(7)
Guzman, A. op. cit., pp. 233-238.
(8)
Julesz, B.,
Towards
(9)
(10)
439
-
1962, North Holland Publishing Co.,
44 3
.
1963, pp.
(AD-692-200)
the Automation of Binocular Depth Perception,
Proceedings of IFIP Congress,
Amsterdam,
1968,
Davson, op. cit., pp. 408-417.
Julesz, B., Texture and Visual Perception,
Scientific American, Vol. 212,
No.2, Feb. 1965, pg.45.
(11)
Disbrow, J. R.,
Studies in Stereoscopy,
S.M. Thesis, M.I.T., E. E. Depart-
ment, 1964.
(12)
Julesz, B. , Texture and Visual Perception, op. cit., pg. 46.
(13)
Simon, C. J., Examples of Computer Processing of Quantized Images, unpublished.
(14)
Horn, B.K. P.,
Shape from Shading:
A Method for Finding the Shape of a
Smooth Opaque Object from One View.
ment, 1970.
46
Ph. D. Thesis, M.I.T., E. E. Depart-
APPENDIX A
The Relationship of Range to Disparity
Figure 1 depicts a typical configuration for obtaining stereo images. Two lenses
of focal length f are separated by a distance a, the interoccular distance. For the
work described in this thesis, the optical axes of the lenses are arranged to be parallel, i. e. they converge at infinity.
A point P in the cross-hatched region will cast an image in both the left and
right image planes. The difference in position of the two images, dr-dt, is known
as the disparity of point P.
For a simple lens, the distance from the lens to the object is related to the
distance from the lens to the image by the following formula:
d.
0 1
d + d.
1
o
d
If d >> d
=
f
where d
=
distance from object to lens
d.
=
distance from image to lens
f
=
focal length of lens
(A-1)
as is the case for a typical stereo configuration, the Eq. (A-1) be-
comes:
(A-2)
d.1 = f
With this result, the relationship between disparity and range can be derived.
From Fig. 1, we may form the following ratios by similar triangles:
dr
a + w
f
z
-
=
Solving for w
w
z
f
in Eq. (A-4) and substituting into Eq. (A-3) we obtain:
afd +d
47
(A-3)
(A-4)
(A-5)
But d
r
is the difference in position of the image of point
-d
P in the two views. This
-L
is the disparity, d.
Letting K I= af , Eq. (A-5) becomes:
Z =
(A -6)
1
d
Thus the range z
is inversely proportional to the disparity d. K 1 is a con-
stant that depends on the optics and geometry of a particular stereo system.
For digital images, the disparity, d,
elements, not inches.
will be measured in terms of picture
An image scanning system may be characterized by the num-
ber of picture elements per inch that it will resolve.
For a system with m pixels/
inch, Eq. (A-6) may be rewritten as:
nK
z
d
where K
z
= K
d
(A-7)
= ma f
= range in inches
disparity in pixels
d
For the camera system used for this study,
m = 1024 pixels/inch
,
a
= 2.0
inches
f
= 1.0
inches
K
=
2048 inches/pixel
If the maximum allowed disparity is 36 pixels, then the nearest range permitted in the images should be:
z =
2048
d
2048
36
57 inches
The camera system, then, will cover ranges from infinity to 5 feet from the
camera.
48
APPENDIX B
A Limit on the Resolution of Range
To be able to resolve a distance
Az at a range z0 implies that z
and z t Az
must have different integer values of disparity. For a system described by Eq.(A-6):
K
K
K
-- iz>1/2
where
K
>
1/2
(B-1)
o
Z0
K
(B-2)
is the stereo configuration constant.
Solving (B-1)and (B-2) for K, we obtain:
z
(z
+ Az)
(B-3)
2Az
K >
z
K>
(z
- Az)
00(B-4)
2 Az
If inequality (B-3) is satisfied, (B-4) will also be satisfied.
solve Az at distance
z0
For a given K,
requires a stereo constant
K
Therefore to re-
given by (B-3).
the smallest Az that will cause a change in disparity is ob-
tained by solving (B-4) for Az:
Z20
Az
>
2K + z
(B-5)
From (B-5) it can be seen that as z0 gets smaller (nearer objects) the resolution will improve since smaller differences Az can be discriminated.
As z0
gets larger (objects far away), Az gets larger, implying that small differences can
not be resolved.
Thus,
small variations in range that may be detected in a near ob-
ject will not be detected if the same object is farther away.
49
APPENDIX C
Listings of Computer Programs
The programs described in Sections 2. 4, 3. 3, 4. 3 and 4. 5 were written for
a Digital Equipment Corporation, PDP-9. The PDP-9 is a medium scale computer
with a one-microsecond cycle time and a repertoire of about 40 instructions.
The
configuration for the particular machine used for this thesis is as follows:
(1)
8K of core memory (18 bit word length)
(2)
Two Dectape transports
(3)
A Colorado Video, Inc., Data Camera, Model 501.
This is an on-line
vidicon TV camera, that produces a 500 x 500 digital image quantized to
64 gray levels.
Geometric linearity is rated at 3%, but this figure is a
lower limit.
(4)
A Tektronix 536 oscilloscope and associated hardware needed to display a 500 x 500 image at 64 gray levels.
A P31 phosphor (fast decay)
is used in the CRT.
guage.
All of the programs that follow were written in the MACRO-9 assembly lan(See MACRO-9 Assembler Programmer's Reference Manual for details,
available from Digital Equipment Corp., Maynard, Mass.)
50
/PRO0RAM NAME:
STERE)
7/15/69
/THIS PkliGRAM WILL CORRELATE A LEFT IMA6E WITH A RIGHT IMAE
/USING TECHNIOUE SIMILAR T) H. JULESZ 'AUT'JMAP.
/THE F IRST 36 DISPARITY PLANES ARE CNEATED AND
/S1)RE
ON DL CTAPE
/'IHE 0UTJtUT TMAGE WILL HE 128 COLUMNS WIDE (Nkl 25h) DUE 10
/MEMAORY
L IAI TAlI ONS.
/]HE
FJLLjWING SWITCH :PTIJNS ARE AVAILAHLE:
/
ACSS UP, D-SPACE IS FROM LE.I VIEWPJINT
/
ACS6 UP, U-SPACE
IS
xJvO1 i<IGHI VIEWPOINT
/
NO SWITCHES IJPD D-SPACE IS
RkJM CENIER VIEWPJINT
/PARAME1 EkS
EeS
SC.)L
0
/)NLY JNE
R
SF
Jo
L)BSEl
R SH It I
LSHIFT
DCHIAS
/0PEN
/1)LEkANC E JF PJIN1-MAITCH
/STAklT INb CLLJMN OF C.R1ELAI TI.N
EACH
0
0
0
THESE eAtRAMETS SHIULD HE NUN
REUEFINE
IRSJ LINE OF R. 1.
/TJ
REDEFINE FIRST LINE J
L.I.
/1) REUEFINE FIRST COL
F R.I.
/TJ REuEINE F IRST CJL J. L.I.
/AVG RI LUM, + UCHIAS = AV
LI LUM
PAIR
/Ti
0
I
J
F ILES
.IJDEV 1,4,5,6
.INIT 4,0 p60
/GFA
1Evj
L- I .
.INIT
5,0, ')
. INI T
6, 1, LG
F I LE NAME
.JYS NAME
v)ES 5
.SEEK
/GET
R. I o
/GET
GUlPUT
)PEN
SLJI
SLOT
4,WJRD
ILE NAME
JMS FNAME
ME
2SS
.EN F
&
i ILE NAME
JMS
SLI'I
/.A
/.DAT
/-DAT
F ILE
& 0E
F~NAME
.ENTER A,#WOUR
/NOW RFAD SWITCHES
LAS
AND
SNA
JA P
(10000
/ACS5
o.4
LAC
DAC
(,JMP V2
VSWIT
JMP
OLJ
LAS
AND
SNA
(4000
JMP
.+4
/ACS6
UPI
51
UP?
4
5
6
1S
L-I.
I\PUl
IS R.. INPU'l
IS JUTrUl
)
LAC (JMP V1
DA C VSWIT
JMi ( L
LAC (LAC
JE
UDiL VSuIT
/MAIN Pk)GtAA
/OH FSET
LEYT & RI(HT IMAUES IF NECESSARY
LAC LOF SET
SNA
JiMP ROF
JMS TCOMP
UAC CTR#
ULOOPL
.REA
4e.,LINPUT,256
/WAS1IE A L-1. ROW
ISZ CTR
JMP OLOOPL
/COUNT WASTED ROWS
k
*F
LAC ROF'SET
SNA
iMP NOFF
JMS TCOMP
DAC CTR
LO.HPR
.READ bp0pkINPUT,2b6
/WASTE A k.l. ROW
ISZ CTR
JMe OLOOPR
/Cj'MPUTE VALUES FOR PROGNAM
NJFP
LAC EPS
-MS TCOMP
DAC "iEPS# /MEPS=
-(EPS)
LAC (RINPUT+372)
/246,246 ANkAYS
JMS TC )MP
DAC MEND# /-(EN[' Ue .. LINE)
LAC (OUTiUF+371)
JMS
CIMP
DAC .)ENU# /-(ENU
0J OUWU)
/START MAIN LOOP
LE.AU2
.READ 4.0,LINPUT,2b6
HRF.AL2
.HEAU
5,pRINPtJT,256
/ZERo) OUTPUT HUHFER
LAW 17402
DAC CTRI#
LAC (OUTHU--1)
['AC 10
DzM* I0
ISZ CTRI
JMP -- 2
/LOUe
/ZERO
TAHLE POR MATCH COUNTS
LAC (TABLE-1)
JLJ
DAC 10
LAW 17734 /2
DAC CTRI
UZM* 10
ISZ CTRI
A
1
PLANES
E -2 / R
/NOW CORkELATE LEFT A RIGHT LINES
52
-JIS COMPAR
.WRITF 6,0,JUTHU -2,256
LAC* LHEAD2+2
AND (5
/END
OF iILE?
SNA
JMP LkEAD2
.CLOSE 6
.CLOSE 5
.CLoSE 4
.wA1T 4
HLT
JMP GO
/RSTARI
/SU.H
/WkITE OUTPUI
HUIFEk
JUT INES
FNAIE
0
.INIT
LAC*
/READ
1, I'b,
NAME
uAC .+3
.WkITE
/cET
I ,PXxl
.CLOSE
I
IN F ILE NAME
ADDUESS
Ji-
MESSA
IEAD 1, 3, WURD, I I
.CLJSE 1
-.
EI- I L
LOOP-7
SW2
LAC (DAC
WJRD
DAC SW1
LAC (LAC Wl)kt+2
DAC SW2
LAW 17775
DAC CTm
DM TEt1P#
LAW 17775
DAG CTXI
LAC W.kRD+2
LRS 6
LAC 1EMP
LLS 6
DAC TEMP
IS! SW?
is/ C Tm
Swi
JMP SW2
bAC WJk1)
Isz SWI
IbZ CR
JMP
IS
JMY
H
LO P7
'NAME /KETURN
FNAME
TJ INAME+1
WJk
R
.HLoCK
M ES
41l)02
.ASCII
41 1002
'TYr-E
9 CHARACTEt<
M1ESS2
.ASCII
'TYPF
9 CHAACTEk NIbGH
13
/HEAE
53
LE1
IMAGE
IMA6E
F ILE NAME
F ILE NAME
' <01
5>
'<015>
* FS S3
411
eASC I
C MPA
'TYkE
9 CHAAGt
/CJMilAk,
'0
TAD
DAC
/")
/SlAK'I
/MAVE
DAC DISWkU#
DAC UISPTH#
kLI
1ENF1
L10N.
K. I.- j~il I
H:.hIZ LJNIALLY
00
/STAHT
LKOT LI NE
/MOVE L.I. OX6bIN
TO L.I.
804ih
/START MARKEN
/POINTER
U JUTPU T
LAW 17602
[AC CTR2# /128 WORUS CAN HE PHJGESSEJ
LAC* 10
/bEl
L.o. ELEMENT
JMS TC)MP
DAC TEA P
/ST-NE
115 NE(ATIVE
LAW 17776 /2 16-HIT PLANES
UAC WCTH#
/SET JUD-EVEN SIA[TCH
UZM UE#
JMS CWOU
/INCEMkKNI HY JNE
ISL 11ISPTX
/CJUNT PLANES
ISZ WCTH
JMP
INCH
A. K
11 /AU1!INUEA
PTk#
TAU SCJL
TAU LSHIKT
DAC 10
/POINTER
DAC LPTR#
LAC (OUTHUF
WL-)P
m i briT
J
1
SE I
HSHI r T
[AC
LAC (LINPUT
COMP2
)UI -lU 1
LEO I ANU
,-A(; ( INPU'i
TAi SCJL
Ek
WLl)JP
ISZ CTR2
/C)UN1
LEMENTS
SKP
.JMP* CJY1PAk
LAC UISWkU
TAD (2
DAC lISWRU
DAG DISPTh
tsL krjTR
LAC kPT8
/kFESET
DAG 11
.J Ae COMP2
CW
Jku
COMP1 I
/i)*OjoE.
wiTH LINK.
/INCREMENT
PJINIE.R TO
iY
'IWO
.1.
0 /LJOP RJUTINE P'dt C JAeAYr
LA N 17756
/18
HITS PEh W.)k
IJAC CTH1
LAC (400000
UAC MASK# /HIbEST Hi] = 4E NJ t-LANE
LAC* 11
TAU ICHIAS
IA
TE AP
SPA
MAbNITUDE
JMS TCOMP /bET
lAU MEPS
SMA!SZA
SK ij
JMS SET
/SET SIT IF T-RES H)LU EXCEEUEL
54
'NTALLY
LAC 11
TAD MEND
SPA
/CHECK
/NO
/YES
SKP
VSWI T
JMP
LAC
RCk
DAC
LAC
TAD
INCR
TO SEE
IF BU
EI-
EXCEEDED
MASK
MASK
OE
/ADVANCE MASK
(1
DAC OE
VI
V2
RAR
SNL
JMS
INC2
/IS JE ODD?
/YES
ISZ CTR1
/CJUNT BITS
JMP COMPI
JMP* CWORD
/INCREMENT
DISPT
LAC DISPIR
TAD (2
[AC DISPTN
'AD JEND
/CHECK T) SEE 1
SPA
JMP* INC2 /NO
0
HY IWO
.
INC2
JMP INCR
0
jzm
LAC
TAD
JmS
DAC
Jmp
NLOO P
LAC
TAD
DAC
ISl
-JMP
LAC
TAD
TAD
TAD
DAC
XX
LAC
/SET A HIT IN JUTPUT AND COUNT
N UM
WCTR
(X 16 PLANES) + I
/2
(3
TCOMP
CTH
NL))P+3
NUM
(22
/COUNT COMPLEIED t-LANES
NUvM
CTR
NLJO0
/COUNT
CTR1
(22
N UM
(ISZ
HITS DONE
TAHLE
.+I
/REPLACED
MASK
XOR* DISPTR
DAC* DISPTN
JMP* SET
TCOMP
EXCEEDED
/YES
#
SEl
UIHU
0
HY IS4 TABLE+N
/TWO'S C)APLEMENT AC
CMA
TAD (1
JMP* T COMP
55
T-iEA
/STUHAGE
TAHLE
RINLIT
LINIPUT
AHRNYS
.BLOCK 44 /2 X 18 UISPARITY PLANES
0
-0
.HLJCK 376
0
0
. HLOCK 376
577000
/HEADEX
0
376
JUlTHUo
.ENU
60
ENU J*
56
/PROGRAM NAME:
STROUT SRC
3/4/70
/FOR EACH COLUMN OF DISPARITY SPACEs 'STROUT' WILL CHOOSE
/THE PLANE WITH THE MAXlMUM NUMBER OF MATCHES IN AN NXN
/NEIGHBORHOOD. IF SEVERAL MAXIMA ARE FOUND 'STROUT' HAS THE
/FOLLOWING OPT IONS:
ACS3 DOWN, THE MIDPOINT JF THE NEAREST AND LONGEST
/
/STRING OF MAXIMA IS CHOJSEN.
ACS3 UP, THE FOLLOWIN6 OPTIONS ARE ALLOWED:
/
ACS4 DOWNP THE MIDPOINT OF THE NEAREST STRING J
/
/MAXIMA IS CHOOSEN.
ACS4 UP, THE MIDPJINT OF THE ARTHEST STRING )F
/
/MAXIMA IS CHOOSEN.
/JUTPUT
IS AN ARRAY OF SIX BIT NUMBERS REekESENTIN6 DISPARITY.
.L)DEV 1,4p5
.SCOM=100
N
LO WL IM
GO
15
/NEIGHHORHJOU SILZE
NEEDED J
iF (N*N)
/PERCENT
0
.INIT
.N IT
.INIT
l1pbJ6
4,
1,60
5,0,O60
.WRITE 1,2,MESS1,20
.CLOSE 1
.INIT Ij0OGO
.REAJ l,3pWJRKl,14
.CLOSE 1
GET IL
FPl
LAC (UAC WORD
DAC SWI
LAC (LAC WORD+2
UAC SW2
LAW 17774
DAC CTRI
DZM TEMP#
LAW 17775
DAG CTR2
SW2
LAC
LRS
LAC
LLS
DAC
WORD+2
6
TEMP
6
TEMP
ISZ SW2
ISZ CTR2
JMP SW2
SW!
[AG WORD
ISZ .- I
ISL
CTR1
iMP FPI
/OPEN
INPUT PILE
.SEEK bWtR[D
/READ SWITCHES
LAS
AND
(40000
/ASC3
57
UP?
kR
VALlD IlY
SZA
JMP
LAC
DAC
JMP
LAS
AND
SNA
TRY4
TRY4
/YES
(LAC MIDPT
-
/NO
SW6
SETUP
(20000
JMP .+4
/ACS4 UP?
/NO
LAC (LAC WORK+I
DAC SW6
JMP SETUP
LAC (JMP NEARST
LAC SW6
/SET UP INTERNAL PARAMETERS
SElUP
LAC .SCOM+2
DAC READl+2
DAC HUFFl#
/YES
/GET LOC OF FIRST FREE REGISTER
ISZ BUFFI
.
.
Al
LAC (OUTBUF+1
DAC 14
/INIT 0 UTPUT POINTER
JMS FMN
LAC LOWLIM
DAC Al
LAC N
CLL
DAC .+2
MUL
/COMPUT E N*N
xx
LACO
/N*N*LOWLIM
MUL
xx
DIV
/DIVIUE BY 100
144
LACO
/OPEN
/COPY
DAC THRESH#
OUTPUT FILE
LAC WORU+3
DAC WORU+2
.ENTER 4,WOR
FIRST (N-1) ROWS
/NUMHER
OF MATCHES NEEDED
/NUMBER
OK COLUMNS OF OUTPUT
LAC MN
READl
UAC CTR#
ISZ CTR
.REA
5*01 XXs256
LAC REAUl+2
TAD (400
DAC READl+2
ISZ CTR
JMP READI
UAC REAU3+2
LAW 17602
TAD N
DAC MCOLS#
/START TO PROCESS IMAGE
58
READ3
.READ
LAC
DAC
DAC
LAC
53 0PXXs256
BUFFH
START#
COLI#
MCOLS
DAC COLCTR#
/ZERO
MATCH
TABLE
LAC MATCHS
JMS CLEAN
FIRST N-1 COLUMNS
/COUNT
LAC MN
DAC CTR
Isz CTR
.WAIT 5
LAC MATCHS
CHER2
JMS COLCT
LAC START
TAD
(2
/TWO
W ORUS PER
ENTRY
DAC START
ISZ CTR
JP CHER2
DAC CoL'J#
/WORK LJOP
LAC
CHER3
.JMS
JMS
JMS
CLC
TAU
SNA
MATCHS
COLCT
MAX
CONV
WITH MAXIMUM
TJ UISPAHITIES
A SINULE MAXIMUM?
/CJUNT 00 MAXIMA
/LJ 1)K FJR PLANE
/CON VE NT MAXIMA
/IS
MAXCTR
TH ERE
JMP CHEk5- I
SW6
/CONVERT
CHERS
JMs STRING
XX
JMP CHEkS
LAC WJR)K+l
TO 64 LEVELS
CLL
MUL
/SEVERAL
77
DIV
44
TAD (-22
SMA! CLL
STL
LAC(
SZL
TA1
(1
UAC* 14
CHER6
ISZ COLCTR
SKP
/ZERO
WORK
JMP CHER4
TABLE
LAC
JMS
/COUNT
WORK
CLEAN
FIRST COLUMN
LAC COLI
59
MAXIMA
DAC START
LAC
WORK
JMS COLCT
JMS SUBTCT
ISZ COLI
ISZ COLI
LAC COLN
TAD (2
DAC COLN
DAC START
/SUBTRACT
FROM MATCHS
JMP CHER3
/RESOLVE AMBIGUITIES IN OUTPUT
CHER4
JMS RESOLV
/WRITE THE FINISHED ROW
JMS WRITE
/TEST IOR END OP FILE
LAC* READ3+2
AND (7
SAD (5
JMvP DONE
/MAKE ROOM FOR NEW ROW
JMS MOVE
JMP READ3
/CLOSE FILES AND QUIT
DONE
.CLOSE 5
.CLOSE 4
-WAIT
4
I
JMP G3
NEARST
LAC 11
TAD (LAC
DAC
.+1
XX
JMP CHERS
COLCT
/COUNT
0
DAC TEMP
MATCHES
/COUNT MATCHES IN A COLUMN
/SAVE TABLE LOCATION
LAC START
COLCTI
DAC
DAC
LAC
DAC
LAC
DAC
PTR#
11
MN
CT81#
TEMP
13
17776 /2 X 18 D-PLANES
DAC CTR2#
LAW 17756 /18 BITS PER WORU
DAC CTR3#
LAC* 11
/GET A WORD
SMA!RCL
/TEST AND GET NEXT BIT
JMP NOBIT /NO MATCH
ISZ* 13
/COUNT A MATCH
ISZ CTR3
JMP COLCT3
LAW
COLCT2
COLCT3
COLCT4
60
NOBIT
SUBTCT
ISZ CTR2
JMP*COLCT2
LAC PTR
TAD (400
/NEXT COLUMN OF
DAC PTR
DAC 11
ISZ CTRI
/END OF PATCH?
JMP COLCTI
/NO
JMP* COLCT
/YES
ISZ 13
/BUMP AUTOINUEX
JMP COLCT4
0
/SUHTRACT
LAC MATCHS
DAC 10
DAC 11
LAC WORK
DAC 12
256 WORDS
FIkST COL FROM MAITCHS
LAW 17734
DAC CTR
SUH I
LAC*
D-PLANES
/36
12
CMA
TAD (1
TAD* 11
IAC* 10
ISZ CTk
-JMP SUHI
JMP* SUBTCT
CL EAN
/ZERO
0
WORK JR MAICH
TAHLE
DAC 10
LAW
17734 /36
D-PLANES
DAC CTRI
uZkl*
10
ISZ CTRl
JMP
-2
JMP* CLEAN
MAX
SHCH
/L'JK FRk MAXIMA
0
LAC MATCHS
DAG PTR
ISZ PTH
LAW 17734 /36 U-PLANES
DAC CWR
D4 M TEMP
LAC* PTR
'lAD TEMP
SPA
JMP
INCR
LAC* PTH
JMS TCOMP
DAC
INCR
TEMP
ISZ PTR
ISZ CTR
JMP SRCH
/MAXIMUM FOUN
/NEXT
61
IN MA'IGH
TABLE
LAC TEMP
TAD THRESH
SMA
,JMP'AMBIG /THRESH
LAC TEMP
/THRESH-MAX
NOT EXCEEDED
JMS TCOMP
AMH 16
DAC TEMP
J*IP* MAX
CL C
/ITEMP=MAX NUMBER JF MATCHES
/MARK AMHIGUITY
JMP CHER6
CONV
CONVI
CONV2
i
/CUNVERT MATCH TAHLE T
SHIP TS
/CJUNT # *J MAXIMA
0WM MAXCTR#
L.AG MATCHS
)AC 10
LAC WORK
DAG 11
LAW 17734 /36 D-PLANES
DAG CTk
LAC* 10
SA1 TEMP
/IS IT A MAXIMUM?
.JMP FNUMAX
/YES
ISz CITR
.JMJ-p C ONVI
JMP* CJNV /MAXClR=#
FNUMAX
LAC CTR
TA!U (44
DAC*
/#
U
0
UZM
LAC
bAC
UAC
K)UNU
SHIFTS
11
ISZ MAXCTR
JMP CONV2
STRING
OF MAXIMA
/FIND
SLENTH#
WOkK
10
11
/COUNT
MAXIMA
'I I1!PJINTS 0h SlTINGS
LJNGEST STkING
/LEN6TH O
LAC MAXCTR
CMA
ST1
ST3
STEND
TAD (I
UAC MAXCTR
/GET FIRST
LAC* 10
DAC BEGIN#
/STAXT JP A NEW STRING
bAC TEMP
ELEMENTS IN STRING
/COUNT
bZvI STGCTH#
ISZ STGCTR
ISZ TEMP
ISZ MAXCTR
SK P
JMP STEND
/GET NEXT ENTRY
LAC* 10
SAD TEMP
JMP ST3
/SAVE START OF NEW STRIN6
LMQ
/COMPUTE MIUPT OF CURRENT STRING
CLC
TAU TEMP
62
TAD BEGIN
RCR
SZL
TAD (1
DAC
/ROUND
OFF NEARDR
TEMP
LAC* 11
/STORE IN TABLE
/IS CURRENT STRING LONGEST?
LAC STGCTR
TAD SLENTH
/LONGEST SO FAR?
SPA
/NO
JMP ST4
/YES
LAC STGCTR
CMA
TAD (I
DAC SLENTH
LAC TEMP
DAC MIDPT#
ST4
JMP*
Ri
R3
/YES,
MIDPT OF LONFEST
/ANY MORE?
/NO
STRING
LACQ
JMP ST1
RESOLV
/SAVE
CLA
SAD MAXCTR
GET NEW START
0
/RESOLV AMBIGUITIES IN OUTPUT
LAC HUKF1
/TEMPORARY STORAGE
DAC 10
LAW 17600 /128 COLUMNS
DAC CTR
LAC (QUTRUF+1
DAC 13
/PTR TO OUTPUT
LAC* 13
/GET AN ELEMENT
SAD (777777
/IS IT CLC?
JMP R2
/YES
DAC TOP# /NO, KEEP LOOKING
ISZ CTR
JMP RI
.JMP* RESOLV
R2
LAC 13
DAC* 10
/SAVE
LAC*
/GET
13
SAD (777777
/YES
JMP R4
DAC BOT# /NO,
CLC
/END
DAC* 10
LAC TOP
LOCATION OF
NEXT
/15
END
CLC
ELEMENT
IT CLC?
OF CLC STRIN G
CODE
CMA
TAD
TAD
SPA
JMP
LAC
R6
R7
(1
HOT
/HOT-TOP
/CHOOSE SMALLDST(0AXT HEST)
R5
TOP
UAC TEMP
LAC BUFF1
DAC PTR
ISZ PTR
LAC* PTR
63
STRIN
SPA
JMP Rs
R8
R4
LAC CTRl
LAC<TEMP
DAC* CTRI
ISZ PTR
JMP R7
LAC BUFF1
UAC 10
LAC BOT
JMP R3
ISZ CTR
JMP R2
CLC
DAC CTR
UAC* 10
LAC TOP
/NO
/YES
/RESET
TEMP STOkAbE
/CONTINUE
/SET END CODE
JMP R6
LAC BOT
JMP R6
/2'S
0
COMPLEMENT O
AC
CMA
TAD (1
JMP* TCOMP
T MNp
/COMPUTE
0
LAC O
CM A
-(N)
TAD (1
DAC MN#
JMP*
MOVE
I/
LAC
FMN
MOVE
LINES UP
MN
DAC CTR
/CTh= -(N-1)
ISZ CTR
LAC HUFFI
TA1
(-2
DAC 10
MJVFF
TAD (400
DAC 11
LAW 17402
DAC
CTRI
/COUNT
WORUS
LAC* 11
DAG* 10
ISZ CTH1
0MP .- 3
ISz
10
isz
10
ISL 11
Isz 11
ISZ GTk
JMP
J.v41e*
/S<IP HEADEK
/SKIP HEADER
MOVEI
MOVE
64
IN HUH Kl
WRITE
/WRITE OUTPUT BUFFER
0
LAW 17600
TAD MN
DAC CTR
DZM* 14
ISZ CTR
JMP .-2
.WRITE 4*0,OUTBUF.p256
LAC (OUTBUF+1
/RESET
DAC 14
JMP* WRITE
WOR)
.BLOCK
MESS)
412002
0
.ASCII 'TYPE 9 CHARACTER FILE NAME & NEW EXTENSION'<15>
MATCHS
/MATCH TABLE
MATCHS
.BLOCK 46 /36 D-PLANES
WORK
.BLOCK 46 /36 D-PLANES
WORK
OUTHUF
16 /FOR FILE NAMES
177000
0
.BLOCK 376
.END GO
EN D
65
/PROGRAM NAME:
EXPER SRC
4/8/70
/FOR EACH COLUMN OF DISPARITY SPACES 'EXPER' WILL CHOOSE
/THE PLANE WITH THE MAXIMUM N.UMBER OF NEIGHBORS IN AN NXN
/AREA. IF THERE ARE SEVERAL MAXIMA, MORE COLUMNS ARE
/CONSIDERED UNTIL A SINGLE MAXIMUM IS FOUND OR THE ALLOWED
/NUMBER OF RECURSIONS ARE MADE. IF THE RECURSION LIMIT IS
/REACHED, THE NEAREST MAXIMUM IS CHOSEN. AFTER RECURSIONs
/THE MATCH TABLE IS RESTORED TO ITS PRE-RECURSION VALUES.
/IF A REGION IS FOUND WITH LESS THAN 'LOWLIM' PERCENT OF THE
/POINTS MATCHED, THE REGION IS CONSIDERED AMBIGUOUS AND
/RESOLVED BY THE ROUTINE 'RESOLV'. OUTPUT IS AN ARRAY OF
/VALUES REPRESENTING SHIFTS NEEDED TO MATCH.
.IODEV 1,4s5
.SCOM=100
N
LOWLIM
RLIM
GO
/MINIMUM NEIGHBORHOOD SIZE
/PERCENT OF (N*N) NEEDED FOR VALIDITY
/MAXIMUM NUMBER OF RECURSIONS
12
62
20
.INIT 101sGO
.INIT 4,1,GU
.INIT 5POGO
.WRITE 1,2,MESSiP20
.CLOSE I
.
INIT IP,0#GO
13.WORDs14
.READ
GETFIL
.CLOSE 1
LAC (DAC WORD
DAC SWI
LAC (LAC WORD+2
DAC SW2
LAW 17774
DAC CTR1
FPI
SW2
DZM TEMP#
LAW 17775
DAC CTR2
LAC WORD+2
LRS 6
LAC TEMP
LLS 6
DAC
ISZ
ISZ
JMP
Sw1
TEMP
SW2
CTR2
SW2
DAC WORD
ISZ .ISZ CTRI
JMP FPI
/OPEN INPUT FILE
.SEEK 5, WORD
/SET UP INTERNAL PARAMETERS
LAC .SCOM+2
DAC READ1+2
TAD (1
DAC BUFF1#
FIRST FREE REGISTER
/GET LOC O
/SETUP FOR INPUT
/PTR
TO ROW BUFFERS
66
TAD (374
DAC BOTL IMLAC (OUTBUF+1
/INIT
DAC 14
LAC LOWLIM
DAC Al
LAC N
CLL
DAC .+2
MUL
/BOTTOM
OF FIRST ROW
OUTPUT POINTER
xx
Al
/OPEN
LACQ
MUL
XX
DIV
/DIVIDE BY 100
144
LACQ
DAC THRESH#
LAC RLIM
JMS TCOMP
DAC MRLIM#
JMS FMN
OUTPTT FILE
LAC WORD+3
DAC WORD+2
.ENTER 4,WoRD
/COPY FIRST (N-1) R)WS
LAC MN
DAC CTR#
READI
ISZ CTR
.READ 5,0,XX,256
LAC REAL1+2
TAD (400
DAC READ1+2
ISZ CTR
JMP REAUl
DAC READ3+2
LAW 176,02
TAD N
DAC
/START
READ3
MCJLS#
/NUlv'iEh
TO PROCESS IMAbE
.READ 5,PXX,256
LAC HUFF1
DAC START#
DAC COL1#
LAC MCOLS
DAC COLCTR#
/ZERO MATCH TABLE
LAC MATCHS
JMS CLEAN
/COUNT FIRST N-1 COLUMNS
LAC MN
DAC CTR
ISZ CTR
.WAIT 5
67
tJ
COLUMNS
OV OUPUT
CHER2
LAC
JMS
LAC
TAD
UAC
ISZ
JMP
DAC
DZM
/WORK LOOP
CHER3
LAC
JMS
JMS
JMS
CLC
TAD
SZA
JMP
LAC
CHERS
TAD
UAC
MATCHS
COLCT
START
(2
/TWO WORDS PER ENTRY
START
CTR
CHER2
COLN#
RFLAG#
/INIT RECUR FLAG OFF
M A TC H S
COLCT
MAX
CONV
/LOOK FOR PLANE WITH MAXIMUM
/CONVERT TO SHIFTS
MAXCTR
RECUR /RECURSE
11
(LAC
IF SEVERAL MAX
0+1
/GET NEAREST MAXIMUM
/CONVERT TO @ TO 63 LEVELS
CLL
M UL
/TIMES 63
77
XX
DIV
44
TAD
(-22
/DIVIDE BY 36
/ROUND OFF
5;VMA!CLL
STL
LACO
SZL
TAD (1
/STORE
DAC* 14
CHER6
ISZ COLCTR
SKP
JMF CHER4
/ZERO WJhK TABLE
LAC WORK
IN OUT HUFiEH
JMS CLEAN
/COUNT
FIRST COLUMN
LAC COLI
DAC START
LAC WORK
JMS COLCT
JMS SUHTCT
LAC COLl
TAD (2
/SUBTXACT FROM MATCHS
/TWO
WORDS PER ENTRY
/TWO
WORDS PER ENTrtY
DAC COLl
LAC COLN
TAD (2
/RESOLVE
bAC COLN
DAC START
JMP CHER3
AMBIGUITIES
68
CHER4
/WRITE
JMS RESOLV
THE FINISHED ROW
JMS WRITE
/TEST WKU? END OF FILE
LAC* READ3+2
AND (7
(5
SA
JMP DONE
/MAKE ROOM FOR NEW ROW
JMS MOVE
JMP READ3
/CLOSE FILES AND QUIT
.CLOSE 5
DONE
.CLOSE 4
.WAIT 4
HLT
JMP GO
COLCT
/COUNT
COLCTI
COLC12
COLCT3
COLCT4
NOIT
SUBTCT
/COUNT MATCHES IN A COLUMN
0
/SAVE TABLE LOCATION
DAC TEMP
MATCHES
LAC START
DAC PTR#
AC 11
LAC MN
DAC CTRI#
LAC TEMP
DAC 13
LAW 17776 /2 X 18 D-PLANES
UAC CTR2#
LAW 17756 /18 HITS PER WORD
DAC CTH3#
/GET A WORD
LAC* 11
/TEST AND GET NEXT HIT
SMA!RCL
JMP NOBIT /NO MATCH
/CJUNT A MATCH
ISZ* 13
ISZ CTR3
JMP COLCT3
ISZ CTR2
JMP COLCT2
LAC PTR
/NEXT COLUMN OF 256 WORDS
TAD (400
DAC PTR
DAG 11
/END OF PATCH?
ISZ CTR1
/NO
JMP COLCTI
/YES
JMP* COLCT
/HUMP AUTOINUEX
ISZ 13
JMP COLCT4
/SUBTRACT
0
LAC MATCHS
DAC 10
DAC 11
LAC WORK
FIRST COL FROM MATCrIS
69
SUBI
DAC 12
LAW 17734
DAC CTR
LAC* 12
36 b-PLANES
CMA
TAD (1
TAD* 11
LAC* 10
ISZ CTR
JMP SUBI
JMP* SUBTCT
CLEAN
/ZERO
0
WORK OR MATCH TABLE
DAC 10
LAW 17734 /36
LAC CTR1
DZM* 10
ISZ CTR1
JMP .- 2
D-PLANES
.JMP* CLEAN
MAX
/LJOK FOR MAXIMA
0
LAC MATCHS
DAG PTR
ISZ PTR
LAW 17734 /36
IN MATCH TABLE
D-PLANES
DAC CTR
DZM TEMP
SRCH
LAC* PTR
TAD TEMP
SPA
JMP
INCR
/MAXIMUM
LAC
AMBI I
INCR
LAC* PTR
JMS TCOMP
DAC TEMP
ISz PTR
[SZ CTR
JMP SRCH
FOUND
/NEXT
TEMP
TAD THRESH
SA A
JMP AMHIG
LAC TEMP
JMS TCOMP
DAC TEMP
/WORK+l
JMP* MAX
CLC
i
CONTAINS FINAL VALUE
JMP CHER6
CONV
/CONVERT
0
DZM MAXCTR#
LAC MATCHS
DAC 10
LAC WORK
DAC 11
MATCH TABLE TO SHIFTS
/COUNT # 01 MAXIMA
70
CONV I
LAW 17734 /36
DAC CTR
LAC* 10
SAD TEMP /IS
ISZ CTR
IT A MAXIMUM?
/YES
JMP FNUMAX
CONV2
D-PLANES
/NO
JMP CONVI
JMP* CONV
FNUMAX
LAC CTR
/# OF SHIFTS
TA) (44
/36 U-PLANES
DAC* 11
ISZ MAXCTR
/COUNT MAXIMA
JMP CONV2
RECUR
LAC
SZA
JMP
ISZ
LAC
DAC
LAC
UAC
/TEST RECUR FLAG
NOTNEW
RFLAG /TURN ON RECUR FLA6
MRLIM
CURCTR#
/CJUNT REC URS IONS
COLI
/FIRRT RECURSION
TEM.I1#
COLN
TEMPN#
MATCHS
10
LAG
RTABLE
DAC
13
DAC
LAC
DAC
LAC
NOTNEW
RECURO
RECUR2
RFLAG
/SAVE
LAW 17734
DAC CTR
LAC* 10
DAC* 13
ISZ CTR
JMP .-3
LAW 17776 /TWO
TAD COLI
SAD BUFF!
JMP RECURI
DAC COLI
DAC START
LAC MATCHS
JMS COLCT /GET
LAC COLN
SAD BOTLIM
JMP RECUR2
bAC START
LAC MATCHS
JMS COLCT /GET
LAC COLN
TAD (2
DAC COLN
JMS MAX
/NOW
JMS CONV
CLC
TAD MAXCTR
MATCH
TABLE
WORDS PER ENTRY
ANOTHER COLUMN
ANOTHER COLUMN
CHECK FOR MAXIM UM
71
SNA
.J 1P
rCL';Nh.
ISL
CUh'WTR
JMR RECUR
/ E':surE MAlCH TAHLE TO PRE-RECURSION VALUES
RDJNE
LAW 17734
DAC CTR
LAC RTABLE
DAC 10
LAC MATCHS
DAC 13
LAC* 10
DAC* 13
ISZ CTR
JMP .- 3
LAC TEMPI
DAC COLI
LAC TEMPN
bAC COLN
DZM R'LAG /RESET FLAG
JMP CHER5
LAC COLN
RECURI
SAD BOTLIM
/NO MORE COLUMNS
JMP RECLJR2+I
JMP RECURO
RESOLV
0
LAC
BUFFI
/RES
OLVE AMBIGUITIES
DAC 10
LAW 17600 /128
DAC CTH
LAC (OUTHUF+1
DAC 13
RI
LAC*
13
COLUMNS
/GET AN
SAD (777777
JMP R2
ELEMENT
/IS
IT HLC?
/YES
/NO, KEEP LNOKING
ISZ CTR
JMP RI
JMP* RESOLV
LAC 13
DAC* 10
LAC* 13
SAD (777777
JMP R4
/END NF CLC RTHINF
DAC HOT#
CLC
DAC* 10
LAC TOP
DAC TOP#
CMA
TAD
TAD
SPA
JMP
LAC
R6
(1
BOT
/BO T-TOP
/CHOOSE
SMALLEST(FAREST)
R5
TOP
ol
DAC TEMP
72
LAC BUFF1
DAC PTR
ISZ PTR
R7
R8
R4
LAC* PTR
SPA.
JMP R8
DAC CTRI
LAC TEMP
DAC* CTR1
ISZ PTR
JMP R7
LAC BUFFI
DAC 10
LAC ROT
JMP R3
ISZ CTR
JMP R2
CLC
DAC CTR
DAC* 10
LAC
JMP
LAC
JMP
TCOMP
TOP
R6
BOT
R6
/2'S COMPLEMENT OF AC
0
CMA
(1
TA
JMP* TCOMP
/COMPUTE -(N)
FM N
0
LAC N
CMA
TAD (1
DAC MN#
JMP* FMN
MOVEl
/MOVE LINES UP IN
0
LAC MN
DAC CTR
/CTR= -(N-1)
ISZ CTR
LAC HUFF1
TAD (-2
DAC 10
TAD (400
DAC 11
LAW 17402
/COUNT WORDS
DAC CTR1
LAC* 11
DAC* 10
ISZ CTR1
JMP .-3
MOVE1
ISZ 10
ISZ
10
/SKIP HEADER
ISz 11
ISZ
11
/SKIP HEADER
73
BUFFER
ISZ CTR
JMP MOVEI
JMP* MOVE
/WRITE
1141)
OUTPUT HUT hLk
AIN
IE
04A 1
/
WRI T E
V
J Wk I 1
BLO C K
WORD
M ES Si1
MATCHS
WORK
412002
0
.ASCII
'TYPE
WORK
HLOCK
44
RTABLE
*BLOCK
OUT HUt
/FN
FILE NAM ES
9 CHARACTER
PILE NAME
& NEW
/MA TCH TAHLFE
MATCHS
. BLOCK 44 /36 D-PLANES
.
RTAHLE
16
/WORK
TASLE
/36 D-PLANES
/STORAGE FOR
PNE-RECURS liN TAHLE
44
57 7000
0
. HLOCK< 376
. EN U
/
UlPUT HLirE.
(1,)
EN D
74
iATibSIN'<15>
FUSEk 4/16/70
/PkOGRAM NAME:
/COMHINE TWO VIEWPOINTS OF SPACE INTO ONE. CENTRAL
/VIEWPOINTs RESOLVIN6 AMBIGUITIES.
.ItLEV a4,5,6
.GLOBL IFILE,)0- ILL.
.SCt)Mx100
.JiS* IVILE
JMP .+2
/DF.VICK
(4)
.JS* IFILE
Jmp .+2
(I
/
(b)
. ESGKSt I i'l VIkf.we.)I'I
4 HAS LEPT VIEke')I-1
i
JFI LE
,J,14S*
.+2
/I)EVjIC'. 6 IS JUlpi1
LAC .SC130+2
DAC RVIELW#
DAC RREAI)+P
DAC JUT+R /HVIEW 15 JUTpul bu0-p-K
TA U (1000
(6)
IJ
DAC LVIEW#
DAG LREAD+2
LHEAD
kEAD
/SHIPI
.READ
490PXX#256
.REAU 5.0oXX#256
kl.GHT VIEW T-J CE.NTEH
LAC RVIEW
TAD (2
DAC PTH#
LAW 17600
DAC CTR# /128
CJLUMNS
5
.WAIT
RSHIFT
VIEWeJINI
LAC* PTR
SNA
SPA
J1P RSI
AND (77
RCR
/DISPAHITY
IS LKSS THMN
17
CMA
TAD (1
]AD
PTR
DAC TPTN#
LAC* PTR
DAC*
CLC
TPTN
DAC* PTI
/KEPLACE HJLL WITH AIHVIGUlY
ISZ ?TR
IS. CTR
Jwle RSH [1T
/SHIFT LEFT VIEW TO CENTER VIEWPOINT
SI
LAC LVIEW
TAD (202
75
LSHIFT
UAC PTR
LAW 17600
LAC CTR
LAC* PTR
SNA
JMP LS1
SPA
.jivp LSI
AN
D (77
HCk
SZL
TAU (I
TAD PT?'
DAC TPTH
LAC* PTH
DAC* TPTk
CLC
OAC*
I r H I6HEX
/HJUN4U
PTh
CLO
LSI
TAb PTH
IAC PTH
ISZ CTK
JMP
LSHIVl
SINOLE, Vlh.W
LAC LVI.W
iA)C
I
TS
10
LAG NVIE:W
TAUe (2
UAG eTN
'USK
I
LAW 17610
DAC Ct
L-A c*
10
SAu*
gTN
CLC
SAME'
JM
ST )k
/10- SAIEP
/FACE.a'T
Ir
SCAL-E
wHhLU11y
CLIL
MUL
7/
/TIMiS
63
DIV
lAD (-2d
HY
/blVII).
/RJUNI.)
36
)
44
SIA!CLL
S'I L
LAC
SZL
(1
UAC* PmT
ISZ PT H
ISZ CTH
J-1P USEI
ANY AMBIGUITIES LEEl
TA.
ST
Ok
/kdSOLVE
OVEx H<UM
LAC LVIEW
76
USIN6
DAC 10
LAW 17600
UAC CTH
LAC RVIEW
UAC 11
ISz 11
LAC* 11
SAD (777777
JM
R2
UAC TtP#
ISZ CTR
JM? OUT
LAC 11
UAC* 10
LAC* I1
SAD (777777
JMP R4
VAC HOT#
CLC
UAC* 10
LAC TUP
CMA
TAD (1
ITAD1)OT
SPA
LAC TO)P
UAC TEMP#
LAC LVIEW
UAC PTR
ISZ PTR
LAC* PTR
Rh
-
j;MP
IJAC TPTR
LAC Tk.vP
DAC* TPTR
ISZ PTH
timek-
R7
LAC LVIEW
DAC
LAC
JAP
ISz
.J.VP
CLC
10
HOT
R3
CTR
R2
DAC CIx
Rb
/OUTPUT
ul
UAC* 10
LAC TUP
Jim P X6
LAC H0l
JMP R6
T'4EF INISHEU LINE
-WRITE 6,0,XXs256
LAC* LVIEW
/CHECK
2
.7.
FOR ENU OF F ILE
(7
SAD (5
AND
-
SKP
J1P LREAU /CONTINUE
/CLOSE UP
.CLOSE" 6
5
.CLOSE
.CLOSE 4
JAP L10
.END G 0
END
/PXO(NHAM NAME: IFILE
10/P9/69
/LI RRAkY kOUTINE T) JPEN AN INPUT 0 ILE ON uAT SLJT = USN
/CALLIN6 SEQUENCV HXJM .OHTRAN:
CALL IFILEU(SN)
/HANDLEk
FJ
DAT SLJT ! IS LJAUD
AUTJMATICALLYS
/THERERHE FO
?'.RT8AN, US. USS 1Ck INeUI.
.IUDLV
I,4vb,6
- ULOHL -1 F I L.E, . DA
TTY= I
IF ILF
uSN
IL
.CLOSE TTY
LAC (DAC WORD
DAC SWI
LAG (LAC W')HD+2
01, C SW2
LAW
LOO
SW2
SW
.DA
Jivh.e .+2
.DSA
.INIT
TTY,1#0O
.W ITE TTY,2#MFSS 16
.INIT
TIY.,0#,6
.kEAD TTY#3,WlNDall
G0
GETO
0
JMSI
17775
DAG Cl#
DZviM TE'4P#
LAW 1777b
DAC C?9#
LAC WJRD+P
LHS 6
EMP
LAC
LLS 6
DtAC T MP
1S SW2
15Z C2
MP SWP
LDAC WND
ISZ .ISz Cl
IMJ LUUP1
Lf-C* DSN
DAC SEEK
DAG INIT
INIT
.INIT
SEKK.
SEEK 5, WJHR
,jmP* I FIL E
.WJRU
MESS
.BLOCK
Sp0sO
114
410002
0
.ASCII
'TYPE 9 CHAkACTEH
EN.U
79
INPUT teILE NA-E'<vwlb>