Range Imaging and Pose Estimation of
Non-Cooperative Targets using Structured Light
Dr Frank Pipitone,
Head, Sensor Based Systems Group
Navy Center for Applied Research in Artificial Intelligence
NRL Code 5515
Structured Light Range Imaging Technology
Conceptual Depiction
Target
Spacecraft
Structured
Light Source
Pursuer
Spacecraft
CCD
Camera
Pursuer illuminates target with structured light
and captures the reflected light on the CCD
focal plane array. A range image of the illuminated
surface is then generated using triangulation.
Structured Light Range Imaging Technology
Moving Correlation Code Triangulation
Structured Light Range Imaging Technology
Triangulation Geometry
1 ~ obtained from mask encoder
1 ~ obtained from illuminate d pixel location
R1 
sin( 1 )
B
sin( 1  1 )
Mask scanning motion allows range
mapping of all accessible points
Structured Light Range Imaging Technology
First Prototype Correlation Scanner
Scanner
Camera
Mask
Target
Range Image
(resolution ~ 0.1 mm at 1 ft)
Desktop Layout
Structured Light Range Imaging Technology
Second Prototype Correlation Scanner
Structured Light Range Imaging Technology
Conceptual Design of Spacecraft Coded Mask Scanner
Housing
Characteristics
Clear Window
Rotor
Motor
Drive Shaft
Coded Mask
Inner
Race
Encoder
Slit
Stator
Base
Bulb
Turret
Outer Race
Light Source
Slit
Integral Mask/Optical Encoder
Mask size, height x arc width
Radial distance to slit
Encoder error
Scan wheel
Xenon flash lamp
3 cm x 1 mm
Eighth Order DeBruijn Sequence
3 cm x 12.56 cm
10 cm
< 0.01 deg
10 rpm + 0.1 % , < 25 cm DIA
Pose Estimation Using Tripod Operators
Description and Key Properties
Twelve Point Tripod Operator
• Tripod operators are a mathematical procedure for rapid
recognition and localization of arbitrary surface shapes in
range images.
C
7
4
• TO’s consist of a set of equilateral rigid triangles joined on one
or more sides, with each joint characterized by a hinge angle.
5
1
2
B
A
3
9
8
6
• Random application to a scene is achieved by “flexing” the 9 hinges
computationally until all vertices lie on the surface
• Three parameters specify a placement: xA, yA, q, so a 3D manifold
(at most) of points in 9-space serves as signature of shape
• Only 12 vertex points are used; most pixels never visited
• Upon comparison with a trained model, the solution consists of
shape recognition and 6 DOF pose estimation, typically
occurring in tens of milliseconds
Pose Estimation Using Tripod Operators
Test Examples of Feature Recognition
Large torus is detected in 8 placements on a
synthetic image
90 degree dihedral is detected in 8 placements on a
LIDAR image with TO edge length of 7 cm
Each detection took approximately 30 milliseconds
Pose Estimation Using Tripod Operators
Industrial CRADA with NRL, Ford, and Perceptron
• Range image of a Ford torque converter generated with a Perceptron
LASAR scanner
• Feature recognition and pose is obtained using 12-point Tripod Operators developed at NRL
• Robot manipulator grabs torque converter from pallet based on 5-state pose estimation
Structured Light Range Imaging
& Pose Estimation Block Diagram
Structured Light
Projector
Mask
Wheel Speed
Control
Auto
GN & C
Range
Imaging
Control
Electronics
Wheel
Drive
Electronics
Scan
Wheel
Xenon
Flash
Tube
Strobe
Timing
Electronics
Flash
Electronics
Slit
Synchronization
Power
Supply
Processor
Pose Estimation
Target
•Interpret code for each pixel
•Generate 3-D surface image
•Apply tripod operator randomly over image
•Match tripod operators to model signature
•Match yields identity and pose of target
object relative to imager
Optical
Position
Encoder
Camera
Frame
Grabber
Structured Light Range Imaging
& Pose Estimation Advantages
 Enables accurate generation of three-dimensional surface models for any space
object, including cooperative and non-cooperative satellites and natural bodies
 Provides full aspect knowledge of target from many partial views (a complete
range image is obtained by meshing partial images)
 Six-state pose estimation is derived for an arbitrary surface shape without the
need for target fiducial features, protuberances, or reflective patterns
 Complete range image, feature recognition, and pose estimation are obtained
in a fraction of a second
 Potential application to a wide variety of complex problems such as AR&C,
inspection, surveillance, repair, and assembly