ROBOT VISION
Lesson 1a: Structured Light 3D Reconstruction
Matthias Rüther, Christian Reinbacher
Robot Vision SS 2013
Matthias Rüther
1
Structured Light Methods
 Goal: Robust 3D Reconstruction through triangulation
 Project artificial pattern on the object
 Pattern alleviates the correspondence problem
 Variants:
– Laser Pattern (point, line)
– Structured projector pattern (several lines, pattern sequence)
– Random projector pattern
Robot Vision SS 2013
Matthias Rüther
2
Structured Light Range Finder
1. Sender (projects plane)
2. Receiver (CCD Camera)
Geometry Z- direction X- direction
Robot Vision SS 2013
Matthias Rüther
Sensor image
3
1 plane -> 1 object profile
To get a 3D profile:
• Move the object
• Scanning Unit for projected plane
• Move the Sensor
Object motion by conveyor band:
=> synchronization: measure distance along conveyor
=> y-accuracy determined by distance measurement
Scanning Units (e.g.: rotating mirror) are rare
(accurate measurement of mirror motion is hard,
small inaccuracy there -> large inaccuracy in
geometry
Move the sensor: e.g. railways: sensor in wagon
coupled to speed measurement
Robot Vision SS 2013
Matthias Rüther
4
Robot Vision SS 2013
Matthias Rüther
5
Commercially Available
Person
Scanners
Cultural
Heritage
Rapid
Prototyping
Robot Vision SS 2013
Matthias Rüther
6
Problems of Laser Profile
 Occlusions:
Object points need to be seen
from Laser and Camera
viewpoint
 Sharpness and Contrast:
Both camera and laser need
to be in focus
 Speckle noise:
Laser always shows “speckle
noise”, caused by
interference of coherent light.
-> where is the center of the
stripe?
Robot Vision SS 2013
Matthias Rüther
7
Multiple Sheets of Light
Project multiple Laser planes simultaneously to reduce measurement time.
Problem:
Separation of stripes in the
image
Application:
Smoothness check of flat
surfaces
Robot Vision SS 2013
Matthias Rüther
8
Pattern projection
Projected light stripes
 Camera: IMAG
CCD,
Res:750x590, f:16
mm
 Projector: Liquid
Crystal Display
(LCD 640), f:
200mm, Distance
to object plane:
120cm
Range Image
Robot Vision SS 2013
Matthias Rüther
9
Projector
Lamp
Lens system
LCD - Shutter
Pattern structure
Line projector (e.g.: LCD-640)
Focusing lens (e.g.: 150mm)
Example
Robot Vision SS 2013
Matthias Rüther
10
Depth decoding
Project Temporal sequence of n binary masks. At each pixel, the temporal sequence of
intensities (I1, …, In) gives a binary number which denoted the corresponding projector
column.
Project  Acquire  Decode  Triangulate
Robot Vision SS 2013
Matthias Rüther
11
Coded Light + Phase Shift
Binary code is limited to pixel accuracy (or less).
Increase accuracy to sub-pixel by projecting sine wave after code
and measuring phase shift between projected and captured
pattern. Decode phase from four samples of sine period, shifted by
pi/2.
Robot Vision SS 2013
Matthias Rüther
12
Coded Light + Phase Shift
Increase accuracy to
sub-pixel by projecting
sine wave after code
and measuring phase
shift between
projected and
captured pattern.
Decode phase from
four samples of sine
period, shifted by pi/2.
code
Image column (x)
phase
+
2
0
Robot Vision SS 2013
Matthias Rüther
Image column (x)
13
Other Coding Methods Possible
Joaquim Salvi,
Pattern codification strategies in
structured light systems
Robot Vision SS 2013
Matthias Rüther
14
The Kinect Working Principle
 Triangulation based depth sensor
 Static pattern projection
 Heavy exploitation of redundancy
 Extremely robust/conservative depth maps
Robot Vision SS 2013
Matthias Rüther
15
The Sensor System
IR Lens:
F~6mm FOV~55°
Diffractive Optical
Element (DOE)
RGB Lens:
F~2.9mm, FOV~65°
Laser
830nm, 60mW
class 3B without optics, 1 with optics,
no amplitude modulation
RGB Camera:
CMOS, rolling shutter, 1.3MP, 1/4“, 10bit
Peltier Element
Temperature Stabilization
IR Bandpass
IR Camera:
CMOS, rolling shutter, 1.3MP, ½“, 10bit
Stereo Processor
Robot Vision SS 2013
Microphone Array
Accelerometer
Matthias Rüther
Tilt Axis
16
The Sensor System
 Tx ~75mm
 DOF 0.5m – 8m
 FOV ~55°
 Res. 640x480 (at most)
 Internal max 1280x1024
Robot Vision SS 2013
Matthias Rüther
17
The Projection Pattern
IR Laser and Diffractive Optical Element create interference
pattern
Pattern is static and identical for all Kinects
Robot Vision SS 2013
Matthias Rüther
18