Combining 3D Shape, Color, and Motion
for Robust Anytime Tracking
David Held, Jesse Levinson, Sebastian Thrun, and Silvio Savarese
Goal: Fast and Robust Velocity Estimation
Our Approach: Alignment Probability
●
●
●
P1
P2
P3
P4
Spatial Distance
Color Distance (if available)
Probability of Occlusion
Annealed Dynamic Histograms
Combining 3D Shape, Color, and Motion
for Robust Anytime Tracking
David Held, Jesse Levinson, Sebastian Thrun, and Silvio Savarese
Goal: Fast and Robust Velocity Estimation
Baseline: Centroid Kalman Filter
t+1
t
Annealed Dynamic Histograms
Baseline: ICP
Local Search
Poor Local Optimum!
Combining 3D Shape, Color, and Motion
for Robust Anytime Tracking
David Held, Jesse Levinson, Sebastian Thrun, and Silvio Savarese
Goal: Fast and Robust Velocity Estimation
Our Approach: Alignment Probability
●
●
●
P1
P2
P3
P4
Spatial Distance
Color Distance (if available)
Probability of Occlusion
Annealed Dynamic Histograms
Motivation
Quickly and robustly estimate the speed of nearby objects
Camera Images
Laser Data
System
Camera Images
System
Laser Data
Previous Work
(Teichman, et al)
Camera Images
System
Velocity
Estimation
Laser Data
This Work
Previous Work
(Teichman, et al)
Velocity Estimation
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
ICP Baseline
ICP Baseline
ICP Baseline
ICP Baseline
ICP Baseline
Local Search
Poor Local Optimum!
Tracking Probability
Velocity Estimation
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
Velocity Estimation
t+1
t
Xt
Velocity Estimation
t+1
t
Xt
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Constant velocity
Kalman filter
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
Motion Model
Tracking Probability
Measurement
Model
k
Motion Model
Tracking Probability
Measurement
Model
Motion Model
k
Sensor
noise
Sensor
resolution
Probability
Color Probability
Delta Color Value
Including Color
Including Color
Including Color
Including Color
Including Color
Probability
Including Color
Delta Color Value
Probability
Including Color
Delta Color Value
Probability
Including Color
Delta Color Value
Probabilistic Framework
3D Shape
Color
Tracking
Motion History
Tracking Probability
P1
P3
P2
P4
Tracking Probability
vy
?
?
?
?
?
vx
Tracking Probability
vy
vx
Dynamic Decomposition
vy
vx
Dynamic Decomposition
vy
vx
Dynamic Decomposition
vy
vx
Dynamic Decomposition
vy
vx
Derived from minimizing KL-divergence between
approximate distribution and true posterior
Annealing
Inflate the
measurement model
Annealing
Inflate the
measurement model
Annealing
Inflate the
measurement model
Algorithm
1. For each hypothesis
A. Compute the probability of the alignment
Measurement
Model
Motion Model
Algorithm
1. For each hypothesis
A. Compute the probability of the alignment
Measurement
Model
B. Finely sample high probability regions
Motion Model
Algorithm
1. For each hypothesis
A. Compute the probability of the alignment
Measurement
Model
Motion Model
B. Finely sample high probability regions
C. Go to step 1 to compute the probability of new hypotheses
Annealing
More time
More accurate
Anytime Tracker
Anytime Tracker
Choose runtime based on:
Total runtime requirements
Importance of tracked object
...
Comparisons
Comparisons
Comparisons
Kalman
Filter
Kalman
Filter
ADH Tracker
(Ours)
Models
Quantitative Evaluation 2
Sampling Strategies
0.8
RMS error (m/s)
0.7
0.6
0.5
0.4
0.3
ADH Tracker (Ours)
Dense sampling
Dense sampling with motion prediction
Top cell sampling
0.2
0.1
0
0
5
10
15
Mean runtime (ms)
20
25
Advantages over Radar
Conclusions
3D Shape
Color
Motion History
Tracking
● Robust to Occlusions, Viewpoint Changes
Conclusions
3D Shape
Color
Motion History
Tracking
● Robust to Occlusions, Viewpoint Changes
● Runs in Real-time
● Robust to Initialization Errors
Probability
Color Probability
Delta Color Value
Error vs Number of Points
Error vs Distance
Error vs Number of Frames
Error vs Number of Frames