Real-time Dense Visual Odometry
for Quadrocopters
Christian Kerl
11.05.2012
1
Outline
•
•
•
•
•
Motivation
Hardware & Software
Approach
Problems
Ideas
11.05.2012
2
Motivation
• Quadrocopters need sensors to fly in unknown
environments
– Motion
– Position
– Obstacles
• Restricted on-board sensors
– IMU
– Visual navigation (no GPS)
• Restricted computing resources
 Autonomous system
11.05.2012
3
Motivation
• Standard approach to visual odometry:
– Sparse feature tracking in intensity / color images
– Examples: Jakob, ETH Zurich, TU Graz, MIT
– On-board frame rates 10 Hz
• Our approach:
– Using full RGB-D image information
– No feature tracking
11.05.2012
4
Hardware – Asctec Pelican
11.05.2012
5
Hardware – Asctec Pelican
IMU
AutoPilot
Board
Atom
Board
11.05.2012
6
Hardware – Asctec Pelican
• IMU
– 3 axis magnetometer, gyroscope, accelerometer
• AutoPilot Board
– Highlevel + Lowlevel Processor (ARM)
• Atom Board
–
–
–
–
Intel Atom Z530 1.6 GHz
1 GB RAM
7 Mini-USB Ports
WirelessLAN
• 600 g payload
11.05.2012
7
Software – Asctec Pelican
•
•
•
•
ROS drivers for Asctec Pelican from ETH Zurich
Nonlinear dynamic inversion for position control
Luenberger Observer for data fusion
Updated version using Extended Kalman Filter to
be presented on ICRA 2012
• Needs absolute position input from external
source
• Allows to command accelerations, velocities or
positions
11.05.2012
8
Hardware – Asus Xtion Pro Live
•
•
•
•
24 bit RGB image
16 bit depth image
640x480 @ 30 Hz
150 g
+
+
-
On-camera RGB and depth image registration
Time synchronized depth and RGB image
Rolling shutter
Auto exposure
11.05.2012
9
Approach
• Estimate transformation minimizing squared
intensity error (energy minimization)
11.05.2012
10
Approach
• Linearization
• with
• minimize
=> solve normal equations
11.05.2012
11
Analysis
• Estimate transformation minimizing squared
intensity error (energy minimization)
11.05.2012
X translation
Y translation
12
Image Data from Quadrocopter
• Hovering
11.05.2012
13
Image Data from Quadrocopter
• Trajectory along camera z-axis
11.05.2012
14
Problems
• Motion blur
• Auto exposure
• Dynamic objects (humans)
11.05.2012
15
Problems – Motion Blur
11.05.2012
16
Problems – Motion Blur
11.05.2012
17
Problems – Motion Blur
11.05.2012
18
Problems – Motion Blur
11.05.2012
19
Problems – Motion Blur
11.05.2012
20
Problems – Motion Blur
11.05.2012
21
Problems – Motion Blur
11.05.2012
22
Problems – Auto Exposure
11.05.2012
23
Problems – Auto Exposure
11.05.2012
24
Problems – Dynamic Objects
11.05.2012
25
Ideas
• Weighted Least Squares
• Initial motion estimate between 2 consecutive
frames from IMU data fusion
• Multiple iterations per level, convergence
checks
• Regularization term to minimize / constrain
least squares solution
• Minimization of intensity and depth error
11.05.2012
26
Ideas – Weighted Least Squares
• Assign smaller weight to residual outliers
=>
• Weight calculation
11.05.2012
27
Ideas – Weighted Least Squares
• Influence function
– Tukey weight
– Huber weight
11.05.2012
28
Ideas – Weighted Least Squares
• Weighted error
11.05.2012
29
Ideas – Weighted Least Squares
• Influence on energy function
X translation w/o weights
11.05.2012
X translation w/ Huber weights
30
Ideas – Weighted Least Squares
• Influence on energy function
Y translation w/o weights
11.05.2012
Y translation w/ Huber weights
31
Ideas – Weighted Least Squares
11.05.2012
32
Ideas – Weighted Least Squares
• Robustification with respect to dynamic
objects
• Slightly degrades tracking performance
• How to choose parameter b?
11.05.2012
33
Ideas – Initialization from IMU
• Use transformation from IMU data fusion as
initial estimate
11.05.2012
34
Ideas – Initialization from IMU
• Use transformation from IMU data fusion as
initial estimate
11.05.2012
35
Ideas – Initialization from IMU
• Use transformation from IMU data fusion as
initial estimate
11.05.2012
36
Ideas – Multiple Iterations
• Perform multiple optimization steps per image
pyramid level
• Stop when increment below threshold
• Bad frames / diverging
results can be recognized
and skipped
11.05.2012
37
Summary/Discussion
• Weighted Least Squares needs more work
(especially weight calculation)
• Initialization from IMU promising
• Multiple Iterations for increased accuracy and
divergence detection promising, but
computationally expensive
• Jumps in trajectory are really problematic!
=> Ideas welcome!
11.05.2012
38