Understanding Smart Sensors
Sensor Fusion - Using the Newest
Techniques for Advanced Sensors
October 12, 2012
Randy Frank
Agenda
• Two Fusion Paths
• Sensor Fusion Models
• Applications
– Fusing Radar & Camera Data
– Motion Fusion
• Sensor Fusion Products
• Sensor Fusion Tools/ Dev Kits
Sensor Fusion
Decision making capability
Sensor (data) fusion
signal processing
neural networks
fuzzy logic
multiplexing
multiple types of
sensors in same
package
multiple measurements of
same type of sensor, i.e. 3axis accelerometer
single parameter
sensor
Sensor output
Direct sensor
fusion
Dasarathy Model
Courtesy Belur Dasarathy
Sensor Fusion Confusion
Fusing Radar and Camera Inputs
Sensor Fusion in Vehicles
Source: Delphi
Sensor Fusion for Virtual Sensing
• NIRA Dynamics
Autonomous Driving
DARPA Challenge
• Tartan Racing,
Carnegie Mellon
University
Sensor Fusion for Motion
Source:
STMicroelectronics
Sensor Fusion for Motion
Pressure
Trim
hi/low/band
pass filtering
Pressure
Shake detection
shake event
3-Axis Acc
FoR
mapping
Trim
hi/low/band
pass filtering
3-Axis Gyro
FoR
mapping
Trim
hi/low/band
pass filtering
ω x,y,z
3-Axis Mag
FoR
mapping
Trim & Hard/Soft
compensation
hi/low/band
pass filtering
B x,y,z
Raw data
Acc x,y,z
calibration
parameters
Calculate hard/soft
iron parameters
FoR = Frame of Reference
Rotation matrix
Kalman
Filter or
similar
function
Quaternion
Geometric
computations
Tilt-compensated mag heading
Inclination (φ, Θ, Ψ)
Sensor Fusion
MANY styles of sensor fusion are possible.
Source: Freescale
Medical Example
Source: Freescale
Motion Capture Systems
• MVN inertial motion capture suit
from Xsens
• System’s 17 inertial trackers and the
sensor fusion software allow motion
capture without requiring cameras
• Used in animation as well as medical
and sports applications
Sensor Fusion Products
• Many sources for motion control
– Analog Devices
– Bosch
– Freescale
– InvenSense
– Kionix
– MEMSIC
– STMicroelectronics
10-DoF MEMS IMU
• Analog Devices ADIS16480 integrates:
– tri-axis gyroscope
– tri-axis accelerometer
– tri-axis magnetometer
– pressure sensor
– ADSP-BF512 Blackfin® processor
• Incorporates an extended Kalman filter (EKF)
to fuse sensor inputs
10-DoF MEMS IMU
• EKF takes multiple measurements over time,
and merging them with a predictive state
estimator.
• Intensive code development, testing and
external processing required by other MEMS
IMUs.
• Targets: military and commercial aircraft
navigation, unmanned vehicles, movable
platform positioning, and industrial robotics.
• Evaluation board
Motion Tracking Device
InvenSense MPU-6500
• Turnkey 6-axis MotionTracking
– MEMS gyroscope and accelerometer
– Onboard Digital Motion Processor™ (DMP)
– MotionFusion algorithm
– 3x3x0.9mm QFN package
Motion Tracking Device
InvenSense MPU-6500
• Applications such as pedestrian navigation,
context-aware advertising, and other locationbased services
• Wearable sensor applications such as remote
health monitoring, sports and fitness tracking,
Architecting Fusion
Source: Movea
Design Options
• Software suite operates either as a hardware
solution on an embedded MCU or as a
software solution on an AP
Software Solution
Sensor Platforms’ FreeMotion™ Library
• Supported microprocessors include:
• 32-bit embedded processors (ARM’s CortexM, Atmel’s AVR and Freescale’s ColdFire
families used as sensor hubs)
• 64-bit application processors (Intel’s Atom,
nVidia’s Tegra, Qualcomm’s Snapdragon, and
TI’s OMAP processors used in smartphones
and tablets)
Sensor Fusion Tools
12-Axis Sensor Platform
• Complete hardware and software solution
Other Development Kits
• Sensor Platforms FreeMotion™ Library and
Software Development Kit
• SmartFusion Development – Actel
• MTi Development Kit – Xsens
• ADIS16480/PCBZ breakout board – Analog
Devices
Summary
Sensor Fusion
•
•
•
•
Algorithms for increased sensor performance
Many different approaches
Motion sensing is a highly pursued area
Many companies offer products and tools