A Novel Tracking System for AAL
Based on Smartphone Technology
DIEEI – University of Catania, Italy
Outline
• AAL research activities at DIEEI
• The proposed methodology for user tracking
• The proposed methodology for the Monitoring of ADL & Fall events
• Experimental results
What is AAL?
Ambient Assisted Living: smart use of technology to support well-being in the preferred living
environment for people who might otherwise find this difficult (e.g. infirm or elderly people who want
to continue living in their own homes)
Quality of life
Social interaction
Active aging
Well-being
Safety
Synergy between competencies to improve life quality of weak (impaired and elderly ) people
Why research on AAL & Assistive technologies?
Social problem
Disabled people represent 50 million persons in the EU
(about 10% of the entire population)
According to ISTAT (2004-2005), 4.8% of the Italian
population (2,800,000 persons) suffer from some kind
of disability, the 75% of which are over-65 years old.
New innovative solutions can have a major effect on
quality of life, thus easing the pressure of increasing
costs in European social and care systems
Attractive potential market
Fundings
AAL Joint Programme
The European Commission has formally proposed a
continuation of the AAL Joint Programme for the period
2014 – 2020.
HORIZON 2020
Accessible and Assistive ICT,
Embedded Accessibility of Future ICT.
AAL@DIEEI-UNICT
Associazione Italiana
“Ambient Assisted Living”
DIEEI is a supporter of the
AALOpenAssociation
focusing on scientific and technological
challenges posed by AAL.
Monitoring of a tactile path
Image enhancement
by noise modulation
Multisensor Haptic cane
Sensors
Methodologies
Color sensor based on fuzzy logic
Tactile perception
The proposed user tracking methodology
Objective: to develop an User-Centered Designed (UCD) Assistive platform for mobility
and assistance of weak users
Approach: sensing and processing features of SmartPhones
Methodology: multi-sensor data fusion and advanced signal processing
Features: user-friendly device, high reliable algorithms for data mining
Smartphone
Server (PC)
Signal Processing
Smartphone:
Accelerometer
Compass
Gyroscope
Android
Communication App
User-Environment
Contextualization/
Interaction
Labview Development
Environment
Feedback
ADL
User tracking:
+
+ • Walking Distance Estimation (Step-Size)
Fall detection
• Heading
Environment info
+
User-Environment
Interaction
(UEI)
The proposed user tracking methodology
Objective: to develop an User-Centered Designed (UCD) Assistive platform for mobility
and assistance of weak users
Approach: sensing and processing features of SmartPhones
Methodology: multi-sensor data fusion and advanced signal processing
Features:
high reliable
algorithms
dataand
mining
UCD is user-friendly
a type of designdevice,
and a process
in which
the needs,for
wants,
limitations of end
users of a product are given extensive attention at each stage of the design process.
Smartphone
Server (PC)
Signal Processing
Smartphone:
Accelerometer
Compass
Gyroscope
Android
Communication App
User-Environment
Contextualization/
Interaction
Labview Development
Environment
Feedback
ADL
User tracking:
+
+ • Walking Distance Estimation (Step-Size)
Fall detection
• Heading
Environment info
+
User-Environment
Interaction
(UEI)
The proposed user tracking methodology
Objective: to develop an User-Centered Designed (UCD) Assistive platform for
mobility and assistance of weak users
Approach: sensing and processing features of SmartPhones
Methodology: multi-sensor
data fusion
andOS
advanced signal processing
App developed
for Android
Features: user-friendly
device,
high
reliable algorithms
- Open
Source
Development
Environmentfor data mining
Smartphone:
Accelerometer
Compass
Smartphone:
Gyroscope
Accelerometer
Compass
Gyroscope
-
Access to all sensor embedded into the
Server (PC)
smartphone
Processing
Wireless communicationSignal
(WiFi,
Bluetooth)
Server
(PC)
Smartphone
Smartphone
Android
Communication App
Signal Processing
App features:
User-Environment
- Read inertial and orientation sensors available
Contextualization/
on the smartphone
User-Environment
Interaction
- Refresh Rate fully configurable
(used 50Hz)
Contextualization/
Feedback
- Sensors Data are transferred to
PC through
Interaction
TCP/IP protocol
Feedback
ADL
User tracking:
+
+ • Walking Distance Estimation (Step-Size)
Fall detection
• Heading
Android
Communication App
Labview Development
Environment
Labview Development
Environment
Environment info
+
User-Environment
Interaction
(UEI)
The proposed user tracking methodology
Objective: to develop an Activities
User-Centered
Designed
(UCD) Assistive platform for
of Daily
Living
mobility and assistance of weak users
Basic routine activities that peopleFall
tenddetection
do everyday without needing assistance.
Approach: sensing and processing features
of SmartPhones
Methodology:
There are sixmulti-sensor
basic ADLs: data fusion and advanced signal processing
The fall in the elderly is a major public health problem as it causes many disabling
• eating
Features:
user-friendly device, high reliable algorithms for data mining
fractures but also has dramatic psychological consequences which reduce the
• bathing
independence of the person.
• dressing
Smartphone
•
toileting
Server
(PC)
•
Step
up
Smartphone:
It was established that the earlier the fall is reported, the lower is the rate of
Signal Processing
• transferring (walking)
Accelerometer
• Step down
morbidity-mortality.
Compass
Android
• continence
• Sitting
Gyroscope
Communication App
User-Environment
An individual's ability to perform ADLs is important
for determining what type of
Contextualization/
Labview Development
long-term care (e.g. nursing-home care or home
care) and coverage the
individual
Environment
Interaction
needs (i.e. Medicare, Medicaid or long-term care insurance).
Feedback
ADL
User tracking:
+
+ • Walking Distance Estimation (Step-Size)
Fall detection
• Heading
Environment info
+
User-Environment
Interaction
(UEI)
Dead Reckoning Techniques
Walking Path Reconstruction
Reconstruction of the single step is performed by the combination of the Heading θi
and the Step Size ∆Si .
The absolute position is obtained by cumulating step by step the relative positions.
N Y’ = ΔS *cos(θ)
X’ = ΔS *sinθ)
y
ΔS2
Y’
ΔS1
∆Si
θi
X’
x
E
• The position error increases step by step.
• It needs periodic correction through absolute
measurement of position.
• Useful to evaluate short distance path
• Useful to estimate physical capability of the user
ΔS3
ΔS4
Schematization of the proposed approach
Methodology: multi-sensor data fusion and advanced signal processing
Data Fusion
Accelerometer
Compensation
Compass
Θ Compass
Θ
CKF
Gyroscope
ʃ
Compensation Θ Gyro
θ
XY
||
Acceleration
Step Size
(SS)
&
Walking
Distance
(WD)
coordinates
Dynamic SS
DFA
Static SS
SS
Δs
Accelerometer
An accelerometer is a device that measures static and dynamic accelerations.
What are accelerometers useful for?
- By sensing the amount of dynamic acceleration, you can analyze the way the device (and
therefore the body to which it is attached) is moving.
- By measuring the amount of static acceleration due to gravity, you can find out the angle
the device is tilted at with respect to the earth.
Accelerometer
• Modeling
• Readout strategy
• Technology
• Output interface
• Analog
• Digital (I2C, SPI)
I2C & SPI
Serial Peripheral Interface
Is a synchronous serial data standard, named by Motorola, that operates in full duplex mode.
SPI is often referred to as SSI (Synchronous Serial Interface).
Devices communicate in master/slave mode where the master device initiates the data frame.
Multiple slave devices are allowed with individual slave select lines.
SCLK: serial clock (output from master);
MOSI: master output, slave input (output from master);
MISO: master input, slave output (output from slave);
SS: slave select (active low, output from master).
Alternative naming conventions are also widely used
Frequencies: 40 MHz to 100 MHz
Inter-Integrated Circuit
Is a multimaster serial single-ended computer bus invented by Philips used for attaching low-speed
peripherals to a motherboard, embedded system, cellphone, or other electronic device.
I²C uses only two bidirectional lines:
• Serial Data Line (SDA)
• Serial Clock (SCL)
I²C defines basic types of messages, each of which begins with a START and ends with a STOP
Frequencies: 10 kbit/s (LSm) to 5 Mbit/s (UFm)
Magnetic Compass
The magnetic compass consists of a magnetized pointer free to align itself with Earth's magnetic field. A
compass is a magnetically sensitive device capable of indicating the direction of the magnetic North
The strength of the earth's magnetic field is about 0.5 to 0.6 gauss and has a component parallel to the
earth's surface that always points toward the magnetic north pole
It is an absolute heading sensor.
It is affected by environmental magnetic fields and ferromagnetic
bodies which alter the Earth's magnetic field.
The tilt problem
Problem: The reading will be more inaccurate
the further the compass is tilted. This problem
occurs because the compass is only using the X
and Y axis of the earths magnetic field.
When the compass is not parallel to these axis
the amount of magnetism felt by the needle will
change based on how out of alignment the
compass is to these axis.
Xh = XM * cos(pitch) + ZM * sin(pitch)
yh = XM*sin(roll)*sin(pitch) + YM*cos(roll) - ZM*sin(roll)*cos(pitch)
heading = arctan(yh/xh)
Gyroscope
A gyroscope is a device for measuring or maintaining orientation, based on the principles of angular
momentum.
Mechanically, a gyroscope is a spinning wheel or disc in which the axle is free to assume any orientation.
Gyroscopes measure angular velocity, how fast something is spinning about an axis.
It is not affected by gravity and magnetic fields.
Step Size Estimation
Az
Accelerometer
Step
detection
Dynamic SS
DFA
Ay , Az
||
Step Size
estimation
SS
Static SS
Methodology:
- Step Detection: Processing of the vertical axis Az of the accelerometer
- Step Size Estimation : Three types of algorithms have been used:
• Static: exploit a constant value of the Step Size, evaluated in a calibration phase in case
of a MEDIUM step size and a NORMAL walking speed.
• Dynamic: computes an adaptive estimation of the Step Size, using the frequency fi and
the variance νi of the forward acceleration:
N
Dss =
∑α fi + β ν i + γ
step =1
α, β evaluated during the calibration phase (10 m
for 3 times with 3 walking speed)
• Adaptive : a DF algorithm provides a combination of both the static and dynamic
estimation depending on the acceleration variance.
Step recognition
Smartphone is placed on the right hip
• Two different heuristic thresholds to
recognize the step:
about 9 and 11 m/s2
• Two set of values (right/left step) of
α, β and γ parameters for the three
walking speed: low , normal and
fast .
Right Step
Left Step
Static and Dynamic Algorithms Test Results
Three Test Speed Mode: SLOW, NORMAL, FAST
Test Pattern Walked using : SMALL step sizes, MEDIUM step sizes.
Static and Dynamic Algorithms Test Results
Three Test Speed Mode: SLOW, NORMAL, FAST
Test Pattern Walked using : SMALL step sizes, MEDIUM step sizes.
J wd % =
WD Nom − WD Estim
WD
LOW Speed
Jwd%
SMALL MEDIUM
Nom
⋅ 100
NORMAL Speed
FAST Speed
SMALL
MEDIUM
SMALL
MEDIUM
WDdyn
3,9
7,2
4,8
0,3
8,5
8,1
WDstat
40,6
3,8
25,8
3,8
40,6
3,8
Heading Determination
Accelerometer
Compass
Gyroscope
Compensation
Θ Compass
CKF
ʃ
Compensation
Θ
Θ Gyro
Methodology:
• Compass: the absolute Heading estimation is obtained by computing the data from the Tilt
Compensated 3-axis Compass (Roll and Pitch)
• Gyroscope : the relative Heading estimation is given by integration of the 3-axis Gyroscope
• Constrained Kalman Filter (CKF): Adaptive Heading estimation by processing the Compass and
Gyroscope by taking into account the variance of the two signals (noise).
The Kalman Filter
The KF addresses the general problem of trying to estimate the state x ϵ Rn of a discrete-time controlled
process governed by the linear stochastic difference equation
with a measurement :
wk and vk represent the process and measurement noise. They are assumed to be indipendent, white and
with normal probability distributions :
p(w) ~ N(0, Q)
p(v) ~ N(0, R)
Target: to find the estimate of the signal x
The Kalman Filter
The KF addresses the general problem of trying to estimate the state x ϵ Rn of a discrete-time controlled
process governed by the linear stochastic difference equation
with a measurement :
wk and vk represent the process and measurement noise. They are assumed to be indipendent, white and
with normal probability distributions :
p(w) ~ N(0, Q)
p(v) ~ N(0, R)
Target: to find the estimate of the signal x
Kalman filter finds the most optimum averaging
factor for each consequent state. Also somehow
remembers a little bit about the past states.
The KF Step-by-Step
Step 1 – Build a Model
A, B and H are in general form matrices, but in most cases can be considered just numeric values.
Moreover they can change between states but most of the time we can assume that they’re constant
Required: estimation of the mean and standard deviation of the noise functions wk-1 and vk.
Step 2 – Start the Process
The next step is to determine the necessary parameters and initial values.
Two distinct set of equations applied at each kth state :
Time Update (prediction)
Measurement Update (correction
To start the process, we need to know the estimate of x0, and P0.
The KF Step-by-Step
Step 3 – Iterate
The previous estimates will be the input for the current state.
prior estimate
Posterior estimate
prior error
covariance
Experimental results and test
•
Users in good health have been requested to perform five repetition of a 20 m long square walking path
•
•
Gyroscope heading information is affected by a drift while compass is influenced by magnetic noise.
The CKF estimation is strongly anchored to the compass heading estimation while rejecting magnetic
noise in the compass signal by following the gyroscope trend.
Experimental results and test
•
Users in good health have been requested to perform five repetition of a 20 m long square walking path
Jθ % =
θ Nom − θ Estim
θ
Nom
⋅ 100
COMPASS
GYRO
CKF
1679°
1501°
1704°
1.8
12.2
0.3
Total Measured Heading
(target 1710°)
Jθ%
•
•
Euclidean Distance (m)
3.04
9.91
0.8
Gyroscope heading information
is affected by a drift
while compass
is influenced
by magnetic noise.
The CKF estimation is strongly anchored to the compass heading estimation while rejecting magnetic
noise in the compass signal by following the gyroscope trend.
Gyroscope Heading - Dynamic StepSize
Kalman Heading - Dynamic StepSize
2
2
0
0
0
-2
-4
-6
-2
Y axis [m]
2
Y axis [m]
Y axis [m]
Compass Heading - Dynamic StepSize
4
-2
-4
2
X axis [m]
4
6
-8
-2
0
2
4
X axis [m]
6
WALKING DISTANCE (nominal 100m):
STATIC
DYNAMIC
Walking distance
101,38
99,60
-4
-6
-6
0
-2
8
-8
-4
-2
0
2
X axis [m]
4
TOTAL ROUNDS (NOMINAL 1710°):
ROUNDS
COMPASS
GYRO
KALMAN HEADING
1679°
1501°
1704°
6
Nominal walking distance = 357 m