KINECT REHABILITATION
Stroke Therapy Research
Kathryn LaBelle
RESEARCH TOPIC
Can the Kinect’s joint-tracking capability
be used in clinical and in-home stroke
rehabilitation tools?
OUTLINE
• Background
– Stroke Therapy
– Kinect
•
•
•
•
•
•
Potential of Kinect in Rehabilitation
Research Questions
Software
Data Gathering
Data Analysis
Conclusions
STROKE THERAPY
• Stroke survivors can experience:
– restricted movement
– loss of sense of balance
– decreased strength
• Regained through physical therapy
– balance exercises
– range of motion activities
– coordination practice
MICROSOFT KINECT
• Developed for the Xbox 360 gaming console
• Tracks your movements: you are the controller
• Sensors
1.
2.
3.
4.
Depth Camera and Sensors
RGB Camera
Microphone array
Motorized base
DEPTH IMAGING
• Infra-red projector shines grid
of light on the scene, encoded
with data.
• Light bounces off objects in the
scene.
• Kinect light sensors receive
reflected light.
• By analyzing time of flight and distoritions in
the encoded data, the Kinect makes a depth
map of the scene.
JOINT TRACKING ALGORITHM
• Input: depth map
• Machine learning algorithm
– Collected recordings of
people
using the Kinect
– Joint positions marked by
hand
– Algorithm was fed this
“training” data and learned
how to correctly identify
joints from a depth image
• Output: x, y, z joint positions
JOINT TRACKING AND
STROKE REHABILITATION
• Clinical applications:
– assess patients’ performance
– track patients’ progress
– pinpoint areas for improvement
• At-home exercise aids:
– provides constructive feedback to patients
– give encourgement and motivation
– generate summary reports for doctors
RESEARCH QUESTIONS
• What SDKs and drivers are available for use with a PC?
• What type of information can be obtained?
• What is the quality of the joint data obtained from the
Kinect?
– Sampling rates
– Consistency
• How resilient is the Kinect’s joint data and performance
to variation in testing conditions?
• What functionality could be provided in a stroke
therapy application that uses the Kinect?
SDK COMPARISON
OpenNI
Microsoft
Raw depth and image data
Yes
Yes
Joint position tracking
Yes
Yes
Save raw data stream to disk
Yes
No
Joint tracking without calibration
No
Yes
Easy installation
No
Yes
Number of joints available
15
20
Quality of documentation
Adequate
Excellent
SOFTWARE DEVELOPED
• Display depth video
and skeleton
• Joint positions and
instantaneous
frames per second
written to file
• Balance board
integration
• Record depth stream
to file
• Obtain joint
positions from
recording
DATA GATHERING
DATA ANALYSIS
• Sampling rates of joint position data
• Identifying phases of movement from joint
positions
• Consistency and stability of joint positions
SAMPLING RATE
OpenNI Microsoft
Average Frame Rate
(fps)
Std Deviation
(between trials)
Minimum
(fps)
Maximum
(fps)
25.0
19.6
5.8
2.3
9.8
14.1
30.0
23.7
IDENTIFYING PHASES OF MOVEMENT
DATA STABILITY
Standard Deviation of Joint Positions
while Subject is Motionless
Joint
Head
Hip
Knee
OpenNI (cm)
0.34
0.42
0.70
Microsoft
(cm)
1.8
1.2
1.5
DATA STABILITY: Assisted Tests
• Clinical therapy often involves an assistant
supporting a patient while he performs
exercises
• Test procedure:
– subject begins by sitting alone
– assistant joins, putting hands on subject’s
shoulders
– subject stands up
DATA STABILITY: Assisted Tests
DATA STABILITY: Assisted Tests
DATA STABILITY: Assisted Tests
CONCLUSIONS
• OpenNI Framework and Microsoft SDK for Windows are best tools
to use
• Can provide significant functionality in a joint-tracking application
– track and record joint positions in three dimensions
– display image of tracked joints in real time
– integrate Kinect with the Wii balance board
• Sampling rate exceeds acceptable level
• Phases of movement are easily identifiable from graphs of joint
positions
• Joint position stability is more than adequate with one subject in
view
• Skeleton merging could pose a problem for clinical use of Kinect
FUTURE WORK
• Deeper investigation into assisted exercises
– Different types of exercises
– Position the assistant differently
– Determine conditions causing skeleton merging
• Further development of software
• Investigate applications in other fields of
physical therapy
QUESTIONS?