EEC 490 Spring 2012
Kinect Fitness Trainer
-Baljeet Aulakh
-Arnold Csok
-Jared Shepherd
-Amandeep Singh
1
Project Overview
Recognition of Body Movements and
Exercises
Everything is Voice Controlled
Ability to Calculate Velocity And
Acceleration
 Write Statistics of Movements to an
Output File
 User Interface Implementation
2
Requirements
 Hardware: Kinect + Laptop(2.66Hz
above)
 Software: Kinect SDK + Microsoft
Speech Platform + C# 2010
 Environment: Quite Room(voice) +
Only one thing moving at a time
3
User Interface
4
Step by Step
 Voice Recognition
 Detection Algorithms
 Velocity and Acceleration
 Ability to Write Statistics to an
Output file
 Documentation and Website
5
Voice Recognition
Allow User To Control Kinect By Voice
Ease of Access to the Program
As Simple as- Start && Stop
Say Which Exercise to Count
6
How Voice Recognition Work
Initialize the Audio Source from the
Sensors
Initialize Speech Recognition by
Speech Recognizer
Create a Speech Recognition Engine
with Exercise Names
Listen to User Speech
Respond to User Speech
7
Problem With Voice Recognition
The First Release of Language Pack
Doesn't Have a Reliable Confidence
Model
Kinect Tries To Match Every Audio
Source It Picks Up
Problem with Matching the Right
Exercise Because of This
8
Possible Solutions For Voice
Recognition
Test Confidence Interval for Best
Accuracy
Use Fitness Trainer in a Quiet
Environment
Introduce Noise Cancelation
Use of Headset/Bluetooth
Wait for a More Reliable Language
Pack
9
Detection Algorithms
 Resources
 Recognition algorithms
 List of Recognized Body Movements
 An Example
 Velocity and Acceleration
10
20 Joint Skeleton System
Provided By Microsoft Research Kinect
11
Resources
20 Joint Skeleton System
Each joint gives x, y and z values
Vector Math to find the angle
Timer functionality
Voice recognition functionality
Flag Variables in programming.
Counters to store the repetition.
12
Recognition algorithms
 Simple cases:
• Displacement of the Joints
• Displacement in the X, Y, Z Direction
 Moderate cases:
• Calculating Angles Between Joints
• Setting Threshold For Some Angles
 Complex cases:
• Set a Step by Step Routine using all of the
above to detect a motion
13
Recognized Exercises
The Following Exercises Are Recognized:
1. Squats
2. Upper Left Punch
3. Upper Right Punch
4. Right Punch
5. Left Punch
6. Right Arm Curl
7. Left Arm Curl
14
Recognized Exercises: Continued
8.
9.
10.
11.
12.
13.
Left Kick
Right Kick
Bowling
Hip Abduction
Lateral Weight Shift
Hamstring Stretch
Counter For All Of These Exercises
15
An example: Bowling motion
16
Bowling final position
17
JOINT STATISTICS
Each exercise has the average
acceleration and velocity of all joints
calculated.
The statistics are time stamped and
saved into a text file.
AVERAGE VELOCITY
Two points
Distance between them
Time to travel between them
AVERAGE VELOCITY
Initial and final position of the joint
The distance formula
A Stopwatch
INITIAL AND FINAL POSITION
Save reference position(point) of skeleton for
use in all calculations in defaultPosition[20].
getDisplayPosition(data.Joints[JointID.HipCenter])
;
getDisplayPosition(data.Joints[JointID.Spine]);
The order is very important
DISTANCE FORMULA
Joints lie on a Cartesian plane
However there is a caveat; The
position is measured in pixels
Pixels Are Converted Into Centimeters
with the conversion factor of 72 DPI
x2 = (x2 * 2.54) / (72);
TEXT FILE
For every joint returned from the array
generated from GetValues
foreach (JointID joint in
Enum.GetValues(typeof(JointID)))
JointID Joint = (JointID)i;
When int is typecasted to JointID it returns the
name of the joint at that position, NOT A
STRING
Display / Interface
The display on the bottom of the
screen shows what exercise is being
done by the user
The state: Start / Stop
Exercise counter
Exercise to be detected
Exercise that was detected
24
Known Limitations
1. Blocked by an object
2. Overlapping joints
3. Distance from the Kinect Throws Off
X, Y, Z coordinates
4. Two People In Front Of Kinect
5. Joints Move During Angle Calculation
 These Limitations Make The Accuracy
of the Exercise Recognition Difficult
 E.g. Crossed arms...
25
Future Approaches
Fix Voice Recognition problem
Have a new accurate Skeleton System
Be able to record and replay exercises
More complex body motions
Introduce more exercises
Find solutions to the Limitations for
example a solution to the overlapping
joints and blocking objects.
26
Conclusion
 Learned about Kinect programming
 Use with windows
 Use of Kinect in different areas other
than gaming
 Learned and getting used to C#
27
Website And Facebook Page
Link To Our Webpage:
“http://www.baljeetaulakh.com”
Link To Our Facebook Page:
“https://www.facebook.com/pages/KinectFitness-Trainer/236918233072748”
28
Questions?????
OR
Suggestions?
29
End of Slides
30
Fun Time!!!!!
Demonstration!!!!!
31