Connect with your
Wow!
Motorized Tilt
3D Depth Sensors
RGB Camera
Microphone Array
www.openni.org
openkinect.org
kinectforwindows.org
Kinect for Windows
Xbox Kinect
That’s not USB
You’ll need this
What you Need
SDK
Demo 0
Hello Tilt
BGR32 Format
Byte Array:
width in pixels
Stride = Bytes Per Row
= bytes per pixel * width
= 4 * width
height in pixels
4 Bytes Per pixel
Demo 1
Hello Colorful World
Depth data
500mm 500mm 500mm 500mm
2500mm 500mm 500mm 500mm
2000mm 2030mm 2010mm 2000mm
Depth data
Max resolution for raw depth data is 640 x 480
2 bytes per pixel of depth data (short)
11011110 00001000 1110101000000000 11101010 00001000
pixel
pixel
pixel
depthInMM = pixel >> DepthImageFrame.PlayerIndexBitmaskWidth
Demo 2
Depth
Depth + Player Data
500mm 500mm 500mm 500mm
player 1 player 1 player 1 player 1
SkeletonStream.Enable();
2500mm 500mm 500mm 500mm
player 1 player 1 player 1
2000mm 2030mm 2010mm 2000mm
var player = pixel & DepthImageFrame.PlayerIndexBitmask;
11011110 00001010
Demo 3
Depth + Player Data
Skeletal Tracking
from skeleton in skeletonFrame.Skeletons
where skeleton.TrackingState == SkeletonTrackingState.PositionOnly
skeletonFrame.Skeletons
skeletonFrame.Skeletons
where skeleton.TrackingState == SkeletonTrackingState.Tracked
hand_left
hand_right
head
wrist_left
wrist_right
shoulder_center
elbow_left
elbow_right
shoulder_left shoulder_right
Skeleton.Joints[JointType]
spine
hip_left
Joint.Position (X,Y,Z)
Skeletons are mirrored
X, Y, Z in meters
Origin(0,0,0) = center of camera.
+Z is out
+X is to the right
+Y is up
knee_left
ankle_left
foot_left
hip_right
hip
center
knee_right
ankle_right
foot_right
Demo 3
Skeletal Tracking
Composites
Color
+
+
Composites
Color
Color
+
MapSkeletonPointToColor
+
MapDepthToColorImagePoint
MapDepthFrameToColorFrame
+
MapDepthToSkeletonPoint
MapSkeletonPointToDepth
Demo 4
Green Screen
Kinect Audio
Echo Cancellation
Echo Cancellation
EchoCancellationMode: None, CancellationOnly, CancellationAndSuppression
int EchoCancellationSpeakerIndex: Index of speaker to cancel
Noise Suppression
Noise Supression
bool NoiseSuppression – true, false
Automatic Gain Control
Automatic Gain Control
bool AutomaticGainControl – true, false
Beam Forming
Beam Forming
0.875 radians
50 degrees
-0.875 radians
-50 degrees
.0175 radians
10 degrees
Demo 5
Speech Recognition
Audio @!%!#!! Gotchas
Must run in an MTA
AEC SystemModes crash if no speaker output.
Use Microsoft.Speech not System.Speech
DevicePairGuid / Qualitymetrics = crash
Should run in ThreadMode Priority Highest?
Must plug Kinect into Wall
Demo 6
Beam Forming
Kinect SDK 1.5!
Seated Skeletal Tracking
Face Tracking
Kinect Studio
Joint Rotation
Creative Commons Acknowledgments
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Flickr by Ian Lewis
Flickr by major nelson
Flickr by markodesign
Flickr by stephantom
Flickr by Abstract Machine
Flickr By Aixo.co.kr
Flickr By Andresmh
Flickr by Antony Bennison
Flickr by Ars Electronica
Flickr By bm.iphone
Flickr by campuspartymexico
Flickr by Chris Jagers
Flickr By ci_polla Paola Sucato
Flickr by creepyhalloweenimages
Flickr by cvander
Flickr by IMAL.org
Flickr by Institut Douglas
Flickr by Joshua and AmberFlickr by Logon Sakai
Flickr by openexhibits
Flickr by Paul_Clarke
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Flickr by petyr.rahl
Flickr by pt
Flickr by Stilldavid
Flickr by y3rdua
Flickr by yesyesnono
Flickr by yum9me
Flickr by Jeff Howard
Flickr by jhscreenwm
Flickr by Kyle McDonald
Flickr by Morten Rand-Hendriksen
Flickr by Stuart Herbert
Flickr by Visiophone
Flickr by orkomedix
Michael Zöllner and Stephan Huber at the University of Konstanz
Openclip by J_Alves
Openclip by jetxee
Openclip by molumen
Openclip by shokunin
Openclip by Jhnri
Openclip by Machovka
Adafruit image of Hector Martin
Thanks!!