Southern Taiwan University of
Science and Technology
Department of Electrical Engineering
An application Kinect camera controls
Vehicles by Gesture
Professor
Student’s name
Student ID
1
: Tsung Fu Chien
: Nguyen Trong Tuyen
: MA02B208
CONTENT
1. Introduction to Kinect
2. Some Libraries for Kinect
3. Implement
4. Conclusion & Future works
2
Introduction to Kinect
1. Kinect camera
Kinect is a motion sensing input device by Microsoft for the Xbox
360 video game console and Windows PCs. Based around a webcamstyle add-on peripheral for the Xbox 360 console, it enables users to
control and interact with Xbox 360 without the need to touch a game
controller, through a natural user interface using gestures and spoken
Kinect included of:
* A camera RGB with sensitive 640x480
* 3D depth sensor
* Microphone
* A motor to control kinect up and down
3
Introduction to Kinect
Figure 1.1 Kinect camera
4
Introduction to Kinect
2. Calculating the depth on Kinect
On Kinect, we have IR projector combined IR camera to calculate
value image depth by Light Coding technology of Primesense.
5
Introduction to Kinect
6
Introduction to Kinect
Suppose, Projector send a single light along the red line. It will be
captured by the camera once it hits a surface.
We have 3 plane, a reference plane, a plane close and a distant
plane.
When the point hits a surface in the close plane it will appear right
in the image than if it was hitting a surface in the reference plane.
Likewise, when the point is projected onto an object on a plane which
is more distant than the reference plane, the point will appear more to
the left.
When the origin and direction of the light is known beforehand,
and the horizontal position of the dot is known for a reference depth,
then it is possible to find out the depth of the surface which the point
hits based on its horizontal position in the cameras image plane
7
Introduction to Kinect
According to Nicolas Burrus, who pioneered with information
about the Kinect from his own experiments, the depth of a point z can
be calculated in meters from the raw disparity of the point d (as
provided by the Kinect hardware) using the following equation[9]:
z = 1.0/(-0.0030711016 x d + 3.3309495161)
d is an 11-bit integer which ranges from 0 to 2047. z will change
sign from positive to negative when d is around 1084, so values
beyond that are useless for depth measurement.
8
Introduction to Kinect
Implementation tests with the Kinect pointing straight at a wall
and found that it is was to unable to reliability measure depth values
below 50 cm (see gure 2). These facts mean that only d values of
about 434 to 1084 represent actual measurable depth values
9
Some Libraries for Kinect
1. Kinect SDK beta
SDK beta released by Microsoft on 16th June 2011, SDK
beta allow to access all function of Kinect but This library only
support on Visual studio 2010.
* Support:
- Process sound
- Control motor angle of repose on Kinect.
- Good for the applications Skeleton tracking.
10
Some Libraries for Kinect
2. Open NI
Open NI supports many API functions, can combine
middleware to increase ability for Kinect.
* Support:
- Detect gesture.
- Skeleton tracking
- Windows, Linux, Mac OSX
11
Some Libraries for Kinect
3. Point cloud library
Point cloud is library to process image in 3D space. Having many
algorithms such as: filtering, surface reconstruction, segmentation,
feature estimation… PCL can apply on many platforms such as:
Window, Mac OS, Linux, Android,…
12
Implement
1. System design
Figure 4.1 General design
13
Implement
1. System design
1
14
2
3
Figure 4.2 Detail design
1
MCU IDE
2
Kinect Driver
3
Kinect IDE
Implement
Start
Define pin
Set communication
Val 1, Val 2, Val 3
Val 1 = 1
Val 1 = 2
No
Yes
No
Yes
Right
Val 1 = 0
Yes
Left
No
Val 2 = 1
Yes
Up
15
Stop
No
Val 2 = 2
Yes
Down
Val 2 = 0
Yes
Disable
Conclusions
The Kinect has been one of the pioneer in the consumer
market of the human / virtual environment interaction without
using some actual buttons or controller, but only by using natural
gesture or speech. We have seen that this field has progressed a
lot since the Kinect has been released (a bit more than 1 year) and
find its application in many different domain whereas the Kinect
was orignally planed to be mostly used for games.
The Kinect has opened the doors of the Virtual and
Augmented Reality to everybody but it is not the perfect device,
it still needs a better reliability (better sensors, cameras and
microphones) also related components, or combined (screens,
robotics...) need to be improved to develop all the capacity of the
Kinect.
16
References
[1] Chumbley, Eric M., Francis G. O'Connor, and Robert P. Nirschl. "Evaluation of
Overuse Elbow Injuries - February 1, 2000 - American Academy of
Family Physicians." American Academy of Family Physicians. American Academy of
Family Physicians, 1 Feb. 2000. Web. 12
July 2011. <http://www.aafp.org/afp/200 <http://www.aafp.org/afp/20000201/691.html>.
[2] “Golf Swing Tips.” Golf Swing Tips.
Golfswingtips.4u.com, 2011. Web. 9 July
2011. <http://golfswingtips4u.com/>.
[3] “OpenNI Modules.” Downloads.
OpenNI TM, Nov. 2010. Web. June-July
2011. <http://www.openni.org/>.
[4] “Physical Therapy Charges.” Akron
General. Akron General Health System,
2011. Web. 14 July 2011.
<http://www.akrongeneral.org/>.
[5] “Shoulder Problems and Injuries – Topic
Overview.” WebMD. WebMD, LLC, 2011.
17
Thank you for your attention
18