Create Natural User Interfaces with the
Intel® RealSense™ SDK Beta 2014
Hon Pong (Gary) Ho
Intel Perceptual Computing
Contents:
• The Intel® RealSense™ SDK for Windows beta
•
What is Intel® RealSense™ Technology?
•
Why should developers care about Intel® RealSense™ Technology?
•
Disclosures of the technical details of Intel® RealSense™ Technology
• Intel® RealSense™ SDK Developer Programs
•
How We Help You to Help Us
• Next Steps: Getting Started with Intel® RealSense™ SDK for Windows beta
• Q&A
2
Interfaces for computing devices have evolved very little in 30 years
What comes next?
3
The Intel® RealSense™ SDK
Free Tools and APIs for building natural user interfaces. Public Beta for Windows available Q3 2014
 Accessible for beginners. Open and extensible for experts.
 Support for Windows 8.1 today. Android support coming.
 Reach a broad-base of customers by writing apps for the Intel RealSense 3D Camera.
 Stand out in a crowded market. Delight your customers.
 Works with languages, frameworks, and game engines you already use
4
The Intel® RealSense™ SDK understands 4 basic
types of input
Use standalone
in combination
Categories
of Input or
Capabilities
Hands
Hand and Finger
Tracking
Gesture Recognition
• 22-point Hand and Finger Tracking
• 9 static and dynamic mid-air gestures
•
Face Detection and
Tracking
•
•
•
•
•
•
Speech Recognition
• Command and Control
• Dictation
• Text to Speech
•
•
•
Segmentation
3D Scanning
Augmented Reality
•
•
•
•
•
Hands
Face
Speech
Environment
•
Face
Speech
Environment
Features
Multiple Face Detection and tracking
78-point Landmark Detection (facial features)
Emotion Recognition (7 emotions, coming post-Beta)
Pulse Estimation
Face Recognition (Coming post-beta)
Background Removal
3D Object / Face / Room Scanning (Coming post-beta)
2D/3D Object Tracking
Scene Perception (coming post-beta)
5
Intel® RealSense™ 3D Camera for Windows:
Designed for close-range interactions
120 cm
20 cm
Intel®RealSense™
3D camera
Competitive
technologies focus on
a living-room
experience or a subset of Intel RealSense
technology features
6
The Intel® RealSense™ 3D Camera
* Other names and brands may be claimed as the property of others
7
The Developer Kit
When we roll out the public beta SDK in Q3
of 2014, we’ll release a developer kit to help
you get started
 Kit includes

The Intel® RealSense™ SDK for Windows Beta

Peripheral depth camera

Full VGA depth resolution

1080p RGB camera

0.2 – 1.2 meter range

USB 3.0 interface

Requires 4th generation Intel® Core™ processor
(or later)
Available for purchase at intel.com/software/realsense
8
Windows today….Android on the way
Look for more details on Intel® RealSense™ SDK for Android later this year
* Other names and brands may be claimed as the property of others
9
What Kinds of Applications can Intel® RealSense™ Technology Enable?
Capture
and Share
Immersive
Collaboration/Creation
Interact
Naturally
Innovate on top of
our application
categories. YOU will
come up with the
next big thing
Gaming
and Play
Learning and
Edutainment
10
Immersive
Collaboration/
Creation
•
•
•
Via the 3D camera, software can
“segment” an environment,
allowing an app to remove the
background
With the background removed,
developers can create applications
ideal for a combination of video
chat and collaboration
Make apps that allow presenters
to be overlaid on top of
applications, allow two people to
share an experience on the web,
or for a single person to use this
“digital green screen” to create
compelling original content
11
Interact
Naturally
•
•
Use our extensive library of
gestures, facial analysis
algorithms, and speech
recognition algorithms to build
interfaces that are truly
natural
Leverage our carefully
developed user-experience
guidelines document to
ensure the interfaces you
build will delight your
customers
12
Gaming
and Play
•
•
Differentiate your game with
gesture, speech, augmented
reality enhanced with depth,
facial analysis, and more
Use the user’s pulse to
determine the stability of a
scope. Allow a user to
physically move their head to
peak around a corner. Use
gesture for a lock-picking
challenge. The possibilities
are endless
13
Capture
and Share
•
•
•
•
* Other names and brands may be claimed as the property of others
Partnership with
scanning/printing experts, 3D
Systems*
Rotate a small object in front
of the 3D camera to create a
3D mesh + color
Developers can import scans
from a proprietary 3DS* app
or eventually build scanning
into their own application
The model is scan, edit,
share. Upload scans to social
media, or sent to a 3D printer.
14
Learning and
Edutainment
•
•
•
Use augmented reality,
powered by Metaio to track
2D or 3D objects and
incorporate them into a
learning experience
Tracking is significantly
improved by the 3D camera.
Enjoy both enhanced
precision and accuracy
Interact with stories using
both facial/emotion
recognition and gesture.
Disguise learning as play.
15
SDK Supported Features
Required Hardware
A system with a minimum of a 4th generation Intel® Core™ processor
(codenamed Haswell), either IA-32 or Intel® 64, with integrated depth camera
Required OS
Microsoft* Windows 8.1 OS (32- and 64-bit)
Microsoft* Windows 8.1 Metro (coming soon)
Supported Programming
Languages
C++, C#, JavaScript, Java (JDK 1.7.0_11 or later for face, voice, and gesture)
Supported IDE
Microsoft* Visual Studio C++ 2010-2013 with service pack 1 or newer
Eclipse.org IDE
Supported Development
Tools
-Microsoft* .NET 4.0 Framework for C# development
-Unity* PRO 4.0.0 or later for Unity game development
-Processing* 2.0.7 or later for Processing framework development
-openFrameworks* v0071 or later for openFrameworks development
-Havok* Vision SDK 2012.2.1 or later for Havok Vision SDK development
-Intel® Media SDK 2014
-Intel® SDK for OpenCL Applications 2013
* Other names and brands may be claimed as the property of others
16
Additional SDK Features
Power Management State
Apps can manage battery life
Privacy Notification Tool
Notifies user when camera is turned on by an app
Input Device Manager
Multiple applications can access data from the Camera simultaneously
Multi-Mode Support
Support multiple usage modes within a single app (e.g. finger tracking + speech + face
tracking) or between apps
Extensible Framework
Developers can plug in their own algorithms. New usage modes can be added. New
devices can be supported.
17
Web / HTML5
Web Socket Communication
127.0.0.1:4181
SDK Local
Web-Socket Server
Intel® RealSense™
SDK runtime
Internet Browser
Unity Web Player App
HTML5 / JavaScript
Operating System
Trigger face and gesture events, just like in C# applications
Switch to the web player platform for deployment
 Gestures and Facial Analysis
 Unity* Web Player games
 HTML5 / JavaScript (interactive websites)
18
* Other names and brands may be claimed as the property of others
18
Unity* Productivity Extension
A set of scripts that provides
configurable actions/rules based on
the capabilities provided in the SDK.
•
Programming: Associate your
game objects with the action
scripts.
•
Programming: Create rules.
Associate your game objects with
rules.
Drag and drop programming
Write less code!
* Other names and brands may be claimed as the property of others
19
Enabling
the
broad
ecosystem
through
How We
Help
You
to Help Us
contests and investments
Online
• RealSense Resource Center (software.intel.com/RealSense)
• RealSense Community (Blogs & Forums)
Physical Events
• Hackathons (12+ worldwide planned for 2014)
• Tradeshow classes/labs
Promotions
• Intel® RealSense™ App Challenge 2014
• Share Your App Showcase
20
Enabling the broad ecosystem through
Online
Resources
contests and investments
21
Enabling the broad ecosystem through
Physical
Events
contests and investments
22
Enabling the broad ecosystem through
Promotions
contests and investments
23
“perceptual computing [now RealSense]…a phrase
that you’re about to hear a lot of…it’s the future of
computing and it’s freaking awesome.”
− Luke Hopewell, Gizmodo
http://www.gizmodo.com.au/2013/06/the-future-of-computing-is-perceptual/
24
Getting Started
• Sign up now to learn when the Intel® RealSense™ SDK and
development kit are available
• Provided by Intel





Free Intel® RealSense™ SDK for Windows Beta in Q3 2014
Intel® RealSense™ Developer Kit available for purchase
Huge opportunity to reach customers with integrated 3D camera
Works with languages/frameworks/engines you already use
High-level APIs for NUI beginners. Low-level APIs for NUI experts
Seize The Opportunity
software.intel.com/RealSense
All products, software, computer systems, dates and figures specified are preliminary based on current expectations, and are subject to change without notice.
25
Q&A
26