CSCI498B/598B
Human-Centered Robotics
September 05, 2014
A Quick Summary
• Definition of human-centered robotics (HCR)
• Applications and challenges of HCR
• Comparisons of HCR with conventional robotics
• Sensors in Robotics
• 3D Machine Vision
• Kinect
2
What to perceive?
• Humans
3
What to perceive?
• Human activities
4
What to perceive?
• Group behaviors
5
What to perceive?
• Human-object interaction
6
What to perceive?
• Objects
7
What to perceive?
• Scenes
8
General Pipeline
1. Data acquisition (e.g., from 3D sensors)
2. Feature extraction and representation
3. Classification/clustering
4. Decision making
5. Taking an action
9
Data Acquisition from Kinect
1. Color-depth images (i.e., RGBD
images)
2. 3D point clouds
10
Data Acquisition from Kinect
3D point clouds and color-depth images are
not equivalent!
• 3D point clouds contains more information then
RGBD images or color-depth images
11
Feature Extraction
• Definition: feature extraction is the process of defining a
set of features or attributes, which will most efficiently or
meaningfully represent the information that is important for
classification, clustering, or other analysis
• Goal: to improve the effectiveness and efficiency of analysis
12
Feature Extraction
• Definition: feature extraction is the process of defining a
set of features or attributes, which will most efficiently or
meaningfully represent the information that is important for
classification, clustering, or other analysis
• eliminating redundancy in the data
• eliminating variability that is of little, or of no value
• restructuring data
• Goal: To form a representation that
• maximize pattern discrimination (effectiveness)
• minimize the number of the features (efficiency)
13
Feature Extraction
• Examples: skeleton of 3D models
14
Feature Extraction
• Examples: surface norm from 3D point clouds
15
Feature Extraction
• Examples: local features from interest points
16
Skeleton features from Kinect
17
Skeleton features from Kinect
• Each joint has 3 values (joint world coordinates in meters)
• Totally 60 elements in a feature vector
• Considering orientations and other elements, we have a large feature
with a lot of noise or irrelevant information
18
Skeleton-based features
• Solution 1: Only use more descriptive joints
19
Skeleton-based features
• Solution 1: Only use more descriptive joints
20
Skeleton-based features
• Solution 2: Compute additional features
𝒅
𝜽
21
3D mapping from 2D Lasers
http://www.youtube.com/watch?v=IMSozUpFFkU
22