CSCI498B/598B
Human-Centered Robotics
September 28, 2015
A Quick Summery of
what we have learned
•Introduction to HCR
•3D robotic perception based on
• Stereo vision
• Structured light
•Robot Operating System
•A bit about robot learning
2
ROS: Robot Operating System
Tools: ROS provides an extensive set of tools for
configuring, starting, introspecting, debugging,
visualizing, logging, testing, and stopping distributed
computing systems.
3
ROS: Robot Operating System
Logging and Visualization Sensor Data: rosbag and rqt_bag
4
ROS: Robot Operating System
3D Visualization: RVIZ
5
ROS: Robot Operating System
2D and 3D Simulation: Player-Stage and Gazebo
6
ROS: Robot Operating System
2D and 3D Simulation: Player-Stage and Gazebo
7
ROS: Robot Operating System
2D and 3D Simulation: Player-Stage and Gazebo
8
ROS: Robot Operating System
Capabilities: ROS provides a broad collection of libraries
that implement useful robot functionality, with a focus
on mobility, manipulation, and perception.
9
Capabilities: Mobility
10
Capabilities: Manipulation
11
Capabilities: Mapping
12
ROS: Robot Operating System
13
Human Action Understanding
for Human-Robot Interaction
14
General Pipeline
1. Data acquisition
2. Representation construction
3. Pattern recognition (classification or clustering)
4. Decision making
5. Taking an action
15
General Pipeline
1. Data acquisition
2. Representation construction
3. Pattern recognition (classification or clustering)
4. Decision making
5. Taking an action
16
Skeletal data
17
Skeletal data from Kinect 1
18
Skeleton features from Kinect 1
• Each joint has 3 values (joint world coordinates in meters)
• Totally 60 elements in a feature vector
• Considering orientations and other elements, we have a large feature
with a lot of noise or irrelevant information
19
General Pipeline
1. Data acquisition
2. Representation construction
3. Pattern recognition (classification or clustering)
4. Decision making
5. Taking an action
20
Skeleton-based representations
• Solution 1: Only use more descriptive joints
21
Skeleton-based representations
• Solution 2: Compute additional features
22
Skeleton-based representations
• Solution 2: Compute additional features
𝒅
𝜽
23
Skeleton-based representations
• Solution 2: Compute additional features
𝒅
𝜽
24
Skeleton-based representations
• How to incorporate spatio-temporal characteristics?
25
Skeleton-based representations
• How to encode spatio-temporal characteristics?
A simplest method is to use
HISTOGRAMS
26