CSCI498B/598B
Human-Centered Robotics
Nov 05, 2014
Slides of LfD are adapted from Dr. Aude G. Billard
Gesture Recognition
Biological
Inspiration
Robotic
Learning by Imitation Implementation
Motor Learning
Gesture Recognition
Robotic
Learning by Imitation Implementation
Motor Learning
Imitation Learning in Robots
Granularity
Cognition
How to imitate?
Level 3: Learning primitives of motion
Level 2: Reproduction of trajectories
Level 1: One-shot learning
Level 0: Implicit imitation
Learning What to imitate
The robot should learn that the important feature in
this task is that the queen should be moved 2 steps
forward vertically
Once the robot has learned the rule of motion for the queen,
it can apply this rule for moving the queen from
locations not seen during the demonstrations
Transmitting human skills and knowledge to robots
Learning a Packaging Task
IMITATION LEARNING VERSUS MOTOR LEARNING
Imitation learning – Programming by
Demonstration:
• A way to speed up learning, to reduce the search
space
• A way to share with the robot’s the same
vocabulary of motor skills
Self Motor Learning - Reinforcement
Learning
• To adapt to novel situations
• To adapt the demonstrated motions to the robot’s body
IMITATION LEARNING VERSUS MOTOR LEARNING
Imitation learning – Programming by
Demonstration:
• A way to speed up learning, to reduce the search
space
• A way to share with the robot’s the same
vocabulary of motor skills
Self Motor Learning - Reinforcement
Learning
• To adapt to novel situations
• To adapt the demonstrated motions to the robot’s body
From Recognizing to Reproducing Gestures
A.G. Billard - SHS Program in Cognitive Psychology - Spring
2007
From Recognizing to Reproducing Gestures
Recovers generalized
signal by regression
GMM Encoding:
Mixture of k Gaussians
A.G. Billard - SHS Program in Cognitive Psychology - Spring
2007
From Recognizing to Reproducing Gestures
Gaussian mixtures
Gaussian mixture model (GMM)
Gaussian mixture regression (GMR)
12
From Recognizing to Reproducing Gestures
Gaussian mixtures
13
From Recognizing to Reproducing Gestures
Calion, S., Guenter, F. and Billard, A. (2007) On Learning, Representing and Generalizing a Task in a Humanoid Robot. IEEE
A.G.37:2.
Billard - Part
SHS Program
in Cognitive
Psychology
- Spring
2007
Transactions on Systems, Man and Cybernetics,
B. Special
issue
on robot
learning
by observation, demonstration and
imitation.
IMITATION LEARNING VERSUS MOTOR LEARNING
Imitation learning – Programming by
Demonstration:
• A way to speed up learning, to reduce the search
space
• A way to share with the robot’s the same
vocabulary of motor skills
Self Motor Learning - Reinforcement
Learning
• To adapt to novel situations
• To adapt the demonstrated motions to the robot’s body
IMITATION LEARNING VERSUS MOTOR LEARNING
Imitation learning – Programming by
Demonstration:
• A way to speed up learning, to reduce the search
space
• A way to share with the robot’s the same
vocabulary of motor skills
Self Motor Learning - Reinforcement
Learning
• To adapt to novel situations
• To adapt the demonstrated motions to the robot’s body
Adapting the Demonstration to Fit the Robot’s Body
A.G. Billard - SHS Program in Cognitive Psychology - Spring
2007
Adapting the Demonstration to Fit the Robot’s Body
A.G. Billard - SHS Program in Cognitive Psychology - Spring 2007
Dynamic Adaptation of Gesture Reproduction
Dynamical system modulation to be robust to perturbations
A.G. Billard - SHS Program in Cognitive Psychology - Spring
2007
Dynamical System Modulation
Different initial conditions
Adaptation to sudden target
displacement
Different initial conditions
Adaptation to sudden target
displacement
Limitation of the System
If the novel situation differs
Importantly from the
demonstrated one, then adapting
the demonstrated trajectory is no
longer sufficient to satisfy the task.
Limitation of the System
If the novel situation differs
Importantly from the
demonstrated one, then adapting
the demonstrated trajectory is no
longer sufficient to satisfy the task.
 Need to relearn the task -- Reinforcement Learning
 Need to define a new metric – the reward
RL - To Adapt to Novel Situations
A.G. Billard - SHS Program in Cognitive Psychology - Spring
2007
RL - To Adapt to Novel Situations
SUMMARY
Learning new tasks relies on various means of teaching the
robots.
 Imitation learning is useful in that it gives hints as to the
optimal solution
 The robot must however rely on generic skills of its own
to adapt the demonstration to its own body and to the
context
 Learning of complex skills is overall relatively slow and
must proceed incrementally
27
28
29
30