CSCI498B/598B
Human-Centered Robotics
Nov 03, 2014
Slides of LfD are adapted from Dr. Aude G. Billard
Gesture Recognition
Biological
Inspiration
Robotic
Learning by Imitation Implementation
Motor Learning
Gesture Recognition
Robotic
Learning by Imitation Implementation
Motor Learning
The Transfer Problem
Imitator
Demonstrator
 , , 
1
2
4
 , , 
5
6
?
3
7
 , , 
1
3
 4

x   x1 , x2 , x3 
 , ,  
5
6
7

x   x1, x2 , x3 
What to imitate?
 
x  x
 
d  d
 
v  v
 
  
Same target location
Same direction of motion
Same speed, same force
Same posture
 , , 
 , , 
1
2
1
3
4
 , , 
5
6
7

v

x   x1 , x2 , x3 

d
2
3

x   x1, x2 , x3 

d
 4
 , ,  
5
6
7

v
How to Imitate?
The correspondence problem
Demonstration
Imitation
?
No solutions
 Find the closest solution according to a metric
Imitation Learning
Following – an imitation mechanism
• While following the teacher, the learner robot learns to
Gesture Recognition
associate a word with a meaning in terms of sensory inputs
Robotic
Learning by Imitation Implementation
• Billard et al, ESANN’1997,
• Billard & Dautenhahn, Robotics & Autonomous Systems 1998,
• Billard & Hayes, 99,00
Imitation Learning
Following – an imitation mechanism
Gesture Recognition
•Teaching path in a Maze
Demiris & Hayes, 1994, 1996;
Robotic
Learning by Imitation Implementation
• Teaching how to climb a hill
Dautenhahn, Robotics & Autonomous Systems, 1995
• Teaching a path in the environment
Billard & Hayes, Adaptive Behavior, 1999
Moga, Gaussier, Applied Artificial Intelligence, 2000
Kaiser et al, Robotics & Autonomous Systems, 2002
Nicolescu & Mataric, AGENTS’ 2003
Imitation Learning
One-Shot Learning Methods
• Segmentation of demonstration into primitives
Gesture Recognition
• Classification of gestures into predefined states (e.g.
grasp, collision)
• Built-in controller for producing sequences of states
Robotic
Learning by Imitation Implementation
• Kuniyoshi et al. IEEE Trans. on Robotics and Automation,1994.
• Dillmann et al, Robotics & Autonomous Systems, 2001.
• Ritter et al, Rev Neuroscience, 2003
• Aleotti et al, Robotics & Autonomous Systems, 2004.
Robot Programming by Demonstration
One-Shot Learning Methods
Sensors: Data Gloves, Fixed cameras, Speech processing
Actuators: Mobile robot, 7 DOF arm, 2 fingers Gripper
R. Dillmann, Robotics & Autonomous Systems 47:2-3, 109-116, 2004
Imitation Learning
One-Shot Learning Methods
Explicit teaching/learning:
- Reasoning about tasks
- Verbal instructions
Gesture Recognition
Gesture Recognition:
For each sensor a context-dependent
Learning
model based on background
knowledgeby
is provided: ‘opening the refrigerator door’,
‘extracting the bottle’ and ‘closing the door’
Task Reproduction:
Store action sequences in a tree-like
structure of macro-operators
R. Dillmann, Robotics & Autonomous Systems
47:2-3, 109-116 2004
Robotic
Imitation Implementation
Imitation Learning
Robot Programming by Demonstration:
Grasping
Gesture Recognition
Because of the large range of possible
shapes, generalizing pre-programmed
grasps to new and general objects is a
rather hard task:
Robotic
Learning by Imitation Implementation
• Orientation of the hand
• Positioning of the fingers (correspondence
problem!)
• Tactile forces, stable object contact
Steil et al, Robotics & Autonomous Systems 47:2-3, 129-141, 2004
Imitation Learning
Robot Programming by Demonstration:
Grasping
Gesture Recognition
(i) a ‘naïve’ imitation strategy, in which the
observed joint angle trajectories (after their
transformation into the three-finger
geometry) were directly applied to control
the fingers of the hand duringLearning
the grasp,
until complete closure around the object
Robotic
by Imitation Implementation
(ii) a strategy in which the visually observed
hand posture is matched to the initial
conditions of a power grip, a precision grip,
a three-finger and two-finger grip,
respectively, in order to identify the grip
type.
Steil et al, Robotics & Autonomous Systems 47:2-3, 129-141, 2004
Robot Programming by Demonstration
Other related works are, e.g.:
•
•
•
Kuniyoshi et al, ICRA, 1994
Aleotti et al, Robotics & Autonomous Systems, 47:2-3, 153-167, 2004
Zhang & Roessler, Robotics & Autonomous Systems 47:2-3, 117-127, 2004
Imitation Learning
Learning of Dynamical Systems
• Learning the optimal controller
Gesture
• Model of physical system
• Reinforcement learning
Recognition
Robotic
Learning by Imitation Implementation
• Atkeson & Schaal, ICML, 1997.
Imitation Learning
Learning of Dynamical Systems
• Learning primitives of the system
Gesture Recognition
Robotic
Learning by Imitation Implementation
•Ijspeert, Nakanishi, Schaal, ICRA’01, NIPS’02
Imitation Learning
Learning of Dynamical Systems
•Learning primitives of the system
Gesture Recognition
Robotic
Learning by Imitation Implementation
•Ijspeert, Nakanishi, Schaal, ICRA’01, NIPS’02
Imitation Learning
Learning of Dynamical Systems
The learned trajectory is not sufficient to control the
Gesture Recognition
actual robot’s walking pattern.
Phase resetting using foot contact information is
necessary.
 on-line adjustment using sensory feedback from the
environment is essential to achieve successful
Learning by Imitation
locomotion
Nakanishi et al, Robotics & Autonomous Systems, 47:2-3, 79-91, 2004.
Robotic
Implementation
Imitation Learning in Robots
Granularity
Cognition
How to imitate?
Level 3: Learning primitives of motion
Level 2: Reproduction of trajectories
Level 1: One-shot learning
Level 0: Implicit imitation
20
21