CSCI598A: Robot Intelligence
Mar. 17, 2015
2
CU-Boulder
3
UTA
4
Rutgers
5
Dataspeed
6
Record and Replay
7
Record and Replay
8
Programming by Demonstration (PbD)
9
Programming by Demonstration (PbD)
To incorporate variations from multiple demonstrations
10
Programming by Demonstration (PbD)
Definition (from Wiki): In computer science, programming
by demonstration (PbD) is an End-user development
technique for teaching a robot new behaviors by
demonstrating the task to transfer directly instead of
programming it through machine commands.
Also called:
• Learning from demonstration
• Learning by demonstration
• Imitation learning
11
Programming by Demonstration (PbD)
Prior to building any capability in robots,
we might want to understand how the
equivalent capability works in humans and
other animals: BIOLOGICAL INSPIRATION
12
Imitation Learning
Imitation Capabilities in Animals
Which species may exhibit imitation is still a main
Gesture Recognition
area of discussion and debate
Biological
Inspiration
One differentiate “true” imitation from copying
(flocking, schooling, following), stimulus
enhancement, contagion or emulation
Learning by Imitation
Imitation Learning
Imitation Capabilities in Animals
Biological
Inspiration
• Copying and Mimicry: Rats, Monkeys
Gesture Recognition
• Observe companion actor rats performing different
spatial tasks differing according to the experimental
requirements. After the observational training,
surgical ablation to block any further learning
Learning by Imitation
• Legio et al, Brain Res. Protocols, 2003
• Heyes, Trends in Cog. Sciences, 2001
Imitation Learning
Imitation Capabilities in Animals
• The observer rats displayed exploration abilities
Gesture Recognition
that closely matched the previously observed
behaviors.
Biological
Inspiration
Learning by Imitation
• Legio et al, Brain Res. Protocols, 2003
• Heyes, Trends in Cog. Sciences, 2001
Imitation Learning
Imitation Capabilities in Monkeys
Subjects who saw the Lever demonstrations tended to
Gesture Recognition
use a levering movement to pop open the lid whereas
subjects who viewed Poke, as well as the controls, did
not display this behavior at all.
Biological
Inspiration
Learning by Imitation
• Whiten et al, Journal of Comparative Psychology, 1996
Imitation Learning
Imitation Capabilities in Animals
• “True” imitation: Ability to learn new actions not part
Gesture Recognition
of the usual repertoire
• The appanage of humans only, and possibly great apes
Biological
Inspiration
Learning by Imitation
• Whiten & Ham, Advances in the Study of Behaviour, 1992
• Savage & Rumbaugh, Child Devel, 1993
Imitation Learning
Imitation Capabilities in Animals
• Complex Imitation capabilities in Dolphins &
Gesture Recognition
Parrots. Large repertoire of imitation capabilities,
demonstrating flexibility and generalization in
different contexts.
Biological
Inspiration
Learning by Imitation
• Moore, Behaviour, 1999.
• Herman, Imitation in Animals & Artifacts, MIT Press, 2002
Imitation Learning
Developmental Stages of Imitation
Biological
Inspiration
• Innate Facial Imitation (newborns  3 months)
Gesture Recognition
Tongue and lips protrusion, mouth-opening, head
movements, cheek and brow motion, eye blinking
• Delayed imitation up to 24 hours
 Imitation is mediated by a stored representation
Learning by Imitation
Meltzoff & Moore, Early Development and Parenting, 1997
Meltzoff & Moore, Developmental Psychology, 1989
Imitation Learning
Developmental Stages of Imitation
• Deferred and delayed imitation - 18 month (Piaget), 9Gesture Recognition
12 months (Meltzoff)
• Deferred imitation of novel behavior
Biological
Inspiration
67% of the infants who saw the display reproduced the act
after the week's delay, as compared to 0% of the control
infants who had not seen the novel display.
Learning by Imitation
• Piaget, Play, Dreams and Imitation in Infancy, 1962 ;
• Meltzoff, Body and the self, 1995
Imitation Learning
Goals and Intentions
• Infants aged 14 months.
Gesture Recognition
• Children imitate new action to achieve the same goal
only if they consider it to be the most rational
alternative.
Biological
Inspiration
Learning by Imitation
•Gergely, Bekkering, Giraly, Nature 415, 755, 2002
Imitation Learning
Goals and Intentions
• 18-months infants
Gesture Recognition
• Differentiate between human and machine
demonstration
 Attribute intentions only to the human
Biological
Inspiration
• Learn from unsuccessful examples
Learning by Imitation
• Meltzoff, Dev. Psychol. 31, 1995.
Imitation Learning
Goals and Intentions
Biological
Inspiration
• Imitation is hierarchical and goal-directed
Gesture Recognition
• Single-hand motions: accurate ipsilateral imitation,
48% subsitution for crosslateral imitation
• Two-hand motions: only 10% substitution for
crosslateral imitation.
Learning by Imitation
• Two-phase motion eliminates mistakes
• Adding constraints of hand gestures increases mistakes
• Bekkering, Wolschlager & Gattis, Quart. J. of Exp. Psych, 2000
Imitation Learning
Imitation in adults
• Reaches highest level of complexity
Gesture Recognition
• Is present in all activities:
Social influence in establishing group norms
Biological
Inspiration
Learning by Imitation
Imitation Learning
Imitation Capabilities in Adults
Movement observation influences movement execution
Gesture Recognition
Priming process occurs involuntarily and is not under
the actor’s control.
Biological
Inspiration
Learning by Imitation
• Brass, Bekkering, Prinz, Acta Psychologia, 2001
Imitation Learning in Animals
Take-Home Message
•
Range of imitative behaviors in animals
 Increasing in complexity across species
•
Stages of development in children imitation
 innate facial imitation
 inferring goals
 hierarchy of goals driving imitation
(hand motion takes precedence over arm gesture and location in
space)
•
Imitation in adulthood is influenced by mvmt observation,
handedness, orientation of the demonstrator
•
The underlying neural mechanisms are not yet completely
deciphered
 A better understanding of those would help shed light on
the different levels of imitation in animal behavior
Imitation Learning in Animals
Take-Home Message
Advantages: When is Imitation useful?
•
It is a powerful means of transferring skills
•
It speeds up the learning process by showing
possible solutions or conversely by showing bad
solutions
Imitation Learning in Animals
Take-Home Message
Disadvantages:
When is Imitation not useful?
•
Not appropriate: When a good solution for the
teacher is not a possible solution for the learner
•
Disadvantageous: When it induces you in error bad teacher (e.g. phoebia of spiders)