Outline
• Human learning behavior
• Introduction to meta learning
• Realizing meta learning
• Experiment result
• conclusion
1
Human Learning Behavior
Human can recognize
an object precisely by
showing one or a few
that object pictures .
Learning to recognize
Human can quickly
grasp motor driving
skill if already
learnt how to
ride bicycle.
Learning new skill
2
Introduction to Meta Learning
error
Meta Learner
info
data
Learner
info
tasks
Base Learner
Architectures of traditional AI model and meta learning
Ø Traditional AI model
–
–
–
–
large amount of data
thousands iterations
learning from scratch
mastering one task
Ø Meta learning
–
–
–
–
a few data for learning
a few iterations
utilizing past experience
Artificial General Intelligence
3
Realizing Meta Learning
Three ways to realize Meta Learning
4
Learning the Optimizer
Architecture of Meta Networks
An augmented layer
5
Learning the Initialization
Viewing the process as :
•Searching internal representation
•Maximizing sensitivity to loss of
parameters
The process of MAML algorithm
6
Experiment Result on Meta Networks
Meta Networks performance on Omniglot and Mini-ImageNet
7
Experiment result on MAML
MAML performance on Omniglot and MiniImagenet
8
Conclusion
Advantage:
pDrawing on the processing of human learning
pFew data to obtain new skill
pFast adapting to new tasks with previous knowledge
pThought as stepping stone for Artificial General Intelligence
Disadvantage:
pPerforming slightly poor
pHard to train
pDistance way to catch up with humans
9
Reference
[1] Tsendsuren Munkhdalai and Hong Yu. "Meta networks." ICML, 2017
[2] Andrychowicz M , Denil M , Gomez S , et al. Learning to learn by
gradient descent by gradient descent. NIPS, 2016
[3] Finn C , Abbeel P , Levine S . "Model-Agnostic Meta-Learning for Fast
Adaptation of Deep Networks. " ICML, 2017.
[4] Nichol A , Achiam J , Schulman J . On First-Order Meta-Learning
Algorithms[J]. 2018.
[5] Gregory Koch, Richard Zemel , Ruslan Salakhutdinov
"Siamese Neural Networks for One-shot Image Recognition ." ICML,
2015
10