CAP6938
Neuroevolution and
Developmental Encoding
Overview of NEAT and
CPPNs
Dr. Kenneth Stanley
August 28, 2006
Why an Overview?
•
•
•
•
Preparation for declaring projects
Get an idea what is possible
Sufficient background without details
Technical details coming later in semester
NEAT: NeuroEvoluton of
Augmenting Topologies
• Evolutionary algorithm applied to evolving
artificial neural networks (ANNs)
• Evolution (i.e. genetic algorithms) is a way
to optimize ANN topology and weights
• NEAT evolves increasingly complex ANNs
• Able to solve difficult control and decisionmaking tasks
Evolution:Survival of the Roundest
Gen 1
Select as parents
Gen 2
Select as parents
Gen 3
Champ!
NeuroEvolution
• Evolve ANNs
• Evaluated for performance on a task
–
–
–
–
–
–
–
Controlling a vehicle or robot
Drawing a picture
Recognizing an image
Warning about danger
Playing a board game
Playing a video game
Playing music
• The fittest survive and reproduce
ANNs can Sense and React
NEAT Features
• NEAT is unique: Networks complexify over
evolution
Crash Warning Champ
• So they can find the right complexity for the task
• NEAT solves problems with systems of diverse
networks using historical marking and speciation
• A first step in evolving complex systems
NEAT Applications
• NEAT makes new applications possible
and has set performance records
Double Pole Balancing Record
Vehicle Warnings
Keepaway Record
NERO: Real-time Neuroevolution
in a Video Game
Robot Duel
Go
Hopper
Not Just About Control and
Classification
Interactive Picture Evolution
Interactive Drum Pattern Evolution
Interactive Particle Effect Evolution
Interactive Chord Progression Evolution
Guitar Effect-Pedal Emulation
Non-neural NEAT
• ANNs aren’t the only thing that can
complexify
• Anything with arbitrary structure can
potentially be evolved
– Cellular Automata
– Lego Structures
– FSMs
– L-systems
L-System Plant Geometry Generator
by Hung-Wen Chen
What Can’t NEAT Do?
• “Regular” NEAT is a direct encoding
• It doesn’t encode regularities or symmetries
• Regularity and symmetry are believed to be
essential to representing and searching complex
structures: gene reuse
• Giant complex structures in nature are
composed of patterns
• Patterns have regularities
• Indirect encodings capture such information
efficiently
Indirect Encoding
• Most are developmental encodings
– Based on embryo to adult development
– “Artificial Embryogeny”
– This is how encoding works in nature
• Many have been proposed
• None close to natural complexity
Proposed Indirect Encodings
What’s missing?
• None implement complexification as in
NEAT: maybe genes can be added and
marked like in NEAT
– For example: L-system NEAT?
• Something’s still missing…
A Recent Discovery
• Drawn by a NEAT-evolved neural
network!?
Compositional Pattern Producing
Networks (CPPNs)
• Networks of functions composed to
produce patterns with regularities
• An abstraction of natural development
without developmental simulation
• Complexifying repeating patterns,
regularities, symmetries, repetition with
variation, etc.
CPPN-NEAT
•
•
•
•
A new kind of indirect encoding
Elegantly captures regularities
Convert patterns into phenotypes
An entirely new research area
Exploiting Regularity Without Development by Kenneth O. Stanley, To appear in: Proceedings of the AAAI Fall
Symposium on Developmental Systems. Meno Park, CA: AAAI Press, 2006 (8 pages)
Music is a Pattern over Time
• A CPPN is a pattern generator that can
produce repetition with variation, just like
music
• A CPPN can produce several related
patterns from an underlying simpler
pattern (multiple instruments)
• Music generation is a promising
demonstration
Homework for 8/30/06
• Neural network weight-learning algorithms
Fausett pp. 39-80 (in Chapter 2)
Fausett pp. 289-316 (in Chapter 6)
Chapter 1 of Sutton (on RL): Online Book ch.1
Optional: Kaelbling RL survey paper
• Remember proposals/partners due 9/11