GENETIC ALGORITHMS AND
GENETIC PROGRAMMING
Ehsan Khoddam Mohammadi
DEFINITION OF THE
GENETIC ALGORITHM (GA)
The genetic algorithm is a probabilistic search
algorithm that iteratively transforms a set (called
a population) of mathematical objects (typically
fixed-length binary character strings), each with
an associated fitness value, into a new
population of offspring objects using the
Darwinian principle of natural selection and
using operations that are patterned after
naturally occurring genetic operations, such as
crossover (sexual recombination) and mutation.
Biological Background
•
•
•
•
•
•
Chromosome (Genome)
Genes
Proteins (A T G C)
Trait
Allele
Natural Selection (survival of fittest)
GA FLOWCHART
Which problems could be solved by
GA?
• Nonlinear dynamical systems - predicting, data analysis
• Designing neural networks, both architecture and
weights
• Robot trajectory
• Evolving LISP programs (genetic programming)
• Strategy planning
• Finding shape of protein molecules
• TSP and sequence scheduling
• َAll Optimization Problems (Knapsack,Graph
coloring,…)
GA Operations
•
•
•
•
•
•
Encodings
Initiate Population
Selection
Reproduction
Crossover (sexual reproduction)
Mutation
GA Operations (Cont.)
ENCODING(1/3)
• Fixed-Length encoding
– 1D encoding: arrays, lists, strings,…
– 2D encoding: matrices,graphs
• Variable-Length encoding
– Tree encoding: binary parser trees like
postfix,infix,…
GA Operations (Cont.)
ENCODING (2/3)
• Permutation Encoding :
– Map Coloring problem , TSP,…
– Array in size of regions, each cell has an integer
corresponding to available colors.
R=1 G=2 B=3 W=4
• Binary Encoding:
– Knapsack problem, equation solving ()
Chromosome A 101100101100101011100101
Chromosome B 111111100000110000011111
GA Operations (Cont.)
ENCODING (3/3)
• Tree encoding
– Genetic programming, finding function of given
values (elementry system identification)
(+ x (/ 5 y))
( do_until step wall )
GA Operations (Cont.)
SELECTION (1/3)
• In GA ,the object is to Maximizing or Minimizing fitness
values of population of Chromes.
• Fitness Function should be applicable to any Chromes
(bounded).
• Mostly a positive number, showing a distance between
present state to goal state.
• In NP-Complete or partially defined problems should
relatively be computed .
• Two important parameters :
– Population diversity (exploring new areas)
– Selective pressure ( degree to which better individuals
are favoured)
GA Operations (Cont.)
SELECTION (2/3)
• Roulette Wheel Selection (improved by Ranking)
– [Sum] Calculate sum of all chromosome fitnesses in population - sum S.
– [Select] Generate random number from interval (0,S) - r.
– [Loop] Go through the population and sum fitnesses from 0 - sum s. When the
sum s is greater then r, stop and return the chromosome where you are
• Not suitable for highly variance populations
• Using RANK Selection
– The worst will have fitness 1, second worst 2 etc. and the best will have fitness
N (number of chromosomes in population).
– Converge Slowly
1
2
GA Operations (Cont.)
SELECTION (3/3)
• Steady-state Selection (threshold)
– Fittest just survived
• Elitism
– Fittest selected, for others we use other selection
manners
• Boltzmann Selection
– P(E)=exp(-E/kT), like SA. Number of selections
reduces in order of growing of age
• Tournament Selection
F.Nitzche
GA Operations (Cont.)
REPRODUCTION(1/1)
• Reproduction rate
• Selected gene transfers directly to new
Generation without any change.
GA Operations (Cont.)
CROSSOVER(1/1)
• CROSSOVER rate
• Single Child
– Single-Point
11001011+11011111 = 11001111
– Multi-Point
– Uniform
– Arithmetic
11001011 + 11011111 = 11001001 (AND)
• Multi Children
GA Operations (Cont.)
MUTATION(1/1)
• Mutation rate
• Inversion
11001001 => 10001001
• Deletion and Regeneration
• …
For TSP is proved that some kind of mutation
causes to most efficient solution
GA EXTENTIONS (part 1)
• GENETIC PROGRAMMING
– solve a problem without explicitly programming
– Writing program to compute X^2+X+1
GENETIC PROGRAMMING
Genetic Programming (1/4)
PREPARATORY STEPS
Objective:
Find a computer program with one input
(independent variable X) whose output
equals the given data
1
Terminal set:
T = {X, Random-Constants}
2
Function set:
F = {+,
3
Fitness:
The sum of the absolute value of the
differences
between
the
candidate
program’s output and the given data
(computed over numerous values of the
independent variable x from –1.0 to +1.0)
-,
*, %}
Genetic Programming (2/4)
initial population
Genetic Programming (3/4)
FITNESS OF THE 4 INDIVIDUALS IN GEN 0
x+1
x2 + 1
2
x
0.67
1.00
1.70
2.67
GENETIC PROGRAMMING (4/4)
Mutant of (c)
Copy of (a)
picking “2”
as mutation
point
First offspring of
crossover of (a)
and (b)
picking “+” of
parent (a) and
left-most “x” of
parent (b) as
crossover points
Second offspring
of crossover of
(a) and (b)
picking “+” of
parent (a) and
left-most “x” of
parent (b) as
crossover points
REPRESENTATIONS
• Decision trees
• If-then production
rules
• Horn clauses
• Neural nets
• Bayesian networks
• Frames
• Propositional logic
• Binary decision
diagrams
• Formal grammars
• Coefficients for
polynomials
• Reinforcement
learning tables
• Conceptual clusters
• Classifier systems
GA EXTENTIONS (part 2)
•
•
•
•
Multi Modal GA
SOCIAL MODEL: religion based
Hybrid Methods ( associate with FL and ANN)
…
REFRENCES
• Neural Networks, Fuzzy Logic and Genetic
Algorithms ,Synthesis and Applications
S.Rajasekaran
G.A.Vijayalakshmi Pai
PSG College of Technology,Coimbatore
• http://www.smi.stanford.edu/people/koza
Doctor John R. Koza
Department of Electrical Engineering
School of Engineering
Stanford University
Stanford California 94305
• http://cs.felk.cvut.cz/~xobitko/ga/
Marek Obitko, obitko@email.cz
‫غالب افراد حق ادامه حیات دارند!‬
‫با تشکر‬