Darwin’s theory of
Evolution
http://www.wsu.edu:8001/vwsu/gened/learn-modules/top_longfor/overview/Overvw5.html
Evolution
http://www.wsu.edu:8001/vwsu/gened/learn-modules/top_longfor/overview/Overvw2.html
http://emporium.turnpike.net/C/cs/cartoon.htm
Genetic Algorithms (GA)
• Derivative-free Stochastic Optimization
method based loosely on the concepts of
natural selection and evolution processes.
• First proposed and investigated by John
Holland at U. of Michigan in 1975.
• Finding significant applications in many
venues.
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Genetic Algorithms (GA)
• Use the concept of Darwin’s theory of
Evolution
– the existence of all living things is based on the
rule of “survival of the fittest”
– New breeds or classes of living things come into
existence through the processes of reproduction,
crossover, and mutation among organisms.
• These concept have been translated into
algorithms to search for solutions to problems
in a more “natural” way.
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Genetic Algorithms (GA)
• GAs are parallel-search procedures that can be
implemented on parallel-processing machines for
massively speeding up their operation.
• GAs are applicable to both continuous and discrete
optimization problems.
• GAs are stochastic and less likely to get trapped in
local minima
• GAs’ flexibility facilitates both structure and parameter
identification in complex models such as NN and Fuzzy
Inference System
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Genetic Algorithms (GA)
• Benefit
– GA searches for a solution from a board
spectrum of possible solutions, rather than
restrict the search to a narrow domain where
the results would be normally expected.
– GA try to perform an intelligent serch for a
solution from a nearly infinite number of
possible solutions.
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA Parameter Encoding
• Parameter set of the problem is coded as a
finite binary string of bits (chromosome)
– An n-bit string can accommodate all integers up to the
value 2n -1.
– Bit string may be mapped to the value of a parameter
(gene), Ci, i = 1,2,… by the mapping
Ci = Cmin + ( b/(2L -1)) *(Cmaxi-Cmini)
– Where “b” is the number in decimal form that is being
represented in binary format.
– Cmax and Cmin are maximun and mininmum value of the
parameters.
– L is the length of the bit strings
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA Parameter Encoding: Example
• Given a data set C
– Assume Cmin = -2 and Cmax = 10,
– Each string, represented X, is 10 bits in length
• When b = 0000000000 or “0” in decimal then
C = -2
• When b = 1111111111 or “1023” in decimal then
C = 10,
• If b = 1000000000 or “512” in decimal then
C = -2 + 512/ (1024-1)*(10- (-2))
= -4.01
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Encoding schemes
• Concatenate genes to form chromosome
–Example: A point (11,6,9) in a three-dimensional
parameter space can be represented as a concatenated
binary string:
11
6
9
1011 0110 1001
Assume each gene…….. min = 0, max =15, and L =4
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Genetic Algorithms (GA)
• Algorithm
– Step 1. Create different possible solutions to a
problem
– Step 2. Test the solutions for their performance (i.e.,
how good a solution they provide)
– Step 3. Select only a fraction of good solutions
(eliminate the others - survival of the fittest)
– Step 4. Create a new generation of possible solutions,
children, by the processes of reproduction, crossover
and mutation of the selected solutions in step 3.
– Step 5. Repeated step 2 to 4 until there is convergence
within a generation.
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Producing the next Generation in GAs
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Three Genetic operators - Reproduction
#1 Reproduction -the process by which
strings with better fitness values receive
correspondingly better copies in the new
generation (survival of the fittest)
• Total number of strings in each generation
is kept constant thus strings with lower
fitness values are eliminated
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Three Genetic operators - Crossover
#2 Crossover -the process by which the strings
are able to mix and match their desirable
qualities in random fashion
- 3 steps
a) two new string are selected at random
b) a random location in both strings is selected
c) the portions of the strings to the right of the
randomly selected location in the two strings
are exchanged
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Three Genetic operators - Mutation
#3 Mutation -helps to increase the searching
power (reproduction or crossover may not be
able to find an optimum solution to a problem)
Occasionally, the value at a certain string
location is changed - Mutation thus ensures
that the vital bit of information is introduced
into the generation!
Mutation take place very rarely, on the
order of once in a thousand bit string locations
(suggested 0.005/bit/generation)
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Summary of simple GA for maximization problems
– #1 Initialize a population with randomly generated
individuals and evaluate the fitness value of each
individual
– #2
• a) select two members from the population with
probabilities proportional to their fitness values.
• b) apply crossover with a probability equal to the
crossover rate.
• c) apply mutation with a probability equal to the
mutation rate.
• Repeated (a) to (d) until enough members are
generated to form the next generation.
– #3 Repeat #2 and #3 until a stopping criterion is met
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Genetic Algorithm: Example 1
• Show how a line may be fit through a given data
set using GA
– Given Data set through which a line fit is required
Data number
X
Y'
1
2
3
4
1.0
2.0
3.0
6.0
1.0
2.0
3.0
6.0
and
Y = C 1x + C 2
Need to solve for C1 and C2 using GA
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA: Example 1 (step 1)
• Step 1. Encode the parameter set (C1,C2) in form
of bit strings.
– Bit strings are created with random assignment of
1s and 0s at different bit locations
– Let’s start with an initial population of four strings
– The strings are 12 bits in length (chromosome)
• Each gene, C1 and C2, is encoded with 6 bits
– Assume that Cmin = -2, and Cmax = 5
– Calculate decimal equivalent of C1 and C2
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA: Example 1 (Step 2)
• Step 2. Calculate Y1, Y2, Y3, and Y4, from the line
equation using Xi , C1 and C2.
– Test the solutions for their performance by finding the
square of the errors in estimating the Y’s for each string.
This summation is subtracted from a large number, 400, to
convert the problem into a maximization problem.
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA: Example 1 (Step 2 Cont.)
• Step 2. Continue
– Column 11 are the fitness values of the four strings
• These fitness values are added and compute its average
– Column 12 are relative fitness of each string
• Take each fitness value and divide by the average fitness of the
whole population
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA: Example 1 (Step 3)
• Step 3. Select only a fraction of good solutions
– Eliminate string #1 due to low expected count
– Duplicate string #3 to replace string #1
– Keep string #2, #3, and #4 ( survival of the fittest)
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA: Example 1 (Step 4)
• Step 4. Create a new generation of possible solutions by the processes of
reproduction, crossover and mutation of the selected solutions
– The first column shows the four strings selected from previous step
aligned for crossover at the locations shown in the strings in the
column
– After crossover, the new strings generated are shown in column 2
Fuzzy Logic with Engineering Applications: Timothy J. Ross
GA: Example 1 (Step 5)
• Step 5. Repeated step 2 to 4 until there is convergence within a generation.
Note: the average fitness of the second generation is greater than
that of first generation.
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: Computing MFs using GA
• A single input, X, and single output, Y with and
input-output values and a functional mapping
for the system between input and output shown
below:
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: MFs
• If X is Small then Y is Large
• If X is Large then Y is Very Large
• Assume that each MF has the shape of a right
triangle as shown below
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: Step 1
• Pick initial Base for input and output MFs
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: First Iteration 1
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: First Iteration 2
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: First Iteration 3
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: 2nd Iteration 1
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: 2nd Iteration 2
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #2: 2nd Iteration 3
Fuzzy Logic with Engineering Applications: Timothy J. Ross
Example #3: Maximization of the “peaks”
function using GAs
Peaks
8
6
4
2
0
-2
-4
-6
2
0
-2
y
-3
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
-2
0
-1
x
1
2
Example #3: Maximization of the “peaks”
function using GAs
Peaks
8
6
4
• Consider [-3,3] X [-3,3]
• Use 8-bit binary coding for each variable
2
0
-2
-4
-6
2
0
-2
y
-3
-2
0
-1
1
x
Search space size of 28 x28 = 65,536
• Each generation in GA contains 20 points
• Each point’s fitness is defined as the value of the
“peaks” function minus the minimum function value
across the population (guarantees non-negative)
• One point crossover with crossover rate = 1
• Uniform mutation with mutation rate = 0.01
• Apply elitism to keep the best twoacross generation
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
2
Example #3: Maximization of the “peaks”
function using GAs
3
2
Y
1
0
-1
-2
-3
-3
-2
-1
0
X
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
1
2
3
Example #3: Maximization of the “peaks”
function using GAs
Initial Population
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Example #3: Maximization of the “peaks”
function using GAs
Population after the fifth generation
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Example #3: Maximization of the “peaks”
function using GAs
Population after the tenth generation
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997
Example #3: Maximization of the “peaks”
function using GAs
Performance of Gas across generations
*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997