High-Performance Genetic Algorithms for Minimization
Winard Britt, Kenan Casey, Hurley Cunningham, Philip Sitton
Department of Computer Science and Software Engineering
Auburn University, AL 36849-5347
[brittwr, caseykl, cunnihu, sittopa]@auburn.edu
Abstract
In this paper we explore a number of different Genetic
Algorithms in order to rapidly find near-optimal solutions to
the F6 Minimization problem. Specifically, we created and
tested a binary-coded Simple Genetic Algorithm (SGA), a
Real-Coded Simple Genetic Algorithm (SRGA), a SteadyState Binary-Coded GA (SSBGA), and a Steady-State RealCoded GA (SSRGA). Further, we explored performanceenhancing modifications to each of these algorithms. Each
algorithm and its corresponding modifications and
parameter settings are described. The algorithms were
tested using a direct mapping between the output of the F6
function and the fitness function for the EP. The results of
those experiments are presented and some rationale is given.
1. Introduction
Genetic Algorithms (GAs) are a subset of Evolutionary
Computation (EC), a collection of stochastic search
algorithms based on biological models. GAs are robust
adaptive systems which have been applied to many
different types of problems [1]. Originally, GAs focused
on manipulating individuals in terms of their genetic
makeup rather than using a phenotypic representation [2].
In order to search, these algorithms first generate a
population of “individuals” which represent points in the
search space. Individuals can then be evaluated using a
evaluation function which should accurately represent the
fitness of an individual. In each evolutionary cycle, the
following basic steps occur: 1) some “parent individuals”
are chosen 2) these parents create “children” through some
means of variation 3) some subset of the parents and
children are chosen to replace the existing population.
Since pressure is applied at either parent selection, survivor
selection, or both the population tends towards better
solutions. This process is repeated until some stopping
condition (finding an optimal solution, for example or a
certain number of evolutionary cycles) is met.
In this paper, we will provide a description of the problem
in Section 2. In Section 3, we will explain in detail the
algorithms themselves and their respective modifications.
The parameters for the experiments are described in
Section 4 and the results of said experiments are detailed in
Section 5. We examine the overall performance and
provide statistical analysis of the algorithms in Section 6.
We provide a work breakdown in Section 7.
2 The F6 Problem
2.1 Description
The various GAs designed were tasked to minimize the F6
function:
F 6( x, y )  0.5 
sin 2 ( x 2  y 2 )  0.5
1.0  0.001  ( x 2  y 2 ) 2
The search space contains many peaks and local minima,
complicating matters slightly since a mutation can easily
destroy a good individual. However, the search space is
certainly too vast to efficiently pursue an exhaustive search.
2.2 Test Suite
Our evaluation function for all the algorithms was to
extract the values of x and y from the individual and
calculate the value of the F6 function. This value would
then be mapped to the fitness of the individual. At the end
of a given run, the algorithm returns its best fit individual.
For each algorithm, we average the best individuals from
all of 100 runs for statistical significance.
3. Algorithm Description
3.1 Simple Genetic Algorithm
3.1.1 Binary Coding
Our binary coded genetic algorithms use a binary
representation for their chromosomes. Our implementation
uses twenty bits to represent the x value, and twenty bits to
represent the y value. We must map the binary strings (x
and y must be decoded separately) to a real value so that
we can use them in the calculation of the f6 value;
therefore, we used the following function to perform this
task, where d(ub,lb,l,chrom) represents the decoding
function, which transforms a binary string into a real
representation: ub is the upper bound of the real value that
the binary string is being mapped to (100), lb is the lower
bound of the real value that the binary string is being
mapped to (-100), l is the number of bits in the string (20),
and chrom is the value of the string.
d(ub,lb,l,chrom) = (ub-lb) decode(chrom)/(2l-1) + lb
3.1.2 Standard Algorithm Description
Our SGA first randomly generates a population of binarycoded individuals (genes are randomly set to either 0 or 1)
and evaluates those individuals. In order to evaluate
individuals, the binary string is decoded to decimal values
as described in section 3.1.1. Fitness is defined according
to the F6 function, where x and y that produce minimal
outputs are said to be more fit than those that produce
larger outputs.
Each evolutionary cycle, a population size (µ) children are
created as follows. A first parent is chosen using binary
tournament selection (two random individuals in the
population are selected and the more fit of the two “wins”
the tournament). This process is repeated twice in order to
generate two parents. With a probability of 1.0, a singlepoint crossover operation is performed on the two parents.
In this operator, a random cut-point is chosen within the
chromosome. All of the first parent’s gene values before
the cut-point go to the child; all of the second parent’s gene
values after the cut-point are similarly taken by the child.
Afterwards, with a mutation rate k, each gene in the child
has a chance to be flipped. This process is repeated until µ
children have been created. These children then replace
the parents.
3.1.3 Modified Algorithm Description
Three modifications were applied, of which two ultimately
provided improved performance.
The first modified version used a modified one-point
crossover algorithm. When creating the child from the two
parents, the larger of the two pieces of the chromosome
created by the cut-point was always taken from the better
fit parent. This was intended to use more “good” genetic
material in the children.
The second modified version involved the introduction of
an elitism-like operator.
After children have been
produced and evaluated, the algorithm checks to see if the
best parent is better than the worst child. If so, the best
parent replaces the worst child.
3.2 Simple Real-Coded GA
3.2.1 Standard Algorithm Description
The simple real-coded GA (SRGA) differs from the simple
GA in that chromosomes are stored in their phenotypic
forms, i.e., as values in the domain of the problem. As a
result no decoding is necessary. In this case, the x and y
values for individuals are represented by two floating point
numbers. Our standard SRGA uses BLX-0.5 crossover and
no mutation. BLX-0.5 is a method of blend crossover
which creates children using the following formulas:
Given genei, parent1 > genei, parent2,
Let  = (genei, parent1 - genei, parent2), where  = 0.5.
Then genei, child = rnd(genei, parent1 - , genei, parent2 + ).
We used binary tournament selection twice in order to
choose the two parents for BLX-0.5 crossover. Binary
tournament selection means that two individuals were
randomly selected from the population. The individual with
the better fitness was selected to be a parent.
Like the SGA, the SRGA also uses a (µ, µ) replacement
strategy, replacing the entire population each generation.
3.2.2 Modified Algorithm Description
We modified the SRGA algorithm in three ways. First, we
experimented with the crossover algorithm. We evaluated
the effect of various values of  in the BLX algorithm in
addition to the default value 0.5. This varied the amount of
disruption that crossover created. Values of  less than 0.5
result in children more likely to be between their parents
while values greater than 0.5 lead to more extreme
offspring. We also implemented a midpoint crossover and a
line crossover. Midpoint crossover calculates the midpoint
(or average) of the real-coded chromosomes of the parents.
Midpoint crossover follows the following formulas:
genei , child 
The third modified version (implemented in conjunction
with modification two) resulted from the observation that
the SGA often became “stuck” with a relatively bad (> 0.01
fitness) best individual for long periods of cycles (1000 or
more) with no improvement. In order to combat this, if the
algorithm recognizes that the best individual is worse than
0.01 fitness after 1000 iterations, it saves the best
individual and re-initializes the population. When all 2000
iterations have finished, the better of the two individuals
(either the individual produced by the first 1000 iterations
or the individual produced by the second 1000 iterations) is
returned. Otherwise, it continues to run with the current
population and returns the best individual.
genei , parent1  genei , parent2
2
Line crossover returns a random value located on the line
connecting the two parents. The following formulas
describe the algorithm:
Given 0.0  t  1.0 ,
genei , child  ( genei , parent1  genei , parent2 )  t
 genei , parent2
The second modification applied to the selection method.
We extended the tournament selection algorithm to include
qn individuals. For binary tournament selection qn=2. When
qn=1, parents are selected randomly without any
tournaments. The value of qn affects the selection pressure
since greater values of qn result in a greater likelihood that
better individuals are selected as parents. Likewise, it
reduces diversity since poor-fitness individuals are less
likely to be chosen as qn increases. We also experimented
with the parameter qp which represents the number of
individuals per tournament as a percentage of the
population size. To make sure we get at least binary
tournaments we chose q  max( q p  popsize , 2) .
Thirdly, we developed an elitist strategy to ensure that the
top e fraction of the population survived to the next cycle.
This strategy prevents the best individuals from being
replaced. Consequently, we are guaranteed to have the best
e   individuals ever created.
3.3 Steady-Steady Binary-Coded GA
3.3.1 Standard Algorithm Description
The steady state binary coded genetic algorithm (SSBGA)
uses the binary representation presented in section 3.1.1.
In order to select parents for the creation of offspring, the
SSBGA uses tournament selection with q equal to 1. This
means that two individuals are randomly selected, and the
better of the two is used as that parent. This process is
performed twice, resulting in two parents for each cycle.
Our SSBGA uses uniform crossover in order to create two
offspring from the two selected parents. Then, the genes
of the resulting offspring are zapped with mutation with a
mutation rate of k. This mutation is performed by flipping
the bit’s value. Once the offspring have been mutated with
the specified rate, the two offspring replace the two worst
individuals in the population, regardless of whether they
are better that the worst or not.
3.3.2 Modified Algorithm Description
In order to improve the performance of the SSBGA
algorithm, we attempted to place some selection pressure
on the search by always making one of the parents the best
individual in the population. Essentially, the first parent of
any offspring creation is the best individual currently
present, and the second parent is chosen as in the normal
algorithm description above.
3.4 Steady-State Real-Coded GA
3.4.1 Standard Algorithm Description
Our SSRGA randomly selects initial individuals from the
set of ordered pairs (x, y) where -100 < x, y < 100. As our
goal is to minimize the F6 function, an individual i’s fitness
is defined by F 6( xi , yi ) .
As populationSize individuals are randomly generated and
their fitnesses evaluated, they are placed into our
population p. Two “parent” individuals a and b are then
randomly chosen to procreate two new “children”
individuals via tournament selection, in which q individuals
are chosen and the individual with the best fit is chosen to
be a parent. Children are created according to the BLX-0.5
scheme described in section 3.2.1.
Once two children have been generated, they replace p’s
two worst-fitting individuals regardless of whether their
fitness is better or worse. The process continues until 2000
individuals have been created and evaluated. The best-fit
individual serves as the output of our SSRGA.
3.4.2 Modified Algorithm Description
Our modification to the SSRGA attempts to improve
performance by using a randomized midpoint crossover
scheme rather than the BLX-0.5 scheme. Supplied a
parameter β, our midpoint crossover scheme defines a
child’s genes in the following way:
genechild 
gene parent1  gene parent2
2
  * N (0,1)
The mid-point crossover scheme was chosen as the basis
for this modification as its results were far superior to
BLX-0.5, probably in part because the best solutions to the
F6 function are near the origin, and thus individuals are
likely scattered about it on all sides. The Gaussian term
promotes diversity, and β helps control the magnitude of its
influence.
Further, in each iteration we used tournament selection to
choose two pairs of parents that each procreate one child,
rather than one pair that procreates two children. This
scheme works around the problem of creating two very
similar children each iteration, as many schemes based
upon midpoint crossover would.
4. Parameter Settings Used
We ran each algorithm for 100 independent runs, with each
run being allowed 2000 iterations before termination.
4.1 SGA
For the SGA, we tested the algorithm with a variety of
population sizes and k values, as follows.
These
population sizes are significant because they represent
values within a range which provided good performance
for at least one of our algorithms. Higher population sizes
than 500 proved ineffective, as did population sizes smaller
than 3. Our exploration of k resulted in similar findings
(larger or smaller k values than those described proved
ineffective).
µ: [3, 6, 12, 25, 50, 100, 200, 400, 500]
k: [0.01,0.03,0.06,0.12,0.25,0.5]
4.2 SRGA
For the SRGA, we tested the algorithm with the same
population sizes as above. We intentionally choose large
population sizes that evenly divided 2000. This ensures that
a full generation is created on the last cycle and avoids the
problem of what to do with a half generation. In the
modified version of the SRGA, we experimented with
values taken from the following sets for , qn, qp, and e:
: [0.0, 0.1, 0.25, 0.3333, 0.5, 0.6667, 0.75, 0.9, 1.0]
qn: [1, 2, 5, 10, 20, 50]
qp: [0.01, 0.02, 0.05, 0.1, 0.2, 0.3 ]
e: [0.01, 0.02, 0.05, 0.1, 0.2, 0.3 ]
4.3 SSBGA
The SSBGA only has two parameters (population size µ
and mutation rate k) that directly affect the performance.
These parameters were taken from the same set as
described in Section 4.1.
4.4 SSRGA
The unmodified SSRGA has two parameters that influence
the performance, population size µ and number of
tournaments q. We tested the algorithm with the same
population sizes as in 4.1. The modified version introduces
a scale factor β. The following parameter settings were
used:
q: [1, 2, 5, 10, 20, 50]
β: [0.01, 0.001, 0.0001]
5. Analysis of Results and Discussion
For all of our analysis, a t-test infers a pairwise, one-tail,
type 1 t-test performed using the statistical analysis
package in Microsoft Excel. Two classes are considered
“statistically different” if the value returned by the t-test
indicates the probability of the two classes being the same
falls below 0.05.
5.1 SGA Results
In general, the SGA performed best with relatively small
populations (3 – 12) and low to mid-level mutation rates
(0.03 – 0.12). However, in the case of the unmodified
SGA, our best results were obtained using a population size
of 25 and a k value of 0.12, which produced an average
best individual of 0.0428. This relatively poor
performance is due to the fact that without any mechanism
for elitism, the algorithm constantly erases the best
individual found so far. The proposed mechanisms to
retain the best individual dramatically improved
performance.
The first modification (altered crossover operator) did not
produce statistically significant improvement in
performance.
The second modification (an added elitism-like operator)
decidedly improved the results. In this case, the best
performing algorithm was with a population size of 3 and a
k value of 0.12, producing an average best individual of
0.008216. These results are statistically significantly
different from the standard algorithm. This is likely caused
by the ability of the algorithm to thoroughly search with a
small number of individuals, rather than searching only a
small area with a large number of individuals.
The third modification (resetting the population, also with
modification two) provided notable improvement for
smaller populations, but seemed to have little effect on
larger populations (population size > 200). Since large
populations have relatively few evolutionary cycles before
the algorithm is terminated, it makes sense that restarting
the population will have little effect. The best settings
produced an average best individual of 0.00579, with
population size 3 and k = 0.12. As stated in Section 3.1.3,
this improvement stems from the tendency of the algorithm
to get “stuck” in a region without seeing improvement for
many iterations. Restarting the population gives it the
opportunity to explore potentially better regions of the
search space.
5.2 Simple Real-Coded GA
For the standard SRGA using BLX-0.5 (=0.5) and binary
tournament selection (qn =2), we found the minimum
solution of 0.027742828 at population size 100. The SRGA
performed better at larger population sizes, Figure 1 shows
that values of μ ≥100 gave better performance. For small
populations the SRGA was unable to produce acceptable
results. Consequently, we tested our modifications with
larger population sizes. Larger population sizes performed
better because they have a greater probability of producing
very good individuals in the first generation. Even though
only a few generations are created, the GA selects the best
individuals more often as parents, allowing them to create
offspring more frequently.
Fitness versus Population Size
0.5
0.45
0.4
Fitness
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0
100
200
300
400
500
600
Population Size
Our third modification to the SRGA was the use of an
elitist strategy. When BLX crossover was employed, there
was no significant difference in performance at any elitism
value especially for larger population sizes. The Midpoint
crossover algorithm, however, did show improvement with
e  0.1 (Figure 3). We found the overall minimum solution
(0.00203948) with Midpoint crossover, population
size=500, qn=5, and e=0.3. Elitism values greater than 0.1
produced results that were about half of the values
generated without an elitist strategy (e=0.0). An elitist
strategy improves the performance of the GA with
Midpoint crossover because it preserves the best
individuals across generations.
Figure 1: Fitness Versus Population Size
Fitness versus Population
0.05
BLX-0.0
Fitness
BLX-0.1
0.04
BLX-0.25
BLX-0.333
0.03
BLX-0.5
Midpoint
0.02
Line
0.01
0
200
300
400
0.008
Midpoint
Line
0.006
BLX-0.0
BLX-0.1
0.004
BLX-0.25
0.002
0
0
0.2
0.4
0.6
0.8
500
600
Population
Figure 2: Fitness Versus Population Size for Various
Crossover Algorithms
Next we varied the number of individuals that competed in
each tournament. We found that a smaller qn performed
best (e.g. 5 ≤ qn ≤10) for our larger population sizes. To
find a correlation between qn and population size we ran
the SRGA with the parameter qp, a percentage of the
population. Not surprisingly, our best performance
occurred with qp around 1%- 5%. Small qn and qp perform
best because they provide the best balance between
diversity and selection pressure for this problem.
1
Elitism (e )
Figure 3: Fitness Versus Elitism for Crossover
Algorithms
For the SSBGA, the changes in population size within a
given mutation rate seemed to have somewhat minimal
impact. Overall, the changes in the average best solution
within a given mutation rate were less than or equal to 0.03.
Looking at the best solution we were able to achieve with
the SSBGA, we see the fitness value 1.68E-8 fairly
frequently within the best individuals found. In fact, this
value is found for every population size for very low
mutation rates, for almost every population size for the
mid-ranged mutation rates, and for very high mutation
rates, the SSBGA was not able to find this solution. We
believe that since this specific solution was so prominent in
the outputs, that this is the best possible solution that can be
obtained using the granularity that we specified for our
chromosomes. Over an average of one hundred runs, we
find that the best average best solution for the standard
SSBGA is 0.0073 which occurs with a population size of
25 and a mutation rate of 0.06. Figure 4 depicts the
average best solutions obtained with both the modified
SSBGA and the original SSBGA. As we can see, on a
scale of zero to 0.01, these values are fairly close to each
other.
Fitness vs. Algorithm
0.009
0.008
0.007
0.006
tness
0.06
100
0.01
5.3 SSBGA
0.07
0
0.012
Fitness
Our first modification evaluated the effect of varying the
blending factor . We found that small  values perform
best (0.0 ≤  ≤ 0.25). Interestingly, the minimum solution
for this modification (0.012129) was found using =0.0
with population size 400. The performance was almost
identical for =0.0 to =0.25 for population sizes greater
than 100. We achieved the best performance using
Midpoint crossover with population size 400. The average
best solution was 0.009409. Line crossover performed next
best at the largest population sizes (Figure 2). Small 
values perform well because they focus the search on
values between the parents. Since midpoint crossover
always creates a child exactly half way between the parents,
it performs even better than BLX with small  values.
Fitness versus Elitism
(popsize =500, q n =5)
0.005
Fitness versus Tournament Size
0.5
0.45
0.4
Pop. Size: 3
0.35
Pop. Size: 6
Fitness
Pop. Size: 12
0.3
Pop. Size: 25
0.25
Pop. Size: 50
Pop. Size: 100
0.2
Pop. Size: 200
0.15
Pop. Size: 400
0.1
Pop. Size: 500
0.05
0
1
2
5
10
20
50
Tournament Size
When we begin looking at the modification that we made to
the SSBGA, we see that applying selection pressure has
only a small impact on the resulting best individuals. As
we found in the standard SSBGA, the value 1.68E-8 again
continues to surface, in much the same manner described
above. This seems to reaffirm our belief that it is the best
solution we can obtain given our granularity. When we
look at the best average best individual over one hundred
runs, our best instance again comes with a mutation rate of
0.06; however, the population size drops to 12 for the
SSBGA with added selection pressure described above.
This average best solution is slightly worse than the
standard SSBGA which we used, garnering an average of
only 0.0082.
In order to check the statistical difference of these
algorithms, we ran an ANOVA test on them, and found that
they are not statistically different, with a p value of 0.76,
and the Fcrit value much higher than the F value. This is
reaffirmed when we run a T-test on the data. When we run
a one-tailed T-test, we find that the returned value is 0.387.
The modified version produced far better results than the
unmodified version (see Figure 6 below), with a best
average of 0.001968. This is mostly due to the strength of
the midpoint crossover algorithm at attacking this particular
function, as demonstrated by the simple real-coded GA.
However, the best results were all found when the Gaussian
term was multiplied by a reasonably large β (0.01) rather
than by smaller values. The diversity introduced by the
Gaussian term was beneficial to the system.
Fitness versus Tournament Size (β = 0.01)
0.5
0.45
0.4
Pop. Size: 3
0.35
Pop. Size: 6
Pop. Size: 12
Fitness
Figure 4: Comparison of Average Best Fitness of
SSBGA with/without Selection Pressure
Figure 5. Average best fitness versus Tournament Size
for the unmodified SSRGA algorithm. Each line
represents a particular population size.
0.3
Pop Size: 25
0.25
Pop Size: 50
Pop Size: 100
0.2
Pop Size: 200
0.15
Pop. Sze: 400
0.1
Pop Size: 500
0.05
5.4 SSRGA
The unmodified version of SSRGA performed better (see
Figure 5), in general, with relatively large population sizes
and small q. Unsurprisingly, this mirrors the effects of large
q on simple real-coded genetic algorithms, and serves as
further confirmation that large tournament sizes restrict
diversity too much for the F6 function, which has many
local minima and maxima. Again, values between 1-5%
seemed to the best. In fact, the overall best average,
0.00629, was found with a population of 50 and a
tournament size of 1, or 2%.
0
1
2
5
10
20
50
Tournament Size
Figure 6. Average best fitness versus Tournament Size
for the modified SSRGA algorithm with β set to 0.01.
Each line represents a particular population size.
6. Why the Best Performed the Best
The values reported below (Table 1) are for the best
performing version of each algorithm, as described in
section 5. Clearly, the real-coded algorithms outperformed
the binary-coded versions in terms of average best
individual. Further, they have considerably lower standard
deviations, indicating that their better performance is also
more consistent than the binary-coded versions. The
reason that this occurs stems from fundamental problems
with the binary representation. First, the granularity of the
binary representation means that the best values for x and y
may not even be able to be represented in 20 bits.
Secondly, the operators performed on legitimately good
binary genes may mutate them in such a way as to produce
drastically different individuals once they are decoded. For
example, a single mutation at the front of the x binary
string results in a massive value change for x when decoded
back into a real value.
Given that the real-coded
algorithms have similar performance, it seems that the
representation plays a strong role in these algorithms.
Algorithm
Avg. Best Individual
Standard Dev.
5.79E-03
0.00587
2.04E-03
0.00262
7.29E-03
0.01613
1.98E-03
0.00264
Table 1: Average Best Individuals and Standard
Deviations
SGA
SRGA
SSBGA
SSRGA
Again, we will consider the algorithms to be of different
equivalence classes if the reported p-values exceed 0.05.
In general, the pairwise t-test comparisons (p-values
reported in Table 2) between each of the algorithms
revealed, not surprisingly, that the binary algorithms are
probably of the same equivalence class. Likewise, the realcoded algorithms are also of their own equivalence class.
Given their similar performances, these results are not
surprising.
t-test
SGA
SGA
SRGA
SSBGA
SSRGA
0
1.872E-08*
0.197
1.143E-08*
SRGA
1.872E-08*
0
9.422E-04*
0.436
SSBGA
0.197
9.422E-04*
0
6.497E-04*
SSRGA
1.143E-08*
0.436
6.497E-04*
0
Table 2: T-Test Matrix for the Best Performing
Algorithms (* indicates a p value < 0.05)
All of our Genetic Algorithms outperformed the
Evolutionary Programming Algorithms in [3].
The
modified
Continuous
Evolutionary
Programming
Algorithm (CEP) produced an average best individual with
0.007347 fitness. By contrast, the best performing GA
(SSRGA modified) produced an average best individual
with a 0.00198 fitness. We attribute this considerable
improvement to the Steady-State GA’s ability to balance
the goals of selection pressure and diversity. By always
replacing the worst individual, the algorithm encourages
diversity. By always keeping the best individuals in the
population, the SSGA places selection pressure on
promising regions.
7. Work Breakdown
We distributed one algorithm for each member of our
group. We engaged in several brainstorming meetings in
order to strive for overall improved algorithm performance.
Each person was responsible for their own implementation,
comments, data collection, data analysis, and algorithm
description. We established as a group a commentary
standard and write-up standard. Win worked on the SGA,
Kenan implemented the SRGA, Hurley designed the
SSBGA, and Philip worked with the SSRGA. The final
compilation of this paper was also done as a group.
References
[1]De Jong, K., Spears, W. “On The State of Evolutionary
Computation.” Proceedings of the Fifth International Conference
on Genetic Algorithms, 1993.
[2]Fogel, David B., Chellapilla, K.., “Revisiting Evolutionary
Programming.” AeroSense'98: Aerospace/Defense Sensing and
Controls, Orlando, Apr. 1998
[3] Britt, Winard R., Casey, Kenan., “High-Performance
Evolutionary Programming Algorithms for Minimization.”
Unpublished work, 2005.