ECE/CS/ME 539 class project
Yong Sun
Introduction to Artificial
Neural Networks and
Fuzzy Systems
ECE/CS/ME 539
Class Project
Instructor: Yu Hen Hu
Student: Yong Sun
-1-
ECE/CS/ME 539 class project
Yong Sun
Engine Operating Parameter Optimization using
Genetic Algorithm
IINTRODUCTION
With more-and-more stringent emission standards, future diesel engine technologies
will need to incorporate advanced combustion strategies with optimized engine
operating parameters for achieving low emissions while maintaining fuel economy
and power density. Genetic algorithms (GA) are being used by engine researchers to
optimize engine design and operating parameters to achieve these goals.
In this study, a micro-Genetic Algorithm (μGA) code was coupled with a 3D engine
Computational Fluid Dynamic (CFD) code KIVA to optimize six engine operating
parameters. A merit function was defined based on three engine performance
parameters to evaluate the merit of the combination of the parameters. Several penalty
functions were also defined based on the physical constraints of the engine. The
results were used as inputs for the development of a multi-layer perceptron (MLP)
configuration. The MLP network can be used to predict the engine performance based
on the engine operating parameters.
GENETIC ALGORITHM
Genetic algorithm overview
Genetic algorithms are global search techniques based on the mechanics of natural
selection which combine a “survival of the fittest” approach with some randomization
and/or mutation. Unlike local search techniques, which are typically highly dependent
on the starting point and have constraints such as solution continuity and
differentiability, as a global search method, GA place few constraints on the solution
domain and are thus much more robust for ill-behaved solution spaces. In addition,
GA tends to converge to a global optimum for multi-modal functions with many local
extrema. GA can also handle high dimensional problems and problems which require
high parameter resolution. These features make GA suitable for the optimization of
engine design and operating parameters.
Micro-genetic algorithm (μGA)
The micro-Genetic Algorithm (μGA) is a “small population” Genetic Algorithm (GA)
that operates on the principles of natural selection or “survival of the fittest” to evolve
the best potential solution (i.e., design) over a number of generations to the most-fit,
or optimal solution. In contrast to the more classical Simple Genetic Algorithm (SGA),
which requires a large number of individuals in each population (i.e., 30 – 200), the
-2-
ECE/CS/ME 539 class project
Yong Sun
μGA uses a μ.population of five individuals (Krishnakumar 1989). This is very
convenient for three-dimensional engine simulations, for example, which require a
large amount of computational time. The small population size allows for entire
generations to be run in parallel with five (or four, as will be discussed later) CPUs.
This feature significantly reduces the amount of elapsed time required to achieve the
most-fit solution. In addition, Krishnakumar (1989) and Senecal (2000) have shown
that the μGA requires a fewer number of total function evaluations compared to
SGAs for their test problems.
Gen4μGA code
The μGA used in the present study is based on the GA code of Carroll (1996) and
can be outlined as follows:
1. A μ-population of five designs is generated randomly.
2. The fitness of each design is determined and the fittest individual is carried to the
next generation (elitist strategy).
3. The parents of the remaining four individuals are determined using a tournament
selection strategy. In this strategy, designs are paired randomly and adjacent pairs
compete to become parents of the remaining four individuals in the following
generation (Krishnakumar 1989).
4. Convergence of the μ-population is checked. If the population is converged, go
to step 1, keeping the best individual and generating the other four randomly. If the
population has not converged, go to step 2.
Note that mutations are not applied in the μGA since enough diversity is introduced
after convergence of a μ-population.
A summary of the gen4 μGA code used in this study is presented in Fig. 2. In this
code, the user is allowed to initialize one individual as the present baseline design,
while randomly sampling the parameter space to determine the other four. The merit
function evaluation part can be considered as a “black box” operation, where 3D CFD
code KIVA is coupled with the gen4 code and the merit function of each individual is
evaluated.
The gen4 code can be run in either serial or parallel mode. In this study, the parallel
mode was used because the parallel option allows for much more rapid population
evaluation since each generation is run in “parallel” on four CPUs (although the μ
GA includes five individuals per generation, the present implementation only requires
four to be calculated because one of the individuals is the optimum individual from all
previous generations).
-3-
ECE/CS/ME 539 class project
Figure 1.
Yong Sun
Flow chart of the gen4 μGA optimization code.
APPROACH, RESULTS AND DISCUSSION
Coding
The gen4 code uses a binary string to represent a set of design parameters (Goldberg
1989). To extend the analogy with nature, the binary string is commonly referred to as
a chromosome, while its features are known as genes.
The precision of each of the design parameters included in a chromosome is based on
the desired range of the parameter (i.e., its minimum and maximum values) and how
many bits are used in its representation. Thus, the precision π of a parameter Xi is
given by (Homaifar et al. 1994):
-4-
ECE/CS/ME 539 class project
Yong Sun
where λi is the number of bits in its representation. Thus if m possible values of a
parameter are desired, its binary representation will include λi=ln(m)/ln(2) bits.
In this study, six engine operating parameters was optimized, intake valve closure
(IVC) timing, start of injection (SOI) timing, the spray angles of two pulses, the dwell
between the two pulses, and the fraction of fuel injected in each pulse. The six
parameters are denoted as A, B, C, D, E and F for simplicity. The range and
resolution of the six parameters are listed in Table 1.
Table 1 Ranges and resolutions of the parameters
Parameter
Baseline
range
Resolution
A
-143
-143~-83
4
B
0.44
0~1
0.067 (1/15)
C
0
0~75
5
D
60
48~141
3
E
125
48~141
3
F
0.5
0.0~1.0
0.143 (1/7)
# of possible values
16
16
16
32
32
8
λi
4
4
4
5
5
3
As a example, consider the coding of the parameter F with Xmax=1, Xmin=0 and λ
=3. Thus, according to the above equation, π =1/7. The resulting binary
representations for the eight numbers are presented in Table 2.
Table 2: Real number and binary representations for parameter F.
real number
binary representation
0
000
1/7
001
2/7
010
3/7
011
4/7
100
5/7
101
6/7
110
1
111
Function evaluation
The fitness, or objective function (merit function), is the “figure of merit” for each
individual chromosome, and thus determines its probability of taking part in
reproduction, In this study the merit function is defined based on three output
variables related to the engine performance, NOx emissions, Unburned Hydrocarbon
(HC) and Break Specific Fuel Consumption (BSFC). They are denoted as X, Y and Z
-5-
ECE/CS/ME 539 class project
Yong Sun
for simplicity. The merit function is defined as:
f1 
1000
X Y2  Z
2
From this definition, the lower X, Y and Z are, the higher merit function value an
individual has.
Several penalty functions have also been applied to the merit function based on the
peak in-cylinder pressure, exhaust temperature and pressure, maximum rate of
pressure rise, soot emissions misfire and the wall-film amount at Exhaust Valve
Opening (EVO). The purpose of applying these penalty functions is to avoid
unphysical cases which may have high merit function but is not practicable.
Reproduction (selectioin)
A reproduction operator combines the relative fitness of a generation’s chromosomes
with some randomness in order to determine parents of the following generation.
Tournament selection strategy is used in the present gen4 code. It can be summarized
in the following way:
1. The present generation is first “mixed up” such that the order of individuals is
completely random.
2. The fitness of individual 1 is compared with the fitness of individual 2. The
individual with the higher fitness is chosen as “parent 1.”
3. The fitness of individual 3 is compared with the fitness of individual 4. The
individual with the higher fitness is chosen as “parent 2.”
4. Parents 1 and 2 are used in the crossover operation.
The above process is repeated until the desired number of children is produced.
Another strategy used in conjunction with the tournament selection strategies in the
gen4 code, is the so-called “elitist approach” (Goldberg 1989). Elitism automatically
forms a child in the new generation with the exact qualities of the parent with the
highest merit. This approach guarantees that the highest fitness in a given generation
will be at least as high as in the previous generation.
Crossover
The crossover operation is the process of mating the selected strings in the
reproduction phase in order to form the individuals of a new generation. In the gen4
code, uniform crossover is used. In this method, a random number between zero and
unity is chosen at each position in the first parent string. If the random number is
larger than a crossover probability, which is set to 0.5, the bit is set to the second
parent’s value, otherwise it remains the same as the first parent’s value.
-6-
ECE/CS/ME 539 class project
Yong Sun
Convergence and restarting
While global optimality is not guaranteed with genetic algorithms, the population of
solutions will evolve over successive generations so that the fitness of the best
individual increases toward the global optimum. In the gen4 code, convergence is
defined as the progression towards chromosome uniformity. Thus, for the genetic
algorithm operators described above, a gene may be considered to be converged, if
95% of that particular gene in the entire population shares the same value. A
population is then converged when all of the genes are converged.
Since multi-dimensional engine simulations require a large amount of CPU time, a
method is needed to stop the search process. In the gen4 code, a maximum number of
generation is defined. The code also has a “restart” feature. If the search has not found
a satisfying optimum within the specified maximum number of generations, the
process can be restarted without any loss of information. If the maximum merit
function does not change for a sufficient number of generations, an optimal, or
near-optimal design is thought to be indicated and the program can be stopped by the
user.
Running the gen4 code
A summary of the source files of the gen4 code is listed in Tables 3. A baseline
individual and its merit is given in init.ga file and the ranges and resolutions of the six
parameters are given in input.ga file. This file also specifies the maximum generation
and the restart flag.
Table 3 Summary descriptions of gen4 source files
-7-
ECE/CS/ME 539 class project
Yong Sun
The gen4 code was modified to communicate with the KIVA code. Gen4 code
generate the six input parameters (A~F) of 4 citizens and sends them to the KIVA
code. The KIVA code calculates the outputs (X, Y and Z) and evaluates the merit of
each individual and sends them to the gen4 code. The gen4 code was also modified to
record the process of convergence. The information of all individuals in all
generations is recorded and written to a perform.ga file.
Results and discussion
In this study, 351 generation was run to get the ‘optimum’ case. Since there are
16×16×16×32×32×8=33,554,432 possible combinations of the six parameters and
running the KIVA code is also time consuming (2 hours for each generation), GA
may not able to find the global maximum. However, GA was able to find a satisfying
case with great improvement compared with the baseline case and the maximum merit
was observed to not change for more than 200 generations. Thus, it is considered that
an ‘optimum’ case was found. Figure 2 shows the history of the maximum merit.
140
Max. Merit Function
120
100
80
60
40
20
0
0
50
100
150
200
250
Generation
300
350
400
Figure 2 Maximum merit as a function of generation number
Table 4 Comparison between the baseline and optimum case
Input
Output
Merit
A
B
C
D
E
F
X
Y
Z
Merit
Baseline -143 0.440
0
60
125
0.5
11.7 139.7 1.386 -5.7
Optimum -139 0.267 15
90
141 0.857 2.3
1.2 1.333 125.4
-8-
ECE/CS/ME 539 class project
Yong Sun
Table 4 shows the comparison between the baseline and optimum case. It can be seen
that significant improvement has been achieved after the optimization using theμGA
algorithm.
MULTI-LAYER PERCRPTRON NETWORK
After running GA optimization, we have a set of data with inputs and outputs of all
the individuals. Next, a Multi-Layer Perceptron (MLP) network is setup to correlate
the six inputs and three outputs. The codes are modified from the bp code used in
class. The subroutines modified are attached.
30% of the data was selected randomly and reserved for testing. Only 70% of the data
was used for training. The inputs were scaled to [-5,5] and outputs [0.2, 0.8] by each
column. The MLP network has 2 layers and the hidden layer has 8 neurons. The
hidden layer uses hyperbolic tangent activation function and output layer uses
sigmoidal function. The parameters used in the code are as below:
Input layer dimension: 6
Output layer dimension: 3
Learning rate: 0.1
Momentum constant: 0.8
Nepoch: 2500
Epoch size K: 64
Nck: 50
The final training error is 0.0090 and the testing error is 0.0127.
The weight matrix of the hidden layer is:
-0.0928
0.9782
-0.7673
0.3791
1.5729
0.3189
-0.3103
-1.8411
1.8331
-0.5381
2.1315
1.1248
2.9580
-0.0267
-0.5908 -0.2677
4.4803
1.4442
-3.8054
-0.5408
-2.0629
0.7693
-2.3483
0.0138
1.4502
-0.2288
-1.4001 -0.7772
-2.1469
2.2675
2.4541
3.2225
5.9811
-4.5168
-0.1184
3.3509
0.7064
-0.5446
4.8072
0.1221
1.4702
1.1668
0.4692
0.0241
0.6601
-0.0094
0.0577
-2.6403
The weight matrix of the output layer is:
0.2365 0.3723 0.1336 0.3715 -0.4441 -0.1610 -0.1951 0.0984
1.3772 0.4357 -0.5307 0.0127 -0.8271 -0.3923 0.5913 0.1959
0.5020 0.0197 -0.1090 0.0477 -0.4352 -0.1570 0.1923 0.0680
-1.3212
0.0312
-0.2042
1.0066
0.1934
1.7161
-2.5117
-0.6212
-0.0809
0.4667
0.7139
Different number of MLP layers and number of neurons in each layer have been tried,
but the training and testing error are quite similar. Different learning rate and
momentum have also been tried, and the results are also not changed so much.
-9-
ECE/CS/ME 539 class project
Yong Sun
CONCLUSIONS
1. Genetic algorithm is a global optimization tool and can be used for engine design
and operating parameter optimization effectively and efficiently.
2. Optimization results can be used to train the MLP network, and the developed
network can be used to predict engine performance based on the inputs of the six
engine operating parameters.
REFERENCE
1. Carroll, D. L., “Chemical Laser Modeling with Genetic Algorithms,” AIAA
Journal, 34, 338, 1996.
2. Krishnakumar, K., “Micro-Genetic Algorithms for Stationary and Non-Stationary
Function Optimization,” SPIE 1196, Intelligent Control and Adaptive Systems,
1989.
3. Senecal, P.K., and Reitz, R.D., “Simultaneous Reduction of Engine Emissions and
Fuel Consumption Using Genetic Algorithms and Multi-Dimensional Spray and
Combustion Modeling,” SAE 2000-01-1890, 2000.
4. Senecal, P.K., “Numerical Optimization Using the Gen4 Micro-Genetic
Algorithm Code”, ERC Document, 2000
5. Goldberg, D. E., Genetic Algorithms in Search, Optimization and Machine
Learning, Addison-Wesley, Reading, MA, 1989.
6. Homaifar, A., Lai H. Y. and McCormick, E., “System Optimization of Turbofan
Engines using Genetic Algorithms,” Appl. Math. Modelling, 18, 1994.
- 10 -