Learning Guided Multiobjective Optimization
Aimin Zhou
East China Normal University, Shanghai, China
7/9, 2015
1
Outline
o Evolutionary Multiobjective Optimization
o A Self-Organizing Map based Approach
o Learning Guided Evolution – A Short Survey
o Conclusions & Future Remarks
2
LGMO - A.Zhou @ ECNU
7/9,2015
Outline
o Evolutionary Multiobjective Optimization
o A Self-Organizing Map based Approach
o Learning Guided Evolution – A Short Survey
o Conclusions & Future Remarks
3
LGMO - A.Zhou @ ECNU
7/9,2015
Multiobjective Optimization Problem
o MOP
min F(x) = ( f1 (x), f 2 (x),… , f m (x))
s.t x Î D
where
D : feasible region of decision variables.
fi : D ® R, objective function
F : D ® R m , objective vector function
F(D) = {F(x) | x Î D} : attainable objective set
o real-world applications
o scientific and engineering problems
4
D
x
F
f2
F (D )
z
z  ( z1,z2 )  F ( x )
f1
z1  f1 ( x ) , z2  f 2 ( x )
LGMO - A.Zhou @ ECNU
7/9,2015
Optimum of an MOP
o For a minimization problem
Let x, y Î D,
x dominates y (or F(x) dominates F(y))
f i (x) £ f i (y) for all i and f j (x) < f j (y) for at least one index j.
o dominate = be better than
o Examples:
x (z ) dominates x (z ).
3
3
1
1
(x ) z and x (z ) cannot be compared with each other.
2
2
1
x1
x2
x3
D
F
f2
z2
z1
1
F (D )
z3
f1
domination is a partial ordering
why MOPs are harder than single opt. problems
5
LGMO - A.Zhou @ ECNU
7/9,2015
Optimum of an MOP
o Pareto optimal solution
Pareto set (PS)
a solution cannot be dominated
by any other solutions.
o Pareto set (PS)
the set of all the Pareto optimal
solutions in decision variable
space.
f2
F
o Pareto front (PF)
PF=F(PS) (in objective space)
F (D )
Pareto front (PF)
f1
The PF is the southwest boundary of F(D).
6
LGMO - A.Zhou @ ECNU
7/9,2015
Task of MOEA
Very often, a decision maker wants
Pareto set (P)
A representative set of Pareto optimal solutions
(uniformly distributed along the PF or PS)
f2
F
F (D )
Task of most
Multiobjective Evolutionary Algorithms
(MOEAs)
Pareto front (PF)
f1
[1] A. Zhou, B. Qu, H. Li, S. Zhao, P. Suganthan, and Q. Zhang, Multiobjective evolutionary algorithms: A survey of the state of the art, Swarm and Evolutionary
Computation, 1(1): 32–49, 2011.
7
LGMO - A.Zhou @ ECNU
7/9,2015
Outline
o Evolutionary Multiobjective Optimization
o A Self-Organizing Map based Approach
o Learning Guided Evolution – A Short Survey
o Conclusions & Future Remarks
8
LGMO - A.Zhou @ ECNU
7/9,2015
Motivation
o Regularity of continuous MOPs:
Pareto set (PS)
Under certain conditions, the PS (PF) is a
(m-1)-dimensional piecewise continuous
manifold in decision (objective) space.
(m is the # of the objs.)
f2
F
o Problem-specific knowledge is
useful for algorithm design.
How can we deal with a continuous MOP
if its PS is (m-1)-D piecewise continuous
manifold?
F (D )
Pareto front (PF)
f1
[1] Q. Zhang, A. Zhou, and Y. Jin, RM-MEDA: a regularity model-based multiobjective estimation of distribution algorithm, IEEE Transactions on Evolutionary
Computation, 12(1):797-799, 2008.
9
LGMO - A.Zhou @ ECNU
7/9,2015
Motivation
o Classical reproduction operators
 scalar-objective optimization
x2
x2
x2
x*
x2
b
A
A
a
B
a
x1
B
x1
(b) 单点杂交
(d) 高斯模型采样
x2
x2
PS
x2
PS
a
B
A
b
x1
PS
a
b
PS
B
A
x1
(b) 单点杂交
x1
(c) 算术杂交
 multiobjective optimization
(a) 当前种群
b
x1
(a) 当前种群
x2
x*
x*
x*
x1
(c) 算术杂交
x1
(d) 高斯模型采样
[1] A. Zhou, Q. Zhang, and G. Zhang, Multiobjective evolutionary algorithm based on mixture Gaussian models, Journal of Software, 25(5):913-928, 2014.
10
LGMO - A.Zhou @ ECNU
7/9,2015
Basic Idea
o Algorithm framework
Population
Reproduction
operators
Competition
Replacement
Selection (Replacement): quite
a lot of works
Reproduction: our focus
New Solutions
11
LGMO - A.Zhou @ ECNU
7/9,2015
Self-Organizing Maps
Z1
D1
X3
R3
Neuron node
Latent space
u
D2
O
wun
1
i
Z2
n
x=(x1 , ... ,xi, ... ,xn)
Iutput space
X2
pu
Q
X1
o MOP
o SOM
 regularity property
 latent model
 mating registration
 similarity detection
[1] H. Zhang, A. Zhou, S. Song, Q. Zhang, X. Gao, and J. Zhang, A self-organizing multiobjective evolutionary algorithm, 2015 (submit).
12
LGMO - A.Zhou @ ECNU
7/9,2015
SOM Assisted MOEA
o Characteristics:
 Call SOM and MOEA main steps iteratively
 detect the population structure in an incremental manner
 save computational cost
 Generate offspring by neighboring parents
13
LGMO - A.Zhou @ ECNU
7/9,2015
Other Issues
o Reproduction operator:
 Differential Evolution (DE)
 Polynominal Mutation
14
o Selection operator:
 Nondominated sorting
scheme
LGMO - A.Zhou @ ECNU
7/9,2015
Experimental Results
o On irregular problems
 GLT test suite
 CellDE, MOEA/D-DE, RM-MEDA, NSGA-II, SMS-EMOA,SOM-NSGA-II
 IGD,HV metrics
15
LGMO - A.Zhou @ ECNU
7/9,2015
Experimental Results
o Run time performance
 Converges faster in most cases.
16
LGMO - A.Zhou @ ECNU
7/9,2015
Experimental Results
o Visual performance
17
LGMO - A.Zhou @ ECNU
7/9,2015
Experimental Results
o Visual performance
18
LGMO - A.Zhou @ ECNU
7/9,2015
Outline
o Evolutionary Multiobjective Optimization
o A Self-Organizing Map based Approach
o Learning Guided Evolution – A Short Survey
o Conclusions & Future Remarks
19
LGMO - A.Zhou @ ECNU
7/9,2015
Basic Questions
Learning + Evolutionary Optimization
o What?
 Learning Guided Evolution (LGE) is a kind of evolutionary algorithms that
utilize statistical and machine learning techniques to guide the search.
o Why?
 Priori & learnt problem specific knowledge to guide the search, and
thus to improve search performance.
o How?
 data organization
 pattern recognition
 pattern usage
20




initialization
reproduction
selection
stop condition
LGMO - A.Zhou @ ECNU
7/9,2015
Related Work
o Adaptive Evolution
 Parameter tuning
 Operator selection
mine populations
 Stopping condition
o Estimation of Distribution Algorithm (EDA)
 Ant Colony Optimization (ACO)
 Cross-entropy method (CE)
model & sample
populations
 Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
o Surrogate Assist Evolutionary Algorithm (SAEA)
replace evaluation
21
LGMO - A.Zhou @ ECNU
7/9,2015
Taxonomy
o Angle of Machine Learning
Regression based EAs
Supervised
Evolution
Classification based EAs
Manifold learning based EAs
Learning Guided
Evolution
Unsupervised
Evolution
Clustering based EAs
Density estimation based EAs
Semisupervised
Evolution
22
LGMO - A.Zhou @ ECNU
7/9,2015
A Short Survey of Our Recent Work
o Regression based approaches
 Surrogate assisted minimax optimization
 Time series prediction for dynamic
multiobjective optimization
 Cheap surrogate model
PS estimation
=
PS manifold learning
+
center point prediction
[1] A. Zhou, and Q. Zhang, A surrogate-assisted evolutionary algorithm for minimax optimization, in IEEE Congress on Evolutionary Computation (CEC 2010),
Barcelona: IEEE Press, 2010, pp.1-7.
[2] A. Zhou, Y. Jin, and Q. Zhang, A population prediction strategy for evolutionary dynamic multiobjective optimization, IEEE Transactions on Cybernetics,
44(1):40-53,2014.
[3] A. Zhou, J. Sun, and Q. Zhang, An estimation of distribution algorithm with cheap and expensive local search, IEEE Transactions on Evolutionary
Computation, 2015. (accepted)
23
LGMO - A.Zhou @ ECNU
7/9,2015
A Short Survey of Our Recent Work
o Classification based approaches
 Classification based preselection
 Classification based selection
selection = classification
[1] J. Zhang, A. Zhou, and G. Zhang, A Classification and Pareto domination based multiobjective evolutionary algorithm, in Proceedings of IEEE Congress on
Evolutionary Computation (CEC 2015), 2015, pp.1-8.
[2] J. Zhang, A. Zhou, and G. Zhang, A classification based preselection for evolutionary algorithms, 2015 (submit).
24
LGMO - A.Zhou @ ECNU
7/9,2015
A Short Survey of Our Recent Work
o Manifold learning based approaches
 Regularity model based multiobjective estimation of distribution
algorithm (RM-MEDA)
1
population
2
C1
3
x1
C2
x2
C3
x
[1] Q. Zhang, A. Zhou, and Y. Jin, RM-MEDA: a regularity model-based multiobjective estimation of distribution algorithm, IEEE Transactions on Evolutionary
Computation, 12(1):797-799, 2008.
[2] A. Zhou, Q. Zhang, and Y. Jin, Approximating the set of Pareto-optimal solutions in both the decision and objective spaces by an estimation of distribution
algorithm, IEEE Transactions on Evolutionary Computation, 13(5):1167-1189, 2009.
25
LGMO - A.Zhou @ ECNU
7/9,2015
A Short Survey of Our Recent Work
o Clustering based approaches
 Clustering based mating selection
 Self-organizing multiobjective evolutionary algorithm
[1] H. Zhang, S. Song, and A. Zhou, A clustering based multiobjective evolutionary algorithm, in IEEE Congress on Evolutionary Computation (CEC 2014), 2014.
[2] H. Zhang, A. Zhou, S. Song, X. Gao, and J. Zhang, A self-organising multiobjective evolutionary algorithm, 2015. (submit)
26
LGMO - A.Zhou @ ECNU
7/9,2015
A Short Survey of Our Recent Work
o Density estimation based approaches

Mixture Gaussian model
 model base reproduction
 model re-use

Non-parametric density estimation
 model based pre-selection
 multi-operator search
 locally weighted model
fitness estimation
by cheap models
[1] L. Zhou, A. Zhou, G. Zhang, C. Shi, An estimation of distribution algorithm based on nonparametric density estimation, in IEEE Congress on Evolutionary
Computation (CEC 2011), New Orleans: IEEE Press, 2011, pp.1597-1604.
[2] A. Zhou, Q. Zhang, and G. Zhang, A multiobjective evolutionary algorithm based on decomposition and probability model, in IEEE Congress of Evolutionary
Computation (CEC 2012), Brisbane: IEEE Press, 2012, pp.1-8.
[3] A. Zhou, Q. Zhang, and G. Zhang, A multiobjective evolutionary algorithm based on mixture Gaussian models, Journal of Software, 25(5):913−928, 2014.
[4] Q. Liao, A. Zhou, and G. Zhang, A locally weighted metamodel for pre-selection in evolutionary optimization, in The IEEE Congress on Evolutionary
Computation (CEC 2014), 2014.
[5] A. Zhou, Y. Zhang, G. Zhang, and W. Gong, On neighborhood exploration and subproblem exploitation in decomposition based multiobjective evolutionary
algorithms, in Proceedings of IEEE Congress on Evolutionary Computation (CEC 2015), 2015, pp.1-8.
[6] W. Gong, A. Zhou, and Z. Cai, A multi-operator search strategy based on cheap surrogate models for evolutionary optimization, IEEE Transactions on
Evolutionary Computation, 2015. (accepted)
27
LGMO - A.Zhou @ ECNU
7/9,2015
A Short Survey of Our Recent Work
o Adaptive approaches
 Adaptive replacement strategy in MOEA/D
cost
 Adaptive resource allocation in MOEA/D
subproblem index
resource control
[1] Z. Wang, Q. Zhang, A. Zhou, M. Gong, and L. Jiao, Adaptive replacement strategies for MOEA/D, IEEE Transactions on Cybernetics, 2015. (accepted)
[2] A. Zhou, and Q. Zhang, Are all the subproblems equally important? Resource allocation in decomposition based multiobjective evolutionary algorithms,
IEEE Transactions on Evolutionary Computation, 2015. (accepted)
28
LGMO - A.Zhou @ ECNU
7/9,2015
Outline
o Evolutionary Multiobjective Optimization
o A Self-Organizing Map based Approach
o Learning Guided Evolution – A Short Survey
o Conclusions & Future Remarks
29
LGMO - A.Zhou @ ECNU
7/9,2015
Conclusions & Future Remarks
Cost
Alg. Cost
Problem Cost
o
Random Search: Alg. Cost is LOW, Problem Cost is HIGH.
o
Mathematical Programming: Alg. Cost is HIGH, Problem Cost is LOW.
o
Evolutionary Optimization: BETWEEN the above two approaches.
o
Learning Guided Evolutionary Optimization
o It Is promising to balance the two costs.
o There is no systematic study yet.
o Which knowledge to detect?
o Which learning method to use?
o How to combine learning methods and evolutionary algorithms?
30
LGMO - A.Zhou @ ECNU
7/9,2015
Thanks!
Dr. Aimin Zhou, East China Normal University
amzhou@cs.ecnu.edu.cn,
http://www.cs.ecnu.edu.cn/~amzhou
http://faculty.ecnu.edu.cn/s/1949/t/22630/main.jspy
31
LGMO - A.Zhou @ ECNU
7/9,2015