Coevolution
Vlado Kvasnička
Department of Applied Informatics FIIT
Slovak Technical University
Bratislava
1
Coevolution
(1) A complex optimization problem is decomposed onto fewer “simpler”
subproblems
(2) Coevolution means an approach how to solve the above “simpler”
problems such that an obtained solution will be closely related (or even
identical) to a global solution of the undecomposed problem.
2
y  f  x
(1)
f : D  Rn  I  R
(2)
f
x
y
D
I
3
Global optimization problem
xopt  arg max f  x 
xD
(3)
In general, very difficult computational problem, discrete problems (e.g.
n
D  0,1 ) are usually NP hard
We propose three different versions of an evolutionary algorithm called the
“chemostat”
4
1. Standard version of Darwinian evolution
evolution
set of genes
population of genes
5
population P is randomly generated;
all replicators are evaluated by fitness;
for t:=1 to tmax do
begin x:=Oselect(P);
if random<prob(x) then
begin x’:=Omut(x);
x’ is evaluated by fitness;
x’’:=Oselect(P);
P:=(P-{x’’})+{x’};
end;
end;
6
2. Competitive version of Darwinian coevolution
x  xP  xQ
(4a)
y  f  x P  xQ 
(4b)
A reformulation of the optimization problem
xopt  xP ,opt  xQ ,opt
(5)
x P ,opt  arg max f  x P  xQ ,opt 
(6a)
xQ ,opt  arg max f  xP ,opt  xQ 
(6b)
x P DP
xQ DQ
7
We have two different populations
P   xP ,1 , xP ,2 ,..., xP ,A  and Q   xQ ,1 , xQ ,2 ,..., xQ ,B  (7)
Fitness of replicators
f P  xP   max f  x P  xQ 
(8a)
fQ  xQ   max f  xP  xQ 
(8b)
xQ U  Q 
xPU  P 
8
evolution
population of genes
set of genes
P
evolution
fitness evaluation
Q
set of memes
population of memes
9
populations P and Q are randomly generated;
all replicators are evaluated by fitness;
for t:=1 to tmax do
begin xP:=Oselect(P);
if random<prob(fitness(xP)) then
begin xP’:=Omut(xP);
xP’ is evaluated by fitness;
xP’’:=Oselect(P);
P:=(P-{xP’’})+{xP’};
end;
xQ:=Oselect(Q);
if random<prob(fitness(xQ)) then
begin xQ’:=Omut(xQ);
xQ’ is evaluated by fitness;
xQ’’:=Oselect(Q);
Q:=(Q-{xQ’’})+{xQ’};
end;
end;
10
xQ
xQ,opt
xP,opt
xP
11
3. Cooperative version of Darwinian coevolution
Function f(x) , for x  xP  xQ , is rewritten in an alternative form
f  x P  xQ   f  x P  x Q 
F  xP 
 f  x P  xQ   f  x P  x Q 

H x P ,xQ
(9)

12
where xQ is a randomly generated sub-argument.
A fitness of a replicator of x  xP  xQ has two parts:
1st part is specified by a function F that evaluates a subfitness of xP,
2nd part is specified by a function H that evaluates an interaction subfitness
between xP and xQ .
Optimization problem
x
P ,opt
, xQ ,opt   arg max  F  xP   H  xP , xQ  
 xP ,xQ 
(10)
13
set of genes
evolution
x
set of memes
population of m-genes
Population is composed of ordered couples
x
P
, xQ 
fitness  x   F  x P   H  x P , xQ 
(11)
14
population P is randomly generated;
all replicators are evaluated by fitness;
for t:=1 to tmax do
begin x:=Oselect(P);
if random<prob(fitness(x)) then
begin xP’:=Omut(xP);
xQ’:=Omut(xQ);
x’:=(xP’, xQ’);
x’ is evaluated by fitness;
x’’:=Oselect(P);
P:=(P-{x’’})+{x’};
end;
end;
Note: There is possible to suggest many heuristic approaches to accelerate the
proposed method
15
16
Cooperative version of Darwinian coevolution of genes and niches with
memes
 xG , x N , x M 
x  xG  xN  xM
(12a)
fitness  x   F  xG   H GN  xG , x N   H NM  x N , xM 
gene
niche
(12b)
meme
Meme affects gene only vicariously through an ecological niche
17
set of genes
evolution
set1 of niches
x
set2 of memes
population of mn-genes
18
population P is randomly generated;
all replicators are evaluated by fitness;
for t:=1 to tmax do
begin x:=Oselect(P);
if random<prob(fitness(x)) then
begin xG’:=Omut(xG);
xN’:=Omut(xN);
xM’:=Omut(xM);
x’:=(xG’,xN’,xM’);
x’ is evaluated by fitness;
x’’:=Oselect(P);
P:=(P-{x’’})+{x’};
end;
end;
19
Hill-Climbing Algorithm with Learniung
(HCwL)
This algorithm has been formulated initially by Baluja (cf. Kvasničku et al.). It
may be interpreted as an abstraction of genetic algorithm, where population
of replicator is substituted by a conception of memory vector
(1) A solution of binary optimization problem
xopt  arg maxn f  x 
x0,1
(2) Basic entity of the algorithm HCwL is a memory vector
n
w   w1 , w2 ,..., wn   0,1 ,its components 0  wi  1 specified a probability of
appearing of '1' in a given position . For instance, if wi=0(1), then xi=0(1), for
0<wi<1 a component xi is stochastically determined by
20
0  if random[0,1]  wi 
xi  
1  otherwice 
This stochastic generation of binary vectors with respect to memory vector w
is formally expressed by
x   x1 , x2 ,..., xn   R  w 
A neighborhood U(w) is composed of randomly generated binary vectors, but
their distribution is specified by components of the memory vector w
U  w    x  R  w 
If entries of the memory vector w are slightly above zero or below one, then a
"diameter" of U(w) is very small, all its elements are closely related to a
binary vector , which is equal to the memory vector w.
In the first diagram A, where entries of the probability vector
w are closely related to 0.5 (initial values), points are
scattered through whole search space S. In the second
diagram B, where the entries of w are substantially deviated
from 0.5, points are concentrated around a "center" state
represented by the heavy dot. The points with highest
objective-function values are denoted by encircled dots.
A
B
21
(2) The learning of memory vector w is introduced with respect to a best
solutions found in the neighborhood U(w). Let xopt be a best solutions that
have greatest functional value of f.
xopt  arg max f  x 
xU  w 
The memory vector w is modified-updated (learned) by the so-called Hebbian
rule (well-known learning rule widely used in artificial neural networks [3])
wnew  wold    xopt  wold 
where  is the learning coefficient (a small positive number, 0<<<1). The
learning rule has a very simple geometric interpretation
wold
wnew
xopt
22
Block diagram of hill-climbing algorithm with learning.
wini (1)
xopt= arg max f ( x) (2)
xU(w)
w: = w +  ( xopt - w) (3)
-
stop

(4)
xopt (5)
(1) Initial initiation of memory vector by wi = 1/2 , for i = 1, 2, ..., n.
(2) A looking for an optimal solution of (6) for neighborhood U(w)
(3) An updating of memory vector w with respect to an optimal solution achieved
in the previous step.
(4) A check of stoping condition,
(5) A stopping of algorithm, an output of vector xopt.
23
procedure HCwL(,cmax,tmax,xopt);
begin for i:=1 to 14 do wi:=0.5;
xopt:=R(w);
for cycle:=1 to cmax do
begin for t:=1 to tmax do
begin x:=R(w);
if f(x)>f(xopt)then xopt:=x;
end;
w : w    x opt  w 
end
end;
24
Elementy vektora pravdepodobnosti w
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0
50
100
150
Počet cyklov
200
250
Plots of memory vector entries wi with respect to the number of cycles. All entries
were initialized at 0.5, as evolution of the HCwL is advanced; they are turning
either to zero or one.
25
Competetive coevolutionary hill-climbing algorithm with
learning
Let
us

1
define
b

two
memory
vectors


wP  wP  ,...,wP   0,1
1
a
a
and
wQ  wQ ,...,wQ  0,1 . A neighborhood of these vectors is specified by
U P  wP    xP  R  wP 
b


andU Q  wQ   xQ  R  wQ  .
For
each
of
these
neighborhoods we solve the following two optimization problems
x P ,opt  arg max f P  x P 
x P U P  w P 
xQ ,opt  arg max fQ  xQ 
xQU Q  wQ 
and
f P  xP   max f  xP  xQ 
xQ U  Q 
fQ  xQ   max f  xP  xQ 
xPU  P 
26
After solving these “local” optimization problems we perform a process of
“learning” of memory vectors
w P ,new  w P ,old    x P ,opt  w P ,old 
wQ ,new  wQ ,old    xQ ,opt  wQ ,old 
This learning process is finished in the moment when all components of
probability vector are closely related either to 1 or 0. Then trhe resulting
probability vectors wP and wQ may be used for a construction of optimal
replicators xP and xQ , which correspond to solutions of optimal truth table
and a minimal set of propositional formulas, respectively.
27
Simple illustrative examples
First example: Let us have a function of two variables, f  u,v  , where u and v
are binary variables of the length n
f : 0,1  0,1  R
n
n
where we have to solve the following optimization problem
f  u,v 
 u,v opt  arg u ,vmax
 0 ,1
 n
This optimization task will be solved by a coevolutionary competitive hillclimbing with learning algorithm.
28
Let
us
have
two
memory
wu   w ,w ,...,w    0,1
1
u
vector
2
u
n
n
u
and
wv   wv1 ,wv2 ,...,wvn    0,1 , we assign two neighborhoods that are constructed
n
with respect to these memory vectors



Ru  α   1 , 2 ,..., n   0,1 , Rv  β   1 ,2 ,...,n   0,1
n
n

where
1  random  0,1  wui 
1  random  0 ,1  wvi 
, i  
i  
0  otherwise 
0  otherwise 
Within these neighborhoods we solve the following simple binary optimization
tasks



 opt  arg max max f   ,  , opt  arg max max f   , 
Ru
Rv
Rv
Ru

29
Finally, a process of learning of memory vectors is specified as follows
wu ,new  wu ,old     opt  wu ,old  , wv ,new  wv ,old     opt  wV ,old 
This process is repeated while all components of memory vectors are closely
related either to “1” or two “0”.
30
Second example: Let us have a function of one variable, f  x  , where x is a
binary variable of the length n
f : 0,1  R
n
where we have to solve the following optimization problem
xopt  arg maxn f  x 
x0 ,1
This optimization task will be solved by a coevolutionary competitive hillclimbing with learning algorithm.
31
Let us divide the binary vector x onto two subparts of the same length
x  u v
n2
u ,v  0,1
x
v
u
u
v
32
Let
us
have
two
memory
wv   wv1 ,wv2 ,...,wvn 2    0,1
n2
vector
wu   w ,w ,...,w
1
u
2
u
n2
u
  0,1
n2
and
, we assign two neighborhoods that are constructed
with respect to these memory vectors

Ru  α   1 , 2 ,..., n 2   0 ,1
n 2


, Rv  β   1 ,2 ,...,n 2   0,1
n 2

where
1  random  0,1  wui 
1  random  0 ,1  wvi 
, i  
i  
0  otherwise 
0  otherwise 
Within these neighborhoods we solve the following simple binary optimization
tasks



 opt  arg max max f      , opt  arg max max f     
Ru
Rv
Rv
Ru

33
Let
us
have
two
memory
wu   w ,w ,...,w    0,1
1
u
vector
2
u
n
n
u
and
wv   wv1 ,wv2 ,...,wvn    0,1 , we assign two neighborhoods that are constructed
n
with respect to these memory vectors



Ru  α   1 , 2 ,..., n   0,1 , Rv  β   1 ,2 ,...,n   0,1
n
n

where
1  random  0,1  wui 
1  random  0 ,1  wvi 
, i  
i  
0  otherwise 
0  otherwise 
Within these neighborhoods we solve the following simple binary optimization
tasks



 opt  arg max max f      , opt  arg max max f     
Ru
Rv
Rv
Ru

34
Finally, a process of learning of memory vectors is specified as follows
wu ,new  wu ,old     opt  wu ,old  , wv ,new  wv ,old     opt  wV ,old 
This process is repeated while all components of memory vectors are closely
related either to “1” or two “0”.
35
Conclusions
(1) Both competitive and cooperative coevolutionary algorithms may be
considered as a type of heuristics to accelerate an evolutionary solution
of complicated optimization problems.
(2) Different coevolutionary models of the above competitive and
cooperative approaches are possible.
(3) Their applicability is strongly restricted to problems with “almost
separable” variables.
(4) Different types of replications and interactions may be introduced.
36
A few years ago I received from father of our student this
email:
February 28, 2010
Dear Professor Kvasnicka:
Please excuse my son Joseph XY, Jr. being absent on February
28, 29, 30, and 31, he will be seriously sick.
With best cordial regards, father Joseph XY, Sr.
The End
37