Sisteme de programe
pentru timp real
Universitatea “Politehnica” din Bucuresti
2005-2006
Adina Magda Florea
http://turing.cs.pub.ro/sptr_06
Curs Nr. 7 si 8
Algoritmi genetici
•
•
•
•
•
•
•
•
•
•
Introducere
Schema de baza
Functionare
Exemplu
Selectie
Recombinare
Mutatie
GA pt TSP
Implementare paralela GA
Co-evolutie
2
1. Introducere



GA – cautare euristica adaptiva
Concept de baza
GA - optimizare
Utili cand
• Spatiu de cautare mare, complex
• Cunostinte domeniu neformalizate, euristice
• Fara model matematic
• Metodele traditionale prea costisitoare
3
Introducere - cont
Directii
 GA - USA, J. H. Holland (1975)
 Algoritmi evolutionisti – Germania, I. Rechenberg, H.-P.
Schwefel (1975-1980)
 Programare evolutionista (1960-1980)
• Optimizare
• Programare automata: evolueaza programe sau proiecteaza
automate celulare
•
•
•
•
•
Invatare automata
Modeel economice
Modele ecologice
Genetica populatiei
Modele ale sistemelor sociale
4
Introductere - cont
• GA – opereaza asupra unei populatii de solutii
potentiale – aplica principiul supravietuirii pe baza de
adaptare (fitness)
• Fiecare generatie – o noua aproximatie
• Evolutie a unei populatii de indivizi mai bine adaptati
mediului
• Modeleaza procese naturale: selectie, recombinare,
mutatie, migratie, localizare
• Populatie de indivizi – cautare paralela
5
2. Schema de baza
Problem
representation
Generate
initial
population
of individuals
(genes)
start
Fitness function
Evaluate
objective
function
Are optimization
criteria met?
no
Generate
new
population
yes
Selection
best
individuals
Crossover/Mate
result
offsprings
Mutation
generations
6
2.1 Rezolvare probleme cu GA
7
Rezolvare probleme cu GA - cont
Rezultate mai bune - subpopulatii
Fiecare populatie evolueaza separat
Indivizi sunt schimbati
Multipopulation GAs
8
3. Functionare
Populatie initiala
• Reprezentare – gena – individ - sir de biti
• Populatia initiala (genele) create aleator
• Lungimea individului
Selectie (1)
• Selectie – extragerea unui subset de gene dintr-o
populatie exsitenta in fct de o definitie a calitatii
• Determina indivizii selectati pt recombinare si cati
descendenti (offsprings) produce fiecare individ
9
3.1 Selectie
Atribuire Fitness :
• Atribuire proportionala (proportional fitness assignment)
• Atribuire bazata pe rang (rank-based fitness assignment)
Selectie: parintii sunt selectati in fct de fitness pe baza unuia
•
•
•
•
•
din algoritmii:
roulette-wheel selection (selectie ruleta)
stochastic universal sampling (esantionare universala)
local selection (selectie locala)
truncation selection (selectie trunchiata)
tournament selection (selectie turneu)
10
Selectie - cont
Proportional fitness assignment
 Each gene has a related fitness
 The mean-fitness of the population is calculated.
 Every individual will be copied as often to the new
population, the better it fitness is, compared to the average
fitness.
 e.g.: the average fitness is 5.76, the fitness of one individuum
is 20.21. This individuum will be copied 3 times.
 All genes with a fitness at the average and below will be
removed.
 New population – may be smaller than the old one.
 New population will be filled up with randomly chosen
individuals from the old population to the size of the old one.
11
3.2 Reinsertion
 Offspring – if less offsprings are produced than the
size of the original population then the offsprings
have to be reinserted into the new population.
 Not all offsprings are to be used at each generation
 More offsprings are generated than needed reinsertion
 The selection algorithm used determines the
reinsertion scheme:
• global reinsertion for all population based on the
selection algorithm (roulette-wheel selection,
stochastic universal sampling, truncation selection),
• local reinsertion for local selection.
12
3.3 Crossover/Recombination
• Recombination produces new individuals in
combining the information contained in the parents
(parents - mating population).
• Several recombination schemes exist
• The b_nX crossover - random mating with a
defined probability
Parallel to the Crossing Over in genetics
1. PM percent of the individuals of the new population
will be selected randomly and mated in pairs.
2. A crossover point will be chosen for each pair
3. The information after the crossover point will be
exchanged between the two individuals of each pair.
13
Crossover
14
3.4 Mutation
• Every offspring undergoes mutation.
• Offspring variables are mutated by small
perturbations (size of the mutation step), with
low probability.
• The representation of the variables determines
the used algorithm.
• Mutation – exploration vs exploitation
• One simple mutation scheme:
– Each bit in every gene has a defined
probability P to get inverted
15
Mutation
16
The effect of mutation is in some way antagonist to selection
17
4 An example
Compute the maximum of a function
• f(x1, x2, ... xn)
• The task is to calculate x1, x2, ... xn for which
the function is maximum
• Using GA´s can be a good solution.
18
Example- Representation
1.
2.
3.
4.
•
•
Scale the variables to integer variables by multiplying them
with 10n, where n is the the desired precision
New Variable = integer(Old Variable ×10 n)
Represent the variables (parameters) in binary form
Connect all binary representations of these variables to form
an individual of the population
If the sign is necessary two ways are possible:
Add the lowest allowed value to each variable and transform
the variables to positive ones
Reserve one bit per variable for the sign
Binary representation or Gray-code representation
19
Example- Representation
20
Example- Computation
1.
2.
3.
4.
5.
Make random initial population
Perform selection
Perform crossover
Perform mutation
Transform the bitstring of each individual back to
the model-variables: x1, x2, ... xn
6. Test the quality of fit for each individual, e.g., the
f(x1, x2, ... xn)
7. Check if the quality of the best individual is good
enough (not significantly improved over last steps)
8. If yes then stop iterations else go to 2
21
5. Selection




The first step is fitness assignment.
Each individual in the selection pool receives a
fitness value
Reproduction probability – depends the own
objective value and the objective value of all other
individuals in the selection pool.
Rp is used for the actual selection step afterwards.
22
Terms
• selective pressure: probability of the best individual being
selected compared to the average probability of selection of all
individuals
• bias: absolute difference between an individual's normalized
fitness and its expected probability of reproduction
• spread: range of possible values for the number of offspring
of an individual
• loss of diversity: proportion of individuals of a population that
is not selected during the selection phase
• selection intensity: expected average fitness value of the
population after applying a selection method to the normalized
Gaussian distribution
• selection variance: expected variance of the fitness
distribution of the population after applying a selection method
to the normalized Gaussian distribution
23
5.1 Rank-based fitness assignment
 The population is sorted according to the objective
values (fitness).
 The fitness assigned to each individual depends only
on its position in the individuals rank and not on the
actual objective value.
24
Rank-based fitness assignment - cont
• Nind - the number of individuals in the population
• Pos - the position of an individual in this population
(least fit individual has Pos=1, the fittest individual
Pos=Nind)
• SP - the selective pressure (probability of the best
individual being selected compared to the average probability of
selection of all individuals)
Linear ranking
Fitness(Pos) = 2 - SP + 2*(SP - 1)*(Pos - 1) / (Nind - 1)
• Linear ranking allows values of SP in [1.0, 2.0].
• The probability of each individual being selected for
mating is its fitness normalized by the total fitness of
the population.
25
Rank-based fitness assignment - cont
Non-linear ranking:
Fitness(Pos) =
Nind*X (Pos - 1) /  i = 1,Nind (X(i - 1))
• X is computed as the root of the polynomial:
0 = (SP - 1)*X (Nind - 1) + SP*X (Nind - 2) + ... + SP*X + SP
• Non-linear ranking allows values of SP in
[1.0, Nind - 2.0]
• The use of non-linear ranking permits higher selective
pressures than the linear ranking method.
• The probability of each individual being selected for
mating is its fitness normalized by the total fitness of
the population.
26
Rank-based fitness assignment - cont
Fitness assignment for linear and non-linear ranking
27
Rank-based fitness assignment - cont
 Rank-based fitness assignment overcomes the scaling problems
of the proportional fitness assignment = Stagnation in the case
where the selective pressure is too small or premature
convergence where selection has caused the search to narrow
down too quickly.
 The reproductive range is limited, so that no individuals
generate an excessive number of offsprings.
 Ranking introduces a uniform scaling across the population and
provides a simple and effective way of controlling selective
pressure.
 Rank-based fitness assignment behaves in a more robust
manner than proportional fitness assignment.
28
Rank-based fitness assignment - cont
Properties of linear ranking
Selection intensity: SelIntRank(SP) = (SP-1)*(1/sqrt()).
Loss of diversity: LossDivRank(SP) = (SP-1)/4.
Selection variance: SelVarRank(SP) = 1-((SP-1)2/ ) = 1-SelIntRank(SP)2.
29
5.2 Roulette wheel selection
Roulette-wheel selection, also called stochastic sampling with replacement
11 individuals, linear ranking and selective pressure of 2
Number
of individual
1
2
3
4
5
6
7
8
9
10
11
Fitness
value
2.0
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
Selection
probability
0.18
0.16
0.15
0.13
0.11
0.09
0.07
0.06
0.03
0.02
0.0
sample of 6 random numbers (uniformly distributed between 0.0 and 1.0):
• 0.81, 0.32, 0.96, 0.01, 0.65, 0.42.
30
Roulette wheel selection - cont
• After selection the mating population consists of the
individuals:
1, 2, 3, 5, 6, 9.
• The roulette-wheel selection algorithm provides a zero bias but
does not guarantee minimum spread
31
5.3 Stochastic universal sampling
• Equally spaced pointers are placed over the line as many as there are
individuals to be selected.
• NPointer - the number of individuals to be selected
• The distance between the pointers are 1/Npointer
• The position of the first pointer is given by a randomly generated
number in the range [0, 1/NPointer].
• For 6 individuals to be selected, the distance between the pointers is
1/6=0.167.
• sample of 1 random number in the range [0, 0.167]: 0.1.
32
Stochastic universal sampling - cont
• After selection the mating population consists of the
individuals:
1, 2, 3, 4, 6, 8.
• Stochastic universal sampling ensures a selection of offspring
which is closer to what is deserved than roulette wheel selection
33
5.4 Local selection
• Every individual resides inside a constrained environment
called the local neighbourhood
• The neighbourhood is defined by the structure in which the
population is distributed.
• The neighbourhood can be seen as the group of potential mating
partners.
• Linear neighbourhood: full and half ring
34
Local selection - cont
The structure of the neighbourhood can be:
• Linear: full ring, half ring
• Two-dimensional:
– full cross, half cross
– full star, half star
• Three-dimensional and more complex with any combination of the above
structures.
35
Local selection - cont
• The distance between possible neighbours together with the structure
determines the size of the neighbourhood.


The first step is the selection of the first half of the mating
population uniform at random (or using one of the other
mentioned selection algorithms, for example, stochastic
universal sampling or truncation selection).
Then a local neighbourhood is defined for every selected
individual. Inside this neighbourhood the mating partner is
selected (best, fitness proportional, or uniform at random).
36
Local selection - cont






Between individuals of a population an 'isolation by distance'
exists.
The smaller the neighbourhood, the bigger the isolation
distance.
However, because of overlapping neighbourhoods, propagation
of new variants takes place.
This assures the exchange of information between all
individuals.
The size of the neighbourhood determines the speed of
propagation of information between the individuals of a
population, thus deciding between rapid propagation or
maintenance of a high diversity/variability in the population.
A higher variability is often desired, thus preventing problems
such as premature convergence to a local minimum.
37
5.5 Tournament selection





A number Tour of individuals is chosen randomly from the
population and the best individual from this group is selected as
parent.
This process is repeated as often as individuals to choose.
The parameter for tournament selection is the tournament size
Tour.
Tour takes values ranging from 2 - Nind (number of individuals
in population).
Relation between tournament size and selection intensity
tournament size
1
2
3
5
10
30
selection intensity
0
0.56
0.85
1.15
1.53
2.04
38
Tournament selection - cont



Selection intensity:
SelIntTour(Tour) = sqrt(2*(log(Tour)-log(sqrt(4.14*log(Tour)))))
Loss of diversity:
LossDivTour(Tour) = Tour -(1/(Tour-1)) - Tour -(Tour/(Tour-1))
(About 50% of the population are lost at tournament size Tour=5).
Selection variance:
SelVarTour(Tour) = 1- 0.096*log(1+7.11*(Tour-1)), SelVarTour(2) = 1-1/
39
6. Crossover / recombination
• Recombination produces new individuals
in combining the information contained in
the parents
40
6.1 Binary valued recombination (crossover)

One crossover position k is selected uniformly at random and
the corrsponding parts (or variables) are exchanged between the
individuals about this point  two new offsprings are
produced.
41
Binary valued recombination - cont



Consider the following two individuals with 11 bits each:
individual 1:
01110011010
individual 2:
10101100101




crossover position is 5
After crossover the new individuals are created:
offspring 1:
0 1 1 1 0| 1 0 0 1 0 1
offspring 2:
1 0 1 0 1| 0 1 1 0 1 0
42
2.2 Multi-point crossover

m crossover positions ki
 individual 1:
01110011010
 individual 2:
10101100101
 cross pos. (m=3) 2 6 10
 offspring 1:
0 1| 1 0 1 1| 0 1 1 1| 1
 offspring 2:
1 0| 1 1 0 0| 0 0 1 0| 0
43
2.3 Uniform crossover







Uniform crossover generalizes single and multi-point crossover
to make every locus a potential crossover point.
A crossover mask, the same length as the individual structure,
is created at random and the parity of the bits in the mask
indicate which parent will supply the offspring with which bits.
individual 1:
01110011010
individual 2:
10101100101
mask 1: 0 1 1 0 0 0 1 1 0 1 0
mask 2: 1 0 0 1 1 1 0 0 1 0 1
(inverse of mask 1)
offspring 1:
11101111111
offspring 2:
00110000000
Spears and De Jong (1991) have poposed that uniform
crossover may be parameterised by applying a probability
to the swapping of bits.
44
2.4 Real valued recombination
2.4.1 Discrete recombination
 Performs an exchange of variable values between the
individuals.
 individual 1:
12 25 5
 individual 2:
123 4 34
 For each variable the parent who contributes its variable
to the offspring is chosen randomly with equal probability.
 sample 1: 2 2 1
 sample 2: 1 2 1
 After recombination the new individuals are created:
 offspring 1:
123 4 5
 offspring 2:
12 4 5
45
Real valued recombination - cont
Discrete recombination - cont
 Possible positions of the offspring after discrete
recombination

Discrete recombination can be used with any kind of
variables (binary, real or symbols).
46
Real valued recombination - cont
2.4.2 Intermediate recombination
 A method only applicable to real variables (and not binary
variables).
 The variable values of the offsprings are chosen
somewhere around and between the variable values of
the parents.
 Offspring are produced according to the rule:
offspring = parent 1 + Alpha (parent 2 - parent 1)
where Alpha is a scaling factor chosen uniformly at
random over an interval [-d, 1 + d].
 In intermediate recombination d = 0, for extended
intermediate recombination d > 0.
 A good choice is d = 0.25.
 Each variable in the offspring is the result of combining
the variables according to the above expression with a
47
Real valued recombination - cont
Intermediate recombination - cont
 individual 1: 12 25 5
 individual 2: 123 4 34
Alpha are:
 sample 1: 0.5 1.1 -0.1
 sample 2: 0.1 0.8 0.5
 The new individuals are calculated


(offspring = parent 1 + Alpha (parent 2 - parent 1)
offspring 1:
67.5 1.9 2.1
offspring 2:
23.1 8.2 19.5
48
Real valued recombination - cont
Intermediate recombination - cont
 Area for variable value of offspring compared to
parents in intermediate recombination
49
Real valued recombination - cont

Intermediate recombination is capable of producing any point
within a hypercube slightly larger than that defined by the
parents

Possible area of the offspring after intermediate recombination
50
Real valued recombination - cont
2.4.3 Line recombination
 Line recombination is similar to intermediate recombination,
except that only one value of Alpha for all variables is used.
 individual 1:
12 25 5
 individual 2:
123 4 34
Alpha are:
 sample 1: 0.5
 sample 2: 0.1
The new individuals are calculated as:
 offspring 1:
67.5 14.5 19.5

offspring 2:
23.1 22.9 7.9
51
Real valued recombination - cont
Line recombination - cont
 Line recombination can generate any point on the line defined
by the parents
52
7. Mutation
• After recombination offsprings undergo
mutation.
• Offspring bits or variables are mutated by
inversion or the addition of small random values
(size of the mutation step), with low probability.
• The probability of mutating a variable is set to be
inversely proportional to the number of variables
(dimensions).
• The more dimensions one individual has as
smaller is the mutation probability.
53
7.1 Binary mutation




For binary valued individuals mutation means flipping of
variable values.
For every individual the variable value to change is chosen
uniform at random.
A binary mutation for an individual with 11 variables, variable 4
is mutated.
Individual before and after binary mutation
before mutation
0
1
1
1
0
0
1
1
0
1
0
after mutation
0
1
1
0
0
0
1
1
0
1
0
54
7.2 Real valued mutation

Effect
of mutation


The size of the mutation step is usually difficult to choose. The optimal step
size depends on the problem considered and may even vary during the
optimization process. Small steps are often successful, but sometimes bigger
steps are quicker.
A mutation operator (the mutation operator of the Breeder Genetic
Algorithm) :
mutated variable = variable ± range·delta (+ or - with equal probability)
range = 0.5*domain of variable
delta = sum(a(i) 2-i), a(i) = 1 with probability 1/m, else a(i) = 0; m = 20.
55
Real valued mutation - cont



This mutation algorithm is able to generate most points in the
hypercube defined by the variables of the individual and range
of the mutation.
With m=20, the mutation algorithm is able to locate the
optimum up to a precision of (range*2-19)
Probability and size of mutation steps (compared to range)
56
8. Using GAs to solve:
0/1 Knapsack Problem
TSP
57
8.1 0/1 Knapsack Problem
Given a set of items, each with a weight (w(i)) and a value/profit
(p(i)), determine the number of each item to include in a
collection so that the total weight is less than some given weight
(W) and the total value is as large as possible.
The 0/1 knapsack problem restricts the number of each items to zero
or one.
• Maximize sum(x(i)*p(i))
• Subject to sum(x(i)*w(i)) <= W
• x(i) = 0 or 1
• Multiobjective optimization problem: maximize the profit and minimize the
weight simultaneously. There is a trade-off between profit and weight. There is
not a single optimal solutions but rather a set of optimal trade-offs which
consists of all solutions that cannot be improved in one criterion without
degradation in another.
58
0/1 Knapsack Problem
Binary string - length - number of items.
Each item is assigned a position within the binary
string, 0 - item is not in the solution, 1 - the
solution contains the corresponding item.
An individual's fitness is equal to the number of
individuals in the population that dominate it.
Genetic operators:
- tournament selection
- one-point crossover
- bit-flip mutation.
59
Population size 100
Tour 2
Mutation rate: 0.01
weight (w) 3 3 3 3 3 4 4 4 7 7 8 8 9
Value (p) 4 4 4 4 4 5 5 5 10 10 11 11 13
•
•
•
•
•
•
•
•
•
•
Generation 74:
No Fit
00 24
01 23
02 23
03 23
04 23
05 23
06 23
07 23
•
Best=24.0, Avg=19.1, Worst=-18.0
Chromosome
0001000011000
0110100000100
0010100101000
0110010001000
0110000110000
0101010001000
1010100000100
0110010001000
08 23
09 23
10 22
11 22
12 22
13 22
14 22
15 22
16 15
17 12
18 10
19 -18
0110010010000
1010101011100
0000010011000
1010101100000
0110101100000
1010101100000
1010101100000
1010101011100
0000010001000
0110010011000
0010100101010
0110011110011
60
8.2 TSP - Problem representation
Evaluation function
• The evaluation function for the N cities is the sum of the
Euclidean distances
N
Fitness   ( xi  xi 1 ) 2  ( yi  yi 1 ) 2
i 1
Representation
• a path representation where the cities are listed in the order in
which they are visited
3 0 1 4 2 5
0 5 1 4 2 3
5 1 0 3 4 2
61
TSP Genetic operators
Crossover
• Traditional crossover is not suitable for TSPs
• Before crossover
1 2 3 4 5 0 (parent 1)
2 0 5 3 1 4 (parent 2)
• After crossover
1 2 3 3 1 4 (child 1)
2 0 5 4 5 0 (child 2)
• Greedy Crossover which was invented by Grefenstette in 1985
62
TSP Genetic operators - cont
Two parents 1 2 3 4 5 0 and 4 1 3 2 0 5
• To generate a child using the second parent as the template, we select city 4
(the first city of our template) as the first city of the child. 4 x x x x x
• Then we find the edges after city 4 in both parents: (4, 5) and (4, 1). If the
distance between city 4 and city 1 is shorter, we select city 1 as the next city of
child 2. 4 1 x x x x
• Then we find the edges after city 1 (1, 2) and (1, 3). If the distance between
city 1 and city 2 is shorter, we select city 2 as the next city. 4 1 2 x x x
• Then we find the edges after city 2 (2, 3) and (2, 0). If the distance between
city 2 and city 0 is shorter, we select city 0. 4 1 2 0 x x
• The edges after city 0 are (0, 1) and (0, 5). Since city 1 already appears in child
2, we select city 5 as the next city. 4 1 2 0 5 x
• The edges after city 5 are (5, 0) and (5, 4), but city 4 and city 0 both appear in
child 2. We select a non-selected city, which is city 3. and thus produce a legal
child 4 1 2 0 5 3
We can use the same method to generate the other child. 1 2 0 5 4 3
63
TSP Genetic operators - cont
Mutation
• We can not use the traditional mutation operator. For example if
we have a legal tour before mutation 1 2 3 4 5 0, assuming the
mutation site is 3, we randomly change 3 to 5 and generate a new
tour 1 2 5 4 5 0
• Instead, we randomly select two bits in one chromosome and
swap their values.
• Swap mutation: 1 2 3 4 5 0
153420
Selection
• When using traditional roulette wheel selection, the best
individual has the highest probability of survival but does not
necessarily survive.
• We use CHC selection to guarantee that the best individual will
always survive in the next generation (Eshelman 1991).
64
TSP Behavior
•
•
•
•
•
CHC selection - population size is N
Generate N children by using roulette wheel selection
Combine the N parents with the N children
Sort these 2N individuals according to their fitness value
Choose the best N individuals to propagate to the next generation.
Comparison of
Roulette and CHC selection
With CHC selection the population
converges quickly compared
to roulette wheel selection
and the performance is also better.
65
TSP Behavior - cont
• But fast convergence may mean less exploration.
• To prevent convergence to a local optimum, when the population
has converged we save the best of the individuals and re-initialize
the rest of the population randomly.
• We call this modified CHC selection R-CHC.
Comparison of CHC selection
with or without re-initialization
66
9 Parallel implementations of GAs
• Migration
• Global model - worker/farmer
• Diffusion model
67
9.1 Migration
• The migration model divides the population in multiple
subpopulations.
• These subpopulations evolve independently from each
other for a certain number of generations (isolation
time).
• After the isolation time a number of individuals is
distributed between the subpopulations = migration.
• How much genetic diversity can occur in the
subpopulations and the exchange of information
between subpopulations depends on:
- the number of exchanged individuals = migration rate
- the selection method of the individuals for migration
- the scheme of migration determines.
68
Migration - cont
• The parallel implementation of the migration
model:
– a speedup in computation time
– needs less objective function evaluations when
compared to a single population algorithm.
• So, even for a single processor computer,
implementing the parallel algorithm in a serial
manner (pseudo-parallel) delivers better results
(the algorithm finds the global optimum more
often or with less function evaluations).
69
Migration - cont
• The selection of the individuals for migration can take
place:
– uniform at random (pick individuals for migration in a random
manner),
– fitness-based (select the best individuals for migration).
• Many possibilities exist for the scheme of the migration
of individuals between subpopulations. For example,
migration may take place:
– between all subpopulations (complete net topology unrestricted)
– in a ring topology
– in a neighbourhood topology
70
Migration - cont
Unrestricted migration topology (Complete net
topology)
71
Migration - cont
Scheme for migration of individuals between
subpopulations
72
Migration - cont
Ring migration topology
Neighbourhood
migration
topology (2-D
implementation)
73
9.2 Global model - worker/farmer
• The global model employs the inherent parallelism of
genetic algorithms (population of individuals).
• The Worker/Farmer algorithm is a possible
implementation.
74
• The diffusion
model selects the
mating partner in
a local
neighbourhood
similar to local
selection. Thus, a
diffusion of
information
through the
population takes
place. During the
search virtual
islands will
evolve.
9.3 Diffusion model
75
10 Coevolution
The “tragedy of commons” problem:
 the rational exploitation of natural
renewable resources by self-interested
agents (human or artificial) that have to
achieve a certain degree of cooperation in
order to avoid the overexploitation of
resources.
76
10.1
Problem Description
“Tragedy of commons” instances:
 use
of common pastures by sheep farmers
 fish and whales catching
 environmental pollution
 urban transportation problems
 preservation of tropical forests
 common computer resources used by different processors
The problem instance considered:
 Fishing companies (Ci) owing several ships (NSi) and having the
possibility to fish in several fishing banks (Rp), during different
seasons (Tj).
 Each company aims at collecting maximum assets expressed by
the amount of money they earn (and the number of ships). The
money is generated by fish catching at fish banks.
 A fish bank is a renewable resource and will not be regenerated if
the companies will be catching too much fish, leading to the
exhaustion of the resource, ecological unbalance, and profit loss.
77
10.2
The model
Company agents (CompA)
A company is represented by a cognitive agent
 A CompA may be ecologically oriented or profit oriented (fix the
goals)
 A CompA have a planning component to develop plans for sending
ships to fish banks (in shore or deep sea), under the uncertainty
induced by the fishing actions of the other company agents.
Environment agent (EnvA)



The attributes of the environment
The evolution of the environment status
Each company in the system has to register at the EnvA (EnvA is
also a facilitator in the MAS)
78
Goals, Plans,
Ships, Profile
Request
Company 3 Agent
Company 1 Agent
Self representation
 World representation
Accept

Declare
Fish banks, Seasons,
Inquiry
Regeneration limit/rate,
Info of other agents
ModifyRequest
Request
Result
Environment Agent
Inform
Companies
 Fish banks
 Environment state

Company 2 Agent
Multi-agent system to model the “tragedy of commons”
79
10.3 GA with genetic co-adapted
components
Cooperative Coadapted Species (Potter and De Jong, 2000):

A subcomponent evolves separately by a genetic algorithm, but the
evolution is influenced by the existence of the other subcomponents in
the ecosystem.
 Each species is evolved in its own population and adapts to the
environment through the repeated application of a genetic algorithm.
 The influence of the environment on the evolution, namely the
existence of the other species, is handled by the evaluation function of
the individuals, which takes into account representatives from other
species.
 The interdependency among subcomponents is achieved by evolving
the species in parallel and evaluating them within the context of each
other.
80
Population of
Company i
Population of
Company 1
Individual1 of Ci
...
Best of C1
Individualj of Ci
...
Individualn of Ci
Population of
Company M
Evaluate individuals
of Company i
Best of CM
Genetic coadapted components to model the companies
81





The fitness of an individual in a population is computed based on its
configuration and on representatives chosen from the other
populations in the ecosystem.
From each population the representative is chosen as the “best”
individual of the population, with two possible interpretations for what
“best” means in the context of a subspecies.
If the company is profit oriented, the best individual is the one that
will bring the maximum profit.
For ecologically oriented companies, the representative is the
individual that will bring maximum profit, while keeping the minimum
amount of fish that allows regeneration. If the profile of the company
is not known, an ecologically profile is assumed by default.
The fitness of the individual is evaluated in the context of the selected
representative and is based on the profit, taking care of the ecological
balance (individuals that do not ensure the minimum regeneration
amount are assigned 0 fitness) or not, depending on the company
profile.
82
Company i
Ship 1
Ship 2
Ship NSi
First representation
(NoBPlace + NoBSeason) * NoShips
Season: T1, T2, T3, T4
Place at Ti: H, R1, R2, R3
Company i
T1
T2
Place of Ship NSi: H, R1, R2, R3
Place of Ship 1: H, R1, R2, R3
Tn
Second representation
NoBPlace * NoShips * NoSeasons
83
10.4 Genetic Model Utilization
- A CompA builds its own plan by genetic evolution while
keeping the ecological balance: it models himself and the
way it believes the other CompAs will act.
- Replanning, after seasons T1..Ti have passed, by the
same approach as above.
- A CompA may keep alternate “good plans” by selecting
the first x “best plans” from genetic evolution, for negotiation
with other CompAs.
- A CompA may model the evolution of the world (the other
CompAs action plans) in case it performs symbolic
distributed planning.
- The genetic model may be used by the EnvA
84
10.5
Experimental results
The planning process was tested for several
situations, with different GA and EA parameters.
We present results for 5 fishing seasons, three fishing
banks (R1, R2, R3) and in-port (H), and 3 companies.
The fitness of an individual was computed as 95% profit
and 5% preservation of resources, with a 0 fitness
value if at least one resource goes beyond the
regeneration level.
85
Genetic Algorithm's parameters:
 Two-point crossover
 Probability of mutation in every individual: 0.1
 Population size: 100
 Length of an individual: 100
 The selection is based on the stochastic remainder
technique
 Number of generations: 20..200
Evolutionary Algorithm’s parameters:
 Crossover rate: 0
 Probability of mutation in every individual: 0.05..0.50
 Population size: 100
 Length of an individual: 100
 Number of generations:150
86
RUN 1
Company 1
Season 1
Season 2
Season 3
Season 4
Season 5
Fitness value
Company 2
Season 1
Season 2
Season 3
Season 4
Season 5
Fitness value
Company 3
Season 1
Season 2
Season 3
Season 4
Season 5
Fitness value
H
R1
R2
3
1
5
2
3
3
1
3
2
2
0
3
0
3
3
0.836166666666
H
R1
R2
2
4
2
1
4
2
1
1
2
3
2
2
0
2
3
0.855166666666
H
R1
R2
1
4
3
3
4
0
2
5
3
1
1
5
2
3
2
0.817166666666
R3
1
2
4
5
4
R3
2
3
6
3
5
R3
2
3
0
3
3
RUN 2
Company1
Season 1
Season 2
Season 3
Season 4
Season 5
Fitness value
Company2
Season 1
Season 2
Season 3
Season 4
Season 5
Fitness value
Company 3
Season 1
Season 2
Season 3
Season 4
Season 5
Fitness value
H
R1
R2
2
2
1
1
3
3
0
3
3
3
4
1
1
3
0
0.856229166666
H
R1
R2
2
3
1
3
3
1
4
2
1
2
3
4
3
2
3
0.730333333333
H
R1
R2
2
4
1
3
2
2
1
2
3
2
0
5
1
0
5
0.826041666666
R3
5
3
4
2
6
R3
4
3
3
1
2
R3
3
3
4
3
4
Results of genetic plan evolution for
3 companies and 2 runs
87
1
Run 1
Run 2
Run 3
Average
0.9
0.8
Fitness
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
20
30
40
50
60
70
80
90
100 110 120 130 140 150 160 170 180 190
Generations No.
Average best fitness of genetic plans
depending on the number of generations
(the average is for 5 runs while only the results of the first three are shown)
88
Evolutionary Algorithm
(150 Generations)
Run 1
Run 2
Run 3
Average
0.900
0.800
0.700
Fitness
0.600
0.500
0.400
0.300
0.200
0.100
0.000
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
Mutation Rate
Average best fitness of evolutionary plans
depending on mutation rate
(the average is for 5 runs while only the results of the first three are shown)
89