Daniel Chivilikhin
03.04.2013

1. Biological inspiration of ACO
2. Solving NP-hard combinatorial problems
3. The ACO metaheuristic
4. ACO for the Traveling Salesman Problem
2

1. Biological inspiration of ACO
2. Solving NP-hard combinatorial problems
3. The ACO metaheuristic
4. ACO for the Traveling Salesman Problem
3

Biological inspiration: from real to artificial ants
4

• Distributed systems of social insects
• Consist of simple individuals
• Colony intelligence >> Individual intelligence
5

• Stigmergy – indirect communication between individuals (ants)
• Driven by environment modifications
6

Denebourg’s double bridge experiments
• Studied Argentine ants I. humilis
• Double bridge from ants to food source
7

Double bridge experiments: equal lengths (1)
8

Double bridge experiments: equal lengths (2)
• Run for a number of trials
• Ants choose each branch ~ same number of trials
9

Double bridge experiments: different lengths (2)
10

Double bridge experiments: different lengths (2)
• The majority of ants follow the short path
11

1. Biological inspiration of ACO
2. Solving NP-hard combinatorial problems
3. The ACO metaheuristic
4. ACO for the Traveling Salesman Problem
12

Solving NP-hard combinatorial problems
13

• Find values of discrete variables
• Optimizing a given objective function
14

Π = ( S , f , Ω) – problem instance
• S – set of candidate solutions
• f – objective function
•
• Ω – set of constraints
~
S

S – set of feasible solutions (with respect to Ω)
• Find globally optimal feasible solution s*
15

NP-hard combinatorial problems
• Cannot be exactly solved in polynomial time
• Approximate methods – generate nearoptimal solutions in reasonable time
• No formal theoretical guarantees
• Approximate methods = heuristics
16

• Constructive algorithms
• Local search
17

• Add components to solution incrementally
• Example – greedy heuristics: add solution component with best heuristic estimate
18

• Explore neighborhoods of complete solutions
• Improve current solution by local changes
– first improvement
– best improvement
19

• A set of algorithmic concepts
• Can be used to define heuristic methods
• Applicable to a wide set of problems
20

• Simulated annealing
• Tabu search
• Iterated local search
• Evolutionary computation
• Ant colony optimization
• Particle swarm optimization
• etc.
21

1. Biological inspiration of ACO
2. Solving NP-hard combinatorial problems
3. The ACO metaheuristic
4. ACO for the Traveling Salesman Problem
22

The ACO metaheuristic
23

• A colony of artificial ants cooperate in finding good solutions
• Each ant – simple agent
• Ants communicate indirectly using stigmergy
24

Combinatorial optimization problem mapping (1)
• Combinatorial problem ( S , f , Ω(t))
• Ω(t) – time-dependent constraints
 Example – dynamic problems
• Goal – find globally optimal feasible solution s *
• Minimization problem
• Mapped on another problem…
25

Combinatorial optimization problem mapping (2)
• C = { c
1
, c
2
, …, c
Nc
} – finite set of components
• States of the problem:
X = { x = < c i
, c j
, …, c h
, …>, | x | < n < + ∞}
• Set of candidate solutions:
S

X
26

Combinatorial optimization problem mapping (3)
• Set of feasible states:
~
X

X satisfying Ω(t)
• Non-empty set of optimal solutions:
S
* 
~
X  S
27

Combinatorial optimization problem mapping (4)
• X – states
• S – candidate solutions
• ~
X
– feasible states
•
S * – optimal solutions
S
X
S*
~
X
28

Combinatorial optimization problem mapping (5)
• Cost g ( s , t ) for each s

S
• In most cases – g ( s , t ) ≡ f ( s , t )
 s

~
S
•
G
C
= ( C , L ) – completely connected graph
• C – set of components
• L – edges fully connecting the components
( connections )
•
G
C
– construction graph
29

Combinatorial optimization problem mapping (last  )
• Artificial ants build solutions by performing randomized walks on G
C
( C , L )
30

• Each component c i associated:
 heuristic information
 pheromone trail or connection l ij have
31

• A priori information about the problem
• Does not depend on the ants
• On components c i
• On connections l ij
• Meaning:
– η
– η ij i cost of adding a component to the current solution
32

• Long-term memory about the entire search process
• On components c i
• On connections l ij
– τ
– τ ij i
• Updated by the ants
33

• Stochastic constructive procedure
• Builds solutions by moving on G
C
• Has finite memory for:
– Implementing constraints Ω( t )
– Evaluating solutions
– Memorizing its path
34

• Has a start state x
• Has termination conditions e k
• From state x r moves to a node from the neighborhood – N k ( x r
)
• Stops if some e k are satisfied
35

• Selects a move with a probabilistic rule depending on:
 Pheromone trails and heuristic information of neighbor components and connections
 Memory
 Constraints Ω
36

• Can update pheromone on visited components (nodes)
• and connections (edges)
• Ants act:
 Concurrently
 Independently
37

While not doStop():
ConstructAntSolutions()
UpdatePheromones()
DaemonActions()
38

• A colony of ants build a set of solutions
• Solutions are evaluated using the objective function
39

• Two opposite mechanisms:
 Pheromone deposit
 Pheromone evaporation
40

UpdatePheromones: pheromone deposit
• Ants increase pheromone values on visited components and/or connections
• Increases probability to select visited components later
41

UpdatePheromones: pheromone evaporation
• Decrease pheromone trails on all components/connections by a same value
• Forgetting – avoid rapid convergence to suboptimal solutions
42

• Optional centralized actions, e.g.:
 Local optimization
 Ant elitism (details later)
43

• Traveling salesman
• Quadratic assignment
• Graph coloring
• Multiple knapsack
• Set covering
• Maximum clique
• Bin packing
• …
44

1. Biological inspiration of ACO
2. Solving NP-hard combinatorial problems
3. The ACO metaheuristic
4. ACO for the Traveling Salesman
Problem
45

ACO for the Traveling Salesman
Problem
46

• N – set of nodes (cities), | N | = n
• A – set of arcs, fully connecting N
• Weighted graph G = ( N , A )
• Each arc has a weight d ij
• Problem:
– distance
Find minimum length Hamiltonian circuit
47

• Identical to the problem graph
• C = N
• L = A
• states = set of all possible partial tours
48

• All cities have to be visited
• Each city – only once
• Enforcing – allow ants only to go to new nodes
49

• Desirability of visiting city j directly after i
50

• η ij
= 1 / d ij
• Used in most ACO for TSP
51

• Select random start city
• Add unvisited cities iteratively
• Until a complete tour is built
52

• Ant System
• Elitist Ant System
• Rank-based Ant System
• Ant Colony System
• MAX-MIN Ant System
53

Ant System: Pheromone initialization
• Pheromone initialization
Τ ij
= m / C nn , where:
 m – number of ants
 C nn – path length of nearest-neighbor algorithm
54

Ant System: Tour construction
• Ant k is located in city i
•
N i k is the neighborhood of city i
• Probability to go to city j

N i k : p ij k 

 k l

N ij
 i
 il

  ij

  il

55

Tour construction: comprehension p ij k 
 l

N k ij
  il
 ij
   il i
• α = 0 – greedy algorithm
• β = 0 – only pheromone is at work
 quickly leads to stagnation
56

Ant System: update pheromone trails – evaporation
Evaporation for all connections ∀ ( i , j ) ∈ L :
τ ij
← (1 – ρ ) τ ij
,
ρ ∈ [0, 1] – evaporation rate
Prevents convergence to suboptimal solutions
57

Ant System: update pheromone trails – deposit
• T k – path of ant k
• C k – length of path T k
• Ants deposit pheromone on visited arcs:
 ij
  ij
 k m 

1
  ij k
,

 

L
  ij k 

1

 0 ,
/ C k
,
 otherwise
T k
58

• Best-so-far ant deposits pheromone on each iteration:
 ij
  ij
 k m 

1
  ij k  e
  ij best
  ij best 

1

 0 ,
/ C best
, otherwise

T best
59

• Rank all ants
• Each ant deposits amounts of pheromone proportional to its rank
60

1. Only iteration-best or best-so-far ant deposits pheromone
2. Pheromone trails are limited to the interval [ τ min,
τ max
]
61

• Differs from Ant System in three points:
 More aggressive tour construction rule
 Only best ant evaporates and deposits pheromone
 Local pheromone update
62

1.Tour Construction j
 arg
 like max
Ant l

N i k
  il
System , il
, if q
 otherwise q
0
2. Local pheromone update:
τ ij
← (1 – ξ ) τ ij
+ ξτ
0
,
63

Comparing Ant System variants
64

• CONCORDE http://www.tsp.gatech.edu/concorde.html
• Solved an instance of 85900 cities
• Computation took 286-2719 CPU days!
65

• New applications
• Theoretical proofs
66

• M. Dorigo, T. Stützle. Ant Colony
Optimization. MIT Press, 2004.
• http://iridia.ulb.ac.be/~mdorigo/ACO/
67

• Some proofs of ACO convergence
68

Thank you!
Any questions?
This presentation is available at: http://rain.ifmo.ru/~chivdan/presentations/
Daniel Chivilikhin [mailto: chivdan@gmail.com]
69

• http://teamaltman.com/wp-content/uploads/2011/06/Uncertainty-Ant-Apple-
1024x1024.jpg
• http://myrealestatecoach.files.wordpress.com/2012/04/ant.jpg
• http://ars.els-cdn.com/content/image/1-s2.0-S1568494613000264-gr3.jpg
• http://www.theorie.physik.uni-goettingen.de/forschung/ha/talks/stuetzle.pdf
• http://www.buyingandsellingwebsites.com/wp-content/uploads/2011/12/Ant-150.jpg
• http://moodle2.gilbertschools.net/moodle/file.php/1040/Event-
100_Days/Ant_Hormiga.gif
• http://4.bp.blogspot.com/_SPAe2p8Y-kg/TK3hQ8BUMtI/AAAAAAAAAHA/p75K-
GcT_oo/s1600/ant+vision.GIF
• http://1.bp.blogspot.com/tXKJQ4nSqOY/UQZ3vabiy_I/AAAAAAAAAjA/IK8jtlhElqk/s1600/16590492-illustrationof-an-ant-on-a-white-background.jpg
70