Nuno Abreu, Zafeiris Kokkinogenis, Behdad Bozorg, Muhammad Ajmal
1
OUTLINE















Introduction to TS
Parameters of TS
Basic concepts of TS
TS vs. other Meta-heuristics
Memory in TS
Use of Memory in TS
Tabus
Aspiration criteria
Stop condition
TS basic algorithm
Search Process – 1
Search Process – 2
Flowchart of a Standard TS Algorithm
Example
Pros and Cons
2
INTRODUCTION TO TS





TS is an iterative procedure designed for the solution of
optimization problems
Invented by Glover (1986)
Uses a neighborhood search procedure to iteratively move
from a solution x to a solution x* in the neighborhood of x
Uses memory structures so that the algorithm does not visit a
given solution repeatedly
Tabu Search Benefits


Cycle avoidance which also saves time
Guide search to promising regions of the search space
3
PARAMETERS OF TABU SEARCH

Space search procedure

Neighborhood structure

Short-term memory: Tabu list

Types of moves

Addition of a Tabu move

Maximum size of Tabu list

Aspiration conditions

Stopping rule
4
BASIC CONCEPTS OF TABU SEARCH








Saves information according to the exploration process
It will be used to limit the moves through the neighborhood
Structure of the neighborhood of the solutions varies from
iteration to iteration
Infeasible solutions can be accepted and evaluated to escape
local minimum.
To prevent from cycling, recent moves are forbidden.
A tabu list records forbidden moves, which are referred to as
tabu moves
A tabu move can be accepted using aspiration criteria.
Allows exploitation of good solution and exploration of
unvisited region of the search space
5
TS VS OTHER META-HEURISTICS

Traditional descent methods cannot allow non-improving
moves, TS can.

SA and GA does not have memory, TS has.

SA uses randomness to escape local minimum, TS uses
forbidden moves.

TS claims that a bad strategic choice can yield more
information than a random choice.
6
MEMORY IN TS

TS uses mainly two types of memory:

Short-term memory




Recent solutions
Structure were tabu moves are stored
Avoids cycling
Long-term memory (frequency-based)

Number of iterations that “solution components” have been
present in the current solution
7
USE OF MEMORY IN TS

Use of memory leads to learning

Prevent the search from repeating moves

Explore the unvisited area of the solution space

By using memory to avoid certain moves, TS can be seen
as global optimizer rather than local.
8
TABUS

Tabus are one of the distinctive elements of TS when
compared to LS

Prevent cycling when moving away from local minimum
through non-improving moves

Stored in the short-term memory – Tabu list

Tabu tenure is the number of iteration the move is tabu

Tabu list can be of fixed-length or dynamically varying
9
ASPIRATION

If a move which is tabu can lead to better solution, the
tabu status should be overruled

This will be performed using aspiration level conditions

Aspiration criteria: accepting an improving solution even if
generated by a tabu move

A tabu move becomes admissible if it yields a solution that
is better than an aspiration value

A tabu move becomes admissible if the direction of the
search (improving or non-improving) does not change
10
BASIC TABU SEARCH ALGORITHM

Step 1: Choose an initial solution i in S. Set i*=i and k=0.

Step 2: Set k=k+1 and generate a subset V of solutions in
N(i,k) such that the Tabu conditions are not violated or the
aspiration conditions hold.

Step 3: Choose a best j in V and set i=j.

Step 4: If f(i) < f(i*) then set i* = i.

Step 5: Update Tabu and aspiration conditions.

Step 6: If a stopping condition is met then stop. Else go to
Step 2.
11
STOPPING CONDITIONS

N(i, K+1) = 0 (no feasible solution in the neighborhood of
solution i)

K is larger than the maximum number of iterations allowed

The number of iterations since the last improvement of i* is
larger than a specified number

Evidence can be given than an optimum solution has been
obtained
12
SEARCH PROCESS - 1

Intensification:





searches solutions similar to the current solution
wants to intensively explore known promising areas of the search
space
creates solutions using the more attractive components of the best
solutions in memory (recency memory)
Another technique consists in changing the neighborhood’s structure
by allowing different moves
Diversification:


Examines unvisited regions of the search space
Generates different solutions



Using rarely components present in the current solution
Biasing the evaluation of a move by modifying the objective function
adding a term related to component frequencies
Intensification and diversification phases alternate during the
search
13
SEARCH PROCESS - 2

Allowing infeasible solution






Surrogate objective


Constraints defining the search space can lead the process to
mediocre solutions
Induces diversification
By dropping some constraints (relaxation) a larger space can be
explored
Penalize objective for the violation
A well known technique: Strategic oscillation
Evaluates neighbors using a simpler function than the objective in
order to spot promising candidates. (Intensification)
Auxiliary objective


Objective function can’t drive the search to more interesting areas
Orient the search by measuring desirable attributes of solutions
14
FLOWCHART OF A STANDARD TABU SEARCH ALGORITHM
Heuristic
procedure
Tabu restrictions
Candidate lists
Aspiration criteria
Elite solutions
Generate initial
solution and initialize
memory structures
Stop
No
Construct modified
neighborhood
Yes
More iteration?
Short and
long term
memory
Modified choice
rules for
diversification or
intensification
Select best
neighbor
Update memory
structures
Restarting
Strategic oscillation
Execute specializes
procedures
Update best
solution
15
EXAMPLE
•
Minimum spanning tree problem with constraints.
•
Objective: Connects all nodes with minimum costs
Costs
B
B
20
A
30
10
C
5
20
E
A
30
10
25
15
C
5
E
25
D
40
15
D
40
An optimal solution without
considering constraints
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
16
EXAMPLE
Iteration 1
Cost=50+200 (constraint penalties)
Add
Delete
Cost
New cost = 75 (iteration 2)
( local optimum)
BE
BE
BE
CE
AC
AB
75+200=275
70+200=270
60+100=160
CD
CD
AD
AC
60+100=160
65+300=365
DE
DE
DE
CE
AC
AD
85+100=185
80+100=180
75+0=75


B
20
A
30
10
C
5
E
25
Delete 15
D
40 Add
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
17
EXAMPLE
Tabu list: DE
Iteration 2 Cost=75


* A tabu move will be considered only if it would result in a better solution than the
best trial solution found previously (Aspiration Condition)
Iteration 3 new cost = 85 Escape local optimum
Delete
Add
Delete
Cost
B
Add
AD
DE*
Tabu move
20
30
AD
CE
85+100=185
AD
AC
80+100=180
10
A
C 5
E
25
15
D
40
Tabu
BE
BE
BE
CE
AC
AB
100+0=100
95+0=95
85+0=85
CD
CD
DE*
CE
60+100=160
95+100=195
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
18
EXAMPLE
Tabu list: DE & BE
Iteration 3 Cost=85


* A tabu move will be considered only if it would result in a better solution than the
best trial solution found previously (Aspiration Condition)
Iteration 4 new cost = 70 Override tabu status
Add
Delete
Cost
B
Tabu
20
A
30
10
25
15
C
5
E
Add
D
40
Tabu
Delete
AB
AB
AB
BE*
CE
AC
Tabu move
100+0=100
95+0=95
AD
AD
AD
DE*
CE
AC
60+100=160
95+0=95
90+0=90
CD
CD
DE*
CE
70+0=70
105+0=105
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
19
EXAMPLE


Optimal Solution
Cost = 70


Additional iterations only find
inferior solutions
B
20
A
30
10
C
5
E
25
15
D
40
20
PROS AND CONS

Pros:




Allows non-improving solution to be accepted in order to
escape from a local optimum
The use of Tabu list
For larger and more difficult problems tabu search can beat
other approaches
Cons:




Too many parameters to be determined
Number of iterations could be very large
Global optimum may not be found, depends on parameter
settings
Too complex
21
REFERENCES

Glover, F., Kelly, J. P., and Laguna, M. 1995. Genetic Algorithms and Tabu
Search: Hybrids for Optimization. Computers and Operations Research. Vol.
22, No. 1, pp. 111 – 134.

Glover, F. and Laguna, M. 1997. Tabu Search. Norwell, MA: Kluwer Academic
Publishers.

Hanafi, S. 2001. On the Convergence of Tabu Search. Journal of Heuristics.
Vol. 7, pp. 47 – 58.

Hertz, A., Taillard, E. and Werra, D. A Tutorial on Tabu Search. Accessed on
April 14, 2005: http://www.cs.colostate.edu/~whitley/CS640/hertz92tutorial.pdf

Gendreau, M. 2002. An Introduction To Tabu Search, Centre de Recherche
sur les Transports et Département d´informatique et de Recherche
opérationnelle, Université de Montréal.

Hillier, F.S. and Lieberman, G.J. 2005. Introduction to Operations Research.
New York, NY: McGraw-Hill. 8th Ed.
22
23
OUTLINE










Introduction
Tabu Search
Basic Approach – 1
Basic Approach – 2
The TS framework for multi-dimensional bin packing
Search methods
Computational test - Dataset
Computational test
Sample of results
Conclusions
24
INTRODUCTION
The paper deals with multi-dimensional cutting and packing
algorithms

Assumes that the item will be packed with fixed orientation
(no rotation)

Lodi, Martello and Vigo present an effective BP Tabu Search
framework

The main characteristic is the adoption of a search scheme
and a neighborhood independent of the packing problem to be
solved

25
TABU SEARCH

The goal is empty a specified target bin
Given a current solution, the moves modify it by changing the
packing of a set S of items



The target bin is the one minimizing a filling function φ(.)
φ(.) measures the easiness of emptying the bin
The idea is to favor target bins packing a small area and a
relatively large number of items

26
BASIC APPROACH - 1

Select target bin
Include one item, j, from the target bin, and contents of k
other bins in a set S


Execute an greedy-type heuristic A on S

The size of the neighborhood k is automatically updated
If the items of S are packed into k (or less) bins, item j is
removed from the target bin. Otherwise, S is changed by
selecting a different set of k bins, or a different item from the
target bin

Then, a new item is selected, a new set S is defined and a
new move is performed

The execution is halted as soon as a proven optimal solution
is found, or a time limit is reached

27
BASIC APPROACH - 2


If the algorithm gets stuck
The target bin is not emptied, the neighborhood is enlarged by
increasing the value of up to a prefixed upper limit

Tabu list stores,for each forbidden move, the sum of the filling
function values of the k+1 involved bins

Variable Neighborhood Search strategy



Small values of k = Small neighborhoods, fast to explore
Accepting moves dealing with increased k = Enlarged neighborhood,
more chances to improve solution
By changing the size of the neighborhood, the algorithm plays
“Intensification” and “Diversification”
The execution is halted as soon as a proven optimal solution is
found, or a time limit is reached

28
THE TS FRAMEWORK FOR MULTI-DIMENSIONAL BIN PACKING
29
SEARCH METHODS
Intensification procedure explores the neighborhood of the
current solution (Inner loop)


Two types of diversification are defined:
“soft” diversification - select as target bin the one having the
second smallest filling function value

“hard” diversification - re-pack into separate bins the items
currently packed in the z/2 bins ( z being the number of bins in the
current solution) with smallest filling function value


The tabu list stores, for each forbidden move:

the sum of the filling function values

the last moves
30
COMPUTATIONAL TESTS - DATASET

The benchmark consists of 500 random instances with n =
{20,40,60,80,100}

Ten different classes of instances were used

wi (width), hi (height) generated from uniform distribution of varying
intervals
31
COMPUTATIONAL TESTS
Measures the improvement brought by TS when comparing
to three greedy-type heuristics:




Hybrid Best-Fit algorithm (HBP)
Knapsack Packing (KP)
Alternate Directions (AD)
Results show that the TS allows a good improvement in the
quality of the solution obtained by the greedy-type heuristics

32
SAMPLE OF RESULTS
33
CONCLUSIONS

The main idea of the framework:


Isolate the information concerning the problem
Let a greedy-type heuristic take care of the structure and construct
feasible solutions
Tabu Search is then used to drive the search through the
solution space by:



re-combining the packed items,
exploring a neighborhood by alternating between “intensification”
and “diversification”
This results in a very general algorithm for multidimensional
bin packing

34