Lecture: 5 Optimization Methods & Heuristic
Strategies
Ajmal Muhammad, Robert Forchheimer
Information Coding Group
ISY Department
Outline





Introduction to Optimization & Intractable Problems
Linear Programming
Heuristic
Metaheuristic
Linear Programming Formulation
Shortest path Problem
Max Flow Problem
Wavelength Assignment Problem
Optimization Problem
Problem that aims to find the best solution from all feasible
solutions
The best solution can be the minimum or maximum
solution
Examples:


Find the route from point A to point B that takes the
shortest times
Determine how a production factory can maximize its
profit using limited materials
The Travelling Salesman Problem
(TSP)
A classic combinatorial optimization problem
Given n cities on a map, find the shortest route that visits
all the cities once, and starts and ends at the same city
Starting and ending in Paris, which
route allow us to visit major cities
with the least amount of travelling ?
The 1-Dimensional Bin Packing
Problem
Given n items of different (1D) sizes, and given some fixedcapacity bins, pack the items into a minimum number of
bins
Is this the minimal number of
Bins needed for these items?
TSP Growth Rates

A route around a map can be represented as a permutation of the n
cities:
E.g. for 5 cities, [B,A,C,E,D] means “start at city B, then go to city A, then city C, then
E, then D, and return to city B”




Given n cities, there is a total of n! permutations
Some permutations represent the same routes - there are actually
½(n-1)! different routes in total
To return the provably optimal solution to any instance of the TSP
the algorithm needs to check the majority of – if not all – possible
routes
The number of routes grows exponentially quickly making the
problem intractable:
Linear Programming (LP)
An optimization problem in which the objective function and all the
constraints are expressed as linear functions
even if just one of them is not a linear function, the problem is not an LP
problem
Linear programming problem
Nonlinear programming problem
LP Formulation
Matrix expression for LP formulation
Nomenclature in LP Problem
Boundary is a constraint that express the upper or lower
bound of an inequality or equality
Feasible region is an area delineated by the boundaries
Corner point is an intersection of the boundaries
Example
Let z= x+y
max z
Rewrite as y= -x + z
Solution by Moving y = -x +z
Simplex Method
Heuristics
•
•
•
•



Provide a shortcut to solve difficult problems.
Used under limited time and/or information to make a
decision.
Problem-dependent techniques, i.e., adapted to the
problem at hand to take advantage of the particularities
of the problem.
Lead to a good decision most of the time.
Heuristics are good when
Making a spur-of-the-moment decision
Having limited information and cannot obtain more
The decision is not that important
Heuristics




Heuristics are bad when
Having plenty of time and information to make an
important decision
Need to be right 100% of the time
Examples of heuristics
Deciding to eat at restaurant B rather than restaurant A
only because B has more cars in its parking lot
When the level in the fuel tank drops to 1/2 tank or less,
always filling up the fuel tank at the very next re-filling
station
Metaheuristic
Meta: in an upper level
Heuristic: to find
A metaheuristic is formally defined as an iterative generation
process which guides a subordinate heuristic by combining
intelligently different concepts for exploring and exploiting the
search space, learning strategies are used to structure
information in order to find efficiently near-optimal solutions.
A metaheuristic is a function that consists of heuristic
subroutines. It is a compound heuristic.
Fundamental Properties of
Metaheuristics





Metaheuristics are strategies that guide the search
process to avoid being trapped in confined areas of the
search space
The goal is to efficiently explore the search space in
order to find (near)-optimal solutions
Techniques which constitute metaheuristic algorithms
range from simple local search procedures to complex
learning processes
Metaheuristic algorithms are approximate and usually
non-deterministic
Metaheuristics are not problem-specific
Metaheuristic Examples

Genetic algorithms (1960s/1970s), Evolutionary strategy
(Rechenberg & Swefel 1960s), Evolutionary programming
(Fogel et al. 1960s).

Simulated annealing (Kirkpatrick et al. 1983), Tabu search
(Glover 1980s), Ant colony optimization (Dorigo 1992),
Genetic programming (Koza 1992), Particle swarm
optimization (Kennedy & Eberhart 1995), Differential evolution
(Storn & Price 1996/1997),

Harmony search (Geem et al. 2001), Honeybee algorithm
(Nakrani & Tovey 2004), ..., Firefly algorithm (Yang 2008),
Cuckoo search (Yang & Deb 2009), ...
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Ant Colony Optimization (in one Slide)
A. Ants in a pheromone trail between nest and food; B. an obstacle interrupts the trail; C. ants find
two paths to go around the obstacle; D. a new pheromone trail is formed along the shorter path.
LP Formulation for Shortest Path
Problem
Network is represented by directed graph G(V,E), where V is the set of
vertices (nodes) & E is the set of links
Link from node i to node j is expressed by (i,j); dij is the link cost of (i,j)
Xpqij : traffic volume from node p to node q routed through (i,j)
Max flow Problem
Network model with link capacities
Max flow routing from node 1 to node 6
ILP formulation for max flow problem ?
Assignment
Wavelength Assignment (WA) Problem
Optical path requests
Graph construction for coloring
LP Formulation for Graph Coloring
Let W be a set of λ, where W = {λ1,λ2,··· ,λ|W|}
Let xλv and yλ, be binary variables.
If λ is assigned to a path corresponding v, xλv = 1, otherwise xλv = 0.
If λ is used at least one time yλ=1, otherwise yλ=0.
Heuristic: Largest Degree First (LDF)
Step 1: select the uncolored vertex with
largest degree.
Step 2: choose the min indexed color
from the colors that are not used by the
adjacent vertices.
Step 3: color the selected vertex using
the color chosen in step 2.
Step 4: if all vertices are colored, LDF
stops. otherwise, LDF returns to step 1.
Wavelength assignment process of LDF