Combinatorial Algorithms
Jessica
INTRODUCTION
What are Combinatorial Algorithms?
• Algorithms to investigate combinatorial structures
• Generation: Construct all the combinatorial structures
of a particular type
– Ex: subsets, permutations, partitions, trees, Catalan
families
• Enumeration: Compute the number of different
structures of a particular type
– Ex: find number of k-subsets of an n element set
• Search: Find at least one example of a structure of
particular type (if it exists)
– Ex: find a clique of a specified size in a given graph
Optimization Problems
• Suppose we are trying to solve an
optimization problem
• There exist backtracking solutions for this
reason
Backtracking Algorithms
• Generate all possible solutions in a certain
lexicographic order
• Advantages:
– Useful for finding one optimal solution and for
counting or enumerating all optimal solutions
• Disadvantages:
– May be inefficient for finding just one solution
– For optimal verification, may need to examine large
part of the state space tree, even with pruning
– Sometimes need to “back up” several levels when
partial solution cannot be extended
Heuristic Algorithm
• Definition: A heuristic algorithm tries to find a
certain combinatorial structure or solve an
optimization problem by the use of heuristics.
• Heuristic: “serving or helping to find out or
discover; proceeding by trial and error”
• Types of problems can be applied to:
– Find 1 optimal solution
– Find “close to” optimal solution
Heuristic Characteristics and Methods
Characteristics
• The state space is not fully
explored
• Randomization is often
employed
• There is a concept of
neighborhood search
Methods
• Hill-climbing
• Simulated annealing
• Tabu search
• Genetic algorithms
General framework for heuristic search
Generic Optimization Problem (maximization):
Instance: A finite set X
An objective function P : X  Z
m feasibility functions g : Xj Z, 1 ≤ j ≤ m
Find:
the maximum value of P(X)
subject to Xj X and g (X) ≥ 0, for 1 ≤ j ≤ m
Designing a heuristic search
• Define a neighborhood function N : X  2X
– Ex: N(X) = {X1 , X2 , X3 , X4 , X5 }
• Design a neighborhood search:
– Algorithm that finds a feasible solution on the
neighborhood of a feasible solution X.
– 2 types of neighborhood searches:
• Exhaustive (chooses best profit among neighbor points)
• Randomized (picks a random point among the neighbor
points)
Defining a neighborhood function
N : X  2X
So, N(X) is a subset of X.
• N(X) should contain elements that are similar or “close
to” X.
• N(X) may contain infeasible elements of X.
• To be useful, want to get to Xopt from X0 via a number
of applications of N(▪)
• ie. Graph G with V(G) = X and E(G) = {{X, Y} : Y N(X)}
should ideally be connected, or at least have one
optimal solution in each of its connected components
• Computing N(X) should be fast, and in particular
|N(X)| shouldn’t be too large
Sumanaruban, Yogesh and Lahiru
HILL CLIMBING ALGORITHM
Hill-Climbing
• Idea: Go up the hill continuously, stop when
stuck.
Hill Climbing Algorithm
1. Pick a random point in the search space
2. Consider all the neighbors of the current
state
3. Choose the neighbor with the best quality
and move to that state
4. Repeat 2 thru 4 until all the neighboring
states are of lower quality
5. Return the current state as the solution state
• Problem: it can get stuck in a local optimum
• Simple (often effective) solution
– Multiple random restarts
Uniform graph partition
• Instance
A complete graph on 2n vertices, G  (V , )
A cost function, cost : E    {0}

• Find the minimum value of
C ([  ,  ])  {u ,v}E cos t (u, v )
0
1
ux 0,vx1
Example
• n = 4;
– cost(1, 2) = 1,
– cost(1, 4) = 5,
– cost(2, 4) = 5,
cost(1, 3) = 2,
cost(2, 3) = 0,
cost(3, 4) = 1
X
2
1
x1
2
5
0
5
x4
1
x3
• Only 3 feasible solutions:
• X0 = {1, 2}, X1 = {3, 4}
X
2
1
x1
2
5
0
5
x4
• C([X0, X1]) = 12
1
x3
• X0 = {1, 4}, X1 = {2, 3}
X
2
1
x1
2
5
0
5
x4
• C([X0, X1]) = 9
1
x3
• X0 = {1, 3}, X1 = {2, 4}
X
2
1
x1
2
5
0
5
x4
1
• C([X0, X1]) = 7 (optimal)
x3
Neighbourhood function: exchange x ∈ X0 and y ∈ X1.
Algorithm UGP(Cmax)
X = [X0, X1] ← SelectRandomPartition
c←1
while (c ≤ Cmax) do
[Y0, Y1] ← Ascend(X)
if not fail then
{X0 ← Y0; X1 ← Y1; }
else return
c←c+1
Algorithm Ascend([X0, X1])
g←0
for each i ∈ X0 do
for each j ∈ X1 do
t ← G[X0,X1](i, j) (gain obtained in exchange)
if (t > g)
{x ← i; y ← j; g ← t}
if (g > 0) then
Y0 ← (X0 ∪ {y}) \ {x}
Y0 ← (X1 ∪ {x}) \ {y}
fail ← false
return [Y0, Y1]
else {fail ← true; return [X0, X1]}
Other drawbacks
local maximum
plateau
ridge
Image from:
http://classes.yale.edu/fractals/CA/GA/Fitness/Fi
tness.html
• Ridge = sequence of local maxima difficult for greedy
algorithms to navigate
• Plateau = an area of the state space where the evaluation
function is flat.
Hill-climbing variations
• How do we make hill climbing less greedy?
• Stochastic hill-climbing
– Random selection among the better neighbors.
– The selection probability can vary with the steepness of the uphill
move.
• What if the neighborhood is too large to
enumerate?
• First-choice hill-climbing
– Randomly generate neighbors, one at a time
– If better, take the move
• Random-restart hill-climbing
– Tries to avoid getting stuck in local maxima.
Hill-Climbing Algorithm
for
Steiner Triple Systems
Stinson's Algorithm
Steiner Triple System
Examples

STS(3), v = 3 (trivial case)
 V = {1,2,3}
 B = {123}
Examples


STS(3), v = 3 (trivial case)
 V = {1,2,3}
 B = {123}
STS(7), v = 7
 V = {1,2,3,4,5,6,7}
 B = {1,2,3}, {1,4,5}, {1,6,7}, {2,4,6}, {2,5,7}, {3,4,7}, {3,5,6}
Examples



STS(3), v = 3 (trivial case)
 V = {1,2,3}
 B = {123}
STS(7), v = 7
 V = {1,2,3,4,5,6,7}
 B = {1,2,3}, {1,4,5}, {1,6,7}, {2,4,6}, {2,5,7}, {3,4,7}, {3,5,6}
STS(9), v = 9
 V = {1,2,3,4,5,6,7,8,9}
 B = {1,2,3}, {4,5,6}, {7,8,9}, {1,4,7}, {2,5,8}, {3,6,9}, {1,5,9},
{2,6,7}, {3,4,8}, {1,6,8}, {2,4,9}, {3,5,7}.
A Graph Theoretic View
Another way to look at Steiner Triple Systems
Image source - http://mathworld.wolfram.com/SteinerTripleSystem.html
A Graph Theoretic View
Another way to look at Steiner Triple Systems
Consider the complete graph on v vertices, Kv.
A decomposition of Kv into edge disjoint triangles
(K3's) is equivalent to a Steiner Triple System.
Image source - http://mathworld.wolfram.com/SteinerTripleSystem.html
Fano Plane
Fano Plane
The Fano plane is an STS(7) Steiner triple system.
The blocks are the 7 lines, each containing 3 points. Every
pair of points belongs to a unique line.
Fano Plane
The Fano plane is an STS(7) Steiner triple system.
The blocks are the 7 lines, each containing 3 points. Every
pair of points belongs to a unique line.
Has application like factoring integers via quadratic forms
Kirkman's Schoolgirl Problem
A problem in combinatorics
proposed by Rev. Thomas
Penyngton Kirkman in 1850
as Query VI in The Lady's and
Gentleman's Diary (pg.48).
Image source - http://delphiforfun.org/programs/kirkman.htm
Kirkman's Schoolgirl Problem
A problem in combinatorics
proposed by Rev. Thomas
Penyngton Kirkman in 1850
as Query VI in The Lady's and
Gentleman's Diary (pg.48).
Fifteen young ladies in a school walk out three abreast for
seven days in succession:
it is required to arrange them daily so that no two shall walk
twice abreast.
Image source - http://delphiforfun.org/programs/kirkman.htm
Kirkman Systems
The solution to Kirkman’s 15 schoolgirl problem is a
resolution of a block design on 15 points with blocks of size 3
Steiner Triple System on 15 points
Kirkman Systems
The solution to Kirkman’s 15 schoolgirl problem is a
resolution of a block design on 15 points with blocks of size 3
Steiner Triple System on 15 points
Solution not unique
Kirkman Systems
The solution to Kirkman’s 15 schoolgirl problem is a
resolution of a block design on 15 points with blocks of size 3
Steiner Triple System on 15 points
Solution not unique
7 possible solutions for STS(15)
1
2
If an STS(n) exists, so too does an STS(2n + 1).
3
If an STS(n) exists, so too does an STS(2n + 7).
1
1
PROOF: 1.1
𝑣−1
𝑟=
2
r = number of blocks in which any point appear
v = total number of points
1
PROOF: 1.1
𝑣−1
𝑟=
2
r = number of blocks in which any point appear
v = total number of points
• Any point x must appear in some block with each of all other (v - 1)
points
1
PROOF: 1.1
𝑣−1
𝑟=
2
r = number of blocks in which any point appear
v = total number of points
• Any point x must appear in some block with each of all other (v - 1)
points
• Point x occurs with 2 other points in each of the r blocks it appears in
1
PROOF: 1.1
𝑣−1
𝑟=
2
r = number of blocks in which any point appear
v = total number of points
• Any point x must appear in some block with each of all other (v - 1)
points
• Point x occurs with 2 other points in each of the r blocks it appears in
• Therefore,
1
PROOF: 1.2
𝑣(𝑣 − 1)
|𝐵| =
6
|B| = number of blocks in set B
v = total number of points
1
PROOF: 1.2
𝑣(𝑣 − 1)
|𝐵| =
6
|B| = number of blocks in set B
v = total number of points
• Let b = |B| be the total number of blocks
1
PROOF: 1.2
𝑣(𝑣 − 1)
|𝐵| =
6
|B| = number of blocks in set B
v = total number of points
• Let b = |B| be the total number of blocks
• We count T, the number of points with their replications appearing
on B, in two ways:
1
PROOF: 1.2
𝑣(𝑣 − 1)
|𝐵| =
6
|B| = number of blocks in set B
v = total number of points
• Let b = |B| be the total number of blocks
• We count T, the number of points with their replications appearing
on B, in two ways:
• T=3Xb
1
PROOF: 1.2
𝑣(𝑣 − 1)
|𝐵| =
6
|B| = number of blocks in set B
v = total number of points
• Let b = |B| be the total number of blocks
• We count T, the number of points with their replications appearing
on B, in two ways:
• T=3Xb
• T=vXr
(r = (v-1)/2 using lemma 1.1)
1
PROOF: 1.2
𝑣(𝑣 − 1)
|𝐵| =
6
|B| = number of blocks in set B
v = total number of points
• Let b = |B| be the total number of blocks
• We count T, the number of points with their replications appearing
on B, in two ways:
• T=3Xb
• T=vXr
(r = (v-1)/2 using lemma 1.1)
• Therefore,
2
If an STS(n) exists, so too does an STS(2n + 1).
3
If an STS(n) exists, so too does an STS(2n + 7).
Reference - http://www.math.mun.ca/~davidm/4341/sts.pdf
1
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• For any x ∈ V, the triples containing x partition V-{x} into
pairs,
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• For any x ∈ V, the triples containing x partition V-{x} into
pairs,
• thus, v-1 is even
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• For any x ∈ V, the triples containing x partition V-{x} into
pairs,
• thus, v-1 is even
• and v is odd
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• For any x ∈ V, the triples containing x partition V-{x} into
pairs,
• thus, v-1 is even
• and v is odd
• Therefore, v≡1,3 or 5(mod 6)
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• For any x ∈ V, the triples containing x partition V-{x} into
pairs,
• thus, v-1 is even
• and v is odd
• Therefore, v≡1,3 or 5(mod 6)
• From lemma 1.2 (b-number of blocks)
b=v(v-1)/6
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• However, if v = 6k+5, computing |B| gives:
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• However, if v = 6k+5, computing |B| gives:
|B| = (6k+5)(6k+4)/6
|B| = (36k2 + 54k + 20)/6
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• However, if v = 6k+5, computing |B| gives:
|B| = (6k+5)(6k+4)/6
|B| = (36k2 + 54k + 20)/6
which is not an integer.
Theorem 1.1
STS(v) ⇒ v ≡ 1, 3 (mod 6)
PROOF:
Let (V,B) be an STS(v)
• However, if v = 6k+5, computing |B| gives:
|B| = (6k+5)(6k+4)/6
|B| = (36k2 + 54k + 20)/6
which is not an integer.
Therefore,
v≡1,3(mod 6)
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
PROOF (Using Mathematical Induction)
• Base Case :
• The base cases are v = 1, v = 3, v = 7 and v = 9, which have been
illustrated
Examples



STS(3), v = 3 (trivial case)
 V = {1,2,3}
 B = {123}
STS(7), v = 7
 V = {1,2,3,4,5,6,7}
 B = {1,2,3}, {1,4,5}, {1,6,7}, {2,4,6}, {2,5,7}, {3,4,7}, {3,5,6}
STS(9), v = 9
 V = {1,2,3,4,5,6,7,8,9}
 B = {1,2,3}, {4,5,6}, {7,8,9}, {1,4,7}, {2,5,8}, {3,6,9}, {1,5,9},
{2,6,7}, {3,4,8}, {1,6,8}, {2,4,9}, {3,5,7}.
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
PROOF (Using Mathematical Induction)
• Base Case :
• The base cases are v = 1, v = 3, v = 7 and v = 9, which have been
illustrated
• Induction Hypothesis :
• Let v = k, (k ≡ 1, 3 (mod 6))
• Assume that there exist an STS(l) for all l < k (l ≡ 1, 3 (mod 6))
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
PROOF (Using Mathematical Induction)
• Induction Step : (Prove STS(k) exist)
• We consider four cases, depending on the congruence of k modulo 12
•
•
•
•
k = 12m + 1
k = 12m + 3
k = 12m + 7
k = 12m + 9
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
PROOF (Using Mathematical Induction)
• Induction Step : (Prove STS(k) exist)
Case 1 : k = 12m + 1
• 6m-3 < k
• Therefore, by the induction hypothesis
• STS(6m - 3) exist
• k = 12m + 1 = 2(6m - 3) + 7
• Lemma 3 says : If an STS(n) exists, so too does an STS(2n + 7)
• Using Lemma 3 there is an STS(k)
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
PROOF (Using Mathematical Induction)
• Induction Step : (Prove STS(k) exist)
Case 2 : k = 12m + 3
• 6m+1 < k
• Therefore, by the induction hypothesis
• STS(6m + 1) exist
• k = 12m + 3 = 2(6m + 1) + 1
• Lemma 2 says : If an STS(n) exists, so too does an STS(2n + 1)
• Using Lemma 2 there is an STS(k)
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
PROOF (Using Mathematical Induction)
• Induction Step : (Prove STS(k) exist)
Case 3 : k = 12m + 7
• 6m+3 < k
• Therefore, by the induction hypothesis
• STS(6m + 3) exist
• k = 12m + 7 = 2(6m + 3) + 1
• Lemma 2 says : If an STS(n) exists, so too does an STS(2n + 1)
• Using Lemma 2 there is an STS(k)
Theorem 1.2
v ≡ 1, 3 (mod 6) ⇒ STS(v)
PROOF (Using Mathematical Induction)
• Induction Step : (Prove STS(k) exist)
Case 4 : k = 12m + 9
• 6m+1 < k
• Therefore, by the induction hypothesis
• STS(6m + 1) exist
• k = 12m + 9 = 2(6m + 1) + 7
• Lemma 3 says : If an STS(n) exists, so too does an STS(2n + 7)
• Using Lemma 3 there is an STS(k)
Partial Steiner Triple System
PSTS (v )
Is a set system (V , B )
In which every block has size 3
Every pair of points in V is contained in
at most one block
| B | v ( v  1) / 6
If | B | v (v  1) / 6  PSTS (v )  STS (v )
Construct Steiner Triple System
Stinson’s Algorithm
6
Switch Heuristic
Remarks
No termination guarantee
In practice, it always terminates
if the choices made by the heuristic Switch()
are random
Usually runs very quickly
Results
The average number of iterations is O (b ln b)
Shalinda , Ramanathan and Pham Nam
GENETIC ALGORITHM
Genetic Algorithm
• The knowledge that each specie gains is encoded in its chromosomes
and they undergo transformation when reproduction occurs.
• Genes from `good‘ parents will produce offspring that are better
than either parent.
• Thus each successive generation will become more suited to their
environment.
Developed by John Holland, University of Michigan (1970’s)
• Provide efficient, effective techniques for optimization and
machine learning applications
Generic Genetic Algorithm
Incorporates a heuristic hN based on a neighborhood function N and a
recombination operation
rec:  x  →  x 
Chromosomes could be:
Bit strings
Real numbers
Permutations of element
Lists of rules
Program elements
... any data structure …
(0101 ... 1100)
(43.2 -33.1 ... 0.0 89.2)
(E11 E3 E7 ... E1 E15)
(R1 R2 R3 ... R22 R23)
(genetic programming)
Generic Genetic Algorithm
Incorporates a heuristic hN based on a neighborhood function N and a
recombination operation
rec:  x  →  x 
Find members that are similar or close to X
Generic Genetic Algorithm
Incorporates a heuristic hN based on a neighborhood function N and a
recombination operation
rec:  x  →  x 
Reproduction
1. Number of generations
2. Until the algorithm converged
Parents are selected to mate, on
the basis of their fitness
Generic Genetic Algorithm
Incorporates a heuristic hN based on a neighborhood function N and a
recombination operation
rec:  x  →  x 
Operator types are:
• Crossover (recombination)
• Mutation
Crossovers
• In biology a crossover happens when chromosomes from different
parents are split and recombine; the new cell has partial strands from
both parents
• Prime distinguished factor of GA from other optimization techniques
Partially matched crossover
 = {1,….,n}
Two parents are α and β, two children are γ and δ
Two crossover points j and k where 1≤j≤k≤n
Example…
Assume that
α =[3,1,4,7,6,5,2,8] and β=[8,6,4,3,7,1,2,5]
Crossover points are j=3 and k=6
i=3
i=4
i=5
i=6
No effects
γ=[7,1,4,3,6,5,2,8] and δ=[8,6,4,7,3,1,2,5]
γ=[6,1,4,3,7,5,2,8] and δ=[8,7,4,6,3,1,2,5]
γ=[6,5,4,3,7,1,2,8] and δ=[8,7,4,6,3,5,2,1]
Mutations
• Adopting terminology from molecular biology,
changes to chromosomes are called mutations
• Mutation alters one or more gene values in a chromosome
from its initial state
• Purposes:
• Inhibit premature convergence.
• User-definable mutation probability.
• Should be a low probability.
Before:
After:
*
*
(6 5 4 3 7 1 2 8)
(6 5 2 3 7 1 4 8)
Generic Genetic Algorithm
Incorporates a heuristic hN based on a neighborhood function N and a
recombination operation
rec:  x  →  x 
Evaluate population
• Determined by an objective function
Genetic TSP
A Genetic Algorithm for TSP
Ramanathan
Travelling Salesman Problem
Instance: a complete graph Kn
a cost function C(X)
Find:
a Hamiltonian circuit [x0, x1, … , xn-1] that
minimizes C(X) = c(x0, x1) + c(x1, x2) + ... + c(xn-1, x0)
Universe: X = set of all (n-1)!/2 permutations
For 10 cities, 181440 tours are possible
Idea
When we find a good tour, try to improve it using
mutation and recombination
Substring from parent 1’s path
Substring from parent 2’s path
The recombined path could be
better than either parents’ path
Preliminaries
Fitness: Overall cost of the tour
Selection: Initial population drawn from a random
set of tours
Recombination: Partially matched crossover
Mutation: Steepest ascent 2-opt heuristic
2-opt move
Once a tour has been generated by some tour
construction heuristic, we need to improve that
solution
2-opt move:
1. Remove two edges from the tour
2. Reconnect the two paths created to form a new
tour
We complete the move only if the new tour will be
shorter
2-opt move
xi
xi+1
xj
xj+1
xi
xi+1
xj
xj+1
The gain in applying a 2-opt move is the decrease
in cost.
G(X,i,j) = C(X) – C(Xij)
= c(xi,xi+1) + c(xj,xj+1) – c(xi+1,xj+1) – c(xi,xj)
Steepest Ascent method
N(x) = all Y ∈ X that can be obtained from x by a 2opt move
Mutation heuristic consists of a sequence of 2-opt
moves
Using a steepest ascent strategy, iteratively apply 2opt moves until no pair of edges can be found to
yield a positive gain
Algorithm
Input: Given tour X
Output: Optimal tour
gainmax = 0;
while (gainmax > 0) do
gainmax = 0;
for i = 0 to (n-1) do
for j = (i+2) to (n-1) do
g = G(X, i, j);
if (g > gainmax) then
gainmax = g; imax = i; jmax = j;
if (gainmax > 0) then
X = Ximax,jmax;
Example
Fitness function : Euclidean distance
Example
Random tour
Tour length = 1.7938
Example
Tour length = 1.7938
Example
Tour length = 1.4329
Example
Tour length = 1.4329
Example
Tour length = 1.2676
Example
Tour length = 1.2676
Example
Tour length = 1.1259
Example
Tour length = 1.1259
Example
Tour length = 1.0436
Example
Tour length = 1.0436
Example
Tour length = 1.006
Example
Tour length = 1.006
Example
Tour length = 0.9874
Example
Tour length = 0.9874
Example
Tour length = 0.9701
Example
Tour length = 0.9701
Example
Optimal tour length = 0.9678
Partially Matched Crossover Algorithm
Input: Parent tours A, B
Minimum bound on the substring length, h
Output: Offspring tours C, D
j = random number (0, n-1); (start of the substring)
k = random number (h, n/2); (length of the substring)
(C, D) = PartiallyMatchedCrossover(A, B, j, (j+k) mod n)
SteepestAscentTwoOpt(C);
SteepestAscentTwoOpt(D);
return (C, D);
Select initial population
Unranking algorithm: intuition
Unranking algorithm: intuition vs
algorithm
Π(1)
r
dn-1
intuition
Unroll recursive
algorithm
r’
Π(2)
Π(n-1)
Π(n)
dn-2
d1
1
Π
Unranking permutation generating
j
0
1
2
3
d
0
1
1
2
r
15
14
12
0
Π
1
2
1
3
2
1
4
2
3
Genetic TSP
Applications
Puneet
Applications
•
•
•
•
•
Speech Recognition
Mobile Robotics
Singapore MRT
Computer Aided Design
Economics
Speech Recognition
• Language Model
• Acoustic Model
• Initially a simple language model and an acoustic
model is used for speech recognition and results
are stored in a speech lattice.
• However it will be error prone.
• Rescoring of the speech lattice is done using the
sophisticated LM.
• Neighborhoods are created using edit distance
metrics.
• Hill Climbing is applied to find the path with the
highest probability (which is the final output).
Speech Recognition
• Edit Distance is used for defining the
neighbourhood.
• Hill Climbing is used to find the highest
probable path in the lattice after rescoring.
Edit Distance
Artificial Intelligence
Mobile Robotics
• Gradients are used to determine most likely
pose of a robot given sensor measurements.
• The computation of gradient gave real time
performance using hill climbing algorithm.
Computer Aided Design
Computer Aided Design
• Genetic Algorithm takes feedback from
evaluation process to select fitter designs.
• Generates new design through the
recombination of selected parts of the design
• Results in a population of high performance
designs.
Satellite Truss Design
• Genetic Algorithms optimized satellite truss
designs for NASA to maximize vibration
control.
• Evolving: Design structures
• Fitness: Vibration resistance
• Evolutionary “creativity”
Optimized Structure
Real Time Systems
Mass Rapid Transit
• Genetic Algorithm was used for TimeTable
Synchronization in Singapore MRT.
Economics
• Game Theory
Game Theory
• Study of multi person decision problems.
• Each rival player has to consider what the
others will do.
• All players play rationally to maximize their
reward
• Genetic algorithms find the most optimal
strategy for a player in a given situation.
Conclusion
• Introduction
• Hill Climbing
– Uniform Graph Partition
– STS
• Genetic Algorithm
– TSP
• Applications