Analysis and Design of
Algorithms
C
P
C
S
O my Lord! Expand for me my chest [with
assurance] and ease for me my task and untie
the knot from my tongue that they may
understand my speech. (Quran 20 : 25-28)
Instructor Dr. Muhammad Umair
2
2
3
Analysis and Design of
Algorithms
C
P
C
S
Exhaustive Search
Introduction
Instructor Dr. Muhammad Umair
2
2
2
3
Analysis and Design of
Algorithms
Exhaustive Search
• A brute force solution to a problem involving
search for an element with a special
property, usually among combinatorial
objects such as permutations, combinations,
or subsets of a set.
• Method:
– generate a list of all potential solutions to the problem in
a systematic manner
– evaluate potential solutions one by one, disqualifying
infeasible ones and, for an optimization problem,
keeping track of the best one found so far
– when search ends, announce the solution(s) found
Instructor Dr. Muhammad Umair
3
C
P
C
S
2
2
3
Analysis and Design of
Algorithms
C
P
C
S
Exhaustive Search
Travelling Salesman Problem (TSP)
2
2
3
Instructor Dr. Muhammad Umair
4
Analysis and Design of
Algorithms
Travelling Salesman Problem
(TSP)
• Given n cities with known
distances between each pair, find
the shortest tour that passes
through all the cities exactly once
before returning to the starting
city
Instructor Dr. Muhammad Umair
5
C
P
C
S
2
2
3
Analysis and Design of
Algorithms
Travelling Salesman Problem
(TSP)
• Find shortest Hamiltonian circuit
in a weighted connected graph
2
a
b
5
3
2
2
3
4
8
c
C
P
C
S
7
d
Instructor Dr. Muhammad Umair
6
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
7
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
8
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
9
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
10
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
11
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
12
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
13
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
14
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
15
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
16
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
17
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
18
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
19
Analysis and Design of
Algorithms
TSP
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
20
Analysis and Design of
Algorithms
C
P
C
S
Exhaustive Search
Knapsack Problem
2
2
3
Instructor Dr. Muhammad Umair
21
Analysis and Design of
Algorithms
Knapsack Problem
• Given n items of known weights
w1, w2, . . . , wn and values v1, v2,
. . . , vn and a knapsack of capacity
W, find the most valuable subset
of the items that fit into the
knapsack
Instructor Dr. Muhammad Umair
22
C
P
C
S
2
2
3
Analysis and Design of
Algorithms
Knapsack Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
23
Analysis and Design of
Algorithms
Knapsack Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
24
Analysis and Design of
Algorithms
Knapsack Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
25
Analysis and Design of
Algorithms
Knapsack Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
26
Analysis and Design of
Algorithms
Knapsack Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
27
Analysis and Design of
Algorithms
Knapsack Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
28
Analysis and Design of
Algorithms
Knapsack Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
29
Analysis and Design of
Algorithms
Assignment Problem
• Input: There are n people and n jobs
• Criteria: each job is assigned to
exactly one person
• Cost: i th person is assigned to the jth
job is a known quantity C[i, j ]
• Output: Assign jobs with the
minimum total cost.
Instructor Dr. Muhammad Umair
30
C
P
C
S
2
2
3
Analysis and Design of
Algorithms
Assignment Problem (Solution)
• It will be described by
–
–
–
–
–
<j1, j2, …, jn>
J1 job is assigned to person 1
J2 job is assigned to person 2
………
Jn job is assigned to person n
Instructor Dr. Muhammad Umair
C
P
C
S
2
2
3
31
Analysis and Design of
Algorithms
Assignment Problem (Solution)
• It will be described by
–
–
–
–
–
<j1, j2, …, jn>
J1 job is assigned to person 1
J2 job is assigned to person 2
………
Jn job is assigned to person n
• <2, 3, 4, 1> means job 2 is assigned to person
1, job 3 is assigned to 2 and so on
Instructor Dr. Muhammad Umair
32
C
P
C
S
2
2
3
Analysis and Design of
Algorithms
Assignment Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
33
Analysis and Design of
Algorithms
Assignment Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
34
Analysis and Design of
Algorithms
Assignment Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
35
Analysis and Design of
Algorithms
Assignment Problem
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
36
Analysis and Design of
Algorithms
C
P
C
S
Exhaustive Search
Graph Traversal Algorithms
2
2
3
Instructor Dr. Muhammad Umair
37
Analysis and Design of
Algorithms
Introduction
C
P
C
S
• Many problems require processing
all graph vertices (and edges) in
systematic fashion
• Graph Traversal Algorithms:
2
2
3
– Depth-first search (DFS)
– Breadth-first search (BFS)
Instructor Dr. Muhammad Umair
38
Analysis and Design of
Algorithms
C
P
C
S
Depth First Search
DFS
Graph Traversal Algorithms
2
2
3
Instructor Dr. Muhammad Umair
39
Analysis and Design of
Algorithms
DFS
• Visits graph’s vertices by always moving away
from last visited vertex to unvisited one,
backtracks if no adjacent unvisited vertex is
available.
• Uses a stack
• a vertex is pushed onto the stack when it’s
reached for the first time
• a vertex is popped off the stack when it
becomes a dead end, i.e., when there is no
adjacent unvisited vertex
• “Redraws” graph in tree-like fashion
(with tree edges and back edges for
undirected graph)
Instructor Dr. Muhammad Umair
40
C
P
C
S
2
2
3
Analysis and Design of
Algorithms
DFS
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
41
Analysis and Design of
Algorithms
DFS (Example)
Pop-off Sequence:
C
P
C
S
2
2
3
Traversal Sequence:
Instructor Dr. Muhammad Umair
42
Analysis and Design of
Algorithms
DFS (Example)
C
P
C
S
• Start with “d”
Pop-off Sequence:
2
2
3
Traversal Sequence:
d,
Instructor Dr. Muhammad Umair
43
Analysis and Design of
Algorithms
DFS (Example)
C
P
C
S
• Start with “d”
• d  (a, b, c, f, g)
Pop-off Sequence:
2
2
3
Traversal Sequence:
d,
Instructor Dr. Muhammad Umair
44
Analysis and Design of
Algorithms
DFS (Example)
C
P
C
S
• Start with “d”
• d  (a, b, c, f, g)
• a  (b, c)
Pop-off Sequence:
2
2
3
Traversal Sequence:
d, a,
Instructor Dr. Muhammad Umair
45
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
Pop-off Sequence:
2
2
3
Traversal Sequence:
d, a, b,
Instructor Dr. Muhammad Umair
46
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
Pop-off Sequence:
2
2
3
Traversal Sequence:
d, a, b, e,
Instructor Dr. Muhammad Umair
47
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
Pop-off Sequence:
2
2
3
Traversal Sequence:
d, a, b, e,
Instructor Dr. Muhammad Umair
48
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g,
Instructor Dr. Muhammad Umair
49
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f,
Instructor Dr. Muhammad Umair
50
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f,
Instructor Dr. Muhammad Umair
51
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f,
Instructor Dr. Muhammad Umair
52
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f,
Instructor Dr. Muhammad Umair
53
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
c  (f)
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f, c
Instructor Dr. Muhammad Umair
54
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
c  (f)
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f, c
Instructor Dr. Muhammad Umair
55
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
c  (f)
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f, c
Instructor Dr. Muhammad Umair
56
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
•
•
•
•
•
•
•
•
C
P
C
S
Start with “d”
d  (a, b, c, f, g)
a  (b, c)
b  (e, g)
e*
g  (e, f)
f*
c  (f)
Pop-off Sequence:
Traversal Sequence:
d, a, b, e, g, f, c
Instructor Dr. Muhammad Umair
57
2
2
3
Analysis and Design of
Algorithms
DFS (Example)
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
58
Analysis and Design of
Algorithms
Count
0
1
2
3
4
5
6
7
8
9
Vertex
a
b
c
d
e
f
g
h
i
j
Traverse
01
50
2
0
03
0
6
40
07
0
8
90
0
10
10
Pop
C
P
C
j
i
S
h
g
e
2
f2
d
c3
b
Traversal: a, c, d, f, b, e, g, h, i, j
PopOff: d e b f c a j i h g
59
Instructor Dr. Muhammad Umair
a
Analysis and Design of
Algorithms
•
DFS
DFS can be implemented with graphs represented as:
Θ(V2)
– adjacency matrices:
– adjacency lists: Θ(|V|+|E|)
• Yields two distinct ordering of vertices:
– order in which vertices are first encountered (pushed
onto stack)
– order in which vertices become dead-ends (popped off
stack)
C
P
C
S
• Applications:
–
–
–
–
checking connectivity, finding connected components
checking acyclicity
finding articulation points and biconnected components
searching state-space of problems for solution (AI)
Instructor Dr. Muhammad Umair
60
2
2
3
Analysis and Design of
Algorithms
C
P
C
S
Breadth First Search
BFS
Graph Traversal Algorithms
2
2
3
Instructor Dr. Muhammad Umair
61
Analysis and Design of
Algorithms
BFS
• Visits graph vertices by moving across to
all the neighbors of last visited vertex
• Instead of a stack, BFS uses a queue
C
P
C
S
• Similar to level-by-level tree traversal
• “Redraws” graph in tree-like fashion
(with tree edges and cross edges for
undirected graph)
Instructor Dr. Muhammad Umair
62
2
2
3
Analysis and Design of
Algorithms
BFS
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
63
Analysis and Design of
Algorithms
BFS (Example)
C
P
C
S
2
2
3
Instructor Dr. Muhammad Umair
64
Analysis and Design of
Algorithms
Final Comments on
Exhaustive Search
• Exhaustive-search algorithms run in a realistic
amount of time only on very small instances
• In some cases, there are much better alternatives!
–
–
–
–
Euler circuits
shortest paths
minimum spanning tree
assignment problem
• In many cases, exhaustive search or its variation is
the only known way to get exact solution
Instructor Dr. Muhammad Umair
65
C
P
C
S
2
2
3
Analysis and Design of
Algorithms
C
P
C
S
Allah (Alone) is Sufficient for us, and He is the
Best Disposer of affairs (for us).(Quran 3 : 173)
Instructor Dr. Muhammad Umair
2
2
3