Analysis & Design of
Algorithms
(CSCE 321)
Prof. Amr Goneid
Department of Computer Science, AUC
Part 12. Graph Algorithms
Prof. Amr Goneid, AUC
1
Graph Algorithms
Prof. Amr Goneid, AUC
2
Graph Algorithms
 Graph Traversal: DFS and BFS
 A Simple Graph Class
 Minimum Cost Spanning Trees (MST)
 Single Source Shortest Paths
 All Pairs Shortest Paths
 Transitive Closure
 Graph Coloring
 Topological Sorting
 The Clique Problem
Prof. Amr Goneid, AUC
3
1. Graph Traversal
Depth-First Search (DFS)
 Exhaustive Search Algorithm
 Visits every node in the graph by following
node connections in depth.
 Recursive algorithm.
 An Array val[v] records the order in which
vertices are visited. Initialized to “unseen”.
 Any edge to a vertex that has not been seen
is followed via the recursive call.
Prof. Amr Goneid, AUC
4
DFS Algorithm
// Assume No. of vertices = v and order = 0 initially
// val[1..v] is an array recording the visit order
void DFS()
{
// Initialize all to unseen
for each vertex k set val[k] = unseen;
// Follow Nodes in Depth
for each vertex k
if (val[k] == unseen) Visit(k);
}
Prof. Amr Goneid, AUC
5
DFS Algorithm (continued)
//Assume an adjacency matrix a[1..v][1..v]
Visit(k)
{
val[k] = ++order;
for each vertex (t), t =1..v
if (vertices (t) and (k) are adjacent)
if (val[t] == unseen) Visit(t);
}
Prof. Amr Goneid, AUC
6
Example
1
A
start
A
B
C
D
E
F
G
A
0
1
1
0
0
1
1
B
1
0
0
0
0
0
0
C
1
0
0
0
0
0
0
VAL[K]
D
0
0
0
0
1
1
0
E
0
0
0
1
0
1
1
F
1
0
0
1
1
0
0
G
1
0
0
0
1
0
0
B
C
D
F
G
E
A B C D E F G
1
Prof. Amr Goneid, AUC
7
Example
1
A
A
B
C
D
E
F
G
A
0
1
1
0
0
1
1
B
1
0
0
0
0
0
0
C
1
0
0
0
0
0
0
VAL[K]
D
0
0
0
0
1
1
0
E
0
0
0
1
0
1
1
F
1
0
0
1
1
0
0
G
1
0
0
0
1
0
0
2 B
C
D
F
G
E
A B C D E F G
1 2
Prof. Amr Goneid, AUC
8
Example
1
A
A
B
C
D
E
F
G
A
0
1
1
0
0
1
1
B
1
0
0
0
0
0
0
C
1
0
0
0
0
0
0
VAL[K]
D
0
0
0
0
1
1
0
E
0
0
0
1
0
1
1
F
1
0
0
1
1
0
0
G
1
0
0
0
1
0
0
2 B
3
C
D
F
G
E
A B C D E F G
1 2 3
Prof. Amr Goneid, AUC
9
Example
1
A
A
B
C
D
E
F
G
A
0
1
1
0
0
1
1
B
1
0
0
0
0
0
0
C
1
0
0
0
0
0
0
VAL[K]
D
0
0
0
0
1
1
0
E
0
0
0
1
0
1
1
F
1
0
0
1
1
0
0
G
1
0
0
0
1
0
0
2 B
3
C
D
4 F
G
E
A B C D E F G
1 2 3
4
Prof. Amr Goneid, AUC
10
Example
1
A
A
B
C
D
E
F
G
A
0
1
1
0
0
1
1
B
1
0
0
0
0
0
0
C
1
0
0
0
0
0
0
VAL[K]
D
0
0
0
0
1
1
0
E
0
0
0
1
0
1
1
F
1
0
0
1
1
0
0
G
1
0
0
0
1
0
0
2 B
3
5
C
D
4 F
G
E
A B C D E F G
1 2 3 5
4
Prof. Amr Goneid, AUC
11
Example
1
A
A
B
C
D
E
F
G
A
0
1
1
0
0
1
1
B
1
0
0
0
0
0
0
C
1
0
0
0
0
0
0
VAL[K]
D
0
0
0
0
1
1
0
E
0
0
0
1
0
1
1
F
1
0
0
1
1
0
0
G
1
0
0
0
1
0
0
2 B
3
5
C
D
4 F
G
E
6
A B C D E F G
1 2 3 5 6 4
Prof. Amr Goneid, AUC
12
Example
1
A
A
B
C
D
E
F
G
A
0
1
1
0
0
1
1
B
1
0
0
0
0
0
0
C
1
0
0
0
0
0
0
VAL[K]
D
0
0
0
0
1
1
0
E
0
0
0
1
0
1
1
F
1
0
0
1
1
0
0
G
1
0
0
0
1
0
0
7
2 B
3
5
C
D
4 F
G
E
6
A B C D E F G
1 2 3 5 6 4 7
Prof. Amr Goneid, AUC
13
Exercises
 Show that the complexity of DFS for adjacency matrix
representation is T(V) = O(V2)
 Show that the complexity of DFS for adjacency list
representation is T(V) = O(V+E)
 Show how DFS can be used to:
1. Determine number of connected components in
a graph.
2. Test if a graph has a cycle.
3. Find the number of edges in a graph.
 Explore the non-recursive version of the DFS using a
stack. What will happen if you use a queue instead of
the stack?
Prof. Amr Goneid, AUC
14
Non-Recursive DFS
// Assume hold = node discovered but not yet visited
// Val[1..v ] is set to “unseen” initially , order = 0
// Assume a stack, initially empty
DFS(k)
{
push (k) on top of stack;
while stack is not empty
{
pop stack top into (k); val[k] = ++order;
for vertices t = v..1 // Scan from right to left
if (vertices (t) and (k) are adjacent))
if (val[t] == unseen)
{ push (t) on top of stack; val[t] = hold;}
}
}
Prof. Amr Goneid, AUC
15
Breadth-First Search (BFS)
// Replacing the stack by a queue, gives the BFS algorithm
// Also an Exhaustive Search Algorithm
// Assume a queue, intially empty
BFS(k)
{
add (k) to end of queue;
while queue is not empty
{
remove queue front into (k); val[k] = ++order;
for vertices t = 1..v // Scan from left to right
if (vertices (t) and (k) are adjacent))
if (val[t] == unseen)
{ add (t) to end of queue; val[t] = hold;}
}
}
Prof. Amr Goneid, AUC
16
Example BFS
1
A
2 B
3
6
C
D
4 F
5 G
E
7
A B C D E F G
1 2 3 6 7 4 5
Prof. Amr Goneid, AUC
17
DFS and BFS of a Tree
 BFS Complexity:
T(V) = O(V2) for adjacency matrix representation.
T(V) = O(V+E) for adjacency list representation.
 BFS Applications
Same as DFS, but can also find paths from a vertex
to all other vertices with the smallest number of
edges
 For a tree structure:
DFS is equivalent to Pre-order traversal
BFS is equivalent to Level-order traversal
Prof. Amr Goneid, AUC
18
Pre-Order Traversal
 DFS is also an Unbound Backtracking algorithm.
1
2
3
5
4
6
Prof. Amr Goneid, AUC
7
19
Graph Traversal Demos
DFS Demo
BFS Demo
http://www.cs.sunysb.edu/~skiena/combinat
orica/animations/search.html
http://www.cosc.canterbury.ac.nz/people/mu
kundan/dsal/GraphAppl.html
Prof. Amr Goneid, AUC
20
Exercise
 Model the shown maze
in
as a graph.
 Show how DFS can be
used to find the exit.
Prof. Amr Goneid, AUC
out
21
2. A Simple Graph Class
 To represent a weighted undirected graph with a maximum of
Vmax vertices and Emax = Vmax(Vmax-1)/2 edges. The verices are
numbered 0,1,...V-1.
 The graph is assumed to be on a text file in the form of an
adjacency matrix. The weights on the edges are assumed to be
positive integers with zero weight indicating the absence of an
edge.
 When loaded from the text file, the weights are stored in a 2-D
array (AdjMatrix) representing the adjacency matrix. Another
array (edges) stores the non-zero edges in the graph.
 An edge (u,v,w) existing between nodes (u) and (v) with weight
(w) is modeled as a class (Edge).
Prof. Amr Goneid, AUC
22
Edge Class
// File: Edge.h
// Definition of Edge class
#ifndef EDGE_H
#define EDGE_H
typedef int weightType;
// weights are positive integers
class Edge
{
public:
int u,v; weightType w;
bool operator < (const Edge &e) { return (w < e.w); }
bool operator <= (const Edge &e) { return (w <= e.w); }
}; // end of class Edge declaration
#endif // EDGE_H
Prof. Amr Goneid, AUC
23
Graph Class
// File: Graphs.h
// Graph library header file
#ifndef GRAPHS_H
#define GRAPHS_H
#include <string>
#include "Edge.h"
using namespace std;
const int Vmax = 50;
// Maximum number of vertices
const int Emax = Vmax*(Vmax-1)/2; // Maximum number of edges
Prof. Amr Goneid, AUC
24
Graph Class
class Graphs
{
public:
Graphs();
// Constructor
~Graphs(); // Destructor
// Map vertex number to a name (character)
char Vname(const int s) const;
void getGraph(string fname); // Get Graph from text File (fname)
void dispGraph( ) const;
// Display Ajacency Matrix
int No_of_Verices( ) const;
// Get number of vertices (V)
int No_of_Edges( ) const;
// Get Number of Non-zero edges (E)
void dispEdges( ) const;
// Display Graph edges
void DFS( );
// Depth First Search Traversal (DFS)
Prof. Amr Goneid, AUC
25
Graph Class
private:
int V, E;
// No.of vertices (V) and edges (E)
weightType AdjMatrix[Vmax][Vmax]; // Adjacency Matrix
Edge edges[Emax];
// Array of non-zero edges
int order;
// Order of Visit of a node in the DFS
int val[Vmax];
// Array holding order of traversal
void getEdges();
// Get edges from adjacency matrix
void printEdge(Edge e) const; // Output an edge (e)
void visit(int k);
// Node Visit Function for DFS
};
#endif // GRAPHS_H
#include "Graphs.cpp"
Prof. Amr Goneid, AUC
26
Graph Class
The CSCI 321 web site contains an implementation of the
simple graph class “Graphs”
Prof. Amr Goneid, AUC
27
3. Minimum Cost Spanning Tree (MST)
Consider houses A..F connected by
muddy roads with the distances
4
indicated. We want to pave some
E
A
roads such that:
7
3 3
 We can reach a house from
any other house via paved roads.
C
B
 The cost of paving is minimum.
This problem is an example of
6
4
finding a Minimum Spanning Tree
D
(MST)
Prof. Amr Goneid, AUC
4
2
G
9
F
5
28
Minimum Spanning Tree (MST)
 Cost:
For a weighted graph,
4
E
A
the cost of a spanning tree
7
is the sum of the weights
3 3
of the edges in that tree.
C
B
 Minimum Spanning tree:
4
A spanning tree of minimum cost 6
 For the shown graph, the
D
minimum cost is 22
Prof. Amr Goneid, AUC
4
2
G
9
F
5
29
Kruskal’s Algorithm for MST
 The algorithm was written by
Joseph Kruskal in 1956
 A Greedy Algorithm:
Builds the MST edge by edge into a set of edges (T). At a given
stage, chooses an edge that results in minimum increase in the
sum of costs included so far in (T).
The set (T) might not be a tree at all stages of the algorithm, but
it can be completed into a tree iff there are no cycles in (T).
Prof. Amr Goneid, AUC
30
Kruskal’s Algorithm for MST
 Builds up forests , then joins them in a
single tree.
 Constraints:
- The graph must be connected.
- Uses exactly V-1 edges.
- Excludes edges that form a cycle.
Prof. Amr Goneid, AUC
31
Abstract Algorithm
Form Set E of edges in increasing order of costs.
Set MST T = empty
Repeat
Select an edge (e) from top.
Delete edge from E set.
Add edge (e) to (T) if it does not form a cycle, otherwise,
reject.
Until we have V-1 successful edges.
Prof. Amr Goneid, AUC
32
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w
2
3
3
4
4
4
5
6
accept
4
E
A
7
3
2
C
B
Prof. Amr Goneid, AUC
3
4
6
4
G
9
F
5
D
33
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w accept
2
yes
3
3
4
4
4
5
6
4
E
A
7
3
2
C
B
Prof. Amr Goneid, AUC
3
4
6
4
G
9
F
5
D
34
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w accept
2
yes
3
yes
3
4
4
4
5
6
4
E
A
7
3
2
C
B
Prof. Amr Goneid, AUC
3
4
6
4
G
9
F
5
D
35
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w accept
2
yes
3
yes
3
yes
4
4
4
5
6
4
E
A
7
3
2
C
B
Prof. Amr Goneid, AUC
3
4
6
4
G
9
F
5
D
36
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w
2
3
3
4
4
4
5
6
accept
yes
yes
yes
4
E
A
7
3
NO
2
C
B
Prof. Amr Goneid, AUC
3
4
6
4
G
9
F
5
D
37
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w
2
3
3
4
4
4
5
6
accept
yes
yes
yes
4
7
3
NO
yes
E
A
2
C
B
Prof. Amr Goneid, AUC
3
4
6
4
G
9
F
5
D
38
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w
2
3
3
4
4
4
5
6
accept
yes
yes
yes
4
7
3
NO
yes
yes
E
A
2
C
B
Prof. Amr Goneid, AUC
3
4
6
4
G
9
F
5
D
39
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w
2
3
3
4
4
4
5
6
accept
yes
yes
yes
4
7
3
NO
yes
yes
E
A
Prof. Amr Goneid, AUC
2
C
B
No
3
4
6
4
G
9
F
5
D
40
Example
Edge
1
2
3
4
5
6
7
8
u
E
A
C
A
C
E
D
B
v
F
C
E
E
D
G
F
D
w
2
3
3
4
4
4
5
6
accept
yes
yes
yes
4
7
3
NO
yes
yes
E
A
2
C
B
NO
3
4
6
4
G
9
F
5
D
yes
Prof. Amr Goneid, AUC
41
How to test for Cycles?
Simple Union- Simple Find
Given a set (1,2,..,n} initially partitioned into n
disjoint sets (each element is its own parent):
 Find (i): return the sub-set that (i) is in (i.e.
return the parent set of i).
 Union (k,j): combine the two sub-sets that (j)
and (k) are in (make k the child of j)
 Union builds up a tree, while find will search
for the root of the tree.
 Cost is O(log n)
Prof. Amr Goneid, AUC
42
How to test for Cycles?
 Represent vertices as Disjoint Sets
A
B
C
D
E
F
G
 Initially, each node is its own parent (-1)
A B C D E F G
1 2 3 4 5 6 7
-1 -1 -1 -1 -1 -1 -1
Prof. Amr Goneid, AUC
Let the parent set
of u be p(u)
43
How to test for Cycles?
 When edge (E,F) is selected, we find that p(E)  p(F). So,
we make a union between p(E) and p(F), i.e., p(F) becomes
the child of p(E). Same for edge (A,C)
A
B
C
D
A B C D E F
1 2 3 4 5 6
-1 -1 1 -1 -1 5
E
G
7
-1
Prof. Amr Goneid, AUC
F
G
Parent table after
selecting (E,F)
then (A,C)
44
How to test for Cycles?
When (C,E) is selected, we find that P(C)  P(E). This means that
the edge will be accepted.
 Select (A,E), Find will give P(A) = P(E) (cycle, reject).

D
A
B
E
C
A B C D E F
1 2 3 4 5 6
-1 -1 1 -1 3 5
G
7
-1
Prof. Amr Goneid, AUC
F
G
Parent table after
accepting (E,F) then
(A,C), then (C,E),
then rejecting (A,E)
45
How to test for Cycles?
When (C,D) is selected, we find that P(C)  P(D). This means that
the edge will be accepted.
 Select (E,G), Find will give P(E)  P(G) (accept).

G
D
A
B
E
C
A B C D E F
1 2 3 4 5 6
-1 -1 1 3 3 5
G
7
5
F
Parent table after 5th
accepted edge
Prof. Amr Goneid, AUC
46
How to test for Cycles?

When (D,F) is selected, we find that P(D) = P(F). This means that
the edge will be rejected.
 Select (B,D), Find will give P(B)  P(D) (accept).
G
D
A
B
E
C
A B C D E F
1 2 3 4 5 6
-1 4 1 3 3 5
G
7
5
F
Parent table after 6th
accepted edge
(Final MST)
Prof. Amr Goneid, AUC
47
Kruskal’s Algorithm
Insert edges with weights into a minimum heap
Put each vertex in a separate set
i=0;
While ((i < V-1) && (heap not empty)) {
Remove edge (u,v) from heap
Find set (j) of connected vertices having (u)
Find set (k) of connected vertices having (v)
if ( j != k ) {
i++; MST [i].u = u; MST [i].v = v;
MST [i].w = w;
9.
Make a Union between set (j) and set (k); }
}
1.
2.
3.
4.
5.
6.
7.
8.
Prof. Amr Goneid, AUC
48
Kruskal’s Algorithm Demo
http://www.cs.sunysb.edu/~skiena/combinatorica/
animations/mst.html
Kruskal's algorithm at work on a graph of distances
between 128 North American cities. Almost imperceptively
at first, short edges get added all around the continent,
slowly building forests until the tree is completed.
MST (Kruskal) Demo1
MST (Kruskal) Demo2
Prof. Amr Goneid, AUC
49
Analysis of Kruskal’s Algorithm
Line(1): generation of the minimum heap O(E log E)
Line(2): O(V)
Line(3): O(1)
Line(5): O(V log E) = O(V log V) since E = O(V2)
Lines(6),(7): O(V log V)
Lines(8),(9): O(V)
Total: O(V) + O(V log V) + O(E log E)
The algorithm is dominated by the generation of the
minimum heap. Hence,
T(n) = O(E log V)
Prof. Amr Goneid, AUC
50
Prim’s Algorithm for MST
 Prim’s algorithm for Minimum Spanning Trees. The
algorithm was developed in 1930 by Czech mathematician
Vojtěch Jarník
and later independently by
computer scientist
Robert C. Prim in 1957
and rediscovered by
Edsger Dijkstra in 1959. Therefore it is also sometimes
called the DJP algorithm, the Jarník algorithm, or the Prim–
Jarník algorithm.
Prof. Amr Goneid, AUC
51
Prim’s Algorithm
 Choose any starting vertex.
 At any stage, the set of selected edges form a
tree
 If A is the set of edges selected so far, then A
is a tree. The next edge (u,v) is a minimum
cost edge not in A such that
A  {(u,v)} is also a tree.
Prof. Amr Goneid, AUC
52
Prim’s Algorithm
// Given a connected graph, a set of vertices V, and a tree T
set T = {empty} , V = {first vertex} , done = false
while (T contains less than V-1 edges and not done) {
 choose an edge (u,v) with (u) in V and (v) not in V
such that (u,v) is of least cost ;
 if there is no such edge then done = true
else { V ← V  {v} , T ← T {(u,v)} };
}
Prof. Amr Goneid, AUC
53
Example
a
a
1
6
b
3
2
b
d
d
3
a
a
1
6
1
6
2
2
b
b
c
4
c
4
3
d
1
6
2
2
c
4
a
1
6
c
4
c
4
b
3
Prof. Amr Goneid, AUC
d
3
d
Prim’s Algorithm
 Explore the differences between Prim’s
Algorithm and Kruskal’s Algorithm
 Make an analysis of the complexity of Prim’s
Algorithm and show that it is O(E log V)
 Prim's Algorithm Demo
Prof. Amr Goneid, AUC
55
4. Single Source Shortest
Paths (General)
 In a graph G(V,E), find shortest paths from a
single source vertex (S) to all other vertices.
 Edsger Dijkstra published an algorithm to
solve this problem in 1959.
Prof. Amr Goneid, AUC
56
Shortest Paths:
Dijkstra’s Algorithm
 Dijkstra’s Algorithm for V vertices:
Uses three arrays:
- Distance[i]: holds distance from (S) to
vertex (i).
- Processed[i]: to flag processed vertices.
- Via[i]: holds index of vertex from which we
can directly reach vertex (i).
Prof. Amr Goneid, AUC
57
Initialization
 Distance[i]:
= 0
if S = i
= Wsi
if (S, i) are adjacent
= 
otherwise
 Processed[i] = yes if i = S , No otherwise
 Via[i] = S if (i , S) are adjacent, 0 otherwise
Prof. Amr Goneid, AUC
58
Method
closest to S
not yet processed
Distance[j]
j
Already Processed
Wij
S
z
y
x
Distance[i]
i
adjacent to j
Prof. Amr Goneid, AUC
59
Dijkstra’s Algorithm
(Greedy Algorithm)
Repeat
 Find j = index of unprocessed node closest to (S)
 Mark (j) as now processed
 For each node (i) not yet processed:
if (i) is adjacent to (j) then {
new_distance = Distance[j] + Wij
if new_distance < Distance[i] then
{ Distance[i] = new_distance ;
Via[i] = j ; }
}
Until all vertices are processed
Prof. Amr Goneid, AUC
60
Example
Initial Source = A
D
E
0 15 35 
20
Dist
Processed
A
B
yes No
0
A
C
No
No
No
A
0
A
15
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
61
Example
j=B
D
E
0 15 35 
20
Dist
No
No
Processed
0
A
A
B
C
yes yes No
0
A
A
15
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
62
Example
j=B
A
i=D
B
C
D
E
0 15 35 55 20
yes yes No
0
A
A
No
No
B
A
Dist
15
A
20
B
Processed
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
63
Example
j=E
A
B
C
D
E
0 15 35 55 20
yes yes No
0
A
A
No
B
Dist
15
yes Processed
A
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
64
Example
j=E
A
i=C
B
C
D
E
0 15 30 55 20
yes yes No
0
A
E
No
B
Dist
15
yes Processed
A
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
65
Example
j=C
A
B
C
D
E
0 15 30 55 20
yes yes yes No
0
A
E
B
Dist
15
yes Processed
A
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
66
Example
j=C
A
i=D
B
C
D
E
0 15 30 55 20
yes yes yes No
0
A
E
B
Dist
15
yes Processed
A
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
67
Example
j=D
A
B
C
D
E
0 15 30 55 20
Dist
15
yes yes yes yes yes Processed
0
A
E
B
A
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
68
Example
Final
A
B
C
D
E
0 15 30 55 20
Dist
15
yes yes yes yes yes Processed
0
A
A→B
A→E→C
A→B→D
A→E
E
B
A
A
20
B
E
35
Via
40
D
Prof. Amr Goneid, AUC
10
35
C
69
How to print the path?
/* The function Vname(k) maps a vertex number to a name (e.g
a character A, B,.. etc). Given the via[ ] array resulting from
Dijkstra’s algorithm, the following recursive function prints
the vertices on the shortest path from source (s) to
destination (i).
*/
void Graphs::printPath(int s, int i) const
{
if (i == s) cout << Vname(s);
else
{
printPath(s,via[i]); cout << Vname(i); }
}
Prof. Amr Goneid, AUC
70
Demo
http://www.cs.sunysb.edu/~skiena/com
binatorica/animations/dijkstra.html
Shortest Paths Demo1
Shortest Paths Demo2
Prof. Amr Goneid, AUC
71
Analysis
 The repeat loop will be done V-1 times
 Finding the node (j) closest to (S) and
not yet processed will cost O(V)
 Updating distances to all nodes
adjacent to (j) and not yet processed will
cost O(V)
 Hence, Dijkstra’s algorithm has a
complexity of O(V2)
Prof. Amr Goneid, AUC
72
5. All Pairs Shortest Paths
(Floyd’s Algorithm: Dynamic Programming)
 Given a weighted directed graph G = (V,E) with n
vertices.
 Find for each pair of vertices (i,j)  V the cost of the
shortest path from i to j.
 Obviously, we can solve this problem by running
Dijkstra’s algorithm (n) times, once for each node
as a source vertex. The complexity would be O(n3).
Prof. Amr Goneid, AUC
73
Floyd’s Algorithm
 An alternate algorithm (Floyd’s Algorithm) uses
dynamic programming and also has a complexity
of O(n3)
 The algorithm was developed by Robert Floyd
(1962)
 Because it has a similarity with
Warshall’s algorithm, it is sometimes
called Floyd-Warshall algorithm.
Prof. Amr Goneid, AUC
74
Example
A0(i,j) = Cost Matrix
0
15
35
Inf
20
15 35 Inf 20
0 Inf 40 Inf
Inf 0 35 10
40 35 0 Inf
Inf 10 Inf 0
15
A
20
B
E
35
A5(i,j) Final Solution
0 15 30 55 20
15 0 45 40 35
30 45 0 35 10
55 40 35 0 45
20 35 10 45 0
40
D
Prof. Amr Goneid, AUC
10
35
C
75
All Pairs Shortest Paths
 Let A be an (nxn) matrix such that Ak(i,j) is the
length of the shortest path from i to j with internal
nodes numbered ≤ k. Hence, the final solution is
An(i,j).
 A dynamic programming approach would first set
A0(i,j) then computes A1(i,j) , A2(i,j), etc.
 For this purpose, let:
0

0
A ( i , j )  wij


if i  j
if i  j and (i, j)  E
if i  j and (i, j)  E
Prof. Amr Goneid, AUC
76
All Pairs Shortest Paths
 A shortest path from i to j with internal vertices no
higher than k either goes through vertex k or it does
not.
i
vertices numbered
at most k-1
j
k
Prof. Amr Goneid, AUC
77
All Pairs Shortest Paths
 If it does not pass through k, then there will be no
change from the previous cost, i.e.,
Ak(i,j) = Ak-1(i,j)
 If it goes through k then
Ak(i,j) = Ak-1(i,k) + Ak-1(k,j)
 On the k-th iteration, the algorithm determines
shortest paths between every pair of vertices i, j
that use only vertices among 1,…,k as intermediate.
Prof. Amr Goneid, AUC
78
All Pairs Shortest Paths
 Hence,
Ak(i,j) = min { Ak-1(i,j) , Ak-1(i,k) + Ak-1(k,j) } , k ≥ 1
A(k-1)(i,k)
k
i
A(k-1)(k,j)
A(k-1)(i,j)
j
Prof. Amr Goneid, AUC
79
All Pairs Shortest Paths
 Prove that Ak(k,j) = Ak-1(k,j)
 Proof:
zero
Ak(k,j) = min { Ak-1(k,j) , Ak-1(k,k) + Ak-1(k,j) }
the same
Hence, Ak(k,j) = Ak-1(k,j)
 Similarly, we can prove that
Ak(i,k) = Ak-1(i,k)
 Hence, the kth row and kth column do not change
and we can do the computation in-place using the
same matrix
Prof. Amr Goneid, AUC
80
All Pairs Shortest Paths
(Algorithm)
void AllPaths (float cost[ ][N], float A[ ][N], int p[ ][N], int n)
// cost[1:n][1:n] is the cost adjacency matrix of a graph with n
// vertices;
// cost[i][i] = 0.0, cost[i][j] = wij for i  j and (i,j)  E,
// and cost[i][j] =  if (i,j)  E
// A[i][j] is the cost of a shortest path from vertex i to vertex j.
// p[i][j] holds the indices of the nodes in the shortest paths.
{
for (int i=1; i<=n; i++)
for (int j=1; j<=n; j++)
{ A[i][j] = cost[i][j]; // Copy cost into A to get A0(i,j)
p[i][j] = 0;
}
Prof. Amr Goneid, AUC
81
All Pairs Shortest Paths
(Algorithm)
for (int k=1; k<=n; k++)
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
if (A[i, k] + A[k, j] < A[i, j])
{
A[i, j] = A[i, k] + A[k, j];
p[i][j] = k;
}
}
Analysis:
It is easy to see that the complexity of Floyd’s algorithm is
(n3)
Prof. Amr Goneid, AUC
82
Example
A0(i,j) = Cost Matrix
0
15
35
Inf
20
15 35 Inf 20
0 Inf 40 Inf
Inf 0 35 10
40 35 0 Inf
Inf 10 Inf 0
A1(i,j)
0 15 35
15 0 50
35 50 0
Inf 40 35
20 35 10
15
A
20
B
E
35
40
Inf
40
35
0
Inf
20
35
10
Inf
0
D
Prof. Amr Goneid, AUC
10
35
C
83
Example
A2(i,j)
0
15
35
55
20
15 35 55 20
0 50 40 35
50 0 35 10
40 35 0 75
35 10 75 0
15
20
B
E
35
A3(i,j)
0 15 35 55 20
15 0 50 40 35
35 50 0 35 10
55 40 35 0 45
20 35 10 45 0
A
40
D
Prof. Amr Goneid, AUC
10
35
C
84
Example
A4(i,j)
0 15 35 55
15 0 50 40
35 50 0 35
55 40 35 0
20 35 10 45
20
35
10
45
0
15
A
20
B
E
35
A5(i,j) Final Solution
0 15 30 55 20
15 0 45 40 35
30 45 0 35 10
55 40 35 0 45
20 35 10 45 0
40
D
Prof. Amr Goneid, AUC
10
35
C
85
All Pairs Shortest Paths
(Listing nodes on shortest paths)
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
if (A[i][j] < )
{ cout << i <<” ”; between(i, j); cout << i <<” ”; }
void between (int i, int j)
{
k = p[i, j];
if (k > 0) { between(i, k); cout << k << ” ”; between(k, j); }
}
Prof. Amr Goneid, AUC
86
6. Transitive Closure
(Warshall’s Algorithm: Dynamic Programming)
 Problem: Given a directed graph G = (V,E), find for
each pair of vertices i,j  V whether there is a path
from i to j.
 Solution: make the cost of all edges 1, and run
Floyd’s algorithm. If on termination A[i, j]  , then
there is a path from i to j.
Prof. Amr Goneid, AUC
87
Warshall’s Algorithm
 A better solution: use Boolean values to obtain
the Transitive Closure (Reachability Matrix) of G.
 The transitive closure is a matrix A* such that
A*(i,j) = 1 iff there is a directed path from vertex i to
vertex j.
 The algorithm has been developed by Stephen
Warshall (1962).
Prof. Amr Goneid, AUC
88
Transitive Closure
3
3
1
1
4
2
0
1
0
0
0
0
0
1
1
0
0
0
4
2
0
1
0
0
0
1
0
1
Prof. Amr Goneid, AUC
0
1
0
1
1
1
0
1
0
1
0
1
89
Warshall’s Algorithm
 Constructs transitive closure T as the last matrix in the
sequence of n-by-n matrices A0, … , Ak, … , An where Ak(i,j)
= 1 iff there is nontrivial path from i to j with only the first k
vertices allowed as intermediate
Note that A0 = (adjacency matrix), An = T (transitive closure)
 Main idea: a path exists between two vertices i, j, iff
• there is an edge from i to j; or
• there is a path from i to j going through vertex 1; or
• there is a path from i to j going through vertex 1 and/or 2; or
• there is a path from i to j going through vertex 1, 2, and/or 3;
or
•...
• there is a path from i to j going through any of the other
vertices
Prof. Amr Goneid, AUC
90
Warshall’s Algorithm
In the kth stage determine if a path exists between two
vertices i, j using just vertices among 1,…,k

A k 1 ( i , j )

k
A ( i, j )  
OR
 A k  1 ( i , k ) AND A k  1 ( k , j )

k
i
kth stage
j
Prof. Amr Goneid, AUC
91
Warshall’s Algorithm
Set A matrix to all false;
for (int i=1; i<=n; i++)
{
for (int j=1; j<=n; j++)
if ((i,j)  E) A[i][j] = true;
A[i][i] = true;
}
for (int k=1; k<=n; k++)
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
A[i][j] = A[i][j] || (A[i][k] && A[k][j]);
The Complexity is also O(n3)
Prof. Amr Goneid, AUC
92
Warshall’s Algorithm Example
3
3
1
1
4
2
0
1
0
0
A0
0 1
0 0
0 0
1 0
0
1
0
0
3
1
1
1
0
0
0
1
0
0
3
1
4
4 2
2
A1
0 0
1 0
0 0
0 1
3
A2
0 0
1 0
0 0
1 1
1
1
0
1
0
1
0
1
Prof. Amr Goneid, AUC
1
4
2
A3
0 0
1 0
0 0
1 1
1
1
0
1
0
1
0
1
4
2
A4
0 0
1 1
0 0
1 1
1
1
0
1
0
1
0
1
93
6a. Single-Source All Shortest Paths, General
Weights (Bellman-Ford Algorithm)
 Dijkstra’s Algorithm computes Single –source all
shortest paths in a directed or undirected graph with
weights Wij > 0
 Floyd’s Algorithm computes All-Pairs shortest paths
in a directed or undirected graph with general weights
(positive or negative), provided that there is no cycle
with a negative length.
 Bellman-Ford Algorithm computes Single –source all
shortest paths in a directed or undirected graph with
general weights, provided that there is no cycle with a
negative length.
Prof. Amr Goneid, AUC
94
Bellman-Ford Algorithm
 Named after Richard Bellman and Lester Ford Jr.
(1956,1958)
 Cannot work if the graph contains a negative cycle
reachable from the source, since any path can be
made cheaper by more walk through the cycle.
 Like Floyd’s and Warshal’s algorithms, it is a DP
algorithm based on the Principle of Optimality.
Prof. Amr Goneid, AUC
95
Bellman-Ford Algorithm
 Dijkstra's algorithm greedily selects the minimum-
weight node that has not yet been processed, and
relaxes the minimum distance on all of its outgoing
edges. Complexity is O(V2) and a use of a heap
makes it O(V log V).
 The Bellman–Ford algorithm relaxes all the edges,
doing this |V|-1 times (when there are no cycles of
negative length, there is a shortest path between any
two vertices that has at most |V|-1 edges).
Complexity is O(VE).
Prof. Amr Goneid, AUC
96
Bellman-Ford Algorithm
 Given that Wsi is the initial cost from source (s) to
nodes (i), a distance array is initially set at
D1si = Wsi
for i = 1…|V|,
 Consider a vertex (u != s) and u has at least one
incoming edge, all vertices (i) such that edge (i,u) is
in the graph are candidates to reach (u). Hence,
there are two distances to compete:
Dsu
and mini {Dsi + Wiu}
Hence, the update rule is:
Prof. Amr Goneid, AUC
97
Bellman-Ford Algorithm
DP Update Rule (Recurrence Relation):
u
Dsu
Wui
S
i
i
Dsi
i
D1su = Wsu
Dksu = min {Dk-1su , mini {Dk-1si + Wiu}, k = 2,3,…, |V|-1
Prof. Amr Goneid, AUC
98
Bellman-Ford Algorithm
Algorithm:
for i = 1 to |V|
Dsi = Wsi
for k = 2 to |V|-1
for (each u != s, u has at least one incoming edge)
for (each edge (i , u) in the graph) // Relax
if (Dsu > Dsi + Wiu) Dsu = Dsi + Wiu
// one more iteration: if there is a shorter path
anywhere, then we have a negative length cycle
Prof. Amr Goneid, AUC
99
7. Graph Coloring
Problem (1)
 m-Colorability Decision: Can we use (m) colors
only to color a graph such that no two adjacent nodes
have the same color? If yes, what are the possible
coloring schemes?
 Example:
A
C
B
B
A
C
D
D
Prof. Amr Goneid, AUC
100
Backtracking Permutation Tree
Solution 1 : G R G R
Solution 2 : R G R G
Gain = 1-15/31 = 51.6%
1
G
R
2
G
3
10
G
4
7
17
R
G
11
14
R
5
6
8
9
12
13
Prof. Amr Goneid, AUC
15
16
101
Graph Coloring
Problem (2) (Backtracking)
 The problem:
Given m colors represented by integers 1 .. m, and a
graph of n vertices.
Find all n-tuples (x1,x2,..xn), where xk is the color of
node (k) such that no two adjacent nodes have the
same color.
 Pre:
The graph is represented by the boolean adjacency
matrix G[1:n] [1:n]
x[k] is assigned zeros.
Prof. Amr Goneid, AUC
102
NextColor
void NextColor(int k, int n)
{
do {
x[k] = (x[k] + 1) % (m+1);
// Next Highest Color
if ( !x[k]) return; // No more colors available
for (int j = 1; j <= n; j++)
if ( G[k] [j] && x[k] == x[j] ) break;
if ( j == n+1) return; // New color found
} while (1); // Otherwise, try to find another color
}
Prof. Amr Goneid, AUC
103
m-Coloring Algorithm
void mColoring(int k, int n)
{
do {
NextColor(k,n); // Assign to x[k]
// a legal color
if( !x[k] ) break ; // No new color possible
if (k == n) output vector x[1:n];
else mColoring(k+1, n);
} while(1);
}
Prof. Amr Goneid, AUC
104
Example
 Find all 3-colorings of the shown 4-node
graph
2
3
1
4
Prof. Amr Goneid, AUC
1
2
4
3
105
Solutions withTwo Colors Only
6 Tuples
2
3
2
1
3
2
1
3
1
4
4
4
2
2
2
3
1
4
3
1
4
Prof. Amr Goneid, AUC
3
1
4
106
Solutions with Exactly 3 Colors
12 Tuples: 1st 6
2
3
2
1
3
2
1
3
1
4
4
4
2
2
2
3
1
4
3
1
4
Prof. Amr Goneid, AUC
3
1
4
107
Solutions with Exactly 3 Colors
12 Tuples: 2nd 6
2
3
2
1
3
2
1
3
1
4
4
4
2
2
2
3
1
4
3
1
4
Prof. Amr Goneid, AUC
3
1
4
108
8. Topological Sorting
 A Topological Sorting of a Directed Acyclic Graph
(DAG) is a linear ordering of its vertices such that, for
every edge (u,v), u comes before v in the ordering.
 For instance, the vertices of the graph may represent
tasks to be performed, and the edges may represent
constraints that one task must be performed before
another. In this application, a topological sorting is
just a valid sequence for the tasks.
 A topological ordering is possible if and only if the
graph has no directed cycles, that is, if it is a directed
acyclic graph (DAG). .
Prof. Amr Goneid, AUC
109
Example
 Consider the following DAG representing the pre-
requisite structure of five courses.
 A valid topological ordering would
be: C1, C2, C3, C4, C5
Also: C2, C1, C3, C4, C5
Prof. Amr Goneid, AUC
110
An Algorithm (exclude & Conquer)
 L ← Empty list that will contain the sorted elements
 S ← Set of all nodes with no incoming edges
 while S is non-empty do
remove a node n from S
insert n into L
for each node m with an incoming edge e from n
remove edge e from the graph
if m has no other incoming edges then
insert m into S
 return L (a topologically sorted order)
Prof. Amr Goneid, AUC
111
An Algorithm (exclude & Conquer)
Prof. Amr Goneid, AUC
112
9. The Clique Problem
(a) Complete and Sub Graphs
 The complete graph on n vertices is Kn = (V,E)
where V = {1, 2, . . . , n}, and every pair of vertices in
V is joined by an edge.The number of edges in Kn is
maximum, i.e.,│E│= n(n-1)/2
 A subgraph of a graph G = (V,E) is a graph
B =(U, F) such that U ⊆ V and
F ⊆ E.
Prof. Amr Goneid, AUC
113
(b) The Clique
 A clique in a graph is complete subgraph of three or
more nodes.
 The clique size (k) is the number of nodes in the
subgraph.
Prof. Amr Goneid, AUC
114
The Clique
 Finding a clique is in a class of NP-complete
problems, along with other hard problems such as
Knapsack packing, graph coloring, and famous
TSP (Traveling salesman problem).
 Up until now, no better than an exponential
algorithm for such problems is known.
 widely seen in numerous fields of science and
engineering applications, even business application.
Prof. Amr Goneid, AUC
115
The Clique
 Example (Map Coloring)
Each state is to be colored
differently from other
neighbors.
The number of different
colors used represents the number of cities that you
can travel directly from one to another, which is
equivalent to a clique in a graph. Note that B, C, F,
and J are adjacent to every other state in the subprovince, forming a 4-clique in the graph.
Prof. Amr Goneid, AUC
116