Graph Algorithms
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 1
Graph Algorithms
Coming up
Useful data structures:
• Heaps & Priority Queues
(Chap 6)
• Disjoint Sets
(Chap 21)
Graph Algorithms
• Elementary Graph Algorithms
(Chap 22)
Graph Representations  Breadth-first search  Depth-first search  Topological sort
• Minimum Spanning Trees
(Chap 23)
Kruskal’s algorithm, Prim’s algorithm
• Single-Source Shortest Paths
(Chap 24)
Bellman-Ford Algorithm  Algorithm for Directed Acyclic Graph  Dijkstra’s Algorithm
• All-Pairs Shortest Paths
(Chap 25)
Shortest Paths by repeated squaring  Floyd-Warshall Alg  Johnson’s Alg
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 2
Heaps & Priority Queues
Complete
binary
tree:
The (binary) heap data structure is:
an array object
that can be viewed as
a nearly complete binary tree
1 2 3 4 5 6 7 8 9 10
1
16 14 10 8 7 9 3 2 4 1
16
2
3
14
Parent(i) =  i/2 
Left(i)
= 2i
Right(i) = 2i+1
• All leaves
have the
same depth
• All internal
nodes have
2 children
10
4
5
6
7
8
7
9
3
8
9 10
2
4
1
A max-heap: child <= parent
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
A min-heap: child >= parent
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 3
Heaps & Priority Queues
MAX-HEAPIFY(A,i)
1
The binary trees rooted at LEFT(i) and
RIGHT(i) are max-heaps
2
But A[i] may be smaller than its children.
3
MAX-HEAPIFY is to “float down” A[i] to
4
O(height of node i)
make the subtree rooted at A[i] a max-heap.
= O(lg n)
..
1
1
16
i=2
4
3
2
10
14
1
16
3
2
10
14
4
5
6
7
i=4
5
6
7
4
14
7
9
3
4
7
9
3
8
16
10
5
6
7
7
9
3
8
9 10
8
9 10
8
i=9 10
2
8
2
8
2
4
1
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
1
http://www.cs.cityu.edu.hk/~helena
3
1
6. Graph Algorithms - 4
Heaps & Priority Queues
Maximum No. of elements
Maximum No. of elements
1
level 0:
1
level 1:
2
level 2:
4
level 3:
2
4
8
3
5
6
7
9 10
8
a one-level tree (height=0):
1
a 2-level tree
(height=1):
3
a 3-level tree
(height=2):
7
a 4-level tree
(height=3):
15
Therefore, for a heap containing n elements :
Maximum no. of elements in level k = 2k
Height of tree = lg n  = (lg n)
Basic procedures:
MAX-HEAPIFY
BUILD-MAX-HEAP
MAX-HEAP-INSERT
O(lg n)
O(n)
O(lg n)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
HEAP-EXTRACT-MAX O(lg n)
HEAP-INCREASE-KEY O(lg n)
HEAP-MAXIMUM
O(lg n)
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 5
Heaps & Priority Queues
Building a heap:
1
2
1
3
2
3
BUILD-MAX-HEAP(Input_numbers)
4
5 6
4
5 6
7
7
1 Copy Input_numbers to a heap
8 9 10
8 9 1011 14 15
2 For i =  n/2  down to 1 /*all non-leaf nodes */
.. ..
3
MAX-HEAPIFY(A,i)
O(n) Note that n/2 the elements are leaf nodes
Casual illustration for a Complete-binary tree:
A complete-binary tree of height h has h+1 levels: 0,1,2,3,.. h.
The levels have 20,21,22,23,…2h elements respectively.
Then, maximum total no. of “float down” carried out by MAX-HEAPIFY
= sum of maximum no. of “float down” of all non-leaf nodes (levels h-1, h-2, .. 0)
= 1 x 2h-1 + 2 x 2h-2 + 3 x 2h-3 + 4 x 2h-4 + .. h x 20
= 2h (1/2 + 2/4 + 3/8 + 4/16…)
[note: 2h+1 = n+1, thus 2h=0.5*(n+1)]
= 0.5(n+1) (1/2 + 2/4 + 3/8 + 4/16…)
[note: 1/2 + 2/4 + 3/8 + 4/16.. <2]
< 0.5(n+1) * 2 = (n+1)
= O(n)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
Show Summation
http://www.cs.cityu.edu.hk/~helena
Continue
6. Graph Algorithms - 6
Illustration: 1/2 + 2/4 + 3/8 + 4/16.. <2
Return
1/2
= 0.5
1/2+2/4
=1
1/2+2/4+3/8
= 1.375
1/2+2/4+3/8+4/16
= 1.625
1/2+2/4+3/8+4/16+5/32
= 1.78125
1/2+2/4+3/8+4/16+5/32+6/64
= 1.875
1/2+2/4+3/8+4/16+5/32+6/64+7/128
= 1.9296875
1/2+2/4+3/8+4/16+5/32+6/64+7/128+8/256
= 1.9609375
1/2+2/4+3/8+4/16+5/32+6/64+7/128+8/256+9/512
= 1.978515625
1/2+2/4+3/8+4/16+5/32+6/64+7/128+8/256+9/512+10/1024
= 1.98828125
1/2+2/4+3/8+4/16+5/32+6/64+7/128+8/256+9/512+10/1024+11/2048 = 1.993652344
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 7
Heaps & Priority Queues
Priority queue
Is a data structure for maintaining a set of elements each
associated with a key.
Maximum priority queue supports the following operations:
INSERT(S,x)
- Insert element x into the set S
MAXIMUM(S)
- Return the ‘largest’ element
EXTRACT-MAX(S)
- Remove and return the ‘largest’ element
INCREASE-KEY(S,x,v) - Increase x’s key to a new value, v
We can implement priority queues
based on a heap structure.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 8
Heaps & Priority Queues
MAXIMUM(A)
1 return A[1]
16
14
(1)
8
2
10
7 9
3
4 1
HEAP-EXTRACT-MAX(A)
O(lg n)
1
1
2
16
14
14
10
14
10
3
8
10
4
8
7 9
3 8
7 9
3
4
7 9
3
5
2 4 1
2 4
2 1
6
Step 1. Save the value of the root that is to be returned.
7
Step 2. Move the last value to the root node.
Step 3. MAX-HEAPIFY(A,1/*the root node*/).
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 9
Heaps & Priority Queues
HEAP-INCREASE-KEY(A,i,v)
O(lg n)
1
16
2
16
16
16
14
10
3
Increase
14
10
14
10
15
10
8
7
9
3
to 15
4
8
7 9 3 15 7 9 3 14 7 9 3
2 4 1
5
8 4 1
15 4 1
8 4 1
6
Keep on exchanging with parent until parent is greater than the current node.
O(lg n)
MAX-HEAP-INSERT(A,key)
1 n = n+1
2 A[n]= - 
3 HEAP-INCREASE-KEY(A,n,key)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
16
14
8
16
10
7 9
2 4 1 -
http://www.cs.cityu.edu.hk/~helena
14
3
8
10
7 9
3
…
2 4 1 15
6. Graph Algorithms - 10
Disjoint Sets
The Disjoint Set data structure
Maintains a collection of disjoint sets.
Each set has a representative member.
Supports the following operations:
MAKE-SET(x): creates a new set containing x.
x is also the representative.
UNION(x,y):
unites the sets that contain x and y.
FIND-SET(x): returns a pointer to the representative
of the set containing x.
{.,q,..}
{.,a,..}
{.,y,..}
{.,q,..}
{.,a,..}
{x} {.,y,..}
{.,q,..} {.,a,..}
{..x,y,..}
{.,q,..} {.,a,..}
{..x,y,..}
Takes O(1) if each element has a
pointer pointing to the representative.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 11
Representation of Graphs
Example of graph:
G = (V,E)
V = {v1,v2,v3..}
E = {(v1,v2),(v2,v3),(v3,v1)..}
Directed graph
1
2
3
4
5
6
Standard representation methods:
1. Adjacency-matrix
a
1
2
3
4
5
6
1
0
0
0
0
0
0
2
10
0
0
10
0
0
3
0
0
0
0
0
0
4 a51,2=16means
0
0 0connects
1an edge
0 10 v10to v2
1
0 10 0
0 0 0
0
1 0 0
1
0 0 0
2. Adjacency-list
1
2
3
4
5
6
2
5
6
2
4
6
4 /
/
5 /
/
/
/
Undirected graph
1
2
3
• Quick to find edges
• Compact size
4
5
6
• Usually used for
dense graphs
• Usually used for
sparse graphs
(|E| << |V|2)
• No loop
• (v1,v2) = (v2,v1)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 12
Representation of Graphs
2. Adjacency-list
1. Adjacency-matrix
Memory requirement:
(|V|2), or (V2)
Directed
graph
1
2
3
4
5
6
Undirected
graph
1
2
3
4
5
6
1
0
0
0
0
0
0
1
2
3
4
5
6
1
2
3
4
5
6
1
0
1
0
0
1
0
0
0
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
2
10
0
0
10
1
0
0
3
0
0
0
0
1
0
1
0
4
10
01
0
0
10
0
5
0
10
10
10
0
0
2
10
0
0
10
0
0
6
0
0
01
0
0
0
3
0
0
0
0
0
0
4
0
1
0
0
0
0
1
0
1
2
3
4
5
6
5
0
10
10
0
0
0
1
0
1
0
0
01
0
0
Memory requirement:
(|V|+|E|), or (V+E)
6
0
0
1
0
0
0
1
0
2
10
0
0
101
1
0
0
1
2
3
4
5
6
3 4 5
0 10 0
Don’t
1 10
0 0
0 Care
0 10
0 0 10
1 10 0
0
0 0 0
http://www.cs.cityu.edu.hk/~helena
6
0
0
01
0
0
0
1
2
3
4
5
6
2
5
6
2
4
6
2
5
6
5
4
3 /
4/
/
5/
/
/
/
4
1
5
1
2
/
/
/
/
/
4 /
2 /
3 /
6. Graph Algorithms - 13
Breadth-First Search
Breadth-First Search (BFS)
r s t u
Explores the edges of a graph    
to reach every vertex from a
   
vertex s, with “shortest paths”
v w x y
The algorithm:
r s t u
   
   
v w x y
s
r


v
t
u 
Start by inspecting
the source vertex S:
So we connect
them:
r s t u
   
r s t u
   
r s t u
   
r s t u
   
   
v w x y
   
v w x y
   
v w x y
   
v w x y
For s, its 2 neighbors
are not yet searched
Now r and w join
our solution
Now v joins
our solution
Now t and x join
our solution
http://www.cs.cityu.edu.hk/~helena


For r, we do the
same to its white
color neighbors:
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong

For w, we do the
same to its white
color neighbors:
w

x
y
…
6. Graph Algorithms - 14
Breadth-First Search
Using 3 colors: white / gray / black
Start by inspecting
the source vertex S:
So we connect
them:
r s t u
   
r s t u
   
   
v w x y
For s, its 2 neighbors
are not yet searched
For r, we do the
same to its white
color neighbors:
For w, we do the
same to its white
color neighbors:
r s t u
   
r s t u
   
   
v w x y
   
v w x y
   
v w x y
Now r and w join
our solution
Now v joins
our solution
Now t and x join
our solution
No more need to
Since s is in our
check s, so mark it
solution, and it is to black.
be inspected, we
r and w join our
mark it gray
solution, we need to
check them later on,
so mark them gray.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
No more need to
check r, so mark it
black.
v joins our solution,
we need to check it
later on, so mark it
gray.
http://www.cs.cityu.edu.hk/~helena
…
No more need to
check w, so mark it
black.
t and x join our
solution, we need to
check them later on,
so mark them gray.
6. Graph Algorithms - 15
Breadth-First Search
For each vertex u, we keep the data:
u.color: black/white/gray u.distance: distance from s to u. u.pred: predecessor of u
Q : a First-in-First-out queue to keep those vertices that we need “to check them later on”
Start by putting
s into Q,
So remove s
from Q and
check its white
neighbors
So remove r
from Q and
check its white
neighbors
So remove w
from Q and
check its white
neighbors
So remove v
from Q and
‘check its white
neighbors’
Qs
r s t u
 0  
Q r w
r s t u
1 0  
Qw v
r s t u
1 0  
Qv t x
r s t u
1 0 2 
Q t x
r s t u
1 0 2 
   
v w x y
 1  
v w x y
2 1  
v w x y
2 1 2 
v w x y
2 1 2 
v w x y
Looking at Q:
the first node to
check is s:
Looking at Q:
the first node to
check is r
Looking at Q:
the first node to
check is w
Looking at Q:
the first node to
check is v
Looking at Q:
the first node to
check is t
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 16
Breadth-First Search
For each vertex u, we keep the data:
u.color: black/white/gray u.distance: distance from s to u. u.pred: predecessor of u
Q : a First-in-First-out queue to keep those vertices that we need “to check them later on”
So remove t
from Q and
check its white
neighbors
So remove x
from Q and
check its white
neighbors
Q t x
r s t u
1 0 2 
Qx u
r s t u
1 0 2 3
Qu y
r s t u
1 0 2 3
r s t u
1 0 2 3
r s t u
1 0 2 3
2 1 2 
v w x y
2 1 2 
v w x y
2 1 2 3
v w x y
2 1 2 3
v w x y
2 1 2 3
v w x y
Looking at Q:
the first node to
check is t
Looking at Q:
the first node to
check is x
Looking at Q:
the first node to
check is u
Looking at Q:
the first node to
check is y
Looking at Q:
no more nodes
to check=> End
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
So remove u
from Q and
‘check its white
neighbors’
So remove y
from Q and
‘check its white
neighbors’
So remove v
from Q and
‘check its white
neighbors’
Q
Qy
6. Graph Algorithms - 17
Breadth-First Search
BFS(G,s) /*G=(V,E)*/
1 For each vertex u in V - {s}
2
u.color = white
(V)
3
u.distance = 
4
u.pred = NIL
5 s.color = gray
6 s.distance = 0
7 s.pred = NIL
8 Q=
9 ENQUEUE(Q,s)
10 while Q  
11
u = DEQUEUE(Q)
12
for each v adjacent to u
13
if v.color = white
14
v.color = gray
15
v.distance = u.distance + 1
16
v.pred = u
17
ENQUEUE(Q,v)
Anal of Alg (2001-2002
SemA)
18CS3381 Des &u.color
= black
City Univ of HK / Dept of CS / Helena Wong
The running
time of BFS is:
O(V+E)
Total number of edges
kept by the adjacency list
is (E)
Total time spent in the
adjacency list is O(E)
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 18
Breadth-First Search
PRINT-PATH(G,s,v)
/* print the shortest path from s to v */
1 if v = s
2
print s
3 else
4
if v.pred = NIL
5
print “no path from” s “to” v “exists”
6
else
7
PRINT-PATH(G, s, v.pred)
8
print v
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 19
Depth-First Search
Depth-First Search (BFS)
Explores the edges of
a graph by searching
“deeper” whenever
possible.
u

v

w


x
u



y
v



z
w






y


x
z
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
DFS(G) /*G = (V,E) */
1 for each vertex u in V
2
u.color = white
3
u.pred = NIL
4 for each vertex u in V
5
if u.color = white
6
DFS-VISIT(u)
DFS-VISIT(u)
1 u.color = gray
2 for each v adjacent to u
3
if v.color = white
4
v.pred = u
5
DFS-VISIT(v)
6 u.color = black
http://www.cs.cityu.edu.hk/~helena
The running
time of DFS
is: (V+E)
(V)
(V) +
Time to
execute
calls to
DFS-VISIT
Total number of
edges kept by
the adjacency
list is (E).
Total time spent
in the adjacency
list is (E).
6. Graph Algorithms - 20
Depth-First Search
DFS(G) /*G = (V,E) */
1 for each vertex u in V
2
u.color = white
3
u.pred = NIL
4 time = 0
In many occasions it is useful to
keep track of the discovery time
and the finishing time while
checking each node.
u
1/8

v
2/7

w
9/12

4/5

3/6

10/11

x
y
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
z
5
6
7
for each vertex u in V
if u.color = white
DFS-VISIT(u)
DFS-VISIT(u)
1 u.color = gray
2 time = time + 1
3 u.discover = time
4
5
6
7
8
for each v adjacent to u
if v.color = white
v.pred = u
DFS-VISIT(v)
u.color = black
9 time = time + 1
10 u.finish = time
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 21
Topological Sort
Topological Sort
11/16 undershorts
socks 17/18
watch 9/10
12/15 pants
shoes 13/14
6/7 belt
shirt 1/8
tie 2/5
• A linear ordering
of vertices : if the
graph contains an
edge (u,v), then u
appears before v.
• Applied to
directed acyclic
graphs (DAG)
jacket 3/4
Sorting according to the finishing times, in descending order:
socks
undershorts
pants
shoes
watch
shirt
17/18
11/16
12/15
13/14
9/10
1/8
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
belt
tie
6/7 2/5
jacket
3/4
6. Graph Algorithms - 22
Topological Sort
TOPOLOGICAL-SORT(G)
11/16 undershorts
socks 17/18
12/15 pants
6/7 belt
1 Call DFS(G) to
watch 9/10
compute finishing
times v.finish for each
shoes 13/14
vertex v
shirt 1/8
2 As each vertex is
finished, insert it onto
the front of a linked
list
tie 2/5
3 Return the linked list
of vertices
jacket 3/4
(V+E)
Sorting according to the finishing times, in descending order:
socks
undershorts
pants
shoes
watch
shirt
17/18
11/16
12/15
13/14
9/10
1/8
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
belt
tie
6/7 2/5
jacket
3/4
6. Graph Algorithms - 23
Minimum Spanning Trees (MST)
Given a connected, undirected graph
G = (V,E).
For each edge (u,v) in E, we have a
weight w(u,v).
4
a
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
4b
8
c 7
2 4 d 9
f 2 e
i
g1
h
8
http://www.cs.cityu.edu.hk/~helena
4
b
8
7 d
c
2
i
6
a 11
8
7
h
1
MINIMUM A set of edges such that
SPANNING (1) They connect all vertices
(2) The sum of their weights
TREE (MST)
is smallest possible.
Why called a tree:
b
4
g
c
9
e
14
2 f
7 d
10
9
2
a 11
i
14 e
4
6
8
7
10
h
1 g 2 f
4 f 2
g
8 c 7
1
b
4
d9
2
h
a
i
e
6. Graph Algorithms - 24
Minimum Spanning Trees (MST)
A Generic MST Algorithm:
GENERIC-MST(G)
1
A=
2
while A does not form a spanning tree
3
Find an edge (u,v) that is a safe choice
4
A = A U {(u,v)}
5
return A
2 elaborations:
Kruskal’s Algorithm
Prim’s Algorithm
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 25
MST - Kruskal’s algorithm
MST - Kruskal’s algorithm
4
Keep on finding the least expensive edge
8
b
11
a
8
h
7
2
i
6
1
7 d
c
e
14
4
g
9
2
f
10
MST-KRUSKAL(G) /* G = (V,E) */
1 A=
2 For each vertex v in V
3
MAKE-SET(v)
4 Sort the edges of E into nondecreasing order by the weight
5 For each edge (u,v) in E, taken in nondecreasing order by weight
6
if FIND-SET(u)  FIND-SET(v)
7
A = A U {(u,v)}
Where MAKE-SET, FIND8
UNION(u,v)
SET, UNION are functions
9 return A
of disjoint-sets.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 26
MST - Kruskal’s algorithm
O(1)
O(V)
O(E lg E)
O(E lg V)
MST-KRUSKAL(G) /* G = (V,E) */
1 A=
2 For each vertex v in V
3
MAKE-SET(v)
4 Sort the edges of E into nondecreasing order by the weight
5 For each edge (u,v) in E, taken in nondecreasing order by weight
6
if FIND-SET(u)  FIND-SET(v)
7
A = A U {(u,v)}
8
UNION(u,v)
9 return A
running time of MIT-KRUSKAL is: O(1) + O(V) + O(E lg E) + O(E lg V)
Since |V|  |E|+1, we prefer a tight upper bound with V terms instead of E terms.
|E| < |V|2
 lg |E| < lg |V|2
 lg |E| < 2 lg |V|
CS3381
running
of MIT-KRUSKAL
is: O(E lg V)
Des & Analtime
of Alg (2001-2002
SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 27
MST - Prim’s algorithm
MST - Prim’s algorithm
Grow a MST by adding vertices that are nearest to the growing tree.
4
2. b
11
1. a
8
8
7
c
8. d
2
4. i
7
7. h
3.
1
14
4
6
9
9.e
10
6.g
2
5. f
• Find a MST rooted at a given vertex r.
• Grow a MST by adding vertices. In each step, a vertex that has least
‘distance’ to the growing tree is added.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 28
MST - Prim’s algorithm
• For each vertex u, define u.dist as this ‘distance’, ie. u.dist is the weight of the
edge connecting u and its closest neighbor in the growing tree. This closest
neighbor is defined as u.pred.
• In each step, after identifying the vertex to be added, we need to check see
whether they can update others’ dist and pred.
8
b
b
dist=4
2.
4
1. a
i
i
dist=2
11
a
dist=0
8
7. hh
h
dist=1
dist=8
dist=7
4.
7
1
7
3. cc
c
dist=8
dist=8
2
d
d
dist=7 9
8.
14
4
6
6. g
g
dist=6
dist=2
f
f
2 dist=4
5.
e
9. ee
dist=10
dist=9
10
• Use a min-priority queue Q to keep the discovered vertices that are to be added
to the tree, keyed by their ‘dist’ values.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 29
MST - Prim’s algorithm
MST-PRIM(G,r) /* G = (V,E) */
1 For each u in V
O(V)
2
u.dist = 
3
u.pred = NIL
4 r.dist = 0
Assume binary min-heap is used
5 Create a min-priority queue Q for V according to values of dist
6 While Q is not empty
O(V lg V) +
O(E *1) +
O(E lg V)
= O(E lg V)
7
u = EXTRACT-MIN(Q)
8
For each v adjacent to u
9
if v exists in Q and weight of (u,v) < u.dist
10
v.pred = u
11
v.dist = weight of (u,v)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
O(lg V)
O(1)
Assume 1 bit is used to tell
whether or not v is in Q
O(lg V)
Needs to run DECREASE-KEY
6. Graph Algorithms - 30
Single-Source Shortest Paths
Single-Source Shortest Paths
Given a weighted, directed graph, find the shortest
paths from a given source vertex s to other vertices.
6
3
3
s
2
9
1
0
4
2
7
3
5
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
5
6
http://www.cs.cityu.edu.hk/~helena
11
6. Graph Algorithms - 31
Single-Source Shortest Paths
Variants:
3
Single-destination
shortest-path problem
By reversing the direction of
each edge, we can reduce this
problem to a single-source
problem.
3
3
5
9
1 4 2
3
2
d 0
6
5
6
7
11
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
5
5
Single-pair
shortest-path
problem
If the single-source
problem is solved, we
can solve this problem
also. There are no
asymptotically faster
algorithms.
http://www.cs.cityu.edu.hk/~helena
9
1 4
2
3
2
s 0
6
3
6
7
11
All-pair
shortest-path
problem
Can be solved by
running a single source
algorithm once for each
source vertex. However,
other faster approaches
exist.
6. Graph Algorithms - 32
Single-Source Shortest Paths
Optimal substructure of a shortest path:
A shortest path between 2 vertices contains
other shortest paths within it.
s 0
3
5
b
a
-4
3
-1
d 2
c 6
5
11
-3
Edge weight & Path weight :
Edge weight: eg. w(c,d) = 6
Path weight: eg. For a path p=<s,c,d>, w(p) = w(s,c) + w(c,d) = 11
Shortest-path weight:
Define shortest-path weight for a path p from s to v as:
(s,v) =
min { w(p): s ~p v}
if there is a path from s to v

otherwise
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 33
Single-Source Shortest Paths
h, i, and j are not reachable from s
=> (s,h), (s,i) and (s,j) are 
Negative-weight edges
eg. w(a,b) = -4
Negative-weight path
eg. <s,a,e>: -1
s 0
Negative-weight cycle
eg. <e,f,e>: -3
3
5
2
a
3
c
5
e
-
b
-1
-4
d
11
6
-3
3
f
-
-6
4
8
7
g
-
h

-8
i

2

3
j
If there is no negative weight cycle reachable from the source vertex s,
then for all v in V, the shortest-path weight (s,v) remains well defined.
A well defined shortest path has no cycle. Prove:
1. A shortest path should not contain non-negative weight cycle.
[otherwise reducing the cycle would give a more optimal path]
2. A well defined shortest path should not contain negative weight cycle
=> A well defined shortest path has no cycle, and has at most |V|-1 edges.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 34
Single-Source Shortest Paths
a
A general function for single-source
shortest paths algorithms:
INITIALIZE-SINGLE-SOURCE()
1 For each vertex v in V
2
v.d = 
3
v.pred = NIL
4 s.d = 0
(V)
s 0
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
c

1

d 2

Where v.d is the upper bound
on the weight of a shortest path
from source vertex s to v.
A general technique for singlesource shortest paths algorithms:
Relaxation
“Relaxing an edge (d,b)” :
Testing whether we can improve the
shortest path to b found so far by going
through d, if so, update b.d and b.pred.
3
1
4

b
3
s 0 1
b
a
4
3
7
d 2
c 1
1
2
RELAX(u,v)
1 if v.d > u.d + w(u,v)
2
v.d = u.d + w(u,v)
3
v.pred = u
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 35
Single-Source Shortest Paths
Three solutions to the problem:
Bellman-Ford algorithm
- By relaxing the whole set of edges |V|-1 times
Algorithm for directed acyclic graphs (DAG)
- By topological sorting the vertices first, then relax the
edges of the sorted vertices one by one.
Dijkstra’s algorithm
- Similar to MST-PRIM. Handle non-negative edges only.
Grow the solution by checking vertices one by one,
starting from the one nearest to the source vertex.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 36
Single-Source Shortest Paths
Bellman-Ford Algorithm
Single-Source Shortest Paths
Bellman-Ford Algorithm
Method: Relax the whole set of edges |V|-1 times.
At 1st time:

6
8
s 0
At 2nd time:

6
6
8
7
6
6
8
s 0
7
9
5
-2
2
7
9
-2
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong

8
s 0


-3
-4 7

6

6
6
8
7
9
5
-2
2
7
5

-3
-4 7
2
7
s 0
5
-2
9


-3
-4 7

6
6
8
s 0
5
-2
7
6
6
8
s 0
7
-3
-4 7
2
7

9
5
-2
4
-3
-4 7
2
7

9
4
-3
-4 7
2
7

-3
-4 7
2
7
s 0
At 3rd , 4th time:
5
-2
9
2
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 37
2
Single-Source Shortest Paths
Bellman-Ford Algorithm
When applied to
a graph with
negative-weight
cycle that is
reachable from
the source vertex:
s0
6
-5
 2 
Characteristics of Bellman-Ford Algorithm:
• Edge weights may be negative.
• Checks whether there is a negative-weight cycle
reachable from the source.
O(V)
O(VE)
s0
6
-5
6 2 
O(E)
s0
6
-5
6 2 1
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
Bellman-Ford Algorithm(G,s) /*G=(V,E)*/
1 INITIALIZE-SINGLE-SOURCE(G,s)
2 for i = 1 to |V| - 1
3
for each edge (u,v) in E
4
RELAX(u,v)
5 for each edge (u,v) in E
6
if v.d > u.d + w(u,v)
7
return false
8 return true
O(VE)
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 38
Single-Source Shortest Paths
Algorithm for directed acyclic graphs (DAG)
Single-Source Shortest Paths
DAG-Shortest-Path
6
Method: By topological sorting the vertices first, then
relax the edges of the sorted vertices one by one.
 5 0 2  7  -1  -2 
s
2
4
3
6
6
6
1
 5 0 2 2 7 6 -1 6 -2 4
s
2
4
3
6
1
 5 0 2 2 7 6 -1 5 -2 4
s
2
4
3
1
 5 0 2  7  -1  -2 
s
2
4
3
1
 5 0 2 2 7 6 -1  -2 
s
2
4
3
6
6
1
1
 5 0 2 2 7 6 -1 5 -2 3
s
2
4
3
1
 5 0 2 2 7 6 -1 5 -2 3
s
2
4
3
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 39
Single-Source Shortest Paths
Algorithm for directed acyclic graphs (DAG)
(V+E)
(V)
(E)
DAT-SHORTEST-PATHS(G,s)
1 Topologically sort the vertices of G
2 INITIALIZE-SINGLE-SOURCE(G,s)
3 For each vertex u, taken in topologically sorted order
4
For each vertex v adjacent to u
5
RELAX(u,v)
(V+E)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 40
Single-Source Shortest Paths
Dijkstra’s Algorithm
Single-Source Shortest Paths
Dijkstra’s Algorithm

10
5

10
8
5
2
1
5
2
10
10
6
13
6
7
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
5
5
10
8
5

2
1
5
9
2
6
http://www.cs.cityu.edu.hk/~helena
5
5
10
8
5
14
2
1
6
7
9
3 9 4
7
2
s 0
1
3 9 4
7
2
s 0

7
8
10
6
3 9 4
7
2
s 0
1
3 9 4
7
2
s 0

3 9 4
7
2
s 0

3 9 4
7
2
s 0
1
Similar to MST-PRIM. Handle non-negative edges only.
Method: Grow the solution by checking vertices one by
one, starting from the one nearest to the source vertex.
5
2
6
7
6. Graph Algorithms - 41
Single-Source Shortest Paths
Dijkstra’s Algorithm
DIJKSTRA(G,s)
1 INITIALIZE-SINGLE-SOURCE(G,s)
2 S=
Assume binary min-heap is used
3 Create a min-priority queue Q for all vertices according to their values of d
4 While Q is not empty
EXTRACT-MIN:
Total: O(|V| lg V)
5
u = EXTRACT-MIN(Q)
6
S = S U {u}
7
For each vertex v adjacent to u
8
DECREASE-KEY:
Total: O(|E| lg V)
RELAX(u,v) /* Need to call DECREASE-KEY */
Assume all vertices are reachable ( ie. |E|>|V|-1 )=> O(E lg V)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 42
All-Pairs Shortest Paths
All-Pairs Shortest Paths
Given a weighted, directed graph, find the
shortest path for every pair of vertices.
Solutions make use of Single-source shortest path methods:
1. Running Dijkstra’s algorithm |V| times, once for each vertex (if no
negative edge exists) => O(VE lg V)
2. Running Bellman-Ford algorithm |V| times, once for each vertex =>
O(V2E)
For dense graphs: |E|=|V|2 approx. => O(V4)
Faster solutions:
1. Shortest paths by repeated squaring => (V3 lg V)
2. Floyd-Warshall Algorithm => (V3)
3. Johnson’s Algorithm for Sparse Graphs => (VE lg V)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 43
All-Pairs Shortest Paths
We typically want the output in tabular form:
2
3
1
8
2
-4
1
7
5
4
6
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
3
i
-5
4
http://www.cs.cityu.edu.hk/~helena
d
1
2
j
3
4
5
1
0
1
-3 2
-4
2
3
0
-4 1
-1
3
7
4
0
5
3
4
2
-1 -5
0
-2
5
8
5
6
0
1
6. Graph Algorithms - 44
All-Pairs Shortest Paths
PRINT-ALL-PAIRS-SHORTEST-PATH(i,j)
1 if i=j
2
print i
3 else
4
if predij = NIL
5
print “no path from ” i “ to” j “ exists.”
6
else
7
PRINT-ALL-PAIRS-SHORTEST-PATH(i,predij)
8
print j
In addition, we
need a
predecessor
matrix to keep
the information
of the paths:
3
1
-4
2
4
3
2
8
7
5 6
1 -5
4
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
d
1
2
i 3
4
5
1 2
0 1
3 0
7 4
2 -1
8 5
j
3
-3
-4
0
-5
1
4 5
2 -4
1 -1
5 3
0 -2
6 0
http://www.cs.cityu.edu.hk/~helena
j
pred 1 2 3 4 5
1 NIL 3 4 5 1
2 4 NIL 4 2 1
i 3 4 3 NIL 2 1
4 4 3 4 NIL 1
5 4 3 4 5 NIL
6. Graph Algorithms - 45
All-Pairs Shortest Paths
Shortest paths by repeated squaring
An all-pairs shortest
paths algorithm
by repeated squaring
A dynamic-programming
approach
Optimal substructure:
A shortest path p from vertex i to j (i j):
i ~(p) j
then, suppose p reaches vertex k just before
reaching j, we describe it as:
p = i ~(p’) k  j
Then, p’ is a shortest path from vertex k to j.
Let l(m)ij be the minimum weight of any path from vertex i to vertex j
that contains at most m edges.
l(m-1)ij = l(m-1)ij+wjj
When m=0:
Otherwise:
0 if i=j
l(m)ij = min (l(m-1)ij , min1k n {l(m-1)ik+wkj})
(0)
l ij =
 if i j
= min
{l(m-1) +w }
1k n
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
ik
kj
6. Graph Algorithms - 46
All-Pairs Shortest Paths
Shortest paths by repeated squaring
Let l(m)ij be the minimum weight of any path from
vertex i to vertex j that contains at most m edges.
0 if i=j
When m=0: l(0)ij =
 if i j
Otherwise: l(m)ij = min1k n {l(m-1)ik+wkj}
Each shortest path in a
graph must contain not
more than |E|-1 edges.
So, we can obtain the
shortest-path weights as
l(n-1)ij (where n=|V|, 1  i,j  n)
Consider the matrix containing values of l(n-1)ij ,
where 1  i,j  n.
It can be computed according to
(1) the matrix containing l(n-2)ij , and
(2) the matrix containing wij
d 1
1 0
2 3
u 3 7
4 2
5 8
2
1
0
4
-1
5
v
3
-3
-4
0
-5
1
4 5
2 -4
1 -1
5 3
0 -2
6 0
Indeed, all matrices containing the values of l(m)ij (where 1  i,j  n, m=1,2,…,n-1)
can be computed according to:
the matrix containing l(m-1)ij and the matrix containing wij
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 47
All-Pairs Shortest Paths
Shortest paths by repeated squaring
Let l(m)ij be the minimum weight of any path from
vertex i to vertex j that contains at most m edges.
0 if i=j
When m=0: l(0)ij =
 if i j
Example:
3
1
2
-4 7
5 6
Otherwise: l(m)ij = min1k n {l(m-1)ik+wkj}
l(0)ij 1
1 0
2 
i3 
4 
5 
2

0



j
3


0


4



0

5




0
j
l(2)ij 1 2 3 4 5 l(3)ij 1
1 0 3 8 2 -4
1 0
2 3 0 -4 1 7
2 3
i 3  4 0 5 11 i 3 7
4 2 -1 -5 0 -2
4 2
8 
6 Alg0(2001-20025 SemA)
5 CS3381
Des & 1
Anal of
8
City Univ of HK / Dept of CS / Helena Wong
l(1)ij 1
1 0
2 
i3 
4 2
5 
2
3
0
4
-1
5
j
3
-3
-4
0
-5
1
4
2
1
5
0
6
2
3
0
4


j
3
8

0
-5

4

1

0
6
1
5
1 0
-4
2 3
-1
11 i 3 7
4 2
-2
5 8
0
l(4)ij
5
-4
7


0
2
1
0
4
-1
5
j
3
-3
-4
0
-5
1
http://www.cs.cityu.edu.hk/~helena
2
w
1
2
i3
4
5
4
2
1
5
0
6
5
-4
-1
3
-2
0
1
0


2

2
3
0
4


4
8 3
1 -5
4
j
3
8

0
-5

4

1

0
6
5
-4
7


0
Eg. l(1)1,2= min { l(0)1,1 + w1,2,
l(0)1,2 + w2,2,
l(0)1,3 + w3,2,
l(0)1,4 + w4,2,
l(0)1,5- +48w5,2 }
6. Graph Algorithms
All-Pairs Shortest Paths
Shortest paths by repeated squaring
THE ALGORITHM
Slower version:
EXTEND-SHORTEST-PATHS(int l[n][n])
1 construct a matrix: l’ [n][n]
2 for i = 1 to n
Let l(m) be the matrix
for j = 1 to n
containing the weights 3
4
l’ [i][j] = 
of the shortest paths
5
for k=1 to n
between all vertex
6
if l [i][j] + wkj < l’[i][j]
pairs that contain at
7
l’[i][j] = l [i][j] +wkj
most m edges.
8 return l’ij
(V3)
The algorithm is to
extend the shortest
paths computed so far SLOW-ALL-PARIS-SHORTEST-PATHS()
by one more edge.
1 copy w to l(1)
2 for m = 2 to n-1
3
l(m) = EXTEND-SHORTEST-PATHS(l(m-1))
4 return l(m)
(V4)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 49
All-Pairs Shortest Paths
Shortest paths by repeated squaring
Let l(m)ij be the minimum weight of any path from
vertex i to vertex j that contains at most m edges.
0 if i=j
When m=0: l(0)ij =
 if i j
Otherwise: l(m)ij = min1k n {l(m-1)ik+wkj}
For a graph with 5 vertices, l(4)ij contains the resultant
weights of the shortest paths in the graph.
However, even we continue to compute l(5)ij, l(6)ij, .., we
won’t get any result different from l(4)ij.
Reason: “l(m)ij is the minimum weight of any path from
vertex i to vertex j that contains at most m edges.”
Example:
3
2
1
2
-4 7
5 6
w
1
2
i3
4
5
1
0


2

2
3
0
4


4
8 3
1 -5
4
j
3
8

0
-5

4

1

0
6
5
-4
7


0
Hence, for a graph with n vertices,
l(n-1)ij = l(n)ij = l(n+1)ij = l(n+2)ij = ..
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 50
All-Pairs Shortest Paths
Shortest paths by repeated squaring
Define a special  operator between 2 matrix,
such that: A  B = C, where Cij = min(Aij + Bjk)
THE ALGORITHM
Faster version:
l(1)ij 1
1 0
2 
3 
4 2
5 
2
3
0
4


3
8

0
-5

4

1

0
6
5
-4
7


0

w
1
2
3
4
5
1
0


2

2
3
0
4


3
8

0
-5

4

1

0
6
5
-4
7


0
l(2)ij 1
1 0
2 3
= 3 
4 2
5 8
2
3
0
4
-1

3
8
-4
0
-5
1
4
2
1
5
0
6
5
-4
7
11
-2
0
Since l(1)
=w
l(2) = l(1)  w = w  w
l(3) = l(2)  w = w  w  w
..
then l(m) = l(m-1)  w = w  w  ..  w /* no. of w in this equation is m-1 */
Traditional matrix multiplication is associative, similarly fewer steps can achieve l(m):
Eg. for l(8)=wwwwwwww, in a simple method, it needs 7  operations.
However, by formulating as {[ (ww)  (ww) ]  [ (ww)  (ww) ]},
we can compute w  w and save as w’, then compute w’  w’ to get (ww)(ww)
CS3381 Des & Anal of Alg (2001-2002 SemA)
Univ of HK
of CS
/ Helena
Wong
Graph Algorithms - 51
andCitysave
as/ Dept
w’’,
then
compute
l8 ashttp://www.cs.cityu.edu.hk/~helena
w’’  w’’, ie. totally only 3 6.operations.
All-Pairs Shortest Paths
Shortest paths by repeated squaring
FASTER-ALL-PARIS-SHORTEST-PATHS()
1 copy w to l(1)
2 m=1
Let |V| = n,
3 while m < n-1
We need the solution l(n-1),
(2m)[n][n]
4
construct
a
matrix:
l
(n-1)
(n)
(n+1)
(n+2)
Since, l
=l =l
=l
= ..
for i = 1 to n
Let q = smallest power of 2 such that q>=(n-1) 5
6
for j = 1 to n
Eg. for (n-1)=1, q=1
7
l(2m)[i][j] = 
for (n-1)=2, q=2
for (n-1)=3 or 4, q=4
8
for k=1 to n
for (n-1)=5, 6, 7 or 8, q=8
9
if l(m)[i][j] + wkj < l(2m)[i][j]
for (n-1)=9,10,11, ..15 or16, q=16
(2m)[i][j] = l(m)[i][j] +w
10
l
kj
Instead of computing l(n-1) , let’s compute 11
m = 2m
l(q):
l(1) = w
12 return l(m)
(2)
(1)
(1)
l =l l
(V3 lg V)
l(4) = l(2)  l(2)
l(8) = l(4)  l(4)
…
l(q) = l(q/2)  l(q/2)
Here, l(q) can be computed in  lg q  steps.
Repeated squaring
THE ALGORITHM
Faster version:
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 52
All-Pairs Shortest Paths
Floyd-Warshall Algorithm
Floyd-Warshall Algorithm
(dynamic-programming)
Optimal Substructure:
Consider a shortest path ~p
from vertex vi to vj:
i
v1, v2, v3, .. vk
j
The intermediate vertices of ~p are all
drawn from {v1, v2, v3, v4, .. vk }
Case 1: p does not involve vk
=>
i ~p j is a shortest path with all intermediate
vertices drawn from v1, v2, .. vk-1.
Case 2: p involves vk
=> 1. i ~p1k is a shortest path with all intermediate
vertices drawn from v1, v2, .. vk-1.
2. k ~p2j is a shortest path with all intermediate
vertices drawn from v1, v2, .. vk-1.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
i
p1
p2
vk
v1, v2, v3, .. Vk-1
j
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 53
All-Pairs Shortest Paths
Floyd-Warshall Algorithm
Let d(k)ij be the weight of a shortest path from vertex i to vertex j for
which all intermediate vertices are in the set {v1, v2, v3, v4, .. vk }
d(k)ij
=
wij
if k=0
min (d(k-1)ij, d(k-1)ik+d(k-1)kj)
if k>=1
d(|V|)ij gives the final answer.
The solution for the predecessor graph is:
NIL
if i=j or wij=
For k=0:
(0)
pred ij =
i
otherwise
For k>=1 pred(k) =
ij
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
pred(k-1)ij
if d(k-1)ij  d(k-1)ik + d(k-1)kj
pred(k-1)kj
otherwise
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 54
All-Pairs Shortest Paths
3
Floyd-Warshall Algorithm
1
2
2
-4 7
5 6
4
8 3
1 -5
4
d(0) 1
1 0
2 
3 
4 2
5 
2
3
0
4


3
8

0
-5

4

1

0
6
5
-4
7


0
pred(0) 1 2 3 4 5
1 NIL 1 1 NIL 1
2 NILNILNIL 2 2
3 NIL 3 NILNILNIL
4 4 NIL 4 NILNIL
5 NILNILNIL 5 NIL
d(3) 1
1 0
2 
3 
4 2
5 
2
3
0
4
-1

3
8

0
-5

4
4
1
5
0
6
5
-4
7
11
-2
0
pred(3) 1 2 3 4 5
1 NIL 1 1 2 1
2 NILNILNIL 2 2
3 NIL 3 NIL 2 2
4 4 3 4 NIL 1
5 NILNILNIL 5 NIL
d(1) 1
1 0
2 
3 
4 2
5 
2
3
0
4
5

3
8

0
-5

4

1

0
6
5
-4
7

-2
0
pred(1) 1 2 3 4 5
1 NIL 1 1 NIL 1
2 NILNILNIL 2 2
3 NIL 3 NILNILNIL
4 4 1 4 NIL 1
5 NILNILNIL 5 NIL
d(4) 1
1 0
2 3
3 7
4 2
5 8
2
3
0
4
-1
5
3
-1
-4
0
-5
1
4
4
1
5
0
6
5
-4
-1
3
-2
0
pred(4) 1 2 3 4 5
1 NIL 1 4 2 1
2 4 NIL 4 2 1
3 4 3 NIL 2 1
4 4 3 4 NIL 1
5 4 3 4 5 NIL
d(2) 1 2 3 4 5
pred(2) 1 2 3 4 5
d(5) 1 2
1 0 3 8 4 -4
1 NIL 1 1 2 1
1 0 1
2  0  1 7
2 NILNILNIL 2 2
2 3 0
3  4 0 5 11
3 NIL 3 NIL 2 2
3 7 4
2 Des5& Anal
-5 of0Alg (2001-2002
-2
4CS3381
4 4 1 4 NIL 1
4 2 -1
SemA)
http://www.cs.cityu.edu.hk/~helena
Univ of
/ Dept 6
of CS0/ Helena Wong5 NILNILNIL
HK 
5 NIL
5City 
5 8 5
3
-3
-4
0
-5
1
4
2
1
5
0
6
5
-4
-1
3
-2
0
pred(5) 1 2 3 4 5
1 NIL 3 4 5 1
2 4 NIL 4 2 1
3 4 3 NIL 2 1
4 4 3 4 NIL 1
4 3 4 - 55
5 NIL
5 Algorithms
6. Graph
All-Pairs Shortest Paths
Floyd-Warshall Algorithm
FLOYD-WARSHALL()
1 Copy w to d(0)
2 For i=1 to |V|
3
For j = 1 to |V|
4
if i=j or wij = 
5
pred(0)ij = NIL
6
else
7
pred(0)ij = i
8 For k = 1 to |V|
9
For i=1 to |V|
10
For j = 1 to |V|
11
If d(k-1)ij  d(k-1)ik +d(k-1)kj
12
d(k)ij = d(k-1)ij
13
pred(k)ij = pred(k-1)ij
14
Else
15
d(k)ij = d(k-1)ik +d(k-1)kj
16
pred(k)ij = pred(k-1)kj
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
(V3)
6. Graph Algorithms - 56
All-Pairs Shortest Paths
Floyd-Warshall Algorithm
Transitive Closure of a Directed Graph
Transitive Closure of a Directed Graph
2
1
Given a directed graph G = (V,E).
Construct ist transitive closure G* = (V,E*) where
E* = {(i,j) : there is a path from vertex i to vertex j in G}
3
5
4
G = (V,E)
Define t(k)ij as a boolean value to describe
whether there is a path from vertex i to
vertex j:
For k=0:
For k>=1:
t(0)ij
1
=
0
t(k)ij= t(k-1)ij
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
if i = j or (i,j)  E
otherwise
V
(t(k-1)ik

t(k-1)kj)
http://www.cs.cityu.edu.hk/~helena
2
1
3
5
4
G* = (V,E*)
6. Graph Algorithms - 57
All-Pairs Shortest Paths
Floyd-Warshall Algorithm
Transitive Closure of a Directed Graph
TRANSITIVE-CLOSURE(G)
1 For i=1 to |V|
2
For j = 1 to |V|
3
if i=j or (i,j)  E
4
t(0)ij = 1
5
else
6
t(0)ij = 0
7 For k = 1 to |V|
8
For i=1 to |V|
9
For j = 1 to |V|
10
t(k-1)ij V (t(k-1)ik  t(k-1)kj)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
(V3)
6. Graph Algorithms - 58
All-Pairs Shortest Paths
Johnson’s Algorithm for Sparse Graphs
Johnson’s Algorithm for Sparse Graphs
O(VE lg V)
Characteristics of Johnson’s Algorithm for Sparse Graphs :
• For sparse graphs, it is asymptotically better than Floyd-Warshall or
repeated squaring of matrices.
• Uses Dijkstra’s algorithm and Bellman-Ford algorithm as subroutines.
• Checks whether there is a negative-weight cycle.
Technique: Reweighting
• If all edge weights are nonnegative, then, run Dijkstra once for
each vertex: O(VE lg V)
• Otherwise, compute a new set of nonnegative weights such that
we can run Dijkstra on it to obtain the equivalent answers.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 59
All-Pairs Shortest Paths
Johnson’s Algorithm for Sparse Graphs
Reweighting
For the weights of the original graph edges (w),
compute a new set of nonnegative weights (w’), such that:
1. For all vertices u,v, a path p is a shortest path from u to v using weights
w if and only if p is also a shortest path from u to v using weights w’.
2. For all edges (u,v), the new weights w’ are all nonnegative.
But how to compute these ŵ values?
Johnson imagined a new vertex s, that has zero weight
edges connecting to each vertex v.
Then he used the Bellman-Ford algorithm to compute
(s,v) , the shortest path weights from s to each vertex v.
Each w’uv is computed as w’uv = wuv + (s,u) - (s,v)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 60
All-Pairs Shortest Paths
Johnson’s Algorithm for Sparse Graphs
For example,
A graph with 5 vertices:
0
0
s
0
3
1
0
2
We add a imagined vertex s.
-1
4
0
2
-4
0
5 7
-4 6
4
3
8
-5
1
-5
0
There are 0-weighted edges
from s to each of the 5 vertices.
By using the Bellman-Ford
Algorithm, we compute the
shortest path weights from s
to each of the 5 vertices.
0
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 61
All-Pairs Shortest Paths
Johnson’s Algorithm for Sparse Graphs
0
s
0
1
0 0
3
2
-4
0
2
-1
0
1
4
3
8 -5
1 -5
7
5
4
-4 6 0
Compute
w’uv=wuv+(s,u)-(s,v)
s
0
1
0 0
2
0
4
0
4
2
-1
5
0
3
13 -5
0
5 10 4 0
-4 2 0
0
Why this approach produces w’ according to our purpose?
We can explain by proving that
A. A path is shorter than another using the weights w if and only if it is
also shorter using w’.
B. A graph has a negative-weight cycle using the weights w if and only
if it has a negative-weight cycle using w’.
C. If the graph has no negative-weight cycle, then w’ are non-negative.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 62
All-Pairs Shortest Paths
Johnson’s Algorithm for Sparse Graphs
To prove A. A path is shorter than another using the weights w
if and only if it is also shorter using w’.
Instead of a prove, we describe an example:
Suppose a path passes the vertices k,l,m,n,o.
Then w’(p) = w’(k,l) + w’(l,m) + w’(m,n) + w’(n,o)
= [w(k,l)
[w(l,m)
[w(m,n)
[w(n,o)
+ (s,k)
+ (s,l)
+ (s,m)
+ (s,n)
-(s,l) ] +
-(s,m) ] +
-(s,n) ] +
-(s,o) ]
= w(k,l) + w(l,m) + w(m,n) + w(n,o) + (s,k) -(s,o)
= w(p) + (s,k) -(s,o)
Indeed, for any source vertex u to any destination vertex v, we have:
w’(u~pv) = w(u~pv) + (s,u) -(s,v)
Therefore, a path from u to v is shorter than another path from u to v
using
w,
if ofand
onlySemA)
if it is also shorter using w’.
CS3381 Des
& Anal
Alg (2001-2002
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 63
(*)
All-Pairs Shortest Paths
Johnson’s Algorithm for Sparse Graphs
To prove B. A graph has a negative-weight cycle using the weights w
if and only if it has a negative-weight cycle using w’.
From (*), we have
w’(u~pv) = w(u~pv) + (s,u) -(s,v)
Hence, if there is a cycle path u~pu, then
w’(u~pu) = w(u~pu) + (s,u) -(s,u)
= w(u ~pu)
Therefore, a cycle has negative-weight using w if and only if it has
negative weight cycle using w’.
To prove C.
If the graph has no negative-weight cycle,
then w’ are non-negative.
(s,u)
Consider an edge (u,v) between 2 vertices
u
and the paths s~pu and s~pv.
w(u,v)
s
We have (s,v)  (s,u) + w(u,v)
=> w(u,v) + (s,u) - (s,v)  0
(s,v)
v
CS3381
Des & Anal of 
Alg 0
(2001-2002(C
SemA)
=> w’(u,v)
is proved)
http://www.cs.cityu.edu.hk/~helena
City Univ of HK / Dept of CS / Helena Wong
6. Graph Algorithms - 64
All-Pairs Shortest Paths
Johnson’s Algorithm for Sparse Graphs
JOHNSON(G) /* G = (V,E) */
1. Copy G to a new graph Gext where Gext = (Vext, Eext),
using w (the same weight function as G)
2. Add a new vertex {s} to Vext, modify Eext such that
and Eext = E U {(s,v) : v V }
and w(s,v) = 0
3 if BELLMAN-FORD(G,s) = FALSE
4
print “The input graph contains a negative-weight cycle”
5 else
6
for each edge (u,v) in Eext
7
w’(u,v) = w(u,v) + (s,u) -(s,u) /* these  values are obtained in step 3 */
8
for each vertex u in V
9
run DIJKSTRA(G,u) using w’ to compute ’(u,v) for all v in V
10
for each vertex v in V
11
(u,v) = ’(u,v) + h(v) - h(u) /* according to (*) */
O(VE lg V)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 65
Graph Algorithms
Summary
Useful data structures:
• Heaps & Priority Queues
• Disjoint Sets
Graph Algorithms
• Elementary Graph Algorithms
Graph Representations  Breadth-first search  Depth-first search  Topological sort
• Minimum Spanning Trees
Kruskal’s algorithm, Prim’s algorithm
• Single-Source Shortest Paths
Bellman-Ford Algorithm  Algorithm for Directed Acyclic Graph  Dijkstra’s Algorithm
• All-Pairs Shortest Paths
Shortest Paths by repeated squaring  Floyd-Warshall Alg  Johnson’s Alg
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
6. Graph Algorithms - 66