# Document

```SFO
CHAPTER 13
GRAPH ALGORITHMS
LAX
ORD
DFW
ACKNOWLEDGEMENT: THESE SLIDES ARE ADAPTED FROM SLIDES PROVIDED WITH
DATA STRUCTURES AND ALGORITHMS IN C++, GOODRICH, TAMASSIA AND
MOUNT (WILEY 2004) AND SLIDES FROM NANCY M. AMATO
2704
BOS
867
849
PVD
ORD
740
621
1846
MINIMUM SPANNING TREES
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
BWI
1090
DFW
946
1121
MIA
2342
REMINDER
WEIGHTED GRAPHS
• In a weighted graph, each edge has an associated numerical value, called the weight of the
edge
• Edge weights may represent, distances, costs, etc.
• Example:
• In a flight route graph, the weight of an edge represents the distance in miles between the endpoint
airports
SFO
PVD
ORD
LGA
HNL
LAX
DFW
MIA
MINIMUM SPANNING TREE
ORD
• Spanning subgraph
• Subgraph of a graph πΊ containing all the
vertices of πΊ
1
DEN
• Spanning tree
• Spanning subgraph that is itself a (free) tree
• Minimum spanning tree (MST)
10
PIT
9
STL
4
8
• Spanning tree of a weighted graph with
6
7
3
DCA
5
2
minimum total edge weight
• Applications
• Communications networks
• Transportation networks
DFW
ATL
EXERCISE
MST
• Show an MST of the following graph.
SFO
PVD
ORD
LGA
HNL
LAX
DFW
MIA
CYCLE PROPERTY
8
f
• Cycle Property:
2
6
• Let π be a minimum spanning tree of a
C
3
e
7
7
• Let π be an edge of πΊ that is not in π and
πΆ let be the cycle formed by π with π
Replacing f with e yields
a better spanning tree
• For every edge π of πΆ, π€πππβπ‘ π ≤
π€πππβπ‘ π
f
• Proof:
spanning tree of smaller weight by replacing
π with π
9
8
weighted graph πΊ
• If π€πππβπ‘ π &gt; π€πππβπ‘(π) we can get a
4
2
6
8
4
C
9
3
8
7
e
7
PARTITION PROPERTY
•
•
Partition Property:
•
Consider a partition of the vertices of πΊ into subsets π
and π
•
•
Let π be an edge of minimum weight across the partition
U
f
9
8
8
Replacing f with e yields
another MST
U
Let π be an MST of πΊ
•
By the cycle property,
π€πππβπ‘ π ≤ π€πππβπ‘(π)
•
•
Thus, π€πππβπ‘ π = π€πππβπ‘ π
We obtain another MST by replacing π with π
3
e
7
There is a minimum spanning tree of G containing edge
π
If π does not contain π, consider the cycle πΆ formed by
π with π and let π be an edge of πΆ across the partition
4
5
2
Proof:
•
•
V
7
f
2
V
7
4
9
5
8
8
e
7
3
PRIM-JARNIK’S ALGORITHM
•
We pick an arbitrary vertex π  and we grow the MST as a cloud of vertices, starting
from π
•
We store with each vertex π£ a label π π£ = the smallest weight of an edge
connecting π£ to a vertex in the cloud
•
At each step:
•
•
We add to the cloud the vertex π’ outside the
cloud with the smallest distance label
We update the labels of the vertices adjacent
to π’
PRIM-JARNIK’S ALGORITHM
• An adaptable priority queue stores the vertices
outside the cloud
• Key: distance, π·[π£]
• Element: vertex π£
• π. ππππππππΎππ¦ π, π changes the key of an item
• We store three labels with each vertex π£:
• Distance π·[π£]
• Parent edge in MST π[π£]
• Locator in priority queue
Algorithm PrimJarnikMST(πΊ)
Input: A weighted connected graph πΊ
Output: A minimum spanning tree π of πΊ
1. Pick any vertex π£ of πΊ
2. π· π£ ← 0; π π£ ← ∅
3. for each vertex π’ ≠ π£ do
4. π· π’ ← ∞; π π’ ← ∅
5. π ← ∅
6. Priority queue π of vertices with π·[π’] as the key
7. while &not;π. empty( ) do
8. π’ ← π. removeMin( )
9. Add vertex π’ and edge π[π’] to π
10. for each π ∈ π’. incidentEdges do
11. π§ ← π. opposite π’
12. if π. weight( ) &lt; π·[π§]
13.
π· π§ ← π. weight( ); π π§ ← π
14.
π. replaceKey π§, π·[π§]
15. return π
EXAMPLE
2
7
B
0
2
B
5
C
0
2
0
A
4
9
5
C
5
F
8
8
7
E
7
7
2
4
F
8
7
ο₯
B
7
D
7
3
9
8
A
D
7
5
F
E
7
2
2
4
8
8
A
ο₯
9
8
C
5
2
D
E
ο₯
2
3
7
7
B
0
3
7
7
4
9
5
C
5
F
8
8
A
D
7
ο₯
E
3
7
4
EXAMPLE
2
2
B
0
4
9
5
C
5
F
8
8
A
D
7
7
7
E
4
3
3
2
2
B
0
4
9
5
C
5
F
8
8
A
D
7
7
7
E
3
3
4
EXERCISE
PRIM’S MST ALGORITHM
• Show how Prim’s MST algorithm works on the following graph, assuming you
• Show how the MST evolves in each iteration (a separate figure for each iteration).
SFO
PVD
ORD
LGA
HNL
LAX
DFW
MIA
ANALYSIS
• Graph operations
• Method incidentEdges is called once for each vertex
• Label operations
• We set/get the distance, parent and locator labels of vertex π§ π deg π§ times
• Setting/getting a label takes π 1 time
• Priority queue operations
• Each vertex is inserted once into and removed once from the priority queue, where each insertion or
•
removal takes π log π time
The key of a vertex π€ in the priority queue is modified at most deg π€ times, where each key change
takes π log π time
• Prim-Jarnik’s algorithm runs in π π + π log π time provided the graph is represented by
• Recall that Σπ£ deg π£ = 2π
• The running time is π π log π since the graph is connected
KRUSKAL’S ALGORITHMS
• A priority queue stores the edges outside the cloud
• Key: weight
• Element: edge
• At the end of the algorithm
• We are left with one cloud that encompasses the MST
• A tree T which is our MST
Algorithm KruskalMST(πΊ)
1. for each vertex π£ ∈ πΊ. vertices do
2. Define a cluster πΆ π£ ← π£
3. Initialize a priority queue π of edges using the
weights as keys
4. π ← ∅
5. while π has fewer than π − 1 edges do
6.
π’, π£ ← π. removeMin( )
7. if πΆ π£ ≠ πΆ(π’) then
8.
9.
Merge πΆ π£ and πΆ π’
10. return π
DATA STRUCTURE FOR KRUSKAL’S ALGORITHM
• The algorithm maintains a forest of trees
• An edge is accepted it if connects distinct
trees
• We need a data structure that maintains a
partition, i.e., a collection of disjoint sets,
with the operations:
• find π’ : return the set storing u
• union π’, π£ : replace the sets storing u and v
with their union
REPRESENTATION OF A PARTITION
• Each set is stored in a sequence
• Each element has a reference back to the set
• Operation ππππ π’ takes π 1 time, and returns the set of which π’ is a member.
• In operation π’ππππ π’, π£ , we move the elements of the smaller set to the sequence of the larger set and
•
update their references
The time for operation π’ππππ(π’, π£) is π min ππ’ , ππ£ , where ππ’ and ππ£ are the sizes of the sets storing
π’ and π£
• Whenever an element is processed, it goes into
a set of size at least double, hence each element
is processed at most log π times
ANALYSIS
• A partition-based version of Kruskal’s Algorithm performs cluster merges as
unions and tests as finds.
• Running time
• There will be at most π removals from the priority queue - π π log π
• Each vertex can be merged at most log π times, as the clouds tend to “double” in size π π log π
• Total: π π + π log π
2704
EXAMPLE
BOS
867
849
ORD
740
621
1846
337
LAX
1391
1464
1235
187
144
JFK
1258
184
802
SFO
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXAMPLE
2704
BOS
867
849
ORD
LAX
1391
1464
1235
144
JFK
1258
184
802
SFO
337
187
740
621
1846
PVD
BWI
1090
DFW
946
1121
MIA
2342
EXERCISE
KRUSKAL’S MST ALGORITHM
• Show how Kruskal’s MST algorithm works on the following graph.
• Show how the MST evolves in each iteration (a separate figure for each iteration).
SFO
PVD
ORD
LGA
HNL
LAX
DFW
MIA
A
8
SHORTEST PATHS
0
4
2
B
2
8
7
5
E
C
3
2
1
9
D
8
F
5
3
WEIGHTED GRAPHS
•
In a weighted graph, each edge has an associated numerical value, called the
weight of the edge
•
•
Edge weights may represent, distances, costs, etc.
Example:
•
In a flight route graph, the weight of an edge represents the distance in miles between the
endpoint airports
SFO
PVD
ORD
LGA
HNL
LAX
DFW
MIA
SHORTEST PATH PROBLEM
•
Given a weighted graph and two vertices π’ and π£, we want to find a path of
minimum total weight between π’ and π£.
•
•
Example:
•
•
Length of a path is the sum of the weights of its edges.
Shortest path between Providence and Honolulu
Applications
•
•
•
Internet packet routing
Flight reservations
Driving directions
HNL
SFO
PVD
ORD
LGA
LAX
DFW
MIA
SHORTEST PATH PROBLEM
• If there is no path from π£ to π’, we denote the distance between them by
π π£, π’ = ∞
• What if there is a negative-weight cycle in the graph?
SFO
PVD
ORD
LGA
HNL
LAX
DFW
MIA
SHORTEST PATH PROPERTIES
• Property 1:
• A subpath of a shortest path is itself a shortest path
• Property 2:
• There is a tree of shortest paths from a start vertex to all the other vertices
• Example:
• Tree of shortest paths
SFO
LGA
from Providence
HNL
PVD
ORD
LAX
DFW
MIA
DIJKSTRA’S ALGORITHM
• The distance of a vertex π£ from a vertex π  is the
length of a shortest path between π  and π£
• Dijkstra’s algorithm computes the distances of all
the vertices from a given start vertex π
(single-source shortest paths)
• Assumptions:
• The graph is connected
• The edges are undirected
• The edge weights are nonnegative
• Extremely similar to Prim-Jarnik’s MST Algorithm
• We grow a “cloud” of vertices, beginning with π
and eventually covering all the vertices
• We store with each vertex π£ a label π· π£
representing the distance of π£ from π  in the
subgraph consisting of the cloud and its adjacent
vertices
• The label π· π£ is initialized to positive infinity
• At each step
• We add to the cloud the vertex π’ outside the cloud with
the smallest distance label, π· π£
• We update the labels of the vertices adjacent to π’, in a
process called edge relaxation
EDGE RELAXATION
•
Consider an edge π = π’, π§ such that
•
•
•
•
D[u] = 50
π’ is the vertex most recently added to the
cloud
s
u
e
D[z] = 75
z
D[y] = 20
y
π§ is not in the cloud
The relaxation of edge π updates
distance π· π§ as follows:
D[u] = 50
π· π§ ← min π· π§ , π· π’ + π. π€πππβπ‘
u
s
D[y] = 20
y
D[z] = 60
e
z
EXAMPLE
A
8
B
2
8
7
Pull in one of the vertices with red labels
2.
The relaxation of edges updates the labels of LARGER
font size
0
4
2
3
ο₯
2
C
1.
1
9
E
D
ο₯
F
A
8
B
2
7
5
E
C
3
4
5
B
8
7
5
E
2
4
2
9
1
C
3
11
F
5
7
B
2
2
1
D
9
8
A
3
5
F
8
3
D
4
2
0
2
8
A
8
0
0
4
2
7
5
E
C
3
2
1
9
D
8
F
5
3
EXAMPLE
A
8
0
4
2
B
2
7
7
C
3
5
E
2
1
9
D
8
F
3
5
A
8
0
4
2
B
2
7
7
C
3
5
E
2
1
9
D
8
F
5
3
EXERCISE
DIJKSTRA’S ALGORITHM
• Show how Dijkstra’s algorithm works on the following graph, assuming you
• Show how the labels are updated in each iteration (a separate figure for each iteration).
SFO
PVD
ORD
LGA
HNL
LAX
DFW
MIA
DIJKSTRA’S ALGORITHM
• A locator-based priority queue stores the
vertices outside the cloud
• Key: distance
• Element: vertex
• We store with each vertex:
•
•
distance π· π£ label
locator in priority queue
Algorithm Dijkstras_sssp πΊ, π
Input: A simple undirected weighted graph πΊ with nonnegative edge weights
and a source vertex π
Output: A label π· π£ for each vertex π£ of πΊ, such that π· π’ is the length
of the shorted path from π  to π£
1. π· π  ← 0; π· π£ ← ∞ for each vertex π£ ≠ π
2. Let priority queue π contain all the vertices of πΊ using π· π£ as the key
3. while &not;π. empty do //π π iterations
4.
//pull a new vertex π’ in the cloud
5.
π’ ← π. removeMin //π log π
6.
for each edge π ∈ π’. incidentEdges( ) do //π πππ π’ iterations
7.
//relax edge π
8.
π§ ← π. opposite π’
9.
if π· π’ + π. weight &lt; π·[π§] then
10.
π· π§ ← π· π’ + π. weight
11.
π. updateKey π§ //π log π
ANALYSIS
• Graph operations
• Method incidentEdges is called once for each vertex
• Label operations
• We set/get the distance and locator labels of vertex π§ π deg π§ times
• Setting/getting a label takes π 1 time
• Priority queue operations
• Each vertex is inserted once into and removed once from the priority queue, where each insertion or removal takes π log π
time
• The key of a vertex in the priority queue is modified at most deg π€ times, where each key change takes π log π time
• Dijkstra’s algorithm runs in π π + π log π time provided the graph is represented by the adjacency list
structure
• Recall that Σπ£ deg π£ = 2π
• The running time can also be expressed as π π log π since the graph is connected
• The running time can be expressed as a function of π, π π2 log π
EXTENSION
• Using the template method pattern, we can extend Dijkstra’s algorithm to
return a tree of shortest paths from the start vertex to all other vertices
• We store with each vertex a third label:
• parent edge in the shortest path tree
• Parents are all initialized to null
• In the edge relaxation step, we update the parent label as well
WHY DIJKSTRA’S ALGORITHM WORKS
•
•
Dijkstra’s algorithm is based on the greedy
method. It adds vertices by increasing distance.
•
•
•
•
Suppose it didn’t find all shortest distances. Let πΉ
be the first wrong vertex the algorithm processed.
When the previous node, π·, on the true shortest
path was considered, its distance was correct.
But the edge π·, πΉ was relaxed at that time!
Thus, so long as π· πΉ &gt; π· π· , πΉ’s distance
cannot be wrong. That is, there is no wrong
vertex.
s
8
0
4
2
B
2
7
7
C
3
5
E
2
9
1
D
8
F
5
3
WHY IT DOESN’T WORK FOR
NEGATIVE-WEIGHT EDGES
A
8
• Dijkstra’s algorithm is based on the
greedy method. It adds vertices by
increasing distance.
• If a node with a negative incident edge
B
8
7
C
4
2
3
ο₯
2
were to be added late to the cloud, it
could mess up distances for vertices
2
0
-3
9
E
D
ο₯
F
A
8
5
0
4
2
B
2
8
7
5
E
C
3
2
9
-3
11
F
D
5
4
C’s true
distance is 1,
in the cloud
with π· πΆ = 2!
0
BELLMAN-FORD ALGORITHM
• Works even with negative-weight edges
• Must assume directed edges (for otherwise
we would have negative-weight cycles)
• Iteration π finds all shortest paths that use π
edges.
• Running time: π ππ
• Can be extended to detect a negativeweight cycle if it exists
• How?
Algorithm BellmanFord πΊ, π
1. Initialize π· π  ← 0 and π· π£ ← ∞ for all
vertices π£ ≠ π
2. for π ← 1 … π − 1 do
3. for each π ∈ πΊ. edges do
4.
//relax edge π
5.
π’ ← π. source ; π§ ← π. target
6.
if π· π’ + π. weight &lt; π· π§ then
7.
π· π§ ← π· π’ + π. weight
• Nodes are labeled with their π·[π£] values
BELLMAN-FORD EXAMPLE
0
8
First
round 8
4
0
-2
ο₯
7
-2
1
ο₯
3
-2
ο₯
8
7
9
ο₯
5
0
-2
ο₯
7
0
-2
1
4
-2
1
-2
3
5
ο₯
-2
5
4
9
Third
round 8
4
1
-2
3
ο₯
Second
round 8
4
-1
5
7
9
3
9
5
-2
1
1
-2
-1
9
4
5
EXERCISE
BELLMAN-FORD’S ALGORITHM
• Show how Bellman-Ford’s algorithm works on the following graph, assuming
• Show how the labels are updated in each iteration (a separate figure for each iteration).
0
8
4
-2
ο₯
7
ο₯
3
-5
ο₯
1
ο₯
ο₯
5
9
ALL-PAIRS SHORTEST PATHS
• Find the distance between every pair of vertices in a weighted
directed graph πΊ
• We can make π calls to Dijkstra’s algorithm (if no negative
edges), which takes π ππ log π time.
• Likewise, π calls to Bellman-Ford would take π π2 π time.
• We can achieve π π3 time using dynamic programming
(similar to the Floyd-Warshall algorithm).
i
Uses only vertices numbered π, … , π
(compute weight of this edge)
Uses only vertices
numbered π, … , π
j
k
Uses only vertices
numbered π, … , π
Input: Graph πΊ with vertices labeled 1, … , π
Output: Distances π· π, π of shortest path lengths between
each pair of vertices
1. for each vertex pair π, π do
2. if π = π then
3.
π·0 π, π ← 0
4. else if π = π, π is an edge in πΊ then
5.
π·0 π, π ← π. weight
6. else
7.
π·0 π, π ← ∞
8. for π ← 1 … π do
9. for π ← 1 … π do
10. for π ← 1 … π do
11.
π·π π, π ← min π·π−1 π, π , π·π−1 π, π + π·π−1 π, π
12. return π·π
```