CSE 421
Algorithms
Richard Anderson
Dijkstra’s algorithm
Single Source Shortest Path
Problem
• Given a graph and a start vertex s
– Determine distance of every vertex from s
– Identify shortest paths to each vertex
• Express concisely as a “shortest paths tree”
• Each vertex has a pointer to a predecessor on
shortest path
1
u
1
2
5
s
3
3
x
4
v
u
s
x
3
v
Construct Shortest Path Tree
from s
d
2
d
1
a
4
4
4
-3
s
e
c
3
7
c
s
g
6
b
e
-2
3
2
a
5
g
b
f
3
f
Warmup
• If P is a shortest path from s to v, and if t is
on the path P, the segment from s to t is a
shortest path between s and t
v
t
• WHY?
s
Assume all edges have non-negative cost
Dijkstra’s Algorithm
S = {};
d[s] = 0;
d[v] = infinity for v != s
While S != V
Choose v in V-S with minimum d[v]
Add v to S
For each w in the neighborhood of v
d[w] = min(d[w], d[v] + c(v, w))
0
s
1
u
1
1
4
2
2
v
4
y
3
1
x 2
2
3
2
z 5
Simulate Dijkstra’s algorithm
(strarting from s) on the graph
Round Vertex
Added
a
1
s
3
1
1
c
2
1
4
3
4
6
b
1
2
d
4
3
5
s
a
b
c
d
Dijkstra’s Algorithm as a greedy
algorithm
• Elements committed to the solution by
order of minimum distance
Correctness Proof
• Elements in S have the correct label
• Key to proof: when v is added to S, it has
the correct distance label.
y
x
s
u
v
Proof
• Let v be a vertex in V-S with minimum d[v]
• Let Pv be a path of length d[v], with an edge (u,v)
• Let P be some other path to v. Suppose P first
leaves S on the edge (x, y)
–
–
–
–
P = Psx + c(x,y) + Pyv
Len(Psx) + c(x,y) >= d[y]
Len(Pyv) >= 0
Len(P) >= d[y] + 0 >= d[v]
y
x
s
u
v
Negative Cost Edges
• Draw a small example a negative cost
edge and show that Dijkstra’s algorithm
fails on this example
Bottleneck Shortest Path
• Define the bottleneck distance for a path
to be the maximum cost edge along the
path
u
5
6
s
2
3
x
5
4
v
Compute the bottleneck shortest
paths
d
6
d
6
a
4
4
4
-3
s
e
c
3
7
c
s
g
6
b
e
-2
3
2
a
5
g
b
f
4
f
How do you adapt Dijkstra’s algorithm
to handle bottleneck distances
• Does the correctness proof still apply?
Who was Dijkstra?
• What were his major contributions?
http://www.cs.utexas.edu/users/EWD/
• Edsger Wybe Dijkstra was one of the most
influential members of computing science's
founding generation. Among the domains in
which his scientific contributions are
fundamental are
–
–
–
–
–
–
–
algorithm design
programming languages
program design
operating systems
distributed processing
formal specification and verification
design of mathematical arguments
Shortest Paths
• Negative Cost Edges
– Dijkstra’s algorithm assumes positive cost edges
– For some applications, negative cost edges make
sense
– Shortest path not well defined if a graph has a
negative cost cycle
a
6
4
-4
-3
s
4
e
c
-2
3
2
3
6
g
b
f
7
4
Negative Cost Edge Preview
• Topological Sort can be used for solving
the shortest path problem in directed
acyclic graphs
• Bellman-Ford algorithm finds shortest
paths in a graph with negative cost edges
(or reports the existence of a negative cost
cycle).
Dijkstra’s Algorithm
Implementation and Runtime
S = {};
d[s] = 0;
d[v] = infinity for v != s
While S != V
Choose v in V-S with minimum d[v]
Add v to S
For each w in the neighborhood of v
d[w] = min(d[w], d[v] + c(v, w))
y
u
s
a
HEAP OPERATIONS
n Extract Mins
x
v
m Heap Updates
b
z
Edge costs are assumed to be non-negative
Bottleneck Shortest Path
• Define the bottleneck distance for a path
to be the maximum cost edge along the
path
u
5
6
s
2
3
x
5
4
v
Compute the bottleneck shortest
paths
d
6
d
6
a
4
4
4
-3
s
e
c
3
7
c
s
g
5
b
e
-2
3
2
a
5
g
b
f
7
f
Dijkstra’s Algorithm
for Bottleneck Shortest Paths
S = {};
d[s] = negative infinity;
d[v] = infinity for v != s
While S != V
Choose v in V-S with minimum d[v]
Add v to S
For each w in the neighborhood of v
d[w] = min(d[w], max(d[v], c(v, w)))
y
4
1
u
s
2
2 4
3
x
1
v
5
4
3
z
a
1
3
b
Minimum Spanning Tree
• Introduce Problem
• Demonstrate three different greedy
algorithms
• Provide proofs that the algorithms work
Minimum Spanning Tree
15
t
a
6
9
14
3
10
13
s
17
1
e
c
11
20
2
5
7
b
u
4
8
12
16
v
g
f
22
Greedy Algorithms for Minimum
Spanning Tree
• Extend a tree by
including the
cheapest out going
edge
• Add the cheapest
edge that joins
disjoint components
• Delete the most
expensive edge that
does not disconnect
the graph
4
b
20
c
5
11
8
a
d
7
e
22
Greedy Algorithm 1
Prim’s Algorithm
• Extend a tree by including the cheapest
out going edge
15
t
a
3
10
Construct the MST
with Prim’s
algorithm starting
from vertex a
Label the edges in
order of insertion
14
9
13
s
17
1
4
e
c
20
2
5
7
b
u
6
8
12
16
v
11
g
f
22
Greedy Algorithm 2
Kruskal’s Algorithm
• Add the cheapest edge that joins disjoint
components
15
t
a
3
10
Construct the MST
with Kruskal’s
algorithm
Label the edges in
order of insertion
14
9
13
s
17
1
4
e
c
20
2
5
7
b
u
6
8
12
16
v
11
g
f
22
Greedy Algorithm 3
Reverse-Delete Algorithm
• Delete the most expensive edge that does
not disconnect the graph
15
t
a
3
10
Construct the MST
with the reversedelete algorithm
Label the edges in
order of removal
14
9
13
s
17
1
4
e
c
20
2
5
7
b
u
6
8
12
16
v
11
g
f
22
Why do the greedy algorithms
work?
• For simplicity, assume all edge costs are
distinct
• Let S be a subset of V, and suppose e =
(u, v) is the minimum cost edge of E, with
u in S and v in V-S
• e is in every minimum spanning tree
Proof
• Suppose T is a spanning tree that does not
contain e
• Add e to T, this creates a cycle
• The cycle must have some edge e1 = (u1, v1)
with u1 in S and v1 in V-S
• T1 = T – {e1} + {e} is a spanning tree with lower
cost
• Hence, T is not a minimum spanning tree
Optimality Proofs
• Prim’s Algorithm computes a MST
• Kruskal’s Algorithm computes a MST
Reverse-Delete Algorithm
• Lemma: The most expensive edge on a
cycle is never in a minimum spanning tree
Dealing with the assumption of no
equal weight edges
• Force the edge weights to be distinct
– Add small quantities to the weights
– Give a tie breaking rule for equal weight
edges
Dijkstra’s Algorithm
for Minimum Spanning Trees
S = {};
d[s] = 0;
d[v] = infinity for v != s
While S != V
Choose v in V-S with minimum d[v]
Add v to S
For each w in the neighborhood of v
d[w] = min(d[w], c(v, w))
y
4
1
u
s
2
2 4
3
x
1
v
5
4
3
z
a
1
3
b
Minimum Spanning Tree
Undirected Graph
G=(V,E) with edge
weights
15
t
a
6
9
14
3
10
13
s
17
1
e
c
11
20
2
5
7
b
u
4
8
12
16
v
g
f
22
Greedy Algorithms for Minimum
Spanning Tree
• [Prim] Extend a tree by
including the cheapest
out going edge
• [Kruskal] Add the
cheapest edge that joins
disjoint components
• [ReverseDelete] Delete
the most expensive edge
that does not disconnect
the graph
4
b
20
c
5
11
8
a
d
7
e
22
Why do the greedy algorithms
work?
• For simplicity, assume all edge costs are
distinct
Edge inclusion lemma
• Let S be a subset of V, and suppose e =
(u, v) is the minimum cost edge of E, with
u in S and v in V-S
• e is in every minimum spanning tree of G
– Or equivalently, if e is not in T, then T is not a
minimum spanning tree
e
S
V-S
e is the minimum cost edge
between S and V-S
Proof
• Suppose T is a spanning tree that does not contain e
• Add e to T, this creates a cycle
• The cycle must have some edge e1 = (u1, v1) with u1 in S
and v1 in V-S
S
e
V-S
• T1 = T – {e1} + {e} is a spanning tree with lower cost
• Hence, T is not a minimum spanning tree
Optimality Proofs
• Prim’s Algorithm computes a MST
• Kruskal’s Algorithm computes a MST
• Show that when an edge is added to the
MST by Prim or Kruskal, the edge is the
minimum cost edge between S and V-S
for some set S.
Prim’s Algorithm
S = { };
T = { };
while S != V
choose the minimum cost edge
e = (u,v), with u in S, and v in V-S
add e to T
add v to S
Prove Prim’s algorithm computes
an MST
• Show an edge e is in the MST when it is
added to T
Kruskal’s Algorithm
Let C = {{v1}, {v2}, . . ., {vn}}; T = { }
while |C| > 1
Let e = (u, v) with u in Ci and v in Cj be the
minimum cost edge joining distinct sets in C
Replace Ci and Cj by Ci U Cj
Add e to T
Prove Kruskal’s algorithm
computes an MST
• Show an edge e is in the MST when it is
added to T
Reverse-Delete Algorithm
• Lemma: The most expensive edge on a
cycle is never in a minimum spanning tree
Dealing with the assumption of no
equal weight edges
• Force the edge weights to be distinct
– Add small quantities to the weights
– Give a tie breaking rule for equal weight
edges
Application: Clustering
• Given a collection of points in an rdimensional space, and an integer K,
divide the points into K sets that are
closest together
Distance clustering
• Divide the data set into K subsets to
maximize the distance between any pair of
sets
– dist (S1, S2) = min {dist(x, y) | x in S1, y in S2}
Divide into 2 clusters
Divide into 3 clusters
Divide into 4 clusters
Distance Clustering Algorithm
Let C = {{v1}, {v2},. . ., {vn}}; T = { }
while |C| > K
Let e = (u, v) with u in Ci and v in Cj be the
minimum cost edge joining distinct sets in C
Replace Ci and Cj by Ci U Cj
K-clustering