Greedy Strategy
Algorithm : Design & Analysis
[14]
In the last class…
„
„
„
Undirected and Symmetric Digraph
UDF Search Skeleton
Biconnected Components
„
„
„
Articulation Points and Biconnectedness
Biconnected Component Algorithm
Analysis of the Algorithm
Greedy Strategy
„
„
Optimization Problem
MST Problem
„
„
„
Single-Source Shortest Path Problem
„
„
Prim’s Algorithm
Kruskal’s Algorithm
Dijstra’s Algorithm
Greedy Strategy
Optimizing by Greedy
„
Coin Change Problem
„
„
„
„
[candidates] A finite set of coins, of 1, 5, 10 and 25 units,
with enough number for each value
[constraints] Pay an exact amount by a selected set of coins
[optimization] a smallest possible number of coins in the
selected set
Solution by greedy strategy
„
For each selection, choose the highest-valued coin as
possible.
Greedy Fails Sometimes
„
If the available coins are of 1,5,12 units, and we
have to pay 15 units totally, then the smallest set
of coins is {5,5,5}, but not {12,1,1,1}
„
However, the correctness of greedy strategy on
the case of {1,5,10,25} is not straightly seen.
Greedy Strategy
„
Constructing the final
setgreedy(set
greedy(setcandidate)
candidate)
set
solution by expanding the
setS=Ø;
S=Ø;
set
partial solution step by step,
n
o
whilefnot
not
solution(S)
and
f
in each of which a selection
while
solution(S)
and
”
ocandidate≠Ø
n
o
g
is made from a set of
i
candidate≠Ø
n
t
i
a
z
candidates, with the choice ad select
i
locally
optimizingxx
r
select
locally
optimizing
m
t
”
i
made must be:
:
y
t
t
y
from
candidate;
p
i
from
candidate;
e
l
o
i
„ [feasible] it has to satisfy
K thecal
b
i
s
candidate=candidate-{x};
problem’s constraints
candidate=candidate-{x};
a
o
e
l
f
“
“
„ [locally optimal] it has to be
iffeasible(x)
feasible(x)then
thenS=S∪{x};
S=S∪{x};
if
d
nall if solution(S) then return S
the best local choice among
a
feasible choices on the step
if solution(S) then return S
„ [irrevocable] the candidate
elsereturn
return(“no
(“nosolution”)
solution”)
else
selected can never be deselected on subsequent steps
Weighted Graph and MST
SpanningTree:
Tree:W(T)=257
W(T)=257
AASpanning
B
26
27
21
A
B
27
26
21
C
A
34
H
36
25
18
29
33
I
J
17
D
21
25
21
28
E
42
22
16
C
36
34 25
33
18 I 21
H
J
28
17
53
G
B
F
53
G
A weighted graph
The nearest neighbor of vertex I is H
The nearest neighbor of shaded
subset of vertex is G
42
29 D
A
34
H
21
C
36
22
16
21
25
F
MST:W(T)=190
W(T)=190
AAMST:
26
27
E
42
E
D 22
33 29
25
16
I
21
18
21
F
25
J
28
17
53
G
Graph Traversal and MST
There are cases that
graph traversal tree
cannot be minimum
spanning tree, with
the vertices explored
in any order.
All other edges
with weight 5
1
1
1
1
1
DFS tree
BFS tree
in any ordering of vertex
Greedy Algorithms for MST
„
Prim’s algorithm:
„
„
„
Difficult selecting: “best local optimization means
no cycle and small weight under limitation.
Easy checking: doing nothing
Kruskal’s algorithm:
„
„
Easy selecting: smallest in primitive meaning
Difficult checking: no cycle
Merging Two Vertices
v0 ’
v0
v0 ”
v2
v2
v1
v5
v5
v4
v4
v6
v3
v5
v4
v6
v3
v6
v3
Constructing a Spanning Tree
a
b
a
b
a
b
a
b
c
d
c
d
c
d
c
d
(0)
(1)
(2)
(3)
Letaabe
bethe
thestarting
startingvertex,
vertex,selecting
selectingedges
edgesone
oneby
byone
oneininoriginal
originalgraph
graph
0.0.Let
Mergingaaand
andccinto
intoa’({a,c}),
a’({a,c}),selecting
selecting(a,c)
(a,c)
1.1.Merging
Merginga’a’and
andbbinto
intoa”({a,c,b}),
a”({a,c,b}),selecting
selecting(c,b)
(c,b)
2.2.Merging
Merginga”
a”and
andddinto
intoa”’({a,c,b,d}),
a”’({a,c,b,d}),selecting
selecting(a,d)
(a,d)or
or(d,b)
(d,b)
3.3.Merging
Ending,as
asonly
onlyone
onevertex
vertexleft
left
Ending,
Prim’s Algorithm for MST
2
A
B
6
3
7
G
1
5
F
I
4
E
3
H
8
2
6
4
1
C
2
2
D
edges included in the MST
Greedy
Greedystrategy:
strategy:
For
Foreach
eachset
setof
offringe
fringe
vertex,
vertex,select
selectthe
theedge
edge
with
withthe
theminimal
minimal
weight,
weight,that
thatis,
is,local
local
optimal.
optimal.
Minimum Spanning Tree Property
„
„
A spanning tree T of a connected, weighted graph has MST property if and
only if for any non-tree edge uv, T∪{uv} contain a cycle in which uv is one of
the maximum-weight edge.
All the spanning trees having MST property have the same weight.
uv-path in T1
edge exchange
u
Must have
same weight
v
u
wi
wi+1 not in T2
edge uv in T2 but not in T1 , with minimum
weight among all different edges
v
×
wi
wi+1
a new spanning tree: same
weight as T1, less different
edges from that of T2
MST Property and Minimum Spanning Tree
„
„
In a connected, weighted graph G=(V,E,W), a tree T
is a minimum spanning tree if and only if T has the
MST property.
Proof
„
„
⇒ For a minimum spanning tree T, if it doesn’t has MST
property. So, there is a non-tree edge uv, and T∪{uv}
contain an edge xy with weight larger than that of uv.
Substituting uv for xy results a spanning tree with less
weight than T. Contradiction.
⇐ As claimed above, any minimum spanning tree has the
MST property. Since T has MST property, it has the same
weight as any minimum spanning tree, i.e. T is a minimum
spanning tree as well.
Correctness of Prim’s Algorithm
„
Let Tk be the tree constructed after the kth step of Prim’s
algorithm is executed, then Tk has the MST property in Gk, the
subgraph of G induced by vertices of Tk.
wa+1
wb-1
assumed first and
Tk-1
wa
wb
……
u1(w1)
edge added in Tk
Note: w(uiv)≥w(u1v), and if wa
added earlier than wb, then
wawa+1 and wb-1wb added later
than any edges in u1wa-path,
and v as well
v,
ui(wp)
last edges with
larger weight than
w(uiv), resulting
contradictions.
added in Tk to form a cycle,
added in Tk only these need be considered
Key Issue in Implementation
„
Maintaining the set of fringe vertices
„
„
„
„
Create the set and update it after each vertex is
“selected” (deleting the vertex having been
selected and inserting new fringe vertices)
Easy to decide the vertex with “highest priority”
Changing the priority of the vertices (decreasing
key).
The choice: priority queue
Implementing Prim’s Algorithm
Main Procedure
ADT operation executions:
insert, getMin, deleteMin: n times
decreaseKey: m times
primMST(G,n)
Initialize the priority queue pq as empty;
Select vertex s to start the tree;
Set its candidate edge to (-1,s,0);
Updating the Queue
insert(pq,s,0);
updateFringe(pq,G,v)
while (pq is not empty)
For all vertices w adjcent to v //2m loops
v=getMin(pq); deleteMin(pq);
newWgt=w(v,w);
add the candidate edge of v to the tree;
if w.status is unseen then
updateFringe(pq,G,v);
Set its candidate edge to (v,w,newWgt);
return
insert(pq,w,newWgt)
else
getMin(pq) always be the
if newWgt<getPriorty(pq,w)
vertex with the smallest
Revise its candidate edge to (v,w,newWgt);
key in the fringe set.
decreaseKey(pq,w,newWgt)
return
Prim’s Algorithm for MST
2
A
×
3
7
1
5
F
I
6
×
G
×4
E
4
3
H
8
2
6
B
1
C
2
2
D
edges included in the MST
Greedy
Greedystrategy:
strategy:
For
Foreach
eachset
setof
offringe
fringe
vertex,
vertex,select
selectthe
theedge
edge
with
withthe
theminimal
minimal
weight,
weight,that
thatis,
is,local
local
optimal.
optimal.
Complexity of Prim’s Algorithm
„
Operations on ADT priority queue: (for a graph with n vertices
and m edges)
„
„
„
„
„
insert: n
getMin: n
deleteMin: n
decreasKey: m (appears in 2m loops, but execute at most m)
So,
T(n,m) = O(nT(getMin)+nT(deleteMin+insert)+mT(decreaseKey))
„
Implementing priority queue using heap, we can get Θ(n2+m)
Kruskal’s Algorithm for MST
2
A
B
6
3
7
G
1
5
F
I
4
E
3
H
8
2
6
4
1
C
2
2
D
edges included in the MST
Also
AlsoGreedy
Greedystrategy:
strategy:
From
Fromthe
theset
setof
ofedges
edges
not
notyet
yetincluded
includedin
inthe
the
partially
partiallybuilt
builtMST,
MST,
select
selectthe
theedge
edgewith
with
the
theminimal
minimalweight,
weight,
that
thatis,
is,local
localoptimal,
optimal,
in
inanother
anothersense.
sense.
Key Issue in Implementation
„
How to know an insertion of edge will result in
a cycle efficiently?
„
For correctness: the two endpoints of the
selected edge can not be in the same connected
components.
„
For the efficiency: connected components are
implemented as dynamic equivalence classes
using union-find.
Kruskal’s Algorithm: the Procedure
„
„
„
„
kruskalMST(G,n,F) //outline
int count;
Build a minimizing priority queue, pq, of edges of G, prioritized by weight.
Initialize a Union-Find structure, sets, in which each vertex of G is in its own set.
„
„
„
„
„
„
„
„
„
„
„
F=φ;
while (isEmpty(pq) == false)
vwEdge = getMin(pq);
deleteMin(pq);
int vSet = find(sets, vwEdge.from);
int wSet = find(sets, vwEdge.to);
if (vSet ≠ wSet)
Add vwEdge to F;
union(sets, vSet, wSet)
return
Simply
Simplysorting,
sorting,the
the
cost
costwill
willbe
beΘΘ(mlogm)
(mlogm)
Prim vs. Kruskal
„
Lower bound for MST
„
„
„
For a correct MST, each edge in the graph should
be examined at least once.
So, the lower bound is Ω(m)
Θ(n2+m) and Θ(mlogm), which is better?
„
Generally speaking, depends on the density of
edge of the graph.
Single Source Shortest Paths
The single source
Red labels on each vertex is the length
of the shortest path from s to the vertex.
1
Note:
Note:
Theshortest
shortest[0,
[0,3]3]The
pathdoesn’t
doesn’tcontain
contain
path
theshortest
shortestedge
edge
the
leavings,s,the
theedge
edge
leaving
[0,1]
[0,1]
7
8
6
2
1
5
2
3
3
3
5
1
3
6
7
s 0
4
4
4
2
2
4
9
6
Dijstra’s Algorithm: an Example
1∞
7
∞
2
2
5
3
1
1
2
∞
68
8
2
s 0
7
3
4
3
∞76
4∞
3
4
4
6
4
∞
5
∞9
Home Assignment
„
pp.416-:
„
„
„
8.7-8.9
8.14-15
8.25