Week 2: Greedy Algorithms
Page 1
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Greedy Algorithms:
 A greedy algorithm always makes the choice
that looks best at the moment.
 It makes a local optimal choice in the hope
that this choice will lead to a globally
optimal solution.
 Greedy algorithms yield optimal solutions
for many (but not all) problems.
Page 2
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Knapsack problem:
Page 3
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Knapsack problem:
Input: n items: (w1, v1), …, (wn, vn) and a number W

the i-th item’s is worth vi dollars with weight wi.

at most W pounds to be carried.
Output: Some items which are most valuable and
with total weight at most W.
 Two versions:


0-1 Knapsack: Items cannot be divided into pieces
fractional knapsack: Items can be divided into pieces
Page 4
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
The 0-1 Knapsack problem:
 The 0-1 knapsack problem:
 N items, where the i-th item is worth vi dollars and
weight wi pounds.
11 p 3 p 4p 58 p 8p 88p

vi and wi are integers.
3$
6$
35$
8$
 We can carry at most W (integer) pounds.
 How to take as valuable a load as possible.

66$
An item cannot be divided into pieces.
 The fractional knapsack problem:


28$
The same setting, but the thief can take fractions of items.
W may not be integer.
W
Page 5
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Solve the fractional Knapsack problem:
 Greedy on the value per pound vi/wi.
 Each time, take the item with maximum vi/wi .





If exceeds W, take fractions of the item.
Example: (1, 5$), (2, 9$), (3, 12$), (3, 11$) and w=4.
vi/wi :
5
4.5
4.0
3.667
First: (1, 5$), Second: (2, 9$), Third: 1/3 (3, 12$)
Total W: 1,
3,
4.
Can only take part of item
Page 6
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
X
Proof of correctness: (The hard part)
Let X = i1, i2, …ik be the optimal items taken.
 Consider the item j : (vj, wj) with the highest v /w.
 if j is not used in X (the optimal solution), get rid of some items
with total weight wj (possibly fractional items) and add item j.
(since fractional items are allowed, we can do it.)

Total value is increased. Why?
v p 1
vp
v
v
 w1  ( 1 )  w2  ( 2 )  ...  w p 1  (
)  q% w p  ( )
w1
w2
w p 1
wp
 w1  (
wj
)  w2  (
vj
wj
)  ...  w p 1  (
 ( w1  w2  ...  w p 1  q % w p )  (
vj
wj
vj
wj
)  q% w p  (
)  wj  (
vj
wj
vj
wj
.
.
ip wp
)
)  vj
One more item selected by greedy is added to X
Repeat the process, X is changed to contain all items selected by
greedy WITHOUT decreasing the total value taken by the
thief.


i2 w2 wj
.
v1  v2  ...  v p 1  q %v p
vj
i1 w1
.
.
.
ikwk
Page 7
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
The 0-1 knapsack problem cannot
be solved optimally by greedy
 Counter example: (moderate part)
 W=10
2
1.8
 Items found (6pounds, 12dollars), (5pounds, 9 dollar),
1.8
1.
1
(5pounds, 9 dollars), (3pounds, 3 dollars), (3 pounds, 3 dollars)
 If we first take (6, 12) according to greedy algorithm,
then solution is (6, 12), (3, 3) (total value is 12+3=15).
 However, a better solution is (5, 9), (5, 9) with total
value 18.
To show that a statement does not hold, we only have to give
an example.
Page 8
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
A subset of
mutually
compatibles
jobs: {c, f}
Page 9
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 10
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 11
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Sorting the n jobs based on fi needs O(nlog n) time
Page 12
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Example:
 Jobs (s, f): (0, 10), (3, 4), (2, 8), (1, 5),
(4, 5), (4, 8), (5, 6) (7,9).
Sorting based on fi:
(3, 4) (1, 5), (4, 5) (5, 6) (4,8) (2,8)
(7, 9)(0,10).
Selecting jobs:
(3,4) (4, 5) (5,6) (7, 9)
Page 13
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 14
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
How to Prove Optimality
 How can we prove the schedule returned is optimal?


Let A be the schedule returned by this algorithm.
Let OPT be some optimal solution.
 It is impossible to show that A = OPT, instead we need
only to show that |A| = |OPT|.
 Note the distinction: instead of proving directly that a
choice of intervals A is the same as an optimal choice,
we prove that it has the same number of intervals as an
optimal. Therefore, it is optimal.
Page 15
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 16
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 17
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Sort jobs based on finish time: b, c, a, e, d, f, g, h.
Greedy algorithm Selects: b, e, h.
Page 18
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Depth: The maximum No. of jobs required at any time.
Depth:
3
Page 19
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 20
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Depth: The maximum No. of jobs required at any time.
Page 21
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Greedy on start time
Page 22
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Greedy Algorithm:
c
d
b
g
j
f
i
Depth: aThe maximum No. of
e jobs required at any
h time.
Depth:
3
Page 23
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
ddepth
Page 24
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 25
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 26
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
l1=0, l2=1
l2=0, l1=0
0
10
11
l1=9, l2=0
l1=0, l2=1
0
1
11
Page 27
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 28
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 29
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
di<dj
n1n2 … nk
Example: 1, 2, 3, 4, 5, 6, 7, 8, 9
1, 2, 6, 7, 3, 4, 5, 8, 9
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
We check every pair
of consecutive
numbers, if there is
not inversion, then the
whole sequenc has no
inversion.
Page 30
di<dj
Page 31
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 32
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Job
1
2
3
4
5
j1
2020/4/11
2
j2
Example:
di
2
4
6
8
10
ti
2
3
4
4
6
5
j3
9
j4
13
j5
19
CS4335 Design and Analysis of
Algorithms/WANG Lusheng
Page 33
Page 34
2020/4/11
CS4335 Design and Analysis of
Algorithms/WANG Lusheng