Tutorial Exercise (Greedy Algorithms)
1.
CS3381 Design and Analysis of Algorithms
Helena Wong, 2001
We have studied a recursive strategy (step 2 of dynamic programming), a dynamic programming approach
and a greedy strategy approach that solve the activity-selection problem. Describe how these approaches
work, using the following example:
Activities sorted in increasing order of finish time:
i
1 2 3 4 5 6 7 8
start_timei
0 3 4 6 5 9 10 11
finish_timei
3 4 7 8 11 11 12 14
0
c(i,j) =
Let Si,j be the set of activities that
start after ai finishes and finish before aj starts.
S 0,1
S0,1
S1,2
S2,3 S3,9 S2,4 S4,9 …
Maxi<k<j {c[i,k]+c[k,j]+1} if Si,jØ
time a0 a1 a2 a3 a4 a5 a6 a7 a8 a9
Top-down recursive algorithm:
solve S0,9: check the followings to see which one is optimal:
Step 1: {a1} plus solutions to S0,1,
S1,9.
Step 2: {a2} plus solutions to S0,2,
S2,9.
Step 3: {a3} plus solutions to S0,3,
S3,9.
Step 4: {a4} plus solutions to S0,4,
S4,9.
Step 5: {a5} plus solutions to S0,5,
S5,9.
Step 6: {a6} plus solutions to S0,6,
S6,9.
Step 7: {a7} plus solutions to S0,7,
S7,9.
Step 8: {a8} plus solutions to S0,8,
S8,9.
For step 1.,
perform step 1a: solve S0,1: no activity => 
perform step 1b: solve S1,9:
1b.1
{a2} plus solutions to S1,2,S2,9.
1b.2
{a3} plus solutions to S1,3,S3,9.
1b.3
{a4} plus solutions to S1,4,S4,9.
1b.4
{a5} plus solutions to S1,5,S5,9.
1b.5
{a6} plus solutions to S1,6,S6,9.
1b.6
{a7} plus solutions to S1,7,S7,9.
1b.7
{a8} plus solutions to S1,8,S8,9.
For step 1b.1,
Perform step 1b.1a: solve S1,2: no activity => 
Perform step 1b.1b: solve S2,9:
1b.1b.1 {a3} plus solutions to S2,3, S3,9
1b.1b.2 {a4} plus solutions to S2,4, S4,9
1b.1b.3 {a5} plus solutions to S2,5, S5,9
1b.1b.4 {a6} plus solutions to S2,6, S6,9
1b.1b.5 {a7} plus solutions to S2,7, S7,9
1b.1b.6 {a9} plus solutions to S2,8, S8,9
S1,2 S2,9 S1,3 S3,9 S1,4 S4,9 …
if Si,j=Ø
S1,8
S2,8
S 8,9
S 8,9
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14

S0,9
S3,9 S0,4 S4,9 …
S0,8
S8,9
S2,3 S3,9 S2,4 S4,9 …
S2,8
S8,9
S1,9 S0,2 S2,9 S0,3
S 0,1
S1,2
Tutorial Exercise (Greedy Algorithms)
CS3381 Design and Analysis of Algorithms
Helena Wong, 2001
Dynamic programming algorithm:
Step 0: Solve all Si,j cases that i=j: S0,0 , S1,1 , S2,2 , S3,3 , .. .. .. , S9,9 : 
Step 1: Solve all Si,j cases that i=j-1: S0,1 , S1,2 , S2,3 , S3,4 , .. .. .. , S8,9 : 
Step 2: Solve all Si,j cases that i=j-2:
S0,2 : Possible solutions:
Solution_of_S0,1 U {a1} U Solution_of_S1,2 = {a1}
S1,3 : Possible solutions:
Solution_of_S1,2 U {a2} U Solution_of_S2,3 = {a2}
S2,4 : Possible solutions: 
.. ..
S7,9 : Possible solutions: 
Step 3: Solve all Si,j cases that i=j-3:
S0,3 : Possible solutions:
Solution_of_S0,1 U {a1} U Solution_of_S1,3
Solution_of_S0,2 U {a2} U Solution_of_S2,3
Which one is better?
S1,4 : Possible solutions:
Solution_of_S1,2 U {a2} U Solution_of_S2,4
S2,5 :
…
S6,9 : Possible solutions:
Solution_of_S6,8 U {a8} U Solution_of_S8,9
Step 4: Solve all Si,j cases that i=j-4:
S0,4 : Possible solutions:
Solution_of_S0,1 U {a1} U Solution_of_S1,4
Solution_of_S0,2 U {a2} U Solution_of_S2,4 S1,5 :
Which one is better?
.. ..
S7,9 :
Step 5: Solve all Si,j cases that i=j-5: S0,5 , S1,6 , S2,7 , S3,8 , S4,9
Step 6: Solve all Si,j cases that i=j-6: S0,6 , S1,7 , S2,8 , S3,9
Step 7: Solve all Si,j cases that i=j-7: S0,7 , S1,8 , S2,9
Step 8: Solve all Si,j cases that i=j-8: S0,8 , S1,9
Step 9: Solve all Si,j cases that i=j-9: S0,9
time a0 a1 a2 a3 a4 a5 a6 a7 a8 a9
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14

Let c[i,j] = number of activities in a maximum-size subset of compatible activities in Si,j.
i
C
j
0
1
2
3
4
5
6
7
8
9
0
0
0
1
2
1
0
0
1
1
Greedy algorithm:
solve S0,9:
solve S1,9:
solve S2,9:
solve S3,9:
solve S6,9:
2
0
0
0
0
3
0
0
0
0
4
0
0
0
0
select a1
select a2
select a3
select a6
select a8
5
0
0
0
0
6
7
8
j
0
0
0
1
0
0
0
0
0
0
i
k
9
0
1
2
3
4
5
6
7
8
9
0
x
x
1
1
1
x
x
2
2
2
x
x
x
x
3
x
x
x
x
4
x
x
x
x
5
x
x
x
x
6
7
8
9
x
x
x
8
x
x
x
x
x
x
Tutorial Exercise (Greedy Algorithms)
2.
CS3381 Design and Analysis of Algorithms
Helena Wong, 2001
For the activity-selection problem, suppose that instead of always selecting the first activity to finish, we
select the last activity to start that is compatible with all previously selected activities. Describe how this
approach is a greedy algorithm.
We solve the problem by a sequence of steps. In each step, we make a locally optimal choice (select
the last activity to start) and leave a subproblem (the remaining time slots to be allocated with
remaining activities) to be solved in next step [Optimal Substructure Property]. The locally optimal
choice here is a safe choice that leaves as much opportunity as possible for the remaining activities to
be scheduled [Greedy Choice Property].
3.
a.) State the key ingredients of dynamic programming.
b.) State the key ingredients of greedy algorithms.
c.) Given an optimization problem, how do you determine an appropriate approach to solve it?
Optimal Substructure Property: …
Overlapping Subproblems Property: …
b) Optimal Substructure Property:
Greedy Choice Property: …
c) If we observe the problem has optimal substructure property, it means we can solve it recursively
(divide-and-conquer). However, since it is an optimization problem that exhibits optimal substructure
property, it is a good clue that dynamic programming might apply. We prefer dynamic programming
solutions since recursive solutions need overhead in repeatedly making function calls, that consume system
resources as the problem size increases.
If the problem also has Overlapping Subproblems Property, then dynamic programming is more efficient then
recursive approach since it saves solutions of subproblems and re-uses them.
If the problem has both optimal substructure property and greedy choice property, that means we can solve it
using greedy strategy. In this case we prefer greedy approach rather than dynamic programming since it is
usually more simple and fast. (In each step, we make decision before subproblem is solved)
a)
4.
Give a dynamic-programming solution to the 0-1 knapsack problem.
5.
Give a greedy-algorithm solution to the fractional knapsack problem.
6.
Fibonacci numbers are defined by the following recurrence:
F0 = 1, F1 = 1, Fi = Fi-1 + Fi-2
What is an optimal Huffman code for the following set of frequencies, based on the first 8 Fibonacci
numbers? Describe your observation.
a:1 b:1 c:2 d:3 e:5 f:8 g:13 h:21
1. a:1
2. b:1
3. c:2
4. d:3
5. e:5
6. f:6
7. g:13
8. h:21
1111111
1111110
111110
11110
1110
110
10
0
Assume there are n characters in the set, the Huffman code for i th character is composed in the
way such that
For i=1, it has n-i bits. All bits are ‘1’.
For i>1 1. it has n-i+1 bits
2. the last bit is ‘0’
3. the first (n-i) bits are ‘1’