Greedy Algorithms
S
Greedy Algorithm
S Greedy Algorithm
- Makes locally optimal choice at each stage.
- For optimization problems.
S If the local optimum is a part of the global optimum, we get
the global optimum.
Greedy Algorithm vs
Dynamic Programming
Dynamic
Programming
Greedy
Algorithm
Knapsack Problem
n items
a thief ’s knapsack
of size W
Knapsack Problem
S 0-1 knapsack problem
- Each item must be either taken or left behind.
S Fractional knapsack problem
- The thief can take fractions of items.
Knapsack Problem
n = 4,
W = 50
$60
$100
$120
($6/unit) ($5/unit) ($4/unit)
$135
($3/unit)
knapsack
Fractional Knapsack Problem
Greedy algorithm:
greatest value per unit
$240
50
$135
($3/unit)
$120
($4/unit)
$100
($5/unit)
$60
($6/unit)
knapsack
0-1 Knapsack Problem
$220
$135
$60
$160
($6/unit)
50
$120
($4/unit)
$100
($5/unit)
$135
($3/unit)
value
per unit
value
optimal
0-1 Knapsack Problem
$220
Difficult to get the
optimal solution with a
greedy strategy.
$135
$160
50
Dynamic
Programming :
value
per unit
value
optimal
Optimal Substructure vs
Subproblem Solution
T1,5
T1,6
Ti,j : the solution of
a subproblem
A subproblem solution?
A local optimum?
T2,5
T2,6
Greedy Algorithm vs
Dynamic Programming
Dynamic Programming
Greedy Algorithm
Computes all subproblems
Find a local optimum
Always finds the optimal
solution
May not be able to find the
optimal solution
Less efficient
More efficient
Optimal Substructure vs
Subproblem Solution
S Subproblem solution
- From all subproblem solutions
Usually
Bottom-up
S Optimal substructure
- With only constant number of parameters
- Without subproblems or future choices
Usually
top-down
Huffman Codes
S A lossless data compression algorithm.
S It uses variable-length code
Variable-Length Code
Six characters : a, b, c, d, e, f
How can we represent them with binary strings?
Fixed-length
a
b
c
d
e
f
000
001
010
011
100
101
Variable-Length Code
Six characters : a, b, c, d, e, f
How can we represent them with binary strings?
Variable-length
a
b
c
d
e
f
0
1
00
01
10
11
What 0010 menas?
0 0 1 0=aaba
0 01 0 = a d a
00 10 = c e
…
Prefix Code
Six characters : a, b, c, d, e, f
How can we represent them with binary strings?
Variable-length
a
b
c
d
e
f
0
101
100
111
1101
1100
No codeword is a prefix of another codeword.
Prefix Code
Six characters : a, b, c, d, e, f
How can we represent them with binary strings?
Variable-length
a
b
c
d
e
f
0
101
100
111
1101
1100
No codeword is a prefix of another codeword.
Variable-Length Code
a
b
c
d
e
f
Frequency
45
13
12
16
9
5
Tota
l
100
Fixed-length
000
001
010
011
100
101
300
Variable-length
0
101
100
111
1101 1100
224
Is it the optimal way?
Variable-Length Code
a
b
c
d
e
f
Frequency
45
13
12
16
9
5
Tota
l
100
Fixed-length
000
001
010
011
100
101
300
Variable-length
0
101
100
111
1101 1100
224
Alternative?
0
1100 1101
10
1110 1111
231
Huffman Tree
Fixed-length code
char code freq
a
000 45
b
c
d
001
010
011
13
12
16
e
f
100
101
9
5
0
0
1
0
0
1
1
0
1
0
1
Huffman Tree
Variable-length code
char code freq
a
0
45
b
c
d
101
100
111
13
12
16
e
f
1101
1100
9
5
0
1
0
0
1
0
1
0
1
1
Huffman’s Algorithm
Observation 1 The longest code : at least 2 characters
0
0
0
Every non-leaf node
has two children.
Huffman’s Algorithm
Observation 1 The longest code : at least 2 characters
Observation 2 The longest 2 codes :
the least frequent two characters
0
1
1
0
1
0
0
1
Huffman’s Algorithm
Observation 1 The longest code : at least 2 characters
Observation 2 The longest 2 codes :
the least frequent two characters
Observation 3 A non-leaf node :
handled like a leaf node
T
T’
Huffman’s Algorithm
Observation 3 A non-leaf node :
handled like a leaf node
T
0
0
1
1
T’
Total length in T
= Total length in T ‘
+ 16 * 1 + (9 + 5) * 2
Huffman’s Algorithm
Observation 1 The longest code : at least 2 characters
Observation 2 The longest 2 codes :
the least frequent two characters
Observation 3 A non-leaf node :
handled like a leaf node
T
T’
Huffman’s Algorithm
Merging two least frequent nodes.
Huffman’s Algorithm
Merging two least frequent nodes.
0
1
Huffman’s Algorithm
Merging two least frequent nodes.
0
1
0
1
Huffman’s Algorithm
Merging two least frequent nodes.
0
1
0
1
0
1
Huffman’s Algorithm
Merging two least frequent nodes.
0
0
1
1
0
1
0
1
Huffman’s Algorithm
Merging two least frequent nodes.
0
1
0
0
1
1
0
1
0
1
char code
a
0
b
101
c
100
d
111
e
f
1101
1100
Greedy Algorithm
S Optimization Algorithms : finds a proper local optimum.
S Not as powerful as Dynamic Programming, but simpler.
S Greedy algorithms
- Knapsack Problem
- Huffman Code
Graphs
S
What is a graph?
G = ( V, E )
V = { 1, 2, 3, 4, 5 }
E = { {1,2}, {1,3}, {2,3},
{2,4}, {2,5}, {3,4} }
Directed and Undirected
Directed graph
Undirected graph
Representations of Graphs
Directed graph
Adjacency-list
1
2
3
4
5
Representations of Graphs
Directed graph
Adjacency-Matrix
1
1
2
3
4
5
2
3
4
5
Representations of Graphs
Adjacency-list
space
Finding all edges
Finding one edge
Adjacency-Matrix
Adjacency List
|V| + |E|
Adjacency Matrix
|V|2
|V| + |E|
num of edges
|V|2
1
Adjacency Matrix of Graphs
Directed graph
Adjacency-Matrix
1
1
2
3
4
5
2
3
4
5
Adjacency Matrix of Graphs
A=
AT =
Weighted Graphs
1.7
0.4
2.0
-0.3
3.1
-0.2
-2.1 3.6