Dynamic Programming
Jeff Chastine
Dynamic Programming
• Usually involves optimization problems
– Optimal solution exists
– Sub-problems arise more than once
– Could use brute force, but…
– Main idea: If you’ve already solved the subproblem, leave yourself a note!
Jeff Chastine
To Get You Thinking…
• Fibonacci
How would you calculate the 100th Fibonacci number?
Fib (100) = Fib(99) + Fib(98)
Fib (99) = Fib (98) + Fib (97)
Jeff Chastine
Assembly Line Scheduling
line 1
a1,1
e1
a1,2
t1,1
a1,3
a1,n-1
t1,2
a1,n
x1
t1,n-1
…
chassis
t2,1
e1
line 2
a2,1
t2,2
a2,2
finished
car
t2,n-1
a2,3
a2,n-1
Jeff Chastine
x2
a2,n
Assembly Line Scheduling
line 1
a1,1
e1
a1,2
t1,1
a1,3
a1,n-1
t1,2
a1,n
x1
t1,n-1
…
chassis
t2,1
e1
line 2
a2,1
t2,2
a2,2
finished
car
t2,n-1
a2,3
a2,n-1
Entry Time – time to get the chassis on the line
Jeff Chastine
x2
a2,n
Assembly Line Scheduling
line 1
a1,1
e1
a1,2
t1,1
a1,3
a1,n-1
t1,2
a1,n
x1
t1,n-1
…
chassis
t2,1
e1
line 2
a2,1
t2,2
a2,2
finished
car
t2,n-1
a2,3
a2,n-1
Station Time – time in each station. Note a1,x != a2,x
Jeff Chastine
x2
a2,n
Assembly Line Scheduling
line 1
a1,1
e1
a1,2
t1,1
a1,3
a1,n-1
t1,2
a1,n
x1
t1,n-1
…
chassis
t2,1
e1
line 2
a2,1
t2,2
a2,2
finished
car
t2,n-1
a2,3
a2,n-1
x2
a2,n
Transfer Time – time to move to the other assembly line
Jeff Chastine
Assembly Line Scheduling
line 1
a1,1
e1
a1,2
t1,1
a1,3
a1,n-1
t1,2
a1,n
x1
t1,n-1
…
chassis
t2,1
e1
line 2
a2,1
t2,2
a2,2
finished
car
t2,n-1
a2,3
a2,n-1
Exit Time – time to get the chassis off the line
Jeff Chastine
x2
a2,n
What’s the fastest way through?
line 1
a1,1
e1
a1,2
t1,1
a1,3
a1,n-1
t1,2
a1,n
x1
t1,n-1
…
chassis
t2,1
e1
line 2
a2,1
t2,2
a2,2
finished
car
t2,n-1
a2,3
a2,n-1
We have a rush order!
Jeff Chastine
x2
a2,n
What about Brute Force?
•
•
•
•
Try all paths
How many? 2n
This is the lower bound!
Example: 100 stations?
– You’ll be dead before it finishes calculating
– Good gift for your grandchildren
Jeff Chastine
Giving the problem structure!
• There’s only 1 path to the first station
• There are 2 paths to the subsequent stations
• For station S1, j:
– S1, j-1 OR
– S2, j-1 with transfer time t2, j-1
Jeff Chastine
What’s the fastest way through?
line 1
S1,1
S1,2
S1,3
S1,4
S1,5
S1,6
7
9
3
4
8
4
2
31
3
2
3
3
4
finished
car
chassis
line 2
1
2
4
2
2
1
2
8
5
6
4
5
7
S2,1
S2,2
S2,3
S2,4
S2,5
S2,6
Jeff Chastine
What’s the fastest way through?
S1,1
S1,2
S1,3
S1,4
S1,5
S1,6
7
9
3
4
8
4
line 1
2
31
3
2
3
3
4
finished
car
chassis
1
2
4
line 2
2
2
1
2
8
5
6
4
5
7
S2,1
S2,2
S2,3
S2,4
S2,5
S2,6
f1[j]
9
?
f2[j]
12
?
f1[2] = min ((9+9), (12+2+9)) = 18 from line 1
f2[2] =Jeff
min
((12+5), (9+2+5)) = 16 from line 1
Chastine
What’s the fastest way through?
S1,1
S1,2
S1,3
S1,4
S1,5
S1,6
7
9
3
4
8
4
line 1
2
31
3
2
3
3
4
finished
car
chassis
1
2
4
line 2
2
2
1
2
8
5
6
4
5
7
S2,1
S2,2
S2,3
S2,4
S2,5
S2,6
f1[j]
9
18
?
f2[j]
12
16
?
l1[j]
*
1
l [j]
*
1
Jeff Chastine 2
What’s the fastest way through?
S1,1
S1,2
S1,3
S1,4
S1,5
S1,6
7
9
3
4
8
4
line 1
2
31
3
2
3
3
4
finished
car
chassis
1
2
4
line 2
2
2
1
2
8
5
6
4
5
7
S2,1
S2,2
S2,3
S2,4
S2,5
S2,6
f1[j]
9
18
?
f2[j]
12
16
?
f1[3] = min ((18+3), (16+1+3)) = 20 from line 2
f2[3] =Jeff
min
((16+6), (18+3+6)) = 22 from line 2
Chastine
What’s the fastest way through?
S1,1
S1,2
S1,3
S1,4
S1,5
S1,6
7
9
3
4
8
4
line 1
2
31
3
2
3
3
4
finished
car
chassis
1
2
4
line 2
2
2
1
2
8
5
6
4
5
7
S2,1
S2,2
S2,3
S2,4
S2,5
S2,6
f1[j]
9
18
20
f2[j]
12
16
22
l1[j]
*
1
2
l [j]
*
1
2
Jeff Chastine 2
What’s the fastest way through?
S1,1
S1,2
S1,3
S1,4
S1,5
S1,6
7
9
3
4
8
4
line 1
2
31
3
2
3
3
4
finished
car
chassis
1
2
4
line 2
2
2
1
2
8
5
6
4
5
7
S2,1
S2,2
S2,3
S2,4
S2,5
S2,6
f1[j]
9
18
20
24
32
35
f2[j]
12
16
22
25
30
37
f*=38
l1[j]
*
1
2
1
1
2
l [j]
Jeff Chastine 2
*
1
2
1
2
2
f*=1
Matrix Multiplication
• [p×q][q×r] →[p×r] for pqr scalar operations
• Most efficient way to multiply?
<A1, A2, A3, A4>
Example:
(A1, (A2, (A3, A4)))
[10×100][100×5][5×50]
(A1, ((A2, A3), A4))
(([10×100][100×5])[5×50]) // 5000
(([10×5])[5×50]) // 2500
((A1, A2), (A3, A4))
((A1, (A2, A3)), A4)
([10×100]([100×5][5×50])) // 25000
([10×100]([100×50])) // 50000
(((A1, A2), A3), A4)
Jeff Chastine
How many Parenthesizations?
(say that 5 times fast)
For any sequence of matrices, we can split at k
M1 M2 M3 M4 … Mk Mk+1 Mk+2 …Mn
1
𝑃 𝑛 =
𝑖𝑓 𝑛 = 1
𝑛−1
𝑃 𝑘 𝑃 𝑛−𝑘
𝑘=1
Jeff Chastine
𝑖𝑓 𝑛 ≥ 2
Optimal Substructure
For any sequence of matrices, we can split at k
M1 M2 M3 M4 … Mk Mk+1 Mk+2 …Mn
M1M2…Mk must be optimal
Mk+1Mk+2…Mn must be optimal as well
// Note the recursion!
Jeff Chastine
Recursive Solution
Let m[i, j] be the min # of multiplications
m[1, n] is cheapest way to compute solution
Then,
m[i, j] = m[i, k] + m[k+1, j] + pi-1 pk pj
Cost to multiply matrices together
Big problem. We don’t know k!
Jeff Chastine
Recursive Solution
Let m[i, j] be the min # of multiplications
m[1, n] is cheapest way to compute solution
Then,
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1 pk pj}
𝑖≤𝑘≤𝑗
Jeff Chastine
6
1
5
2
4
3
3
?
2
1
4
Matrix
M1
M2
M3
M4
M5
M6
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
6
1
5
2
4
3
3
7875
2
1
4
Matrix
M1
M2
M3
M4
M5
M6
5
(30x5)
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
𝑚𝑖𝑛
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
2625 + 30 ∙ 35 ∙ 5 = 7875
Jeff Chastine
15750 + 30 ∙ 15 ∙ 5 = 18000
6
1
5
2
4
3
?
3
2
1
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
1
5
2
4
3
?
3
2
1
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
0 + 2500 + 35 ∙ 15 ∙ 20 = 13000
𝑚𝑖𝑛
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
1
5
2
4
3
?
3
2
1
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
0 + 2500 + 35 ∙ 15 ∙ 20 = 13000
𝑚𝑖𝑛 2625 + 1000Jeff+Chastine
35 ∙ 5 ∙ 20 = 7125
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
1
5
2
4
3
7125
3
2
1
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
0 + 2500 + 35 ∙ 15 ∙ 20 = 13000
𝑚𝑖𝑛 2625 + 1000Jeff+Chastine
35 ∙ 5 ∙ 20 = 7125
4375 + 0 + 35 ∙ 10 ∙ 20 = 11375
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
?
5
4
3
2
1
1
2
11875
10500
(30x20)
(35x25)
3
9375
7125
5375
(30x10)
(35x20)
(15x25)
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
?
5
4
3
2
1
1
2
11875
10500
(30x20)
(35x25)
3
9375
7125
5375
(30x10)
(35x20)
(15x25)
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
?
5
4
3
2
1
1
2
11875
10500
(30x20)
(35x25)
3
9375
7125
5375
(30x10)
(35x20)
(15x25)
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
?
5
4
3
2
1
1
2
11875
10500
(30x20)
(35x25)
3
9375
7125
5375
(30x10)
(35x20)
(15x25)
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
?
5
4
3
2
1
1
2
11875
10500
(30x20)
(35x25)
3
9375
7125
5375
(30x10)
(35x20)
(15x25)
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
?
5
4
3
2
1
1
2
11875
10500
(30x20)
(35x25)
3
9375
7125
5375
(30x10)
(35x20)
(15x25)
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
6
15125
5
4
3
2
1
1
2
(30x25)
11875
10500
(30x20)
(35x25)
3
9375
7125
5375
(30x10)
(35x20)
(15x25)
4
7875
4375
2500
3500
(30x5)
(35x10)
(15x20)
(5x25)
Matrix
M1
M2
M3
M4
M5
M6
5
15,750
2,625
750
1,000
5,000
(30x15)
(35x5)
(15x10)
(5x20)
(10x25)
6
0
0
0
0
0
0
M1
M2
M3
M4
M5
M6
Jeff Chastine
Dimension
30 × 35
35 × 15
15 × 5
5 × 10
10 × 20
20 × 25
Summary
• Elements of Dynamic Programming
– Bottom-up approach
– Optimal substructure
• Fastest way through station j contained fastest way
through station (j-1)
• Determining correct split contained optimal solutions
to subproblems
– Overlapping subproblems
• Doesn’t solve the same subproblems over and over
Jeff Chastine
A Note about Memoization
• Top-down approach
• Leave a note about solutions to sub-problems
• Example: Fibonacci numbers
1, 1, 2, 3, 5, 8, 13…
Calculate the nth Fibonacci number
𝑓𝑖𝑏 𝑛 = 𝑓𝑖𝑏 𝑛 − 1 + 𝑓𝑖𝑏(𝑛 − 2)
Jeff Chastine
Why this could be bad
F(10)
F(8)
+
Jeff Chastine
F(9)
Why this could be bad
F(10)
F(8)
F(6)
F(9)
F(7)
F(7)
Jeff Chastine
F(8)
Why this could be bad
F(8)
F(9)
F(6)
F(4)
F(7)
F(5)
height = ~10
F(10)
F(5)
F(7)
F(6)
F(5)
10 nodes!
Ballpark:Jeff 2Chastine
F(8)
F(6)
F(6)
F(7)
What to notice
F(10)
F(8)
F(9)
F(6)
F(4)
F(7)
F(5)
F(5)
F(7)
F(6)
F(5)
F(8)
F(6)
F(6)
Look at the redundancy!
Jeff Chastine
F(7)
Memoization
• Leave a note
• Once you solve a problem
– Write it in a table
– Instead of calculating/branching again, look it up!
1
2
3
4
5
6
…
1
1
2
3
5
8
…
Jeff Chastine