DYNAMIC PROGRAMMING
Optimization Problems

If a problem has only one correct solution, then optimization is not required

For example, there is only one sorted sequence containing a given set of
numbers.

Optimization problems have many solutions.

We want to compute an optimal solution e. g. with minimal cost and maximal
gain.

There could be many solutions having optimal value.

Dynamic programming is very effective technique.

Development of dynamic programming algorithms can be broken into a
sequence steps as in the next.
Why Dynamic Programming?

Dynamic programming, like divide and conquer method, solves
problems by combining the solutions to sub-problems.

Divide and conquer algorithms:
•
Partition the problem into independent sub-problem
•
Solve the sub-problem recursively
•
Combine their solutions to solve the original problem

In contrast, dynamic programming is applicable when the subproblems are not independent.

Dynamic programming is typically applied to optimization problems.
Dynamic programming
Dynamic programming is a way of improving on inefficient
divide-and-conquer algorithms.
 By “inefficient”, we mean that the same recursive call is
made over and over.
 If same subproblem is solved several times, we can use
table to store result of a subproblem the first time it is
computed and thus never have to recompute it again.
 Dynamic programming is applicable when the subproblems
are dependent, that is, when subproblems share
subsubproblems.
 “Programming” refers to a tabular method

Difference between DP and Divideand-Conquer

Using Divide-and-Conquer to solve these problems is
inefficient because the same common subproblems
have to be solved many times.

DP will solve each of them once and their answers are
stored in a table for future use.
Elements of Dynamic Programming (DP)
DP is used to solve problems with the following characteristics:
 Simple subproblems


Optimal substructure of the problems


We should be able to break the original problem to smaller subproblems that have
the same structure
The optimal solution to the problem contains within optimal solutions to its
subproblems.
Overlapping sub-problems

there exist some places where we solve the same subproblem more than once.
Steps to Designing a Dynamic Programming
Algorithm
1. Characterize optimal substructure
2. Recursively define the value of an optimal
solution
3. Compute the value bottom up
4. (if needed) Construct an optimal solution
Fibonacci Numbers

Fn= Fn-1+ Fn-2
n≥2

F0 =0, F1 =1

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, …

Straightforward recursive procedure is slow!

Let’s draw the recursion tree
Fibonacci Numbers
Fibonacci Numbers



We can calculate Fn in linear time by remembering
solutions to the solved subproblems – dynamic programming
Compute solution in a bottom-up fashion
In this case, only two values need to be remembered at
any time
Assembly-line scheduling
Assembly-line scheduling

Automobiles factory with two assembly lines.

Each line has the same number “n” of stations. Numbered j = 1, 2, ..., n.

We denote the jth station on line i (where i is 1 or 2) by Si,j .

The jth station on line 1 (S1,j) performs the same function as the jth station on line 2
(S2,j ).

The time required at each station varies, even between stations at the same position
on the two different lines, as each assembly line has different technology.

Time required at station Si,j is (ai,j) .

There is also an entry time (ei) for the auto to enter assembly line i and an exit time
(xi) for the completed auto to exit assembly line i.
13
Assembly-line scheduling

After going through station Si,j, can either
q
stay on same line
 next
 no
q
station is Si,j+1
transfer cost , or
transfer to other line
 next
station is S3-i,j+1 or Si+i,j+1
 transfer
 No
cost from Si,j to S3-i,j+1 is ti,j ( j = 1, … , n–1)
ti,n, because the assembly line is complete after Si,n
Assembly-line Scheduling
Station S1,1 Station S1,2 Station S1,3 Station S1,4 … Station S1,n-1 Station S1,n
Assembly
line 1
a1,1
e1
auto
enters
t1,1
a1,3
t1,2
a1,n-
a1,4
1
a1,n
x1
t1,n-1
t1,3
t2,1
a2,1
t2,2
a2,2
t2,n-1
t2,3
a2,3
a2,4
Stations Si,j;
2 assembly lines, i = 1,2;
Completed
auto exits n stations, j = 1,...,n.
…
e2
Assembly
line 2
a1,2
a2,n1
x2
a2,n
Station S2,1 Station S2,2 Station S2,3 Station S2,4 … Station S2,n-1 Station S2,n
ai,j = assembly time at Si,j;
ti,j = transfer time to other
line after station Si,j
ei = entry time from line i;
xi = exit time from line i .
Problem Definition
Which stations should be chosen from line 1 and
which from line 2 in order to minimize the total
time through the factory for one car?
To minimize the total processing time for one auto.
Brute Force


Enumerate all possibilities of selecting stations

Compute how long it takes in each case and choose the best one
Solution:
1
2
3
4
n
1
0
0
1
1
0 if choosing line 2
at step j (= 3)
2n

There are
possible ways to choose stations

Infeasible when n is large!!
1 if choosing line 1
at step j (= n)
Requires to
examine Ω(2n)
possibilities
17
Step 1: Find Optimal Structure

An optimal solution to a problem contains
within it an optimal solution to subproblems.

The fastest way through station Si,j contains
within it the fastest way through station S1,j-1
or S2,j-1 .

Thus can construct an optimal solution to a
problem from the optimal solutions to
subproblems.
Dynamic Programming
Step 1: Optimal Solution Structure

optimal substructure : choosing the best path to Sij.

The structure of the fastest way through the factory (from
the starting point)

The fastest possible way to get through Si,1 (i = 1, 2)


Only one way: from entry starting point to Si,1
Take time is entry time (ei)
Step 1: Optimal Solution Structure
a1,j-
a1,j
1

The fastest possible way to get through Si,j (i = 1, 2)
(j = 2, 3, ..., n). Two choices:
 Stay
in the same line: Si,j-1  Si,j
 Time
is fi[j-i]+ ai,j
 Transfer
 Time
or
a1,j
to other line: S1,j-1  S2,j or S2,j-1  S1,j
is f2[j-1] + t2,j-1 + a1,j
The fastest
way to reach
S1,j-1
or f1[j-1]
+ t1,j-1 + a2,j
t2,n-
The fastest
way to reach
S2,j-1
1
a2,j1
Step 2: Recursive Solution


Define the value of an optimal solution recursively in terms of the optimal
solution to sub-problems.
Sub-problem here

Finding the fastest way through station j on both lines (i=1,2)

Let fi [j] be the fastest possible time to go from starting point through Si,j

The fastest time to go all the way through the factory: f*

x1 and x2 are the exit times from lines 1 and 2, respectively
Step 2: Recursive Solution

The fastest time to go through Si,j

e1 and e2 are the entry times for lines 1 and 2
2
In
f1[1] = e1 + a1,1;
4
f2[1] = e2 + a2,1.
f1[j] = min (f1[j-1] + a1,j, f2[j-1] + t2,j-1 + a1,j) for j ≥ 2;
f2[j] = min (f2[j-1] + a2,j, f1[j-1] + t1,j-1 + a2,j) for j ≥ 2;
3
Out
2
22
Step 2 : Recursive Solution

Recursive solution:
․ Overlapping subproblem: The fastest way through station S is
1,j
either through S1,j-1 and then S1,j , or through S2,j-1 and then transfer to
line 1 and through S1,j.
․ f [j]: fastest time from the starting point through S
i
i,j
․ The fastest time all the way through the factory.
fi[j] (i=1,2; j=1,2,…,n) records optimal values to the subproblems.
f* = min(f1[n] + x1, f2[n] + x2)
Step 2: Recursive Solution



To keep the track of the fastest way, introduce li[j] to record the
line number (1 or 2), whose station j-1 is used in a fastest way
through Si,j.
Introduce l* to be the line whose station n is used in a fastest way
through the factory.
We avoid defining li[1] because no station precedes station 1 on either lines.
Step 2: Recursive Solution
Base Cases
 f1[1]
= e1 + a1,1
 f2[1]
= e2 + a2,1
Step 2: Recursive Solution
Two possible ways of computing f1[j],For j = 2, 3, . . ., n
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2,
j-1
+ a1, j)
Symmetrically, f2[j], For j = 2, 3, . . ., n
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
Objective function = f* = min(f1[n] + x1, f2[n] + x2)
Example: Computation of f1[1] & f2[1]]
2
7
9
2
In
4
2
8
3
4
3
1
1
2
5
6
8
3
3
4
2
4
4
Out
1
5
7
8
4
2
f1[1] = e1 + a1,1 = 2 + 7 = 9
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
3
4
2
4
f2[1] = e2 + a2,1 = 4 + 8 = 12
3
Out
1
5
7
2
Example: Computation of f1[2]
2
7
2
In
4
9
3
1
2
8
3
5
4
1
2
6
4
8
4
3
4
2
1
5
3
Out
7
2
j=2
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
f1[2] = min (f1[1] + a1, 2, f2[1] + t2, 1 + a1, 2)
= min (9 + 9, 12 + 2 + 9) = min (18, 23) = 18
l1[2] =

l1[2] = 1
f1[1] = 9
f2[1] = 12
Example: Computation of f2[2]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
Out
1
5
7
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j=2
f2[2] = min (f2[1] + a2, 2, f1[1] + t1, 1 + a2, 2)
= min (12 + 5, 9 + 2 + 5) = min (17, 16) = 16
2[2]
l l[2]
= 1=
2
3
4
2
4
4
2
f1[1] = 9
f2[1] = 12
Example: Computation of f1[3]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
3
4
2
4
4
Out
1
5
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
j=3
f1[3] = min (f1[2] + a1, 3, f2[2] + t2, 2 + a1, 3)
= min (18 + 3, 16 + 1 + 3)
= min (21, 20) = 20
=2
l1l[3]
1[3] =
7
2
f1[2] = 18
f2[2] = 16
Example: Computation of f2[3]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
Out
1
5
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j=3
f2[3] = min (f2[2] + a2, 3, f1[2] + t1, 2 + a2, 3)
= min (16 + 6, 18 + 3 + 6)
= min (22, 27) = 22
l2l[3]
=
2[3] = 2
3
4
2
4
4
7
2
f1[2] = 18
f2[2] = 16
Example: Computation of f1[4]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2,
j=4
Out
1
5
j-1
+ a1, j)
f1[4] = min (f1[3] + a1, 4, f2[3] + t2, 3 + a1, 4)
= min (20 + 4, 22 + 1 + 4)
= min (24, 27) = 24
=1
l1l[4]
1[4] =
3
4
2
4
4
7
2
f1[3] = 20
f2[3] = 22
Example: Computation of f2[4]
2
In
4
7
9
2
2
3
3
1
1
2
4
8
3
1
j-1
5
7
+ a1, j)
f2[4] = min (f2[3] + a2, 4, f1[3] + t1, 3 + a2, 4)
= min (22 + 4, 20 + 1 + 4)
= min (26, 25) = 25
ll22[4]
[4] ==1
3
4
2
8
5
6
4
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2,
j=4
4
Out
2
f1[3] = 20
f2[3] = 22
Example: Computation of f1[5]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
3
4
2
4
4
Out
1
5
7
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
j=5
f1[5] = min (f1[4] + a1, 5, f2[4] + t2, 4 + a1, 5)
= min (24 + 8, 25 + 2 + 8)
= min (32, 35) = 32
l1l[5]
[5]= 1=
2
f1[4] = 24
f2[4] = 25
Example: Computation of f2[5]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
3
4
2
4
4
Out
1
5
7
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j=5
f2[5] = min (f2[4] + a2, 5, f1[4] + t1, 4 + a2, 5)
= min (25 + 5, 24 + 3 + 5)
= min (30, 32) = 30
ll2[5]
[5]==2
2
f1[4] = 24
f2[4] = 25
Example: Computation of f1[6]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
Out
1
4
5
j-1
+ a1, j)
f1[6] = min (f1[5] + a1, 6, f2[5] + t2, 5 + a1, 6)
= min (32 + 4, 30 + 1 + 4)
= min (36, 35) = 35
ll11[6]
[6] ==2
3
4
2
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2,
j=6
4
7
2
f1[5] = 32
f2[5] = 30
Example: Computation of f2[6]
2
7
2
In
4
9
2
8
5
3
3
1
1
2
6
4
8
3
3
4
2
4
4
Out
1
5
7
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j=6
f2[6] = min (f2[5] + a2, 6, f1[5] + t1, 5 + a2, 6)
= min (30 + 7, 32 + 4 + 7)
= min (37, 43) = 37
[6] =
l2l2[6]
=2
2
f1[5] = 32
f2[5] = 30
Example: Computation of f*
f* = min (f1[6] + x1, f2[6] + x2)
= min (35 + 3, 37 + 2)
= min (38, 39) = 38
L* = 1
f1[6] = 35
f2[6] = 37
Entire Solution Set: Assembly-Line Scheduling
2
7
9
2
In
4
j 1
Fi[j]
2
8
2
3
3
1
1
2
5
3
4
3
5
3
Out
1
4
6
4
4
2
6
4
8
5
2
7
j 2
3
4
5
6
Li[j]
1
9
18 20 24 32 35
1
1
2
1
1
2
2
12 16 22 25 30 37
2
1
2
1
2
2
f* = 38
l* = 1
Fastest Way: Assembly-Line Scheduling
Let li[j]
be the line number, 1 or
2, whose station j-1 is
used in the solution.
eg. l1[5] = 2 means that the fastest
way to reach station 5 of
line 1 should pass
through station 4 of line
2.
Let l*
be the line whose station
n is used in the solution.
eg. l*=2 means that the fastest way
involves station n of line
2.
2
7
2
In
4
l* = 1 =>
l1[6] = 2 =>
l2[5] = 2 =>
l2[4] = 1 =>
l1[3] = 2 =>
l2[2] = 1 =>
9
2
8
3
3
1
1
2
5
Station S1, 6
Station S2,5
Station S2,4
Station S1,3
Station S2,2
Station S1,1
4
8
3
3
4
2
6
4
Out
1
4
5
2
7
j 2
3
4
5
6
1
1
2
1
1
2
2
1
2
1
2
2
Li[j]
Step 3: Optimal Solution Value
What we are doing, is:
•
Starting at the bottom level, continuously filling in
the tables (f1[], f2[] and l1[], l2[]), and
•
Looking up the tables to compute new results for
next higher level.
Most of the sub-problems are visited more than
once.
Any clever method to handle them?
•
•
Exercise: what is the complexity of this algorithm?
Step 3: Optimal Solution Value
Compute initial values of f1 and f2
Compute the
values of
f1[j] and l1[j]
Compute the
values of
f1[j] and l1[j]
O(N)
Compute the values of
the fastest time through the
entire factory
Step 3: Optimal Solution Value
Step 4: Construct an optimal solution
Step 4.
Construct an optimal solution from computed information.
PRINT-STATIONS()
1 i  l*
2 print “line ” i “, station ” n
3 for j  n down to 2
4
do i  li[j]
5
print “line ” i “, station ” j-1
line 1,
line 2,
line 2,
line 1,
line 2,
line 1,
station
station
station
station
station
station
6
5
4
3
2
1
Assembly-line Scheduling
What we have done is indeed
Dynamic Programming
Now it is time to test your memory:
Dynamic Programming
Step 1 Characterize the structure of an optimal solution
Eg. Study the structure of the fastest way through the factory.
Step 2 Recursively define the value of an optimal solution
Step 3 Compute the value of an optimal solution in a bottom-up fashion
Step 4 Construct an optimal solution from computed information.