Dynamic Programming
2012/11/20
1
Dynamic Programming (DP)

Dynamic programming is typically
applied to optimization problems.

Problems that can be solved by dynamic
programming satisfy the principle of
optimality.
P.2
Principle of optimality


Suppose that in solving a problem, we have to
make a sequence of decisions D1, D2, …, Dn-1,
Dn
If this sequence of decisions D1, D2, …, Dn-1, Dn
is optimal, then the last k, 1  k  n, decisions
must be optimal under the condition caused by
the first n-k decisions.
3
Dynamic method v.s.
Greedy method


Comparison: In the greedy
method, any decision is locally
optimal.
These locally optimal solutions
will finally add up to be a globally
optimal solution.
4
The Greedy Method

E.g. Find a shortest path from v0 to v3.


The greedy method can solve this problem.
The shortest path: 1 + 2 + 4 = 7.
5
The Greedy Method





E.g. Find a shortest path from v0 to v3 in the multi-stage
graph.
Greedy method: v0v1,2v2,1v3 = 23
Optimal: v0v1,1v2,2v3 = 7
The greedy method does not work for this problem.
This is because decisions at different stages influence
one another.
6
Multistage graph





A multistage graph G=(V,E) is a directed
graph in which the vertices are partitioned
into k2 disjoint sets Vi, 1i  k
In addition, if <u,v> is an edge in E then
uVi and vVi+i for some i, 1i<k
The set V1 and Vk are such that V1
=Vk=1
The multistage graph problem is to find a
minimum cost path from s in V1 to t in Vk
Each set Vi defines a stage in the graph
7
Greedy Method vs. Multistage
graph

E.g.
4
A
D
1
11
S
2
5
B
18
9
E
16
13
T
2
5
C


2
F
The greedy method cannot be applied to this
case: S A D T 1+4+18 = 23.
The shortest path is:
S C F T 5+2+2 = 9.
8
Dynamic Programming

Dynamic programming approach:
1
S
2
B
5

A
C
d(A, T)
d(B, T)
T
d(C, T)
d(S, T) = min{1+d(A, T), 2+d(B, T),
5+d(C, T)}
9
Dynamic Programming
4
A
D
1
A
4
11


11
D
E
d(D, T)
S
T
d(E, T)
2
5
B
18
9
E
16
13
T
2
5
C
2
F
d(A, T) = min{4+d(D, T), 11+d(E, T)}
= min{4+18, 11+13} = 22.
10
Dynamic Programming
d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.



4
A
D
1
11
S
2
9
5
B
E
16
13
T
B
D
d (D , T )
d (E , T )
5
E
T
2
5
C
9
18
2
F
16
d (F , T )
F
11
4
A
D
1
11
S
Save computation

2
5
B
18
9
E
16
13
T
2
5
C
2
F
For example, we never calculate (as a whole)
the length of the path
SBDT
( namely, d(S,B)+d(B,D)+d(D,T) )


because we have found d(B, E)+d(E,
T)<d(B,D)+d(D,T)
There are some more examples…
Compare with the brute-force method…
12
The advantages of dynamic
programming approach



To avoid exhaustively searching the entire solution
space (to eliminate some impossible solutions and
save computation).
To solve the problem stage by stage
systematically.
To store intermediate solutions in a table (array)
so that they can be retrieved from the table in
later stages of computation.
13
Comment

If a problem can be described by a
multistage graph then it can be solved
by dynamic programming.
14
The longest common subsequence
(LCS or LCSS) problem





A sequence of symbols A = b a c a d
A subsequence of A: deleting 0 or more symbols
(not necessarily consecutive) from A.
E.g., ad, ac, bac, acad, bacad, bcd.
Common subsequences of A = b a c a d and
B = a c c b a d c b : ad, ac, bac, acad.
The longest common subsequence of A and B:
a c a d.
15
DNA Matching
DNA = {A|C|G|T}*
S1=ACCGGTCGAGTGCGGCCGAAGCCGGCCGAA
S2=GTCGTTCGGAATGCCGTTGCTGTAAA
Are S1 and S2 similar DNAs?
The question can be answered by figuring out
the longest common subsequence.
P.16
Networked virtual environments
(NVEs)



virtual worlds full of numerous virtual objects to simulate a
variety of real world scenes
allowing multiple geographically distributed users to
assume avatars to concurrently interact with each other
via network connections.
E.G., MMOGs: World of Warcraft (WoW), Second Life
(SL)
Avatar Path Clustering


Because of similar personalities, interests, or habits, users
may possess similar behavior patterns, which in turn lead
to similar avatar paths within the virtual world.
We would like to group similar avatar paths as a cluster
and find a representative path (RP) for them.
How similar are two paths in
Freebies island of Second Life?
19
LCSS-DC－path transfers sequence
SeqA:C60.C61.C62.C63.C55.C47.C39.C31.C32
20
LCSS-DC － similar path thresholds
SeqA :C60.C61.C62.C63.C55.C47.C39.C31.C32
SeqB :C60.C61.C62.C54.C62.C63.C64
LCSSAB :C60.C61.C62. C63
21
Longest-common-subsequence
problem:


We are given two sequences X =
<x1,x2,...,xm> and Y = <y1,y2,...,yn> and
wish to find a maximum length common
subsequence of X and Y.
We define Xi = < x1,x2,...,xi >
and Yj = <y1,y2,...,yj>.
P.22
Brute Force Solution
m * 2n = O(2n )
or
 n * 2m = O(2m)

23
A recursive solution to
subproblem

Define c [i, j] is the length of the LCS of
Xi and Yj .
if i=0 or j=0

0

if i,j>0 and xi=y
c[i, j] = 
c[i -1, j -1] +1
max{c[i, j -1], c[i -1, j]} if i,j>0 and x  y
j

i
P.24
j
Computing the length of an LCS
LCS_LENGTH(X,Y)
1 m  length[X]
2 n  length[Y]
3 for i  1 to m
4
do c[i, 0]  0
5 for j  1 to n
6
do c[0, j]  0
P.25
7 for i  1 to m
8
for j  1 to n
9
if xi = yj
10
then c[i, j]  c[i-1, j-1]+1
11
b[i, j]  “”
12
else if c[i–1, j]  c[i, j-1]
13
then c[i, j]  c[i-1, j]
14
b[i, j]  “”
15
else c[i, j]  c[i, j-1]
16
b[i, j]  “”
17 return c and b
P.26
Complexity: O(mn) rather than O(2m) or O(2n) of Brute force method
P.27
PRINT_LCS
PRINT_LCS(b, X, i, j )
1 if i = 0 or j = 0
Complexity: O(m+n)
2
then return
3 if b[i, j] = “”
4
then PRINT_LCS(b, X, i-1, j-1)
5
print xi
6 else if b[i, j] = “”
7
then PRINT_LCS(b, X, i-1, j)
8 else PRINT_LCS(b, X, i, j-1)
By calling PRINT_LCS(b, X, length[X], length[Y]) to print LCSP.28
Matrix-chain multiplication


How to compute A1 A2 ... An where Ai
is a matrix for every i.
Example: A1 A2 A3 A4
( A1 ( A2 ( A3 A4 )))
( A1 (( A2 A3 ) A4 ))
(( A1 A2 )( A3 A4 ))
(( A1 ( A2 A3 )) A4 )
((( A1 A2 ) A3 ) A4 )
Chapter 15 P.29
MATRIX MULTIPLY
MATRIX MULTIPLY(A,B)
1 if columns[A]  rows[B]
2
then error “incompatible dimensions”
3
else for i  1 to rows[A]
4
for j  1 to columns[B]
C [i , j ]  0
5
6
for k  1 to columns[A]
C [ i , j ]  C [ i , j ] + A[ i , k ] B [ k , j ]
7
8 return C
Chapter 15 P.30
Complexity:

Let A be a p  q matrix, and B be a
q  r matrix. Then the complexity of
A xB is p  q  r .
Chapter 15 P.31
Example:

is a 10  100 matrix, A2 is a 100  5
matrix, and A3 is a 5  50 matrix. Then
(( A1 A2 ) A3 ) takes 10  100  5 + 10  5  50 = 7500
time. However ( A1 ( A2 A3 )) takes
100  5  50 + 10  100  50 = 75000 time.
A1
Chapter 15 P.32
The matrix-chain multiplication
problem:


Given a chain  A , A ,..., A  of n
matrices, where for i=0,1,…,n, matrix
Ai has dimension pi-1pi, fully
parenthesize the product A1 A2 ... An in
a way that minimizes the number of
scalar multiplications.
1
2
n
A product of matrices is fully parenthesized if it is either a
single matrix, or a product of two fully parenthesized matrix
product, surrounded by parentheses.
Chapter 15 P.33
Counting the number of
parenthesizations:
if n = 1
1

 n -1
P( n ) = 
 P ( k ) p( n - k )

 k =1

if n  2
[Catalan number]
P( n ) = C ( n - 1 )
n
 2 n
4
=

 = ( 3 / 2 )
n +1 n 
n
1
Chapter 15 P.34
Step 1: The structure of an
optimal parenthesization
O ptim al
(( A1 A2 ... A k )( A k + 1 A k + 2 ... A n ))
C om bine
Chapter 15 P.35
Step 2: A recursive solution

Define m[i, j]= minimum number of
scalar multiplications needed to
compute the matrix Ai .. j = Ai Ai +1 .. A j
goal m[1, n]

m[ i, j ] =



 min
0
i k  j
{ m [ i , k ] + m [ k + 1, j ] + p i -1 p k p j }
i= j
i j
Chapter 15 P.36
Step 3: Computing the optimal
costs


Instead of computing the solution to
the recurrence recursively, we compute
the optimal cost by using a tabular,
bottom-up approach.
The procedure uses an auxiliary table
m[1..n, 1..n] for storing the m[i, j]
costs and an auxiliary table s[1..n, 1..n]
that records which index of k achieved
the optimal cost in computing m[i, j].
Chapter 15 P.37
MATRIX_CHAIN_ORDER
MATRIX_CHAIN_ORDER(p)
1 n  length[p] –1
2 for i  1 to n
3
do m[i, i]  0
4 for l  2 to n
5
do for i  1 to n – l + 1
6
do j  i + l – 1
7
m[i, j]  
8
for k  i to j – 1
9
do q  m[i, k] + m[k+1, j]+ pi-1pkpj
10
if q < m[i, j]
11
then m[i, j]  q
12
s[i, j]  k
13 return m and s
3
Complexity: O ( n )
Chapter 15 P.38
Example:
A1
30  35
= p 0  p1
A2
35  15
= p1  p 2
A3
15  5
= p2  p3
A4
5  10
= p3  p4
A5
10  20
= p4  p5
A6
20  25
= p5  p6
Chapter 15 P.39
the m and s table computed by
MATRIX-CHAIN-ORDER for n=6
Chapter 15 P.40
m[2,5]=
min{
m[2,2]+m[3,5]+p1p2p5=0+2500+351520=13000,
m[2,3]+m[4,5]+p1p3p5=2625+1000+35520=7125,
m[2,4]+m[5,5]+p1p4p5=4375+0+351020=11374
}
=7125
Chapter 15 P.41
MATRIX_CHAIN_MULTIPLY
PRINT_OPTIMAL_PARENS(s, i, j)
1 if i=j
2 then print “A”i
3 else print “(“
4
PRINT_OPTIMAL_PARENS(s, i, s[i,j])
5
PRINT_OPTIMAL_PARENS(s, s[i,j]+1, j)
6
print “)”
(( A1 ( A2 A3 ))(( A4 A5 ) A6 ))
 Example:
Chapter 15 P.42
Q&A
43