Dynamic programming
3.1 The Binomial Coefficient
 Divide-and-conquer approach
C kn 
n!
(n  k )!k!
C kn  C kn11  C kn 1
1
0k n
k  0 or k  n
 Algorithm 3.1
This algorithm computes
2Ckn  1
terms to
n
C
determine k
 Proof by induction
Tkn1  1  Tkn  Tkn1  1  2Ckn  1  2Ckn1  1
 2Ckn  2Ckn1  1
n!
n!
2
2
1
k!(n  k )!
(k  1)!(n  k  1)!
n!(n  k  1)
n! k
2
2
1
k!(n  k  1)!
k!(n  k  1)!
2
(n  1)n!
(n  1)!
1  2
1
k!(n  k  1)!
k!(n  k  1)!
 2C kn 1  1
 Dynamic Programming approach
Figure 3.1
Algorithm 3.2
Time complexity of algorithm 3.2 is
1+2+3+…+k+(k+1)+(k+1)…(k+1)
=
k (k  1)
(2n  k  2)( k  1)
 (n  k  1)( k  1) 
  (nk )
2
2
All-Pairs shortest path
 A sample Graph—Figure 3.2
The representation matrix W—Fig 3.3
D(k)[i,j] be the length of a shortest path
from vi to vj using only vertices in the set
{v1,v2,…,vk} as intermediate nodes
D(k)[i,j]=min{ D(k-1)[i,j],
D(k-1)[i,k]+ D(k-1)[k,j]}
D0 D1 D2……. Dn-1 Dn
 Algorithm 3.3
D0
1
2
3
4
5
1
0
9


3
2
1
0



3
5
5


3
0
D1
1
2
3
4
5
1
0
9


3
2
1
0


4
3

3
0
2

4
1
2
4
0

D2
1
2
3
4
5
1
0
9


3
D4
1
2
3
4
5
1
0
9


3

3
0
2

4
1
2
4
0
4
5
5
14

3
0
2
1
0


4
3
4
3
0
2
7
4
1
2
4
0
4
5
5
14

3
0
D3
1
2
3
4
5
1
0
9


3
2
1
0


4
3
4
3
0
2
7
4
1
2
4
0
4
5
5
14

3
0
2
1
0


4
3
3
3
0
2
6
4
1
2
4
0
4
5
4
5
7
3
0
D5
1
2
3
4
5
1
0
8
10
6
3
2
1
0
11
7
4
3
3
3
0
2
6
4
1
2
4
0
4
5
4
5
7
3
0
 Finding path routes
P[i,j]: highest index of an intermediate
node on the shortest path from vi to vj
Algorithm 3.4 reserves path information
Algorithm 3.5 prints path routes
P0
1
2
3
4
5
1
0
0
0
0
0
2
0
0
0
0
0
3
0
0
0
0
0
4
0
0
0
0
0
5
0
0
0
0
0
P1
1
2
3
4
5
1
0
0
0
0
0
2
0
0
0
0
1
3
0
0
0
0
0
4
0
0
0
0
1
5
0
0
0
0
0
P2
1
2
3
4
5
1
0
0
0
0
0
2
0
0
0
0
1
3
2
0
0
0
2
4
0
0
0
0
1
5
0
0
0
0
0
P3
1
2
3
4
5
1
0
0
0
0
0
2
0
0
0
0
1
3
2
0
0
0
2
4
0
0
0
0
1
5
0
0
0
0
0
P4
1
2
3
4
5
1
0
0
0
0
0
2
0
0
0
0
1
3
4
0
0
0
4
4
0
0
0
0
1
5
4
4
4
0
0
P5
1
2
3
4
5
1
0
5
5
5
0
2
0
0
5
5
1
3
4
0
0
0
4
4
0
0
0
0
1
5
4
4
4
0
0
 Principle of optimality: a problem possess
this property if an optimum solution to an
instance always contains optimum
solutions to all subinstances
The shortest path problem does but the
longest path problem does not
v1
1
1
3
V2
V3
2
4
v4
3.4 Chained matrix multiplication
 AijBjk needs ijk multiplications
A202(B230(C3012D128))
(A202B230) (C3012D128)
A202((B230C3012)D128)
( (A202B230) C3012)D128
(A202(B230C3012)D128)
 Brute-force method
Let tn be the number of different orders
where we can multiply n matrices.
A1A2…An-1An can be multiply by either
A1(A2…An-1An) or (A1A2…An-1)An
tntn-1+tn-1=2tn-1
and t2=1
 tn2n-2
 Let Ai be a matrix of di-1 di
A2A3 needs d1d2 d3 乘法
A1(A2A3) needs d0d1 d3 乘法
(A1A2A3)A4 needs d0d3 d4 乘法
A2(A3A4A5) needs d1d2 d5 乘法
 Dynamic Programming approach
Let M[i,j] be the minimum number of 
needed to multiply Ai through Aj
M [i, j ]  min {M [i, k ]  M [k  1, j ]  d i 1d k d j }
i  k  j 1
M[i, i]  0
Examples 3.5, 3.6 and Algorithm 3.6
 Every case time complexity
n 1
 (n  diagonal )  diagonal
diagonal1
n
n 1

diagonal1
diagonal 
n 1
 diagonal
2
diagonal1
n 2 (n  1) (n  1)( n)( 2n  1) n 3  n n(n  1)( n  1)




2
6
6
6
 ( n 3 )
 Algorithm 3.7 prints the sequence of
matrix multiplication
3.5 Optimal binary search tree
 Every key has a distinct access frequency.
These frequencies are known.
How can we build the most efficient
binary search tree?
Example 3.7
 Let cm be the level of keym in the tree.
Assume that keyk is the root of the tree.
Let cm be the level of keym in the subtree
after removing the root keyk.
Let A[i,j] be the average number of
comparisons needed for the optimal
binary search tree constructed for keyi
through keyj
n
k 1
i 1
i 1
A[1, n]   pi ci  pk ck   pi ci 
k 1
 pk   pi (ci  1) 
i 1
k 1
 pk   pi ci 
i 1
n
pc
i  k 1
n
 p (c  1)
i  k 1
i
i
k 1
n
n
 p c   p   p
i  k 1
i i
i 1
i
i  k 1
i
i i
n
k 1
i 1
i 1
  pi   pi ci 
n
 p c
i  k 1
i i
n
  pi  A[1, k  1]  A[k  1, n]
i 1
Similarly, we can get the following
j
A[i, j ]   pm  A[i, k  1]  A[k  1, j ]
m i
K2 0.3
K3
K1
0.2
0.05
0.1
0.2
K4
0.1
K5
K6
0.05
K7
A[1,1]=0.2, A[5,5]=0.1, A[7,7]=0.05
A[4,5]=0.2+A[5,5]=0.3
=(10.1)+(20.05)=0.3
A[3,7]=0.5+A[4,5]+A[6,7]=0.5+0.3+0.15
=0.95=(10.2)+(20.1)+(20.05)+(30.1)+(30.0)
A[1,7]=1.0+A[1,1]+A[3,7]=1.0+0.2+0.95=2.15
=(1*0.3)+(2*0.2)+(2*0.2)+(3*0.15)+(4*0.15)
 The recursive relation
A[1, n]  min
1 k  j
A[i, j ]  min
n
p
i k  j
i 1
i
 A[1, k  1]  A[k  1, n]
j
p
m i
m
 A[i, k  1]  A[k  1, j ]
A[i, i ]  pi
A[i, i  1]  0 and A[ j  1, j ]  0
 Algorithm 3.9 finds the matrix A
Algorithm 3.10 builds the tree
Example 3.9
 Please compute the following data
Key1
0.2
Key2
0.1
Key3
0.05
Key4
0.15
Key5
Key6
Key7
Key8
0.15
0.2
0.05
0.1
Diagonal=1 Diagonal=2 Diagonal=3
j
i
1
2
3
4
5
6
7
8
9
0
0
1
2
3
0.2
0.4
0.55 0.95
0
0.1
0.2
0
0.05 0.25 0.55
0
4
5
6
7
8
0.5
0.15 0.45 0.85
0
0.15
0.5
0.6
0
0.2
0.30 0.55
0
0.05
0.2
0
0.1
0
Diagonal=1 Diagonal=2 Diagonal=3
j
i
1
2
3
4
5
6
7
8
9
0
0
1
2
3
4
5
6
7
8
1
1
1
2
0
2
2
4
0
3
4
4
0
4
4,5
5
0
5
6
6
0
6
6
6
0
7
8
0
8
0
3.6 The traveling salesman problem
 A tour in a directed graph is a path from a
node to itself that passes through each of
the other nodes exactly once.
An optimum tour is such a path of
minimum length.
Figure 3.16 is the directed graph.
Figure 3.17 is the adjacency matrix.
 Let D[vi,A] is the length of a shortest path
from vi to v1 passing each node in A
exactly once.
optimal length  min (W [1, j ]  D[v j ,V  {v1, v j }])
2 j  n
D[vi , A]  min (W [i, j ]  D[v j , A  {v j }]) if A  
jA
D[vi ,  ]  W [i,1]
A-{vj}
W(i,j)
vi
vj
v1
A
 Examples 3.10, 3.11
Algorithm 3.11
n
 Theorem 3.1:  kC
k 1
kCkn  k
n
 kC
k 1
 n 2 n 1
n!
n!
(n  1)! n


 nCkn11
(n-k )!k! (n-k )!(k  1)! (n-k )!(k  1)!
n
n
k
n
k
 n C
k 1
n 1
k 1
n 1
n C kn1  n2 n1
k 0
 Time complexity analysis
n2
T (n)   (n  1  k )kCkn1
k 1
 (n  1  k )Ckn1  (n  1)Ckn2
n2
T (n)  (n  1) kCkn2  (n  1)( n  2)2 n3  (n 2 2 n )
k 1
 The memory size D[vi,A] and P[vi,A]
n2
 (n  1  k )C
k 1
n 1
k
n2
  (n  1)C
k 1
 (n  1)  2n  2  (n2n )
 Example 3.12
n2
k
n2
 (n  1) C kn 2
k 1