CPSC 335
Dr. Marina Gavrilova
Computer Science
University of Calgary
Canada

Dynamic programming
• Optimal structure: optimal solution to problem
• consists of optimal solutions to subproblems
• Overlapping subproblems: few subproblems in total,
• many recurring instances of each
• Solve in “near linear” time through recursion, using a table to show state of problem at any given moment of time
• Reduces computation time from exponential to linear in some cases

Dynamic programming



Longest Common Subsequence
⟨
Given 2 sequences, A = a 1 ,...,an
⟩ and B =
⟨ b 1 ,...,bm
⟩
, find a common subsequence of characters not necessarily adjacent whose length is maximum.
Subsequence must be in order.
3

Dynamic programming

Straight-forward solution finds all permutations of substrings in string A in exponential time 2^n, then checks for every beginning position in string B – another m times, O(m2^n)
4

Dynamic programming





Application: comparison of two DNA strings
Ex: X= {A B C B D A B }, Y= {B D C A B
A}
Longest Common Subsequence:
X = A B C BD A B
Y = B D C A B A
5

Dynamic programming

Straight-forward solution finds all permutations of substrings in string A in exponential time 2^n, then checks for every beginning position in string B – another m times, O(m2^n)
6

Dynamic programming

Similarly to Linear programming,
Dynamic programming defines an algorithmic technique to solve the class of problems through recursion on small subproblems and noticing patterns.
7

Dynamic programming
We determine the length of Longest Common Substring and at the same time record the substring as well.
Define Ai, Bj to be the prefixes of A and B of length i and j respectively
Define L[i,j] to be the length of LCS of Ai and Aj
Then the length of LCS of A and B will be L[n,m]
We start with i = j = 0 (empty substrings of A and B)
LCS of empty string and any other string is empty, so for every i and j:
L[i,0] = 0 c[0, j] =
First case: A[ of LCS i ] =B [ j ]: one more symbol in strings A and B matches, so the length
Ai and A j equals to the length of LCS L [AX i-1, Bi-1]+1
Second case: As symbols don’t match, solution is not improved, and the length of
L(Ai , Bj) is the same as before (i.e. maximum of L(Ai, Bj-1) and L(Ai-1,Bj))
8


LCS algorithm calculates the values of each entry of the array ARRAY[n,m] in O(m*n) since each c[i,j] is calculated in constant time, and there are m*n elements in the array
9

Knapsack problem
10

Knapsack problem
Given some items, pack the knapsack to get the maximum total value.
Each item has some size and some value. Total size of the knapsack is is no more than some constant C.
We must consider sizes of items as well as their value.
Item# Size Value
1 2 5
2 3 7
3 5 6
11

Knapsack problem
Given a knapsack with maximum capacity C, and a set U consisting of n items {U1,U2,Un}
Each item j has some size sj and value vj
Problem: How to pack the knapsack to achieve maximum total value of packed items?
12

Knapsack problem
There are two versions of the problem:
(1) “0-1 knapsack problem” and
(2) “Fractional knapsack problem”
(1) Items are single; you either take an item or not. Solved with dynamic programming
(2) Items can be taken a number of times.
Solved with a greedy algorithm
13

Knapsack problem

Let’s first solve this problem with a straightforward algorithm

Since there are n items, there are 2n possible combinations of items. We go through all combinations and find the one with the most total value and with total size less or equal to C

Running time will be O(2n)
14

Knapsack problem
If items are labeled 1..n
, then a subproblem would be to find an optimal solution for U labeled 1, 2, .. i} i = {items
However, the solution for U 4 might not be part of the solution for is flawed.
U5, s o this definition of a subproblem
Adding another parameter: j, which represents the exact size for each subset of items U, is the solution, woth the subproblem to be computed as V [i,j]
15

Knapsack problem
Now, the best subset of U j that has total size J is one of the two:
1) the best subset of U size s , i-1 that has total or
2) the best subset of Ui -1 size (C-si) plus the item i that has total
16

Knapsack problem

Intuitively, the best subset of Ui that has the total size s either contains item i or not:
First case: si>C . Item i can’t be part of the solution, since if it was, the total size would be more than C, which is unacceptable
Second case: si <= C . Then the item i can be in the solution, and we choose the case with greater value benefit (whether it includes item i or not)
17

Conclusions

Dynamic programming is a useful technique of solving certain kinds of optimization problems


When the solution can be recursively described in terms of partial solutions, we can store these partial solutions and re-use them to save time
Running time (Dynamic Programming) is much better than straight-forward approach
18