COSC 3100
Decrease and Conquer
Instructor: Tanvir
Decrease and Conquer
• Exploit the relationship between a
solution to a given instance of a
problem and a solution to its smaller
instance
• Top-down: recursive
• Bottom-up: iterative
• 3 major types:
– Decrease by a constant
– Decrease by a constant factor
– Variable size decrease
Decrease and Conquer
(contd.)
• Decrease by a constant
– Compute an where a ≠ 0 and n is a
nonnegative
Can you improve
an computation ?
– an = an-1 × a
– Top down: recursive
• f(n) =
f(n-1) × a if n > 0
1
if n = 0
– Bottom up: iterative
• Multiply 1 by a, n times
We shall see more
interesting ones in
this chapter!
Decrease and Conquer
(contd.)
• Decrease by a constant factor
(usually 2)
𝟐
𝒏
𝟐
𝒂
•
an
=
𝒂
𝒏−𝟏 𝟐 𝟐
If n is even and positive
∙𝒂
𝟏
If n is odd
If n = 0
529 =?
Decrease and Conquer (contd.),
Decr. By const. factor
ALGORITHM Exponentiate(a, n)
if n = 0
return 1
tmp <- Exponentiate(a, n >> 1)
if (n & 1) = 0 // n is even
return tmp*tmp
else
return tmp*tmp*a
𝒏 𝟐 𝟐
If n even
𝒂
𝒂
𝒏−𝟏 𝟐 𝟐
𝟏
What’s the time complexity ?
∙𝒂
If n is odd
If n = 0
Θ(lgn)
Decrease and Conquer (contd.)
• Variable-size decrease
We already have seen one example.
Which one ?
gcd(m, n) = gcd(n, m mod n)
gcd( 31415, 14142 ) = gcd( 14142, 3131 )
= gcd( 3131, 1618 )
= gcd( 1618, 1513 )
Decr. By 11011
Decr. By 1513
Decr. By 105
= gcd( 1513, 105 )
Decr. By 1408
= gcd( 105, 43 )
Decr. By 62
= gcd( 43, 19 )
= gcd( 19, 5 )
= gcd( 5, 4 )
= gcd( 4, 1 )
= gcd( 1, 0 )
Decr. By 24
Decr. By 14
Decr. By 1
Decr. By 3
Decr. By 1
Decrease and Conquer (contd.)
• Apply decrease by 1 technique to
sorting an array A[0..n-1]
• We assume that the smaller instance
A[0..n-2] has already been sorted.
• How can we solve for A[0..n-1] now?
(Straight)
INSERTION
SORT
Find the appropriate position for A[n-1]
among the sorted elements and insert it
there.
Scan the sorted subarray A[0..n-2] from
right to left until an element smaller than
or equal to A[n-1] is found, then insert A[n-1]
right after that element.
Insertion Sort
ALGORITHM InsertionSort(A[0..n-1])
for i <- 1 to n-1 do
89 | 45 68 90 29 34 17
v <- A[i]
j <- i-1
45 89 | 68 90 29 34 17
while j ≥ 0 and A[j] > v do
45 68 89 | 90 29 34 17
A[j+1] <- A[j]
45 68 89 90 | 29 34 17
j <- j-1
A[j+1] <- v
29 45 68 89 90 | 34 17
Input size: n
Basic op: A[j] > v
Why not j ≥ 0 ?
C(n) depends on input type ?
29 34 45 68 89 90 | 17
17 29 34 45 68 89 90
Insertion Sort (contd.)
ALGORITHM InsertionSort(A[0..n-1])
for i <- 1 to n-1 do
What is the worst case scenario ?
v <- A[i]
A[j] > v executes highest # of times
j <- i-1
When does that happen ?
while j ≥ 0 and A[j] > v do
A[j+1] <- A[j]
A[j] > A[i] for j = i-1, i-2, …, 0
Can you improve it ?
j <- j-1
Worst case input:
A[j+1] <- v
An array of strictly decreasing values
Cworst(n) =
For almost sorted files,
insertion sort’s performance
is excellent!
Cbest(n) =
𝒏−𝟏
𝒊=𝟏
𝒊−𝟏
𝒋=𝟎 𝟏
=
𝒏−𝟏
𝒊=𝟏 𝒊
=
𝒏−𝟏 𝒏
𝟐
є Θ(n2)
What is the best case ?
A[i-1] ≤ A[i] for i = 1, 2, …, n-1
𝒏−𝟏
𝒊=𝟏 𝟏
In place? Stable ?
= n-1 є Θ(n)
Cavg(n) ≈
𝒏𝟐
𝟒
є Θ(n2)
Topological Sorting
• Ordering of the vertices of a directed
graph such that for every edge uv, u comes
before v in the ordering
• First studied in 1960s in the context of
PERT (Project Evaluation and Review
Technique) for scheduling in project
management.
– Jobs are vertices, there is an edge from x to y
if job x must be completed before job y can be
started
– Then topological sorting gives an order in which
to perform the jobs
Directed Graph or Digraph
• Directions for all edges
a
b
c
d
e
a
b
c
b
a
c
c
e
a
b
c
d
a
0
1
b
1
0
1
0
0
c
0
0
0
0
0
d
0
0
1
0
1
e
0
0
0
0
1
0
e
0
0
Not symmetric
c
d
e
One node for one edge
Digraph (contd.)
a
b
a
b
c
b
a
c
c
e
c
c
d
e
d
a
d
Tree edge
e
Directed cycle: a, b, a
b
e
Back edge
Forward edge
Cross edge
c
DFS forest
If no directed cycles, the digraph
is called “directed acyclic graph” or “DAG”
Topological Sorting Scenario
Consider a robot professor Dr. Botstead. He gets dressed
in the morning. The professor must put on certain garments
before others (e.g., socks before shoes). Other items may be
put on in any order (e.g., socks and pants).
socks
undershorts
watch
Wont work
if directed
cycle exists!
shoes
pants
shirt
belt
tie
jacket
Let us help him to decide
on a correct order using
topological sorting!
Method 1: DFS
u
so
sho
p
shi
b
t
j
Reverse the pop order!
w9
so8 shi7
w9, 9
w
so8, 8
t7, 6
u -> p -> sho
p -> sho -> b
b -> j
shi -> b -> t
t -> j
so -> sho
w -> null
sho -> null
j -> null
t6
u5
p4
shi6, 7
j5,2
b4, 3
sho3, 1
p2, 4
u1,5
b3
j2
sho1
Method 2: Decrease (by 1)
and Conquer
u
so
sho
p
w
shi
b
t
j
u, p, shi, b, t, j, so, sho, w
Topological Sorting Scenario 2
•
•
•
•
•
Set of 5 courses: { C1, C2, C3, C4, C5 }
C1 and C2 have no prerequisites
C3 requires C1 and C2
C4 requires C3
C1, C2, C3, C4, C5
C5 requires C3 and C4
C1
C4
C3
C2
C5
!
Topological Sorting: Usage
• A large project – e.g., in construction,
research, or software development –
that involves a multitude of
interrelated tasks with known
prerequisites
– Schedule to minimize the total
completion time
• Instruction scheduling in program
compilation, resolving symbol
dependencies in linkers, etc.
Decrease and Conquer:
Generating Permutations
• Have to generate all n! permutations
of the set of integers from 1 to n
– They can be indices of n-element set {a1,
a2, …, an}
• Assume that we already have (n-1)!
permutations of integers 1, 2, …, n-1
• Insert n in each of the n possible
positions among elements of every
permutation of n-1 elements
Generating Permutations (contd.)
• We shall generate all permutations of
{1, 2, 3, 4}, 4! = 24 permutations in
Insert 4 in 4 possible
total
positions of each previous
Start
1
permutation
Insert 2 in
2 possible
positions
21,
12
4321, 3421, 3241, 3214,
4231, 2431, 2341, 2314,
4213, 2413, 2143, 2134,
Insert 3 in
3 possible
positions of
each previous
permutation
321, 231, 213,
312, 132, 123
4312, 3412, 3142, 3124,
4132, 1432, 1342, 1324,
4123, 1423, 1243, 1234
Generating Permutations (contd.)
2
If we can produce a permutation
from the previous one just by exchanging
5 8
7 3
two adjacent elements, cost calculation takes
c
d
constant time!
1
This is possible and called
a  b  c  d  a 2+8+1+7 = 18
minimal-change requirement
a
b
a  b  d  c  a 2+3+1+5 = 11
a  c  b  d  a 5+8+3+7 = 23
a  c  d  b  a 5+1+3+2 = 11
Every time we process one of the
old permutations to get a bunch of new
ones, we reverse the order of insertion
a  d  b  c  a 7+3+8+5 = 23
a  d  c  b  a 7+1+8+2 = 18
1
Right to Left
12,21
R to L
L to R
123, 132, 312,
321, 231, 213
R2L:
L2R:
R2L:
L2R:
R2L:
L2R:
1234,
4132,
3124,
4321,
2314,
4213,
1243,
1432,
3142,
3421,
2341,
2413,
1423,
1342,
3412,
3241,
2431,
2143,
4123,
1324,
4312,
3214,
4231,
2134
Generating Permutations (contd.)
• It is possible to get the same
ordering of permutations of n
elements without explicitly
generating permutations for smaller
values of n
• Associate a direction with each
element k in a permutation
• 3241
Element k is mobile if its arrow points to
a smaller number adjacent to it.
mobile
Generating Permutations (contd.)
ALGORITHM JohnsonTrotter(n)
//Output: A list of all permutations of {1, …, n}
initialize the first permutation with 1 2 … 𝑛
while the last permutation has a mobile element do
find its largest mobile element k
swap k with the adjacent element k’s arrow points to
reverse direction of all the elements that are larger than k
add the new permutation to the list
12𝟑
1𝟑2
31𝟐
𝟑21
2𝟑1
213
This version
is by the
Shimon
Even
𝟑 2 1 should
have been
Lexicographic
Can
beone!
implemented in Θ(n!) order
time
last
Generating Permutations: in
Lexicographic Order
ALGORITHM LexicographicPermute(n)
initialize the first permutation with 12…n
while last permutation has two consecutive elements in increasing
order do
let i be its largest index such that 𝑎𝑖 < 𝑎𝑖+1
find the largest index j such that 𝑎𝑖 < 𝑎𝑗
swap ai with aj
reverse the order from ai+1 to an inclusive
add the new permutation to the list
1
2
i
3
i+1=j
3
1
i
2 1
3
2
i+1 j
3
2
i
1
2
3
1
i+1 j
2
i i+1=j
3
i
2
i+1=j
1
3
i i+1=j
1
2
i
3 1
i+1=j
Narayana Pandita in 14th century India
Generating Subsets
• Generate all 2n subsets of the set
A = {a1, a2, …, an}
Can divide the subsets into two groups:
Those that do not contain an
Those that do contain an
It’s the subsets of {a1, a2, …, an-1}
Take each subset of {a1, a2, …, an-1}
and add an to it
Generating Subsets (contd.)
• Let us generate 23 subsets of {a1, a2,
a3 }
n=0
Ø
n=1
Ø {a1}
n=2
Ø {a1} {a2} {a1, a2}
n=3
Ø {a1} {a2} {a1, a2} {a3} {a1, a3} {a2, a3} {a1, a2, a3}
Generating Subsets (contd.)
• There is a one-to-one
correspondence between all 2n
subsets of an n element set A = {a1,
a2, …, an} and all 2n bit strings b1b2…bn
of length n.
Bit strings
Subsets
000 001 010
Ø
011
100
101
110
111
{a3} {a2} {a2, a3} {a1} {a1, a3} {a1, a2} {a1, a2, a3}
How to generate subsets in “squashed order” from bitstrings?
Ø {a1} {a2} {a1, a2} {a3} {a1, a3} {a2, a3} {a1, a2, a3}
Generating Subsets (contd.)
Is there a minimal-change algorithm for generating
bit strings so that every consecutive two differ by
only a single bit ?
In terms of subsets, every consecutive two differ
by either an addition or deletion, but not both of
a single element.
Yes, there is!
For n = 3,
000
001
011
010
110
111
101
100
It is called “Binary reflected Gray code”
Generating Subsets (contd.)
ALGORITHM BRGC(n)
//Generates recursively the binary reflected Gray code of order n
//Output: A list of all bit strings of length n composing the Gray code
if n = 1
make list L containing bit strings 0 and 1 in this order
else
generate list L1 of bit strings of size n-1 by calling BRGC(n-1)
copy list L1 to list L2 in reverse order
add 0 in front of each bit string in list L1
add 1 in front of each bit string in list L2
append L2 to L1 to get list L
L : 00
01
return L
L1 : 0
1
L2 : 1
0
L1 : 00
01
L2 : 11
10
11
10
Binary Reflected Gray Codes (contd.)
10 11
0
1
110
010
00
01
011
100
000
001
111
101
Decrease-by-a-Constant-Factor
Algorithms
• Binary Search
– Highly efficient way to search for a key
K in a sorted array A[0..n-1]
Compare K with A’s middle element A[m]
If they match, stop.
Else if K < A[m] do the same for the first half of A
Else if K > A[m] do the same for the second half of A
Binary Search (contd.)
K (key)
A[0] … A[m-1]
A[m]
A[m+1] … A[n-1]
Search here if
K > A[m]
Search here if
K < A[m]
Let us apply binary search for K = 70 on the following array:
3, 14, 27, 31, 39, 42, 55, 70, 74, 81, 85, 93, 98
0
1
2
3
4
5
6
7
8
9
10
11
12
Binary Search (contd.)
ALGORITHM BinarySearch(A[0..n-1], K)
//Input: A[0..n-1] sorted in ascending order and a search key K
//Output: An index of A’s element equal to K or -1 if no such element
l <- 0
r <- n-1
Input size: n
while l ≤ r do
Basic operation: “comparison”
m <- 𝑙 + 𝑟 2
Does C(n) depend on input type?
if K = A[m]
return m
YES!
else if K < A[m]
Let us look at the worst-case…
r <- m-1
Worst-case input: K is absent or some
else
K is at some special position…
l <- m+1
return -1
To simplify, assume n = 2k
Cworst(n) = Cworst( 𝑛 2 ) + 1
Then Cworst(n) є Θ(lgn)
Cworst(1) = 1
For any integer n > 0, Cworst(n) = 𝐥𝐨𝐠 𝟐 𝒏 + 𝟏= 𝐥𝐨𝐠 𝟐 𝒏 + 𝟏
Variable-Size-Decrease
• Problem size decreases at each
iteration in variable amount
• Euclid’s algorithm for computing the
greatest common divisor of two
integers is one example
Computing Median and the
Selection Problem
• Find the k-th smallest element in a
list of n numbers
• For k = 1 or k = n, we could just scan
the list
• More interesting case is for k = 𝑛 2
– Find an element that is not larger than
one half of the list’s elements and not
smaller than the other half; this middle
element is called the “median”
– Important problem in statistics
Median Selection Problem
Any idea ?
Sort the list and select the k-th element
Time efficiency is ? O(n lgn)
It is an overkill, the problem needs only the
k-th smallest, does not ask to order the entire list!
We shall take advantage of the idea of “partitioning”
a given list around some value p of, say the list’s first
element. This element is called the “pivot”.
p
all are < p
p
all are ≥ p
Lomuto partitioning & Hoare’s partitioning
p
i
i
61
93
56
Lomuto Partitioning
90
11
s
p
61
56
A[i] not less than p
p
93
s
s
56
61
i
56
s
93
90
11
A[i] not less than p
90
11
p
61
A[i] less than p
p
i
s
i
61
93
i
i
56
s
i
90
11
p
93
s
90
11
A[i] less than p
i
61
56
11
s
90
93
11
56
61
90
93
Lomuto Partitioning (contd.)
s
p
i
<p
≥p
?
?
A[i] < p
p
s
<p
≥p
swap
<p
s
p
≥p
Lomuto Partitioning (contd.)
ALGORITHM LomutoPartition(A[l..r])
//Partition subarray by Lomuto’s algo using first element as
pivot
//Input: A subarray A[l..r] of array A[0..n-1], defined by its
//left and right indices l and r (l ≤ r)
//Output: Partition of A[l..r] and the new position of the pivot
p <- A[l]
s <- l
for i <- l+1 to r do
Time complexity is Θ(r-l)
if A[i] < p
s <- s+1
swap(A[s], A[i])
Length of subarray A[l..r]
swap(A[l], A[s])
return s
Median Selection Problem
(contd.)
• We want to use the Lomuto partitioning to
efficiently find out the k-th smallest element of
A[0..n-1]
• Let s be the partition’s split position
• If s = k-1, pivot p is the k-th smallest
• If s > k-1, the k-th smallest (of entire array) is
the k-th smallest of the left part of the
partitioned array
• If s < k-1, the k-th smallest (of entire array) is
the [(k-1)-(s+1)+1]-th smallest of the right part of
the partitioned array
Median Selection Problem
(contd.)
ALGORITHM QuickSelect(A[l..r], k)
//Input: Subarray A[l..r] of array A[0..n-1] of orderable
elements and integer k (1 ≤ k ≤ r-l+1)
//Output: The value of the k-th smallest element in A[l..r]
s <- LomutoPartiotion(A[l..r])
if s = k-1
return A[s]
else if s > l+k-1
QuickSelect(A[l..s-1], k)
else
QuickSelect(A[s+1..r], k-1-s)
Median Selection: QuickSelect
k= 9 2 =5
s
i
4
1
10
s
i
1
10
4
8
7
12
9
2
15
8
7
12
9
2
15
i
s
4
1
10
8
7
12
9
1
2
8
7
12
9
10
s
4
1
2
15
i
s
4
2
15
i
8
7
12
9
10
15
8
7
12
9
10
15
s
2
1
4
s = 2 < k-1 = 4, so we proceed with the right part…
Median Selection: QuickSelect
(contd.)
k= 9 2 =5
2
1
4
8
7
s
i
8
8
12
9
10
15
7
12
9
10
15
s
i
7
12
9
10
15
s
8
7
i
12
9
10
15
12
9
10
15
s
7
8
Now s (=4) = k-1 (=5-1=4), we have found the median!
Median Selection (contd.)
ALGORITHM QuickSelect(A[l..r], k)
//Input: Subarray A[l..r] of array A[0..n-1] of orderable elements and integer
//k (1 ≤ k ≤ r-l+1)
//Output: The value of the k-th smallest element in A[l..r]
s <- LomutoPartiotion(A[l..r])
What is the time efficiency ?
if s = k-1
Does it depend on input-type?
return A[s]
Partitioning A[0..n-1] takes n-1 comparisons
else if s > l+k-1
QuickSelect(A[l..s-1], k) If now s = k-1, this is the best-case,
Cbest(n) = n є Θ(n)
else
QuickSelect(A[s+1..r], k-1-s)
What’s the worst-case ?
Why is QuickSelect
in variable-size-decrease ?
Say k = n and we have a strictly
increasing array!
Then, Cworst(n) = (n-1)+(n-2)+……+1 =
𝒏−𝟏 𝒏
𝟐
= 𝜣 𝒏𝟐
Cworst(n) is worse than sorting based solution!
Fortunately, careful mathematical analysis
has shown that the average-case is linear