Algorithm Analysis …
CSE 331
Algorithm Analysis II
Algorithm Analysis
1
Objectives
•
•
•
•
•
•
Start using frameworks for describing and analysing
algorithms.
Understand the technique used to prove the correctness
of an algorithm and its similarity to the mathematical
induction technique
See how to describe algorithms in pseudocode.
Begin using asymptotic notation to express running-time
analysis.
Examine two algorithms for sorting: insertion sort and
merge sort.
Learn the technique of “divide and conquer” in the
context of merge sort
Algorithm Analysis
2
Divide and Conquer
•
•
•
•
•
•
•
The design method of the insertion sort algorithm is called
“Incremental”.
Having sorted A[1..j-1], place A[j] correctly, so that A[1..j] is
sorted.
Another approach is the divide and conquer:
These type of algorithms are recursive in nature.
Divide the problem into a number of subproblems.
Conquer the subproblems by solving them recursively.
– Base case: If the subproblems are small enough, just solve them by brute
force.
Combine the subproblem solutions to give a solution to the
original problem.
Algorithm Analysis
3
Merge sort algorithm
• A sorting algorithm based on divide and conquer.
• Its worst-case running time has a lower order of growth
•
than insertion sort.
Because we are dealing with subproblems, we state
each subproblem as sorting a subarray A[p . . r ].
Initially, p = 1 and r = n, but these values change as we
recurse through subproblems.
Algorithm Analysis
4
Merge-Sort algorithm details
To Sort A[p..r] :
Divide by splitting into two subarrays A[p . . q] and A[q + 1 . . r],
where q is the halfway point of A[p . . r].
Conquer by recursively sorting the two subarrays A[p . . q] and
A[q+1 . . r].
Combine by merging the two sorted subarrays A[p..q] and
A[q+1..r] to produce a single sorted subarray A[p . . r]. To
accomplish this step, define a procedure MERGE(A, p, q, r ).
The recursion bottoms out when the subarray has just 1 element,
so that it’s trivially sorted.
•
•
•
•
Algorithm Analysis
5
Merge algorithm
•
•
•
Input: Array A and indices
p, q, r such that p≤q<r.
Subarray A[p..q] is sorted
and subarray A[q+1..r] is
sorted. None of them is
empty.
Output: The two
subarrays are merged into
a single sorted subarray in
A[p..r].
We implement it so that it
takes Θ 𝑛 time, where n is
the number of elements
being merged.
Algorithm Analysis
6
Merge algorithm Example
Algorithm Analysis
7
Merge algorithm Example
Algorithm Analysis
8
Analysis of recursive algorithms
Correctness of Merge Sort algorithm
Algorithm Analysis
9
Proof by induction
•
•
•
•
•
We must first prove the correctness of the iterative Merge
algorithm.
Prove the correctness of the entire algorithm Using proof by
induction.
Base case: The base case is sorting an array of length one, where
the recursion stops. An array of length one is trivially sorted. The
base case is correct.
Induction hypothesis: assume that the algorithm correctly sort
any array of length less than 𝑛.
Induction step: Sorting an array of length 𝑛 {𝐴 𝑝. . 𝑟 } is done by
𝑛
sorting two arrays of length < 𝑛 {𝐴 𝑝. . 𝑞 , 𝐴[𝑞 + 1. . 𝑟]. By the
2
induction hypothesis, these calls will sort these arrays correctly.
After calling the merge the array 𝐴 𝑝. . 𝑟 of length 𝑛 will bbe sorted
correctlly
Algorithm Analysis
10
Loop invariant of the Merge Algorithm
The for loop performs the rp+1 basic steps by
maintaining the following loop
invariant:
– At the start of each iteration
of the for loop, the subarray
A[p..k-1] contains the k-p
smallest elements of
L[1..n1+1] and R[1..n2+1],
in sorted order.
– Moreover, L[i] and R[j] are
the smallest elements of their
arrays that have not been
copied back into A.
Algorithm Analysis
11
Loop invariant of the Merge Algorithm
We must show that:
this loop invariant holds
prior to the first iteration
of the for loop,
that each iteration of the
loop maintains the
invariant,
and that the invariant
provides a useful
property to show
correctness when the
loop terminates
Algorithm Analysis
12
Correctness of the Merge algorithm
Initialization:
Prior to the first iteration of
the loop, we have k=p, so
that the subarray A[p..k-1] is
empty. This empty subarray
contains the k-p=0 smallest
elements of L and R,
and since i=j=1, both L[i]
and R[j] are the smallest
elements of their arrays that
have not been copied back
into A.
Algorithm Analysis
13
Correctness of the Merge algorithm
Maintenance:
Let us suppose that L[i]
≤R[j]. Then L[i] is the
smallest element not yet
copied back into A. Because
A[p..k-1] contains the k-p
smallest elements, after the
loop copies L[i] into A[k],
the subarray A[p..k] contain
the k-p-1 smallest
elements. Incrementing k
and i re-establishes the loop
invariant for the next
iteration.
Algorithm Analysis
14
Correctness of the Merge algorithm
Termination:
At termination, k=r+1. The
subarray A[p..k-1] which is
A[p..r], contains k-p=r-p+1
smallest elements of
L[1..n1+1] and R[1..n2+1] in
sorted order.
The arrays L and R together
contain n1+n2+2=r-p+3
elements. All but the two
largest have been copied back
into A, and these two largest
elements are the sentinels
(value = ∞).
Algorithm Analysis
15
Analysis of recursive algorithms
Running Time of divide and conquer
algorithms
Algorithm Analysis
16
Analyzing recursive algorithms
•
•
•
•
•
•
•
Let T (n) = running time on a problem of size n.
If the problem size is small enough (say, n ≤ c for some constant c), we
have a base case. The brute-force solution takes constant time: Θ 1 .
Otherwise, suppose that we divide into a subproblems, each 1/b the size
of the original. (In merge sort, a=b=2)
Let the time to divide a size-n problem be D(n).
There are a subproblems to solve, each of size n/b ⇒ each subproblem
takes T(n/b) time to solve ⇒we spend aT(n/b) time solving
subproblems.
Let the time to combine solutions be C(n).
We get the recurrence:
Algorithm Analysis
17
Analyzing merge sort
•
•
For simplicity, assume that n is a power of 2 ⇒ each divide step yields two
subproblems, both of size exactly n/2.
The base case occurs when n = 1.
When n ≥ 2, time for merge sort steps:
Divide: Just compute q as the average of p and r ⇒ D(n)= Θ 1
Conquer: Recursively solve 2 subproblems, each of size n/2⇒2T (n/2).
Combine: MERGE on an n-element subarray takes Θ 𝑛 time
⇒C(n)=Θ 𝑛 .
Since D(n) = Θ 1 and C(n) = Θ 𝑛 , summed together gives Θ 𝑛 .
•
Solving this equation gives
•
•
•
•
•
Algorithm Analysis
18
Solving the recurrent equation
Assume that n is an
exact power of 2
Recursion tree method
Algorithm Analysis
19