Algorithm Analysis
Algorithm Analysis
• Pseudo code
• Analyzing algorithms and Problems
– Time Complexity
– Rate of growth of functions
– Asymptotic notation
Pseudo code
• Describes
main
ideas
behind
the
implementation of data structure or algorithm
• No precise definition of pseudo code
• Mixture of natural language and high-level
programming constructs (C,C++, and Java)
Algorithm arrayMax
Algorithm arrayMax(A,n)
Input: An array A storing n ≥ 1 integers.
Output: The maximum element in A.
currentMax ← A[0]
for i ← 1 to n - 1 do
if currentMax < A [i] then
currentMax ← A[i]
return currentMax
Pseudo code
• Expression
–Assignment Operator ←
–Equality in Boolean expression ==
• Method Declaration
Algorithm name (param I, parmn2, . .. )
input:
output:
Pseudo code
• Decision structures
if condition then
true-actions
else
false-actions
• While-loops:
while condition do
actions
Pseudo code
• Repeat-loops
repeat
actions
until condition
• for-loops:
for variable-increment-definition do
actions
Pseudo code
• Array Indexing: A[i]
• Method Calls :
object.method(args)
• Method Returns:
return value
//object is optional if it
is understood
Primitive Operations:
Random Access Machine Model
• Assigning a value to a variable
• Calling a method
• Performing an arithmetic operation (for example, adding
two numbers)
• Comparing two numbers
• Indexing into an array
• Following an object reference
• Returning from a method.
Counting Primitive Operations
Algorithm arrayMax(A,n)
Input: An array A storing n ≥ 1 integers.
Output: The maximum element in A.
currentMax ← A[0]
for i ← 1 to n - 1 do
2
1+ n
if currentMax < A [i] then
currentMax ← A[i]
return currentMax 1
2+1+n+6(n-1)+1=7n-2
(worst case)
2+1+n+4(n-1)+1=5n
(best case)
(n-1)*2
(n-1)*2 in worse case
(n-1)*2 for increment i
Algorithm:
Worst-case and average-case analysis
• Worst Case
– Gives upper limit for running time
– Easy to analyze
• Average case
– Cost of the algorithm on average
– Difficult to calculate
Growth Rates
• rate of growth or order of growth
• Growth rate functions:
– Linear
– Quadratic
– Exponential
Running time : Growth Rate
Source :
http://homes.soic.indiana.edu/yye/lab/teaching/spring2014-C343/bigO_basics.php
Asymptotic Notation
• Big- Oh Notation (O)
• Big-Omega (Ω)
• Big-Theta (θ )
• Little-oh (o)
• Little-omega (ω)
Asymptotic
Notation
f(n)=O(g(n))
f(n)= Ω(g(n))
f(n)= θ(g(n))
Big Oh Notation: Rules
Rule 1:
If d(n) is O(f(n)), then ad(n) is O(f(n)) , for any constant
a>0
Rule 2:
If d(n) is O(f(n)) and e(n) is O(g(n)), then
d(n)+e(n) is O(f(n)+g(n))
Rule 3:
If d(n) is O(f(n)) and e(n) is O(g(n)), then d(n)e(n) is
O(f(n)g(n))
Big Oh Notation: Rules
Rule 4:
If d(n) is O(f(n)) and f(n) is O(g(n)) , then d(n) is O(g(n))
Rule 5:
If f(n) is polynomial of degree d (that is,
f(n)=a0+a1n+….+adnd), then f(n) is O(nd)
Rule 6:
nx is O(an) for any fixed x>0 and a>1
Rule 7:
log nx is O(log n) for any fixed x>0
Rule 8:
logx n is O(ny) for any fixed constants x>0 and y>0
BigOh Rules: Example
• 2n3+4n2logn is O(n3)
– log(n) is O(n) (rule 8)
– 4n2logn is O(4n3) (rule 3)
– 2n3+4n2logn is O(2n3+4n3) (rule 2)
– 2n3+4n3 is O(n3) (rule 5 or rule 1)
– then 2n3+4n2logn is O(n3) (rule 4)
BigOh Rules: Example
• 10n2+4n+2 is O(n2)
– 10n2 is O(n2)
– 4n is O(n)
– 2 is O(1)
– 10n2+4n is O(n2+n)
– 10n2+4n+2 is O(n2+n+1)
– n2+n+1 is O(n2)
– then, 10n2+4n+2 is O(n2)
(rule 2)
(rule 2)
(rule 5)
(rule 4)
BigOh and LittleOh
• f(n) is O(g(n)) if there is a real constant
c > 0 and an integer constant n0 ≥ 1 such
that
f(n) ≤ cg(n) for every integer n ≥ n0
• f(n) is o(g(n)) for any constant c > 0,
there is integer constant n0 > 0 such that
f(n) ≤ cg(n) for every integer n ≥ n0
Asymptotically
not a tight
bound
BigOmega and
LittleOmega
• f(n) is Ω(g(n)) if there is a real constant
c > 0 and an integer constant n0 ≥ 1 such
that
f(n) ≥ cg(n) for every integer n ≥ n0
• f(n) is ω(g(n)) for any constant c > 0,
there is integer constant n0 > 0 such that
f(n) ≥ cg(n) for every integer n ≥ n0
Calculate no of Primitives operations :
Problem
for (i = 1; i <= 2n; i++) {
for (j = 1; j <= i ; j ++)
sum++;
}
Calculate no of Primitives operations :
Problem
for (i = 1; i <= n; i++) {
for (j = 1; j <= n; j += i)
x = x + 1;
}
Proof of Correctness:
Justification Techniques
•
•
•
•
By Example
Contra Attack
Induction
LoopInvariants
Justification : By Example
• Counterexample
• Example:
– Claim : that every number of the form 2i - 1 is a
prime, when i is an integer greater than 1
– Proof:
24 - 1 = 15 = 3*5
Justification : Contra Attack
• Contrapositive
“if p is true, then q is true”
“if q is not true, then p is not true."
• Contradiction
To Prove statement q is true
Assume that q is false and then show that
this assumption is wrong.
Justification : Induction
• To prove q(n) is true
– Step 1 : q(n) is true for n = 1 (Base Case)
– Step 2: suppose (or assume that) q(n) is true for
some k < n (inductive assumption or
inductive hypothesis)
Then justify that q(n) is true for n>k
(Induction Step)
Justification : Induction Example
• Claim : F(n) < 2n . Consider the Fibonacci sequence: F (
l ) = 1 , F(2) = 2, and F(n) = F(n - 1 ) + F(n - 2) for n >
2.
• Proof:
Base Case : F(1) = 1 < 21
Induction step: Assume that claim is true for n`<n
F(n) = F (n - 1 ) + F(n - 2)
claim is correct for n-1 and n-2
Hence F(n) < 2n-1+2n-2
< 2n-1+2n-1
< 2*2n-1
< 2n
Justification : Loop Invariant
• To prove some statement S about a loop is correct, define
S in terms of a series of smaller statements S0 , S1 , . . , Sk
where:
1. The initial claim, S0, is true before the loop begins.
2. If Si-1 is true before iteration i begins, then one can
show that Si will be true after iteration i is over.
3. The final statement, Sk , implies the statement S
that we wish to justify as being true.
Algorithm arrayMax
Algorithm arrayMax(A,n)
Input: An array A storing n ≥ 1 integers.
Output: The maximum element in A.
currentMax ← A[0]
for i ← 1 to n - 1 do
if currentMax < A [i] then
currentMax ← A[i]
return currentMax