Asymptotic Analysis
Asymptotic Analysis/Dr.A.Sattar/1
Asymptotic Analysis
Topics
ƒ Asymptotic notation
ƒ Basic methods for analysis
ƒ Using Limits for asymptotic analysis
ƒ Properties of Asymptotic Notation
ƒ Analysis of summation
ƒ Analysis of operations on elementary data structures
Asymptotic Analysis/Dr.A.Sattar/2
1
Using Basic Methods
Asymptotic Analysis/Dr.A.Sattar/3
O-Notation
Definition
ƒ If f(n) is a growth function for an algorithm, and g(n) is a function such that for
some positive real constant c and for n ≥ n0 ,
0 < f(n) ≤ c.g(n)
then f(n) = O(g(n)) (Read f(n) is Big-Oh of g(n) )
ƒ The behavior of f(n) and g(n) is portrayed in the graph. It follows, that for n<n0, f(n) may
be above or below g(n), but for all n ≥ n0, f(n) falls consistently below g(n). The function g(n)
is said to be asymptotic upper bound for f(n)
ƒ There can be a several functions for which the g(n) is the upper bound. All such
functions are said belong to the group identified by g(n). Symbolically, we denote the
relationship as f(n)∈ O(g(n))
Asymptotic Analysis/Dr.A.Sattar/4
2
O-Notation
Example-1
ƒUsing basic definition we show that 3n2 + 10n
Consider,
10 ≤ n
10n ≤ n2
Therefore,
for
n ≥ 10
for n ≥ 10
∈
O(n2)
( obvious !)
( Multiplying both sides with n )
3n2+10n ≤ 4n2 for n ≥ 10
(Adding 3n2 to both sides )
3n2 + 10 n ≤ c.n2
( c=4 and n0=10 )
for n ≥ n0
it follows from the basic definition that
∈
3n2 +10n
O(n2)
Asymptotic Analysis/Dr.A.Sattar/5
O-Notation
Example-1 cont’d
ƒ The preceding result shows that
3n2 + 10 n ≤ c. n2, for c=4 and n ≥ 10
ƒHowever, the choice of c is not unique. We can select some other c and corresponding n0 so that the
relation still holds.
Consider,
3n2 + 10n ≤ 3n2 + 10 n2 for n ≥1 ( Replacing n with n2 on the right side )
=13 n2
for n≥ 1
3n2 +10n ≤ c.n2,
for n ≥ n0 , ( c=13 and n0=1)
ƒ The graph depicts the two solutions. Observe that both 13n2 and 4n2 eventually grow faster than
3n2+10
g(n)= cn2
c=13
g(n)=cn2
c=4
f(n)=3n2 + 10 n
n0=10
n0=1
Asymptotic Analysis/Dr.A.Sattar/6
3
O- Notation
Example-2
ƒ The technique can be applied to prove the Big-oh behavior of polynomials. For instance,
it can be shown that 7n3 - 5n2 - 20n - 16 ∈ O(n3).
ƒConsider, ,
7n3 - 5n2 -20n -16
≤ 7n3 + 5n2 +20n +16
for n ≥ 1 ( changing negative terms to positive)
≤ 7n3 + 5n3+20n3 +16n3 for n ≥ 1 ( replacing n2, n, and 1 with n3)
= (7+5+20+16) n3
=48n3
for n ≥ 1
7n3 - 5n2 -20n -16 ≤ c.n3 for n ≥ n0
( c= 48, n0=1)
Therefore, 7n3 - 5n2 -20n -16 ∈ O(n3)
Asymptotic Analysis/Dr.A.Sattar/7
O- Notation
Example-3
ƒIt can be shown that in general, a0 + a1n + a2n2+……….+aknk ∈ O(nk) , ak>0
For,
a0 + a1n + a2n2+……….+aknk ≤ |a0 |+ |a1|n + |a2|n2+……….+|ak|nk for n≥1
≤ |a0 |nk+ |a1|nk + |a2|nk+……….+|ak|nk for n≥1
= (|a0| + |a1| + |a2|+……….+|ak| ) nk
= c. nk,
n≥ n0
where c=|a0| + |a1| + |a2|+……….+|ak| and n0=1
Therefore , a0 + a1n + a2n2+……….+ aknk ∈ O(nk)
Asymptotic Analysis/Dr.A.Sattar/8
4
O-Notation
Example-4
ƒ We show that 2n ∈O(n!)
ƒ For, we have
2 <3
2<4
2<5
……..
……..
2< n for n>2
ƒ Multiplying both sides of inequalities
2(n-2)<3.4.5…..n for n>2
2n <2.2.3……….n (Multiply both sides with 2.2)
Or,
2n <2. n! for n>2
2n <c. n! for n > 2, c=2
Therefore, 2n
∈ O(n!)
Asymptotic Analysis/Dr.A.Sattar/9
Ω-Notation
Definition
ƒ If f(n) is a growth function for an algorithm, and g(n) is a function such that for
some positive real constant c and for all n ≥ n0 ,
0 < cg(n) ≤ f(n)
then f(n) = Ω(g(n)) (Read f(n) is Big-Omega of g(n) )
ƒ The behavior of f(n) and g(n) is portrayed in the graph. It follows, that for n<n0, f(n) may
be above or below g(n), but for all n ≥ n0, f(n) falls consistently above g(n). The function g(n)
is said to be asymptotic lower bound for f(n)
ƒ There can be a several functions for which the g(n) is the lower bound. All such
functions are said belong to the group identified by g(n). Symbolically, we denote the
relationship as f(n) ∈ Ω( g(n))
Asymptotic Analysis/Dr.A.Sattar/10
5
Ω-Notation
ƒWe show that n2 -10n
Example-1
∈ Ω(n ).
2
For, n
≥ n/2
for n ≥ 1
( Obvious !)
n - 10 ≥ n / (2 x 10) for n ≥ 10 ( Divide right side by 10)
= n / 20
n2 – 10n ≥ n2 / 20 for n ≥ 10 ( Multiply both sides with n to maintain inequality )
n2 – 10n ≥ c. n2 for n ≥ n0, where c=1 / 20 and n0=10
Therefore,
n2 -10n
∈
Ω(n2).
ƒ The behavior of functions n2-10n and n2 / 20 is shown in the graph. Observe that for
n≥10, the function n2 / 20 falls below the function n2-10n
f(n)=n2 -10n
n0=10
g(n)=c.n2, c = 1/20
Lower asymptotic bound
Asymptotic Analysis/Dr.A.Sattar/11
Ω -Notation
Example-2
ƒNext we show that 3n2 -25n
∈
Ω(n2).
n
≥ n / 2 for n ≥ 1
( Obvious !)
n- 25/3 ≥ 3 n / (2 x 25) for n ≥ 9 ( Divide right side by 25/3 ≈ 8.3)
= 3 n / 50
for n ≥ 9
3n2 – 25n ≥ 9n2 / 50 for n ≥ 9 ( Multiply both sides with 3n to maintain inequality )
3n2 – 25 n≥ c. n2 for n ≥ n0, where c=9 / 50 and n0=9
Therefore,
3n2 -25n ∈ Ω(n2).
ƒ The behavior of functions 3n2-25n and 9n2 / 50 is shown in the graph. Observe that for
n≥9, the function 9n2 / 50 falls below the function 3n2-25n
f(n)=3n2 -25n
g(n)=c.n2, c= 9/50
n0=9
Lower asymptotic bound
n
Asymptotic Analysis/Dr.A.Sattar/12
6
θ-Notation
Definition
ƒ If f(n) is a growth function for an algorithm, and g(n) is a function such that for
some positive real constant c1 , c2 and for all n ≥ n0 ,
0 < c2.g(n) ≤ f(n) ≤ c1.g(n)
then f(n) = θ(g(n)) (Read f(n) is theta of g(n) )
ƒ The behavior of f(n) and g(n) is portrayed in the graph. It follows, that for n < n0, f(n) may
be above or below g(n), but for all n ≥ n0, f(n) falls consistently between c1.g(n) and c2.g(n).
Function g(n) is said to be the asymptotic tight bound for f(n)
ƒ There can be a several functions for which the g(n) is asymptotic tight bound. All such
functions are said belong to the group identified by g(n). Symbolically, we denote the
relationship as f(n)∈ θ(g(n))
Asymptotic Analysis/Dr.A.Sattar/13
θ-Notation
ƒ We show that 5n2 -19n
∈
θ(n2).
Example
Consider the upper bound,
5n2 – 19n ≤ 5n2 for n ≥ 0
5n2 – 19 n ≤ c1. n2 for n ≥ n1, where c1=5 and n1=0
Next, consider the lower bound,
n ≥ n/2
for n ≥ 1
n- 19/5 ≥ 5 n / (2 x 19)
= 5 n / 38
( Obvious !)
for n ≥ 4 ( Divide right side by 19/5= 3.8)
for n ≥ 4
5n2 – 19n ≥ 25n2 / 38 for n ≥ 4
( Multiply both sides with 5n )
5n2 – 19 n≥ c2. n2 for n ≥ n2, where c2=25 / 38 and n2=4
It follows, 0 < c2.n2 ≤ 5n2 - 19n ≤ c1.n2
Therefore,
5n2
-19n
∈
for n ≥ n0 , where n0=4, c1=5 and c2=25/38.
θ(n2)
Asymptotic Analysis/Dr.A.Sattar/14
7
θ-Notation
Example-cont’d
ƒThe relation 5n2 -19n ∈ θ(n2) is illustrated in the figure below. The graph shows the
Asymptotic upper and lower bounds of the function f(n)= 5n2-19 n
Upper bound
g(n)=c1n2
c1=5
f(n)=5n2 -19n
g(n)=c2.n2
c2=25/38
n0=4
Lower bound
n
Asymptotic Analysis/Dr.A.Sattar/15
o-Notation
Definition
ƒIf f(n) is a growth function for an algorithm and g(n) is some function such that
for all real c > 0 and n ≥ n0
0 ≤ f(n) ≤ c.g(n)
then
f(n) = o(g(n)) ( Read f(n) is small-oh of g(n) )
ƒSymbolically, the relation is denoted by
f(n) = o( g(n))
Or,
f(n)
∈ o(g(n))
Asymptotic Analysis/Dr.A.Sattar/16
8
o-Notation
Example-1
ƒ We show that n ∈ o(n2)
For, if n ≤ c.n2 where c is a real constant
then
1 ≤ c.n ( Dividing both sides with n)
Or,
n≥ 1/c
ƒ It follows that for each c >0, we need to find n0 such that for n ≥ n0, the above inequality
holds true. This is indeed possible for all choices of c>0.
For example, the table shows some typical choices of c and the corresponding values of n0.
c
1/c
n0
0.01
100
100
0.1
10
10
0.2
10/2=5
5
0.3
10/3=3.3
4
0.4
10/4=2.5
3
0.5
10/5=2
2
1
1
1
5
0.2
1
10
0.1
1
ƒIt follows that in general for any c we can find some n0 so that n ≤ cn2 for n ≥ n0
Therefore, n∈ o(n2)
Asymptotic Analysis/Dr.A.Sattar/17
ω-Notation
Definition
ƒ If f(n) is a growth function for an algorithm and g(n) is some function such that
for all real c>0 and n ≥ n0
0< c.g(n) ≤ f(n)
then
f(n) = ω(g(n)) (Read f(n) is small omega of g(n) )
ƒSymbolically, the relation is denoted by
f(n)= ω(g(n))
Or, f(n)
∈
ω( g(n))
Asymptotic Analysis/Dr.A.Sattar/18
9
ω-Notation
ƒ We show that 10-3.n3
Example
∈ ω( n2)
ƒConsider,
10-3 .n3 ≥ c.n2, where c is a real positive number
Or,
10-3. n ≥ c
n ≥ 1000c
ƒIt is possible to choose any arbitrary c>0 and a corresponding n0 so that the above
inequality holds true. The table below gives some illustrative choices for c and
corresponding n0
c
n0
2
2000
1
1000
0.1
100
0.01
10
0.001
1
ƒWhatever c is chosen, it is possible to select n0 such that for n ≥ n0, 10-3.n3 ≥ c.n2
Therefore, 10-3.n 3 ∈ ω( n2)
Asymptotic Analysis/Dr.A.Sattar/19
Using Limits
Asymptotic Analysis/Dr.A.Sattar/20
10
O-Notation
Using Limits
ƒ If f(n) and g(n) are growth functions such that
f(n)
lim
—— = c , where 0 ≤ c < ∞
n → ∞ g(n)
then∈f(n)
∈ O(g(n)).
Example(1): 3n2 + 5n+ 20
∈ O(n2)
3n2 + 5n + 20
lim
n2
n→∞
Therefore, 3n2 + 5n+ 20
Example(2): 10n2 + 25n+ 7
=3+ 5 / n +20 / n2 = ( 3+0+0)=3
∈ O(n2)
∈
O(n3)
10n2 + 25n + 7
lim
=10 / n+ 25 / n2 + 7 / n3 = (0+0+0)=0
n3
n→∞
Therefore, 10n2 + 25n+ 7
∈
O(n3)
Asymptotic Analysis/Dr.A.Sattar/21
Computing Limits
L’Hopital Rule
ƒIn some cases, when n tends to infinity, both f(n) and g(n) tend to infinity. In other words,
the numerator and denominator both become infinite. In such cases, a theorem called,
L’ Hopital Rule can be used to compute the limit.
ƒ If f(n) and g(n) are growth functions such that
lim
n→∞
f ( n) / g ( n ) = ∞ / ∞
then by L’Hopital’s Rule
lim
n→∞
f ( n) / g ( n) =
lim
f ′(n) / g ′(n)
n→∞
where f ‘(n), g ‘(n) are differentials of f(n) and g(n) with respect to n.
ƒ If after taking differential, the numerator and denominator both tend to infinite, the
L’Hopital rule can be applied again
Asymptotic Analysis/Dr.A.Sattar/22
11
O-Notation
Examples
Example(1): lg n ∈ O(n)
In order compute differential of lg n we first convert binary logarithm to natural logarithm.
Converting lg n (binary ) to ln n ( natural), by the using formula lg n = ln n / ln 2
lg n
(ln n)
lim
=
=∞ / ∞
n→∞ n
(ln 2)n
1
lim
n → ∞ ln 2. n = 0 ( Differentiating the numerator and denominator)
lg n ∈ O(n)
Therefore,
Example(2):
n2
∈ O(2n)
n2
lim
=∞ / ∞
n → ∞ 2n
2n
lim
n → ∞ ln 2 2n
= ∞ / ∞ (Differentiating again the numerator and denominator)
2
lim
n → ∞ (ln2)2. 2n
Therefore,
n2
(Differentiating the numerator and denominator)
=0
∈ O(2n)
Asymptotic Analysis/Dr.A.Sattar/23
Ω -Notation
Using Limits
ƒ If f(n) and g(n) are growth functions such that
f(n)
lim
—— = c , where 0 < c ≤ ∞
n → ∞ g(n)
then∈f(n)
∈ Ω(g(n)).
Example(1): 7n2 + 12n+8
∈ Ω(n2)
7n2 + 14n + 8
lim
n2
n→∞
Therefore, 7n2 + 14n+ 8
=7+ 14 / n +8 / n2 =(7 + 0 + 0)=7
∈ Ω(n2)
Example(2): 10n3 + 5n+ 2 ∈ Ω(n2)
10n3 + 5n + 2
lim
=10 n+ 5 / n + 2 / n2 = ( ∞ + 0 + 0 ) = ∞
n2
n→∞
Therefore, 10n3 + 5n+ 2
∈ Ω (n2)
Asymptotic Analysis/Dr.A.Sattar/24
12
θ-Notation
Using Limits
ƒ If f(n) and g(n) are growth functions such that
f(n)
lim
—— = c , where 0 < c < ∞ , ( c=0 is excluded )
n → ∞ g(n)
then∈f(n)
∈ θ(g(n)).
∈ θ(n3)
Example(1): 45n3 - 3n2 - 5n+ 20
45n3 -3n2 - 5n + 20
lim
= 45-3 / n -5 / n2 +20/n3 =(45-0 -0)=45
n3
n→∞
Therefore, 45n3 - 3n2 - 5n+ 20 ∈ θ(n3)
Example(2): n lg n + n + n2 ∈ θ(n2)
n lg n + n + n2
lim
n→∞
n2
Thus, n lg n + n + n2
= lg n / n +1/n +1=(0+0+1)=1
∈ θ(n2)
Asymptotic Analysis/Dr.A.Sattar/25
o-Notation
Using Limits
ƒ If f(n) and g(n) are growth functions such that
f(n)
lim
—— = 0 ,
n → ∞ g(n)
then∈f(n)
∈ o(g(n)).
Example(1): 3n2 + 5n
3n2 + 5n
lim
n3
n→∞
Therefore, 3n2 + 5n
Example(2): 2n
∈ o(n3)
=3/n+ 5 / n =(0+ 0 )=0
∈ o(n3)
∈ o ( n!). Using Stirling’s approximation for n!
2n
2n
lim
=
=
n → ∞ n!
√2 π n ( n/e)n
Thus,
1
1
=
(√2 π n (n/e)n) /2n
√2 πn ( n/2e)n
=0
2n ∈ o ( n!)
Asymptotic Analysis/Dr.A.Sattar/26
13
ω-Notation
Using Limits
ƒIf f(n) and g(n) are growth functions such that
f(n)
lim
—— = ∞ ,
n → ∞ g(n)
then∈f(n)
Example :
∈ ω(g(n)).
3n3 + 5n2
∈ ω(n)
3n3 + 5n2
lim
=3n2+ 5 n = (∞ + ∞) =∞
n
n→∞
Therefore, 3n3 + 5n2
∈ ω(n)
Asymptotic Analysis/Dr.A.Sattar/27
Asymptotic Notation
Summary
lim
n→∞
f(n)
——— = α
g(n)
Notation
Using Basic Definition
Using Limits
Bound
f(n)=O( g(n) )
f(n) ≤ c.g(n) for some c>0, and n ≥ n0
0≤ α <∞
Tight upper
f(n)=o( g(n) )
f(n) ≤ c.g(n) for all c>0 and n ≥ n0
α=0
Loose upper
f(n)=Ω( g(n) )
f(n) ≥ c.g(n) for some c>0 and n ≥ n0
0< α ≤∞
Tight lower
f(n)=ω( g(n) )
f(n) ≥ c.g(n) for all c>0 and n ≥ n0
α=∞
Loose lower
f(n)=θ( g(n) )
c1.g ≤ f(n) ≤ c2.g(n) for some c1>0, c2>0
and n ≥ n0
0<α <∞
Tight, upper and
lower
Asymptotic Analysis/Dr.A.Sattar/28
14
Asymptotic Notation Theorems
Asymptotic Analysis/Dr.A.Sattar/29
Asymptotic Notation
Constants
ƒ If c is a constant then by convention
∈O(1) ,
O(c)
∈ θ(1),
θ (c)
∈ Ω(1)
Ω(c)
The convention implies that the running time of an algorithm ,which does not depend
on the size of input, can be expressed in any of the above ways.
ƒ If c is constant then
O( c.f(n) ) = O(f(n))
θ( c.f(n) )
= θ(f(n))
Ω( c.f(n) ) = Ω(f(n))
The above relations imply that in a summation the multiplier constants can be ignored
For example, O(1000n)=O(n), θ(7lgn ) = θ (lg), Ω(100n!)= Ω(n!)
Asymptotic Analysis/Dr.A.Sattar/30
15
Asymptotic Notation
θ,O, Ω Relations
ƒ If f(n)
f(n)
∈
∈
θ( g(n) ) then
Ω( g(n) ),
f(n)
∈ O( g(n) )
∈
ƒ Conversely if f(n) ∈ Ω( g(n) ) and f(n)
then f(n)
∈
O( g(n) ) then
θ( g(n) )
Example: Since, n(n-1)/2 ∈ θ(n2 ), therefore
n(n-1)/2 ∈ Ω(n2 )
n(n-1)/2 ∈ O(n2 )
Asymptotic Analysis/Dr.A.Sattar/31
Asymptotic Notation
Set Representation
ƒ The relationship among the O-,Ω-,θ-Notation can be expressed using set notation
Consider, for example, the following sets of growth functions
• SO(n2) =
{ f(n): √n, n+5, lg n+4n, n1.5+n, √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 }
where SO(n2) is a set of functions f(n)
∈ O(n )
2
• SΩ(n2) = { f(n): √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 , 5n2+n3, n3 +n2+n, lg n+4n4, nlg n+3n4 }
where SΩ(n2) is a set of functions f(n)
∈ Ω(n )
2
• Sθ(n2) = f(n):{ √n+5n2 , n2+5n, lg n+4n2, n1.5+3n2 }
where Sθ(n2) is a set of functions f(n)
ƒ It follows that
∈θ(n )
2
Sθ(n ) = SO(n ) ∩ SΩ(n )
2
2
2
Asymptotic Analysis/Dr.A.Sattar/32
16
O-, Ω-, θ-Notation Relation
Set Representation
ƒThe following Venn diagram represents the preceding relation
√n
n+5
lg n+4n
n1.5+n
√n+5n2
n2+5n
lg n+4n2
n1.5+3n2
√n+5n2
5n2+n3
n2+5n
n3 +n2+n
lg n+4n2 lg n+4n4
n1.5+3n2 nlg n+3n4
SO(n2)
SΩ(n2)
√n
n+5
lg n+4n
n1.5+n
√n+5n2
n2+5n
lg n+4n2
n1.5+3n2
5n2+n3
n3 +n2+n
lg n+4n4
nlg n+3n4
Sθ(n2)
Asymptotic Analysis/Dr.A.Sattar/33
θ,Ω, O relation
Set Representation
ƒIn general, the relation among θ,Ω, O is expressed using set notation, as shown below.
Asymptotic Analysis/Dr.A.Sattar/34
17
Asymptotic Notation
o -O Relation
ƒ If f(n)
∈
o( g(n) ) then
f(n) ∈ O( g(n) )
Example(1): Since, 2n ∈ o( n! ), therefore
∈ O( n!
2n
)
ƒ Converse is not true
That is, if f(n)
∈ O( g(n) ),
then f(n)
Example (2): n2 + n ∈ O(n2),
∉ o(g(n))
but n2 + n
∉ o(n )
2
Asymptotic Analysis/Dr.A.Sattar/35
Asymptotic Notation
ƒ If f(n)
∈
ω-Ω Relation
ω( g(n) ) then
f(n) ∈ Ω( g(n) )
Example(1): Since, n! ∈ω(2n)
n!
therefore
∈ Ω (2n)
ƒ Converse is not true
That is, if f(n)
∈ ω( g(n) ),
Example (2): n2 + n ∈ Ω(n2),
then f(n)
∉ Ω(g(n))
but n2 + n
∉ ω(n )
2
Asymptotic Analysis/Dr.A.Sattar/36
18
Asymptotic Notation
Order Theorem
Theorem: If f1(n)∈ O( g1(n) ) and f2(n)∈ O( g2(n) ) then
f1(n) + f2(n)
∈
O( max( g1(n) , g2(n) )
Proof: By definition,
f1(n) ≤ c1. g1(n)
for n ≥ n1
for n ≥ n2
f2(n) ≤ c2. g2(n)
Let n0 = max( n1, n2) c3=max(c1, c2)
f1(n) ≤ c3. g1(n)
f2(n) ≤ c3 g2(n)
for n ≥ n0
for n ≥ n0
f1(n) + f2(n) ≤ c3.g1(n) + c3. g2(n) )
for n ≥ n0
Let h(n) = max( g1(n) , g2(n) )
f1(n) + f2(n) ≤ 2c3.h(n) = c. h(n) where c=2c3
for n ≥ n0
f1(n) + f2(n) ≤ c. h(n) = c. max( g1(n) , g2(n) )
for n ≥ n0
Therefore, f1(n) + f2(n) ∈ O( max( g1(n) , g2(n) )
ƒ The theorem also applies to θ and Ω notations
Asymptotic Analysis/Dr.A.Sattar/37
Asymptotic Notation
Using Order Theorem
The relation
f1(n) + f2(n)
∈
O( max( g1(n) , g2(n) )
implies that in a summation, the lower order growth functions can be discarded in favor
of the highest ranking function
Example: Consider the summation f(n) consisting of basic functions :
f(n) = n +√n + n1.5+ lg n + n lg n + (lg n)2 + n2
The function n2 grows faster than all other functions in the expression
Thus, O( max( n + √n + n1.5+ lg n + n lg n + (lg n)2 + n2 ) ) =O( n2)
It follows that f(n)
∈ O( n2 )
Asymptotic Analysis/Dr.A.Sattar/38
19
Analysis of Summations
Asymptotic Analysis/Dr.A.Sattar/39
Arithmetic Summation
Asymptotic Behavior
ƒ The sum of first of n terms of arithmetic series is :
1 + 2 + 3………..+n = n(n+1)/2
Let f(n)= n(n+1)/2
and g(n)= n2
lim
n→∞
g(n)
n2 /2 + n/2
n(n+1)/2
f(n)
=
n2
=
n2
= 1 /2 + 1/2n =(1/2 + 0)=1/2
ƒ Since the limit is non-zero and finite it follows
∈ θ( g(n)) = θ(n )
1 + 2 + ………..+ n ∈ θ(n )
f(n)
Or,
2
2
Asymptotic Analysis/Dr.A.Sattar/40
20
Geometric Summation
Asymptotic Behavior
ƒ The asymptotic behavior of geometric series
1 +r + r2+……..+rn
depends on geometric ratio r. Three cases need to be considered
Case r > 1: It can be shown that sum f(n) of first n terms is as follows
rn+1 - 1
f(n) = 1 +r + r2+……..+rn = —————
r - 1
Case r = 1:
f(n)= 1 + 1 + 1+ ……+ = n
Case r < 1: It can be shown that
1 - rn+1
f(n) = 1 +r + r2+……..+rn = —————
1-r
ƒ The asymptotic behavior in first case and third case is explored by computing limits.
Asymptotic Analysis/Dr.A.Sattar/41
Geometric Summation
Case r > 1
rn+1 - 1
Let f(n) = 1 +r + r2+……..+rn = —————
r - 1
Let g(n)= rn
Consider, the limit
f(n)
lim
n→∞
=
g(n)
rn+1 - 1
(r - 1).rn
=
r - 1/rn
(r - 1)
Since r>1 , 1/rn →0 as n→ ∞
lim
n→∞
Therefore, f(n)
Or,
f(n)
=
g(n)
∈θ(r )
n
r
(r - 1)
>0, since r > 1
for r>1
1 +r + r2+……..+rn
∈ θ(r )
n
for r > 1
Asymptotic Analysis/Dr.A.Sattar/42
21
Geometric Summation
Case r < 1
rn+1 - 1
Let f(n) = 1 +r + r2+……..+rn = —————
r - 1
Let g(n)=c where c is some constant
Consider, the limit
f(n)
lim
n→∞
rn+1 - 1
=
g(n)
=
(r - 1).c
1- rn+1
(1- r )c
Since r<1 , rn+1 →0 as n→ ∞
lim
n→∞
Therefore, f(n)
f(n)
g(n)
1
=
∈ θ(g(n))=θ(c) = θ(1)
Or, 1 +r + r2+……..+rn
>0, since r < 1
(1 - r)c
∈ θ(1)
for r <1
for r <1
Asymptotic Analysis/Dr.A.Sattar/43
Logarithm Summation
Asymptotic Behavior
ƒThe logarithmic series has the summation
lg(1)+lg(2)+ lg(3)+….. +lg(n)
Let f(n) = lg(2)+ lg(3)+….+.lg(n)
and g(n) = nlg n
lim
n→∞
f(n)
g(n) =
lg n!
n lg n =
= lg(2.3……..n)
lg ( √(2πn)(n/e)n )
= lg( n!)
(using Stirling’s Approximation)
n lg n
Now, lg ( √(2πn)(n/e)n ) = (1+lg π + lg n)/2 + n lg n - n lg e , therefore
n
lim lg ( √(2πn)(n/e) )
n→∞
n lg n
= ( 1+lg π + lg n ) /(2 nlg n) + 1 - lg e / lg n = (0+ 1- 0)=1
Since limit is non-zero and finite, it follows
f(n)
θ(g(n))
∈
Or, lg(1)+lg(2)+ lg(3)+….. +lg(n)
∈
θ(n lg n) …
Asymptotic Analysis/Dr.A.Sattar/44
22
Harmonic Summations
Asymptotic Behavior
ƒThe sum of first n terms of Harmonic series is
1+ 1/2 + 1/3+…..+1/n
Let f(n) = 1+ 1/2 + 1/3+…..+1/n
and g(n) = lg n
1+1/2+1/3+ …..+1/n
f(n)
=
g(n)
lg n
I t can be shown that 1+ 1/2+ 1/3+…+1/n = lg (n) + γ + 1/2n – 1/12n2+.. where γ ≈ 0.5772
lim
n→∞
lim
n→∞
lg n + γ + 1/2n – 1/12n2 +…
f(n)
= 1+0+0-0+0+…=1
g(n) =
lg n
Since limit is finite and non-zero, it follows
f(n) ∈ θ( g(n))
Or, 1+ 1/2 + 1/3+…..+1/n
∈
θ (lg n )
Asymptotic Analysis/Dr.A.Sattar/45
Polynomials
Asymptotic Behavior
ƒLet p(n) be a polynomial of degree k,
p(n) = a0 + a1n + a2n2+……..+aknk,
(1) Let g(n)= nk
where ak > 0
a0 + a1n + a2n2+……..+aknk,
p(n)
lim
n→∞ g(n)
nk
Thus, p(n) ∈θ(nk),
p(n)
∈ O(nk)
,and p(n)
= ak > 0
∈ Ω(nk)
(2) Let g(n)= nk+1
p(n)
lim
n→∞ g(n)
Thus, p(n)
∈
a0 + a1n + a2n2+……..+aknk,
nk+1
=0
o(nk+1) ,and p(n) ∈ O(nk+1)
(3) Let g(n)= nk-1
p(n)
lim
n→∞ g(n)
a0 + a1n + a2n2+……..+aknk,
nk-1
Thus, p(n)∈ ω(nk-1) ,and p(n)∈ Ω(nk-1)
= ∞
Asymptotic Analysis/Dr.A.Sattar/46
23
Analysis
of
Elementary Data Structures
Asymptotic Analysis/Dr.A.Sattar/47
Arrays
Basic Operations
ƒAn array is a collection of finite, linear , homogenous data items. Elements are
accessed using an index set
ƒ The basic operations on an array are:
• Accessing
• Insertion
• Deletion
• Linear Search
ƒ Since an element is accessed direct using index, the accessing operation has running time θ(1)
Asymptotic Analysis/Dr.A.Sattar/48
24
Arrays
Insertion Operation
ƒTo insert an element in a given position, all succeeding elements are shifted one position to
to right to make room for the new element.
A[1]
A[2]
--
A[k-1]
A[k]
A[k+1]
….
…
..
A[n]
ƒThe pseudo code for inserting an element key in kth position of Array A[1..n] is
ARRAY-INSERT (A, k, key)
1 j ←n
2 while j > k do
3
A[j+1] ← A[j] ► Shift right
4
j← j-1
5 A[j] ← key
ƒA total of n-k+1 shifting and looping operations are executed. The running time Tinsert(n) is
expressed as:
a + b. (n-k+1), where a and b are constants
Tinsert(n) =
initialization Shifting and looping
ƒ To insert an element in last position we set k = n. Thus, Tinsert(n) = a +b = θ(1)
ƒ To insert an element in first position we set k = 1. Thus, Tinsert(n) = a +b.n = θ(n)
Asymptotic Analysis/Dr.A.Sattar/49
Arrays
Deletion Operation
ƒTo delete an element in a given position, all succeeding elements are shifted one position to
left. The element in given position is overwritten.
A[1]
A[2]
--
A[k-1]
A[k]
A[k+1]
….
…
..
A[n]
ƒThe pseudo code for deleting an element in kth position of Array A[1..n] is
ARRAY-DELETE (A[1..n], k)
1 j ←k
2 while j ≤ n do
3
A[j] ← A[j+1] ► Shift to left position
4
j ← j+ 1
ƒA total of n-k+1 shifting and looping operations are executed. The running time Tdelete(n) is
expressed as:
a
+
b. (n-k+1), where a and b are constants
Tdelete(n) =
initialization
Shifting and looping
ƒ To delete last element we set k = n. Thus, Tdelete(n) = a +b = θ(1)
ƒ To delete first element we set k = 1. Thus, Tdelete(n) = a +b.n = θ(n)
Asymptotic Analysis/Dr.A.Sattar/50
25
Arrays
Linear Search
ƒIn linear search, the array is scanned sequentially. Pseudo code for searching Array A[1..n] is
as follows. If research is successful the index of key is returned; otherwise, -1 is returned.
ARRAY-SEARCH (A, key)
1 for j ← 1 to j ≤ n do
2
if A[j] = key
3
return j
► Search successful
4 return -1
► Search unsuccessful
ƒ In the worst case the loop will be executed n times. The running time is:
Tworst(n)= a+ b.n , where a and b are constants
=θ(n)
ƒ We use probabilistic analysis to find average running time. Assuming that key has
equal probability , 1/n ,of being in any position, and c is unit cost, the expected ( average)
running time is given by
Taverage(n) =1c.1/n + 2.c.1/n+…….+(n-1).c.1/n + n.c.1/n = [1+2+3+….+n ].c./ n
= c. (n+1)/2 = c.n/2 + c/2
= θ(n)
Asymptotic Analysis/Dr.A.Sattar/51
Stacks
Basic Operations
ƒ A stack is a data collection in which an item is added and removed from the same end
Stack implements the principle of LIFO (Last In First Out) The basic operations on
stack are:
• Push- adds an item to stack
• Pop- removes and returns an item from stack
• Peek -returns an item from stack without removing it
• Empty- determines stack underflow condition
• Full- determines stack overflow condition
Asymptotic Analysis/Dr.A.Sattar/52
26
Stacks
Push Operation
ƒStack is implemented using an array S[1..n]. An index variable top identifies the
most recently inserted element. When an item is added, top is incremented and new item is
placed in the array cell pointed by top. The pseudo code for the Push operation is shown below.
It pushes the item x onto the stack.
x
filled
top
STACK-PUSH( x)
1 top ← top+1
2 S[top] ← x
ƒ The running time for push consist the cost (Ci) of incrementing top , and cost (Ca)
of inserting x into the array in the last cell. Both the costs are fixed . Thus, the
running time Tpush is
Tpussh= Ci + Ca,
where Ci and Ca are constants
= θ(1)
Asymptotic Analysis/Dr.A.Sattar/53
Stacks
Pop Operation
ƒThe Pop operation returns the most recently inserted element. After the item is returned
top is decremented by 1. Pseudo code for Pop operation is given below.
.
returned
x
filled
top
STACK-POP()
1 temp ← top
2 top ← top-1
3
return S [temp]
ƒ The running time for push consist of cost (Cs) of storing index top, cost (Cd) of
decrementing top , and cost (Cr )of accessing and returning array element. Thus, the
running time Tpop for the push operation is
Tpop= Cs + Cd+ Cr,
where Cs, Cd, and Cr are constants
= θ(1)
Asymptotic Analysis/Dr.A.Sattar/54
27
Queues
Basic Operations
ƒ A queue is a data collection in which an item added first is removed first. Queue
implements the principle of FIFO (First In First Out) Basic operations on a queue are:
•
Enqueue- inserts a new item into the queue
• Dequeue – fetches an item from the queue
•
Empty- determines if queue is empty
•
Full- determines if queue is full
Asymptotic Analysis/Dr.A.Sattar/55
Queues
Enqueue Operation
ƒAn array implementation of a queue uses two indexes: tail ,head . The tail pointer
indexes the next vacant location at which new element will be inserted. The head
pointer indexes the element to be returned from the queue.
filled
head
tail
ƒ The pseudo code for enqueue operation is as follows. Array Q is used to implement the
queue.
ENQUEUE( x)
1 Q [tail] ← x
2
tail←tail + 1
ƒ If is Ca the cost of accessing array element and assigning x to it , and Ci is the cost of
incrementing pointer tail, the running time Tenqueue for enqueue operation is given by
Tenqueue= Ca + Ci, where Ca and Ci are constants
Tenqueue = θ(1)
Asymptotic Analysis/Dr.A.Sattar/56
28
Queues
Dequeue Operation
ƒ The dequeue operation returns the element that was first inserted.
filled
head
tail
ƒ The pseudo code for dequeue operation is as follows. Array Q is used to implement the
queue.
DEQUEUE( )
1 x ←Q[ head]
2 head←head + 1
3
return x
ƒ If is Ca the cost of accessing array element and assigning it x, Ci is the cost of
incrementing pointer head, and Cr is cost of returning x, the running time Tdequeue for dequeue
operation is given by
Tdequeue= Ca + Ci + Cr , where Ca, Ci and Cr are constants
Tdequeue = θ(1)
Asymptotic Analysis/Dr.A.Sattar/57
Linked
Lists
Basic Operations
ƒ A linked list is collection data items, called nodes. Each node contains a key and one or
more pointers (addresses ) of the neighboring nodes. Basic operations on a linked list are:
• Insert- adds a new at the front, in the middle or at the tail
• Delete- removes a front, middle, or tail node
• Search- searches a node for a given key
Asymptotic Analysis/Dr.A.Sattar/58
29
Linked
Lists
Inserting Front Node
ƒTo insert a front node, a new node is created. The next node pointer of new node is set to
point to the front node in the original list. The head pointer is reset to point to the new node.
new node
head
head
head
ƒ The pseudo code for inserting a front node is as follows. Note x is pointer to newly
created node.
LIST-INSERT(x)
1 next[ x] ← head ► Set next node pointer of x to point to previous front node
2 x← head
► Reset head pointer to point to new node
ƒ The basic operations involve resetting the pointers. If C is the cost of resetting a pointer,
The running time for inserting front node is
Tinsert = 2. C, where C is constant
=θ(1)
Asymptotic Analysis/Dr.A.Sattar/59
Linked
Lists
Deleting Front Node
ƒTo delete the front node, the head pointer is temporarily stored in temp. The head pointer is
reset to point to next node. The next node pointer of front node is set to NULL to de-link it from
the front node.
temp head
temp
temp
head
head
ƒ The pseudo code for deleting a front node is as follows.
LIST-DELETE()
1 temp ← head
► temp points to the node to be deleted
2 head ←next[head]
► Reset head pointer to point to next node
3 next [temp]← NULL ► de-link the front node
ƒ The basic operations involve resetting the pointers. If C is the cost of resetting a pointer,
the running time for deleting front node is
Tdelete = 3. C, where C is constant
=θ(1)
Asymptotic Analysis/Dr.A.Sattar/60
30
Linked
Lists
Search Operation
ƒ To search key k, the head pointer is temporarily stored in temp. The pointer temp is used to
scan the list. If key matches with the list data element, the search is abandoned, and temp gives
the pointer to the node that contains the matched data item..
k
head
x
temp
ƒ The pseudo code for searching a linked list is as follows.
LIST-SEARCH( k )
1 temp ← head
2 while temp ≠ NULL and key [temp] ≠ k do
3 temp ←next [temp]
4 return temp
ƒ In worst case the while loop is executed n times. If a is total fixed and initializing
cost and b ist looping and comparing cost, the worst running time for search is given by
T worst = a + b. n , where a and b are constants.
= θ(n)
Asymptotic Analysis/Dr.A.Sattar/61
31