θ-Notation
Definition
If f(n) is running time of an algorithm, and g(n) is some standard growth function
such that for some positive constants c1 , c2 and positive integer n0 ,
0 < c2.g(n) ≤ f(n) ≤ c1.g(n)
for all n ≥ n0
then f(n) = θ(g(n)) (Read f(n) is theta of g(n) )
The behavior of f(n) and g(n) is portrayed in the graph. It follows that for n < n0, f(n) may
be above or below g(n), but for all n ≥ n0, f(n) falls consistently between c1.g(n) and c2.g(n). It
also implies that g(n) grows as fast as f(n) The function g(n) is said to be the asymptotic tight
bound for f(n)
Asymptotic Upper Bound
Asymptotic Lower Bound
Asymptotic tight bound of f(n)
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/18
Set Builder θ-Notation
Definition
There can be a several functions for which the g(n) is asymptotic tight bound. All such
functions are said to belong to the class Ө(g(n))
 Using set builder notation,, if
Ө(g(n)) = { f(n): there exit positive constants c1,c2 and positive integer n0 such that
c1 g(n) ≤ f(n) ≤ c2 g(n) }
then
f(n)  θ(g(n))
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/29
θ-Notation
Basic Method
Example: We show that 5n2 -19n = θ(n2).
First consider the upper bound . As explained before,
5n2 – 19n ≤ 5n2 for n ≥ 0
5n2 – 19 n ≤ c1. n2 for n ≥ n1, where c1=5 and n1=0 ………..(1)
Next, consider the lower bound,
n ≥ n/2
for n ≥ 0
n- 19/5 ≥ 5 n / (2 x 19)
= 5 n / 38
( Obvious !)
for n ≥ 4 ( Dividing right side by 19/5= 3.8)
for n ≥ 4
5n2 – 19n ≥ 25n2 / 38 for n ≥ 4
( Multiplying both sides with 5n )
5n2 – 19 n≥ c2. n2 for n ≥ n2, where c2=25 / 38 and n2=4 …….(2)
From (1) and (2) it follows ,
0 < c2.n2 ≤ 5n2 - 19n ≤ c1.n2
for n ≥ n0 , where n0=4, c1=5 and c2=25/38. (Solution)
Therefore, 5n2 -19n = θ(n2)
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/20
θ-Notation Example
Comparison of Growth Rates
The relation 5n2 -19n = θ(n2) is illustrated in the figure below. The graph shows the
asymptotic upper and lower bounds for the function f(n)= 5n2-19 n
Asymptotic Upper bound
cg(n)=5n2
f(n)=5n2 -19n
cg(n)=25n2/38
n0=4
AsymptoticLower bound
n
Growth of functions 5 n2 and 25n2/38
versus the function f(n)=5 n2 -19 n
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/ 24
Asymptotic Notation
Constant Running Time
If running time T(n)=c is a constant, i. e independent of input size, then by convention,
the asymptotic behavior is denoted by the notation
O(c) =
O(1) ,
θ (c) = θ(1), Ω(c) = Ω(1)
The convention implies that the running time of an algorithm ,which does not depend
on the size of input, can be expressed in any of the above ways.
 If c is constant then using basic definition it can be shown that
O( c.f(n) ) = O( f(n) )
θ( c.f(n) ) = θ( f(n) )
Ω( c.f(n) ) = Ω( f(n ))
Example: (i) O(1000n) = O(n),
(ii) θ(7lgn ) = θ (lg n),
(ii) Ω(100 n!) = Ω(n!)
The above results imply that in asymptotic notation the multiplier constants in an
expression for the running time can be dropped
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/22
Asymptotic Notations
Relationships
Theorem: If f(n) = θ( g(n) ) then
f(n) = Ω( g(n) ),
and f(n)= O( g(n) )
Conversely, if f(n) = Ω( g(n) )
and
f(n) = O( g(n) )
then f(n) = θ( g(n) )
 The above relations can be established by using basic definitions
Example(1): Since, n(n-1)/2 = θ(n2 ), therefore it follows that
n(n-1)/2 = Ω(n2 )
n(n-1)/2 = O(n2 )
Example(2): It can be shown that
5n2+1 = Ω(n2 )
and
5n2+1 = O(n2 )
Therefore, 5n2 + 1 = θ(n2 )
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/23
Asymptotic Set Notations
Relationship
We have seen that if f(n) = O( g(n) ) and f(n) = Ω(g(n)) then f(n) = Ө( g(n) )
Using set notation we can express the relationship as follows:
if
f(n)
O( g(n) ),
and f(n)
Ω(g(n)),
then
f(n)
 Ө( g(n) ) ,
where Ө(g(n)) = O( g(n) ) ∩ Ω (g(n) )
 The above property is illustrated by the following Venn diagram
O(g(n))
Ө(g(n))
Ω(g(n))
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/24
Asymptotic Set Notation
Example
The relationship among the O ,Ω , and θ notations can be expressed using set notation
Consider, for example, the following sets of growth functions
O(n2) =
{√n, n+5, lg n+4n, n1.5+n, √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 }
Ω(n2) = { √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 , 5n2+n3, n3 +n2+n, lg n+4n4, nlg n+3n4 }
θ(n2) =
{ √n+5n2 , n2+5n, lg n+4n2, n1.5+3n2 }
 It follows that
θ(n2) = O(n2) ∩ Ω(n2)
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/25
Asymptotic Set Notation
Example
The following Venn diagram represents the preceding set relations
√n
n+5
lg n+4n
n1.5+n
√n+5n2
n2+5n
lg n+4n2
n1.5+3n2
√n+5n2
n2+5n
lg n+4n2
n1.5+3n2
O(n2)
5n2+n3
n3 +n2+n
lg n+4n4
nlg n+3n4
Ω(n2)
√n
n+5
lg n+4n
n1.5+n
√n+5n2 5n2+n3
n3 +n2+n
n2+5n
lg n+4n2 lg n+4n4
n1.5+3n2 nlg n+3n4
θ(n2)
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/26
Asymptotic Notation
Order Theorem
Theorem: If f1(n) = O( g1(n) ) and f2(n) = O( g2(n) ) then
f1(n) + f2(n)=
O( max( g1(n) , g2(n) )
Proof: By definition,
f1(n) ≤ c1. g1(n)
for n ≥ n1
for n ≥ n2
f2(n) ≤ c2. g2(n)
Let n0 = max( n1, n2) c3=max(c1, c2)
f1(n) ≤ c3. g1(n)
f2(n) ≤ c3 g2(n)
for n ≥ n0
for n ≥ n0
f1(n) + f2(n) ≤ c3.g1(n) + c3. g2(n) )
for n ≥ n0
Let h(n) = max( g1(n) , g2(n) )
f1(n) + f2(n) ≤ 2c3.h(n) = c. h(n) where c=2c3
for n ≥ n0
f1(n) + f2(n) ≤ c. h(n) = c. max( g1(n) , g2(n) )
for n ≥ n0
Therefore, f1(n) + f2(n) = O( max( g1(n) , g2(n) )
 The theorem also applies to θ and Ω notations
Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/10
Asymptotic Notations
General Theorem
Theorem: If f1(n)= O(g1(n)) , f2(n) =O(g2(n)), f3(n)=O(g3(n))…., fk(n)=O(gk(n))
then
f1(n) + f2(n) + f3(n) ….+ fk(n) = O( max (g1 (n), g2 (n), g3 (n)…, gk(n) ) )
where max means fastest growing function
The theorem can be proved by using basic definition of Big-Oh
It follows from the theorem that in an expression consisting of sum of several functions,
the comparatively slower growing functions can be discarded in favor of the fastest
growing function to obtain the Big oh notation for the whole expression This also true for
the Ө and Ω notations
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/28
Asymptotic Notations
Using Order Theorem
Example: Consider the summation f(n) consisting of basic functions :
f(n) = n +√n + n1.5+ lg n + n lg n n2
We have seen that
lg n < √n < n <n lg n <n1.5 < n2
which means that the function n2 grows faster than all other functions in the expression
Thus,
f(n) = n + √n + n1.5+ lg n + n lg n + n2
= O( max (n , √n , n1.5, lg n , n lg n , n2 ) )
= O(n2)
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/29
Asymptotic Analysis-2
Asymptotic Analysis-2
Topics
 Asymptotic notation using Limits
 Definitions of O, Ω,Ө, o,ω notations
 Analysis of Summations
Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/14
Asymptotic Analysis
Using Limits
Use of basic definition for determining the asymptotic behavior is often awkward. It
involves ad hoc approach or some kind of manipulation to prove algebraic relations.
Calculus provides an alternative method for the analysis. It depends on evaluating
the following limit.
f(n)
lim
= c
n→∞ g(n)
where f(n) is a growth function for an algorithm and g(n) is a standard function
Depending upon the value c , the relation between f(n) and g(n) can be expressed in terms
of asymptotic notations. In most cases it is easier to use limits, compared to basic method, to
determine asymptotic behavior of growth functions.
 It will be seen that the Calculus notation n→ ∞ is equivalent to the algebraic condition
for all n ≥ n0 . Either of these conditions implies large input
O-Notation Using Limit
Definition
If f(n) is running time of an algorithm and g(n) is some standard growth function such
that
f(n)
lim —— = c ,
n → ∞ g(n)
where c is a positive constant such that 0 ≤ c < ∞
then f(n) = O ( g(n) )
 Note that infinity is excluded from the range of permissible values for the constant c
O-Notation
Examples
Example(1): 3n2 + 5n+ 20 = O(n2)
3n2 + 5n + 20
lim
n2
n→∞
=3+ 5 / n +20 / n2
= 3+0+0 =3 ( positive constant)
Therefore, 3n2 + 5n+ 20 = O(n2)
Example(2): 10n2 + 25n+ 7 = O(n3)
10n2 + 25n + 7
lim
n→∞
n3
=10 / n+ 25 / n2 + 7 / n3
=0+0+0 = 0
Therefore, 10n2 + 25n+ 7 = O(n3)
O-Notation
Examples
Example(3): lg n =
O(n)
lg n
lim
=
n→∞ n
∞/∞
( Need to apply the L’ Hopital Rule )
In order to compute differential of lg n we first convert binary logarithm to natural logarithm.
Converting lg n (binary log ) to ln( n) ( natural log) , by the using formula lg n = ln n / ln 2
lg n
(ln n)
lim
=
n→∞ n
(ln 2) n
1
lim
n → ∞ ln 2. n
(Converting to natural log)
( Differentiating numerator and denominator)
= 0 ( Evaluating limits)
Therefore, lg n =
O(n)
O-Notation
Example(4):
n2 = O(2n)
Examples
n2
lim
=∞ / ∞ ( Need to apply the L’ Hopital Rule )
n → ∞ 2n
Since d ( 2n ) = ln 2 . 2n,
dn
lim
n→∞
2n
( Calculus rule for differentiating the exponential functions)
(Differentiating numerator and denominator )
ln 2 2n
= ∞/∞
2
lim
n → ∞ (ln2)2. 2n
=0
Therefore, n2 = O(2n)
(Need again to apply the L’Hopital rule)
(Again differentiating numerator and denominator )
(Evaluating the limits )