The Growth of Functions:
Selected Exercises
Goals
• Introduce big-O & big-Omega
• Show how to estimate the size of functions
using this notation.
Preface
You may use without proof that: The functions below
increase asymptotically from top to bottom:
• f( n ) = k, for some constant k.
• f( n ) = logk n, for all k  N & any constant log base ≥ 2.
• f( n ) = nq, for all q  Q+
• f( n ) = kn < (k+1)n, for all k  N
• f( n ) = n!
Copyright © Peter Cappello
2
Preface continued
• The book says that f(n) is O( g( n ) ) when
k c n ( n > k  | f( n ) |  c | g( n ) | )
• In computational complexity, we deal exclusively with
functions whose domains & ranges are positive.
• We thus may simplify the definition of O( ) as follows:
k c n > k f( n )  c g( n ).
• You are not responsible for knowing o().
This is different from O().
Copyright © Peter Cappello
3
Exercise 10
Defn: f( n ) is O( g( n ) ) if k c n > k f( n )  cg( n ).
Show that:
1) n3 is O( n4 )
2) n4 is not O( n3 ).
Copyright © Peter Cappello
4
Exercise10: Solution
Defn: f( n ) is O( g( n ) ) when k c n > k f( n )  cg( n ).
Show that:
1) n3 is O( n4 )
2) n4 is not O( n3 ).
1)
For n ≥ 1 & c = 1: n3  1n4  1  n.
2)
Proof (by contradiction)
1)
Assume k c n > k n4  cn3.
2)
k c n > k n4  cn3 
k c n > k n  c ), which is false. ( Divide both sides by n3 )
3) Therefore, n4 is not O( n3 ).
Copyright © Peter Cappello
5
Theorems You Can Use
Thm 1: Let f(x) = anxn + an-1xn-1 + … + a1x + a0,
where the ai are real. Then, f(x) is O( xn ).
Let
f1(x) is O (g1(x) )
& f2(x) is O( g2(x) ).
Thm 2: (f1 + f2)(x) is O( max( g1(x), g2(x) ) )
Thm 3: (f1 f2)(x) is O( (g1g2)(x) ).
Copyright © Peter Cappello
6
Exercise 20
Give a big-O estimate for the functions:
(Use a simple g of smallest order.)
a) f( n ) = ( n3 + n2logn )( logn + 1 ) + ( 17logn + 19 )( n3 + 2 ).
Copyright © Peter Cappello
7
Exercise 20 a) Solution
Give a big-O estimate for the functions:
(Use a simple g of smallest order.)
a) f( n ) = ( n3 + n2logn )( logn + 1 ) + ( 17logn + 19 )( n3 + 2 ).
Using our theorems,
( n3 + n2logn )( logn + 1 ) + ( 17logn + 19 )( n3 + 2 )
Is O( ( n3 )( logn ) + ( 17logn )( n3 ) )
Is O( ( n3 logn ) + ( n3 17logn )
Is O( ( n3 logn ).
Copyright © Peter Cappello
8
Exercise 20 b)
b) f( n ) = ( 2n + n2 )( n3 + 3n ).
Copyright © Peter Cappello
9
Exercise 20 b) Solution
b) f( n ) = ( 2n + n2 )( n3 + 3n ).
Using our theorems,
f( n ) = ( 2n + n2 )( n3 + 3n ) is O( ( 2n )( 3n ) )
which is O( 2n3n ) which is O( 6n ).
Copyright © Peter Cappello
10
Exercise 20 c)
c) f( n ) = ( nn + n2n + 5n )( n! + 5n )
Copyright © Peter Cappello
11
Exercise 20 c) Solution
Defn: f( n ) is O( g( n ) ) when k c n > k f( n )  cg( n ).
c) f( n ) = ( nn + n2n + 5n )( n! + 5n )
Using our theorems, f( n ) is O( ( nn + n2n )( n! ) )
In nn + n2n, which is the fastest growing term?
Claim: n2n is O ( nn ) :
1.
n ≥ 2  2n-1  nn-1.
2.
n ≥ 2  n2n  2nn. (Multiply both sides of 1. by 2n.)
Thus, f( n ) is O( nnn! ).
Copyright © Peter Cappello
12
Exercise 30
Defn: f( n ) is O( g( n ) ) when k c n > k f( n )  cg( n ).
Defn. f( n ) is Ω( g( n ) ) when k c > 0 n > k f( n ) ≥ cg( n ).
What does it mean for f( n ) to be Ω( 1 )?
Hint: graph f( n ).
Copyright © Peter Cappello
13
Exercise 30 Solution
Defn. f( n ) is Ω( g( n ) ) when k c > 0 n > k f( n ) ≥ c g( n ).
What does it mean for a function to be Ω( 1 )?
• From the definition, f( n ) is Ω( 1 ) when
k c > 0 n > k f( n ) ≥ c.
• f( n ) ≥ c > 0, for sufficiently large n.
f( n ) = 1/n is Ω( 1 ). True or false?
Copyright © Peter Cappello
14
Generalizing the definitions
Defn: f( n ) is O( g( n ) ) when k c n > k f( n )  cg( n ).
What is a good definition of
f( n, m ) is O( g( n, m ) )?
Copyright © Peter Cappello
15
Time Complexity of Bubble Sort
void bubblesort( int[] a )
{
for ( int i = 0; i < a.length – 1; i++ )
for ( int j = 0; j < a.length – 1 – i; j++
)
{
if ( a[ j ] < a[ j + 1 ] )
{
int temp = a[ j ];
a[ j ] = a[ j + 1 ];
a[ j + 1 ] = temp;
}
}
}
• Let a.length = n.
• What is the total # of
comparisons as a function of n?
• This number is O( ? )
Copyright © Peter Cappello
16
End 3.2
Copyright © Peter Cappello
17
40
Defn. f( n ) is Θ( g( n ) ) when
f( n ) is O( g ( n ) ) and f( n ) is Ω( g( n ) ).
Show that:
if f1( x ) & f2( x ) are functions from Z+ to R and f1( x )
is Θ( g1( x ) ) and f2( x ) is Θ( g2( x ) ),
then f1f2 ( x ) is Θ( g1g2( x ) ).
Copyright © Peter Cappello 2011
18
40 Proof
1. Assume f1( x ) & f2( x ) are functions from Z+ to R and
f1( x ) is Θ( g1( x ) ) and f2( x ) is Θ( g2( x ) ).
2. f1( x ) is O( g1( x ) ).
3.
k1, C1, x > k1  f1( x )  C1g1( x )
4. f2( x ) is O( g2( x ) ).
5.
k2, C2, x > k2  f2( x )  C2g2( x )
(1. and defn of Θ)
(2.,Defn of O)
(1. and defn of Θ)
(4., Defn of O)
6. x > max{ k1, k2 }  f1 f2( x )  C1C2g1g2( x )
7. f1 f2( x ) is O( g1g2( x ) ).
Copyright © Peter Cappello 2011
(6., Defn of O)
19
40 Proof continued
1.
f1( x ) is Ω( g1( x ) ).
(Previous 1. & defn of Θ)
2.
k’1, C’1, x > k’1  f1( x ) ≥ C’1g1( x )
(1. & Defn of Ω)
3.
f2( x ) is Ω( g2( x ) ).
(Previous 1. & defn of Θ)
4.
k’2, C’2, x > k’2  f2( x ) ≥ C’2g2( x )
(3. & Defn of Ω)
5.
x > max{ k’1, k’2 }  f1 f2( x ) ≥ C’1C’2g1g2( x )
6.
f1 f2( x ) is Ω( g1g2( x ) ).
(5. & Defn of Ω)
7.
f1 f2( x ) is Θ( g1g2( x ) ).
(6., previous 7., defn Θ)
Copyright © Peter Cappello 2011
20
50
Show that ┌ xy ┐ is Ω(xy).
Proof:
1.
┌
xy ┐ ≥ xy.
(Defn of ceiling)
2. Let c = 1.
3. For x > 0, y > 0, ┌ xy ┐ ≥ cxy.
4. Therefore,┌ xy ┐ is Ω(xy).
Copyright © Peter Cappello 2011
(Defn of Ω)
21