COP 3530 Spring2012
Data Structures & Algorithms
Discussion Session Week 5
Outline
• Growth of functions
– Big Oh
– Omega
– Theta
– Little Oh
Growth of Functions
Gives a simple view of the algorithm’s efficiency.
Allows us to compare the relative performance of
alternative algorithms.
f1(n) is O(n)
f2(n) is O(n^2)
Growth of Functions
Exact running time of an algorithm is usually hard to
compute, and it’s unnecessary.
For large enough inputs, the lower-order terms of
an exact running time are dominated by high-order
terms.
f(n) = n^2 + 5n + 234
n^2 >> 5n + 234, when n is large enough
Asymptotic Notation: Big Oh (O)
f(n)= O(g(n)) iff there exist positive constants c and n0 such that
f(n) ≤ cg(n) for all n ≥ n0
O-notation to give an upper bound on a function
Asymptotic Notation: Big Oh (O)
Example 1[linear function] f(n) = 3n+2
For n >= 2, 3n+2 <= 3n+n <= 4n. So f(n) = O(n).
We can arrive the same conclusion in other ways. For example,
3n + 2 <= 10n for n > 1.
The specific values of c and n0 used to satisfy the definition of big
oh are not important, we only say f(n) is big oh of g(n). c/n0 do
not matter.
Asymptotic Notation: Big Oh (O)
Example 2[quadric function] f(n) = 10n^2+4n+2
For n >= 2, f(n) <= 10n^2+5n.
For n>= 5, 5n < n^2.
Hence for n>= n0 = 5, f(n) <= 10n^2+n^2 = 11n^2. Therefore, f(n)
= O(n^2).
Only the highest order term matters !!!
Asymptotic Notation: Big Oh (O)
Example 3[exponential function] f(n) = 6*2^n + n^2
For n>=4, n^2 <= 2^n.
f(n) <= 6*2^n + 2^n = 7*2^n for n>=4
Therefore, 6*2^n + n^2 = O(2^n)
Example 4[constant function] f(n) = 3
For any n, f(n) <= 4*1
Therefore, f(n) = O(1)
Asymptotic Notation: Big Oh (O)
Example 5[loose bounds] f(n) = 3n+3
For n >= 10, 3n+3 <= 3n^2.
Therefore, f(n) = O(n^2).
Usually, we mean tight upper bound when using big oh notation.
Example 6[Incorrect bounds] 3n+2 != O(1)
since we cannot find n0/c such that 3n + 2 <= c, when n>=c0 (n
can be infinity).
Similarly, 10n^2 + 6n + 2 != O(n).
Asymptotic Notation: Big Oh (O)
The specific values of c and n0 used to satisfy the definition of big
oh are not important, we only say f(n) is big oh of g(n).
c/n0 do NOT matter, WHY?
f(n)=n. It’s close to 10n when comparing
with n^2, n^3
f(n) is relatively small when n<n0
Asymptotic Notation: Omega Notation
Big oh provides an asymptotic upper bound on a function.
Omega provides an asymptotic lower bound on a function.
Asymptotic Notation: Omega Notation
Example 7 f(n) = 3n+3 > 3n for all n. So f(n) = Omega(n)
Example 8[loose bounds] f(n) = 3n+3 > 1 for all n, so f(n) =
Omega(1)
Asymptotic Notation: Theta Notation
Theta notation is used when function f can be bounded both
from above and below by the same function g
Asymptotic Notation: Theta Notation
Example 9: f(n) = 3n+3 is Theta(n), since n <= 3n+3 <= 4n, when
n >= 3.
Similarly, f(n) = 3n+2 is Theta(n)
f(n) = 5n^2 - 10n + 9 is Theta(n^2)
f(n) is Theta(g(n)) iff f(n) is Omega(g(n)) and O(g(n))
Asymptotic Notation: Little oh (o)
The asymptotic upper bound provided by O-notation may or may
not be asymptotically tight. 2n = O(n) is tight, 2n = O(n^2) is not
tight.
We use o-notation to denote an upper bound that is NOT
asymptotically tight.
Asymptotic Notation: Little oh (o)
f(n) = o(g(n)) iff f(n) = O(g(n)) and f(n) != Omega(g(n))
Example 10 3n+2 = o(n^2) as 3n+2 = O(n^2) and 3n+2 !=
Omega(n^2)
Example 11 3n+2 != o(n) as 3n+2 = Omega(n)
Review
Big oh: upper bound on a function.
Omega: lower bound.
Theta: lower and upper bound.
- f(n) is Theta(g(n)) iff f(n) is O(g(n)) and Omega(g(n))
Little oh: loose upper bound.
- f(n) = o(g(n)) iff f(n) = O(g(n)) and f(n) != Omega(g(n))
Office Hour This Week:
Thursday 9th period at E309