Growth of Functions
(Big-O notation)
Reading: sections 3.9
1 / 11
log n,
n,
n log n,
n ,
n ,
2
Overview
he rates 1.
of growth
for these standard functions are indicated in Fig. 3-7, which gives their appro
tain values ofIn. Observe that the functions are listed in the order of their rates of growth: th
Study a way to describe the growth of functions in the limit –
on log2 n grows most slowly, the exponential function 2n grows most rapidly, and the polynom
asymptotic efficiency
w according to the exponent c.
Example. Rates of growth of standard functions
3-7 important
Rate of growth
of standard
I Focus onFig.
what’s
(leading
factor) byfunctions
abstracting
lower-order terms and constant factors
I A way
he way we compare
our complexity
with one of the standard functions is to use
to comparefunction
“sizes” fof(n)
functions
” notation which we formally define below.
tion 3.4: Let f (x) and g(x) be arbitrary functions defined on R or a subset of R. We say “f (
written
f (x) = O(g(x))
2 / 11
2. Big-O notation
3.1 Asymptotic notation
Definition. Let f (x) and g(x) be
functions defined on the a subset
c2 g.n/
of real numbers. If there exists
a
real number k and a positive
f .n/
constant C such that
|f (x)| ≤ C|g(x)|
whenever
f (x) =
x > k,
f .n/ D ‚.g.n//
O(g(x)).
(a)
f .n/
c1 g.n/
then we say “f (x) is big-O of
g(x)”, written
n0
cg.n/
n
n0
n
f .n/ D O.g.n//
(b)
Figure 3.1 Graphic examples of the ‚, O, and notations. In ea
is the minimum possible value; any greater value would also work.
tion to within constant factors. We write f .n/ D ‚.g.n// if there
and c2 such that at and to the right of n0 , the value of f .n/ always
3 / 11 l
3. Example: 2x + 10 = O(x2 ).
Since
i.e., C = 1 and k = 5.
2x + 10 ≤ x2
for x ≥ 5,
Note: k and C are not unique, we can also pick k and C as
follows:
2x + 10 ≤ 2x2 + 10x2 = 12x2 for x ≥ 1,
i.e., k = 1 and C = 12.
More examples:
I
I
I
I
f (x) = x2 + 2x + 1 = O(x2 )
1 + 2 + 3 + · · · + n = O(n2 ).
f (n) = log n! = O(n log n).
f (n) = n! = O(nn )
4 / 11
4. Theorem: Let f (x) = an xn + an−1 xn−1 + · · · + a1 x + a0 ,
where an , . . . , a0 are real numbers. Then f (x) = O(xn ).
Proof: If x > 1,
|f (x)| ≤ |an |xn + |an−1 |xn−1 + · · · + |a1 |x + |a0 |
≤ (|an | + |an−1 | + · · · + |a1 | + |a0 |)xn ≡ Cxn .
Examples.
I 5x3 + 3x = O(x3 )
I x5 − 4000000x2 + 50000x = O(x5 )
5 / 11
5. Theorem: If f1 (x) = O(g1 (x)) and f2 (x) = O(g2 (x)), then
I (f1 + f2 )(x) = O(max(|g1 (x)|, |g2 (x)|)).
I (f1 f2 )(x) = O(g1 (x)g2 (x)).
Example: give a big O-notation estimate for
f (n) = 3n log(n!) + (n2 + 3) log n,
where n is a positive integer.
6 / 11
2
according to the exponent c.
6. Rates of growth of standard functions
Fig. 3-7
Rate of growth of standard functions
way we compare our complexity function f (n) with one of the standard functions is to use the
notation which we formally define below.
n 3.4: Let f (x) and g(x) be arbitrary functions defined on R or a subset of R. We say “f (x)
ritten
f (x) = O(g(x))
xists a real number k and a positive constant C such that, for all x > k, we have
|f (x)| ≤ C|g(x)|
7 / 11
her words, f (x) = 0(g(x)) if a constant multiple of |g(x)| exceeds |f (x)| for all x greater
7. An ordering of the growth of commonly used functions in
computer science, namely if f (n) comes before g(n) in the
order then f (n) = O(g(n)).
Example. Ordering of standard functions:
log n,
n,
n log n,
nk ,
2n
8 / 11
8. Big-Ω notation
Definition. Let f (x) and g(x) be functions defined on the set
(or subset) of real numbers. If there exists a real number k and
a positive constant C such that
|f (x)| ≥ C|g(x)|
whenever
x > k,
then we say “f (x) is big-Omega of g(x)”, written
f (x) = Ω(g(x)).
Example: f (x) = 8x3 + 5x2 + 6 is Ω(x3 ).
9 / 11
9. Big-Θ notation
Definition. Let f (x) and g(x) be functions defined on the set
(or subset) of real numbers.
f (x) = Θ(g(x))
if and only if
f (x) = O(g(x)) and f (x) = Ω(g(x)).
reads: “f (x) is big-Theta of g(x)”, or f (x) is of (same) order
g(x).
Example: 1 + 2 + · · · + n is of order n2 , i.e.,
1 + 2 + · · · + n = Θ(n2 ).
10 / 11
10. Theorem. Let f (x) = an xn + an−1 xn−1 + · · · + a1 x + a0 ,
where an , . . . , a0 are real numbers and an 6= 0. Then f (x) is of
order xn , i.e. f (x) = Θ(xn ).
11 / 11