Big-O
Algorithm Analysis
• Exact analysis: produce a function f(n)
measuring how many basic steps are needed for
a given inputs n On any input of size n, the
algorithm runs for at most 7n2 + 3n + 8 steps
• Do we really need such a precise
measurement?
• To compare two algorithm, we are concerned
with how well the algorithm scales,
• Too much details of f(n) is meaningless!
• (asymptotic) analysis
Asymptotic Analysis
• Asymptotic analysis of an algorithm describes
the relative efficiency of an algorithm when n
gets very large
•
Asymptotic Analysis (cont’d)
N
For large values of n, Algorithm 1 grows faster than Algorithm 2
Why does Complexity Matter?
• Assume that you have 3 algorithms to sort a list
– f(n) = nlog2n
– g(n) = n2
– h(n) = n3
• Assume that each step takes 1 microsecond
(i.e., 10-6 seconds)
Asymptotic Analysis (cont’d)
Bound
• f(n) is contributed from N^2
• The grow rate is bounded
• with such bound, we only need to refer to well
known functions and do not need to be
concerned with the details of the function
• Big-O notion: upper bound on the growth rate of
a function, for sufficiently large values of n.
Big-O Notation
• Definition:
f(n) = O(g(n)) if there exist positive constants c
and N such that f(n)  cg(n) for all n  N
• Big-O expresses an upper bound on the growth
rate of a function, for sufficiently large values of
n.
• Graph to understand Big-O
An Example
• Prove that 2n2 + 1 = O(n2)
Proof with Induction
Typical functions
Properties of Big-O Notation
• Fact 1 (Transitivity)
– If f(n)=O(g(n)) and g(n)=O(h(n)), then f(n)=O(h(n))
– Prove it (done in class)
• Fact 2
– If f(n)=O(h(n)) and g(n)=O(h(n)), then
f(n)+g(n)=O(h(n))
• Fact 3
– The function ank=O(nk)
• Fact 4
– The function nk=O(nk+j) for any positive j
Properties of Big-O Notation (cont’d)
• It follows from those facts that every polynomial
is big-O of n raised to the largest power
f(n) = aknk + ak-1nk-1 + . . . + a1n1 + a0 = O(nk)
Properties of Big-O Notation (cont’d)
• Fact 5
– If f(n)=cg(n), then f(n)=O(g(n))
• Fact 6
– The function logan = O(logbn) for any positive
numbers a and b  1
– WHY?
• Fact 7
– logan = O(lgn) for any positive a  1, where lg n =
log2n