Analysys & Complexity of
Algorithms
Big Oh Notation
Complexity
• Two kinds of complexity
– Space complexity
• Effects of data type choices on size of data
• Effects of amount of data (input and variables)
– Time Complexity
• Effects of choices of program design
• Given 2 algorithms, which is "the best"
2
Big Oh (or Big O)
• A measure of complexity related to "n"
(problem size=amount of data) e.g.:
– # of records to process
– #of files
– # of numbers or times through a loop
• This is asymptotic analysis
– associates n, the problem size, with t, the processing
time required to solve the problem
3
Big O Examples
• x=x+1;
– O(1) ->
constant degradation
• Binary Search of a SORTED set of n elements
– O(log n)
• for (i=1;i<n;i++)
– O(n) ->
linear degradation
• selection sort, compare two 2D arrays, find duplicates in an
UNSORTED list
– O(n2) ->
quadratic degradation
• Generate all premutations of n symbols
– O(an) ->
'a' is some constant independtent of 'n'
• O(l) < O(log n) < O(n) < O(n log n) < O(n2) < O(an)
4
3 cases
• Best case
– minimum path lengths
• Average case
– constant path length
• Worst case
– maximum path length
– most useful!! Leads to better design
– answers question: will it be good enough
tomorrow??
5
Frequency Counting
• Examine a piece of code and predict the
number of instructions to be executed
• e.g. predict how many times (max) each
statement will run.
Inst # Code
1
2
for (int i=0; i< n ; i++)
{ printf ("%d",i);
3
p = p + i;
}
Freq
count
n+1
n
n
3n+1
6
Order of magnitude
• In the previous example:
– best case = average case = worst case
– Example is based on iteration limit: n
• To convert Frequency Count to order of magnitude:
– pick the most significant term if polynomial
– discard constant terms (like the +1)
– disregard coefficients (like the 3)
– yields worst case path through algorithm
• Big O (represented as O(n))
• O(n) for the previous example
7
Common growth rates
8
Big Oh - Formal Definition
• f(n)=O(g(n)),
iff
ᴲ
{c, n0 | f(n) <= c  g(n)
for all
n >= n0}
• Thus, g(n) is an upper bound on f(n)
• Note:
f(n) = O(g(n))
– f(n) has a complexity of g(n)"
this is NOT the same as
O(g(n)) = f(n)
• The '=' is not the usual mathematical "=" operator (it is not
reflexive)
• We will see more about this in chapter 6
9