Algorithm Analysis
Lectures 3 & 4
Resources
Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2
Introduction to Algorithms (Cormen, Leiserson, & Rivest): Chap.1
Algorithms Theory & Practice (Brassard & Bratley): Chap. 1
Algorithms
• An algorithm is a well-defined computational
procedure that takes some value or a set of
values, as input and produces some value, or
a set of values as output.
• Or, an algorithm is a well-specified set of
instructions to be solve a problem.
Efficiency of Algorithms
• Empirical
– Programming competing algorithms and trying
them on different instances
• Theoretical
– Determining mathematically the quantity of
resources (execution time, memory space, etc)
needed by each algorithm
Analyzing Algorithms
• Predicting the resources that the algorithm
requires:
• Computational running time
• Memory usage
• Communication bandwidth
• The running time of an algorithm
• Number of primitive operations on a particular input size
• Depends on
– Input size (e.g. 60 elements vs. 70000)
– The input itself ( partially sorted input for a sorting
algorithm)
Order of Growth
• The order (rate) of growth of a running time
– Ignore machine dependant constants
– Look at growth of T(n) as n
–  notation
• Drop low-order terms
• Ignore leading constants
• E.g.
– 3n3 + 90n2 – 2n +5 =  (n3)
Mathematical Background
Mathematical Background
• Definitions:
– T(N) = O(f(N)) iff  c and n0 
T(N)  c.f(N) when N  n0
– T(N) = (g(N)) iff  c and n0 
T(N)  c.g(N) when N  n0
– T(N) = (h(N)) iff T(N) = O(h(N)) and
T(N) = (h(N))
Mathematical Background
• Definitions:
– T(N) = o(f(N)) iff  c and n0 
T(N)  c.f(N) when N  n0
– T(N) = (g(N)) iff  c and n0 
T(N)  c.g(N) when N  n0
Mathematical Background
• Rules:
– If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
a) T1(N) + T2(N) = max( O(f(N)),O(g(N))
b) T1(N) * T2(N) = O(f(N) * g(N))
– If T(N) is a polynomial of degree k, then
T(N) = (Nk)
– Logk N = O(N) for any constant k.
More …
1.
2.
3.
4.
5.
6.
3n3 + 90n2 – 2n +5 = O(n3 )
2n2 + 3n +1000000 = (n2)
2n = o(n2) ( set membership)
3n2 = O(n2) tighter (n2)
n log n = O(n2)
True or false:
–
–
–
–
n2 = O(n3 )
n3 = O(n2)
2n+1= O(2n)
(n+1)! = O(n!)
Ranking by Order of Growth
1
n
(3/2)n
n log n
n2
nk
2n
(n)!
(n+1)!
Running time calculations
•
Rule 1 – For Loops
The running time of a for loop is at most the running
time of the statement inside the for loop
(including tests) times the number of iterations
•
Rule 2 – Nested Loops
Analyze these inside out. The total running time of a
statement inside a group of nested loops is the
running time of the statement multiplied by the
product of the sizes of all the loops
Running time calculations: Examples
Example 1:
sum = 0;
for (i=1; i <=n; i++)
sum += n;
Example 2:
sum = 0;
for (j=1; j<=n; j++)
for (i=1; i<=j; i++)
sum++;
for (k=0; k<n; k++)
A[k] = k;
How to rank?