Algorithm Analysis
Chris Kiekintveld
CS 2401 (Fall 2010)
Elementary Data Structures and Algorithms
Algorithm Analysis
 There are many different algorithms to solve the
same problem
 Ask 5 programmers to write a non-trivial program,
you will get 5 different solutions
 Which is best?
 Correctness
 Efficiency
Java Programming: Program Design Including Data Structures
2
Computational Resources
 Algorithms require resources to run




Time (processor operations)
Space (computer memory)
Network bandwidth
Programmer time
 Two types of costs
 Fixed: same every time we run the algorithm
 Variable: depends on the size of the input
Java Programming: Program Design Including Data Structures
3
Measuring Resource Use
 How can we compare the resources used by different
algorithms?
 Empirical
 Code both algorithms
 Run them an record the resources used
 You did this in the Fibonacci lab!
Java Programming: Program Design Including Data Structures
4
Empirical Analysis Problems
 Depends on code quality/implementation
 Better/worse programmers, not the algorithm itself
 Depends on computer speed/architecture
 Depends on language/compiler efficiency
 Depends on the input
 E.g. linear search is very fast for some inputs, and
very slow for others
Java Programming: Program Design Including Data Structures
5
Analytical Approach
 Analyze the algorithm itself
 Abstract away from implementation details
 How many operations will be executed?
 How much memory is used?
 Consider different cases (depending on input)
 Best
 Worst
 Average
Java Programming: Program Design Including Data Structures
6
Counting Operations
int i = 2;
int j = 2;
int k = i + j;
System.out.println(k);
How many operations are there?
Assignment:
Addition:
Print:
Total:
Java Programming: Program Design Including Data Structures
7
Counting Operations
int i = 2;
int j = 2;
int k = i + j;
System.out.println(i+j+k);
How many operations are there?
Assignment:
Addition:
Print:
Total:
Java Programming: Program Design Including Data Structures
8
Counting Operations
int i = 0;
while (i < 10) {
System.out.println(i);
i++;
}
How many operations are there?
Assignment:
Comparison:
Increment:
Print:
Total:
Java Programming: Program Design Including Data Structures
9
Counting Operations
for (int i=0; i < 10; i++) {
System.out.println(i);
}
How many operations are there?
Assignment:
Comparison:
Increment:
Print:
Total:
Java Programming: Program Design Including Data Structures
10
Counting Operations
for (int i=0; i < n; i++) {
System.out.println(i);
}
How many operations are there?
Assignment:
Comparison:
Increment:
Print:
Total:
Java Programming: Program Design Including Data Structures
11
Counting Operations
for (int i=0; i < n; i++) {
for (int j=0; j < n; j++) {
System.out.println(i);
}
}
How many operations are there?
Assignment:
Comparison:
Increment:
Print:
Total:
Java Programming: Program Design Including Data Structures
12
Counting Operations
 So far, we have counted every operation
 This is quite tedious, especially for infrequent
operations
 Focus on the most important operation
 Most frequent
 May need to figure out what this is
Java Programming: Program Design Including Data Structures
13
Another Look at Search Algorithms
 We have discussed two ways to search a list
 Linear search (unordered data)
 Binary search (sorted data)
 Data is sorted by “keys”
 Unique for each element
 Well-defined order
Java Programming: Program Design Including Data Structures
14
Linear (Sequential) Search
public int seqSearch(T[] list, int length, T searchItem)
{
int loc;
boolean found = false;
for (loc = 0; loc < length; loc++)
{
if (list[loc].equals(searchItem))
{
found = true;
break;
}
}
if (found)
return loc;
else
return -1;
}
Java Programming: Program Design Including Data Structures
15
Sequential Search Analysis
 The statements in the for loop are repeated several
times
 For each iteration of the loop, the search item is
compared with an element in the list
 When analyzing a search algorithm, you count the
number of comparisons
 Suppose that L is a list of length n
 The number of key comparisons depends on where
in the list the search item is located
Java Programming: Program Design Including Data Structures
16
Sequential Search Analysis
(continued)
 Best case
 The item is the first element of the list
 You make only one key comparison
 Worst case
 The item is the last element of the list
 You make n key comparisons
 What is the average case
Java Programming: Program Design Including Data Structures
17
Sequential Search Analysis
(continued)
 To determine the average case
 Consider all possible cases
 Find the number of comparisons for each case
 Add them and divide by the number of cases
 Average case
 On average, a successful sequential search searches
half the list
Java Programming: Program Design Including Data Structures
18
Binary Search
public int binarySearch(T[] list, int length, T searchItem)
{
int first = 0;
int last = length - 1;
int mid = -1;
boolean found = false;
while (first <= last && !found)
{
mid = (first + last) / 2;
Comparable<T> compElem = (Comparable<T>) list[mid];
Java Programming: Program Design Including Data Structures
19
Binary Search (continued)
if (compElem.compareTo(searchItem) == 0)
found = true;
else
if (compElem.compareTo(searchItem) > 0)
last = mid - 1;
else
first = mid + 1;
}
if (found)
return mid;
else
return -1;
}//end binarySearch
Java Programming: Program Design Including Data Structures
20
Binary Search Example
Figure 18-1 Sorted list for a binary search
Table 18-1 Values of first, last, and middle and the Number of
Comparisons for Search Item 89
Java Programming: Program Design Including Data Structures
21
Performance of Binary Search
 Suppose that L is a sorted list of size n
 And n is a power of 2 (n = 2m)
 After each iteration of the for loop, about half the
elements are left to search
 The maximum number of iteration of the for loop
is about m + 1
 Also m = log2n
 Each iteration makes two key comparisons
 Maximum number of comparisons: 2(m + 1)
Java Programming: Program Design Including Data Structures
22
Comparison: Linear vs Binary
Worst case number of comparison
List Size
Linear
Binary
4
4
6
8
8
8
32
32
12
512
512
20
1048576
1048576
42
Java Programming: Program Design Including Data Structures
23
Asymptotic Analysis: Motivation
 So far, we have counted operations exactly
 We don’t really care about the details
 Computers execute billions of operations per second
 A few here or there is negligible
 Care about overall scalability
 As the input size grows, does computation grow
quickly or slowly?
 Don’t lose the forest for the trees
Java Programming: Program Design Including Data Structures
24
Asymptotic Analysis
 Asymptotic means the study of the function f as n
becomes larger and larger without bound
 Consider functions g(n) = n2 and f(n) = n2 + 4n + 20
 As n becomes larger and larger, the term 4n + 20 in
f(n) becomes insignificant
 g(1000) = 1,000,000 and f(1000) = 1,004,020
 You can predict the behavior of f(n) by looking at the
behavior of g(n)
Java Programming: Program Design Including Data Structures
25
Asymptotic Algorithm Analysis
 Identify a function that describes the growth in
runtime as the input gets large
 An “upper bound” of sorts on the running time
 Typically worst-case, but occasionally average case
 Describe the number of operations done using a
function
 Focus only on most important operations
 Ignore one time initializations, etc.
Java Programming: Program Design Including Data Structures
26
Common Asymptotic Functions
Table 18-4 Growth Rate of Various Functions
Java Programming: Program Design Including Data Structures
27
Common Functions, visual
Figure 18-9 Growth Rate of Various Functions
Java Programming: Program Design Including Data Structures
28
Java Programming: Program Design Including Data Structures
29
Asymptotic Notation: Big-O
Notation (continued)
Table 18-7 Some Big-O Functions That Appear in Algorithm Analysis
Java Programming: Program Design Including Data Structures
30
Big-Oh Notation (Definition)
A function f(n) is O(g(n)) if there exist positive
constants c and n0 such that:
f(n) ≤ cg(n) for all n ≥ n0
Java Programming: Program Design Including Data Structures
31
Big-Oh Notation
 Translation: After some point, f(n) is always smaller
than g(n)
 “Some point” refers to increasing problem size
 The constant c says that we don’t care about
multipliers
 So, 2n and n have the same essential growth rate
 2n is O(n)
Java Programming: Program Design Including Data Structures
32
Asymptotic Notation: Big-O
Notation (continued)
Table 18-8 Number of Comparisons for a List of Length n
Java Programming: Program Design Including Data Structures
33