Chapter 1
Analysis Basics
1
Chapter Outline
•
•
•
•
•
•
What is analysis?
What to count and consider
Mathematical background
Rates of growth
Tournament method
Analyzing programs
2
Prerequisites
• Before beginning this chapter, you should
be able to:
– Read and create algorithms
– Identify comparison and arithmetic operations
– Use basic algebra
3
Goals
• At the end of this chapter you should be
able to:
– Describe how to analyze an algorithm
– Explain how to choose the operations that are
counted and why others are not
– Explain how to do a best-case, worst-case,
and average case analysis
4
Goals (continued)
– Work with logarithms, probabilities, and
summations
– Describe (f), Ω(f), O(f), growth rate, and
algorithm order
– Use a decision tree to determine a lower bound
on complexity
5
Analyzing an Algorithm
• Show that it properly solves the problem
• Determine how efficient it solves the problem
– Approximate the number of operation
6
Algorithm Classification
• Iterative
– Based on loop and conditional statements
– Determine the work in the loop and the number of
times the loop executes
• Recursive
– Based on calls to the same subprogram with a
smaller problem
– Determine the number of smaller problems and
the amount of work done to produce them and
then put their solutions together
7
What is Analysis?
• An approximation of how long an algorithm
will take or how much extra space it needs
• Comparison of two or more algorithms that
solve the same problem
• Independent of the computer because a
faster computer does not make an algorithm
more efficient
• Independent of initializations because those
may be done faster
8
Input Classes
• Groups of input sets that cause the same
amount of work to be done
• For example, finding the largest value
– Input classes could be based on the location of
the largest element
9
Space Complexity
• The amount of space needed for an algorithm
to complete its task
• Algorithms are classified as either in place or
needing extra space
– Depends on whether the algorithm moves data
around within its current storage or copies
information to a new space while it’s working
• Use of extra space can be critical in software
designed for small embedded systems
10
What to Count
• Comparisons
– Equal, greater, not equal, …
• Arithmetic
– Additions
• add, subtract, increment, decrement
– Multiplications
• multiply, divide, modulus, remainder
11
Cases to Consider
• Best Case
– The least amount of work done for any input set
• Worst Case
– The most amount of work done for any input set
• Average Case
– The amount of work done averaged over all of the
possible input sets
12
Average Case
• Determining the average case:
– Find the number of input set classes
– Find the probability that the input will be from each
of these classes
– Find the amount of work done for each class
• The average case is given by:
m
A (n)   pi * ti
i 1
13
Mathematical Background
• The floor of X is the largest integer  to X
• The ceiling of X is the smallest integer  to X
• The logarithm (base Y) of X is the value Z
that satisfies the equation X = YZ
14
Binary Trees
• A binary tree with N nodes has at least
lg N+1 levels
• There are 2K-1 nodes on level K
• In a complete binary tree all of the nodes on
levels 1 to J-1 have two children, so all of the
leaves are on level J
• A complete binary tree with J levels has
2J-1 nodes
15
Probabilities
• A probability is a number between 0 and 1
• If something will never occur, it has a
probability of 0
• If something always occurs, it has a
probability of 1
• If there are N equally likely events, each one
has a probability of 1/N
16
Summations
• A summation is a compact way to express the
sum of a series of values
• The sum of the numbers from 1 to 7 would be
written:
7
i
i1
• There are closed forms for many summations
given in the text
17
Rate of Growth
• The rate of growth of a function determines how fast
the function value increases when the input increases
18
Rate of Growth
• If algorithm A does x3 operations on an input of size
x and algorithm B does x2 operations, algorithm B is
more efficient
• Because of the relative growth rate, we will consider
x3 + x2 + x equivalent to x3
19
Classification of Growth
• Big Omega Ω(f)
– The class of functions that grow at least as fast as
the function f, and maybe faster
• Big Oh O(f)
– The class of functions that grow no faster than f,
and maybe slower
• Big Theta Θ(f)
– The class of functions that grow at the same rate
as the function f
20
Tournament Method
• Finding the second largest element
–
–
–
–
Build a binary tree with the values in the leaves
The larger of a pair is copied into the parent node
The root has the largest value
The second largest value must have lost to the largest
21
Decision Trees
• Each choice is represented by a
different child
• Path from the root to a leaf indicates
how that solution is found
• Different algorithms that solve a
problem will produce different decision
trees
22
Decision Tree Example
23
Lower Bounds
• The absolute smallest number of operations
needed to solve a problem
• Based on the problem not a particular
algorithm
• Any algorithm that claims to solve the
problem faster must fail
24
Lower Bounds Example
• For example, sorting a list
– Each leaf holds one of the N! possible initial
orderings of elements
– Each internal node indicates an element
comparison
– If the tree is packed as tightly as possible, the
number of levels indicate the lower bound
– Thus, the lower bound for sorting is N lg N
25
Analyzing Programs
• Program profilers are available that will
indicate where the program is spending its
execution time
• Can be done by hand by:
– Incrementing a unique counter every time each
subprogram is called
– Incrementing a unique counter every time a
particular code segment is reached
26