Algorithms &
Complexity
Dr. Ranette Halverson
CMPS 2433 – Discrete Systems & Analysis
Chapter 1: Introduction, 1.4
1
Solving a Problem
• Existence of a Solution
• How many Solutions
• Optimal Solution
• Sample problems on page 1 of book.
2
Algorithm
• Step-by-Step instructions to
accomplish a task or solve a problem
• Computer Program
• Assembly instructions
• Driving Directions
• Recipe
• Format & Detail level depends on user
3
Efficiency of Algorithms
• Different algorithms for a single task
may have different efficiency
• Driving Directions – short route vs. long
route
• Searching Algorithms
• Sorting Algorithms
• Must be able to evaluate efficiency
(speed) of computer algorithms
4
Comparing Algorithms
• Want to choose the “best” algorithm
for tasks
• Generally, best = fastest
• But there are other considerations
• Hardware (e.g. LaTex)
• Size of data set
• Data Structure
• Need a standard measurement
5
Complexity = Speed = Efficiency
• NOT Really – but sort of
• *Complexity is the number of basic
operations required by an algorithm
• Will 2 algorithms with same complexity
take the same actual amount of time to
run??
• Why or Why Not?
* Means memorize
6
Complexity Examples
Read x, y, z
x = y + z
Print x
How many operations?
Read x, y, z
x = y / z
Print x
Are these algorithms
the same complexity?
The same speed?
7
Complexity Examples
Read x, y, z
x = y + z
Print x
Do 10 times
Read x, y, z
x = y + z
Print x
8
Complexity Examples
Do 10 times
Read x, y, z
x = y + z
Print x
Do n times
Read x, y, z
x = y + z
Print x
9
Big Oh Notation
O(f(n)) – “Big Oh of f of n”
n represents size of data set
Examples
• O(1) or O(c)
• O(n)
• O(n2)
Big Oh is an upper bound.
10
Constant Complexity
• An algorithm with the same number
of operations regardless of the data
set is said to have CONSTANT
COMPLEXITY
• O(1) or 0(c)
• See previous algorithms
• Most significant algorithms are NOT
constant complexity
11
Complexity Examples
Do 10 times
Read x, y, z
x = y + z
Print x
Do n times
Read x, y, z
x = y + z
Print x
Complexity?
O(c) or O(1) - constant
Complexity?
Depends on value of n!
NOT Constant
6 * n basic operations
O(6n)  O(n)
12
Algorithm Complexity
• Number of operations in terms of the
data set size
Do 10 times
Read x, y, z
x = y + z
Print x
Constant Time Complexity
– O(1) or O(c)
13
Algorithm Complexity
Do n times
Read x, y, z
x = y + z
Print x
Time Complexity is dependent upon n
6n Operations - O(6n)  O(n)
14
*Big Oh (yes, memorize)
O(f(n)) – Big Oh of f of n
A function g(n) = O(f(n)) if there
exist 2 constants K and n0 such that
|g(n)| <= K|f(n)| for all n >=n0
Big Oh is an upper bound.
15
Evaluating xn
• Operations???
P = x
k = 1
While
P
k
Print
k < n
= P * x
= k + 1
P
16
Evaluating xn
P = x
k = 1
While
P
k
Print
k < n
= P * x
= k + 1
P
• Operations
1
1
n
2(n-1)
2(n-1)
1
Total =3 + n + 2(n-1) + 2(n-1)
3 + n + 2n -2 + 2n – 2
5n – 1 = O(5n -1)  O(n)
17
Trace: xn  54 
P = x
k = 1
While
P
k
Print
k < n
= P * x
= k + 1
P
(n=4, x=5)
P = 5, k = 1
P = 5*5 (25); k= 2
P = 25*5 (125); k = 3
P = 125*5 (625); k=4
Print 625
18
Complexity & Rate of Growth
• Complexity measures growth rate of
algorithm time in terms of data size
• O(1)
Constant
• O(n)
Linear
• O(n2)
Polynomial (any constant power)
• O(5n)
Exponential (any constant base)
What do these functions look like when
graphed?
How big are real world data sets? (name some)
19
Polynomial Evaluation
P(x) = anxn + an-1xn-1 + … +a1x + a0
Given values for x, a0, a1,…an
How many ops for anxn ?
Operations to calculate each term in poly?
n + n-1 + n-2 +…+ 1 + 0 = (n*(n+1))/2
Additions to combine the n+1 terms = n
Total Ops = (n2+n)/2 + n  O(n2)
What did we omit from analysis???
20
Polynomial Evaluation (p.26)
P(x) = anxn + an-1xn-1 + … +a1x + a0
Given values for
x, a0, a1,…an
S = a0 , k = 1
while k <=1
S = S + ak x k
k = k+ 1
Print S
Can you spot any
inefficiencies?
21
Polynomial Evaluation Algorithm
(p.26) P(x) = anxn + an-1xn-1 + … +a1x + a0
Given values for
n, x, a0, a1,…an
S = a0 , k = 1
while k <=n
S = S + ak x k
k = k+ 1
Print S
2
n+1
#n(2+5(k+1)+1)
2n
1
*Slide 17: xk = 5k + 1
#Total = 2n +5kn + 5n + n
5kn + 8n
22
Complexity of Poly. Evaluation
Total = (by line on previous slide)
=2 + (n + 1) + (5kn + 8n) + 2n + 1
= 5kn + 11n + 4
Worst case for k??
= 5n2 + 11 n + 4 O(n2)
Note: this is for inserting code for xk
Function call adds additional overhead.
23
Analysis of Polynomial
Compare Slides 20 and 22/23
• Same Big Oh  O(n2)
• “Different details” but same results
24
Horner’s Rule (Polynomial Evaluation)
P(x) = anxn + an-1xn-1 + … +a1x + a0
S = an , k = 1
While k <= n
S = Sx + an-k
k=k=1
Print s
2
n+1
n*3
n *2
1
Total = 2 + n+1 + 3n + n2 + 1 = 6n + 4
 O(n)
25
Summary & Homework
• See table on page 31
• In general, an algorithm is considered “good”
if its complexity is no more than some
polynomial in n
• For small n, some non-polynomial algorithms
may be “acceptable”
Homework: Page 33+
1 – 10, 23 – 26 – will discuss in class
27 – 30 (detail for each step as on slides, state
Total & Big Oh) – Turn in 27-30 for grading
26