CSCE 310J: Data Structures &
Algorithms
Fundamentals of Algorithm Analysis
Dr. Steve Goddard
goddard@cse.unl.edu
http://www.cse.unl.edu/~goddard/Courses/CSCE310J
Most of the slides in this lecture
were originally created by
Jeff Edmonds
from York University
Relevant Mathematics
Classifying Functions
The Time Complexity of an Algorithm
Adding Made Easy
Recurrence Relations
Jeff Edmonds
York University
Start With Some Math
Time Complexity
t(n) = Q(n2)
Time
Classifying Functions
f(i) = nQ(n)
Input Size
Adding Made Easy
∑i=1 f(i).
Recurrence Relations
T(n) = a T(n/b) + f(n)
Assumed Knowledge
• Logarithms and Exponentials
• Existential and Universal Quantifiers,
g "b Loves(b,g)
"b g Loves(b,g)
Quantifiers: all, some
• “for all x” "x P(x) is true iff P(x) is true for all x
– universal quantifier (universe of discourse)
• “there exist x” x P(x) is true iff P(x) is true for
some value of x
– existential quantifier
• "x A(x) is logically equivalent to  x(A(x))
• x A(x) is logically equivalent to "x(A(x))
• "x (A(x)  B(x))
“For all x such that if A(x) holds then B(x) holds”
Classifying Functions
Giving an idea of how fast a function grows
without going into too much detail.
Which are more alike?
Which are more alike?
Mammals
Which are more alike?
Which are more alike?
Dogs
Classifying Animals
Vertebrates
Fish
Reptiles
Mammals
Birds
Giraffe
Dogs
Classifying Functions
T(n) 10
100
1,000 10,000
6
9
13
amoeba
10
31
100
bird
n 10
100
1,000 10,000
n log n 30
600
9,000 130,000 my father
log n
3
n1/2 3
10,000 106
n2 100
109
n3 1,000 106
2n 1,024 1030
10300
108
1012
103000
human
elephant
dinosaur
the universe
Note: The universe contains approximately 1050 particles.
Which are more alike?
n1000
n2
2n
Which are more alike?
n1000
n2
Polynomials
2n
Which are more alike?
1000n2
3n2
2n3
Which are more alike?
1000n2
Quadratic
3n2
2n3
Classifying Functions?
Functions
Classifying Functions
Functions
Exp
Double Exp
25n
Polynomial
Poly Logarithmic
Logarithmic
Constant
2
2
25n
n5
(log n)5
5 log n
5
Exponential
n5
Classifying Functions?
Polynomial
Classifying Functions
Polynomial
Linear
Quadratic
Cubic
?
5n
5n2
5n3
5n4
Logarithmic
• log10n = # digits to write n
• log2n = # bits to write n
= 3.32 log10n
• log(n1000) = 1000 log(n)
Differ only by a
multiplicative
constant.
Poly Logarithmic
(log n)5 = log5 n
Logarithmic << Polynomial
log1000 n << n0.001
For sufficiently large n
Linear << Quadratic
10000 n << 0.0001 n2
For sufficiently large n
Polynomial << Exponential
n1000 << 20.001 n
For sufficiently large n
Ordering Functions
Functions
25n << 2
n5
<<
Double Exp
<<
<<
Exp
5 << 5 log n << (log n)5 << n5
<<
Exponential
<<
Polynomial
<<
Poly Logarithmic
Logarithmic
Constant
<<
25n
<<
2
Which Functions are Constant?
Yes • 5
Yes • 1,000,000,000,000
Yes • 0.0000000000001
Yes • -5
Yes • 0
No • 8 + sin(n)
Which Functions are “Constant”?
The running time of the algorithm is a “Constant”
It does not depend significantly
on the size of the input.
Yes
Yes
Yes
No
No
Yes
•5
• 1,000,000,000,000
9
• 0.0000000000001
7
• -5
• 0
Lie in between
• 8 + sin(n)
Which Functions are Quadratic?
• n2
•…?
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
Some constant times n2.
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Which Functions are Quadratic?
• n2
• 0.001 n2
Lie
in
between
2
• 1000 n
• 5n2 + 3n + 2log n
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Ignore low-order terms
Ignore multiplicative constants.
Ignore "small" values of n.
Write θ(n2).
Which Functions are
Polynomial?
• n5
•…?
Which Functions are
Polynomial?
• nc
• n0.0001
• n10000
n to some constant power.
Which Functions are
Polynomial?
• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n
• 5n2 log n
• 5n2.5
Which Functions are
Polynomial?
• nc
• n0.0001
Lie
in
between
10000
•n
• 5n2 + 8n + 2log n
• 5n2 log n
• 5n2.5
Which Functions are
Polynomials?
• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n
• 5n2 log n
• 5n2.5
Ignore low-order terms
Ignore power constant.
Ignore "small" values of n.
Write nθ(1)
Which Functions are
Exponential?
• 2n
•…?
Which Functions are
Exponential?
• 2n
• 20.0001 n
• 210000 n
n times some constant power
raises to the power of 2.
Which Functions are
Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 too small?
•2n · n100 too big?
Which Functions are
Exponential?
• 2n
• 20.0001 n
• 210000 n
= 23n
• 8n
0.5n
n
100
>
2
•2 /n
2n
n
100
<
2
•2 · n
Lie in between
Which Functions are
Exponential?
• 2n
• 20.0001 n
• 210000 n
= 23n
• 8n
0.5n
n
100
>
2
•2 /n
2n
n
100
<
2
•2 · n
20.5n > n100
2n = 20.5n · 20.5n > n100 · 20.5n
2n / n100 > n0.5n
Which Functions are
Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100
•2n · n100
Ignore low-order terms
Ignore base.
Ignore "small" values of n.
Ignore polynomial factors.
Write 2θ(n)
Classifying Functions
Functions
2
Double Exp
2θ(n)
Exp
nθ(1)
Exponential
(log n)θ(1)
Polynomial
θ(log n)
Poly Logarithmic
Logarithmic
Constant
θ(1)
nθ(1)
2θ(n)
2
Notations
Theta
f(n) = θ(g(n))
f(n) ≈ c g(n)
BigOh
f(n) = O(g(n))
f(n) ≤ c g(n)
Omega
f(n) = Ω(g(n))
f(n) ≥ c g(n)
Little Oh
f(n) = o(g(n)) f(n) << c g(n)
Little Omega
f(n) = ω(g(n)) f(n) >> c g(n)
Definition of Theta
f(n) = θ(g(n))
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
Definition of Theta
f(n) = θ(g(n))
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
f(n) is sandwiched between c1g(n) and c2g(n)
Definition of Theta
f(n) = θ(g(n))
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
f(n) is sandwiched between c1g(n) and c2g(n)
for some sufficiently small c1 (= 0.0001)
for some sufficiently large c (= 1000)
Definition of Theta
f(n) = θ(g(n))
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
For all sufficiently large n
Definition of Theta
f(n) = θ(g(n))
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
For all sufficiently large n
For some definition of “sufficiently large”
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
? ?
c1·n2  3n2 + 7n + 8  c2·n2
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
3 4
3·n2  3n2 + 7n + 8  4·n2
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
1
3·12  3·12 + 7·1 + 8  4·12
False
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
7
3·72  3·72 + 7·7 + 8  4·72
False
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
8
3·82  3·82 + 7·8 + 8  4·82
True
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
9
3·92  3·92 + 7·9 + 8  4·92
True
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
10
3·102  3·102 + 7·10 + 8  4·102
True
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
?
3·n2  3n2 + 7n + 8  4·n2
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
8 n8
3·n2  3n2 + 7n + 8  4·n2
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
n8
3·n2  3n2 + 7n + 8  4·n2
True
Definition of Theta
3n2 + 7n + 8 = θ(n2)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
n8
3·n2  3n2 + 7n + 8  4·n2
7n + 8  1·n2
True 7 + 8/n  1·n
Definition of Theta
3n2 + 7n + 8 = θ(n2)
True
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
3 4
8
n  8 3·n2  3n2 + 7n + 8  4·n2
Definition of Theta
n2 = θ(n3)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
Definition of Theta
n2 = θ(n3)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
? ?
c1 · n3  n2  c2 · n3
Definition of Theta
n2 = θ(n3)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
0
0 · n3  n2  c2 · n3
True, but ?
Definition of Theta
n2 = θ(n3)
False
 c1  0, c2  0,  n0, " n  n0,...
0
Constants c1 and c2 must be positive!
Definition of Theta
3n2 + 7n + 8 = θ(n)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
Definition of Theta
3n2 + 7n + 8 = θ(n)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
? ?
c1·n  3n2 + 7n + 8  c2·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
3 100
3·n  3n2 + 7n + 8  100·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
?
3·n  3n2 + 7n + 8  100·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
100
3·100  3·1002 + 7·100 + 8 
100·100
False
Definition of Theta
3n2 + 7n + 8 = θ(n)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
3 10,000
3·n  3n2 + 7n + 8  10,000·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
10,000
3·10,000  3·10,000 2 + 7·10,000 + 8  100·10,000
False
Definition of Theta
3n2 + 7n + 8 = θ(n)
What is the
reverse statement?
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
Understand Quantifiers!!!
" b, Loves(b, MM)
" b, Loves(b, MM)]
 b, Loves(b, MM)]
 b, Loves(b, MM)
Sam
Mary
Sam
Mary
Bob
Beth
Bob
Beth
John
Marilin
Monro
John
Marilin
Monro
Fred
Ann
Fred
Ann
Definition of Theta
3n2 + 7n + 8 = θ(n)
The reverse statement
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
Definition of Theta
3n2 + 7n + 8 = θ(n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
3 100
?
3n2 + 7n + 8 > 100·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
3 100
100
3·1002 + 7·100 + 8 > 100·100
True
Definition of Theta
3n2 + 7n + 8 = θ(n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
3 10,000
?
3n2 + 7n + 8 > 10,000·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
3 10,000 10,000
3·10,0002 + 7·10,000 + 8 >
10,000·10,000
True
Definition of Theta
3n2 + 7n + 8 = θ(n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
c1 c2
?
3n2 + 7n + 8 > c2·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
c1 c2
c2
3·c22 + 7 · c2 + 8 > c2·c2
True
Definition of Theta
3n2 + 7n + 8 = θ(n)
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
c1 c2 n 0 ?
3n2 + 7n + 8 > c2·n
Definition of Theta
3n2 + 7n + 8 = θ(n)
True
" c1, c2, " n0,  n  n0, c1 g( n)  f ( n) or f ( n)  c2 g( n)
c1 c2 n0 max(c2,n0 )
3·c22 + 7 · c2 + 8 > c2·c2
True
Definition of Theta
3n2 + 7n + 8 = g(n)θ(1)
3n2 + 7n + 8 = nθ(1)
 c1, c2, n0, " n  n0 g(n)c1  f(n)  g(n)c2
Definition of Theta
3n2 + 7n + 8 = nθ(1)
 c1, c2, n0, " n  n0 g(n)c1  f(n)  g(n)c2
? ?
nc1  3n2 + 7n + 8  nc2
Definition of Theta
3n2 + 7n + 8 = nθ(1)
 c1, c2, n0, " n  n0 g(n)c1  f(n)  g(n)c2
2 3
n2  3n2 + 7n + 8  n3
Definition of Theta
3n2 + 7n + 8 = nθ(1)
 c1, c2, n0, " n  n0 g(n)c1  f(n)  g(n)c2
2 3
?
n?
n2  3n2 + 7n + 8  n3
Definition of Theta
3n2 + 7n + 8 = nθ(1)
 c1, c2, n0, " n  n0 g(n)c1  f(n)  g(n)c2
2 3
5
n5
n2  3n2 + 7n + 8  n3
Definition of Theta
3n2 + 7n + 8 = nθ(1)
True
 c1, c2, n0, " n  n0 g(n)c1  f(n)  g(n)c2
2 3
5
n5
n2  3n2 + 7n + 8  n3
True
Order of Quantifiers
f(n) = θ(g(n))
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
 n0, " n  n0,  c1, c2, c1 g( n)  f ( n )  c2 g( n)
?
Understand Quantifiers!!!
 g, " b, loves( b, g )
One girl
" b,  g, loves( b, g )
Could be a separate
girl for each boy.
Sam
Mary
Sam
Mary
Bob
Beth
Bob
Beth
John
Marilin
Monro
John
Marilin
Monro
Fred
Ann
Fred
Ann
Order of Quantifiers
f(n) = θ(g(n))
 c1, c2,  n0, " n  n0, c1 g( n)  f ( n )  c2 g( n)
 n0, " n  n0,  c1, c2, c1 g( n)  f ( n )  c2 g( n)
No! It cannot be a different c1 and c2
for each n.
Other Notations
Theta
f(n) = θ(g(n))
f(n) ≈ c g(n)
BigOh
f(n) = O(g(n))
f(n) ≤ c g(n)
Omega
f(n) = Ω(g(n))
f(n) ≥ c g(n)
Little Oh
f(n) = o(g(n)) f(n) << c g(n)
Little Omega
f(n) = ω(g(n)) f(n) >> c g(n)
BigOh Notation
• n2 = O(n3)
• n3 = O(n2)
BigOh Notation
• O(n2) = O(n3)
• O(n3) = O(n2)
Odd Notation
f(n) = O(g(n))
Standard
f(n) ≤ O(g(n))
Stresses one function dominating another.
f(n)O(g(n))
Stress function is member of class.
The Time Complexity of an
Algorithm
Specifies how the running time
depends on the size of the input.
Purpose?
Purpose
• To estimate how long a program will run.
• To estimate the largest input that can reasonably
be given to the program.
• To compare the efficiency of different algorithms.
• To help focus on the parts of code that are
executed the largest number of times.
• To choose an algorithm for an application.
Time Complexity Is a Function
Specifies how the running time depends on
the size of the input.
A function mapping
“size” of input
“time” T(n) executed .
Definition of Time?
Definition of Time
• # of seconds (machine dependent).
• # lines of code executed.
• # of times a specific operation is performed
(e.g., addition).
Which?
Definition of Time
• # of seconds (machine dependent).
• # lines of code executed.
• # of times a specific operation is performed
(e.g., addition).
These are all reasonable definitions of time,
because they are within a constant factor of
each other.
Size of Input Instance?
83920
Size of Input Instance
5
83920
1’’
2’’
•
•
•
•
Size of paper
# of bits
# of digits
Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
Which are reasonable?
Size of Input Instance
5
83920
1’’
2’’
•
•
•
•
Size of paper
# of bits
# of digits
Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
• Intuitive
Size of Input Instance
5
83920
1’’
2’’
•
•
•
•
Size of paper
# of bits
# of digits
Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
• Intuitive
• Formal
Size of Input Instance
5
83920
1’’
2’’
•
•
•
•
Size of paper
# of bits
# of digits
Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
• Intuitive
• Formal
• Reasonable
# of bits =
3.32 * # of digits
Size of Input Instance
5
83920
1’’
2’’
•
•
•
•
Size of paper
# of bits
# of digits
Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
•
•
•
•
Intuitive
Formal
Reasonable
Unreasonable
# of bits = log2(Value)
Value = 2# of bits
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Time?
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
Time = N
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Time = N
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
Time = N
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Is this reasonable?
Time = N
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
Time = N
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Time = N
No!
One is considered fast and the other slow!
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Size of Input Instance?
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
size n = N
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
size n = log N
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
size n = N
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
size n = log N
Time as function of input size?
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
size n = N
Time = N = n
size n = log N
Time = N = 2n
Two Example Algorithms
• Sum N array entries:
A(1) + A(2) + A(3) + …
• Factor value N:
Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
size n = N
Time = N = n
size n = log N
Time = N = 2n
Linear vs Exponential Time!
Size of Input Instance?
14,23,25,30,31,52,62,79,88,98
Size of Input Instance
10
14,23,25,30,31,52,62,79,88,98
• # of elements - n = 10 elements
Size of Input Instance
10
14,23,25,30,31,52,62,79,88,98
• # of elements - n = 10 elements
Is this reasonable?
Size of Input Instance
10
14,23,25,30,31,52,62,79,88,98
• # of elements - n = 10 elements
~ Reasonable
If each element has size c
# of bits = c * # of elements
Size of Input Instance
10
14,23,25,30,31,52,62,79,88,98
• # of elements - n = 10 elements
Reasonable
If each element is in [1..n]
each element has size log n
# of bits = n log n ≈ n
Time Complexity Is a Function
Specifies how the running time depends on
the size of the input.
A function mapping
# of bits n needed to represent the input
# of operations T(n) executed .
Which Input of size n?
There are 2n inputs of size n.
Which do we consider
for the time T(n)?
Which Input of size n?
Typical Input
Average Case
Worst Case
Which Input of size n?
Typical Input
But what is typical?
Average Case
For what distribution?
Worst Case
•Time for all inputs is
bound.
•Easiest to compute
Time Complexity of Algorithm
The time complexity of an algorithm is
the largest time required on any input
of size n.
O(n2): Prove that for every input of size n,
the algorithm takes no more than cn2 time.
Ω(n2): Find one input of size n, for which the
algorithm takes at least this much time.
θ (n2): Do both.
Time Complexity of Problem
The time complexity of a problem is
the time complexity of the fastest algorithm
that solves the problem.
O(n2): Provide an algorithm that solves the
problem in no more than this time.
Ω(n2): Prove that no algorithm can solve it faster.
θ (n2): Do both.
Adding
The Classic Techniques
n
Evaluating ∑i=1 f(i).
Arithmetic Sum
Gauss
∑i=1..n i = 1 + 2 + 3 + . . . + n
= ?
1
+
2
+
n
+
n-1 +
3
+
...
+
n-1 +
n
=
S
n-2 +
...
+
2
1
=
S
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
+
=
1
+
2
+
n
+
n-1 +
3
+
...
+
n-1 +
n
=
S
n-2 +
...
+
2
1
=
S
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
+
=
1
+
2
+
n
+
n-1 +
3
+
...
+
n-1 +
n
=
S
n-2 +
...
+
2
1
=
S
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
+
=
1
+
2
+
n
+
n-1 +
3
+
...
+
n-1 +
n
=
S
n-2 +
...
+
2
1
=
S
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
n (n  1)
S
2
+
=
1
+
2
+
n
+
n-1 +
3
+
...
+
n-1 +
n
=
S
n-2 +
...
+
2
1
=
S
+
Algebraic argument
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
=
Let’s restate this argument
using a geometric
representation
1
+
2
+
n
+
n-1 +
3
+
...
+
n-1 +
n
=
S
n-2 +
...
+
2
1
=
S
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
+
=
= number of white dots.
1
2........n
+
...
+
n-1 +
n
=
S
= number of white dots
n-2 +
...
+
2
1
=
S
= number of yellow dots
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
1
+
2
+
n
+
n-1 +
3
n ....... 2 1
+
=
1
2........n
+
...
+
n-1 +
n
=
S
= number of white dots
n-2 +
...
+
2
1
=
S
= number of yellow dots
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
1
+
2
+
n
+
n-1 +
3
+
=
n
n
There are n(n+1)
dots in the grid
n
n
n
n
n+1 n+1 n+1
n+1 n+1
+
...
+
n-1 +
n
=
S
= number of white dots
n-2 +
...
+
2
1
=
S
= number of yellow dots
(n+1) + (n+1) + (n+1) +
...
+ (n+1) + (n+1) =
2S
n (n+1)
2S
1
+
2
+
n
+
n-1 +
3
+
=
n
n (n  1)
S
2
n
n
n
n
n
n+1 n+1 n+1
Note = Q(# of terms · last term))
n+1 n+1
Arithmetic Sum
Gauss
∑i=1..n i = 1 + 2 + 3 + . . . + n
= Q(# of terms · last term)
True when ever terms increase slowly
Geometric Sum
∑i=0..n
2i = 1 + 2 + 4 + 8 +. . . + 2n
=
?
Geometric Sum
Geometric Sum
∑i=0..n
2i = 1 + 2 + 4 + 8 +. . . + 2n
= 2 · last term - 1
Geometric Sum
∑i=0..n
ri = r0 + r1 + r2 +. . . + rn
=
?
Geometric Sum
S  1  r  r  r ... r
2
Sr 
3
n
n 1
r  r  r ... r  r
2
3
n
n 1
S - Sr  1
-r
n 1
-1
S
r -1
r
Geometric Sum
∑i=0..n
n 1
1
ri 
r -1
r
When r>1?
Geometric Sum
∑i=0..n
n 1
1
ri 
 θ(rn)
r -1
r
When r>1
Biggest Term
Geometric Increasing
∑i=0..n
=
ri = r0 + r1 + r2 +. . . + rn
Q(biggest term)
True when ever terms increase quickly
Geometric Sum
∑i=0..n
ri
1- r

1-r
n 1
When r<1?
Geometric Sum
∑i=0..n
ri
1- r

1-r
n 1
When r<1
 θ(1)
Biggest Term
Bounded Tail
∑i=0..n
=
ri = r0 + r1 + r2 +. . . + rn
Q(1)
True when ever terms decrease quickly
Harmonic Sum
∑i=1..n 1/i
= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n
= ?
Sum of Shrinking Function
f(i) = 1
∑i=1..n f(i) = n
n
Sum of Shrinking Function
f(i) = 1/2i
∑i=1..n f(i) = 2

Sum of Shrinking Function
f(i) = 1/i
∑i=1..n f(i) = ?
n
Harmonic Sum
Harmonic Sum
∑i=1..n 1/i
= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n
= Q(log(n))
Approximating Sum
by Integrating
∑i=1..n f(i) ≈ ∫x=1..n f(x) dx
The area under the curve
approximates the sum
Approximating Sum
Arithmetic Sums
by Integrating
∑i=1..n ic ≈ ∫x=1..n xc dx = 1/c+1 nc+1 = θ(nc+1)
 θ(# of terms · last term)
True when ever terms increase slowly
Approximating Sum
Geometric Sums
by Integrating
∑i=1..n bi ≈ ∫x=1..n bx dx = 1/ln(b) bn = θ(bn)
 θ(last term)
True when ever terms increase quickly
Approximating Sum
by Integrating
Harmonic Sum
Approximating Sum
by Integrating
Problem: Integrating may be hard too.
Outline
I) What is the sum
Eg. ∑i=1..n f(i) = ∑i=1..n 1/i0.9999 =θ(n0.0001)
II) Math proofs of sums
I flash it up
If you follow great.
If not don’t panic.
III) Pattern of sums
∑i=1..n f(i) = θ(n · f(n))
IV) Intuition why this is the sum
V) Ability to know the answer fast
Adding Made Easy
We will now classify (most) functions f(i)
into four classes:
–Geometric Like
–Arithmetic Like
–Harmonic
–Bounded Tail
Adding Made Easy
We will now classify (most) functions f(i)
into four classes
For each class, we will give
an easy rule for approximating
it’s sum
θ( ∑i=1..n f(i) )
Classifying Animals
Vertebrates
Fish
Reptiles
Mammals Giraffe
Dogs
Birds
Mammal  Complex social
networks
Dog
 Man’s best friend
Classifying Functions
Functions
Exp.
Poly.
nθ(1) 2θ(n)
f(n) = 8·2n / n100 + n3
2θ(n)  ∑i=1..n f(i) = θ(f(n))
θ(2n / n100)
 ∑i=1..n f(i) = θ(2n / n100)
θ(2n / n100)
θ(2n)
8·2n / n100 + n3 8·2n + n3
Irrelevant
20.5n < 8· 2n / n100 + n3 < 2n
Significant
Less significant
Adding Made Easy
Four Classes of Functions
1
1
2
4
3
Adding Made Easy
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
If the terms f(i) grow sufficiently
quickly, then the sum will be
dominated by the largest term.
Silly example:
1+2+3+4+5+ 1,000,000,000 ≈ 1,000,000,000
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
If the terms f(i) grow sufficiently
quickly, then the sum will be
dominated by the largest term.
Classic example:
?
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
If the terms f(i) grow sufficiently
quickly, then the sum will be
dominated by the largest term.
Classic example:
∑i=1..n 2i = 2n+1-1 ≈ 2 f(n)
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
If the terms f(i) grow sufficiently
quickly, then the sum will be
dominated by the largest term.
For which functions f(i) is this true?
How fast and how slow can it grow?
Geometric
Like:
Ω(n)
f(i)  2
Upper Extreme:
 ∑i=1..n f(i) = θ(f(n))
∑i=1..n (1000)i ≈ 1.001(1000)n
= 1.001 f(n)
Recall, when f(n) = ri = 2θ(n)
2
3
n
i





 1 r
r
r
... r
∑i=1..n r
n 1
-1
r
n


θ(
) Last Term
r
r
1
 θ(f(n)) when r>1.
i ≈ 10,000 (1.0001)n
∑
(1.0001)
Lower Extreme: i=1..n
= 10,000 f(n)
Geometric
Like:
Ω(n)
f(i)  2
Upper Extreme:
 ∑i=1..n f(i) = θ(f(n))
∑i=1..n (1000)i ≈ 1.001(1000)n
= 1.001 f(n)
Because the constant is sooo big,
the statement is just barely true.
i ≈ 10,000 (1.0001)n
∑
(1.0001)
Lower Extreme: i=1..n
= 10,000 f(n)
Geometric
Like:
Ω(n)
f(i)  2
Upper Extreme:
 ∑i=1..n f(i) = θ(f(n))
∑i=1..n (1000)i ≈ 1.001(1000)n
= 1.001 f(n)
Even bigger?
i ≈ 10,000 (1.0001)n
∑
(1.0001)
Lower Extreme: i=1..n
= 10,000 f(n)
Geometric
Like:
Ω(n)
f(i)  2
No Upper Extreme:
 ∑i=1..n f(i) = θ(f(n))
∑i=1..n
i
2
22
≈
n
2
22 =
1f(n)
Even bigger?
i ≈ 10,000 (1.0001)n
∑
(1.0001)
Lower Extreme: i=1..n
= 10,000 f(n)
Geometric
Like:
Ω(n)
f(i)  2
No Upper Extreme:
 ∑i=1..n f(i) = θ(f(n))
∑i=1..n
i
2
22
≈
n
2
22 =
1f(n)
Functions in between?
i ≈ 10,000 (1.0001)n
∑
(1.0001)
Lower Extreme: i=1..n
= 10,000 f(n)
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
f(n) = 8·2n + n3 = θ( 2n )
n100
n100
∑i=1..n 8·2i + i3
i100
= 8·21 + 8·22 + 8· 23 + 8· 24 +…+ 8· 2n
100
1100 2100 3100 4100
n
3
3
3
3
3
+
n
+
4
+1 +2 +3
= θ( 2n )
= θ(f(n))
=
≈ 8·[2·2n –2]
100
100
n
n
+ n3
Dominated by the largest term.
Geometric
Like:
Ω(n)
f(i)  2
f(n) = 8·2n
n100
f(i+1)
f(i)
=
 ∑i=1..n f(i) = θ(f(n))
Easy
Hard
f(n)  ∑i=1..n f(i)  c·f(n)
f(i)  ?·f(n)
f(i)  ?·f(i
+1+1
) )
.51·f(i
8·2(i+1)
(i+1)100 8·2i
i100
=
=
100
i
2
(i+1)100
1
2 (1+1/ )100  1.9  1/.51
i
Geometric
Like:
Ω(n)
f(i)  2
f(n) = 8·2n
n100
 ∑i=1..n f(i) = θ(f(n))
f(n)  ∑i=1..n f(i)  c·f(n)
(n-i)·f(n)
f(i)  .51
?·f(n)
f(i)  .51·f(i+1)
f(i)  .51·f(i+1)  .512·f(i+2)  .513·f(i+3)
...  .51(n-i)·f(i+(n-i))  .51(n-i)·f(n)
Eg. i = n, f(n)  .51(n-n)·f(n)  1·f(n)
i = 0, f(0)  .51n·f(n)
Geometric
Like:
Ω(n)
f(i)  2
f(n) = 8·2n
n100
 ∑i=1..n f(i) = θ(f(n))
f(n)  ∑i=1..n f(i)  c·f(n)
(n-i)·f(n)
f(i)  .51
?·f(n)
f(i)  .51·f(i+1)
∑i=1..n f(i)  ∑i=1..n .51(n-i)·f(n)
= [∑i=1..n .51(n-i)] · f(n)
= θ(1) · f(n)
Geometric
Like:
Ω(n)
f(i)  2
Functions
Exp.
Poly.
nθ(1) 2θ(n)
 ∑i=1..n f(i) = θ(f(n))
f(n) = 8·2n / n100 + n3
2θ(n)  ∑i=1..n f(i) = θ(f(n))
θ(2n / n100)
 ∑i=1..n f(i) = θ(2n / n100)
θ(2n / n100)
θ(2n)
8·2n / n100 + n3 8·2n + n3
Irrelevant
20.5n < 8· 2n / n100 + n3 < 2n
Significant
Less significant
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
If f(n) = c ban nd loge (n)
c >?0
a  ?0
b  ?1
d  ?(-,)
e  ?(-,)
f(n) =Ω,
θ?
2Ω(n)
then ∑i=1..n f(i) = θ(f(n)).
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
All functions in 2Ω(n)?
Maybe not.
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
Functions that oscillate continually
do not have the property.
Geometric
Like:
Ω(n)
f(i)  2
 ∑i=1..n f(i) = θ(f(n))
Functions expressed with +, -, ,  , exp, log
do not oscillate continually.
They are well behaved for sufficiently large n.
These do have the property.
Adding Made Easy
Done
Adding Made Easy
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
If most of the terms f(i) have roughly the
same value, then the sum is roughly the
number of terms, n, times this value.
Silly example:
1,001 + 1,002 + 1,003 + 1,004 + 1,005
≈ 5 · 1,000
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
If most of the terms f(i) have roughly the
same value, then the sum is roughly the
number of terms, n, times this value.
Another silly example:
∑i=1..n 1 = n · 1
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
If most of the terms f(i) have roughly the
same value, then the sum is roughly the
number of terms, n, times this value.
Is the statement true for this function?
∑i=1..n i = 1 + 2 + 3 + . . . + n
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
If most of the terms f(i) have roughly the
same value, then the sum is roughly the
number of terms, n, times this value.
Is the statement true for this function?
∑i=1..n i = 1 + 2 + 3 + . . . + n
The terms are not roughly the same.
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
∑i=1..n i =
1 + . . . + n/2 + . . . + n
But half the terms are roughly the same.
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
∑i=1..n i =
1 + . . . + n/2 + . . . + n
But half the terms are roughly the same
and the sum is roughly the number
terms, n, times this value
∑i=1..n i = θ(n · n)
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Is the statement true for this function?
∑i=1..n i2 = 12 + 22 + 32 + . . . + n2
Even though, the terms
are not roughly the same.
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
∑i=1..n i =
12 + . . . + (n/2)2 + . . . + n2
1/
4
n2
Again half the terms
are roughly the same.
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
∑i=1..n i =
12 + . . . + (n/2)2 + . . . + n2
1/
4
n2
Again half the terms
are roughly the same.
∑i=1..n
2
i
= θ(n ·
2
n)
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
∑i=1..n f(i)
≈ area under curve
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
area of small square
 ∑i=1..n f(i)
≈ area under curve
 area of big square
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
n/
2
· f(n/2)
= area of small square
 ∑i=1..n f(i)
≈ area under curve
 area of big square
= n · f(n)
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
θ(n · f(n))
= n/2 · f(n/2)
= area of small square
 ∑i=1..n f(i)
≈ area under curve
 area of big square
= n · f(n)
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
θ(n · f(n))
?
n
n
= /2 · f( /2)
= area of small square
 ∑i=1..n f(i)
≈ area under curve
 area of big square
= n · f(n)
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
f(i) = n2
∑i=1..n i2 = 12 + 22 + 32 + . . . + n2
=?
The key property is
f(n/2) = θ(f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
f(i) = n2
∑i=1..n i2 = 12 + 22 + 32 + . . . + n2

= θ(n·f(n)) = θ(n · n2)
The key property is
f(n/2) = (n/2)2 = 1/4 n2 = θ(n2) = θ(f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
f(i) = nθ(1)
∑i=1..n f(i) = ?
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
f(i) = nθ(1)
∑i=1..n ir = 1r + 2r + 3r + . . . + nr
The key property is
f(n/2) = θ(f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
f(i) = nθ(1)
∑i=1..n ir = 1r + 2r + 3r + . . . + nr

= θ(n·f(n)) = θ(n · nr)
The key property is
f(n/2) = (n/2)r = 1/2r nr = θ(nr) = θ(f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
f(i) = nθ(1)
∑i=1..n 2i = 21 + 22 + 23 + . . . + 2n
The key property is
f(n/2) = θ(f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
f(i) = nθ(1)
∑i=1..n 2i = 21 + 22 + 23 + . . . + 2n

= θ(n·f(n)) = θ(n · 2n)
The key property is
f(n/2) = 2(n/2) = θ(2n) = θ(f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Upper Extreme: ∑i=1..n i1000 = 1/1001 n1001
= 1/1001 n · f(n)
All functions
in between.
Middle Extreme: ∑i=1..n 1 = n · 1
= n · f(n)
Adding Made Easy
Half done
Sum of Shrinking Function
f(i) = 1
∑i=1..n f(i) = n
n
Sum of Shrinking Function
f(i) = ?1/i1/2
∑i=1..n f(i) = n1/2
n
Sum of Shrinking Function
f(i) = 1/i
∑i=1..n f(i) = log n
n
Sum of Shrinking Function
f(i) = 1/2i
∑i=1..n f(i) = 2

Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
If most of the terms f(i) have roughly the
same value, then the sum is roughly the
number of terms, n, times this value.
Does the statement hold
for functions f(i) that shrink?
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
If most of the terms f(i) have roughly the
same value, then the sum is roughly the
number of terms, n, times this value.
Does the statement hold
for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i) = ?
θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i) = ∑i=1..n 1/i = ?
θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i) = ∑i=1..n 1/i = θ(log n)
θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i) = ∑i=1..n 1/i = θ(log n)
? = θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i) = ∑i=1..n 1/i = θ(log n)
θ(1) = θ(n · 1/n) = θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i) = ∑i=1..n 1/i = θ(log n)
≠
θ(1) = θ(n · 1/n) = θ(n · f(n))
No the statement does not hold!
Adding Made Easy
not included
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
Shrinks slightly slower
than harmonic.
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i) = ∑i=1..n 1/i0.9999 = ?
= θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i) = ∑i=1..n 1/i0.9999 = θ(n0.0001)
= θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i) = ∑i=1..n 1/i0.9999 = θ(n0.0001)
? = θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i) = ∑i=1..n 1/i0.9999 = θ(n0.0001)
θ(n0.0001) = θ(n · 1/n0.9999) = θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Does the statement hold
for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i) = ∑i=1..n 1/i0.9999 = θ(n0.0001)
=
θ(n0.0001) = θ(n · 1/n0.9999) = θ(n · f(n))
The statement does hold!
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Middle Extreme: ∑i=1..n 1 = n · 1
= n · f(n)
All functions
in between.
1/ 0.9999 = θ(n0.0001)
∑
Lower Extreme: i=1..n i
= θ(n · f(n))
Arithmetic
Like:
θ(1)-1
f(i) = n
 ∑i=1..n f(i) = θ(n·f(n))
Upper Extreme: ∑i=1..n i1000 = 1/1001 n1001
= 1/1001 n · f(n)
Middle Extreme: ∑i=1..n 1 = n · 1
= n · f(n)
1/ 0.9999 = θ(n0.0001)
∑
Lower Extreme: i=1..n i
= θ(n · f(n))
Arithmetic Like
Conclusion
If f(n) = c ban nd loge (n)
c > ?0
a  ?0
or bb1?
d
?
> -1
e  ?(-,)
n-1+θ(1)
f(n) = Ω,
θ?
then ∑i=1..n f(i) = θ(n · f(n)).
(For +, -, ,  , exp, log functions f(n))
Adding Made Easy
Done
Adding Made Easy
Harmonic
Harmonic Sum
∑i=1..n 1/i
= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n
= Q(log(n))
Adding Made Easy
f(n)  n
Bounded
Tail:
-1-Ω(1)
 ∑i=1..n f(i) = θ(1)
If the terms f(i) decrease towards zero
sufficiently quickly,
then the sum will be a constant.
The classic example
∑i=0..n 1/2i = 1 + 1/2 + 1/4 + 1/8 + … = 2.
f(n)  n
Bounded
Tail:
-1-Ω(1)
 ∑i=1..n f(i) = θ(1)
If f(i) decays even faster, then the
tail of the sum is more bounded.
1
∑i=1..n
2i
22
= θ(1).
f(n)  n
Bounded
Tail:
-1-Ω(1)
 ∑i=1..n f(i) = θ(1)
1/ 1.0001 = θ(1)
∑
Upper Extreme:
i=1..n i
f(n)  n
Bounded
Tail:
-1-Ω(1)
 ∑i=1..n f(i) = θ(1)
1/ 1.0001 = θ(1)
∑
Upper Extreme:
i=1..n i
All functions
in between.
1
No Lower Extreme:
∑i=1..n
2i
22
= θ(1).
Bounded
Tail
Conclusion
c nd loge (n)
If f(n) = c ban nbdanloge (n)
c >0?0
a <0?0
or bb 
< 11?
d  (-,)
?
e  ?(-,)
f(n) = Ω, θ ?
then ∑i=1..n f(i) = θ(1).
Bounded Tail
Conclusion
If f(n) = c nd loge (n)
c > ?0
a0
b=1
d
?
< -1
e  (-,)
?
nθ(1)-1
f(n) = Ω,
θ?
then ∑i=1..n f(i) = θ(1).
Adding Made Easy
Done
Adding Made Easy
Missing Functions
n
n
n log n
logn/
n
1/
nlogn
Adding Made Easy
• Geometric Like: If f(n)  2Ω(n),
then ∑i=1..n f(i) = θ(f(n)).
• Arithmetic Like: If f(n) = nθ(1)-1, then ∑i=1..n f(i) = θ(n · f(n)).
• Harmonic:
If f(n) = 1/n ,
then ∑i=1..n f(i) = logen + θ(1).
• Bounded Tail:
If f(n)  n-1-Ω(1), then ∑i=1..n f(i) = θ(1).
(For +, -, ,  , exp, log
functions f(n))
This may seem confusing, but it is really not.
It should help you compute most sums easily.
Recurrence Relations
T(1) = 1
T(n) = a T(n/b) + f(n)
Recurrence Relations
 Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Recurrence relations
arise from the timing
of recursive programs.
Let T(n) be the # of “Hi”s
on an input of “size” n.
Recurrence Relations
 Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Given size 1,
the program outputs
T(1)=1 Hi’s.
Recurrence Relations
 Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Given size n,
the stackframe outputs
f(n) Hi’s.
Recurrence Relations
 Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Recursing
on a instances of size n/b
generates T(n/b) “Hi”s.
Recurrence Relations
 Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Recursing
a times
generates a·T(n/b) “Hi”s.
Recurrence Relations
 Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
For a total of
T(1) = 1
T(n) = a·T(n/b) + f(n)
“Hi”s.
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac =
bd =
(a+b)(c+d) =
XY =
Stack Frame: A particular
execution of one routine on one
particular input instance.
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac =
bd =
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = ?
bd =
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = ?
bd =
(a+b)(c+d) =
XY =
X=2
Y=2
XY=
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = ?
bd =
(a+b)(c+d) =
XY =
X=2
Y=2
XY = 4
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd =
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd = ?
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd = ?
(a+b)(c+d) =
XY =
X=1
Y=3
XY = 3
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd = 3
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd = 3
(a+b)(c+d) = ?
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd = 3
(a+b)(c+d) = ?
XY =
X=3
Y=5
XY = 15
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd = 3
(a+b)(c+d) = 15
XY = ?
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = ?
bd =
(a+b)(c+d) =
XY =
X = 21
Y = 23
ac = 4
bd = 3
(a+b)(c+d) = 15
XY = 483
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd =
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd = ?
(a+b)(c+d) =
XY =
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd = ?
(a+b)(c+d) =
XY =
X = 33
Y = 12
ac = ?
bd =
(a+b)(c+d) =
XY = 15
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd = ?
(a+b)(c+d) =
XY =
X = 33
Y = 12
ac = ?
bd =
(a+b)(c+d) =
XY = 15
X=3
Y=1
XY = 3
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd = ?
(a+b)(c+d) =
XY =
X = 33
Y = 12
ac = 3
bd = ?
(a+b)(c+d) =
XY = 15
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd = ?
(a+b)(c+d) =
XY =
X = 33
Y = 12
ac = 3
bd = ?
(a+b)(c+d) =
XY = 15
X=3
Y=2
XY = 6
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd = ?
(a+b)(c+d) =
XY =
X = 33
Y = 12
ac = 3
bd = 6
(a+b)(c+d) = ?
XY = 15
MULT(X,Y):
Stack of Stack Frames
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 2133
Y = 2312
ac = 483
bd = ?
(a+b)(c+d) =
XY =
X = 33
Y = 12
ac = 3
bd = 6
(a+b)(c+d) = ?
XY = 396
An so on ….
Tree of Stack Frames
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
X = 21
Y = 23
ac = 4
bd = 3
(a+b)(c+d) = 15
XY = 483
X=2
Y=2
XY=4
X=1
Y=3
XY=3
X=3
Y=5
XY=15
X = 2133
Y = 2312
ac = 483
bd = 396
(a+b)(c+d) = 1890
XY = 4931496
X = 33
Y = 12
ac = 3
bd = 6
(a+b)(c+d) = 18
XY = 396
X=3
Y=1
XY=3
X=3
Y=2
XY=6
X = 54
Y = 35
ac = 15
bd = 20
(a+b)(c+d) = 72
XY = 1890
X=6
Y=3
XY=18
X=5
Y=3
XY=15
X=4
Y=5
XY=20
X=9
Y=8
XY=72
Solving Technique 1
Guess and Verify
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = 2n2 – n
•Verify: Left Hand Side Right Hand Side
T(1) = 2(1)2 – 1
T(n)
= 2n2 – n
1
4T(n/2) + n
= 4 [2(n/2)2 – (n/2)] + n
= 2n2 – n
Solving Technique 2
Guess Form and Calculate Coefficients
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 + bn + c
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2
Guess Form and Calculate Coefficients
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 + bn + 0
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
c=4c
c=0
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2
Guess Form and Calculate Coefficients
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 - 1n + 0
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
b=2b+1
b=-1
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2
Guess Form and Calculate Coefficients
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 - 1n + 0
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
a=a
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2
Guess Form and Calculate Coefficients
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = 2n2 - 1n + 0
•Verify: Left Hand Side Right Hand Side
a+b+c=1
a-1+0=1
a=2
T(1) = a+b+c
T(n)
= an2 +bn+c
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 3
Approximate Form and
Calculate Exponent
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
which is bigger?
Guess
Solving Technique 3
Calculate Exponent
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
•Guess: aT(n/b) << f(n)
•Simplify: T(n)  f(n)
In this case, the answer is easy.
T(n) = Q(f(n))
Solving Technique 3
Calculate Exponent
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
•Guess: aT(n/b) >> f(n)
•Simplify: T(n)  aT(n/b)
In this case, the answer is harder.
Solving Technique 3
Calculate Exponent
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b)
( / )
a
•Guess: G(n) = cn = cn
•Verify: Left Hand Side Right Hand Side
log a
T(n)
= cna
1 = a b-a
ba = a
a log b = log a
a = log a/log b
log b
aT(n/b)
= a [c (n/b) a ]
= c a b-a na
Solving Technique 3
Calculate Exponent
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2)
( / )
( / )
2
a
•Guess: G(n) = cn = cn
= cn
= cn
•Verify: Left Hand Side Right Hand Side
log a
T(n)
= cna
1 = a b-a
ba = a
a log b = log a
a = log a/log b
log b
log 4
log 2
aT(n/b) + f(n)
= c [a (n/b) a ]
= c a b-a na
Solving Technique 3
Calculate Exponent
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
If bigger then
(
T(n) = Q(n
log a/
log b
If bigger then
)
)
T(n) = Q(f(n))
And if aT(n/b)  f(n)
what is T(n) then?
Technique 4: Decorate The Tree
•T(1)
=
1
T(1)
=
• T(n)
=
a T(n/b) + f(n)
T(n)
=
f(n)
T(n/b)
1
a
T(n/b)
T(n/b)
T(n/b)
T(n)
T(n/b)
=
f(n)
a
T(n/b)
T(n/b)
T(n/b)
=
T(n)
f(n)
a
f(n/b)
a
T(n/b2) T(n/b2) T(n/ b2)T(n/ b2)
T(n/b)
T(n/b)
T(n/b)
=
T(n)
f(n)
a
a
T(n/b2) T(n/b2) T(n/ b2)T(n/ b2)
f(n/b)
f(n/b)
f(n/b)
f(n/b)
a
T(n/b2) T(n/b2) T(n/ b2)T(n/ b2)
a
T(n/b2) T(n/b2) T(n/ b2)T(n/ b2)
a
T(n/b2) T(n/b2) T(n/ b2)T(n/
=
T(n)
f(n)
a
a
T(n/b2) T(n/b2) T(n/ b2)T(n/ b2)
f(n/b)
f(n/b)
f(n/b)
f(n/b)
a
T(n/b2) T(n/b2) T(n/ b2)T(n/ b2)
a
T(n/b2) T(n/b2) T(n/ b2)T(n/ b2)
a
T(n/b2) T(n/b2) T(n/ b2)T(n/
11111111111111111111111111111111 . . . . . . 11111111111111111111111111111111
Evaluating: T(n) = aT(n/b)+f(n)
Level
0
1
2
i
h
Evaluating: T(n) = aT(n/b)+f(n)
Level
0
1
2
i
h
Instance
size
Evaluating: T(n) = aT(n/b)+f(n)
Level
0
1
2
i
h
Instance
size
n
Evaluating: T(n) = aT(n/b)+f(n)
Level
0
1
2
i
h
Instance
size
n
n/b
Evaluating: T(n) = aT(n/b)+f(n)
Level
0
1
2
i
h
Instance
size
n
n/b
n/b2
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
0
1
2
n
n/b2
i
n/bi
h
n/bh
n/b
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
0
1
2
n
n/b2
i
n/bi
h
n/bh
n/b
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
0
1
2
n
n/b2
i
n/bi
h
n/bh = 1
n/b
base case
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
0
1
2
n
n/b2
i
n/bi
h
n/bh = 1
n/b
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
0
1
2
n
n/b2
i
n/bi
h = log n/log b
n/bh = 1
n/b
bh = n
h log b = log n
h = log n/log b
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
0
1
2
n
n/b2
i
n/bi
h = log n/log b
1
n/b
Work
in stack
frame
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
0
1
2
n/b
f(n/b)
n/b2
f(n/b2)
i
n/bi
f(n/bi)
h = log n/log b
1
T(1)
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
0
1
2
n/b
f(n/b)
n/b2
f(n/b2)
i
n/bi
f(n/bi)
h = log n/log b
n/bh
T(1)
# stack
frames
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
0
1
2
n/b
f(n/b)
1
a
n/b2
f(n/b2)
a2
i
n/bi
f(n/bi)
ai
h = log n/log b
n/bh
T(1)
ah
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
0
1
2
n/b
f(n/b)
1
a
n/b2
f(n/b2)
a2
i
n/bi
f(n/bi)
ai
h = log n/log b
n/bh
T(1)
ah
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
0
1
2
n/b
f(n/b)
1
a
n/b2
f(n/b2)
a2
i
n/bi
f(n/bi)
ai
h = log n/log b
n/bh
T(1)
ah
ah =
a
log n/
log b
log a/
log b
=n
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
0
1
2
n/b
f(n/b)
1
a
n/b2
f(n/b2)
a2
i
n/bi
f(n/bi)
ai
h = log n/log b
n/bh
T(1)
log a/
log b
n
Work in
Level
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
Work in
Level
1 · f(n)
a · f(n/b)
0
1
2
n/b
f(n/b)
1
a
n/b2
f(n/b2)
a2
a2 · f(n/b2)
i
n/bi
f(n/bi)
ai
ai · f(n/bi)
h = log n/log b
n/bh
T(1)
log a/
log b
n
Total Work T(n) = ∑i=0..h aif(n/bi)
log a/
log b
n
· T(1)
Evaluating: T(n) = aT(n/b)+f(n)
= ∑i=0..h aif(n/bi)
If a Geometric Sum
∑i=0..n xi  θ(max(first term, last term))
Evaluating: T(n) = aT(n/b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
Work in
Level
1 · f(n)
a · f(n/b)
0
1
2
n/b
f(n/b)
1
a
n/b2
f(n/b2)
a2
a2 · f(n/b2)
i
n/bi
f(n/bi)
ai
ai · f(n/bi)
h = log n/log b
n/bh
T(1)
log a/
log b
n
Dominated by Top Level or Base Cases
log a/
log b
n
· T(1)
Evaluating: T(n) = aT(n/b)+f(n)
Is the sum Geometric?
Simplify by letting f(n) = nc logkn
T(n) = ∑i=0..h aif(n/bi)
= ∑i=0..h ai(n/bi)c logk(n/bi)
c
c
loga]·i
b]·i
=
(../bi)c(n/..)
=i = [-c[log
a 22
n
= nc ∑i=0..h 2[log a - c log b] · i logk(n/bi)
= nc ∑i=0..h 2-d i logk(n/bi)
c > log a/log b
log a - c log b < 0
log a/
c
<
log b
c = log a/log b & k>-1
c = log a/log b & k<-1
log a - c log b = 0 log a - c log b < 0
Geometrically Decreasing
Arithmetic Sum
Geometrically Increasing
Sum = θ(first term)
Sum = θ(any term # terms)
Sum = θ(last term)
log a/
log b )
= θ(
= θ(f(n)).
= θ(f(n)  log n)
n
Evaluating: T(n) = aT(n/b)+f(n)
Is the sum Geometric?
Simplify by letting f(n) = nc logkn
T(n) = ∑i=0..h aif(n/bi)
= ∑i=0..h ai(n/bi)c logk(n/bi)
= nc ∑i=0..h 2[log a - c log b] · i
= nc ∑i=0..h 20 i logk(n/bi)
c > log a/log b
log a - c log b < 0
k=-1  logk(n/bi)
logk(n/bi) = 1/
(logn - i  logb)
(backwards)
log a/
c
<
log b
c = log a/log b & k>-1
c = log a/log b & k<-1
log a - c log b = 0 log a - c log b < 0
Geometrically Decreasing
Arithmetic Sum
Sum = θ(first term)
Sum = θ(any term # terms)
= θ(f(n)).
 1/i
= θ(f(n)  log n)
Geometricallyde
Increasing
(back wards)
Sum = θ(last term)
log a/
log b )
= θ(
n
Evaluating: T(n) = aT(n/b)+f(n)
Is the sum Geometric?
Simplify by letting f(n) = nc logkn
T(n) = ∑i=0..h aif(n/bi)
= ∑i=0..h ai(n/bi)c logk(n/bi)
= nc ∑i=0..h 2[log a - c log b] · i logk(n/bi)
= nc ∑i=0..h 2+d i logk(n/bi)
c > log a/log b
log a - c log b < 0
log a/
c
<
log b
c = log a/log b & k>-1
c = log a/log b & k<-1
log a - c log b = 0 log a - c log b < 0
Geometrically Decreasing
Arithmetic Sum
Geometrically Increasing
Sum = θ(first term)
Sum = θ(any term # terms)
Sum = θ(last term)
log a/
log b )
= θ(
= θ(f(n)).
= θ(f(n)  log n)
n
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Time for top level?:
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Time for top level: f(n) = n3/log n
Time for base cases?:
5
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Time for top level: f(n) = n3 /log n
log ?/
log a/
log ?
log b
θ(n
)
)=
Time for base cases: θ(n
5
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Time for top level: f(n) = n3 /log n
log 4/
log a/
log 2
log b
θ(n
)=?
)=
Time for base cases: θ(n
5
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Time for top level: f(n) = n3/log n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = ? log a/log b = ?
5
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Time for top level: f(n) = n3/log n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 3 > 2 = log a/log b
5
Hence, T(n) = ?
Evaluating: T(n) = 4T(n/2)+ n3/log n
5
Time for top level: f(n) = n3/log n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 3 > 2 = log a/log b
5
Hence, T(n) = θ(top) = θ(f(n)) = θ(n3/log n).
5
bigger
Evaluating: T(n) = 4T(n/2)+ 2n
Time for top level: f(n) = 2n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c =>big
? > 2 = log a/log b
Hence, T(n) = θ(top) = θ(f(n)) = θ(2n).
The time is even more dominated
by the top level of recursion
Evaluating: T(n) = 4T(n/2)+ n log n
5
Time for top level: f(n) = n log5n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 1<
? 2 = log a/log b
Hence, T(n) ==?θ(base cases) = θ(n
log a/
log b
) = θ(n2).
Evaluating: T(n) = 4T(n/2)+ log5n
Time for top level: f(n) = log5n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 0<
? 2 = log a/log b
Hence, T(n) ==?θ(base cases) = θ(n
log a/
log b
) = θ(n2).
Evaluating: T(n) = 4T(n/2)+ n2
Time for top level: f(n) = n2
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 2=
? 2 = log a/log b
Hence, T(n) ==?θ(f(n) log(n) ) = θ(n2 log(n)).
Evaluating: T(n) = 4T(n/2)+ n2 log n
5
Time for top level: f(n) = n2 log5n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 2 = 2 = log a/log b
5 ?> -1
k=?
Hence, T(n) == ?θ(f(n) log(n) ) = θ(n2 log6(n)).
Evaluating: T(n) = 4T(n/2)+ n2/log n
5
Time for top level: f(n) = n2/log5n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 2 = 2 = log a/log b
k =-5
? ?< -1
Hence, T(n) ==?θ(base cases) = θ(n
log a/
log b
) = θ(n2).
Evaluating: T(n) = 4T(n/2)+ n2/log n
Time for top level: f(n) = n2/log n
log 4/
log a/
log 2
log b
θ(n
) = θ(n2)
)=
Time for base cases: θ(n
Dominated?: c = 2 = 2 = log a/log b
k =-1
? ?= -1
Hence, T(n) ==?θ(nc loglog n) = θ(n2 loglog n).
Did not do before.
Evaluating: T2(n) = 4 T2(n/2+n1/2)+ n3
Sufficiently close to:
T(n) = 4T(n/2)+ n3 = θ(n3).
T2(n) = ?θ(n3).
Evaluating: T(n) = aT(n-b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
Work in
Level
1 · f(n)
a2 · f(n-2b)
ai · f(n-ib)
0
1
2
n-b
f(n-b)
n-2b
f(n-2b)
1
a
a2
i
n-ib
f(n-ib)
ai
hh==n?/b
n-hb
T(0)
n/
b
a
|base case| = 0 = n-hb
Likely dominated by base cases
n
h = /b
a · f(n-b)
n/
b
a
· T(0)
Exponential
Evaluating: T(n) = 1T(n-b)+f(n)
Level
Instance
size
Work
in stack
frame
n
f(n)
# stack
frames
Work in
Level
f(n)
f(n-2b)
0
1
2
n-b
f(n-b)
n-2b
f(n-2b)
1
1
1
i
n-ib
f(n-ib)
1
f(n-ib)
hh==n?/b
n-hb
T(0)
1
f(0)
f(n-b)
Total Work T(n) = ∑i=0..h f(b·i) = θ(f(n)) or θ(n·f(n))
Math Review: Exponents
XA XB = XA+B (not XAB !!)
XA / XB = XA-B
(XA )B = XAB
XA + XA = 2XA
2A + 2A = 2A+1
Logarithms
• Definition: XA = B if and only if logXB =
A (x is the “base” of the logarithm).
– Example: 102 = 100 means log10100 = 2.
• Theorems
logXAB = logXA + logXB
logXA/B = logXA – logXB
logX AB = B logXA
Logarithms…
• Theorem: logAB = logCB / logCA
Proof: Let X = logCB, Y = logCA, and
Z = logAB. By the definition of
logarithm: CX = B, CY = A, and AZ =
B.
Thus CX = B = AZ = CYZ , X = YZ, Z
= X/Y.
Logarithms…
• The notation for logs can be confusing.
There are two alternatives:
– log2(x) = log (log x)
– log2(x) = (log x)2
or
• Generally, we use the 2nd definition.
• Note: log2(x) is not (log x2)
• Note: log is not a multiplicative entity, it is a
function.
End