This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License .
CS 312: Algorithm Design &
Analysis
Lecture #2: Asymptotic Notation
Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

Pop Quiz
1. What is the reward for submitting project reports early?
2. How many penalty-free late days are in your budget for the semester?
3. How many penalty-free late days can you use on any one project?
4. What is the penalty for late submission of project reports (after you’ve used your penalty-free late days)?
5. What is the time of day for project deadlines?
6. T/F: You may submit late regular homework assignments.
7. T/F: You can do a whiteboard experience alone.

Announcements
 TA and instructor office hours are on the course wiki
 Project #1
 Due dates on the schedule
 Instructions linked from the online schedule
 We’ll cover the mathematical ideas in class
 Help session with TA
 Purpose of help sessions
 HW #0
 C# surprises?

Objectives
 Revisit orders of growth
 Formally introduce asymptotic notation:
O,
W, Q
 Classify functions in asymptotic orders of growth

Orders of Growth
 𝐶(𝑛) : the cost (e.g., number of steps) required by an algorithm on an input of size / difficulty 𝑛 .
 Efficiency : how cost grows with the size/ difficulty 𝑛 of a given problem instance.
 Order of growth: the functional form of 𝐶(𝑛) up to a constant multiple as 𝑛 goes to infinity.

Orders of Growth n
10
10 2
10 3
10 4
10 5
10 6 log
2 n
3.3
6.6
10
13
17
20 n n log
2 n
3.3*10 n 2
10 10 2
10 2 6.6*10 2 10 4
10 3 1.0*10 4 10
10 4 1.3*10 5 10 8
6
10 5 1.7*10 6 10 10
10 6 2.0*10 7 10 12 n 3
10 3
10 6
10 9
10 12
10 15
10 18

Orders of Growth n
10
10 2
10 3
10 4
10 5
10 6 log
2 n
3.3
6.6
10
13
17
20 n n log
2 n
3.3*10 n 2
10 10 2
10 2 6.6*10 2 10 4
10 3 1.0*10 4 10
10 4 1.3*10 5 10 8
6
10 5 1.7*10 6 10 10
10 6 2.0*10 7 10 12 n 3 2 n n !
10 3 ~10 3 ~3.6*10 6
10 6 ~1.3*10 30 ~9*10 157
10 9 … …
10 12
10 15
10 18
Efficient

Kinds of Efficiency
 Need to decide which instance of a given size to use as the representative for that class:
Algorithm Domain
• Worst Case
• Best Case
• Average Case
(over all possible instances)
1 2 3 4 5
Instance size

Asymptotic Notation
 Definition: given an asymptotically nonnegative fn. 𝑔(𝑛) ,
𝛩 𝑔 𝑛 = 𝑓 𝑛 : ∃ ( 𝑐
1
, 𝑐
2
, 𝑛
0
> 0 such that 0
≤ 𝑐
1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐
2 𝑔 𝑛 ∀𝑛 ≥ 𝑛
0
}
 Translation:
 An asymptotically non-negative function 𝑓(𝑛) belongs to the set Θ(𝑔(𝑛)) if and only if there exist positive constants 𝑐
1 and 𝑐
2 can be “sandwiched” between 𝑐
1 such that 𝑔(𝑛) and 𝑐 𝑓(𝑛)
2 𝑔(𝑛) , scaled versions of 𝑔(𝑛) , for sufficiently large 𝑛 .

Asymptotic Notation
Definitions: given an asymptotically non-negative fn. g(n),
Q
( g ( n ))

{ f ( n ) :

( c
1
, c
2
, n
0
)
0
 c
1 g ( n )

 f (
0 such that n )
 c
2 g ( n )
 n
 n
0
} g n
 f n
 c n
0
)

0 such that
0

( )

( )
  n
0
}
W g n
 f n
 c n
0
)

0 such that
0

( )

( )
  n
0
}

Notational Convention
 Unfortunately, 𝑓(𝑛) = 𝛩(𝑔(𝑛)) means 𝑓(𝑛) ∈ 𝛩(𝑔(𝑛))

Asymptotic Notation c g(n) f(n) n
0 f ( n )

O ( g ( n )) f(n) c g(n) n
0 f ( n )
 W
( g ( n )) c
2 g(n) f(n) c
1 g(n) n
0 f ( n )
 Q
( g ( n ))

Cases vs. Orders
 Do not confuse worst-case with 𝑂()

Distinguishing 𝑂 from Θ
 Can you draw a function that is in 𝑂 𝑛 2 but not Θ 𝑛 2
?

Distinguishing 𝑂 from Θ
 Can you draw a function that is in 𝑂 𝑛 2 but not Θ 𝑛 2
?

Example
• Show that
1 n
2 
3 n
Q n
2
( )
2
• Must find positive constants c
1
, c
2
, n
0 c
1 n
2 
1
2 n
2 
• Divide through by n 2 to get
3 c
1 n


1
2 c
2 n
 such that
2  n
3 n
 c
2

 n n
0
 n
0
• Hint: consider one side of the inequality at a time:
– for n
0
=7 we have 𝑐
1
≤
1
14
– for n
0
=6 we have 𝑐
2
≥ 1
2
; works for larger n
0
=7 as well
• This proof is constructive

Another Example
Show that 6𝑛 3 ∉ 𝛩 𝑛 2
 Use proof by contradiction: assume that 6𝑛 3 ∈ Θ 𝑛 2
 Suppose positive constants c
2
6𝑛 3 ≤ 𝑐
2 𝑛 2 ∀𝑛 ≥ 𝑛
0
, n
0 exist such that
 But this implies that 𝑛 ≤ 𝑐
2 ∀𝑛 ≥ 𝑛
6
 i.e., 𝑛 is bounded by 𝑐
2
6
, a constant
0
 But 𝑛 is unbounded; hence we have a contradiction.
 Thus, our assumption was false; hence we have shown that 6𝑛 3 ∉ 𝛩 𝑛 2

Other proof types?
 Induction
 Optional example on HW #1
 Deduction
 Others!

Duality
( )
W g n g n

O f n

The Limit Rule

The Limit Rule

Example

Example

Useful Identity: L’Hopital’s Rule
Applicable when:
• lim 𝑓(𝑛) = lim 𝑔(𝑛) = 0
• lim 𝑓(𝑛) = lim 𝑔(𝑛) = ∞
As 𝑛 → ∞

Review: Log Identities

Review: Log Identities

Review: More Logarithms

Review: More Logarithms

Assignment
 HW #1: 0.1 (a-e, g, m) in the textbook
 Problems like these will appear on the mid-term and final exams
 Justify your answers and show your work
 Remember:
 Don’t spend more than 2 focused hours on the homework exercises.
 If you reach 2 hours of concentrated effort , stop and make a note to that effect on your paper (unless you’re having fun and want to continue).
 The TAs will take that into consideration when grading.

 The following slides are extra

Addition

Addition

Multiplication

Multiplication

Classic Multiplication
American Style
5001 x 502
10002
0
+ 25005
2510502
English Style
5001 x 502
25005
0
+ 10002
2510502
O(n 2 ) for 2 n-digit numbers

Multiplication a la Francais / Russe

Multiplication a la Francais / Russe

Mulitplication a la Francais / Russe

0 if y

0 x if y

1
 2( x
  y / 2 )
 x 2( x
  y / 2 ) if is odd function multiply(x,y)
Input: Two n-bit integers x and y, where y

0
Output: Their product if y = 0: return 0 if y = 1: return x z = multiply(x, floor(y/2)) if y is even: return 2z else: return x+2z

Mulitplication a la Francais / Russe

0 if y

0 x if y

1
 2( x
  y / 2 )
 x 2( x
  y / 2 ) if is odd function multiply(x,y)
Input: Two n-bit integers x and y, where y

0
Output: Their product if y = 0: return 0 if y = 1: return x z = multiply(x, floor(y/2)) if y is even: return 2z else: return x+2z

Arabic Multiplication
1
2
1
0
9 8 1
0 0 0
9 8 1
1 1 0
8 6 2
2 2 0
7 4 3
3 3 0
6 2 4
5 5 4
1
2
3
4

Division function divide(x,y)
Input: Two n-bit integers x and y, where y

1
Output: The quotient and remainder of x divided by y if x=0: return (q, r) = (0,0)
(q, r) = divide(floor(x/2),y) q=2*q, r=2*r if x is odd: r=r+1 if r
 y: r=r-y, q=q+1 return (q, r)

Division function divide(x,y)
Input: Two n-bit integers x and y, where y

1
Output: The quotient and remainder of x divided by y if x=0: return (q, r) = (0,0)
(q, r) = divide(floor(x/2),y) q=2*q, r=2*r if x is odd: r=r+1 if r
 y: r=r-y, q=q+1 return (q, r)