Lecture 6
Efficiency of Algorithms (2)
(S&G, ch.3)
5/29/2016
CS 100 - Lecture 6
1
Read S&G chapter 4
Warning: There are printing errors in
ch. 4 in formulas!
We will send out corrections by email
5/29/2016
CS 100 - Lecture 6
2
Sorting
• One of the most common activities of a
computer is sorting data.
• Arranging data into numerical or
alphabetical order for purposes of
– Reports by category
– Summarizing data
– Searching data
5/29/2016
CS 100 - Lecture 6
slide courtesy of S. Levy
3
Sorting (2)
• Given: n, N1, N2, …, Nn
• Want: Rearrangement M1, M2, …, Mn
where M1 …  Mn according to the appropriate
understanding of .
• There are many well-known algorithms for doing
this. Preference may be due to
–
–
–
–
5/29/2016
list size
degree of order already present
ease of programming
program available
CS 100 - Lecture 6
slide courtesy of S. Levy
4
Sorting - Selection Sort
• Strategy:





find the largest element in list
swap it with last element in list
find next largest
swap it with next to last element in list
etc
• Variables:
 U - Marker for unsorted section
 L - Position of largest so far
 C - Current position
5/29/2016
CS 100 - Lecture 6
slide courtesy of S. Levy
5
Selection Sort - The Algorithm:
High Level
Set U to n
Repeat until U = 1
Search For Largest of N [1] to N [U]
and put its location in L
Exchange N [L] and N [U]
Set U to U – 1
Stop
5/29/2016
CS 100 - Lecture 6
6
Selection Sort - The Algorithm
Set U to n
Repeat until U = 1
Set L to 1
Set C to 2
Repeat until C > U
Search For Largest of
N [1], …, N [U] and
If N [C] > N [L] then
put location in L
Set L to C
Set C to C + 1
Exchange N [L] and N [U]
Set U to U – 1
Stop
5/29/2016
CS 100 - Lecture 6
slide courtesy of S. Levy
7
Selection Sort - Example (Pass 1)
L
6 5
19
U
9 12
6
5
L
19
U
9 12
L
19
C
U
9 12
C
L
6 5
6
6
5
5
5/29/2016
19
C
U
9 12
6
5
C
L
19
C
U
9 12
L
19
U
9 12
C
6
5
L
12
U
9 19
C
CS 100 - Lecture 6
slide courtesy of S. Levy
8
Selection Sort - Example (Pass 2)
L
6 5
U
12 9 19
6
5
L U
12 9
C
C
L
6 5 12
C
6
5
19
U
9
L U
12 9
C
19
6
5
L U
9 12
19
C
19
L U
6 5 12 9 19
5/29/2016
C
CS 100 - Lecture 6
slide courtesy of S. Levy
9
Selection Sort - Example (Pass 3)
L
6
U
5 9 12 19
6
5
LU
9 12
C
L
6 5
6
5
19
C
U
9 12 19
C
LU
9 12 19
19
C
LU
6 5 9 12 19
C
5/29/2016
CS 100 - Lecture 6
slide courtesy of S. Levy
10
Selection Sort - Example (Pass 4)
L U
6 5 9
C
L U
6 5
6
9
12
19
9 12 19
C
LU
6 9
5
12 19
U
5
12
19
C
5/29/2016
CS 100 - Lecture 6
slide courtesy of S. Levy
11
Efficiency Analysis
of Selection Sort
• Makes n – 1 passes through the list
• Each pass uses Search For Largest, on the
unsorted part of the list, to find the largest
element
• If the unsorted part has length U, Search For
Largest takes U – 1 comparisons
5/29/2016
CS 100 - Lecture 6
12
Efficiency of Selection Sort (2)
• To sort n element list:
n – 1 comparisons to find largest
n – 2 comparisons to find second largest
…
2 comparisons to find largest of last three
1 comparisons to find largest of last two
• Total number of comparisons:
n 1n

n 1  n  2   2  1 
2
5/29/2016
CS 100 - Lecture 6
13

A Useful Formula
In analysis of algorithms,
a sum of consecutive integers :
S  1 2   N
Note that we can write :
S  1 
2

S  N  (N 1) 
Add 2S  N  1  N  1 
 (N 1)  N

2

1
 N  1  N  1
N
 N N  1


we often encounter
Hence, S  N N  1 2
5/29/2016
CS 100 - Lecture 6
14
A Useful Formula (2)
5/29/2016
CS 100 - Lecture 6
15
Best, Worst, & Average
 Suppose there are N possible problem instances
and that Tk  time to solve instance k
 Worst case : W  max T1,T2, ,TN 
 Best case : B  min T1,T2 , ,TN 
 Average case : A  T1  T2 
 TN  /N
(if they are equally likely)
 If pk is the probability of instance k,
then weighted average M  p1T1  p2T2 
5/29/2016
CS 100 - Lecture 6
 pN TN
16
Motivation for
Asymptotic Complexity
• We are interested in comparing the
efficiency of various algorithms,
independent of computer running them
• Therefore, we choose to ignore:
– the absolute time taken by the instructions
– constant initialization overhead
– time taken by the less frequently executed
instructions
5/29/2016
CS 100 - Lecture 6
17
Motivation for
Asymptotic Complexity (2)
• Not concerned about efficiency for small
problems (they’re all quick enough)
• Usually interested in using on large
problems
• We investigate how resource usage varies
on progressively larger problems
• Which is asymptotically better, i.e., for
sufficiently large problems
5/29/2016
CS 100 - Lecture 6
18
Quadratic vs. Linear Growth
QuickTime™ and a
Animation decompressor
are needed to see this picture.
• On big enough problem, quadratic algorithm
takes more time than linear algorithm
5/29/2016
CS 100 - Lecture 6
19
Quadratic vs. Linear Growth (2)
y  2x  10
y  0.1x  x  5
2

5/29/2016

CS 100 - Lecture 6
20
Equivalent Complexity
• For our purposes:
An algorithm taking time 10n sec. is as good as
one taking 2n sec.
– because absolute time for individual instructions
doesn’t matter
• An algorithm taking 2n + 10 sec. is as good as one
taking 2n sec.
– because fixed overhead doesn’t matter
• An algorithm taking n2 + 2n sec. is as good as one
taking n2 sec.
– less frequently executed instructions don’t matter
5/29/2016
CS 100 - Lecture 6
21
Complexity Classes
•
•
•
•
•
•
•
No growth: (1)
Linear growth: (n)
Quadratic growth: (n2)
Cubic growth: (n3)
Polynomial growth: (nk) for any k
Exponential growth: (kn) for any k
 (theta) is called the “exact order” of a
function
5/29/2016
CS 100 - Lecture 6
22
Comparison of Growth Orders
5/29/2016
CS 100 - Lecture 6
23
Combinatorial Explosion
• Consider a game with k possible moves on
each play
• Therefore:
 k possible first moves
 k2 possible replies
 k3 possible replies to the replies, etc.
• In general, must consider kn possibilities to
look n moves ahead (exponential algorithm)
• “Brute force” solutions to many problems
lead to “combinatorial explosion”
5/29/2016
CS 100 - Lecture 6
24
Intractable Problems
• For some problems, all known algorithms
are exponential-time
• No known polynomial-time algorithms —
not even (n100)!
• Sometimes there are efficient
approximation algorithms
– give good, but not perfect answer
– may be good enough for practical purposes
5/29/2016
CS 100 - Lecture 6
25
Limitations of
Asymptotic Complexity
• Note that 1,000,000 n is (n),
but 0.000001 n2 is (n2)
• Paradox: 1,000,000 n hrs >> 0.000001 n2 hrs
(for n << 1012),
but (n) is “more efficient” than (n2)
• Not useful if concerned with absolute real-time
constraints (e.g., must respond in 2 sec.)
• Not useful if concerned with fixed size or bounded
problems (e.g., n < 1,000,000)
• Sometimes, “the constants matter”
5/29/2016
CS 100 - Lecture 6
26