Lecture 6
1/28/04 18:02
Read S&G chapter 4
Lecture 6
Efficiency of Algorithms (2)
(S&G, ch.3)
1/28/04
CS 100 - Lecture 6
1
1/28/04
• Given: n, N1, N2, …, Nn
• Want: Rearrangement M1, M2, …, Mn
where M1 ≤… ≤ Mn according to the appropriate
understanding of ≤.
• There are many well-known algorithms for doing
this. Preference may be due to
• One of the most common activities of a
computer is sorting data.
• Arranging data into numerical or
alphabetical order for purposes of
– Reports by category
– Summarizing data
– Searching data
CS 100 - Lecture 6
–
–
–
–
3
1/28/04
slide courtesy of S. Levy
Sorting - Selection Sort
 U - Marker for unsorted section
 L - Position of largest so far
 C - Current position
CS 100 - Lecture 6
CS 100
4
Set U to n
Repeat until U = 1
Set L to 1
Set C to 2
Search For Largest of
Repeat until C > U
N [1], …, N [U]
If N [C] > N [L] then
Set L to C
Set C to C + 1
Exchange N [L] and N [U]
Set U to U – 1
Stop
Find the largest element in list
Swap it with last element in list
Find next largest
Swap it with next to last element in list
Etc
slide courtesy of S. Levy
CS 100 - Lecture 6
Selection Sort - The Algorithm
• Variables:
1/28/04
List size
Degree of order already present
Ease of programming
Program available
slide courtesy of S. Levy
• Strategy:





2
Sorting (2)
Sorting
1/28/04
CS 100 - Lecture 6
5
1/28/04
CS 100 - Lecture 6
6
slide courtesy of S. Levy
1
Lecture 6
1/28/04 18:02
Selection Sort - Example (Pass 1)
L
6
U
5
19
9
L
12
6 5
19
9
C
L
6
U
5
19
9
12
U
L
12
6
C
U
L
19
6 5
9
U
19
9
12
6 5
9
6
5
12
19
6
C
5
9
CS 100 - Lecture 6
U
5
7
19
12 19
6
5
9
12
C
L
6
5
U
12
9
12
5
19
L
U
6
5
C
U
9 19
CS 100 - Lecture 6
12 19
6
5
5
9
U
9
12 19
9
12 19
5 6
9
12
19
C
LU
12 19
19
5
6
9
12
19
C
LU
1/28/04
8
C
C
6
19
L U
9
9
9 12
19
C
1/28/04
C
LU
6
5
Selection Sort - Example (Pass 4)
U
5
6
slide courtesy of S. Levy
LU
9
19
C
L
L
6
Selection Sort - Example (Pass 3)
6
9
L U
slide courtesy of S. Levy
L
12
C
9 12
1/28/04
5
C
U
19
6
U
C
L
5
L U
9 19
C
L
U
L
12
C
6
12
C
L
5
U
5
C
12
C
6
Selection Sort - Example (Pass 2)
12 19
C
CS 100 - Lecture 6
9
1/28/04
slide courtesy of S. Levy
CS 100 - Lecture 6
10
slide courtesy of S. Levy
Efficiency Analysis
of Selection Sort
Efficiency of Selection Sort (2)
• To sort n element list:
• Makes n – 1 passes through the list
• Each pass uses Search For Largest, on the
unsorted part of the list, to find the largest
element
• If the unsorted part has length U, Search For
Largest takes U – 1 comparisons
n – 1 comparisons to find largest
n – 2 comparisons to find second largest
…
2 comparisons to find largest of last three
1 comparisons to find largest of last two
• Total number of comparisons:
(n −1) n
(n −1) + ( n − 2) + L + 2 + 1 =
2
1/28/04
CS 100 - Lecture 6
11
1/28/04
CS 100 - Lecture 6
12
€
CS 100
2
Lecture 6
1/28/04 18:02
A Useful Formula
€
A Useful Formula (2)
In analysis of algorithms, we often encounter
a sum of consecutive integers :
S = 1+ 2 + L + N
Note that we can write :
S = 1 +
2
+ L + (N −1) + N
S = N + (N −1) + L +
2
+ 1
Add 2S = (N + 1) + ( N + 1) + L + (N + 1) + ( N + 1)
1444444
424444444
3
N
= N ( N + 1)
€
Hence, S = N (N + 1) 2
€
1/28/04
CS 100 - Lecture 6
13
1/28/04
CS 100 - Lecture 6
14
€
Best, Worst, & Average
Motivation for
Asymptotic Complexity
• Suppose there are N possible problem instances
and that Tk = time to solve instance k
• We are interested in comparing the
efficiency of various algorithms,
independent of computer running them
• Therefore, we choose to ignore:
• Worst case : W = max{T1,T2 ,K,TN }
€
• Best case : B = min{T1,T2 ,K,TN }
€
• Average case : A = (T1 + T2 + L + TN ) /N
– the absolute time taken by the instructions
– constant initialization overhead
– time taken by the less frequently executed
instructions
(if they are equally likely)
€
€
• If pk is the probability of instance k,
then weighted average M = p1T1 + p2T2 + L + pN TN
1/28/04
CS 100 - Lecture 6
15
1/28/04
CS 100 - Lecture 6
16
€
Quadratic vs. Linear Growth
Motivation for
Asymptotic Complexity (2)
• Not concerned about efficiency for small
problems (they’re all quick enough)
• Usually interested in using on large
problems
• We investigate how resource usage varies
on progressively larger problems
• Which is asymptotically better, i.e., for
sufficiently large problems
1/28/04
CS 100
CS 100 - Lecture 6
• On big enough problem, quadratic algorithm
takes more time than linear algorithm
17
1/28/04
CS 100 - Lecture 6
18
3
Lecture 6
1/28/04 18:02
Equivalent Complexity
Complexity Classes
• For our purposes:
an algorithm taking time 10n sec. is as good as
one taking 2n sec.
•
•
•
•
•
•
•
– because absolute time for individual instructions
doesn’t matter
• algorithm taking 2n + 10 sec. is as good as one
taking 2n sec.
– because fixed overhead doesn’t matter
• algorithm taking n2 + 2n sec. is as good as one
taking n2 sec.
– less frequently executed instructions don’t matter
1/28/04
CS 100 - Lecture 6
19
No growth: Θ(1)
Linear growth: Θ(n)
Quadratic growth: Θ(n2)
Cubic growth: Θ(n3)
Polynomial growth: Θ(nk) for any k
Exponential growth: Θ(kn) for any k
Θ (theta) is called the “exact order” of a
function
1/28/04
Combinatorial Explosion
 k possible first moves
 k2 possible replies
 k3 possible replies to the replies, etc.
• In general, must consider kn possibilities to
look n moves ahead (exponential algorithm)
• “Brute force” solutions to many problems
lead to “combinatorial explosion”
CS 100 - Lecture 6
20
Intractable Problems
• Consider a game with k possible moves on
each play
• Therefore:
1/28/04
CS 100 - Lecture 6
21
• For some problems, all known algorithms
are exponential-time
• No known polynomial-time algorithms —
not even Θ(n100)!
• Sometimes there are efficient
approximation algorithms
– give good, but not perfect answer
– may be good enough for practical purposes
1/28/04
CS 100 - Lecture 6
22
Limitations of
Asymptotic Complexity
• Note that 1,000,000 n is Θ(n),
but 0.000001 n2 is Θ(n2)
• Paradox: 1,000,000 n hrs >> 0.000001 n2 hrs
(for n << 1012),
but Θ(n) is “more efficient” than Θ(n2)
• Not useful if concerned with absolute real-time
constraints (e.g., must respond in 2 sec.)
• Not useful if concerned with fixed size or bounded
problems (e.g., n < 1,000,000)
• Sometimes, “the constants matter”
1/28/04
CS 100
CS 100 - Lecture 6
23
4