Efficiency of Algorithms
Csci 107
Lecture 7
• Last time
– Data cleanup algorithms and analysis
– (1), (n), (n2)
• Today
– Binary search and analysis
– (lg n)
– Sorting
• Selection sort
Searching
• Problem: find a target in a list of values
• Sequential search
– Best-case : (1) comparison
• target is found immediately
– Worst-case: (n) comparisons
• Target is not found
– Average-case: (n) comparisons
• Target is found in the middle
• Can we do better?
– No…unless we have the input list in sorted order
Searching a sorted list
• Problem: find a target in a sorted list
– How can we exploit that the list is sorted, and come up
with an algorithm faster than sequential search in the
worst case?
– How do we search in a phone book?
– Can we come up with an algorithm?
• Check the middle value
• If smaller than target, go right
• Otherwise go left
Binary search
•
•
•
•
Get values for list, A1, A2, ….An, n , target
Set start =1, set end = n
Set found = NO
Repeat until ??
– Set m = middle value between start and end
– If target = m then
• Print target found at position m
• Set found = YES
– Else if target < Am then end = m-1
Else start = m+1
• If found = NO then print “Not found”
• End
Efficiency of binary search
• What is the best case?
• What is the worst case?
– Initially the size of the list in n
– After the first iteration through the repeat loop, if not found, then
either start = m or end = m ==> size of the list on which we search
is n/2
– Every time in the repeat loop the size of the list is halved: n, n/2,
n/4,….
– How many times can a number be halved before it reaches 1?
Orders of magnitude
• Order of magnitude ( lg n)
– Worst-case efficiency of binary search: ( lg n)
• Comparing order of magnitudes
(1) << (lg n) << (n) << (n2)
Sorting
• Problem: sort a list of items into alphabetical or
numerical order
• Why sorting?
– Sorting is ubiquitous (very common)!!
– Examples:
• Registrar: Sort students by name or by id or by department
• Post Office: Sort mail by address
• Bank: Sort transactions by time or customer name or accound
number …
• For simplicity, assume input is a list of n numbers
• Ideas for sorting?
Selection Sort
• Idea: grow a sorted subsection of the list from the back to
the front
57216483|
5721643|8
521643|78
52134|678
…
|1 2 3 4 5 6 7 8
Selection Sort
• Pseudocode (at a high level of abstraction)
– Get values for n and the list of n items
– Set marker for the unsorted section at the end of the list
– Repeat until unsorted section is empty
• Select the largest number in the unsorted section of the list
• Exchange this number with the last number in unsorted section of list
• Move the marker of the unsorted section forward one position
– End
Selection Sort
• Level of abstraction
– It is easier to start thinking of a problem at a high level
of abstraction
• Algorithms as building blocks
– We can build an algorithm from “parts” consisting of
previous algorithms
– Selection sort:
• Select largest number in the unsorted section of the list
• We have seen an algorithm to do this last time
• Exchange 2 values
Selection Sort Analysis
• Iteration 1:
– Find largest value in a list of n numbers : n-1 comparisons
– Exchange values and move marker
• Iteration 2:
– Find largest value in a list of n-1 numbers: n-2 comparisons
– Exchange values and move marker
• Iteration 3:
– Find largest value in a list of n-2 numbers: n-3 comparisons
– Exchange values and move marker
• …
• Iteration n:
– Find largest value in a list of 1 numbers: 0 comparisons
– Exchange values and move marker
Total: (n-1) + (n-2) + …. + 2 + 1
Selection Sort
• Total work (nb of comparisons):
– (n-1) + (n-2) + …. + 2 + 1
– This sum is equal to .5n2 -.5n (proved by Gauss)
=> order of magnitude is ( ? )
• Questions
– best-case, worst-case ?
– we ignored constants, and counted only comparisons.. Does this
make a difference?
• Space efficiency
– extra space ?
Selection Sort
• In conclusion: Selection sort
– Space efficiency:
• No extra space used (except for a few variables)
– Time efficiency
• There is no best-case and worst-case
• the amount of work is the same: (n2) irrespective of
the input
• Other sorting algorithms? Can we find more
efficient sorting algorithms?
This week…
• Tuesday: Lab 4
• Exam 1 (Wednesday or Monday?)
– Material: Algorithms
– Chapter 1, 2 & 3 from textbook
• For next time
– Binary search
– Review chapter 1,2 & 3
– Practice exam