Analysis of Algorithms
CS 477/677
Instructor: Monica Nicolescu
The Heap Data Structure
• Def: A heap is a nearly complete binary tree with
the following two properties:
– Structural property: all levels are full, except
possibly the last one, which is filled from left to right
– Order (heap) property: for any node x
Parent(x) ≥ x
8
7
5
4
2
It doesn’t matter that 4 in
level 1 is smaller than 5 in
level 2
Heap
CS 477/677
2
Array Representation of Heaps
• A heap can be stored as an
array A.
– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]
– Heapsize[A] ≤ length[A]
• The elements in the subarray
A[(n/2+1) .. n] are leaves
• The root is the maximum
element of the heap
A heap is a binary tree that is filled in order
CS 477/677
3
Maintaining the Heap Property
•
Suppose a node is smaller than a
child
– Left and Right subtrees of i are max-heaps
•
Invariant:
– the heap condition is violated only at that
node
•
To eliminate the violation:
– Exchange with larger child
– Move down the tree
– Continue until node is not smaller than
children
CS 477/677
4
Building a Heap
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2
1
4
2
3
1
3
4
8
2
14
A:
4
1
3
2
5
6
7
16 9
9
10
8
7
16
9
10 14
CS 477/677
10
8
7
5
Heapsort
• Goal:
– Sort an array using heap representations
• Idea:
– Build a max-heap from the array
– Swap the root (the maximum element) with the last
element in the array
– “Discard” this last node by decreasing the heap size
– Call MAX-HEAPIFY on the new root
– Repeat this process until only one node remains
CS 477/677
6
Alg: HEAPSORT(A)
1. BUILD-MAX-HEAP(A)
O(n)
2. for i ← length[A] downto 2
3.
4.
•
n-1 times
do exchange A[1] ↔ A[i]
MAX-HEAPIFY(A, 1, i - 1)
O(lgn)
Running time: O(nlgn)
CS 477/677
7
Example:
MAX-HEAPIFY(A, 1, 4)
A=[7, 4, 3, 1, 2]
MAX-HEAPIFY(A, 1, 3)
MAX-HEAPIFY(A, 1, 2)
MAX-HEAPIFY(A, 1, 1)
CS 477/677
8
HEAP-MAXIMUM
Goal:
– Return the largest element of the heap
Alg: HEAP-MAXIMUM(A)
1.
Running time: O(1)
return A[1]
Heap A:
Heap-Maximum(A) returns 7
CS 477/677
9
HEAP-EXTRACT-MAX
Goal:
–
Extract the largest element of the heap (i.e., return the max
value and also remove that element from the heap
Idea:
–
Exchange the root element with the last
–
Decrease the size of the heap by 1 element
–
Call MAX-HEAPIFY on the new root, on a heap of size n-1
Heap A:
Root is the largest element
CS 477/677
10
HEAP-EXTRACT-MAX
Alg: HEAP-EXTRACT-MAX(A, n)
1. if n < 1
2.
then error “heap underflow”
3.
max ← A[1]
4.
A[1] ← A[n]
5.
MAX-HEAPIFY(A, 1, n-1)
6.
return max
remakes heap
Running time: O(lgn)
CS 477/677
11
Example: HEAP-EXTRACT-MAX
16
14
7
4
max = 16
10
8
2
1
9
14
3
10
8
1
2
7
9
3
4
Heap size decreased with 1
14
Call MAX-HEAPIFY(A, 1, n-1)
8
4
2
10
7
9
3
1
CS 477/677
12
HEAP-INCREASE-KEY
• Goal:
– Increases the key of an element i in the heap
• Idea:
– Increment the key of A[i] to its new value
– If the max-heap property does not hold anymore:
traverse a path toward the root to find the proper
place for the newly increased key
16
14
Key [i] ← 15 2
8 i
4
10
7
9
3
1
CS 477/677
13
HEAP-INCREASE-KEY
Alg: HEAP-INCREASE-KEY(A, i, key)
1.
2.
3.
4.
5.
6.
•
if key < A[i]
then error “new key is smaller than current key”
A[i] ← key
16
while i > 1 and A[PARENT(i)] < A[i]
do exchange A[i] ↔ A[PARENT(i)]
14
i ← PARENT(i)
2
Running time: O(lgn)
8 i
4
7
10
9
3
1
Key [i] ← 15
CS 477/677
14
Example: HEAP-INCREASE-KEY
16
14
2
8 i
4
16
10
7
9
14
3
1
2
10
8 i
7
15 1
9
3
Key [i ] ← 15
16
i
15
2
14
i
15
10
7
8
16
9
3
10
14
1
2
CS 477/677
7
8
9
3
1
15
MAX-HEAP-INSERT
• Goal:
16
– Inserts a new element into a maxheap
14
10
8
• Idea:
– Expand the max-heap with a new
element whose key is -
– Calls HEAP-INCREASE-KEY to
set the key of the new node to its
correct value and maintain the
max-heap property
CS 477/677
2
7
4
1
9
3
-
16
14
10
8
2
7
4
1
9
3
15
16
MAX-HEAP-INSERT
16
Alg: MAX-HEAP-INSERT(A, key, n)
14
1. heap-size[A] ← n + 1
8
2
2. A[n + 1] ← -
10
7
4
1
9
3
-
3. HEAP-INCREASE-KEY(A, n + 1, key)
Running time: O(lgn)
CS 477/677
17
Example: MAX-HEAP-INSERT
Insert value 15:
- Start by inserting -
Increase the key to 15
Call HEAP-INCREASE-KEY on A[11] = 15
16
16
14
8
2
7
4
14
10
1
9
8
3
2
-
10
7
4
1
9
3
15
The restored heap containing
the newly added element
16
16
14
8
2
15 9
4
15
10
1
7
10
8
3
2
CS 477/677
14 9
4
1
3
7
18
Summary
• We can perform the following operations on
heaps:
– MAX-HEAPIFY
O(lgn)
– BUILD-MAX-HEAP
O(n)
– HEAP-SORT
O(nlgn)
– MAX-HEAP-INSERT
O(lgn)
– HEAP-EXTRACT-MAX
O(lgn)
– HEAP-INCREASE-KEY
O(lgn)
– HEAP-MAXIMUM
O(1)
CS 477/677
19
The Search Problem
• Find items with keys matching a given search key
Applications:
• Keeping track of customer account information at
a bank
– Search through records to check balances and perform
transactions
• Keep track of reservations on flights
– Search to find empty seats, cancel/modify reservations
• Search engine
– Looks for all documents containing a given word
CS 477/677
20
Symbol Tables (Dictionaries)
• Dictionary = data structure that supports two
basic operations: insert a new item and return
an item with a given key
• Queries: return information about the set
– Search (S, k)
– Minimum (S), Maximum (S)
– Successor (S, x), Predecessor (S, x)
• Modifying operations: change the set
– Insert (S, k)
– Delete (S, k)
CS 477/677
21
Implementations of Symbol Tables
• Key-indexed-array
– Key values are distinct, small numbers
– Store the items in an array, indexed by keys
• Ordered/unordered arrays
• Ordered/unordered linked lists
Insert
key-indexed array
ordered array
ordered list
unordered array
unordered list
1
N
N
1
1
CS 477/677
Search
1
N
N
N
N
22
Binary Search Trees
• Support many dynamic set operations
– SEARCH, MINIMUM, MAXIMUM, PREDECESSOR,
SUCCESSOR, INSERT
• Running time of basic operations on binary
search trees
– On average: (lgn)
• The expected height of the tree is lgn
– In the worst case: (n)
• The tree is a linear chain of n nodes
CS 477/677
23
Binary Search Trees
• Tree representation:
– A linked data structure in which each
node is an object
• Node representation:
–
–
–
–
–
Key field
Satellite data
Left: pointer to left child
Right: pointer to right child
p: pointer to parent (p [root [T]] =
NIL)
L
parent
key data
Left child
R
Right child
• Satisfies the binary-search-tree
property
CS 477/677
24
Binary Search Tree Example
• Binary search tree property:
– If y is in left subtree of x,
then key [y] ≤ key [x]
– If y is in right subtree of x,
then key [y] ≥ key [x]
CS 477/677
5
3
2
7
5
9
25
Traversing a Binary Search Tree
•
Inorder tree walk:
– Prints the keys of a binary tree in sorted order
– Root is printed between the values of its left and
right subtrees: left, root, right
•
Preorder tree walk:
– root printed first: root, left, right
•
Postorder tree walk: left, right, root
– root printed last
Inorder: 2 3 5 5 7 9
5
3
2
Preorder: 5 3 2 5 7 9
7
5
9
Postorder: 2 5 3 9 7 5
CS 477/677
26
Traversing a Binary Search Tree
Alg: INORDER-TREE-WALK(x)
1. if x  NIL
2.
then INORDER-TREE-WALK ( left [x] )
3.
print key [x]
4.
INORDER-TREE-WALK ( right [x] )
• E.g.:
5
3
2
Output: 2 3 5 5 7 9
7
5
9
• Running time:
– (n), where n is the size of the tree rooted at x
CS 477/677
27
Searching for a Key
•
Given a pointer to the root of a tree and a key k:
–
Return a pointer to a node with key k
if one exists
–
•
5
3
Otherwise return NIL
2
7
4
9
Idea
–
Starting at the root: trace down a path by comparing k with the
key of the current node:
•
If the keys are equal: we have found the key
•
If k < key[x] search in the left subtree of x
•
If k > key[x] search in the right subtree of x
CS 477/677
28
Searching for a Key
Alg: TREE-SEARCH(x, k)
5
1. if x = NIL or k = key [x]
3
7
2
4
9
2.
then return x
3. if k < key [x]
4.
then return TREE-SEARCH(left [x], k )
5.
else return TREE-SEARCH(right [x], k )
Running Time: O (h),
h – the height of the tree
CS 477/677
29
Example: TREE-SEARCH
15
• Search for key 13:
6
3
2
– 15  6  7  13
18
7 17
4
20
13
9
CS 477/677
30
Iterative Tree Search
Alg: ITERATIVE-TREE-SEARCH(x, k)
1. while x  NIL and k  key [x]
2.
do if k < key [x]
3.
then x  left [x]
4.
else x  right [x]
5. return x
CS 477/677
31
Finding the Minimum in a Binary
Search Tree
•
Goal: find the minimum value in a BST
– Following left child pointers from the root,
until a NIL is encountered
15
Alg: TREE-MINIMUM(x)
1. while left [x]  NIL
2.
do x ← left [x]
3. return x
6
3
2
18
7 17
4
20
13
9
Minimum = 2
Running time: O(h), h – height of tree
CS 477/677
32
Finding the Maximum in a Binary
Search Tree
•
Goal: find the maximum value in a BST
– Following right child pointers from the root,
until a NIL is encountered
15
Alg: TREE-MAXIMUM(x)
1. while right [x]  NIL
2.
do x ← right [x]
3. return x
6
3
2
18
7 17
4
20
13
9
Maximum = 20
•
Running time: O(h), h – height of tree
CS 477/677
33
Successor
Def: successor (x ) = y, such that key [y] is the
smallest key > key [x]
• E.g.: successor (15) = 17
6
successor (13) = 15
3
successor (9) = 13
2
• Case 1: right (x) is non empty
15
18
7 17
4
20
13
9
– successor (x ) = the minimum in right (x)
• Case 2: right (x) is empty
– go up the tree until the current node is a left child:
successor (x ) is the parent of the current node
– if you cannot go further (and you reached the root):
x is the largest element
CS 477/677
34
Finding the Successor
Alg: TREE-SUCCESSOR(x)
1.
2.
3.
4.
5.
6.
7.
if right [x]  NIL
then return TREE-MINIMUM(right [x])
y ← p[x]
while y  NIL and x = right [y]
6
do x ← y
3
y ← p[y]
2
4
return y
15
y
7 17
13
18
20
x
9
Running time: O (h), h – height of the tree
CS 477/677
35
Predecessor
Def: predecessor (x ) = y, such that key [y] is the
biggest key < key [x]
15
• E.g.: predecessor (15) = 13
predecessor (9) = 7
6
predecessor (13) = 9
3
• Case 1: left (x) is non empty
2
18
7 17
4
– predecessor (x ) = the maximum in left (x)
20
13
9
• Case 2: left (x) is empty
– go up the tree until the current node is a right child:
predecessor (x ) is the parent of the current node
– if you cannot go further (and you reached the root):
x is the smallest element
CS 477/677
36
Insertion
• Goal:
– Insert value v into a binary search tree
• Idea:
– If key [x] < v move to the right child of x,
Insert value 13
else move to the left child of x
12
– When x is NIL, we found the correct position
– If v < key [y] insert the new node as y’s left child
else insert it as y’s right child
5
2
1
18
9 15
3
13
19
17
– Begining at the root, go down the tree and maintain:
• Pointer x : traces the downward path (current node)
• Pointer y : parent of x (“trailing pointer” )
CS 477/677
37
Example: TREE-INSERT
Insert 13:
x, y=NIL
12
y
12
x
5
2
1
18
9 15
3
5
19
18
2
1
17
9 15
3
17
12
12
5
2
1
x
y
18
9 15
3
19
5
2
19
1
17
18
9 15
3
13
19
17
x = NIL
y = 15
CS 477/677
38
Alg: TREE-INSERT(T, z)
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
y ← NIL
x ← root [T]
while x ≠ NIL
do y ← x
if key [z] < key [x]
then x ← left [x]
else x ← right [x]
p[z] ← y
if y = NIL
then root [T] ← z
else if key [z] < key [y]
then left [y] ← z
else right [y] ← z
CS 477/677
12
5
2
1
18
9 15
3
13
19
17
Tree T was empty
Running time: O(h)
39
Readings
• Chapter 6
• Chapter 12
CS 477/677
40