Splay Trees, Fibonacci Heaps,
Persistent Data Structures
1

Splay Trees
Fibonacci Heaps
Persistent Data Structures:
Summary
2

SOURCES:
Splay Trees
Base slides from: David Kaplan, Dept of Computer Science & Engineering,
Autumn 2001
CS UMD Lecture 10 Splay Tree
UC Berkeley 61B Lecture 34 Splay Tree
Fibonacci Heap
Lecture slides adapted from:
 Chapter 20 of Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein.
 Chapter 9 of The Design and Analysis of Algorithms by Dexter Kozen.
Persistent Data Structure
Some of the slides are adapted from:
 http://electures.informatik.uni-freiburg.de
3

Pre-knowledge: Amortized Cost Analysis

Amortized Analysis
– Upper bound, for example, O(log n)
– Overall cost of a arbitrary sequences
– Picking a good “credit” or “potential” function

Potential Function: a function that maps a data structure onto a real valued, nonnegative “potential”
–
High potential state is volatile, built on cheap operation
– Low potential means the cost is equal to the amount allocated to it
Amortized Time = sum of actual time + potential change
4

Splay Tree
Muthu Kumar C.
Xie Shudong
CS6234 Advanced Algorithms

Balanced Binary Search Trees
Unbalanced binary search tree Balanced binary search tree x
Balancing by rotations
A B
Rotations preserve the BST property y
C
Zig
A x
B y
C
6

Problems with AVL Trees

Extra storage/complexity for height fields

Ugly delete code
Solution: Splay trees (Sleator and Tarjan in 1985)

Go for a tradeoff by not aiming at balanced trees always.




Splay trees are self-adjusting BSTs that have the additional helpful property that more commonly accessed nodes are more quickly retrieved.
Blind adjusting version of AVL trees.
Amortized time (average over a sequence of inputs) for all operations is O(log n).
Worst case time is O(n).
7

10
You’re forced to make a really deep access:
17
Since you’re down there anyway, fix up a lot of deep nodes!
5
2
3
9
Why splay?
This brings the most recently accessed nodes up towards the root.
8

Bring the node being accessed to the root of the tree, when accessing it, through one or more splay steps .
A splay step can be:
Zig
Zig-zig
Zig-zag
Zag Single rotation
Zag-zig
Zag-zag
Double rotations
9

Node being accessed (n) is:


 the root a child of the root
Do single rotation: Zig or Zag pattern has both a parent (p) and a grandparent (g)
Double rotations:
(i) Zig-z i g or Zag-zag pattern: g  p  n is left-left or right-right
(ii) Zig-z a g pattern: g  p  n is left-right or right-left
10

root n
X Y root n
X Y
11

X n root p Zig – right rotation
Z
Zag – left rotation
X root n p
Y Y Z
12

Zig root p n
Z
X Y
13

g
X
Y n p
W n g p
X Y Z W
Z
14

g p
Zig
X n
W
Y Z
15

Zag g n
X p
Y
Z W
16

W g
1
2 p p n g
X Y
Y Z W X
No more cookies! We are done showing animations.
n
Z
17

Do nothing | Single rotation | Double rotation cases
Zig y x z n
A B x
C A y y x
D
Zig-Zag y x z
A B
B C A
B C
A B C D
18

6
1 1
1
2 2 zag
3
4
3 zags
4
2
3
5
6
6
4
5
Tree still unbalanced.
No change in height!
5
19

1
2
1 1
6
2 2
1
3
3 3
3
4 5
6
2 5
5 4 6
5
4
6
4
20

If a node n on the access path, to a target node say x, is at depth d before splaying x, then it’s at depth <= 3+d/2 after the splay. (Proof in Goodrich and Tamassia)
Overall, nodes which are below nodes on the access path tend to move closer to the root
Splaying gets amortized to give O(log n) performance. (Maybe not now, but soon, and for the rest of the operations.)
21

Find the node in normal BST manner
Note that we will always splay the last node on the access path even if we don’t find the node for the key we are looking for.
Splay the node to the root
Using 3 cases of rotations we discussed earlier
22

Find( 6 )
1
2
3
1
2 zag-zag
3
4 6
5 5
6 4
23

1
2
4
5
3
6
zag-zag
1
6
3
2 5
4
24

1
6
3
2 5
4
zag
6
1
3
2 5
4
25

Can we just do BST insert?
Yes. But we also splay the newly inserted node up to the root.
Alternatively, we can do a Split(T,x)
26

Split(T, x) creates two BSTs L and R:
 all elements of T are in either L or R ( T = L  R )
 all elements in L are  x

 all elements in R are  x
L and R share no elements ( L  R =  )
27

How can we split?

We can do Find(x), which will splay x to the root.

Now, what’s true about the left subtree L and right subtree R of the root?

So, we simply cut the tree at x, attach x either L or R
28

split(x)
T splay
L
R
L
 x
R
> x
OR
L
< x
R
 x
29

split(x)
L
< x
R
> x x
L R
30

4
1
6
4 7
9 split(5) 1
2
6
9
7
2
Insert( 5 )
1
2
4
4
1
5
6
7
6
9
9
2 7
31

Do a BST style delete and splay the parent of the deleted node. Alternatively, find(x)
L x
R delete (x)
L
< x
R
> x
32

Join(L, R): given two trees such that L < R, merge them
L splay
L R
R
Splay on the maximum element in L, then attach R
33

T find(x)
L x
R delete x
L
< x
R
> x
Join(L,R)
T - x
34

4
6
1 6
1 9 find(4)
2
4 7
7
2
Delete( 4 )
Compare with BST/AVL delete on ivle
1
2
6
9
7
9
1
2
Find max
1
2
7
6
7
6
9
9
35

2
Bottom-up Top Down
A x
B y
C
Zig
A x y
B C
L x
A B y
C
R
Zig
L
A x
B y
R
C
Why top-down?
Bottom-up splaying requires traversal from root to the node that is to be splayed, and then rotating back to the root – in other words, we make 2 tree traversals. We would like to eliminate one of these traversals.
1
How? time analysis.. We may discuss on ivle.
1. http://www.csee.umbc.edu/courses/undergraduate/341/fall02/Lectures/Splay/ TopDownSplay.ppt
36

Splay Trees: Amortized Cost Analysis
• Amortized cost of a single splay-step
• Amortized cost of a splay operation: O(logn)
• Real cost of a sequence of m operations: O((m+n) log n)
CS6234 Advanced Algorithms

Splay Trees: Amortized Cost Analysis
CS6234 Advanced Algorithms

Splay Trees Amortized Cost Analysis
Amortized cost of a single splay-step
Lemma 1: For a splay-step operation on x that transforms the rank function r into r’, the amortized cost is:
(i) a i
(ii) a i
≤ 3(r’(x) − r(x)) + 1 if the parent of x is the root, and
≤ 3(r’(x) − r(x)) otherwise.
x y
Zig x y y z x
Zig-Zag y x z
CS6234 Advanced Algorithms

Splay Trees Amortized Cost Analysis
Lemma 1: root, and
(i) a i
≤ 3(r’(x) − r(x)) + 1 if the parent of x is the
Proof : ≤ 3(r’(x) − = c +
φ
’
− φ three cases of splay-step operations (zig/zag, zigzig/zagzag, and zigzag/zagzig).
Case 1 (Zig / Zag) : The operation involves exactly one rotation.
Real cost c i
= 1 y x x
Zig y
CS6234 Advanced Algorithms

Splay Trees Amortized Cost Analysis
Amortized cost is a i
− φ
= 1 +
φ
’
In this case, we have r’(x)= r(y), r’(y) ≤ r’(x) and r’(x) ≥ r(x).
So the amortized cost: a i
= 1 +
φ
’ − φ
= 1 + r’(x) + r’(y) − r(x) − r(y)
= 1 + r’(y) − r(x)
≤ 1 + r’(x) − r(x)
≤ 1 + 3(r’(x) − r(x)) y x x
Zig y
CS6234 Advanced Algorithms

Splay Trees Amortized Cost Analysis
Lemma 1: root, and
(i) a i
≤ 3(r’(x) − r(x)) + 1
(ii) a i
≤ 3(r’(x) − r(x)) otherwise.
if the parent of x is the
The proofs of the rest of the cases, zig-zig pattern and zig-zag/zagzig patterns, are similar resulting in amortized cost of a i r(x))
≤ 3(r’(x) −
Zig z y x y x x y x
Zig-Zag y z
CS6234 Advanced Algorithms

Splay Trees Amortized Cost Analysis
Amortized cost of a splay operation: O(logn)
Building on Lemma 1 (amortized cost of splay step),
Zig z y x y x y Zig-Zag x y x z
We proceed to calculate the amortized cost of a complete splay operation.
Lemma 2: The amortized cost of the splay operation on a node x in a splay tree is O(log n) .
CS6234 Advanced Algorithms

Splay Trees Amortized Cost Analysis x y
Zig x y y z x
Zig-Zag y x z
CS6234 Advanced Algorithms

Splay Trees Amortized Cost Analysis
Theorem: For any sequence of m operations on a splay tree containing at most n keys, the total real cost is O((m + n)log n) .
Proof: Let a i be the amortized cost of the i-th operation. Let c i real cost of the i-th operation. Let
φ
0 be the be the potential before and
φ m be the potential after the m operations. The total cost of m operations is:
( From )
We also have
φ
0
− φ m
≤ n log n, since r(x) ≤ log n. So we conclude:
CS6234 Advanced Algorithms

Range Removal [7, 14]
3
5
10
17
6 13
8 16
22
7 9
Find the maximum value within range (-inf, 7), and splay it to the root.
CS6234 Advanced Algorithms

Range Removal [7, 14]
6
5 10
8
17
3
7 9
13 22
16
Find the minimum value within range (14, +inf), and splay it to the root of the right subtree.
CS6234 Advanced Algorithms

Range Removal [7, 14]
3
5
7
6
8
16
10
9
13
[7, 14]
17
22
Cut off the link between the subtree and its parent.
CS6234 Advanced Algorithms

Find
Insert
Delete
Range Removal
Memory
Implementation
AVL
O(log n)
O(log n)
O(log n)
O(nlog n)
More Memory
Complicated
Splay
Amortized O(log n)
Amortized O(log n)
Amortized O(log n)
Amortized O(log n)
Less Memory
Simple
58

Can be shown that any M consecutive operations starting from an empty tree take at most O(M log(N))
 All splay tree operations run in amortized O(log n) time
O(N) operations can occur, but splaying makes them infrequent
Implements most-recently used (MRU) logic

Splay tree structure is self-tuning
59

Splaying can be done top-down; better because:
 only one pass
 no recursion or parent pointers necessary
Splay trees are very effective search trees
 relatively simple: no extra fields required
 excellent locality properties:
– frequently accessed keys are cheap to find (near top of tree)
– infrequently accessed keys stay out of the way (near bottom of tree)
60

Fibonacci Heaps
Agus Pratondo
Aleksanr Farseev
CS6234 Advanced Algorithms

Fibonacci Heaps: Motivation
It was introduced by Michael L. Fredman and Robert E. Tarjan in
1984 to improve Dijkstra's shortest path algorithm from
O
( E log V ) to
O
( E + V log V ).
62

Fibonacci Heaps: Structure
Fibonacci heap.

Set of heap-ordered trees.

 each parent < its children
Maintain pointer to minimum element.
Set of marked nodes.
roots
Heap H
17
30
24
26 46
35
23 7 heap-ordered tree
3
18
39
52 41
44
63

Fibonacci Heaps: Structure
Fibonacci heap.

Set of heap-ordered trees.


Maintain pointer to minimum element.
Set of marked nodes.
find-min takes O(1) time
Heap H
17
30
24
26 46
35
23 7 min
3
18
39
52 41
44
64

Fibonacci Heaps: Structure
Fibonacci heap.

Set of heap-ordered trees.


Maintain pointer to minimum element.
Set of marked nodes.
• True if the node lost its child, otherwise it is false
• Use to keep heaps flat
• Useful in decrease key operation min
23
Heap H
17
30
24
26 46
35
7 3 marked
18
39
52 41
44
65

Fibonacci Heap vs. Binomial Heap
Fibonacci Heap is similar to Binomial Heap , but has a less rigid structure
 the heap is consolidate after the delete-min method is called instead of actively consolidating after each insertion
.....This is called a “lazy” heap”....
min
66

Fibonacci Heaps: Notations
Notations in this slide
 n = number

 rank(x) rank(H)
= number
= max rank

 trees(H) = number marks(H) = number of nodes in heap.
of children of node x.
of any node in heap H.
of trees in heap H.
of marked nodes in heap H.
trees(H) = 5 marks(H) = 3 n = 14 rank = 3 min
Heap H
17
30
24
26 46
35
23 7 3 marked
18
39
52 41
44
67

Fibonacci Heaps: Potential Function
 (H)
=
trees(H) + 2  marks(H) potential of heap H
Heap H trees(H) = 5 marks(H) = 3
17
30
24
26 46
35
23
 (H) = 5 + 2  3 = 11 min
7 3 marked
18
39
52 41
44
68

69

Fibonacci Heaps: Insert
Insert.

Create a new singleton tree.

Add to root list; update min pointer (if necessary).
insert 21
21
Heap H
17
30
24
26 46
35
23 7 min
3
18
39
52 41
44
70

Fibonacci Heaps: Insert
Insert.

Create a new singleton tree.

Add to root list; update min pointer (if necessary).
insert 21
Heap H
17
30
24
26 46
35
23 7 min
21 3
18
39
52 41
44
71

Fibonacci Heaps: Insert Analysis
Actual cost. O(1)
 (H)
=
trees(H) + 2  marks(H)
Change in potential. +1
Amortized cost. O(1) potential of heap H min
Heap H
17
30
24
26 46
35
23 7 21 3
18
39
52 41
44
72

73

Linking Operation
Linking operation. Make larger root be a child of smaller root.
56
77 tree T
1
24 larger root
15 3 smaller root
18
39
52 41
44 tree T
2
74

Linking Operation
Linking operation. Make larger root be a child of smaller root.
15 is larger than 3
 Make ‘15’ be a child of ‘3’ larger root smaller root
15 3
56
77 tree T
1
24 18
39
52 41
44 tree T
2
75

Linking Operation
Linking operation. Make larger root be a child of smaller root.
15 is larger than 3
 Make ‘15’ be a child of ‘3 larger root smaller root still heap-ordered
15 3 3
56
77 tree T
1
24 18
39
52 41
44 tree T
2
56
77
15
24
18
39
52 41
44 tree T'
76

77

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
7 24
30 26 46
35
23 17 min
3
18
39
52 41
44
78

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
min
7 24
30 26 46
35
23 17 18
39
52 41
44
79

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
min
7 current
24
30 26 46
35
23 17
39
52 41
44
80

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 min
7 current
24 23 17 18
39
52
30 26 46
35
41
44
81

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 min
7 current
24 23 17 52
39
30 26 46
35
41
44
82

min
Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3
7 24 23
30 26 46
35 current
17
39
52 41
44
83

min
Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3
7 24
30 26 46
35
23 current
17
39
52 41
44 link 23 into 17
84

min
Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3
7 24
30 26 46
35
17 current 23 39
52 41
44 link 17 into 7
85

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3
24
26 46
35 min
17 30
23
7 current
39
52 link 24 into 7
41
44
86

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 min
24 17 30
26 46
35
23
7 current
39
52 41
44
87

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 current
52 min
7
24 17 30
26 46
35
23
39
41
44
88

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 min
7
24 17 30
26 46
35
23
39
52 current
41
44
89

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 min
7
24 17 30
26 46
35
23
39
52 current
41
44 link 41 into 18
90

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 min
7
24 17 30
26 46
35
23
52 current
41 39
44
91

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
rank
0 1 2 3 min
7
24 17 30
26 46
35
23
52 current
41 39
44
92

Fibonacci Heaps: Delete Min
Delete min.

Delete min; meld its children into root list; update min.

Consolidate trees so that no two roots have same rank.
min
7
24 17 30
26 46
35
23 stop
52
41 39
44
18
93

Fibonacci Heaps: Delete Min Analysis
Delete min.
 (H)
=
trees(H) + 2  marks(H) potential function
Actual cost. O(rank(H)) + O(trees(H))

O(rank(H)) to meld min's children into root list.


O(rank(H)) + O(trees(H)) to update min.
O(rank(H)) + O(trees(H)) to consolidate trees.
Change in potential. O(rank(H)) - trees(H)

trees(H' )  rank(H) + 1 since no two trees have same rank.

 (H)  rank(H) + 1 - trees(H).
Amortized cost. O(rank(H))
94

95

Fibonacci Heaps: Decrease Key
Intuition for deceasing the key of node x.

If heap-order is not violated, just decrease the key of x.


Otherwise, cut tree rooted at x and meld into root list.
To keep trees flat: as soon as a node has its second child cut, cut it off and meld into root list (and unmark it).
min
7 18 marked node: one child already cut
24 17 23 21 39
26 46 30
35 88 72
52
38
41
96

Fibonacci Heaps: Decrease Key
Case 1. [heap order not violated]

Decrease key of x.

Change heap min pointer (if necessary).
min
7 38
24 17 23 21 39
35
26
88 72 x
30 52
41 decrease-key of x from 46 to 29
97

Fibonacci Heaps: Decrease Key
Case 1. [heap order not violated]

Decrease key of x.

Change heap min pointer (if necessary).
min
7 38
24 17 23 21 39
35
26
88
29 x
72
30 52
41 decrease-key of x from 46 to 29
98

Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
min
7 38
35
26
88
24 p
17
30 x
72
23 21
52
39 41 decrease-key of x from 29 to 15
99

Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
min
7 38
35
26
88
24 p
17
15 x
72
30
23 21
52
39 41 decrease-key of x from 29 to 15
100

Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
x
15 7 min
38
72
35
26
24 p
17
30
23
88
21 39 41
52 decrease-key of x from 29 to 15
101

Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
x
15 7 min
38
72 mark parent
26
35 88 p
17
30
23 21 39 41
52 decrease-key of x from 29 to 15
102

Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
min
15 7 38
72 x p 26
88
17 23
30
21 39 41
52 decrease-key of x from 35 to 5
103

Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
min
15 7 38
72 p 26 x 5 88
17 23
30
21 39 41
52 decrease-key of x from 35 to 5
104

Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
15 min x
5 7 38
72 p 26
88
17 23
30
21 39 41
52 decrease-key of x from 35 to 5
105

Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
15 min x
5 7 38
72 second child cut p 26
88
17 23
30
21 39 41
52 decrease-key of x from 35 to 5
106

Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
15 min x
5 p
26 7 38
72 88 17 23
30
21 39 41
52 decrease-key of x from 35 to 5
107

Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
15 min x
5 p
26 7 38
72 88 p' second child cut
17 23
30
21 39 41
52 decrease-key of x from 35 to 5
108

Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated]

Decrease key of x.


Cut tree rooted at x, meld into root list, and unmark.
If parent p of x is unmarked (hasn't yet lost a child), mark it;
Otherwise, cut p, meld into root list, and unmark
(and do so recursively for all ancestors that lose a second child).
15 min x
5 p
26 p'
24 p''
7 38
72 88 don't mark parent if it's a root
17
30
23 21 39 41
52 decrease-key of x from 35 to 5
109

Fibonacci Heaps: Decrease Key Analysis
Decrease-key.
 (H)
=
trees(H) + 2  marks(H) potential function
Actual cost. O(c)

O(1) time for changing the key.

O(1) time for each of c cuts, plus melding into root list.
Change in potential. O(1) - c



trees(H') = trees(H) + c.
marks(H')  marks(H) - c + 2.
  c + 2  (-c + 2) = 4 - c.
Amortized cost. O(1)
110

111

Fibonacci Heaps: Bounding the Rank
Lemma. Fix a point in time. Let x be a node, and let y
1
, …, y k denote its children in the order in which they were linked to x. Then: x rank ( y i
) 


0 if i  1 i  2 if i  1 y
1 y
2
… y k

Def. Let F k be smallest possible tree of rank k satisfying property.
F
0
F
1
F
2
F
3
F
4
F
5
1 2 3 5 8 13
112

Fibonacci Heaps: Bounding the Rank
Lemma. Fix a point in time. Let x be a node, and let y
1
, …, y k denote its children in the order in which they were linked to x. Then: x rank ( y i
) 


0 if i  1 i  2 if i  1 y
1 y
2
… y k

Def. Let F k be smallest possible tree of rank k satisfying property.
F
4
F
5
F
6
8 13 8 + 13 = 21
113

Fibonacci Heaps: Bounding the Rank
Lemma. Fix a point in time. Let x be a node, and let y
1
, …, y k denote its children in the order in which they were linked to x. Then: x rank ( y i
) 


0 if i  1 i  2 if i  1 y
1 y
2
… y k

Def. Let F k be smallest possible tree of rank k satisfying property.
Fibonacci fact. F k
  k , where  = (1 +  5) / 2  1.618.
Corollary. rank(H)  log

n .
golden ratio
114

115

Fibonacci Numbers: Exponential Growth
Def. The Fibonacci sequence is: 0, 1, 1, 2, 3, 5, 8, 13, 21, …
F k

0



1
F k 1

F k 2 if k

0 if k if k

1 , 2

3
116

117

Fibonacci Heaps: Union
Union. Combine two Fibonacci heaps.
Representation. Root lists are circular, doubly linked lists.
23 24
Heap H'
30 26 46
35 min
17 min
3 7
Heap H''
18
39
52 41
44
21
118

Fibonacci Heaps: Union
Union. Combine two Fibonacci heaps.
Representation. Root lists are circular, doubly linked lists.
23 24
30 26 46
35
17
Heap H
7 min
3
18
39
52 41
44
21
119

Fibonacci Heaps: Union
Actual cost. O(1)
Change in potential. 0
Amortized cost. O(1)
 (H)
=
trees(H) + 2  marks(H) potential function
23 24
30 26 46
35
17
Heap H
7 min
3
18
39
52 41
44
21
120

121

Fibonacci Heaps: Delete
Delete node x.

decrease-key of x to  .

delete-min element in heap.
Amortized cost. O(rank(H))

O(1) amortized for decrease-key.

O(rank(H)) amortized for delete-min.
 (H)
=
trees(H) + 2  marks(H) potential function
122

Application:
Priority Queues => ex.Shortest path problem
Operation make-heap is-empty insert delete-min decrease-key delete union find-min
Binomial
Heap
1
1 log n log n log n log n log n log n
n = number of elements in priority queue
† amortized
Fibonacci
Heap †
1
1
1 log n
1 log n
1
1
123

Persistent Data Structures
Li Furong
Song Chonggang
CS6234 Advanced Algorithms

Motivation
Version Control



Suppose we consistently modify a data structure
Each modification generates a new version of this structure
A persistent data structure supports queries of all the previous versions of itself

Three types of data structures
– Fully persistent all versions can be queried and modified
– Partially persistent all versions can be queried, only the latest version can be modified
– Ephemeral only can access the latest version
125

Making Data Structures Persistent
In the following talk, we will

Make pointer-based data structures persistent, e.g., tree

Discussions are limited to partial persistence
Three methods

Fat nodes

Path copying

Node Copying (Sleator, Tarjan et al.)
126

Fat Nodes
Add a modification history to each node value time
1 value time
2

Modification
– append the new data to the modification history, associated with timestamp

Access
– for each node, search the modification history to locate the desired version

Complexity (Suppose m modifications)
Modification
Access
Time
O(1)
O(log m) per node
Space
O(1)
127

Path Copying
Copy the node before changing it
Cascade the change back until root is reached
128

Path Copying
Copy the node before changing it
Cascade the change back until root is reached
0
5
1
3
7 version 0: version 1:
Insert (2) version 2:
Insert (4)
129

Path Copying
Copy the node before changing it
Cascade the change back until root is reached
0
5
1
3 3
7 version 1:
Insert (2)
2
130

Path Copying
Copy the node before changing it
Cascade the change back until root is reached
0
5
1
3 3
2
7 version 1:
Insert (2)
131

Path Copying
Copy the node before changing it
Cascade the change back until root is reached
0 1
5 5
1
1
3 3
2
7 version 1:
Insert (2)
132

Path Copying
Copy the node before changing it
Cascade the change back until root is reached
0 1 2
5 5 5
1
1 1
3 3
3
2 4
7 version 1:
Insert (2) version 2:
Insert (4)
133

Path Copying
Copy the node before changing it
Cascade the change back until root is reached
0 1 2
5 5 5
1
1 1
3 3
3
2 4
7 version 1:
Insert (2) version 2:
Insert (4)


Each modification creates a new root
Maintain an array of roots indexed by timestamps
134

Path Copying
Copy the node before changing it
Cascade the change back until root is reached

Modification
– copy the node to be modified and its ancestors

Access
– search for the correct root, then access as original structure

Complexity (Suppose m modifications, n nodes)
Modification
Access
Time
Worst: O(n)
Average: O(log n)
O(log m)
Space
Worst: O(n)
Average: O(log n)
135

Node Copying
Fat nodes: cheap modification, expensive access
Path copying: cheap access, expensive modification
Can we combine the advantages of them?
Extend each node by a timestamped modification box

A modification box holds at most one modification

When modification box is full, copy the node and apply the modification

Cascade change to the node‘s parent
136

k lp mbox rp
1
3
5
Node Copying
7 version 0 version 1:
Insert (2) version 2:
Insert (4)
137

1
2
3
1 lp
5
Node Copying edit modification box directly like fat nodes
7 version 0: version 1:
Insert (2)
138

Node Copying
5
1
2
3
1 lp
3
1 lp
4 copy the node to be modified
7 version 1:
Insert (2) version 2:
Insert (4)
139

Node Copying
5
1
2
7
3
1 lp
3
4 apply the modification in modification box version 1:
Insert (2) version 2:
Insert (4)
140

1
2
Node Copying
5
7
3
1 lp
3 version 1:
Insert (2) version 2:
Insert (4)
4 perform new modification directly the new node reflects the latest status
141

1
2 rp
2
Node Copying
5
7
3
1 lp
3
4 cascade the change to its parent like path copying version 1:
Insert (2) version 2:
Insert (4)
142

Node Copying

Modification
– if modification box empty, fill it
–
– otherwise, make a copy of the node, using the latest values cascade this change to the node’s parent (which may cause
– node copying recursively) if the node is a root, add a new root

Access
– search for the correct root, check modification box

Complexity (Suppose m modifications)
Time
Modification
Access
Amortized: O(1)
O(log m) +
O(1) per node
Space
Amortized: O(1)
143

Modification Complexity Analysis
Use the potential technique

Live nodes

– Nodes that comprise the latest version
Full live nodes

– live nodes whose modification boxes are full
Potential function f (T)
– number of full live nodes in T (initially zero)


Each modification involves k number of copies
– each with a O(1) space and time cost
– decrease the potential function by 1-> change a full modification box into an empty one
Followed by one change to a modification box (or add a new root)



Δ f = 1-k
Space cost: O(k+
Δ f ) = O(k+1–k) = O(1)
Time cost: O(k+1+
Δ f) = O(1)
144

Applications

Grounded 2-Dimensional Range Searching

Planar Point Location

Persistent Splay Tree
145

Applications: Grounded 2-Dimensional Range Searching

Problem
– Given a set of n points and a query triple (a,b,i)
– Report the set of points (x,y), where a<x<b and y<i y i a b x
146

Applications: Grounded 2-Dimensional Range Searching

Resolution
– Consider each y value as a version , x value as a key
–
–
–
Insert each node in ascending order of y value
Version i contains every point for which y<i
Report all points in version i whose key value is in [a,b]
147

Applications: Grounded 2-Dimensional Range Searching

Resolution
– Consider each y value as a version , x value as a key
–
–
–
Insert each node in ascending order of y value
Version i contains every point for which y<i
Report all points in version i whose key value is in [a,b] i a b


Preprocessing
– Space required O(n) with Node Copying and O(n log n) with Path
Copying
Query time O(log n)
148

Applications: Planar Point Location

Problem
– Suppose the Euclidian plane is divided into polygons by n line
– segments that intersect only at their endpoints
Given a query point in the plane, the Planar Point Location problem is to determine which polygon contains the point
149


Applications: Planar Point Location
Solution
– Partition the plane into vertical slabs by drawing a vertical line
– through each endpoint
Within each slab, the lines are ordered
–
–
Allocate a search tree on the x-coordinates of the vertical lines
Allocate a search tree per slab containing the lines and with each line associate the polygon above it
150


Applications: Planar Point Location
Answer a Query (x,y)
– First, find the appropriate slab
– Then, search the slab to find the polygon slab
151


Applications: Planar Point Location
Simple Implementation
– Each slab needs a search tree, each search tree is not related to
– each other
Space cost is high: O(n) for vertical lines, O(n) for lines in each slab

Key Observation
– The list of the lines in adjacent slabs are related a) The same line b) End and start

Resolution
– Create the search tree for the first slab
– Obtain the next one by deleting the lines that end at the corresponding vertex and adding the lines that start at that vertex
152

1
2
3
First slab
Applications: Planar Point Location
153

1
2
3
First slab
Second slab
Applications: Planar Point Location
154

1
1
2
3
First slab
Second slab
Applications: Planar Point Location
155

1
1
2
3
First slab
Second slab
Applications: Planar Point Location
156

1
2
1
4
5
3
First slab
Second slab
Applications: Planar Point Location
157

1
2
1
4
5
3 3
First slab
Second slab
Applications: Planar Point Location
158

Applications: Planar Point Location

Preprocessing
– 2n insertions and deletions
– Time cost O(n) with Node Copying, O(n log n) with Path Copying

Space cost O(n) with Node Copying, O(n log n) with Path Copying
159



Applications: Splay Tree
Persistent Splay Tree
– With Node Copying, we can access previous versions of the splay tree
Example 0
5
3
1
160



Applications: Splay Tree
Persistent Splay Tree
– With Node Copying, we can access previous versions of the splay tree
Example 0
5
3 splay
1
1
161



Applications: Splay Tree
Persistent Splay Tree
– With Node Copying, we can access previous versions of the splay tree
Example 0 1
5 1
3
3 splay
1
1 5
162

1
3
5
0
Applications: Splay Tree
163

1
3
5
0
Applications: Splay Tree
1
1
1 rp
0
3
1 rp
0
1
1
5
0
164

Summary
Hong Hande
CS6234 Advanced Algorithms

Splay tree

Advantage
– Simple implementation
– Comparable performance
– Small memory footprint
– Self-optimizing

Disadvantage
– Worst case for single operation can be O(n)
– Extra management in a multi-threaded environment
166

Fibonacci Heap

Advantage
– Better amortized running time than a binomial heap
– Lazily defer consolidation until next delete-min

Disadvantage
– Delete and delete minimum have linear running time in the worst case
– Not appropriate for real-time systems
167

Persistent Data Structure

Concept
– A persistent data structure supports queries of all the previous versions of itself

Three methods
– Fat nodes
– Path copying
– Node Copying (Sleator, Tarjan et al.)

Good performance in multi-threaded environments.
168

Key Word to Remember

Splay Tree --- Self-optimizing AVL tree

Fibonacci Heap --- Lazy version of Binomial Heap

Persistent Data Structure --- Extra space for previous version
169