3.3 BALANCED SEARCH TREES
Algorithms
F O U R T H
R O B E R T
S E D G E W I C K
Algorithms, 4th Edition
·
‣ 2-3 search trees
‣ red-black BSTs
‣ B-trees
E D I T I O N
K E V I N
W A Y N E
Robert Sedgewick and Kevin Wayne
·
Copyright © 2002–2012
·
March 4, 2012 1:33:02 PM
Symbol table review
average case
(after N random inserts)
worst-case cost
(after N inserts)
implementation
ordered
iteration?
key
interface
search
insert
delete
search hit
insert
delete
sequential search
(unordered list)
N
N
N
N/2
N
N/2
no
equals()
binary search
(ordered array)
lg N
N
N
lg N
N/2
N/2
yes
compareTo()
BST
N
N
N
1.39 lg N
1.39 lg N
?
yes
compareTo()
goal
log N
log N
log N
log N
log N
log N
yes
compareTo()
Challenge. Guarantee performance.
This lecture. 2-3 trees, left-leaning red-black BSTs, B-trees.
introduced to the world
in COS 226, Fall 2007
2
‣ 2-3 search trees
‣ red-black BSTs
‣ B-trees
3
2-3 tree
Allow 1 or 2 keys per node.
•
•
2-node: one key, two children.
3-node: two keys, three children.
Symmetric order. Inorder traversal yields keys in ascending order.
Perfect balance. Every path from root to null link has same length.
M
3-node
smaller than E
2-node
E J
AC
between E and J
H
R
larger than J
L
P
SX
null link
4
Global properties in a 2-3 tree
Invariants. Maintains symmetric order and perfect balance.
Pf. Each transformation maintains symmetric order and perfect balance.
root
parent is a 3-node
b
a b c
a
left
c
d e
a
a b c
parent is a 2-node
a b c
right
a
middle
b d
d
left
a
a c
b c d
b
a e
right
d
c
a c e
b c d
c
b d e
a b
b
d
a b d
c d e
c
e
Splitting a temporary 4-node in a 2-3 tree (summary)
7
2-3 tree: performance
Perfect balance. Every path from root to null link has same length.
Typical 2-3 tree built from random keys
Tree height.
•
•
•
•
Worst case:
lg N.
[all 2-nodes]
Best case:
log3 N ≈ .631 lg N. [all 3-nodes]
Between 12 and 20 for a million nodes.
Between 18 and 30 for a billion nodes.
Guaranteed logarithmic performance for search and insert.
9
2-3 tree: implementation?
Direct implementation is complicated, because:
•
•
•
•
Maintaining multiple node types is cumbersome.
Need multiple compares to move down tree.
Need to move back up the tree to split 4-nodes.
Large number of cases for splitting.
Bottom line. Could do it, but there's a better way.
11
‣ 2-3 search trees
‣ red-black BSTs
‣ B-trees
12
Left-leaning red-black BSTs (Guibas-Sedgewick 1979 and Sedgewick 2007)
3-node
1. Represent 2–3 tree as a BST.
−black tree 2.
less forbetween
Use "internal"Mleft-leaning links as "glue"
3–nodes.greater
J
C
H
a bP
L
less
than a
A
than a
R
3-node
E
between
a and b
b
M
J
E
C
H
less
than a
greater
than b
less
than a
R
P
L
X
M
J
R
E J
L
black links connect
2-nodes and 3-nodes
red−black tree
M
H
between
a and b
red links "glue"
nodes within a 3-node
Encoding a 3-node with two 2-nodes
connected by a left-leaning red link
tree
A C
S
greater
than b
Encoding a 3-node with two 2-nodes
connected by a left-leaning red link
greater
than b
between
a and b
larger key is root
a
S
a
zontal red links
than b
a and b
X
b
A
a b
P
E
C
S X
R
P
L
X
S
H
A
2-3 tree
corresponding red-black BST
−1 correspondence between red-black and 2-3 trees
horizontal red links
M
E
J
13
R
An equivalent definition
A BST such that:
•
•
•
No node has two red links connected to it.
Every path from root to null link has the same number of black links.
Red links lean left.
"perfect black balance"
red−black tree
M
J
E
C
R
P
L
X
S
H
A
horizontal red links
M
J
E
A
C
H
R
L
P
S
X
14
Left-leaning red-black BSTs: 1-1 correspondence with 2-3 trees
Key property. 1–1 correspondence between 2–3 and LLRB.
red−black tree
M
J
E
C
R
P
L
X
S
H
A
horizontal red links
M
J
E
A
C
H
R
P
L
2-3 tree
S
X
M
R
E J
A C
H
L
P
S X
1−1 correspondence between red-black and 2-3 trees
15
Search implementation for red-black BSTs
Observation. Search is the same as for elementary BST (ignore color).
but runs faster because of better balance
public Val get(Key key)
{
Node x = root;
while (x != null)
{
int cmp = key.compareTo(x.key);
if
(cmp < 0) x = x.left;
else if (cmp > 0) x = x.right;
else if (cmp == 0) return x.val;
}
return null;
}
red−black tree
M
J
E
C
R
P
L
X
S
H
A
horizontal red links
M
J
E
A
C
H
R
P
L
2-3 tree
S
X
M
Remark. Most other ops (e.g., ceiling, selection, iteration)
R
E Jare also identical.
A C
H
L
P
S X
16
Red-black BST representation
Each node is pointed to by precisely one link (from its parent) ⇒
can encode color of links in nodes.
private static final boolean RED
= true;
private static final boolean BLACK = false;
private class Node
{
Key key;
Value val;
Node left, right;
boolean color;
// color of parent link
}
private boolean isRed(Node x)
{
if (x == null) return false;
return x.color == RED;
}
null links are black
h
h.left.color
is RED
E
C
A
J
D
h.right.color
is BLACK
G
private static final boolean RED
= true;
private static final boolean BLACK = false;
private class Node
{
Key key;
Value val;
Node left, right;
int N;
boolean color;
//
//
//
//
//
//
key
associated data
subtrees
# nodes in this subtree
color of link from
parent to this node
Node(Key key, Value val)
{
this.key
= key;
this.val
= val;
17
Elementary red-black BST operations
Left rotation. Orient a (temporarily) right-leaning red link to lean left.
rotate E left
(before)
h
E
S
x
less
than E
between
E and S
greater
than S
private Node rotateLeft(Node h)
{
assert isRed(h.right);
Node x = h.right;
h.right = x.left;
x.left = h;
x.color = h.color;
h.color = RED;
return x;
}
Invariants. Maintains symmetric order and perfect black balance.
18
Elementary red-black BST operations
Left rotation. Orient a (temporarily) right-leaning red link to lean left.
rotate E left
(after)
S
h
x
E
greater
than S
less
than E
between
E and S
private Node rotateLeft(Node h)
{
assert isRed(h.right);
Node x = h.right;
h.right = x.left;
x.left = h;
x.color = h.color;
h.color = RED;
return x;
}
Invariants. Maintains symmetric order and perfect black balance.
19
Elementary red-black BST operations
Right rotation. Orient a left-leaning red link to (temporarily) lean right.
rotate S right
(before)
S
x
h
E
greater
than S
less
than E
between
E and S
private Node rotateRight(Node h)
{
assert isRed(h.left);
Node x = h.left;
h.left = x.right;
x.right = h;
x.color = h.color;
h.color = RED;
return x;
}
Invariants. Maintains symmetric order and perfect black balance.
20
Elementary red-black BST operations
Right rotation. Orient a left-leaning red link to (temporarily) lean right.
rotate S right
(after)
x
E
S
h
less
than E
between
E and S
greater
than S
private Node rotateRight(Node h)
{
assert isRed(h.left);
Node x = h.left;
h.left = x.right;
x.right = h;
x.color = h.color;
h.color = RED;
return x;
}
Invariants. Maintains symmetric order and perfect black balance.
21
Elementary red-black BST operations
Color flip. Recolor to split a (temporary) 4-node.
flip colors
(before)
h
E
A
less
than A
S
between
A and E
between
E and S
greater
than S
private void flipColors(Node h)
{
assert !isRed(h);
assert isRed(h.left);
asset isRed(h.right);
h.color = RED;
h.left.color = BLACK;
h.right.color = BLACK;
}
Invariants. Maintains symmetric order and perfect black balance.
22
Elementary red-black BST operations
Color flip. Recolor to split a (temporary) 4-node.
flip colors
(after)
h
E
A
less
than A
S
between
A and E
between
E and S
greater
than S
private void flipColors(Node h)
{
assert !isRed(h);
assert isRed(h.left);
asset isRed(h.right);
h.color = RED;
h.left.color = BLACK;
h.right.color = BLACK;
}
Invariants. Maintains symmetric order and perfect black balance.
23
Insertion in a LLRB tree: overview
Basic strategy. Maintain 1-1 correspondence with 2-3 trees by
applying elementary red-black BST operations.
insert
C C
insert
EE
E
AA
add
addnew
new
node
here
node here
right link red
sorotate
rotate left
left
AA
A
A
SS
R S
RR
EE
CC
C
C
A
RR
E
E
R
R
SS
S
S
Insert into a 2-node
LLRB
tree
at the
bottom
E
A C
R S
2-3 tree
24
b
Insertion in a LLRB tree
search ends
at this null link
Warmup 1. Insert into a tree with exactly 1 node. b
a
left
root
right
b
a
root
b
a
right
root
a
a
red link to
new node
containing a
converts 2-node
to 3-node
search ends
at this null link
red link to
new node
containing a
converts 2-node
to 3-node
root
search ends
at this null link
a
search ends
at this null link
root
attached new node
with red link
b
root
b
a
rotated left
to make a
legal 3-node
Insert into a single
2-node (two cases)
attached new node
with red link
b
root
25
Insertion in a LLRB tree
Case 1. Insert into a 2-node at the bottom.
•
•
Do standard BST insert; color new link red.
If new red link is a right link, rotate left.
insert
C C
insert
EE
E
AA
add
addnew
new
node
here
node here
right link red
sorotate
rotate left
left
AA
A
A
SS
R S
RR
EE
CC
C
C
A
RR
E
E
R
R
SS
S
S
Insert into a 2-node
LLRB
tree
at the
bottom
E
A C
R S
2-3 tree
26
Insertion in a LLRB tree
Warmup 2. Insert into a tree with exactly 2 nodes.
larger
b
a
larger
smaller
between
smaller between
between
searchcends
c
search ends c
c
b
search ends
cc
b
at
this
at this
at this
a
search ends
b
a
search ends
bnull link
null link
a
search
ends
a link
ba
null
at
at this
null link this null link
at
this
null
link
search ends
search endssearch ends
at this null link
c
at this null link
larger
smaller
at this null link
b
a
b
a
attached new
c
node with
red link
b
attached newb
a b
attached
b new node with c
node with red link
c
ab
a b
attached new
ared link c
a
c
attached newnode with
a
node with red link
red link
rotated b
b
colors
flippedrighta
rotated
b
a
to blackc
colors flipped
b
a
right
b
c
a
colors flipped to black
c
a
a black c
to
bb
c
colors flipped
b
a
a
to black
colors flipped
a to blackc
b
a
c
c
c
a new
attached
attached new
node withb
node
b withred link
attached new
ca
nodered
withlink
red link c
c
b
rotated
c
b
a
right
a
rotated left
c
rotated left
rotated
right
a
rotated
colors
flipped
a to right
black c
b
cc
attached new
node with
red link
rotated left
rotated
right
b
c
colors flipped
to black
colors flipped
c
a
to black
b
b
colors flipped
c
to black
Insert into a single 3-node
(three cases)
c
a
b
a
Insert into a single 3-node (three cases)
27
Insert into a single 3-node (three cases)
so rotate right
E
C
Insertion in a LLRB tree
S
A
inserting H
R
E
Case 2. Insert into a 3-node at the bottom.
•
•
•
•
C
S
A
Rotate to balance the 4-node (if needed).
E
C level.
S
Flip colors
toH pass red link up one
inserting
A
E
add new
node here
R
A
inserting H
S
R
add new
node here
C
two lefts in a row
so rotate right
E
Rotate to make lean
left (if needed).
C
both children red
so flip colors
R
Do standard BST insert; color new link red.
inserting H
H
A
two lefts in a row
H
so rotate right
S
H
right link red
so rotate left
S
R
add new
node here
E
C
both children red
S
two lefts in a rowC
S
so flip colors
so rotate right
A
R
E
A
R
E
C
R
H
C
S
add new
S
A
H
node here A
R
both children red
two lefts in a row
H
so flip colors
so rotate right
right link red
E
so rotate left
E
both children redC
R
C
S
so flip colors
E
S
A
H
A
R E
C
R
R
H C
right link red
S
A
H
S so rotate left
A
H
both children red
so flip colors
right link red
R
E
E
so rotate left
S
E
C
R
C
R
H
SC
A
H
A
H E S
E
A
E
C
R
E
C
R
A
S
H
R
S
E
C
H
A
Insert into a 3-node
at the bottom
28
node here
C
Insertion in a LLRB tree: passing red links up the tree
A R
E
C
A
Case 2. Insert into a 3-node at the bottom.
•
•
•
•
•
Rotate to balance the 4-node (if needed).
S
M
A
Rotate to make lean left (if needed).
R
H
add new
node here
S
E
E
C
inserting P
R
E
A
S
M
H
H
add new
node hereC
A
R
M
H
R
both children
red so
flip colors
C
R
H
M
A
both children
red so
P
flip colors
add new
S
node here E
C
M
right link red
C
M
so rotate
left
R
both
children
A
H
R
red so
P
A
H
S
E
add new E
S
flip colors
node here
C
M
C
M
both children
right
link
red
P
A
H
R
red so
P
A
H
so rotate
left
R
flip colors
S
E
two lefts in a row
S
E
C
M
so rotate right
right
link red
both children
C
M
red soleft
R
P so rotate
A
H
R
flip colors
P
A
H
M
S
S
E
P
E
C
M
right link red
two lefts in a row
so rotate left
so rotate right
C
R
H
P
A
H
R
A
S
E
both children red
two lefts in a row
M
S
C
M
so flip colors
so rotate right
P
E
P
A
H
R
C
H
M
M
S
two lefts in a row
A
R
E
S
two lefts in a row
P so rotate right
E
S
E
P
H
S
M
R
A
S
A
M
Mright link red
so P
rotate left
R
A
H
S
E
two lefts in a row
M
so rotate right C
P
A
R H
inserting
P
Repeat case
1 or
case 2 up
the Mtree (if needed).
C
R
C
C
R
E
inserting P
add new
node here
M
both children
R
red so
P
H
flip colors
S
E
E
inserting P
Flip colors to pass red link up one level.
C
H
S
right link red
so rotate left
Do standard BST insert; color new link red.
M
both children red
E
so flip colors
C
H
MA
R
E
C
P
H
A
M
E
C
H
C
A
S
P
both children red
so flip colors
S
M
R
E
P
H
R
P
S
S
M
A
R
E
C the tree
H
Passing a red link up
A
P
S
Passing a red link up the tree
29
Insertion in a LLRB tree: Java implementation
Same code for both cases.
•
•
•
Right child red, left child black: rotate left.
Left child, left-left grandchild red: rotate right.
Both children red: flip colors.
h
h
left
rotate
h
right
rotate
flip
colors
Passing a red link up a red-black tree
private Node put(Node h, Key key, Value val)
{
if (h == null) return new Node(key, val, RED);
int cmp = key.compareTo(h.key);
if
(cmp < 0) h.left = put(h.left, key, val);
else if (cmp > 0) h.right = put(h.right, key, val);
else if (cmp == 0) h.val = val;
if (isRed(h.right) && !isRed(h.left))
h = rotateLeft(h);
if (isRed(h.left) && isRed(h.left.left)) h = rotateRight(h);
if (isRed(h.left) && isRed(h.right))
flipColors(h);
return h;
}
insert at bottom
(and color red)
lean left
balance 4-node
split 4-node
only a few extra lines of code
provides near-perfect balance
31
Insertion in a LLRB tree: visualization
255 insertions in ascending order
32
Insertion in a LLRB tree: visualization
255 insertions in descending order
33
Insertion in a LLRB tree: visualization
255 random insertions
34
Balance in LLRB trees
Proposition. Height of tree is ≤ 2 lg N in the worst case.
Pf.
•
•
Every path from root to null link has same number of black links.
Never two red links in-a-row.
Property. Height of tree is ~ 1.00 lg N in typical applications.
35
ST implementations: summary
worst-case cost
(after N inserts)
average case
(after N random inserts)
implementation
ordered
iteration?
key
interface
search
insert
delete
search hit
insert
delete
sequential search
(unordered list)
N
N
N
N/2
N
N/2
no
equals()
binary search
(ordered array)
lg N
N
N
lg N
N/2
N/2
yes
compareTo()
BST
N
N
N
1.39 lg N
1.39 lg N
?
yes
compareTo()
2-3 tree
c lg N
c lg N
c lg N
c lg N
c lg N
c lg N
yes
compareTo()
red-black BST
2 lg N
2 lg N
2 lg N
1.00 lg N *
1.00 lg N *
1.00 lg N *
yes
compareTo()
* exact value of coefficient unknown but extremely close to 1
36