Data Structures and Algorithms
Lesso n 1: Dat a St ruct ure s and Algo rit hm s using J ava
Data Structures and Algo rithms Overview
Algo rithm Perfo rmance
Co nstant Perfo rmance
Lo garithmic Perfo rmance
Linear Perfo rmance
Quadratic Perfo rmance
Co mparing Classificatio n Families
Lesso ns Learned
ArrayList Amo rtized Reallo catio n
Quiz 1 Pro ject 1
Lesso n 2: Dat a St ruct ure s and t he J ava Co lle ct io ns Fram e wo rk
Intro ductio n to Java Co llectio ns Framewo rk
Set Interface
List Interface
Queue Interface
Map Interface
Summarizing the Implementatio ns Yo u Need To Kno w
Impo rtant Metho ds Fo r Keys And Values
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 3: Algo rit hm s Using J ava
Designing Algo rithms
Skyline Pro blem
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 4: Wo rking Wit h Big Dat a
Wo rking with Big Data
So rting Large Sets Using External Sto rage
Characterizing Sto rage Requirements fo r an Algo rithm
MergeSo rt with O(n) Sto rage Requirements
Wo rking with Large Datasets
Never Be Satisfied
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 5: Re pre se nt ing Graph Dat a St ruct ure s
Representing Graphs
Using Adjacency Matrix To Represent Graph
Searching a Graph
Practical Applicatio n
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 6 : Graph Adjace ncy List and Sho rt e st Pat h Algo rit hm s
Searching Fo r Optimal Paths
Representing Graph By Adjacency List
Breadth-First Search
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 7: Prio rit y Que ue s
Prio rity Queue Data Structure
Minimum Spanning Tree
Heap Data Structure
Prim's Algo rithm Implementatio n
Evaluating Minimum Spanning Tree Implementatio ns
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 8 : Binary T re e Dat a St ruct ure
Binary Tree Data Structure
Naive Binary Tree Implementatio n
Evaluating Binary Tree Implementatio n
Rebalancing Binary Trees
Using Co llectio ns TreeSet
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 9 : Mult idim e nsio nal Algo rit hm s
A Data Structure Fo r Multidimensio nal Algo rithms
Traversing a kd-tree
Using kd-trees to Search fo r Po ints
Lesso ns Learned
Pro ject
Quiz 1 Pro ject 1
Lesso n 10 : Mat he m at ical Algo rit hm s and Flo at ing Po int Co m put at io ns
Mathematical Algo rithms and Flo ating Po int Co mputatio ns
Gauss Jo rdan Eliminatio n
Ro unding Erro rs
Partial Input Data
Matrix Determinant
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 11: Brut e Fo rce Algo rit hm s
Using Brute Fo rce To So lve Permutatio n Pro blems
Finding All Five-Letter wo rds in PALINDROME
N Queens Pro blem
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 12: Pat h Finding f o r Single -Playe r Gam e s
Path Finding Fo r Single-Player Games
Breadth-First Search
Evaluating Search Tree Algo rithms
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 13: Pat h Finding f o r T wo -Playe r Gam e s
Path Finding Fo r Two -Player Games
Minimax Implementatio n
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 14: Algo rit hm s On So und Dat a
Signal Pro cessing Algo rithms
Co mpo sed Wave Fo rms
Analyzing Co mpo sed Wave Fo rms
Using FFT o n WAV file samples
Lesso ns Learned
Quiz 1 Pro ject 1
Lesso n 15: Co nclusio n
Co ncluding Lesso n Fo r Algo rithms
Remo ving Elements Fro m a So rted Array
Remo ving Elements Fro m Binary Search Trees
Remo ving Elements Fro m AVL Trees
Remo ving Elements Fro m KD-trees
Lesso ns Learned
Quiz 1 Pro ject 1
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Data Structures and Algorithms using Java
Welco me to the O'Reilly Scho o l o f Techno lo gy co urse o n Data Structures and Algo rithms Using Java!
Course Objectives
When yo u co mplete this co urse, yo u will be able to :
identify the co re data structures pro vided by the JDK.
identify appro priate data structures based o n pro blems yo u are likely to face.
explain the essential design techniques necessary fo r develo ping algo rithms.
develo p algo rithms that efficiently pro cess data.
characterize the perfo rmance o f an algo rithm in bo th space and time.
In this Java co urse, yo u'll learn ho w to write efficient Java co de, which means learning abo ut data structures and algo rithms.
Here yo u'll refine yo ur Java skills to identify the appro priate data structures to use when so lving real-wo rld pro blems. These
data structures are already pro vided fo r yo u in the Java Develo pment Kit (JDK) release. Yo u'll learn key algo rithms that yo u'll
use again and again so yo ur co de perfo rms efficiently every time.
In each lab, yo u'll learn abo ut data structures and algo rithms within the co ntext o f a so lutio n to a real-wo rld pro blem. Once yo u
understand the so lutio n, yo u'll demo nstrate mastery by extending the existing co de in a pro ject. Thro ugho ut this co urse yo u will
write Java co de fro m scratch while so lving real pro blems. There will also be references to Algo rit hm s in a Nut she ll, the
asso ciated textbo o k fo r this co urse. The bo o k co mes with an o nline co de base, the Algo rithms Develo pment Kit (ADK), that can
be used as a reference in additio n to the co de described in these lesso ns.
Each quiz will validate that yo u learned the key info rmatio n and the pro jects and will describe likely extensio ns to the data
structures and algo rithms.
As yo u pro gress thro ugh the co urse, yo u'll write pro fessio nal test cases to verify the behavio r o f yo ur data structures and
algo rithms.
Lesson Objectives
When yo u co mplete this lesso n, yo u will be able to :
explain the limitatio n o f using arrays to sto re dynamic co llectio ns.
characterize the input, pro cessing and o utput steps fo r an algo rithm.
explain why using classes to mo del structured info rmatio n is preferred to just using multiple arrays co ntaining
primitive types.
characterize the run-time perfo rmance o f an algo rithm based o n the size o f a pro blem instance.
Welco me to the O'Reilly Scho o l o f Techno lo gy's co urse o n Data Structures and Algo rithms. Altho ugh it's unlikely that this sixth
co urse in the Java series is yo ur first OST co urse, we'll describe ho w OST wo rks, just in case. If yo u already have a so lid
understanding o f o ur to o ls and metho ds, feel free to skip ahead to the Data Structures and Algo rithms Overview sectio n.
Learning with O'Reilly School of T echnology Courses
As with every O'Reilly Scho o l o f Techno lo gy co urse, we'll take a user-active appro ach to learning. This means that yo u
(the user) will be active! Yo u'll learn by do ing, building live pro grams, testing them and experimenting with them—
hands-o n!
To learn a new skill o r techno lo gy, yo u have to experiment. The mo re yo u experiment, the mo re yo u learn. Our system
is designed to maximize experimentatio n and help yo u learn to learn a new skill.
We'll pro gram as much as po ssible to be sure that the principles sink in and stay with yo u.
Each time we discuss a new co ncept, yo u'll put it into co de and see what YOU can do with it. On o ccasio n we'll even
give yo u co de that do esn't wo rk, so yo u can see co mmo n mistakes and ho w to reco ver fro m them. Making mistakes
is actually ano ther go o d way to learn.
Abo ve all, we want to help yo u to learn to learn. We give yo u the to o ls to take co ntro l o f yo ur o wn learning experience.
When yo u co mplete an OST co urse, yo u kno w the subject matter, and yo u kno w ho w to expand yo ur kno wledge, so
yo u can handle changes like so ftware and o perating system updates.
Here are so me tips fo r using O'Reilly Scho o l o f Techno lo gy co urses effectively:
T ype t he co de . Resist the temptatio n to cut and paste the example co de we give yo u. Typing the co de
actually gives yo u a feel fo r the pro gramming task. Then play aro und with the examples to find o ut what else
yo u can make them do , and to check yo ur understanding. It's highly unlikely yo u'll break anything by
experimentatio n. If yo u do break so mething, that's an indicatio n to us that we need to impro ve o ur system!
T ake yo ur t im e . Learning takes time. Rushing can have negative effects o n yo ur pro gress. Slo w do wn and
let yo ur brain abso rb the new info rmatio n tho ro ughly. Taking yo ur time helps to maintain a relaxed, po sitive
appro ach. It also gives yo u the chance to try new things and learn mo re than yo u o therwise wo uld if yo u
blew thro ugh all o f the co ursewo rk to o quickly.
Expe rim e nt . Wander fro m the path o ften and explo re the po ssibilities. We can't anticipate all o f yo ur
questio ns and ideas, so it's up to yo u to experiment and create o n yo ur o wn. Yo ur instructo r will help if yo u
go co mpletely o ff the rails.
Acce pt guidance , but do n't de pe nd o n it . Try to so lve pro blems o n yo ur o wn. Go ing fro m
misunderstanding to understanding is the best way to acquire a new skill. Part o f what yo u're learning is
pro blem so lving. Of co urse, yo u can always co ntact yo ur instructo r fo r hints when yo u need them.
Use all available re so urce s! In real-life pro blem-so lving, yo u aren't bo und by false limitatio ns; in OST
co urses, yo u are free to use any reso urces at yo ur dispo sal to so lve pro blems yo u enco unter: the Internet,
reference bo o ks, and o nline help are all fair game.
Have f un! Relax, keep practicing, and do n't be afraid to make mistakes! Yo ur instructo r will keep yo u at it
until yo u've mastered the skill. We want yo u to get that satisfied, "I'm so co o l! I did it!" feeling. And yo u'll have
so me pro jects to sho w o ff when yo u're do ne.
Lesson Format
We'll try o ut lo ts o f examples in each lesso n. We'll have yo u write co de, lo o k at co de, and edit existing co de. The co de
will be presented in bo xes that will indicate what needs to be do ne to the co de inside.
Whenever yo u see white bo xes like the o ne belo w, yo u'll type the co ntents into the edito r windo w to try the example
yo urself. The CODE TO TYPE bar o n to p o f the white bo x co ntains directio ns fo r yo u to fo llo w:
CODE TO TYPE:
White boxes like this contain code for you to try out (type into a file to run).
If you have already written some of the code, new code for you to add looks like this.
If we want you to remove existing code, the code to remove will look like this.
We may also include instructive comments that you don't need to type.
We may run pro grams and do so me o ther activities in a terminal sessio n in the o perating system o r o ther co mmandline enviro nment. These will be sho wn like this:
INTERACTIVE SESSION:
The plain black text that we present in these INTERACTIVE boxes is
provided by the system (not for you to type). The commands we want you to type look lik
e this.
Co de and info rmatio n presented in a gray OBSERVE bo x is fo r yo u to inspect and absorb. This info rmatio n is o ften
co lo r-co ded, and fo llo wed by text explaining the co de in detail:
OBSERVE:
Gray "Observe" boxes like this contain information (usually code specifics) for you to
observe.
The paragraph(s) that fo llo w may pro vide additio n details o n inf o rm at io n that was highlighted in the Observe bo x.
We'll also set especially pertinent info rmatio n apart in "No te" bo xes:
Note
T ip
No tes pro vide info rmatio n that is useful, but no t abso lutely necessary fo r perfo rming the tasks at hand.
Tips pro vide info rmatio n that might help make the to o ls easier fo r yo u to use, such as sho rtcut keys.
WARNING
Warnings pro vide info rmatio n that can help prevent pro gram crashes and data lo ss.
Data Structures and Algorithms Overview
In 19 76 , Niklaus Wirth, the invento r o f the Pascal language and a pio neering figure in Co mputer Science, published a
fundamental textbo o k o n So ftware Engineering called Algorithms + Data Structures = Programs. In sho rt, he pro po sed
that develo pers must understand data structures and algo rithms as a prerequisite to writing efficient pro grams.
In yo ur earliest pro grams, yo u no do ubt used variables to sto re info rmatio n that yo u pro cessed. Fo r example, think o f
the first time yo u wro te a pro gram that co nverted temperatures between Celsius and Fahrenheit (and yo u kno w that
yo u did). To understand ho w to write this pro gram, a develo per must identify the appro priate Algorithm and Data
Structure to use.
An algorithm is a step-by-step pro cedure fo r co mputatio n that pro ce sse s input dat a to pro duce an o ut put re sult .
We'll highlight input dat a, pro ce sse s, and o ut put re sult s with these co lo rs thro ugho ut this lesso n to identify the
different functio nal parts o f the algo rithm implementatio ns.
The co mputatio n fo r temperature co nversio n is a straightfo rward calculatio n, so yo u o nly need to determine the fo rmat
o f input. In mo st situatio ns, the input is mo st easily represented as text strings typed by the user; ho wever, yo u co uld
also retrieve input as a binary sequence o f bits, perhaps fro m an embedded micro co ntro ller.
A problem instance is a particular input data set to which a pro gram is applied.
In practical terms, different pro grams will pro cess their input using co de that deco des o r translates the input into the
pro per data structures that they need to functio n. Let's try an example.
Create a new pro ject fo r this lesso n named Dat aSt ruct , and assign the J ava6 _Le sso ns wo rking set to it:
When pro mpted to o pen the perspective, check Re m e m be r m y answe r and click No .
In the Dat aSt ruct pro ject /src so urce fo lder, create a T e m pe rat ure Co nve rsio n class:
CODE TO TYPE: TemperatureCo nversio n class
import java.util.*;
public class TemperatureConversion {
public static void main(String[] args) {
System.out.println("Enter Celsius: ");
Scanner sc = new Scanner (System.in);
double c = Double.valueOf(sc.nextLine());
double f = c*9.0/5 + 32;
System.out.println("Fahrenheit is: " + f);
}
}
Right-click the class and select Run As | J ava Applicat io n. When pro mpted "Enter Celsius:", enter a number. The
Fahrenheit equivalent prints:
Run TemperatureCo nversio n
Enter Celsius:
100
Fahrenheit is: 212.0
Let's take a clo ser lo o k at the pro gram.
OBSERVE: TemperatureCo nversio n class
import java.util.*;
public class TemperatureConversion {
public static void main(String[] args) {
System.out.println("Enter Celsius: ");
Scanner sc = new Scanner (System.in);
double c = Double.valueOf(sc.nextLine());
double f = c*9.0/5 + 32;
System.out.println("Fahrenheit is: " + f);
}
}
The abo ve pro gram co nfo rms to the generic Input - Pro ce ss - Out put structure o f mo st pro grams. The Ce lsius
t e m pe rat ure is t ype d by t he use r and yo u co nve rt it t o Fahre nhe it , and m ult iply by 9 .0 /5 to retain precisio n
in yo ur co mputatio n. Fo r o utput, print t he co m put e d Fahre nhe it value t o t he co nso le .
The wo rld gets mo re co mplicated when yo u need to pro cess gro ups o f data. Let's extend the abo ve pro gram to
pro duce a table o f Fahrenheit co nversio ns given a number o f Celsius values. This first attempt sto res just the
co llectio n o f Celsius values and co mputes the Fahrenheit co nversio n values as needed. We'll assume that the user
enters the requested values co rrectly; adding erro r-handling lo gic wo uld o nly co mplicate these small pro grams and
o bscure the po ints we're trying to make in this lesso n.
In the Dat aSt ruct pro ject /src so urce fo lder, create a T e m pe rat ure Co nve rsio nT able class, and mo dify it as
sho wn:
CODE TO TYPE: TemperatureCo nversio nTable class
import java.util.*;
public class TemperatureConversionTable {
public static void main(String[] args) {
System.out.println("Enter Celsius values separated by spaces then press Enter: ");
Scanner sc = new Scanner (System.in);
String values = sc.nextLine();
StringTokenizer st = new StringTokenizer(values, " ");
double cValues[] = new double[st.countTokens()];
int index = 0;
while (st.hasMoreTokens()) {
cValues[index] = Double.valueOf(st.nextToken());
index++;
}
System.out.println("Celsius\tFahrenheit");
for (int i = 0; i < cValues.length; i++) {
double f = cValues[i]*9.0/5 + 32;
System.out.println(cValues[i] + "\t" + f);
}
}
}
Save and run T e m pe rat ure Co nve rsio nT able to see ho w it executes o n a sample pro blem instance:
Run TemperatureCo nversio nTable
Enter Celsius values separated by spaces then press Enter:
3 2.7 1.3 9.9 123.4
Celsius Fahrenheit
3.0 37.4
2.7 36.86
1.3 34.34
9.9 49.82
123.4 254.12000000000003
Let's lo o k at the different functio nal elements o f this co de.
TemperatureCo nversio nTable bro ken do wn into its parts
import java.text.*;
import java.util.*;
public class TemperatureConversionTable {
public static void main(String[] args) {
System.out.println("Enter Celsius values separated by spaces then press Enter: ");
Scanner sc = new Scanner (System.in);
String values = sc.nextLine();
StringTokenizer st = new StringTokenizer(values, " ");
double cValues[] = new double[st.countTokens()];
int index = 0;
while (st.hasMoreTokens()) {
cValues[index] = Double.valueOf(st.nextToken());
index++;
}
System.out.println("Celsius\tFahrenheit");
for (int i = 0; i < cValues.length; i++) {
double f = cValues[i]*9.0/5 + 32;
System.out.println(cValues[i] + "\t" + f);
}
}
}
The abo ve co de uses an array o f do uble cValue s[] to sto re the Celsius values entered by the user. With arrays yo u
need to specify the size o f the co llectio n in advance. In many cases, yo u either kno w the pro per size in advance o r yo u
can set the size to be so me value large eno ugh fo r any co nceivable pro gram executio n. The Input co nsists o f
Celsius values separated by spaces. Yo u can use St ringT o ke nize r to extract each o f these values as a String to ken,
which is then co nverted into a do uble value using the Do uble .value Of () metho d. As an added bo nus,
StringTo kenizer can tell yo u the to tal number o f to kens that will be extracted using the co unt T o ke ns() metho d. The
abo ve co de uses an o bvio us array structure to sto re a set o f singularly typed values (do ubles, in this case) and yo u
easily traverse each element in the array using a f o r lo o p to iterate o ver every element.
The o utput o f this pro gram is still a bit ro ugh. Let's make so me enhancements:
1. Retrieve the Celsius values fro m the user o ne per line; after that, the user simply presses Ent e r.
2. Present the co nversio n table so rted in ascending o rder.
3. Ro und Temperature values to two digits o f precisio n.
4. Do n't display any duplicate values in the table.
The first enhancement changes the entire Input phase o f the pro gram. The pro gram no lo nger kno ws in advance ho w
many values are to be read; rather, it must read them o ne at a time until to ld to sto p. Once the entire array o f values has
been created, yo u can satisfy the seco nd enhancement by so rting the array. Finally, yo u need to do so me extra
pro cessing to ensure thet there are no duplicates fo r the fo urth enhancement. The upco ming co de handles all o f these
enhancements while still using a simple array to sto re its values. The structure o f the co de has changed to allo w yo u
to co nduct testing at the end o f this lesso n.
CODE TO TYPE: Mo dified TemperatureCo nversio nTable class
import java.io.*;
import java.text.*;
import java.util.*;
public class TemperatureConversionTable {
static double cValues[];
static NumberFormat nf;
public static void main(String[] args) {
System.out.println("Enter Celsius values, one per line, then press Enter when done:
");
Scanner sc = new Scanner (System.in);
String values = sc.nextLine();
StringTokenizer st = new StringTokenizer(values, " ");
double cValues[] = new double[st.countTokens()];
int index = 0;
while (st.hasMoreTokens()) {
cValues[index] = Double.valueOf(st.nextToken());
index++;
}
process(System.in);
output(System.out);
}
static void process(InputStream is) {
Scanner sc = new Scanner (is);
nf = NumberFormat.getInstance();
nf.setMaximumFractionDigits(2);
cValues = new double[0];
while (true) {
String value = sc.nextLine();
if (value.equals ("")) { break; }
double val = Double.valueOf(value);
String formatVal = nf.format(val);
boolean found = false;
for (double d : cValues) {
if (nf.format(d).equals(formatVal)) {
found = true;
break;
}
}
if (found) {
System.err.println(" ** omitting duplicate value:" + formatVal);
} else {
cValues = java.util.Arrays.copyOf(cValues, cValues.length+1);
cValues[cValues.length-1] = val;
}
}
}
static void output(PrintStream out) {
java.util.Arrays.sort(cValues);
System.out.println("Celsius\tFahrenheit");
for (int i = 0; i < cValues.length; i++) {
double f = cValues[i]*9.0/5 + 32;
System.out.println(nf.format(cValues[i]) + "\t" + nf.format(f));
}
}
}
Try running this revised T e m pe rat ure Co nve rsio nT able o n the fo llo wing pro blem instance:
INTERACTIVE SESSION: Sample Run
Enter Celsius values, one per line, then press Enter when done:
22.4
13.7
18.003
31
18
** omitting duplicate value:18
Celsius
13.7
18
22.4
31
Fahrenheit
56.66
64.41
72.32
87.8
Let's break this co de do wn and try to deal with a number o f separately identified co de blo cks:
OBSERVE: revised main and new pro cess metho d
public static void main(String[] args) {
System.out.println("Enter Celsius values, one per line, then press Enter when done: "
);
process(System.in);
output(System.out);
}
static void process(InputStream is) {
Scanner sc = new Scanner (is);
nf = NumberFormat.getInstance();
nf.setMaximumFractionDigits(2);
cValues = new double[0];
while (true) {
String value = sc.nextLine();
if (value.equals ("")) { break; }
double val = Double.valueOf(value);
String formatVal = nf.format(val);
boolean found = false;
for (double d : cValues) {
if (nf.format(d).equals(formatVal)) {
found = true;
break;
}
}
if (found) {
System.err.println(" ** omitting duplicate value:" + formatVal);
} else {
cValues = java.util.Arrays.copyOf(cValues, cValues.length+1);
cValues[cValues.length-1] = val;
}
}
}
The abo ve co de will read strings fro m Syst e m .in which co ntain the Celsius values. All Celsius values will be sto red
in an array o f do uble values; ho wever, since the pro gram canno t determine in advance the number o f Celsius values
entered, it starts—literally—with an empty array o f do uble . A Num be rFo rm at instance accurate to two digits o f
precisio n is created to be used bo th during pro ce ssing and o ut put .
The bulk o f the wo rk is handled by the co de that reads o ne string line at a time to extract Celsius values to be added to
the array o f do uble values. The pro gram must ensure that no duplicate value appears in the o utput table. Of co urse yo u
co uld filter the o utput to avo id printing duplicate values, but it's better idea to just avo id sto ring duplicate values in the
first place. No te that yo u must avo id having two values in the table which wo uld o therwise "ro und" to the same two
digits o f precisio n. So , if the input co ntained bo th 15 .234 and 15 .2321, o nly the first value sho uld be entered into the
array, because bo th values ro und to 15 .23.
As each String value is read fro m the input, the co de checks fo r the empty string as a signal that the user is do ne;
o therwise, it co nverts the string value into a do uble using Do uble .value Of () and also co nstructs a f o rm at Val
String representing the two -digit ro unded value that it wo uld represent in the o utput table. The f o r (do uble d :
cValue s) lo o p checks to see whether any o ther Celsius value (o nce fo rmatted) also equals f o rm at Val. If it do es, the
pro gram co nsiders the new value to be a duplicate and alerts the user that the value will be o mitted.
If the value is no t eliminated, yo u must add it to the cValue s array. Since the size o f the array canno t be kno wn in
advance, this co de uses the java.ut il.Arrays.co pyOf () metho d to extend the array to be o ne greater in size. The
metho d co pies values fro m the o ld array into the new o ne because the new array is allo cated as a new Java o bject.
No te that the last value in the array will be the value typed in by the user, val.
OBSERVE: o utput metho d
static void output(PrintStream out) {
java.util.Arrays.sort(cValues);
out.println("Celsius\tFahrenheit");
for (int i = 0; i < cValues.length; i++) {
double f = cValues[i]*9.0/5 + 32;
out.println(nf.format(cValues[i]) + "\t" + nf.format(f));
}
}
The final lo gic in the co de so rt s t he Ce lsius value s in cValue s using the java.ut il.Arrays.so rt () metho d pro vided
by Java. Once cValue s is so rted, the Fahrenheit temperatures are co nverted as needed, and the table is o utput in
ascending Celsius o rder, line by line.
Yo u may be satisfied with this pro gram as it is. It lo o ks like it so lves the pro blem. Ho wever, perfo rmance may be an
issue; the run-time perfo rmance o n this small data is fine, but it might no t wo rk as well o n a much larger data set.
With algo rithms, the key perfo rmance questio n to co nsider is what happens when the size o f a rando m pro blem
instance gro ws; mo re specifically, when the size do ubles. To anticipate the perfo rmance o f this co de, yo u need to
understand ho w practitio ners evaluate the perfo rmance o f algo rithms.
Algorithm Performance
Cho o sing an algo rithm depends o n the pro blem being so lved and the pro blems it will likely face. Algo rithms are
typically presented with three co mmo n cases in mind:
Wo rst case : The class o f pro blem instances fo r which an algo rithm exhibits its wo rst runtime behavio r.
Instead o f trying to identify the specific input, algo rithm designers typically describe properties o f the input
that prevent an algo rithm fro m running efficiently.
Ave rage case : The expected behavio r when executing the algo rithm o n rando m pro blem instances. This
measure describes the expectatio ns an average user o f the algo rithm sho uld have.
Be st case : The class o f pro blem instances fo r which an algo rithm exhibits its best runtime behavio r. In
reality, the best case rarely o ccurs.
We co mpare algo rithms by evaluating their perfo rmance o n pro blem instances o f size n. The go al is to determine the
number o f steps o r operations the algo rithm needs to so lve the pro blem. This is an abstract way o f measuring the co st
o f an algo rithm. Intuitively, an o peratio n can be the assignment o f a variable, co mparing two numbers to gether, o r
perfo rming a mathematical o peratio n. This metho do lo gy is the standard means fo r co mparing algo rithms. By co unting
the number o f o peratio ns, we can determine which algo rithms scale to so lve pro blems o f no ntrivial size by evaluating
the running time needed by the algo rithm in relatio n to the size o f the pro vided input. This fo rm o f evaluatio n is
co nsistent and do es no t depend o n the pro gramming language used o r the specific pro cesso r o n which the pro gram
is run.
When yo u determine the number o f o peratio ns perfo rmed by an algo rithm, yo u must represent the to tal co unt with
regards to the o riginal size, n, o f the pro blem instance. Fo r example, the fo llo wing sample co unt co unts the number o f
times an integer value appears in an arbitrary array o f integer values:
OBSERVE: sample co unt metho d
static int count (int[] A, int val) {
int count = 0;
for (int i = 0; i < A.length; i++) {
if (A[i] == val) {
count++;
}
}
return count;
}
There are six individual statements in the abo ve co de. Fo r each statement, yo u can determine the m axim um number
o f times it executes o n a pro blem instance o f size n:
St at e m e nt
Exe cut io n Co unt
1. int co unt = 0
executes o nce
2. int i = 0
executes o nce
3. if (i < A.le ngt h) executes n+1 times
4. if (A[i] == val)
executes n times
5. co unt ++
executes NO MORE THAN n times
6 . i++
executes n times
In to tal, there will be no mo re than 4*n+3 statements executed. If n is very large, the co nstant +3 beco mes insignificant
and yo u can just say that the number o f o peratio ns will be fo ur times the to tal number, n, o f values. The phrase used in
this co urse is that the number o f o peratio ns fo r the abo ve algo rithm is on the order of n o r O(n). The statement "an
o rder n algo rithm"—written as O(n)—means that the to tal number o f o peratio ns is bo unded by a co nstant (in this case
it was 4) multiplied by n.
In so me cases, the number o f o peratio ns is co nstant and do es no t depend o n the pro blem instance size. In these
cases, yo u wo uld represent the behavio r as O(1). Fo r a mo re detailed discussio n o n the "big O" no tatio n used here,
review Chapter 2 in the Algo rit hm s In A Nut she ll bo o k.
There are a number o f classificatio ns in this co urse. They are o rdered here by decreasing efficiency:
Co nstant O(1)
Lo garithmic O(log n)
Linear O(n)
Lo glinear O(n log n)
Quadratic O(n 2 )
Expo nential O(2 n )
Yo u will see examples o f each o f these classificatio ns during this co urse. Fo r no w, let's fo cus o n these fo ur examples:
Constant Performance
Suppo se yo u want to determine the first element o f an uno rdered array o f n elements. The effo rt yo u'd expend
to acco mplish this task wo uld be the same even if yo u had 2*n, o r twice as many, elements. When an
algo rithm can so lve a pro blem in a f ixe d num be r o f o pe rat io ns, regardless o f the size o f the pro blem
instance, the algo rithm exhibits Constant Performance.
Logarithmic Performance
Co nsider lo o king fo r a given last name in the pho ne bo o k with 10 0 0 pages. Yo u do n't typically start o n page
1; rather yo u start o n page 50 0 and determine "which side" o f the pho nebo o k yo u need to search further. Yo u
repeat this pro cess until yo u find the pro per page. With each step o f wo rk, yo u reduce the size o f the pro blem
by half. When an algo rithm can so lve a pro blem in a number o f steps relative to the lo garithm (base 2) o f the
pro blem instance size, we say that the algo rithm exhibits Logarithmic Performance.
Linear Performance
When the number o f steps required by an algo rithm to so lve a pro blem gro ws at the same rate as the
pro blem size gro ws, then the algo rithm exhibits Linear Performance. If yo u're searching fo r a value in an
uno rdered array o f n elements, yo u wo uld require twice as much wo rk to search thro ugh an array with 2*n
elements.
Quadratic Performance
Fo r so me algo rithms, do ubling the size o f the pro blem instance makes the executio n fo ur times lo nger,
resulting in Quadratic Performance. Co nsider the pro blem o f determining whether there are two values in an
uno rdered array o f n elements that are the same. Fo r each value in the array, yo u may have to co mpare it
against each o f the o ther values in o rder to find a match.
Whenever yo u identify a nested lo o p o ver all elements in a co llectio n, yo u can be sure that the perfo rmance is
at least Quadratic.
OBSERVE: Sample nested fo r lo o p exhibiting quadratic perfo rmance
for (int i = 0; i < values.length; i++) {
for (int j = 0; j < values.length; j++) {
Inner Code Block
}
}
The Inner Code Block executes n 2 times, where n is the number o f elements in the value s array. Here's
ano ther co mmo n nesting pattern:
OBSERVE: Ano ther sample nested fo r lo o p exhibiting quadratic perfo rmance
for (int i = 0; i < values.length-1; i++) {
for (int j = i+1; j < values.length; j++) {
Inner Code Block
}
}
In that co de, the Inner Code Block executes o nce fo r every unique pair o f elements in value s. The to tal
number o f times this executes is n*(n-1)/2 o r n 2 /2 - n/2. The perfo rmance o f this nested fo r lo o p is still
co nsidered to be Quadratic with respect to the size o f the pro blem instance, n, despite the subtractio n o f n/2
and the co efficient 1/2. This is due to the do minance o f n 2 in the equatio n. As n co ntinues to increase, the
gro wth in size o f this equatio n will always be larger than a co rrespo nding gro wth in a linear equatio n.
Comparing Classification Families
The run-time behavio rs o f algo rithms can be co mpared by classificatio n. That is, an O(1) algo rithm is
co nsidered to be mo re efficient than an O(n) algo rithm. When two algo rithms exhibit the same perfo rmance
classificatio n—say, O(n log n)—o ne might still be mo re efficient than ano ther "because o f the co nstants."
Recall ho w earlier we said that the co ntants beco me insignificant with increasing sizes o f n? Theo retically this
is true, but o ne implementatio n o f an algo rithm may be mo re efficient than ano ther even when they belo ng to
the same classificatio n. Yo u may also find that an O(n 2 ) algo rithm is mo re efficient than a co mparable O(n log
n) algo rithm f o r sm all value s o f N. The asso ciated co nstants fo r the O(n log n) algo rithm make the co de
run slo wer than it do es with the O(n 2 ) algo rithm fo r small values o f n. Once n increases, the O(n log n)
algo rithm will o utperfo rm any O(n 2 ) algo rithm regardless o f co nstants.
Let's go back and evaluate the perfo rmance o f T e m pe rat ure Co nve rsio nT able .
In yo ur Dat aSt ruct pro ject, Create a /pe rf o rm ance so urce fo lder to sto re all perfo rmance-related classes.
Create a T im e T e m pe rat ure Co nve rsio n class in the default package o f the /pe rf o rm ance so urce fo lder.
CODE TO TYPE: TimeTemperatureCo nversio n class
import java.util.*;
import java.io.*;
public class TimeTemperatureConversion {
public static void main(String[] args) throws Exception {
Scanner sc = new Scanner (System.in);
System.out.println("Enter number of different values to add.");
int numItems = Integer.valueOf(sc.nextLine());
StringBuilder sb = new StringBuilder();
for (int i = 0; i < numItems; i++) {
sb.append(i).append("\n");
}
sb.append("\n");
// empty string to terminate the input
// execute with timing in place
double total = 0;
int numTrials = 10;
for (int run = 0; run < numTrials; run++) {
ByteArrayInputStream is = new ByteArrayInputStream(sb.toString().getBytes());
System.gc();
long now = System.currentTimeMillis();
TemperatureConversionTable.process(is);
long end = System.currentTimeMillis();
total += (end - now);
}
System.out.println(numItems + "," + (total/numTrials));
}
}
Run T im e T e m pe rat ure Co nve rsio n no w fo r 6 4, 128 , 256 , 512, 10 24, 20 48 , and 40 9 6 and no te the results, then
we'll make so me changes and re-run the co de using the same values.
The abo ve co de creates a string, sb, fro m which the input is to be read. The pro ce ss() metho d o f
T e m pe rat ure Co nve rsio nT able was designed to pro cess its input fro m an Input St re am o bject; in the actual co de,
input came fro m Syst e m .in, but in this perfo rmance co de a Byt e ArrayInput St re am o bject is created and used
instead. This co de design allo ws yo u to write auto mated test cases so yo u do n't have to input yo ur data manually like
we're do ing here. Ten trials are executed and the average o ver all executio ns is printed.
As the pro blem size do ubles, the time required by T e m pe rat ure Co nve rsio nT able to so lve each pro blem
increases (tripling o r even quadrupling), suggesting that this implementatio n exhibits Quadratic Performance. But do es
that mean that there is no faster implementatio n?
To impro ve o ur results, we'll use data structures pro vided by the JDK. (In the next lab, we'll intro duce a number o f co re
data structures with behavio rs that will be useful as yo u write mo re advanced algo rithms.)
Create a Ce lsiusValue class in the default package o f the Dat aSt ruct /src so urce fo lder.
CODE TO TYPE: CelsiusValue class
import java.text.NumberFormat;
public class CelsiusValue implements Comparable<CelsiusValue> {
public String formatted;
public double value;
public String fahrenheit;
static NumberFormat nf = null;
public CelsiusValue(double v) {
value = v;
if (nf == null) {
nf = NumberFormat.getInstance();
nf.setMaximumFractionDigits(2);
}
formatted = nf.format(v);
fahrenheit = nf.format(9.0*v/5 + 32);
}
public boolean equals (Object o) {
if (o == null) { return false; }
if (o instanceof CelsiusValue) {
CelsiusValue other = (CelsiusValue) o;
return (formatted.equals (other.formatted));
}
return (false);
}
public int compareTo(CelsiusValue other) {
return formatted.compareTo(other.formatted);
}
}
This class represents an entry in the Celsius co nversio n table. The Ce lsiusValue co nstructo r sto res the f o rm at t e d
Celsius value (accurate to two digits) and its co mputed f ahre nhe it equivalent. To increase efficiency, there's a static
Num be rFo rm at field, nf , that is co nstructed the very first time a Ce lsiusValue o bject is co nstructed.
Yo u need two o ther metho ds to arrive at this so lutio n. Yo u may be familiar with the Java standard e quals(Obje ct )
metho d, which determines whether two o bjects are equal to each o ther. Fo r this pro blem, two Ce lsiusValue o bjects
are equal if they have the same f o rm at t e d representatio n; this will prevent two duplicate entries fro m appearing in the
table when their fo rmatted values are the same. The co m pare T o metho d determines the o rdering o f two
Ce lsiusValue o bjects, to so rt entries in the table pro perly. The names o f these metho ds sho uld be familiar to Java
pro grammers and in the next lesso n we will further investigate the naming co nventio ns and standard interfaces and
classes pro vided by the JDK. As yo u develo p classes to represent info rmatio n in the pro blem do main, yo u will see
that these classes are no lo nger exclusively Input o r Pro ce ss classes.
No w mo dify T e m pe rat ure Co nve rsio nT able as sho wn:
CODE TO TYPE: Mo dified TemperatureCo nversio nTable class
import java.io.*;
import java.text.*;
import java.util.*;
public class TemperatureConversionTable {
static TreeSet<CelsiusValue> cValues;
static double cValues[];
static NumberFormat nf;
public static void main(String[] args) {
System.out.println("Enter Celsius values, one per line, then press Enter when done:
");
process(System.in);
output(System.out);
}
static void process(InputStream is) {
Scanner sc = new Scanner (is);
nf = NumberFormat.getInstance();
nf.setMaximumFractionDigits(2);
cValues = new TreeSet<CelsiusValue>();new double[0];
while (true) {
String value = sc.nextLine();
if (value.equals ("")) { break; }
double val = Double.valueOf(value);
CelsiusValue cval = new CelsiusValue(val);
String formatVal = nf.format(val);
boolean found = false;
for (double d : cValues) {
if (nf.format(d).equals(formatVal)) {
found = true;
break;
}
}
if (cValues.contains(cval)found) {
System.err.println(" ** omitting duplicate value:" + valueformatVal);
} else {
cValues.add(cval);
cValues = java.util.Arrays.copyOf(cValues, cValues.length+1);
cValues[cValues.length-1] = val;
}
}
}
static void output(PrintStream out) {
java.util.Arrays.sort(cValues);
out.println("Celsius\tFahrenheit");
for (CelsiusValue cv : cValues) {
out.println(cv.formatted + "\t" + cv.fahrenheit);
}
for (int i = 0; i < cValues.length; i++) {
double f = cValues[i]*9.0/5 + 32;
out.println(nf.format(cValues[i]) + "\t" + nf.format(f));
}
}
}
No w rerun T im e T e m pe rat ure Co nve rsio n fo r the same values we used in the earlier test (6 4, 128 , 256 , 512, 10 24,
20 48 , and 40 9 6 ).
The seco nd co lumn belo w sho ws ho w T e m pe rat ure Co nve rsio nT able perfo rmed the first time; the third co lumn
sho ws appro ximate perfo rmance after the last changes (yo ur results may differ so mewhat):
sho ws appro ximate perfo rmance after the last changes (yo ur results may differ so mewhat):
Pro ble m
T e m pe rat ure Co nve rsio nT able
T e m pe rat ure Co nve rsio nT able wit h ArrayList
Size
Ave rage Exe cut io n T im e (m illise co nds)
Ave rage Exe cut io n T im e (m illise co nds)
64
3.0
2.6
128
8 .0
3.0
256
22.8
6 .0
512
79 .7
7.0
10 24
29 3.6
9 .0
20 48
9 49 .8
13.7
40 9 6
38 14.8
24.2
8 19 2
*
47.1
16 38 4
*
9 7.4
Yo u can see that, as the pro blem instance size increases, this revised implementatio n is ten times faster (size 512)
and even 10 0 times faster (size 40 9 6 ). Clearly the seco nd implementatio n is much mo re efficient! In this case, it
appears that the cho ice o f data structure vastly impro ved the efficiency o f the co de, reaffirming the o bservatio n by Wirth
that Algorithms + Data Structures = Programs.
Lessons Learned
Real-wo rld pro blems are no t always as clean and simple as tho se presented here. In particular, yo u must ro utinely
maintain a highly dynamic co llectio n o f values. So metimes yo u might want to add o r remo ve a value to o r fro m the
co llectio n. Yo u might want to sto re the entire co llectio n to persistent sto rage (such as a database o r the file system) so
yo u can retrieve it entirely at a later po int. Since release 1.5 o f the JDK, Java pro vides the Collections Framework, which
is a so phisticated set o f classes to represent and manipulate co llectio ns. Yo u have likely co me acro ss these classes
because o f their versatility (classes such as ArrayList and HashMap, fo r example). As a pro grammer, yo u need to
kno w abo ut these classes and—mo re impo rtantly—to kno w the specific circumstances under which to use each o ne.
There is no need fo r yo u to reimplement these data structures o n yo ur o wn, but yo u do need to understand ho w to
select the appro priate classes fo r yo ur needs.
We started this co urse by co mpleting a pro blem using the mo st primitive capabilities o ffered by the Java pro gramming
language. The T im e T e m pe rat ure Co nve rsio n class wo rks co rrectly, but it's no t efficient eno ugh when tackling
larger pro blem instances. Yo u can find ways to mo dify yo ur pro grams to impro ve their efficiency, but in mo st cases it's
easier to use the available data structures rather than implement yo ur o wn versio ns.
At the end o f each lab, we'll review key co ncepts regarding data structures and algo rithms. Here is the first list o f key
co ncepts:
Use arrays in t he way t he y we re de signe d: Use arrays when yo u have a fixed and bo unded number o f
values and yo u need immediate access to any o f these values using a po sitio n index.
Avo id se arching t hro ugh uno rde re d arrays: It's inefficient. If searching fo r an item is a key part o f yo ur
algo rithm, do no t sto re yo ur items in an uno rdered array.
Avo id dynam ically re sizing arrays t o be just o ne size large r: If yo u are frequently adding an item to a
co llectio n, review the Java Co llectio ns Framewo rk (described in next lesso n) to find a mo re suitable data
structure fo r dynamic behavio r.
ArrayList Amortized Reallocation
If yo u have access to the so urce co de o f the JDK, review the e nsure Capacit y() metho d o f the
java.ut il.ArrayList class. Yo u can see that whenever a new element is added to an ArrayList o bject,
e nsure Capacit y() first ensures that it has eno ugh capacity fo r the new element. The size o f the new array is
ro ughly 1.5 times larger whenever it needs to expand the size o f the underlying e le m e nt Dat a array.
OBSERVE: ArrayList add() metho d uses Amo rtizatio n thro ugh ensureCapacity()
public boolean add(E e) {
ensureCapacity(size + 1);
elementData[size++] = e;
return true;
}
// Increments modCount!!
public void ensureCapacity(int minCapacity) {
modCount++;
int oldCapacity = elementData.length;
if (minCapacity > oldCapacity) {
Object oldData[] = elementData;
int newCapacity = (oldCapacity * 3)/2 + 1;
if (newCapacity < minCapacity)
newCapacity = minCapacity;
// minCapacity is usually close to size, so this is a win:
elementData = Arrays.copyOf(elementData, newCapacity);
}
}
When yo u expand an array to acco mmo date mo re elements, make sure that yo u do n't call the o peratio n to o
frequently. Instead, increase the size o f the array by mulitplying it by so me number rather than by adding ro o m
fo r a co nstant number o f items.
The next lesso n will intro duce the Java Co llectio ns Framewo rk, which we'll use extensively in this co urse.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Data Structures and the Java Collections Framework
Lesson Objectives
When yo u finish this lesso n yo u will be able to :
identify the co re interfaces pro vided by the Java Co llectio ns Framewo rk.
explain the difference between a Set and a List.
implement equals(Object) and hashCo de() metho ds as required by Set and List implementatio ns.
Introduction to Java Collections Framework
In this co urse, we use the co mmo nly-accepted term "Java Develo pment Kit (JDK)" to refer to the platfo rm fo r
develo ping Java applicatio ns. The JDK defines a range o f Applicatio n Pro gramming Interfaces (APIs) fo r general
purpo se functio nality, including netwo rk pro gramming packages (java.net) and graphical user interfaces (java.awt,
javax.swing). Alto gether, the JDK release dated August 1, 20 13 co ntains o ver 12,0 0 0 classes (18 ,0 0 0 classes if yo u
include ano nymo us and inner classes). In this lesso n, we are co ncerned with the java.ut il package, which co ntains
the Co llectio ns Framewo rk, legacy co llectio n classes, event mo del, date and time facilities, internatio nalizatio n, and
miscellaneo us utility classes (a string to kenizer, a rando m-number generato r, and a bit array).
Befo re describing the classes in the Co llectio ns Framewo rk, we need to discuss the nature o f an interface in Java.
Each Java class defines public metho ds to be used by external classes. The java.lang.St ring class, fo r example,
co ntains 8 2 public metho ds. Did yo u kno w that yo u can determine the last index po sitio n o f a character within a String?
Yo u might if yo u had read the do cumentatio n and disco vered that there is a last Inde xOf (char ch) metho d. This class
also has a co m pare T o (St ring s) metho d that co mpares two strings in alphabetical o rder; it returns 0 if the two string
o bjects are equal to each o ther and returns a negative o r po sitive number to determine the alphabetic o rder o f the two
String o bjects. Because co mparing two o bjects is a fundamental o peratio n fo r so many classes, Java designers
develo ped an interface that declares this behavio r (here's part o f the do cumentatio n in the interface):
OBSERVE: java.lang.Co mparable Interface
package java.lang;
public interface Comparable<T> {
/**
* Compares this object with the specified object for order. Returns a
* negative integer, zero, or a positive integer as this object is less
* than, equal to, or greater than the specified object.
*/
public int compareTo(T o);
}
An interface co ntains a set o f metho ds that represents a behavio r; in this case, the Co m parable interface represents
the ability to o rder two o bjects. This behavio r is the same whether the o bject is a String, an Integer o r a Do uble o bject.
The interface specifies the behavio r and a class pro vides the behavio r by declaring that the class implements the
interface. The String class, fo r example, declares this:
OBSERVE: String interfaces
public final class String implements java.io.Serializable, Comparable<String>, CharSequ
ence { ... }
Once yo u kno w that the String class implements Co m parable , yo u kno w that it must pro vide an implementatio n o f its
metho ds, and specifically that it will have the co m pare T o metho d defined by the interface.
To describe the Co llectio ns Framewo rk, let's start with the java.ut il.Co lle ct io n<E> interface, which is the
fundamental interface in this package. A Co lle ct io n<E> represents a gro up o f elements o f type E defined using the
Java Generics co ncept. This general definitio n applies to a wide range o f behavio rs. This interface defines 15
metho ds, including these six fundamental metho ds:
int size () returns size o f the co llectio n.
bo o le an isEm pt y() determines if the co llectio n is empty.
co nt ains (Obje ct e le m e nt ) determines whether it co ntains a given element.
It e rat o r<E> it e rat o r() enables retrieval o f elements in the co llectio n in some order.
bo o le an add(E e le m e nt ) adds an element to the co llectio n (o ptio nal metho d).
re m o ve (Obje ct e le m e nt ) remo ves an element fro m the co llectio n (o ptio nal metho d).
Because so me co llectio n o bjects are immutable—that is, their elements canno t be changed—the last two metho ds
abo ve are o ptio nal and a Co llectio n o bject may o r may no t ho no r requests to add o r remo ve an element. Ho wever, if
an add metho d returns t rue , the Co llectio n guarantees that the element was added to the set. If the add metho d
returns f alse , the co llectio n already co ntains the element and will no t allo w duplicates. If a Co llectio n refuses to add a
particular element fo r a reaso n o ther than it being a duplicate, it must thro w an Exceptio n.
The fo llo wing co de sample sho ws ho w to use all o f these metho ds with an ArrayList o bject to determine the unique
letters used in a given sentence.
Create a Co lle ct io ns pro ject and add it to the J ava6 _Le sso ns wo rking set.
In the default package o f the Co lle ct io ns/src so urce fo lder, create a Unique Le t t e rs class as sho wn:
CODE TO TYPE: UniqueLetters class
import java.util.*;
public class UniqueLetters {
static final String vowels = "aeiou";
public static void main(String[] args) {
System.out.println("Enter at least one character:");
Scanner sc = new Scanner (System.in);
String s = sc.nextLine();
Collection<Character> unique = new ArrayList<Character>();
for (int i = 0; i < s.length(); i++) {
char c = Character.toLowerCase(s.charAt(i));
if (Character.isWhitespace(c)) { continue; }
if (!unique.contains(c)) {
unique.add(c);
}
}
if (unique.isEmpty()) {
System.out.println("Please enter at least one character");
System.exit(1);
}
System.out.println("There are " + unique.size() + " unique characters");
System.out.print(" Vowels:");
for (int i = 0; i < vowels.length(); i++) {
char c = vowels.charAt(i);
if (unique.contains(c)) {
System.out.print(c);
unique.remove(c);
}
}
System.out.print("\n Consonants:");
for (Iterator<Character> it = unique.iterator(); it.hasNext(); ) {
System.out.print(it.next());
}
System.out.println();
}
}
Run this pro gram o n the sample input belo w:
INTERACTIVE SESSION: Sample executio n o f UniqueLetters
Enter at least one character:
Now is the time for all good men to come to the aid of their country
There are 18 unique characters
Vowels:aeiou
Consonants:nwsthmfrlgdcy
Let's take a clo ser lo o k at this co de.:
OBSERVE: Adding elements to a Co llectio n
Collection<Character> unique = new ArrayList<Character>();
for (int i = 0; i < s.length(); i++) {
char c = Character.toLowerCase(s.charAt(i));
if (Character.isWhitespace(c)) { continue; }
if (!unique.contains(c)) {
unique.add(c);
}
}
The unique o bject is co nstructed as an ArrayList o f Charact e r o bjects. Fo r each no n-whit e space charact e r in
the input st ring (co nve rt e d t o lo we rcase ), we check to make sure that the charact e r is no t alre ady a m e m be r
o f unique befo re adding t he charact e r t o it . Just keep in mind that o nce the add metho d co mpletes, the element
is guaranteed to be a member o f the co llectio n. The specific implementatio n o f Co llectio n—in this case ArrayList —is
respo nsible fo r handling the add request; ArrayList appends the character to the end o f the gro wing list o f
Charact e r o bje ct s.
OBSERVE: Searching fo r and remo ving elements fro m a Co llectio n
static final String vowels = "aeiou";
...
for (int i = 0; i < vowels.length(); i++) {
char c = vowels.charAt(i);
if (unique.contains(c)) {
System.out.print(c);
unique.remove(c);
}
}
The co de iterates o ver the kno wn vo we ls and remo ves these characters fro m the co llectio n unique . Finally, all
Co llectio n classes pro vide a co nsistent means to iterate o ver their elements, as sho wn in this co de:
OBSERVE: Iterate o ver elements in a Co llectio n using an Iterato r
for (Iterator<Character> it = unique.iterator(); it.hasNext(); ) {
System.out.print(it.next());
}
The Co llectio n classes also implement the It e rable interface, which means yo u can use Java's enhanced for loop to
iterate o ver the elements. The abo ve iteratio n co de co uld have been written mo re simply as:
OBSERVE: Iterate o ver elements in a Co llectio n using enhanced fo r lo o p
for (Character c : unique) {
System.out.print(c);
}
The Co llectio n interface fo rms the ro o t o f the hierarchy fo r three impo rtant interfaces in the Co llectio ns Framewo rk:
List, Se t , and Queue. This co de example uses the ArrayList class, which is a co ncrete implementatio n o f the List
class. We'll discuss each interface in the upco ming sectio ns o f the lesso n.
class. We'll discuss each interface in the upco ming sectio ns o f the lesso n.
Each implementatio n o f Co llectio n guarantees certain perfo rmance o f the metho ds defined earlier. In describing this
perfo rmance, the JDK do cumentatio n uses the same terms used in the previo us lesso n to describe the wo rst-case
run-time perfo rmance o f algo rithms. Here is the "specificatio n sheet" fo r ArrayList fo r the six fundamental Co llectio n
metho ds:
O(1): size, isEmpty
Am o rt ize d Co nst ant T im e : add
O(n): remo ve, co ntains, iterate
The add metho d is declared to have amortized constant time because ArrayList must reallo cate mo re memo ry when
it beco mes full. Recall the discussio n in the previo us lesso n abo ut the dangers o f resizing arrays? When ArrayList
needs to resize its array o f n elements, it makes sure to request (3*n/2+1) elements to reduce the number o f times that
it has to reallo cate memo ry. In the lo ng run, adding n elements requires o nly a to tal o f O(n) time, which means each
individual add o peratio n is co nsidered to be amortized constant time.
There are two reaso ns to cho o se a particular data structure: (1) because o f the functio nal behavio r that it pro vides; (2)
because o f the perfo rmance asso ciated with that behavio r. All o peratio ns fo r these classes will be described using the
algo rithm perfo rmance classificatio n.
Set Interface
We start with the Se t interface because it adds no metho ds to the Co llectio n interface; it o nly co nstrains
metho ds that add an element to a Se t . Specifically, a Se t is a Co llectio n that co ntains no duplicate elements
and never co ntains null as an element. The interface matches the mathematical co ncept o f a set. This
co nstraint changes the behavio r o f add(E e le m e nt ) to return f alse if the pro vided element already exists in
the Se t .
The Co llectio ns Framewo rk is designed to co nfo rm to basic Java principles, so there are so me subtle
changes to bo th the standard e quals and hashCo de metho ds. The e quals metho d is the standard means
in Java to determine whether two o bjects are equal. When dealing with two Co llectio n o bjects, yo u'll o ften
want to determine if they represent the same co llectio n o f o bjects regardless of the actual concrete class
implementing this interface. Ho wever, it wo uld be cumberso me to require this capability o f all Co llectio n
subclasses. Java's designers have so lved that pro blem by requiring that Se t implementatio ns o nly suppo rt
e quals with o ther Se t implementatio ns. The main reaso n fo r this is that Se t o bjects canno t be o rdered, so
the e quals metho d fo r all implementatio ns o f Se t return f alse whenever the pro vided o bject is itself no t a
class that implements Se t . In o ther wo rds, Se t o bjects can o nly be co mpared fo r equality against o ther Se t
o bjects.
Finally, when se t 1.e quals(se t 2) is true, it must be the case that se t 1.hashCo de () equals
se t 2.hashCo de (). This is the essential relatio nship between the hashCo de and e quals metho ds as
required by the Java specificatio n. Ho wever, classes that implement Se t may cho o se ho w to implement
these metho ds. To ensure the pro per relatio nship between e quals and hashCo de , regardless o f the specific
implementatio n, any implementatio n o f Se t must make sure that its hashCo de metho d returns the sum o f
the hashCo de o f all o f its members. Because additio n is a co mmuting o peratio n (that is, a+b equals b+a),
this ensures that the respective hashCo de values o f two Se t o bjects will always be the same if they
represent the same elements. This design principle is incredibly insightful because it separates the co ntract o f
the interface fro m any o f its po tential implementatio ns.
Let's reimplement the sample pro blem described earlier, which actually seems mo re easily implemented
using sets. Fo r this implementio n, replace ArrayList with HashSe t :
CODE TO TYPE: Mo dify UniqueLetters class
import java.util.*;
public class UniqueLetters {
static final String vowels = "aeiou";
public static void main(String[] args) {
System.out.println("Enter at least one character:");
Scanner sc = new Scanner (System.in);
String s = sc.nextLine();
Collection<Character> unique = new HashSetArrayList<Character>();
for (int i = 0; i < s.length(); i++) {
char c = Character.toLowerCase(s.charAt(i));
if (Character.isWhitespace(c)) { continue; }
if (!unique.contains(c)) {
unique.add(c);
}
}
if (unique.isEmpty()) {
System.out.println("Please enter at least one character");
System.exit(1);
}
System.out.println("There are " + unique.size() + " unique characters");
System.out.print(" Vowels:");
for (int i = 0; i < vowels.length(); i++) {
char c = vowels.charAt(i);
if (unique.contains(c)) {
System.out.print(c);
unique.remove(c);
}
}
System.out.print("\n Consonants:");
for (Iterator<Character> it = unique.iterator(); it.hasNext(); ) {
System.out.print(it.next());
}
System.out.println();
}
}
No w execute the mo dified Unique Le t t e rs class using the same input as befo re to generate the same
o utput:
INTERACTIVE SESSIONS: Output o f UniqueLetter remains the same
Enter at least one character:
Now is the time for all good men to come to the aid of their country
There are 18 unique characters
Vowels:aeiou
Consonants:fgdcnlmhwtsry
A HashSe t o ffers impro ved perfo rmance fo r the co re o peratio ns described earlier by using a scheme that
subdivides a co llectio n into b buckets. Elements are placed into a bucket based o n the hashing functio n,
hashCo de (); these perfo rmance characteristics are valid if the hashing functio n disperses the elements
pro perly amo ng the buckets.
O(1): size, isEmpty, co ntains, add, remo ve
O(n): iterate o ver the set o f elements
When yo u use a HashSe t , yo u have the o ptio n o f predeclaring an initial Capacity which specifies the number
o f buckets to use to sto re the co llectio n. If yo u set this to o high, the iteratio n o ver all elements in the set
requires time pro po rtio nal to n+b (where b is the number o f buckets in the HashSe t ); it's better to let
HashSe t manage its o wn structure.
List Interface
The List interface pro vides a sequence-o riented perspective o n a co llectio n. First, it is o rdered, which
pro vides the first distinctio n with regard to Se t . Seco nd, a List may co ntain duplicate elements o r even null
values. Third, this interface is much mo re po werful than the "list" co ncepts that mo st pro grammers intuitively
have in mind—specifically, the List interface o ffers additio nal behavio rs to Co llectio n:
Inde x (po sit io nal) acce ss. Yo u can retrieve, remo ve o r replace any element by its po sitio n in the
List. Yo u can also insert elements into a List at a specific lo catio n, bumping all elements up by o ne
spo t fro m that po int in the list.
Se arch. Yo u can identify the o rdinal lo catio n (0 .. n-1) in the list o f a given element (fro m either the
fro nt o r the end o f the list).
As with Se t , any class implementing List must pro perly implement the e quals() metho d to return f alse
whenever it is co mpared against a no n-List o bject. Two List o bjects are co nsidered to be equal if they are o f
the same size and they co ntain the same elements in the same o rder. Ho wever, the hashCo de metho d must
be defined to wo rk with e quals. Fo r this reaso n, any class that implements List must be sure that its
hashCo de metho d fo llo ws the co ntract as defined by the List class.
Yo u've already seen ho w useful ArrayList can be. Ano ther useful List implementatio n is Linke dList , which
implements a do ubly-linked list o f items. Why wo uld yo u cho o se to use o ne class o ver the o ther? Suppo se
yo u wanted to implement a double-ended queue (o r "dequeue" fo r sho rt; pro no unced "deek"). The fo llo wing
benchmark directly co mpares ArrayList to Linke dList . First, it creates a list by adding integer elements at
the "end" o f the list and then rando mly remo ving either the first o r he last element in the list. Once max
iteratio ns have co mpleted, it drains the remaining elements o f the list by repeatedly remo ving the first o ne.
Create a Co m pare De que ue class in the default package o f the /src so urce fo lder:
CODE TO TYPE: Co mpareDequeue class
import java.util.*;
public class CompareDequeue {
static long[] performance(int max, List<Integer> list) {
long nanoTime = System.nanoTime();
while (max > 0) {
list.add(max);
list.add(max+1);
if (Math.random() < 0.5) {
list.remove(0);
} else {
list.remove(list.size()-1);
}
max--;
}
long inner = System.nanoTime();
while (!list.isEmpty()) {
list.remove(0);
}
long lastTime = System.nanoTime();
return new long[]{ inner-nanoTime, lastTime-inner};
}
public static void main(String[] args) {
int max = 65536;
float m = 1000000;
System.out.println("\t\tConstruct\tDrain\t\tTotal");
System.gc();
long[] arraylist = performance(max, new ArrayList<Integer>());
System.out.println("ArrayList\t" + arraylist[0]/m + "\t" + arraylist[1]/m +
"\t" + (arraylist[0] + arraylist[1])/m);
System.gc();
long[] linkedlist = performance(max, new LinkedList<Integer>());
System.out.println("Linkedlist\t" + linkedlist[0]/m + "\t" + linkedlist[1]/m
+
"\t" + (linkedlist[0] + linkedlist[1])/m);
}
}
Save and run this pro gram to pro duce the perfo rmance pro file. The co de measures the executio n time o f
two phases o f the pro gram (gro wing phase and draining phase). The final number in the co lumn is the to tal
time in milliseco nds:
INTERACTICE SESSION: Output fro m Co mpareDequue (Time in milliseco nds)
Construct Drain Total
ArrayList 253.11002 469.56882 722.67883
Linkedlist 17.202131 1.695052 18.897184
In this benchmark, Linke dList perfo rms much faster than ArrayList . ArrayList suffers in co mpariso n
because it must co nstantly resize its internal array to meet the gro wing demand. In additio n, when remo ving
the first element fro m the ArrayList , all subsequent items in the List must be co pied do wn.
So , what if yo u made a small change to the benchmark? Remo ve a rando m element instead o f just remo ving
an element fro m either the head o r tail o f the list. Mo dify the co de as sho wn:
CODE TO TYPE: Co mpareDequeue class
import java.util.*;
public class CompareDequeue {
static long[] performance(int max, List<Integer> list) {
long nanoTime = System.nanoTime();
while (max > 0) {
list.add(max);
list.add(max+1);
list.remove(max % list.size());
if (Math.random() < 0.5) {
list.remove(0);
} else {
list.remove(list.size()-1);
}
max--;
}
long inner = System.nanoTime();
while (!list.isEmpty()) {
list.remove(0);
}
long lastTime = System.nanoTime();
return new long[]{ inner-nanoTime, lastTime-inner};
}
public static void main(String[] args) {
int max = 65536;
float m = 1000000;
System.out.println("\t\tConstruct\tDrain\t\tTotal");
System.gc();
long[] arraylist = performance(max, new ArrayList<Integer>());
System.out.println("ArrayList\t" + arraylist[0]/m + "\t" + arraylist[1]/m +
"\t" + (arraylist[0] + arraylist[1])/m);
System.gc();
long[] linkedlist = performance(max, new LinkedList<Integer>());
System.out.println("Linkedlist\t" + linkedlist[0]/m + "\t" + linkedlist[1]/m
+
"\t" + (linkedlist[0] + linkedlist[1])/m);
}
}
Save and run it; the tables have turned!
INTERACTIVE SESSION: Output fro m Co mpareDequue (Time in milliseco nds)
Construct Drain Total
ArrayList 320.66714 466.6363 787.3034
Linkedlist 1834.3232 1.668898 1835.9921
When cho o sing the appro priate data structure, yo u need to understand exactly ho w the data structure is to be
used, especially if it is updated frequently, yo u need rando mized access to any element, o r the ratio o f
add/remo ve co mpared to the number o f queries is used to determine whether an element exists in the
co llectio n.
Queue Interface
A Queue is a data structure that allo ws yo u to remo ve an element o nly at the head o f an o rdered sequence
and insert elements o nly at the end o f the sequence. The actual implementatio n determines whether the
Queue is first-in, first-o ut (what wo uld be expected) o r a variatio n (such as a last-in, first-o ut). Further, o ne can
o nly remo ve an element fro m the head o f the queue as determined by the implementatio n. There is a Deque
interface that extends Queue to o ffer do uble-ended queueing behavio r.
In the Co llectio ns Framewo rk, a Queue specifies a co llectio n designed to ho ld elements prio r to pro cessing.
The Queue interface extends Co llectio n. This interface is designed to suppo rt co llectio ns that have a
maximum size (such as bo unded queues) in additio n to mo re general queues with no restrictio ns. Yo u can
offer an element to the Queue, which simply returns f alse if the Queue denies this request (usually because it
is a bo unded queue). The remaining fo ur metho ds (re m o ve , po ll, e le m e nt , and pe e k) each return (witho ut
mo difying the queue) o r remo ve the element at the head o f the queue.
It may seem o dd that a Queue is no t predefined to be a subinterface o f List. Ho wever, it is critical when
designing framewo rks to separate structure fro m behavio r. Using list-based semantics is no t the o nly way to
implement a queue (yo u co uld use circular buffers, fo r example, which is an efficient way to implement a
bo unded queue). The Linke dList class cho o ses to implement bo th List and De que (and thus by extensio n,
Queue). ArrayList do es no t implement Queue tho ugh, likely because its perfo rmance as a queue wo uld be
ho rrible.
Map Interface
The Map interface in the Co llectio ns Framewo rk allo ws yo u to create co llectio ns that asso ciate a value with a
unique key. Given a Map, yo u can add (o r replace) a key mapping with put (ke y,value ); to retrieve a value o r
to determine whether the value exists in the Map, use ge t (ke y).
The designers o f Map had to decide whether a Map was a Co llectio n o bject; after all, it sto res a co llectio n o f
values. Ho wever, the fundamental o peratio n o n a Co llectio n is the add metho d, and there is no easy way to
apply this o peratio n to a Map. Instead, Map o bjects o ffer two Co llectio n views o ver its o bjects. Because the
keys in a Map are unique, yo u can retrieve the Se t o f all keys in the Map, but their values might no t be unique,
so Map o nly allo ws yo u to retrieve the Co llectio n o f values in the map.
Map is designed to o ptimize the insertio n and remo val o f (key, value) pairs. In do ing so , any o rdering
pro perties amo ng the keys are lo st. Fo r example, try to o rder the keys in a Map alphabetically.
Create a So rt ingMap class in the default package o f the /src so urce fo lder:
CODE TO TYPE: So rting key values in map
import java.util.*;
public class SortingMap {
public static void main(String[] args) {
float m = 1000000;
Map<String,Integer> map = new HashMap<String,Integer>();
long start = System.nanoTime();
byte[] word = new byte[3];
for (byte c0 = 'A'; c0 <= 'Z'; c0++) {
word[0] = c0;
for (byte c1 = 'A'; c1 <= 'Z'; c1++) {
word[1] = c1;
for (byte c2 = 'A'; c2 <= 'Z'; c2++) {
word[2] = c2;
String s = new String(word);
map.put(s, c0*c1*c2);
}
}
}
long created = System.nanoTime();
ArrayList<String> keys = new ArrayList<String>(map.keySet());
Collections.sort(keys);
long sorted = System.nanoTime();
long total = 0;
for (String k : keys) {
total += map.get(k);
}
long done = System.nanoTime();
System.out.println("Total\t" + total);
System.out.println("Created\t" + (created-start)/m +
"\nSorted\t" + (sorted-created)/m +
"\nDone\t" + (done - sorted)/m);
int print = 10;
for (String s : map.keySet()) {
if (--print < 0) { break; }
System.out.println(s);
}
}
}
Save and run it.
INTERACTIVE SESSION: Output o f So rtedMap
Total
Created
Sorted
Done
GDC
GDD
GDA
GDB
GDK
GDL
GDI
GDJ
GDG
GDH
8181353375
99.77721
68.29778
13.256433
The co de inserts pairs (s, n) fo r max entries and then o utputs the keys in so rted value. To do that, it must
retrieve the ke ySe t () fro m the Map, but the set must be co nverted into a List so it can be so rted, so an
ArrayList keys is co nstructed fro m the set o f keys. Finally, it so rts the keys using the Co lle ct io ns.so rt
metho d. The executio n sho ws the perfo rmance, as well as the first ten keys in the Map; no te that these keys
aren't so rted because the Map iteratio n do es no t maintain o rdering.
Summarizing the Implementations You Need T o Know
We refer to a number o f co mmo n implementatio ns pro vided fo r yo u in the JDK. These are the "go to " classes
yo u'll use again and again to so lve yo ur pro gramming issues. Yo u must never re-implement these data
structures o n yo ur o wn; these implementatio ns have already been fine-tuned by experts.
Int e rf ace
Hash t able
Re sizable array
T re e
Linke d List
Im ple m e nt at io ns Im ple m e nt at io ns Im ple m e nt at io ns Im ple m e nt at io ns
Set
HashSet
List
Map
TreeSet
ArrayList
HashMap
LinkedList
TreeMap
Each o f these default implementatio ns pro vide distinct perfo rmance pro files fo r the Co llectio n metho ds. In
additio n, bo th List and Map add metho ds to their interface definitio ns. It's impo rtant to differentiate between
the behavio rs o f the classes that implement these interfaces as well.
Important Methods For Keys And Values
All o f the classes in the Co llectio n Framewo rk suppo rt Java Generics, so yo u do n't just refer to an ArrayList ,
but an ArrayList <St ring>. Do ing so makes yo ur co de mo re ro bust because it enables the co mpiler to
detect many class cast exceptio ns. The co nt ains metho d—co mmo n to many Co llectio ns classes— must be
implemented pro perly, o therwise the Co llectio n class wo n't wo rk. Co nsider this example:
Create a class T uple in the default package o f the /src so urce fo lder:
CODE TO TYPE: Tuple class
public class Tuple {
String value;
int
x;
int
y;
public Tuple (String v, int x, int y) {
this.value = v;
this.x = x;
this.y = y;
}
public String toString () {
return "(" + value + "," + x + "," + y + ")";
}
}
No w, create the fo llo wing driver class named T uple Drive r in the default package o f the /src so urce fo lder:
CODE TO TYPE: TupleDriver class
import java.util.*;
public class TupleDriver {
public static void main(String[] args) {
ArrayList<Tuple> al = new ArrayList<Tuple>();
Tuple t1 = new Tuple("Sample", 10, 20);
al.add(t1);
System.out.println("ArrayList Contains tuple:" + al.contains(t1));
Tuple t2 = new Tuple("Sample", 10, 20);
System.out.println("ArrayList Contains tuple:" + al.contains(t2));
}
}
Save and run the abo ve co de to see so me surprising o utput:
INTERACTIVE SESSION: Output o f TupleDriver
ArrayList Contains tuple:true
ArrayList Contains tuple:false
Altho ugh T uple t1 and t2 are exactly the same, o nly o ne o f them is fo und in the ArrayList . Yo u see this
behavio r because yo u haven't implemented the requisite e quals(Obje ct o ) metho d required by the
Co llectio ns Framewo rk. The co nt ains(o ) metho d will return t rue if there is an o bject in the Co llectio n fo r
which e quals(o ) is t rue . Go ahead and add the e quals metho d no w:
CODE TO TYPE: Mo dified Tuple class
public class Tuple {
String value;
int
x;
int
y;
public Tuple (String v, int x, int y) {
this.value = v;
this.x = x;
this.y = y;
}
public boolean equals (Object o) {
if (o == null) { return false; }
if (!(o instanceof Tuple)) { return false; }
Tuple other = (Tuple) o;
if (value == null) {
if (other.value != null) {
return false;
}
} else if (!value.equals(other.value)) {
return false;
}
return x == other.x && y == other.y;
}
public String toString () {
return "(" + value + "," + x + "," + y + ")";
}
}
Review the e quals metho d. It sho uld no t thro w any Exceptio n, but rather handle all cases (such as null
o bject references). This co de is able to wo rk with T uple o bjects that have a value String attribute o f null. No w
go back and rerun the T uple Drive r:
INTERACTIVE SESION: Output o f TupleDriver
ArrayList Contains tuple:true
ArrayList Contains tuple:true
No w let's try to use this T uple as a key in HashSe t . Mo dify T uple Drive r as sho wn:
CODE TO TYPE: Mo dified TupleDriver class
import java.util.*;
public class TupleDriver {
public static void main(String[] args) {
ArrayList<Tuple> al = new ArrayList<Tuple>();
Tuple t1 = new Tuple("Sample", 10, 20);
al.add(t1);
System.out.println("ArrayList Contains tuple:" + al.contains(t1));
Tuple t2 = new Tuple("Sample", 10, 20);
System.out.println("ArrayList Contains tuple:" + al.contains(t2));
HashSet<Tuple> values = new HashSet<Tuple>();
values.add(t1);
System.out.println("HashSet Contains tuple:" + values.contains(t1));
System.out.println("HashSet Contains tuple:" + values.contains(t2));
}
}
save and run it again:
INTERACTIVE SESSION: Output o f TupleDriver
ArrayList Contains tuple:true
ArrayList Contains tuple:true
HashSet Contains tuple:true
HashSet Contains tuple:false
When using a class as a key value in any o f the "Hash" co llectio n classes (that is, HashSe t , HashMap,
Linke dHashSe t , o r Linke dHashMap) yo u need to implement the hashCo de () metho d. Specifically, if two
o bjects are equal, their hashCo de () values must be identical. If yo u do n't pro vide yo ur o wn hashCo de
metho d, then yo u will inherit the default fro m java.lang.Obje ct which means hashCo de () values must be
identical. Mo dify T uple to add a reaso nable implementatio n o f hashCo de .
CODE TO TYPE: Mo dified Tuple class
public class Tuple {
String value;
int
x;
int
y;
public Tuple (String v, int x, int y) {
this.value = v;
this.x = x;
this.y = y;
}
public boolean equals (Object o) {
if (o == null) { return false; }
if (!(o instanceof Tuple)) { return false; }
Tuple other = (Tuple) o;
if (value == null) {
if (other.value != null) {
return false;
}
} else if (!value.equals(other.value)) {
return false;
}
return x == other.x && y == other.y;
}
public int hashCode() {
int hash = 0;
if (value != null) { hash += value.hashCode(); }
return hash + x + y;
}
public String toString () {
return "(" + value + "," + x + "," + y + ")";
}
}
The hashCo de metho d must always return the same value upo n each executio n. Because all o f the
fundamental classes in the JDK have suitable hashCo de metho ds, yo u may want to co mpo se yo ur o wn
metho ds using their values in numerical co mputatio ns. Rerun T uple Drive r:
INTERACTIVE SESSION: Output o f TupleDriver
ArrayList Contains tuple:true
ArrayList Contains tuple:true
HashSet Contains tuple:true
HashSet Contains tuple:true
But wait—we're no t do ne. Yo u have to be especially careful with classes that have o bjects that will be key
values. Specifically, yo u must make these classes immutable, o therwise strange things can happen. Fo r
example, mo dify T uple Drive r as sho wn:
CODE TO TYPE: Mo dified TupleDriver class
import java.util.*;
public class TupleDriver {
public static void main(String[] args) {
ArrayList<Tuple> al = new ArrayList<Tuple>();
Tuple t1 = new Tuple("Sample", 10, 20);
al.add(t1);
System.out.println("ArrayList Contains tuple:" + al.contains(t1));
Tuple t2 = new Tuple("Sample", 10, 20);
System.out.println("ArrayList Contains tuple:" + al.contains(t2));
HashSet<Tuple> values = new
values.add(t1);
System.out.println("HashSet
t1.value = "Other";
System.out.println("HashSet
System.out.println("HashSet
HashSet<Tuple>();
Contains tuple:" + values.contains(t1));
Contains tuple:" + values.contains(t1));
Contains tuple:" + values.contains(t2));
}
}
save and run it again:
INTERACTIVE SESSIONS: Output o f TupleDriver
ArrayList Contains tuple:true
ArrayList Contains tuple:true
HashSet Contains tuple:true
HashSet Contains tuple:false
HashSet Contains tuple:false
Once yo u change the value o f an o bject that has been used as a key value within HashSe t , yo u may no t be
able to find it again. Use the f inal mo difier fo r the attributes o f yo ur key classes to avo id these situatio ns:
CODE TO TYPE: Mo dified Tuple class
public class Tuple {
final String value;
final int
x;
final int
y;
public Tuple (String v, int x, int y) {
this.value = v;
this.x = x;
this.y = y;
}
public boolean equals (Object o) {
if (o == null) { return false; }
if (!(o instanceof Tuple)) { return false; }
Tuple other = (Tuple) o;
if (value == null) {
if (other.value != null) {
return false;
}
} else if (!value.equals(other.value)) {
return false;
}
return x == other.x && y == other.y;
}
public int hashCode() {
int hash = 0;
if (value != null) { hash += value.hashCode(); }
return hash + x + y;
}
public String toString () {
return "(" + value + "," + x + "," + y + ")";
}
}
With this change, the T uple Drive r class no lo nger co mpiles because it's trying to mo dify the (no w
immutable) attribute. Delete the o ffending line in T uple Drive r. No w yo u have a wo rking implementatio n o f a
class suitable fo r use as a key value in any o f the "Hash" Co llectio ns classes.
Lessons Learned
The Co llectio ns Framewo rk co ntains many implementatio ns o f the fundamental data structures that yo u'll use
in yo ur algo rithms. When yo u Understand these structures it will help yo u write efficient co de:
Rat he r t han re im ple m e nt yo ur o wn, use de f ault im ple m e nt at io ns o f Se t , List , and
Que ue . The default implementatio ns are made to wo rk efficiently under the mo st co mmo n
usages.
Se e k dat a st ruct ure s t hat le ad t o O(n lo g n) be havio r. Many pro blems have "naive"
so lutio ns that result in O(n 2 ) run-time perfo rmance. In mo st cases yo u'll be able to increase
perfo rmance to O(n log n) by applying the appro priate data structure. Yo u'll see this happen o ften in
furture lesso ns.
The Co llectio ns Framewo rk co ntains a Co lle ct io ns class that has a number o f static metho ds useful fo r
yo ur algo rithms. Again, these metho ds are o ptimized fo r use by the vario us classes in the Co llectio ns
Framewo rk. Use these metho ds rather than reimplementing them. Each o f these metho ds has a published
perfo rmance co ntract to which it adheres, which makes it po ssible to use them and be assured o f reaso nably
go o d perfo rmance. Review the class in the API do cumentatio n when yo u get a chance. Here are fo ur that
yo u'll use o ften:
so rt(List): efficiently so rt the List in place in O(n log n) time.
binarySearch(List): assuming that List is o rdered, lo cate the index o f the given key in O(log n) time.
reverse(List, key): lo cate the key in the o rdered List o bject in O(log n) time.
shuffle(List): permutes the List in place in O(n) time.
This lesso n co vered just the highlights o f the Co llectio ns Framewo rk. The java.ut il package co ntains 28 3
classes and 19 interfaces. To delve deeper into Co llectio ns, fo llo w the standard Tuto rial o n the Co llectio ns
framewo rk after yo u co mplete this co urse.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Algorithms Using Java
Lesson Objectives
After co mpleting this lesso n, yo u will be able to :
maintain the o rdering pro perty o f a List when inserting values.
use a TreeSet to iterate o ver the elements o f a set in their natural o rdering.
Designing Algorithms
There are well-kno wn algo rithms yo u can use to so rt an array o f strings, such as QuickSo rt, but the real value o f an
algo rithm is that it is a co ncise explainatio n o f an efficient way to so lve a specific pro blem. The algo rithms yo u will
enco unter are as varied as the pro blems they so lve. Let's start by trying to so lve a given pro blem. As yo u wo rk to ward
a so lutio n, yo u'll so lve numero us sub-tasks and make impo rtant decisio ns and develo p are essential in the pro cess.
Let's get started.
Skyline Problem
Let's say yo u have a set o f rectangular building pro files in two dimensio ns, co mpute the Skyline view based
o n the partially o verlapping and fully o bstructed building pro files at yo ur dispo sal. Assume there are seven
buildings as sho wn. No te that two buildings (4 and 5) share the same left co o rdinate, altho ugh building 5 is
taller:
The Skyline fo r the so lutio n is sho wn in re d with black shado w highlights:
If yo u so lve this pro blem by hand, yo u'd pro bably have no tro uble tracing the appro priate edges, but yo u
might find it hard to explain the exact sequence o f steps yo u to o k. No do ubt, yo u wo uld be able to determine
the skyline fo r any co nceivable set o f rectangular buildings. So , ho w can yo u write a pro gram to do the same
thing? Let's start by identifying the Input - Pro ce ss - Out put phases fo r this pro blem.
Each skyline pro blem instance is defined by a set o f buildings. A building can be represented any number o f
ways; here we'll represent a building with three integers: left, right, and height. Building 1 abo ve, fo r example,
can be defined as (left=1, right=4, height=3). The Input to o ur final pro gram will be a sequence o f text lines,
each o f which co ntains three integers separated by spaces. When co nsidering the input, ask yo urself these
questio ns: Will the sequence o f buildings in the Input already be so rted so meho w fro m left to right? Is it
po ssible fo r two buildings in the input to share the same left co o rdinate? Can two buildings have the exact
same values fo r all three co o rdinates? As yo u co nsider these questio ns, make the fewest po ssible
assumptio ns. This will help make yo ur co de mo re ro bust and able to handle diverse situatio ns.
Fo r this pro blem, assume that each building co o rdinate is represented by an integer value greater than 0 . This
prevents bizarre situatio ns (a building with zero o r negative height). Also , assume left < right fo r each building.
This ensures yo u will kno w that the smaller co o rdinate is truly "left" o f a building's right co o rdinate. There may
be duplicate co o rdinates (even buildings) in the input. It wo uld be mo re difficult to try to find and remo ve
duplicate buildings fro m the input set than to write yo ur pro gram to wo rk even when the input co ntains
duplicate buildings.
To reco rd the input, yo u will need data structures to sto re the info rmatio n abo ut each building. First yo u need
to define a way to sto re info rmatio n abo ut a building.
Create a pro ject named Algo rit hm s and assign it to yo ur J ava6 _Le sso ns wo rking set.
In the Algo rit hm s pro ject /src so urce fo lder, create a package named skyline to co ntain all the co de yo u
need to so lve this pro blem.
Create a Building class in the /src so urce fo lder skyline package to represent a building:
CODE TO TYPE: Building class
package skyline;
public class Building {
final public int left;
final public int right;
final public int height;
public Building(int left, int right, int height) {
this.left = left;
this.right = right;
this.height = height;
if (left <= 0 || right <= 0 || height <= 0 || left >= right) {
throw new IllegalArgumentException ("Invalid building parameters: " +
left + "," + right + "," + height);
}
}
public String toString () {
return "[" + left + "," + right + "] @ " + height;
}
}
This class represents a valid building; any attempt to co nstruct an invalid Building o bject will thro w an
Exceptio n. The input fo r a Skyline pro blem instance co nsists o f a set o f Building o bjects which yo u can sto re
as an ArrayList <Building> o bject.
Create a Skyline class in the skyline package o f the /src so urce fo lder and enter the co de as sho wn (the
co de pro cesses a set o f text lines representing buildings and the o utputs the buildings it fo und):
CODE TO TYPE: Skyline class
package skyline;
import java.io.*;
import java.util.*;
public class Skyline {
public static Collection<Building> retrieveInput(InputStream is) {
ArrayList<Building> buildings = new ArrayList<Building>();
Scanner sc = new Scanner (is);
while (sc.hasNextLine()) {
String s = sc.nextLine();
if (s.equals("")) { break; }
try {
StringTokenizer st = new StringTokenizer(s);
int left = Integer.valueOf(st.nextToken());
int right = Integer.valueOf(st.nextToken());
int height = Integer.valueOf(st.nextToken());
Building b = new Building (left, right, height);
buildings.add(b);
} catch (NumberFormatException nfe) {
System.err.println(" ** Ignoring " + s + ": all values must be integers.
");
} catch (Exception e) {
System.err.println(" ** Ignoring " + s + ": " + e.getMessage());
}
}
return (buildings);
}
public static void main(String[] args) {
Collection<Building> buildings = retrieveInput(System.in);
for (Building b : buildings) {
System.out.println("[" + b.left + "," + b.right + "] @ " + b.height);
}
}
}
This "scaffo lding" co de co ntains a re t rie ve Input metho d that reads a series o f lines that represent the
buildings in the Skyline pro blem instance. re t rie ve Input returns an ArrayList o f Building o bjects that it
parsed. When in do ubt as to which class to use when representing a list, start with ArrayList . The m ain
metho d o utputs the building info rmatio n fo r all buildings retrieved fro m the input.
Run Skyline with the input set belo w (press Ent e r twice when yo u finish); the o utput sho ws that all buildings
were pro perly pro cessed by this scaffo lding class:
INTERACTIVE SESSION: Demo nstrate Skyline pro cesses sample input
1 4 3
6 7 1
8 15 4
8 11 5
9 12 3
2 5 4
13 16 5
[1,4] @ 3
[6,7] @ 1
[8,15] @ 4
[8,11] @ 5
[9,12] @ 3
[2,5] @ 4
[13,16] @ 5
No w yo u are ready to co nsider ho w to represent a valid so lutio n to the Skyline pro blem—ho w the Output
sho uld be represented. In the abo ve graphic, the so lutio n is represented as a sequence o f "(x,y)" po ints that
determines the Skyline fro m left to right. The bo tto m o f each building is y=0 and the first po int in the so lutio n is
(L, 0 ) where L is the left co o rdinate o f the leftmo st building. Finally, the last po int in the so lutio n is (R, 0 ) where
R is the right co o rdinate o f the rightmo st building. Because the buildings are rectangular, yo u kno w that the
Skyline is co mpo sed o f a sequence o f alternating vertical and ho rizo ntal edges. With this info rmatio n in hand,
yo u can see that yo ur pro gram sho uld co mpute the set o f edges fro m the building info rmatio n in the Input.
Fro m the sequence o f edges, yo u can easily co mpute the sequence o f po ints in the Skyline.
When tackling a co mplex pro blem, it really helps to spend the time (like yo u did) to understand the input and
o utput requirements. No w yo u're ready to address the Pro cess phase o f this algo rithm. When trying to
determine ho w to so lve the Skyline pro blem, start with what yo u kno w. Yo u kno w that the leftmo st vertical
(re d) edge o f the leftmo st building will fo rm the start o f the Skyline; the first po int is (1,0 ) and the "end" o f the
Skyline is (1,3). Let's use two variables to co mpute the Skyline; right X=1 and t o pY=3 are the x- and yco o rdinates o f the rightmo st po int in the Skyline. Starting with a first (blue ) building, co nsider three different
po ssibilities when pro cessing the "next" seco nd building to determine ho w to extend the Skyline. The seco nd
building is the building in the input set with the left co o rdinate that is clo sest to the left co o rdinate o f the first
building.
In Case One , the seco nd building is taller than the first building, so the Skyline rises (as sho wn by thick black
lines) adding the po int (2,3) and ending at the "next end" o f (2,4 ). Ho wever, in Case T wo , the seco nd building
is smaller than the first building, so the Skyline will co me back do wn (again, sho wn in thick black lines) adding
the po int (4 ,3) and ending at the "next end" o f (4 ,2). In additio n to maintaining the "end po int" o f the Skyline
(right X, t o pY) yo u also need to kno w buildingRight , o r the right co o rdinate o f the current building being
pro cessed (in Case T wo yo u need this value so yo u kno w where to "turn").
No w, neither o f these cases handles the situatio n when the seco nd building do esn't actually o verlap the first
building (as sho wn by Case T hre e ). We can handle this case tho ugh because the Skyline co mes back do wn
to "gro und zero " and then co ntinues back up the left edge o f the seco nd building. Here three po ints are added
to the Skyline—(2,3), (2,0 ), (3,0 ) and yo u end up with a "next end" o f (3,2).
These cases lay the fo undatio n fo r an algo rithm. No w yo u need to co nsider the initializatio n phase o f the
algo rithm (where do yo u start?) and the terminatio n phase (ho w do yo u end?). To start this algo rithm, yo u
need to find the leftmo st building with tallest height (just in case two o r mo re buildings start with the same
smallest co o rdinate) and start the Skyline with its vertical edge. To terminate this algo rithm, there will be no
seco nd building (since they will have all been pro cessed), so yo u can "clo se" the algo rithm by extending the
Skyline to buildingRight and then back do wn to the "gro und zero ".
Befo re implementing the algo rithm, yo u sho uld describe its lo gic using pseudocode; this allo ws yo u to see
the structure witho ut the co mplicated syntax o f a regular pro gramming language. When sketching an
algo rithm using pseudo co de, yo u can define helper functio ns as needed. Define a ne xt T alle st (x) functio n
that returns the building who se left co o rdinate is clo sest to the right o f x. In the event o f a "tie," this functio n
must return the tallest o f all such buildings starting at that co o rdinate. Review the fo llo wing pseudo co de
descriptio n and no te ho w Case T hre e is handled first, since that detects when the next building do esn't
intersect the "current building." With each pass thro ugh the lo o p, curre nt and ne xt are updated acco rdingly
to represent the index o f the current building being pro cessed and the next building to pro cess.
OBSERVE: Pseudo co de descriptio n o f algo rithm
compute ()
current = nextTallest(0)
rightX = current.left
topY = current.height
skyline starts with (rightX, 0) and (rightX, topY)
buildingRight = current.right
while exists next = nextTallest(rightX)
if next.left > buildingRight then
add edges to skyline according to
rightX = next.left
else if next.height > topY then
add edges to skyline according to
rightX = next.left
else if next.height < topY then
add edges to skyline according to
rightX = current.right
do
Case Three
Case One
Case Two
topY = next.height
buildingRight = next.right
current = next
close skyline by adding (buildingRight, topY) and (buildingRight, 0)
Befo re jumping into implementatio n, review ho w this algo rithm wo uld wo rk o n the sample pro blem yo u saw
earlier. If yo u fo llo w the abo ve pseudo co de o n the sample data, yo u will see that it inco rre ct ly co m put e s
this Skyline:
The actual po ints in the Skyline are:
OBSERVE: Skyline pro cesses sample input
[(1,0) - (1,3)]
[(1,3) - (2,3)]
[(2,3) - (2,4)]
[(2,4) - (5,4)]
[(5,4) - (5,0)]
[(5,0) - (6,0)]
[(6,0) - (6,1)]
[(6,1) - (7,1)]
[(7,1) - (7,0)]
[(7,0) - (8,0)]
[(8,0) - (8,5)]
[(8,5) - (11,5)]
[(11,5) - (11,3)]
[(11,3) - (12,3)]
[(12,3) - (12,0)]
[(12,0) - (13,0)]
[(13,0) - (13,5)]
[(13,5) - (16,5)]
[(16,5) - (16,0)]
Note
** Here is where the Skyline is incorrect **
In my first attempt to so lve this pro blem, I co ded the so lutio n before I co nsidered all po ssible
cases, then realized the implementatio n was inco rrect. Yo u can avo id wasting time o n faulty
implementatio ns by checking yo ur pseudo co de against a real example.
As yo u unco ver the missing case that yo u hadn't co nsidered befo re, it lo o ks like the who le appro ach will have
to change.
In Case Fo ur, o nce yo u have pro cessed the seco nd building, the height o f the next Skyline po int will be at the
o riginal first building, no t the third building pro cessed. The o riginal appro ach fo cused to o much o n the le f t
co o rdinates o f the buildings; no w yo u can see that the right co o rdinate o f the buildings is just as impo rtant.
Let's appro ach the pro blem fro m ano ther perspective. Ano ther way to define the Skyline fo r a set o f buildings
is to co nsider o nly the to ps o f buildings, that is, just the ho rizo ntal edges. The to p o f a building at a given xco o rdinate is part o f the Skyline if no other building at that x-coordinate is taller. This is a simpler way to
appro ach the who le pro blem; o ften after yo u have wo rked o n a pro blem fo r so me time, a simpler so lutio n will
materialize. The earlier o bservatio n that the Skyline co ntained alternating vertical and ho rizo ntal edges was
actually a distractio n fro m this simpler appro ach.
The revised appro ach to this pro blem (as sho wn abo ve in Case Fo ur) still invo lves attempts to "sweep" the
buildings fro m left to right, but no w it gives equal weight to bo th the left- and right- co o rdinates o f a building.
No w yo u need to maintain an o rdered heightList o f buildings (fro m tallest do wn to smallest) as the xco o rdinate sweeps fro m left to right. The o rdered list (represented by a chain o f circles) keeps track o f the
buildings. When yo u sweep x fro m left to right and disco ver the left edge o f a building at that x-co o rdinate, add
the building to this o rdered list at its proper location (according to its height). When the x-co o rdinate matches
the right-co o rdinate o f a building in heightList, re m o ve that building fro m the list. No w co mes the key
o bservatio n: Whenever the top of the ordered list changes, the Skyline changes as well.
Here's the expected behavio r when we manage the heightList structure fo r the sample input set described
earlier:
Let's get started with adding co de to sweep thro ugh the co o rdinates and co nstructs heightList. Mo dify the
Skyline class as sho wn:
CODE TO TYPE: Mo dificatio ns to Skyline class
package skyline;
import java.io.*;
import java.util.*;
public class Skyline {
public static Collection<Building> retrieveInput(InputStream is) {
ArrayList<Building> buildings = new ArrayList<Building>();
Scanner sc = new Scanner (is);
while (sc.hasNextLine()) {
String s = sc.nextLine();
if (s.equals("")) { break; }
try {
StringTokenizer st = new StringTokenizer(s);
int left = Integer.valueOf(st.nextToken());
int right = Integer.valueOf(st.nextToken());
int height = Integer.valueOf(st.nextToken());
Building b = new Building (left, right, height);
buildings.add(b);
} catch (NumberFormatException nfe) {
System.err.println(" ** Ignoring " + s + ": all values must be integers.
");
} catch (Exception e) {
System.err.println(" ** Ignoring " + s + ": " + e.getMessage());
}
}
return (buildings);
}
public static void compute(Collection<Building> buildings) {
TreeSet<Integer> S = new TreeSet<Integer>();
HashMap<Integer,ArrayList<Building>> lefts = new HashMap<Integer,ArrayList<B
uilding>>();
HashMap<Integer,ArrayList<Building>> rights = new HashMap<Integer,ArrayList<
Building>>();
ArrayList<Building> list = null;
for (Building b : buildings) {
S.add(b.left);
list = lefts.get(b.left);
if (list == null) {
list = new ArrayList<Building>();
lefts.put(b.left, list);
}
list.add(b);
S.add(b.right);
list = rights.get(b.right);
if (list == null) {
list = new ArrayList<Building>();
rights.put(b.right, list);
}
list.add(b);
}
ArrayList<Building> heightList = new ArrayList<Building>();
for (int x : S) {
list = rights.get(x);
if (list != null) {
for (Building b : list) {
heightList.remove(b);
}
}
list = lefts.get(x);
if (list != null) {
for (Building b : list) {
int i;
for (i = 0; i < heightList.size(); i++) {
if (heightList.get(i).height < b.height) {
heightList.add(i, b);
break;
}
}
if (i == heightList.size()) {
heightList.add(b);
}
}
}
System.out.println(x + ":" + heightList);
}
}
public static void main(String[] args) {
Collection<Building> buildings = retrieveInput(System.in);
compute(buildings);
for (Building b : buildings) {
System.out.println("[" + b.left + "," + b.right + "] @ " + b.height);
}
}
}
Execute this pro gram o n the sample input fro m befo re:
INTERACTIVE SESSION: Maintaining HeightList
1 4 3
6 7 1
8 15 4
8 11 5
9 12 3
2 5 4
13 16 5
1:[[1,4] @ 3]
2:[[2,5] @ 4, [1,4] @ 3]
4:[[2,5] @ 4]
5:[]
6:[[6,7] @ 1]
7:[]
8:[[8,11] @ 5, [8,15] @ 4]
9:[[8,11] @ 5, [8,15] @ 4, [9,12] @ 3]
11:[[8,15] @ 4, [9,12] @ 3]
12:[[8,15] @ 4]
13:[[13,16] @ 5, [8,15] @ 4]
15:[[13,16] @ 5]
16:[]
Each line o f o utput sho ws the heightList o f buildings, in reverse o rder o f height. Co mpare the o utput to the
heightList image we just saw; the results at each x-co o rdinate accurately reflect the o rder o f buildings in
heightList at each co o rdinate. Let's take a clo ser lo o k at the co de:
OBSERVE: Creating initial data structures S, lefts and rights
public static void compute(Collection<Building> buildings) {
TreeSet<Integer> S = new TreeSet<Integer>();
HashMap<Integer,ArrayList<Building>> lefts = new HashMap<Integer,ArrayList<B
uilding>>();
HashMap<Integer,ArrayList<Building>> rights = new HashMap<Integer,ArrayList<
Building>>();
ArrayList<Building> list = null;
for (Building b : buildings) {
S.add(b.left);
list = lefts.get(b.left);
if (list == null) {
list = new ArrayList<Building>();
lefts.put(b.left, list);
}
list.add(b);
S.add(b.right);
list = rights.get(b.right);
if (list == null) {
list = new ArrayList<Building>();
rights.put(b.right, list);
}
list.add(b);
}
This co de co nstructs a set S fro m the left- and right- co o rdinates o f the buildings so it can sweep thro ugh the
co o rdinates fro m left to right. In the previo us lesso n, we learned that sets have no inherent o rdering
asso ciated with them; they simply maintain a co llectio n o f unique elements. In practice, ho wever, the
T re e Se t class in the Co llectio ns Framewo rk can sto re the elements o f a set efficiently and allow you to
iterate over these elements in sorted order. The algo rithm pseudo co de sho ws that yo u need to be able to
retrieve quickly, all buildings who se left- (o r right-) co o rdinate is a specific value. This behavio r calls fo r an
asso ciative Map o f so me kind. Here the co de creates two HashMap o bjects. le f t s enables the retrieval o f an
ArrayList o f Building o bjects that all share the same left x-co o rdinate. Similarly, the right s HashMap sto res
an ArrayList o f Building o bjects that all share the same right x-co o rdinate.
To co mpute the number o f o peratio ns in the abo ve co de, co nsider these actio ns:
The f o r lo o p executes n times.
Each add to a T re e Se t is guaranteed to perfo rm with O(log n) behavio r.
Each ge t o n a HashMap is O(1) time.
Each put o peratio n o n a HashMap is O(1) time.
Each add o n a ArrayList is amo rtized co nstant time.
The to tal number o f o peratio ns is 2*n*(O(log n) + O(1) + O(1) + Amortized Constant). The abo ve is classifed as
an O(n log n) algo rithm because tho se are the do minant terms in the co mputatio n.
Thro ugho ut this co urse yo u will be asked to evaluate the run-time perfo rmance o f an algo rithm in the same
manner. Make sure yo u understand the reaso n behind classifying the perfo rmance o f this initializatio n co de
as O(n log n).
Once le f t s, right s, and S are co nstructed, the co m put e metho d must sweep thro ugh the co o rdinates fro m
left to right. It do es so by iterating o ver all integer values in S, which are pro cessed in ascending o rder.
OBSERVE: Manage heightList
ArrayList<Building> heightList = new ArrayList<Building>();
for (int x : S) {
list = rights.get(x);
if (list != null) {
for (Building b : list) {
heightList.remove(b);
}
}
list = lefts.get(x);
if (list != null) {
for (Building b : list) {
int i;
for (i = 0; i < heightList.size(); i++) {
if (heightList.get(i).height < b.height) {
heightList.add(i, b);
break;
}
}
if (i == heightList.size()) {
heightList.add(b);
}
}
}
System.out.println(x + ":" + heightList);
}
The f o r lo o p iterates o ver every co o rdinate value x. First it re m o ve s f ro m he ight List all buildings with the
right-co o rdinate o f x; these buildings can no lo nger affect the Skyline. Then the f o r lo o p inse rt s int o
he ight List all o f the buildings with a left-co o rdinate o f x. If any e xist , the Building o bjects in that list are
inserted at the proper location in heightList. Observe ho w the abo ve co de keeps heightList in o rder (fro m
tallest to sho rtest). The clo sing println statement o utputs heightList so yo u can validate that the sweep is
wo rking pro perly.
No w that yo u have a wo rking sweep that maintains the heightList, it's time to design the revised pseudo co de
fo r the algo rithm.
OBSERVE: Pseudo co de descriptio n o f revised algo rithm
compute ()
S = set of integers containing all left- and right-coordinates of buildings
lefts = HashMap of buildings by left-coordinate
rights = HashMap of buildings by right-coordinate
heightList = empty
skyline = empty
foreach x in S in sorted order do
if heightList is empty then
top = 0
else
top = tallest building in heightList
foreach building
remove b from
foreach building
insert b into
b whose b.right=x do
heightList
b whose b.left=x do
heightList at appropriate location
if heightList is empty then
newTop = 0
else
newTop = tallest building in heightList
if top is 0 then
left = x
else if top != newTop then
add edge (left, top) - (x, top) into skyline
left = x
return skyline
The pseudo co de demo nstrates ho w to generate a set o f edges while sweeping the co o rdinates fro m left to
right. With each pass thro ugh the fo reach lo o p, the algo rithm determines if the to p o f the tallest building in
heightList changes because a building is remo ved fro m o r added to the heightList. If a change happens, then
ne wT o p != t o p and a ho rizo ntal edge can be determined fo r the Skyline. With each pass thro ugh the
f o re ach lo o p, left reco rds the mo st recent x-co o rdinate fo r pro cessing.
To co mplete the implementatio n, yo u need a class to represent the edges in the Skyline.
Create an Edge class in the skyline package o f the /src so urce fo lder:
CODE TO TYPE: Edge class
package skyline;
import java.awt.Point;
public class Edge {
final Point start;
final Point end;
public Edge (Point start, Point end) {
this.start = start;
this.end = end;
}
public String toString() {
return "[(" + start.x + "," + start.y + ") - (" + end.x + "," + end.y + ")]"
;
}
}
The Edge class simply reco rds an edge by using two java.awt .Po int o bjects. It has a co nvenient t o St ring
metho d fo r debugging.
To co mpute the Skyline o f ho rizo ntal edges yo u need to reco rd whenever the top of the ordered heightList
changes. Make these co de mo dificatio ns to Skyline :
CODE TO TYPE: Mo dificatio ns to Skyline to co mpute edges o f Skyline
package skyline;
import java.io.*;
import java.util.*;
import java.awt.Point;
public class Skyline {
public static Collection<Building> retrieveInput(InputStream is) {
ArrayList<Building> buildings = new ArrayList<Building>();
Scanner sc = new Scanner (is);
while (sc.hasNextLine()) {
String s = sc.nextLine();
if (s.equals("")) { break; }
try {
StringTokenizer st = new StringTokenizer(s);
int left = Integer.valueOf(st.nextToken());
int right = Integer.valueOf(st.nextToken());
int height = Integer.valueOf(st.nextToken());
Building b = new Building (left, right, height);
buildings.add(b);
} catch (NumberFormatException nfe) {
System.err.println(" ** Ignoring " + s + ": all values must be integers.
");
} catch (Exception e) {
System.err.println(" ** Ignoring " + s + ": " + e.getMessage());
}
}
return (buildings);
}
public static ArrayList<Edge>void compute(Collection<Building> buildings) {
TreeSet<Integer> S = new TreeSet<Integer>();
HashMap<Integer,ArrayList<Building>> lefts = new HashMap<Integer,ArrayList<B
uilding>>();
HashMap<Integer,ArrayList<Building>> rights = new HashMap<Integer,ArrayList<
Building>>();
ArrayList<Building> list = null;
for (Building b : buildings) {
S.add(b.left);
list = lefts.get(b.left);
if (list == null) {
list = new ArrayList<Building>();
lefts.put(b.left, list);
}
list.add(b);
S.add(b.right);
list = rights.get(b.right);
if (list == null) {
list = new ArrayList<Building>();
rights.put(b.right, list);
}
list.add(b);
}
int left = 0, top = 0;
ArrayList<Edge>skyline = new ArrayList<Edge>();
ArrayList<Building> heightList = new ArrayList<Building>();
for (int x : S) {
if (heightList.isEmpty()) {
top = 0;
} else {
top = heightList.get(0).height;
}
list = rights.get(x);
if (list != null) {
for (Building b : list) {
heightList.remove(b);
}
}
list = lefts.get(x);
if (list != null) {
for (Building b : list) {
int i;
for (i = 0; i < heightList.size(); i++) {
if (heightList.get(i).height < b.height) {
heightList.add(i, b);
break;
}
}
if (i == heightList.size()) {
heightList.add(b);
}
}
}
int newTop;
if (heightList.isEmpty()) {
newTop = 0;
} else {
newTop = heightList.get(0).height;
}
if (top == 0) {
left = x;
} else if (top != newTop) {
Edge e = new Edge(new Point (left, top), new Point (x, top));
skyline.add(e);
left = x;
}
System.out.println(x + ":" + heightList);
}
return (skyline);
}
public static void main(String[] args) {
Collection<Building> buildings = retrieveInput(System.in);
for (Edge e : compute(buildings)) {;
System.out.println(e);
}
}
}
This appro ach will reco rd all o f the ho rizo ntal lines (sho wn in red in previo us images) that fo rm the to ps o f
buildings. Execute this pro gram o n the o riginal set o f buildings to pro duce this set o f ho rizo ntal edges:
INTERACTIVE SESSION: Output o f ho rizo ntal edges in Skyline
1 4 3
6 7 1
8 15 4
8 11 5
9 12 3
2 5 4
13 16 5
[(1,3) - (2,3)]
[(2,4) - (5,4)]
[(6,1) - (7,1)]
[(8,5) - (11,5)]
[(11,4) - (13,4)]
[(13,5) - (16,5)]
Co mpare these edges with the image belo w that highlights the edges in the Skylinein red; these are the
ho rizo ntal edges in the final Skyline:
To co mplete this algo rithm, yo u add a helper metho d that co mpletes the Skyline which co ntains o nly
ho rizo ntal edges fo rming the to ps o f each building. Since the edges in the Skyline were added fro m left to
right, yo u must "stitch to gether" vertical edges to co nnect them, but yo u must also handle gaps that fo rm
when there is a space between two buildings. The pseudo co de belo w fo r co m ple t e () describes this
pro cess. In the pseudo co de, e dge .st art refers to the left po int o f a ho rizo ntal edge and e dge .e nd refers to
its right po int. Each po int has an x-co o rdinate and a y-co o rdinate, so e dge .e nd.x refers to the x-co o rdinate
o f the right po int o f the given edge:
OBSERVE: Pseudo co de fo r co mplete() metho d
complete(skyline)
skylinepoints = empty
left = leftmost coordinate of edges in skyline
right = rightmost coordinate of edges in skyline
append (left, 0) to skylinepoints
foreach edge in skyline do
append edge.start to skylinepoints
append edge.end to skylinepoints
nextEdge = next edge in skyline
if nextEdge exists then
if edge.end does not have same x-coordinate as nextEdge.start then
append (edge.end.x, 0) to skylinepoints
append (nextEdge.start.x, 0) to skylinepoints
append (right, 0) to skylinepoints
return skylinepoints
The fo llo wing implementatio n sho ws the final mo dificatio ns to co mplete the Skyline pro blem.
Skyline Final Implementatio n
package skyline;
import java.io.*;
import java.util.*;
import java.awt.Point;
public class Skyline {
public static Collection<Building> retrieveInput(InputStream is) {
ArrayList<Building> buildings = new ArrayList<Building>();
Scanner sc = new Scanner (is);
while (sc.hasNextLine()) {
String s = sc.nextLine();
if (s.equals("")) { break; }
try {
StringTokenizer st = new StringTokenizer(s);
int left = Integer.valueOf(st.nextToken());
int right = Integer.valueOf(st.nextToken());
int height = Integer.valueOf(st.nextToken());
Building b = new Building (left, right, height);
buildings.add(b);
} catch (NumberFormatException nfe) {
System.err.println(" ** Ignoring " + s + ": all values must be integers.
");
} catch (Exception e) {
System.err.println(" ** Ignoring " + s + ": " + e.getMessage());
}
}
return (buildings);
}
public static ArrayList<Edge> compute(Collection<Building> buildings) {
TreeSet<Integer> S = new TreeSet<Integer>();
HashMap<Integer,ArrayList<Building>> lefts = new HashMap<Integer,ArrayList<B
uilding>>();
HashMap<Integer,ArrayList<Building>> rights = new HashMap<Integer,ArrayList<
Building>>();
ArrayList<Building> list = null;
for (Building b : buildings) {
S.add(b.left);
list = lefts.get(b.left);
if (list == null) {
list = new ArrayList<Building>();
lefts.put(b.left, list);
}
list.add(b);
S.add(b.right);
list = rights.get(b.right);
if (list == null) {
list = new ArrayList<Building>();
rights.put(b.right, list);
}
list.add(b);
}
int left = 0, top = 0;
ArrayList<Edge>skyline = new ArrayList<Edge>();
ArrayList<Building> heightList = new ArrayList<Building>();
for (int x : S) {
if (heightList.isEmpty()) {
top = 0;
} else {
top = heightList.get(0).height;
}
list = rights.get(x);
if (list != null) {
for (Building b : list) {
heightList.remove(b);
}
}
list = lefts.get(x);
if (list != null) {
for (Building b : list) {
int i;
for (i = 0; i < heightList.size(); i++) {
if (heightList.get(i).height < b.height) {
heightList.add(i, b);
break;
}
}
if (i == heightList.size()) {
heightList.add(b);
}
}
}
int newTop;
if (heightList.isEmpty()) {
newTop = 0;
} else {
newTop = heightList.get(0).height;
}
if (top == 0) {
left = x;
} else if (top != newTop) {
Edge e = new Edge(new Point (left, top), new Point (x, top));
skyline.add(e);
left = x;
}
}
return (skyline);
}
public static Collection<Point> complete(ArrayList<Edge> skyline) {
ArrayList<Point> skylinepoints = new ArrayList<Point>();
if (skyline.isEmpty()) { return skylinepoints; }
int left = skyline.get(0).start.x;
int right = skyline.get(skyline.size()-1).end.x;
skylinepoints.add(new Point (left, 0));
for (int i = 0; i < skyline.size(); i++) {
Edge edge = skyline.get(i);
skylinepoints.add(edge.start);
skylinepoints.add(edge.end);
Edge nextEdge = null;
if (i+1 < skyline.size()) {
nextEdge = skyline.get(i+1);
if (edge.end.x != nextEdge.start.x) {
skylinepoints.add(new Point (edge.end.x, 0));
skylinepoints.add(new Point (nextEdge.start.x, 0));
}
}
}
skylinepoints.add(new Point (right, 0));
return (skylinepoints);
}
public static void main(String[] args) {
Collection<Building> buildings = retrieveInput(System.in);
ArrayList<Edge> skyline = compute(buildings);
Collection<Point> skylinepoints = complete(skyline);
for (Point p : skylinepoints) {
System.out.print("(" + p.x + "," + p.y + ") ");
}
for (Edge e : compute(buildings)) {
System.out.println(e);
}
}
}
Execute the final pro gram to validate that it wo rks o n the sample input set:
INTERACTIVE SESSION: Final o utput fro m Skyline pro gram
1 4 3
6 7 1
8 15 4
8 11 5
9 12 3
2 5 4
13 16 5
(1,0) (1,3) (2,3) (2,4) (5,4) (5,0) (6,0) (6,1) (7,1) (7,0) (8,0) (8,5) (11,5) (
11,4)
(13,4) (13,5) (16,5) (16,0)
Lessons Learned
This lesso n demo nstrates an iterative appro ach to algo rithm develo pment. The challenge is to identify
milesto nes alo ng the way where yo u can validate yo ur pro gress. Instead o f trying to so lve the who le pro blem
all at o nce, find ways to break the pro blem into sub-tasks. Our first attempt to so lve the Skyline pro blem
identified a number o f cases that we believed to be every po ssible way that the Skyline wo uld gro w when
buildings intersected with each o ther. In retro spect, this ad ho c so lutio n didn't capture all the ways that n
buildings co uld intersect each o ther. Yo u need to find meaningful milesto nes that represent the different
stages with the pro cessing phase o f an algo rithm. Each milesto ne has a well-defined validatio n co nditio n that
yo u co uld test using test cases. In Skyline, everything started to wo rk o nce the heightList abstractio n was
identified; that's allo wed us to identify the ho rizo ntal edges in the Skyline. Once that wo rk was co mpleted and
validated, the seco nd stage o f the algo rithm just stitched the edges to gether to fo rm the Skyline.
1. Whe n yo u canno t f ully o rde r t he e le m e nt s, t ry t o f ind a way t o swe e p f ro m le f t t o
right acro ss a part ially o rde re d se t : the sweeping technique described in this lesso n can be
used to fully pro cess each element in a set that canno t be co mpletely o rdered.
2. Eve n t ho ugh a se t is inhe re nt ly uno rde re d, t he T re e Se t allo ws yo u t o it e rat e o ve r
it s e le m e nt s in o rde r: while Se t implementatio ns canno t be so rted in the same way that List
implementatio ns can, yo u can use its efficient iterato r to inspect each o f the elements in so rted
o rder.
3. Use ArrayList whe n yo u ne e d t o m aint ain a list in so m e so rt e d o rde r and t he n inse rt
ne w e le m e nt s int o t he ir pro pe r lo cat io ns wit hin t he list : while LinkedList and ArrayList are
bo th Lists that can be so rted, yo u canno t insert the element into its pro per lo catio n efficiently in
LinkedList because its ge t (idx) metho d executes in O(n) time where n represents the number o f
elements in the list. Only ArrayList guarantees a co nstant time perfo rmance fo r this o peratio n.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Working With Big Data
Lesson Objectives
When yo u finish this lesso n yo u will be able to :
characterize the sto rage requirement fo r an algo rithm.
access the co ntents o f a structured binary file in the same way that yo u wo uld an array sto red in main memo ry.
read fro m and write to a memo ry-mapped file.
Working with Big Data
What if yo u had to so rt a co llectio n o f integers? The fo llo wing example sho ws ho w to use the built-in so rting
capabilities pro vided by the JDK.
Create a BigDat a pro ject, and assign it to the J ava6 _Le sso ns wo rking set.
Then, create a So rt Rando m Int e ge rs class in the default package o f the /src so urce fo lder:
CODE TO TYPE: So rtingExample
import java.util.Arrays;
public class SortRandomIntegers {
public static void main(String[] args) {
int numIntegers = 1000;
int[] group = new int[numIntegers];
for (int i = 0; i < numIntegers; i++) {
group[i] = (int)(Math.random()*numIntegers);
}
Arrays.sort(group);
for (int i = 0; i < 10; i++) {
System.out.println(group[i]);
}
}
}
Run the co de to verify that it prints o ut ten numbers in so rted o rder.
This small pro gram generates a rando m array co ntaining 10 0 0 integers, so rts them, and prints o ut the smallest ten in
the array. Yo u sho uld always use the Arrays.so rt built-in metho ds to so rt arrays because it pro vides tuned
algo rithms with a perfo rmance that is nearly always O(n log n). Fo r comparison-based so rting algo rithms (where yo u
can o nly so rt the elements by directly co mparing the individual elements) this is the best we've go t.
No w what if yo u have a large co llectio n o f integers? Like 450 millio n? If yo u mo dify the settings o f the abo ve pro gram
to generate 4 5 0 0 0 0 0 0 0 integers instead o f 10 0 0 integers, yo u'll see this erro r message:
OBSERVE: Unable to create large arrays in memo ry
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at SortRandomIntegers.main(SortRandomIntegers.java:8)
The primary issue with this co de is that it is simply impo ssible to create a co ntiguo us array to co ntain a large co llectio n
o f elements. Yo u can try to increase the heap space available to yo ur Java virtual machine, but eventually the co mputer
o n which yo u are running will exhaust its available memo ry. So ho w is it po ssible to deal with extremely large data
sets? Yo u will need to develo p techniques that manage the transfer o f data fro m external sto rage (such as a hard disk)
into main memo ry (what is co mmo nly called RAM). In the early days o f co mputing, main memo ry was measured in
kilo bytes (no t gigagbytes!) and pro grammers learned ho w to wo rk within these co nstraints. In this era o f "Big Data"
where data can be measured in terabytes and petabytes, even mo dern pro grammers have to make so me fundamental
adjustments.
In this lesso n, yo u'll learn ho w to so rt large sets sto red o n disk, sets that may be to o large to sto re in main memo ry.
We'll sho w examples using small data sets, but they can scale to much larger data sets as needed.
Sorting Large Sets Using External Storage
Mo st so rting algo rithms o perate o ver an array o f values, swapping elements in the array until the elements
are in o rder. The earlier so rt metho d do es that. Ho wever, when the number o f elements being so rted is to o
large to sto re in main memo ry, there are so rting algo rithms that allo w us to use external sto rage. The
fundamental algo rithm to learn is called MergeSort. Yo u've pro bably used this technique in the real wo rld
already. Suppo se that yo u had a stack o f 50 no tecards, each co ntaining a single number. To so rt the who le
stack, divide it into two stacks o f 25 no tecards each. So rt each o f these two stacks individually, which results
in two so rted stacks o f no tecards where yo u can see the to pmo st visible card in each stack. Yo u can "merge"
these two smaller stacks into a third so rted stack by repeatedly taking the card who se visible number is the
smaller o f the two . This merging pro cess gives the algo rithm its name.
MergeSo rt is recursive, since it breaks up a pro blem instance into two smaller instances o f half the size. To
sto p the recursio n, co nsider two cases:
1. So rting a co llectio n o f two values: swap the first value with the seco nd if they are o ut o f o rder.
2. So rting a co llectio n with a single value: the co llectio n is already so rted, so sto p.
Yo u have eno ugh info rmatio n to write the pseudo co de no w. The no tatio n |A| represents the size o f the
co llectio n A:
OBSERVE: pseudo co de fo r Mergeso rt
MergeSort (A)
if |A| < 2 then return A
if |A| = 2 then
swap elements of A if out of order
return A
sub1 = MergeSort(left half of A)
sub2 = MergeSort(right half of A)
merge sub1 and sub2 into a new array B
return B
Try this o ut by manually executing MergeSo rt o n the co llectio n [6 , 2, 1, 5, 3]. In the fo llo wing graphic, the blue
arro ws represent invo catio ns o f MergeSort and the red arro ws represent the returned so rted arrays. The
newly created arrays are depicted in red and co ntain three o r mo re elements (that is, [1,3,5] and [1,2,3,5,6 ]):
No w it's yo ur turn to implement this algo rithm:
Create a so rt package in the /src so urce fo lder.
Create a Co pyMe rge So rt class in the so rt package as sho wn.
CODE TO TYPE: Co pyMergeSo rt class
package sort;
import java.util.Arrays;
public class CopyMergeSort {
public static void main(String[] args) {
int[] group = new int []{6, 2, 1, 5, 3};
group = copymergesort(group);
for (int i : group) {
System.out.print (i + " ");
}
}
static int[] copymergesort(int[] A) {
if (A.length < 2) {
return A;
}
if (A.length == 2) {
if (A[0] > A[1]) {
int tmp = A[0];
A[0] = A[1];
A[1] = tmp;
}
return A;
}
int mid = A.length/2;
int[] left = Arrays.copyOfRange(A, 0, mid);
int[] right = Arrays.copyOfRange(A, mid, A.length);
left = copymergesort(left);
right = copymergesort(right);
for (int i
if (j >=
A[idx]
} else {
A[idx]
}
}
= 0, j = 0, idx=0; idx < A.length; idx++) {
right.length || (i < left.length && left[i] < right[j])) {
= left[i++];
= right[j++];
return A;
}
}
Run this co de to verify that it wo rks. Yo u might experiment with different initial arrays to see ho w the co de
handles arrays with just 1 o r 2 (o r larger number o f) elements:
OBSERVE: Output o f Co pyMergeSo rt
1 2 3 5 6
Let's take a clo ser lo o k at this co de. The base cases o f the recursio n are as fo llo ws:
OBSERVE: Base cases o f Co pyMergeSo rt recursio n
static int[] copymergesort(int[] A) {
if (A.length < 2) {
return A;
}
if (A.length == 2) {
if (A[0] > A[1]) {
int tmp = A[0];
A[0] = A[1];
A[1] = tmp;
}
return A;
}
...
}
The co pym e rge so rt metho d must return an int [] array, so these if statements bo th return the input array,
A. When there are two elements in the array, the seco nd if statement swaps t he t wo e le m e nt s if they are
o ut o f o rder.
OBSERVE: Recursive steps
int mid = A.length/2;
int[] left = Arrays.copyOfRange(A, 0, mid);
int[] right = Arrays.copyOfRange(A, mid, A.length);
left = copymergesort(left);
right = copymergesort(right);
The true lo gic o f this algo rithm o ccurs when the array A is subdivide d int o t wo arrays, le f t and right ,
which are then recursively so rted using co pym e rge so rt .
OBSERVE: Merging two so rted arrays
for (int i
if (j >=
A[idx]
} else {
A[idx]
}
}
= 0, j = 0, idx=0; idx < A.length; idx++) {
right.length || (i < left.length && left[i] < right[j])) {
= left[i++];
= right[j++];
return A;
}
Once the two recursive calls return, le f t and right will be so rted (this is the fundamental pro perty o f any
recursive functio n). All that remains is the pro cess o f selecting the smaller o f the two elements while merging
these two lists. The co de abo ve reuses array A to sto re the so rted values. Variable i will iterate o ver the
indices in le f t , while j will iterate o ver the indices in right . idx identifies the index lo catio n in A into which the
smaller value o f le f t [i] o r right [j] will be written. The lo o p terminates o nce all values have been transferred
into A (that is, when idx = A.length). Once right has exhausted its elements (because j >= right .le ngt h),
elements o f le f t are transferred to A. Similarly, o nce le f t has exhausted its elements (because i >=
le f t .le ngt h), elements o f right are transferred to A.
This co de wo rks, and it's reaso nably efficient o n small sets o f numbers, but we also need space fo r the le f t
and right arrays. To address this issue, yo u need to learn ho w to characterize the sto rage requirements fo r
an algo rithm.
Characterizing Storage Requirements for an Algorithm
Thro ugho ut this co urse, yo u have characterized the running time o f an algo rithm to determine its efficiency.
This is ho w algo rithms are mo st o ften co mpared. Yo u can also co mpare algo rithms by their sto rage
requirements. There is a "Time vs. Space" tradeo ff in pro gramming that explains many o f the design
decisio ns that a pro grammer must make. Fo r example, each Java class that is used as a key value in a
HashMap must implement a hashCo de () metho d as part o f the Co llectio ns Framewo rk. As we've
mentio ned, if two o bjects are equal to each o ther, then the value returned by hashCo de must also be the
same. Fo r immutable classes (such as St ring), a pro gram can save co mputatio n time by co mputing the
hash value just o nce and then caching the result fo r subsequent invo catio ns. Here is the co de fro m
java.lang.St ring:
OBSERVE: String.hashCo de() metho d
public int hashCode() {
int h = hash;
int len = count;
if (h == 0 && len > 0) {
int off = offset;
char val[] = value;
for (int i = 0; i < len; i++) {
h = 31*h + val[off++];
}
hash = h;
}
return h;
}
Whenever hashCo de is executed, it checks to see if the cached value hash is equal to zero ; o nly then do es it
co mpute and sto re the value in the hash class attribute. This co de is mo re efficient because o f the extra
integer being sto red. Ho w much extra sto rage is required? In this case it's a fixed amo unt o f sto rage—just
o ne additio nal int value. When addressing mo re co mplicated algo rithms, yo u will need to determine whether
the amo unt o f extra sto rage is fixed, o r is based o n the size o f the pro blem instance. Fo r example, if yo u
needed 2*n additio nal sto red array elements to so rt an existing array o f n elements, yo u wo uld characterize
the sto rage requirements as being O(n). If, ho wever, yo u needed n*n additio nal array elements to so rt an
existing array o f n elements, the required sto rage is O(n 2 ). We use the fo llo wing no tatio n in this co urse. T(n)
refers to the running time characterizatio n o f an algo rithm, S(n) refers to the storage requirements o f an
algo rithm. Mo dify the Co pyMe rge So rt co de as sho wn:
CODE TO TYPE: Mo dificatio ns to Co pyMergeSo rt to co mpute sto rage requirements
package sort;
import java.util.Arrays;
public class CopyMergeSort {
static int total=0;
public static void main(String[] args) {
int[] group = new int []{6, 2, 1, 5, 3};
int numIntegers = 512;
for (; numIntegers < 65536; numIntegers *= 2) {
int[]group = new int[numIntegers];
for (int i = 0; i < numIntegers; i++) {
group[i] = (int)(Math.random()*numIntegers);
}
total = 0;
group = copymergesort(group);
System.out.println(total + " locations for " + numIntegers);
}
for (int i : group) {
System.out.print (i + " ");
}
}
static int[] copymergesort(int[] A) {
if (A.length < 2) {
return A;
}
if (A.length == 2) {
if (A[0] > A[1]) {
int tmp = A[0];
A[0] = A[1];
A[1] = tmp;
}
return A;
}
int mid = A.length/2;
int[] left = Arrays.copyOfRange(A, 0, mid);
int[] right = Arrays.copyOfRange(A, mid, A.length);
left = copymergesort(left);
right = copymergesort(right);
for (int i
if (j >=
A[idx]
} else {
A[idx]
}
}
= 0, j = 0, idx=0; idx < A.length; idx++) {
right.length || (i < left.length && left[i] < right[j])) {
= left[i++];
= right[j++];
total += A.length;
return A;
}
}
Execute this revised co de to pro duce this table:
OBSERVE: Output sho wing sto rage requirements fo r Co pyMergeSo rt
4096 locations for 512
9216 locations for 1024
20480 locations for 2048
45056 locations for 4096
98304 locations for 8192
212992 locations for 16384
458752 locations for 32768
When so rting 512 elements yo u need 8 times as much tempo rary sto rage; wo rse, when so rting 2,0 48
elements yo u need 10 times as much tempo rary sto rage. Based o n the abo ve table, when so rting n elements
yo u need 2*n*log2(n) tempo rary sto rage where log2(n) is the lo garithm o f n in base 2. So , the sto rage
requirement fo r Co pyMergeSo rt is O(n log n). Even tho ugh Co pyMergeSo rt executes efficiently, there is a
serio us issue regarding its sto rage requirements. Can so mething be do ne to remedy this? Yes.
MergeSort with O(n) Storage Requirements
Mo st so rting algo rithms already perfo rm "in place" with no additio nal sto rage requirements, so yo u might
think that so me intermediate co mpro mise can be reached to reduce the sto rage requirements. Yo u do n't
need to instantiate two sub-arrays le f t and right if yo u inst e ad pass param e t e rs t hat re f e r t o
subrange s wit hin t he array it se lf . Let's start by revising the pseudo co de fo r MergeSort to create a metho d
that takes an array, A, and two internal indices, [st art , e nd) where index lo catio n st art is inclusive in the
range 0 .. A.le ngt h-1 while e nd is e xclusive in the range 0 .. A.le ngt h. So , to so rt an array o ne wo uld
invo ke Me rge So rt (A, 0 , A.le ngt h). No te that the so rting is do ne "in place" so an array is no lo nger returned
by this functio n.
OBSERVE: po tential revised pseudo co de fo r Mergeso rt
MergeSort (A, start,
if end - start < 2
if end - start = 2
swap elements of
end)
then return
then
A if out of order
mid = (end + start)/2;
MergeSort(A, start, mid);
MergeSort(A, mid, end);
merge A's left- and right- sorted sub-arrays
The tro uble with this appro ach is that merging in place will ultimately require just as many co mpariso ns (and
po ssibly mo re element swaps) as so rting in place. To avo id this situatio n, co nsider making these change to
the pseudo co de which intro duces a co py o f the initial array being so rted, which means the sto rage
requirement is O(n):
OBSERVE: final pseudo co de fo r Mergeso rt
MergeSort (A)
copy = copy of A
MergeSort (copy, A, 0, |A|)
MergeSort (A, result, start, end)
if end - start < 2 then return
if end - start = 2 then
swap elements of result if out of order
mid = (end + start)/2;
MergeSort(result, A, start, mid);
MergeSort(result, A, mid, end);
merge A's left- and right- sorted sub-arrays
merge left- and right- of A into result
Because co py is a true co py o f the entire array, the terminating base cases o f the recursio n will wo rk because
they reference the original elements of the array directly at their respective index locations. This o bservatio n is
a so phisticated o ne; when yo u run this implementatio n in the debugger, yo u can validate it fo r yo urself. In
additio n, the final merge step requires o nly O(n) o peratio ns.
No w it's yo ur turn to implement this pseudo co de.
In the so rt package o f the /src so urce fo lder, create a Me rge So rt Int e ge r class as sho wn.
COE TO TYPE: MergeSo rtInteger class
package sort;
import java.util.Arrays;
public class MergeSortInteger {
public static void main(String[] args) {
int numIntegers = 1024;
int[] group = new int[numIntegers];
for (int i = 0; i < numIntegers; i++) {
group[i] = (int)(Math.random()*numIntegers);
}
mergesort(group);
for (int i = 0; i < 10; i++) {
System.out.println(group[i]);
}
}
static void mergesort (int[] A) {
int[] copy = Arrays.copyOf(A, A.length);
mergesort (copy, A, 0, A.length);
}
static void mergesort(int[] A, int[] result, int start, int end) {
if (end - start < 2) {
return;
}
if (end - start == 2) {
if (result[end-2] > result[end-1]) {
int tmp = result[end-2];
result[end-2] = result[end-1];
result[end-1] = tmp;
}
return;
}
int mid = (end + start)/2;
mergesort(result, A, start, mid);
mergesort(result, A, mid, end);
for (int i = start, j = mid, idx=start; idx < end; idx++) {
if (j >= end || (i < mid && A[i] < A[j])) {
result[idx] = A[i++];
} else {
result[idx] = A[j++];
}
}
}
}
Run this co de; yo u see the first ten rando mly generated integers in so rted o rder. Let's review this co de
mo re clo sely:
OBSERVE: MergeSo rt invo catio n
static void mergesort (int[] A) {
int[] copy = Arrays.copyOf(A, A.length);
mergesort (copy, A, 0, A.length);
}
To so rt the array, we make a full co py and then internally invo ke m e rge so rt to so rt the co py with A as the
ultimate destinatio n. No te that the arguments to pass in are 0 and A.le ngt h, which reflect the index values
into A, namely inclusive o n the left side with 0 and exclusive o n the right side with A.le ngt h.
All lo gic o nce again resides in the recursive metho d. Let's review the base cases:
OBSERVE: Recursive base case o f MergeSo rt
static void mergesort(int[] A, int[] result, int start, int end) {
if (end - start < 2) {
return;
}
if (end - start == 2) {
if (result[end-2] > result[end-1]) {
int tmp = result[end-2];
result[end-2] = result[end-1];
result[end-1] = tmp;
}
return;
}
If e nd - st art is le ss t han 2, there is either no element o r a single element to be so rted, which means
no thing needs to be do ne. When e nd-st art e quals 2, there are two elements to be so rted. This co de
executes o nly as a base case in the recursio n, which means that it's the first time the metho d is inspecting the
array subrange o f [st art ,e nd). Because the result must be sto red in the re sult array, this co de reo rders the
values it finds there.
The final elements in m e rge so rt sho w ho w to merge the so rted left and right sub-arrays:
OBSERVE: Merging in O(n) time
int mid = (end + start)/2;
mergesort(result, A, start, mid);
mergesort(result, A, mid, end);
for (int i = start, j = mid, idx=start; idx < end; idx++) {
if (j >= end || (i < mid && A[i] < A[j])) {
result[idx] = A[i++];
} else {
result[idx] = A[j++];
}
}
This co de first re cursive ly so rt s t he le f t half and right half o f t he range [st art , e nd), placing the
pro perly o rdered elements in the array referenced as A. Then it uses two indices, i and j, to iterate o ver each
o f these sub-ranges, always co pying the smaller o f A[i] and A[j] into the pro perly lo cated re sult [idx]. There
are three cases to co nsider:
1. The right side is exhausted (j >= e nd), in which case yo u can grab the remaining elements fro m
A[i].
2. The left side is exhausted (i >= m id), in which case yo u can grab the remaining elements fro m
A[j].
3. The left and right side have elements; if A[i] < A[j], insert A[i], o therwise insert A[j].
Once the fo r lo o p co mpletes, re sult has the merged (and so rted) elements fro m the subarray [st art , e nd)
o f the o riginal array A.
Working with Large Datasets
Yo u are go ing to use MergeSort to so rt large co llectio ns o f values. Yo u're go ing to need additio nal sto rage
fo r that to wo rk. Yo u do n't actually need to sto re the entire co llectio n in main memo ry to so rt its co ntents.
Let's start by defining the pro blem instance. The input o f n integers will be sto red in a binary file co ntaining 4*n
bytes. Practice using this structure by writing this sample pro gram:
In the so rt package o f the /src so urce fo lder, create a class BinaryInt e ge rFile as sho wn:
CODE TO TYPE: BinaryIntegerFile
package sort;
import java.io.*;
public class BinaryIntegerFile {
public static void main(String[] args) throws IOException {
int numIntegers = 4096;
File f = new File ("IntegerFile.bin");
DataOutputStream dos = new DataOutputStream(new FileOutputStream(f));
for (int i = 0; i < numIntegers; i++) {
dos.writeInt((int)(Math.random()*numIntegers));
}
dos.close();
DataInputStream dis = new DataInputStream(new FileInputStream(f));
System.out.println("First five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.skipBytes(4*(numIntegers-10));
System.out.println("Last five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.close();
}
}
Run this pro gram; yo u'll see so mething like this (yo ur numbers will be different because they are
rando mly generated):
INTERACTIVE SESSION: Output fro m BinaryIntegerFile. No te that so rt functio n is no t yet implemented.
First five sorted numbers
2935
3918
245
2885
2496
Last five sorted numbers
2748
3716
1972
2086
1350
Let's take a clo ser lo o k at this co de. The java.io package co ntains a number o f classes to read and write
info rmatio n to the file system. The fundamental abstractio n is a stream, which represents a sequence o f data.
An Input St re am reads data fro m a so urce and an Out put St re am writes data to a so urce. In the co de, a
Dat aInput St re am is use d t o re ad prim it ive J ava dat a t ype s f ro m t he input st re am (such as int
and f lo at value s) while a Dat aOut put St re am writ e s prim it ive J ava dat a t ype s t o an o ut put
st re am .
OBSERVE: Creating a rando m binary file o f integers
int numIntegers = 4096;
File f = new File ("IntegerFile.bin");
DataOutputStream dos = new DataOutputStream(new FileOutputStream(f));
for (int i = 0; i < numIntegers; i++) {
dos.writeInt((int)(Math.random()*numIntegers));
}
dos.close();
Using a Dat aOut put St re am o bject, the abo ve co de writes 4,0 9 6 integers in binary fo rmat to the file and
clo ses it. Once created, this file will co ntain 16 ,38 4 bytes because the integers are written in binary fo rmat
where each integer value requires fo ur bytes. Do n't bo ther trying to o pen this file in Eclipse because the data
is sto red in binary fo rmat so Eclipse will just present yo u with the raw data. Yo u can retrieve the integer values
that were sto red using Dat aInput St re am , which pro perly deco des the binary fo rmatted enco ding o f the
integer values in the file. The seco nd part o f the co de reads in this file and prints the first five integers and the
last five integers in the file.
OBSERVE: Read integers fro m file
DataInputStream dis = new DataInputStream(new FileInputStream(f));
System.out.println("First five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.skipBytes(4*(numIntegers-10));
System.out.println("Last five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.close();
This co de uses a Dat aInput St re am to retrieve the values fro m the file. No te that it reads the first five
integers and then skips t he re quisit e num be r o f byt e s (there are fo ur bytes fo r each integer) so it can
then read the last five numbers fro m the file. Clearly this file isn't so rted; yo u'll so lve this by implementing a
MergeSort that o perates o ver a File co ntaining integer values, rather than an in-memo ry array o f integer
values.
To make this wo rk, yo u have to access a file in the same way that yo u wo uld o therwise access an array. Yo u
kno w the structure o f MergeSort fro m the implementatio n yo u co mpleted earlier, all yo u need to do no w is
map tho se co ncepts to a file. Co nsider using Rando m Acce ssFile , pro vided by the java.io package, which
allo ws yo u to access any byte within a file rando mly. Kno wing that the file co ntains a co llectio n o f integers in
4-byte fo rmat, yo u can determine that to read the nth int value fro m the file, yo u need to start reading 4 bytes
fro m po sitio n n*4. Similar lo gic is used to write an integer to replace the nth int value in the file. All o f these
o peratio ns will succeed with index values o f type lo ng, which means yo u can pro cess extremely large files if
yo u want.
In the so rt package o f the /src so urce fo lder, create a Me rge So rt File class as sho wn:
CODE TO TYPE: MergeSo rtFile class
package sort;
import java.io.*;
public class MergeSortFile {
static void mergesort (File A) throws IOException {
File copy = new File (A.getPath() + ".tmp");
copyFile(A, copy);
// TBA: invoke MergeSort
}
static void copyFile(File src, File dest) throws IOException {
FileInputStream fis = new FileInputStream(src);
FileOutputStream fos = new FileOutputStream (dest);
byte[] bytes = new byte[4*1048576];
int numRead;
while ((numRead = fis.read(bytes)) > 0) {
fos.write(bytes, 0, numRead);
}
fis.close();
fos.close();
}
}
The m e rge so rt metho d prepares fo r the algo rithm by making a full co py o f the so urce file, A. Fo r
demo nstratio n purpo ses, the file co py is created in yo ur wo rkspace, but no rmally yo u wo uld use the static
metho d File .cre at e T e m pFile instead to create a tempo rary file in the default tempo rary directo ry. The
co pyFile metho d co pies bytes in chunks o f fo ur megabytes to replicate the file. To test o ut the abo ve co de,
mo dify BinaryInt e ge rFile to use the m e rge so rt metho d in Me rge So rt File .
CODE TO TYPE: Mo dified BinaryIntegerFile
package sort;
import java.io.*;
public class BinaryIntegerFile {
public static void main(String[] args) throws IOException {
int numIntegers = 4096;
File f = new File ("IntegerFile.bin");
DataOutputStream dos = new DataOutputStream(new FileOutputStream(f));
for (int i = 0; i < numIntegers; i++) {
dos.writeInt((int)(Math.random()*numIntegers));
}
dos.close();
long now = System.currentTimeMillis();
MergeSortFile.mergesort(f);
System.out.println((System.currentTimeMillis() - now) + " ms.");
DataInputStream dis = new DataInputStream(new FileInputStream(f));
System.out.println("First five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.skipBytes(4*(numIntegers-10));
System.out.println("Last five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.close();
}
}
No w execute BinaryInt e ge rFile and refresh yo ur wo rkspace. Yo u will see two to p-level files:
Int e ge rFile .bin and Int e ge rFile .bin.t m p. Select bo th o f these files in the Java package bro wser (ho lding
do wn the Shif t key and click each ico n), and right-click o n either file to select Co m pare Wit h | Each Ot he r.
The files are identical, because yo u haven't yet written any co de to so rt the data.
Yo u are no w ready to co mplete the MergeSort implementatio n. Mo dify Me rge So rt File as fo llo ws:
Mo dified MergeSo rtFile
package sort;
import java.io.*;
public class MergeSortFile {
static void mergesort (File A) throws IOException {
File copy = new File (A.getPath() + ".tmp");
copyFile(A, copy);
// TBA: invoke MergeSort
RandomAccessFile src = new RandomAccessFile(A, "rw");
RandomAccessFile dest = new RandomAccessFile(copy, "rw");
mergesort (dest, src, 0,
src.close();
dest.close();
copy.delete();
A.length());
}
static void copyFile(File src, File dest) throws IOException {
FileInputStream fis = new FileInputStream(src);
FileOutputStream fos = new FileOutputStream (dest);
byte[] bytes = new byte[4*1048576];
int numRead;
while ((numRead = fis.read(bytes)) > 0) {
fos.write(bytes, 0, numRead);
}
fis.close();
fos.close();
}
static void mergesort(RandomAccessFile A, RandomAccessFile result,
long start, long end) throws IOException {
if (end - start < 8) {
return;
}
if (end - start == 8) {
result.seek(end-8);
int left = result.readInt();
int right = result.readInt();
if (left > right) {
result.seek(end-8);
result.writeInt(right);
result.writeInt(left);
}
return;
}
long mid = (end + start)/8*4;
mergesort(result, A, start, mid);
mergesort(result, A, mid, end);
result.seek(start);
for (long i = start, j = mid, idx=start; idx < end; idx += 4) {
A.seek(i);
int Ai = A.readInt();
int Aj = 0;
if (j < end) { A.seek(j); Aj = A.readInt(); }
if (j >= end || (i < mid && Ai < Aj)) {
result.writeInt(Ai);
i += 4;
} else {
result.writeInt(Aj);
j += 4;
}
}
}
}
The mo dified m e rge so rt metho d no w o pens two Rando m Acce ssFile o bjects o n the two files and they are
bo th o pened in read/write mo de because they will bo th be updated during the MergeSort algo rithm. The
m e rge so rt metho d is invo ked by requesting to so rt the co ntents o f the co pied file into the o riginal file. Once
do ne, the co pied file can be deleted.
The mergeso rt(Rando mAccessFile A, Rando mAccessFile result, lo ng start, lo ng end) metho d perfo rms the
recursive MergeSort o f the given range [st art , e nd) o f the underlying files. These parameters are bo th o f type
lo ng to enable this metho d to so rt files that can be several gigabytes in size. Fo r this lesso n, the files will
o nly be several megabytes in size; feel free to generate files o f this size o n yo ur ho me co mputer!
The structure o f this metho d fo llo ws the earlier examples.
Go back and execute BinaryInt e ge rFile and the o utput will appear pro perly (tho ugh yo ur rando m numbers
will be different):
Output fro m BinaryIntegerFile
First five sorted numbers
1
3
4
6
6
Last five sorted numbers
4091
4091
4093
4093
4095
Refresh the files in yo ur wo rkspace; the tempo rary file used during the so rt has been deleted. Let's take a
clo ser lo o k at this co de. First, let's inspect the base cases o f the recursio n:
OBSERVE: mergeso rt base cases fo r recursio n
static void mergesort(RandomAccessFile A, RandomAccessFile result,
long start, long end) throws IOException {
if (end - start < 8) {
return;
}
if (end - start == 8) {
result.seek(end-8);
int left = result.readInt();
int right = result.readInt();
if (left > right) {
result.seek(end-8);
result.writeInt(right);
result.writeInt(left);
}
return;
}
Recall that integers are sto red using 4 bytes. The o ffsets st art and e nd are index lo catio ns within the
Rando m Acce ssFile ; in additio n, st art and e nd must be evenly divisible by 4. The co nditio n e nd - st art <
8 determines when the subrange [st art , e nd) co ntains zero o r o ne element; when this o ccurs, no so rting
needs to take place and the metho d can simply return.
The seco nd base case needs to swap the two neighbo ring integers in re sult if they are o ut o f o rder. When
e nd - st art == 8 yo u kno w that [st art , e nd) co ntains two elements exactly. The abo ve co de uses the se e k
metho d to find that lo catio n in the file fo r the first o f these two integers. It then reads in two integers
sequentially fro m the 8 bytes sto red at that po sitio n in that file. If these two numbers are o ut o f o rder, it go es
back to the beginning o f that range in the file and writes the two integers in the pro per o rder.
The final case demo nstrates ho w to merge the two sub-ranges to gether. It starts with a little mathematical
o ptimizatio n. Mergeso rt must divide the range into two parts, but each sub-range must co ntain a number o f
bytes that is divisible by fo ur. Fo r example, if the range co ntained 7 integers fo r a to tal o f 28 bytes, it might be
represented as [0 ,28). Simply dividing (0 +28)/2 wo uld give 14, which is no t divisible by 4. Instead, divide
(0 +28)/8 (to get 3 using integer divisio n) and then multiply by 4 to get 12, which is ro ughly half o f the range.
OBSERVE: Merging case in MergeSo rt
long mid = (end + start)/8*4;
mergesort(result, A, start, mid);
mergesort(result, A, mid, end);
result.seek(start);
for (long i = start, j = mid, idx=start; idx < end; idx += 4) {
A.seek(i);
int Ai = A.readInt();
int Aj = 0;
if (j < end) { A.seek(j); Aj = A.readInt(); }
if (j >= end || (i < mid && Ai < Aj)) {
result.writeInt(Ai);
i += 4;
} else {
result.writeInt(Aj);
j += 4;
}
}
The abo ve co de recursively invo ke s m e rge so rt o n t he le f t and right sub-range s, after which the file o n
disk referenced by A will co ntain the two so rted sub-ranges waiting to be merged. The co de takes advantage
o f a nice o ptimizatio n in that the integers written to re sult will be written sequentially, so it o nly needs to se e k
t o t he st art ing lo cat io n o f that o utput sequence in re sult befo re starting the f o r lo o p. When the co de
determines the Ai and Aj values, it must seek the pro per file po sitio n within A and then read the integer
enco ded there.
No te that, in this f o r lo o p, the index values i, j, and idx are all incremented by 4 because they reference
po sitio ns inside the file that co ntains the 4-byte integer enco dings.
The co nditio n (j >= e nd || (i < m id && Ai < Aj)) takes advantage o f "sho rt-circuit" lo gical evaluatio n. That is,
if j >= e nd, the seco nd part o f the co nditio n (after the "||") is no t executed. Ho wever, if j < e nd, yo u can
retrieve Aj fro m the file. Fo r this reaso n, the co de first lo ads up Aj if it exists to prepare fo r the sho rt-circuit
co nditio nal.
Never Be Satisfied
Despite the success o f the co de, it still feels like it takes to o lo ng to co mplete. The pro blem is likely that as the
files get larger, the number o f disk accesses begins to do minate the perfo rmance o f the algo rithm. Fo rtunately
there is a "dro p-in replacement" fo r file access based o n an o perating-system capability kno wn as "Memo ry
Mapped Files." The implementatio n is fo und in the java.nio package, kno wn as the "new input/o utput" Java
package that co ntains many high-perfo rmance classes.
In the so rt package o f the /src so urce fo lder, create a class named Me rge So rt File Mappe d. Much o f this
co de will be familiar to yo u because it fo llo ws the exact implementatio n style o f the earlier MergeSo rt.
CODE TO TYPE: MergeSo rtFileMapped class
package sort;
import java.io.*;
import java.nio.*;
import java.nio.channels.*;
public class MergeSortFileMapped {
static void mergesort (File A) throws IOException {
File copy = File.createTempFile("Mergesort", ".bin");
MergeSortFile.copyFile(A, copy);
RandomAccessFile src = new RandomAccessFile(A, "rw");
RandomAccessFile dest = new RandomAccessFile(copy, "rw");
FileChannel srcC = src.getChannel();
FileChannel destC = dest.getChannel();
MappedByteBuffer srcMap = srcC.map(FileChannel.MapMode.READ_WRITE, 0, src.le
ngth());
MappedByteBuffer destMap = destC.map(FileChannel.MapMode.READ_WRITE, 0, dest
.length());
mergesort (destMap, srcMap, 0, (int) A.length());
src.close();
dest.close();
}
static void mergesort(MappedByteBuffer A, MappedByteBuffer result,
int start, int end) throws IOException {
if (end - start < 8) {
return;
}
if (end - start == 8) {
result.position(start);
int left = result.getInt();
int right = result.getInt();
if (left > right) {
result.position(start);
result.putInt(right);
result.putInt(left);
}
return;
}
int mid = (end + start)/8*4;
mergesort(result, A, start, mid);
mergesort(result, A, mid, end);
result.position(start);
for (int i = start, j = mid, idx=start; idx < end; idx += 4) {
int Ai = A.getInt(i);
int Aj = 0;
if (j < end) { Aj = A.getInt(j); }
if (j >= end || (i < mid && Ai < Aj)) {
result.putInt(Ai);
i += 4;
} else {
result.putInt(Aj);
j += 4;
}
}
}
}
To execute this co de instead o f the earlier versio n, mo dify BinaryInt e ge rFile as sho wn:
CODE TO TYPE: Mo dificatio ns to BinaryIntegerFile
package sort;
import java.io.*;
public class BinaryIntegerFile {
public static void main(String[] args) throws IOException {
int numIntegers = 655364096;
File f = new File ("IntegerFile.bin");
DataOutputStream dos = new DataOutputStream(new FileOutputStream(f));
for (int i = 0; i < numIntegers; i++) {
dos.writeInt((int)(Math.random()*numIntegers));
}
dos.close();
long now = System.currentTimeMillis();
MergeSortFile.mergesort(f);
MergeSortFileMapped.mergesort(f);
System.out.println((System.currentTimeMillis() - now) + " ms.");
DataInputStream dis = new DataInputStream(new FileInputStream(f));
System.out.println("First five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.skipBytes(4*(numIntegers-10));
System.out.println("Last five sorted numbers");
for (int i = 0; i < 5; i++) {
System.out.println(dis.readInt());
}
dis.close();
}
}
Run it to so rt just o ver 6 5,0 0 0 integer values. The co de spends mo st o f its time writing the rando m
numbers to the disk file to prepare fo r the algo rithm. The executio n no w takes far less time (32 milliseco nds
instead o f 10 276 milliseco nds). This co de executes so much faster because when yo u're wo rking with data
o n disk, yo u need to limit the frequency o f disk access to maximize the efficiency o f yo ur co de. Inside the
Java Virtual Machine, the java.nio package is integrated with the virtual memo ry manager o f the o perating
system. Memo ry-mapped files are lo aded into memo ry o ne entire page at a time, and each o perating system
is fine-tuned so these o peratio ns execute as efficiently as po ssible. When yo u mo dify info rmatio n in a
memo ry-mapped file, it will be written o ut to the file o ne page at a time; the o perating system is respo nsible
fo r carrying this o peratio n o ut efficiently as well. No w yo ur pro gram is no lo nger in charge o f reading and
writing bytes fro m a file directly; it updates memo ry directly, as managed by the Mappe dByt e Buf f e r class.
Ultimately this class determines when the updated memo ry is written to the file.
Let's review this co de:
OBSERVE: Using MappedByteBuffers to access file data
static void mergesort (File A) throws IOException {
File copy = File.createTempFile("Mergesort", ".bin");
MergeSortFile.copyFile(A, copy);
RandomAccessFile src = new RandomAccessFile(A, "rw");
RandomAccessFile dest = new RandomAccessFile(copy, "rw");
FileChannel srcC = src.getChannel();
FileChannel destC = dest.getChannel();
MappedByteBuffer srcMap = srcC.map(FileChannel.MapMode.READ_WRITE, 0, src.le
ngth());
MappedByteBuffer destMap = destC.map(FileChannel.MapMode.READ_WRITE, 0, dest
.length());
mergesort (destMap, srcMap, 0,
src.close();
dest.close();
(int) A.length());
}
One unfo rtunate drawback with using MappedByteBuffer is that o n many o perating systems (and o n Windo ws
in particular) o nce a file has been mapped, it canno t be deleted fro m within the Java pro gram. The abo ve
co de, therefo re, cre at e s a t e m po rary f ile in t he de signat e d de f ault t e m po rary dire ct o ry which will
eventually be cleaned up by the user. Fro m a Rando m Acce ssFile , it is po ssible t o re t rie ve it s
File Channe l de script o r, which is used to co nst ruct t he re spe ct ive Mappe dByt e Buf f e r o bje ct s.
Changes to the m e rge so rt metho d are mo re subtle:
OBSERVE: mergeso rt revised to use MappedByteBuffer
static void mergesort(MappedByteBuffer A, MappedByteBuffer result,
int start, int end) throws IOException {
if (end - start < 8) {
return;
}
if (end - start == 8) {
result.position(start);
int left = result.getInt();
int right = result.getInt();
if (left > right) {
result.position(start);
result.putInt(right);
result.putInt(left);
}
return;
}
int mid = (end + start)/8*4;
mergesort(result, A, start, mid);
mergesort(result, A, mid, end);
result.position(start);
for (int i = start, j = mid, idx=start; idx < end; idx += 4) {
int Ai = A.getInt(i);
int Aj = 0;
if (j < end) { Aj = A.getInt(j); }
if (j >= end || (i < mid && Ai < Aj)) {
result.putInt(Ai);
i += 4;
} else {
result.putInt(Aj);
j += 4;
}
}
}
st art and e nd are int values again. The MappedByteBuffer class o nly suppo rts integer indexing, which
means the files to be so rted canno t be greater than 2 32 bytes in size (ro ughly 4 gigabytes).
Let's review the base cases o f the recursio n:
OBSERVE: MergeSo rtFileMapped base recursive cases
if (end - start < 8) {
return;
}
if (end - start == 8) {
result.position(start);
int left = result.getInt();
int right = result.getInt();
if (left > right) {
result.position(start);
result.putInt(right);
result.putInt(left);
}
return;
}
When asked to so rt two elements, the co de uses the ge t Int metho d o f the MappedByteBuffer class to
retrieve the integer sto red at the pro per o ffset o f st art . If the MappedByteBuffer do es no t have this info rmatio n
in main memo ry, it will read the info rmatio n into memo ry o ne page at a time. If this memo ry is updated (using
the put Int metho ds) it wo n't be written to disk until the Mappe dByt e Buf f e r determines that it can be written
eficiently. As a pro grammer, yo u no lo nger kno w whether a ge t Int o r e nd metho d accesses the file system;
yo u can simply pro gram it co rrectly while leaving MappedByteBuffer respo nsible fo r the persistent sto rage o f
the info rmatio n.
OBSERVE: Co mpleting the mergeso rt
int mid = (end + start)/8*4;
mergesort(result, A, start, mid);
mergesort(result, A, mid, end);
result.position(start);
for (int i = start, j = mid, idx=start; idx < end; idx += 4) {
int Ai = A.getInt(i);
int Aj = 0;
if (j < end) { Aj = A.getInt(j); }
if (j >= end || (i < mid && Ai < Aj)) {
result.putInt(Ai);
i += 4;
} else {
result.putInt(Aj);
j += 4;
}
}
Despite the large number o f read and write statements that access the re sult and A files, the
MappedByteBuffer class ensures that these o peratio ns act o n info rmatio n sto red in main memo ry. Yo u can't
predict when (Java) will write the info to a file, so the MappedByteBuffer class ensures that data is read fro m
memo ry (instead o f the file). It'll be mo re efficient because it's faster to read fro m memo ry than fro m a file (in
this case, almo st 30 0 times faster).
Lessons Learned
Often yo u can impro ve the efficiency o f an algo rithm by sto ring additio nal state info rmatio n. Yo u see this o n a
small scale in java.lang.St ring, which caches its co mputed hash value to impro ve the perfo rmance o f
hashCo de . Often yo u can achieve efficient O(n log n) perfo rmance by sto ring additio nal O(n) sto rage
info rmatio n.
To determine the appro priate algo rithm to use, be sure to characterize the sto rage requirements in additio n to
the run-time perfo rmance. In mo st cases, the additio nal sto rage will be O(n), which typically is an acceptable
trade-o ff to make.
Accessing info rmatio n o n disk is typically thousands of times slower than accessing information in main
memory. When designing algo rithms that access data o n disk, yo u must find ways to reduce the number o f
individual reads and writes, cho o sing instead to let the o perating system o ptimize input/o utput access.
Practice so me o f the things yo u learned in this lesso n in the pro ject. See yo u in the next lesso n!
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Representing Graph Data Structures
Lesson Objectives
After co mpleting this lesso n, yo u will be able to :
co mpute the adjacency matrix fo r any graph.
explain the co ncept o f backtracking.
use a backtracking depth-first search algo rithm to traverse a graph.
Representing Graphs
Yo u've already seen ho w to use Se t and List classes fro m the Java Co llectio ns Framewo rk to represent uno rdered
sets o r lists o rdered linearly. Ho wever, so metimes yo u need to represent mo re than a co llectio n o f items; yo u also
need to capture relationships between the items themselves. One co mmo n way to acco mplish this is to define a Graph
co nstruct co mpo sed o f a set o f vertices, V, representing a set o f items, and a set o f edges, E, co nnecting pairs o f these
vertices, representing the relatio nships. In this lesso n, we'll fo cus o n learning ho w to represent graphs. Alo ng the way,
yo u'll also learn ho w to explo re a graph by traversing its edges fro m o ne vertex to ano ther.
Let's start with an example. Assume yo u live in a city with a subway system with three subway lines and ten statio ns as
sho wn:
Yo u wo uld like to determine a ro ute fro m a given starting statio n to a destinatio n statio n. Fo r example, to go fro m
statio n 4 to 10 yo u can likely see two distinct paths to take: (4, 5, 3, 10 ) and (4, 2, 8 , 9 , 10 ). If the subway system has
do zens o f statio ns, the pro blem increases in co mplexity and yo u might no t always be able to "see a path" at a glance.
To write a pro gram that so lves this pro blem, yo u need to develo p an algo rithm to traverse the subway fro m a starting
statio n to an ending statio n. The algo rithm needs a data structure that represents the subway system so it can execute
efficiently. No ne o f the classes pro vided by the JDK can be used "as is" to so lve this pro blem, so yo u have to do this
yo urself.
Using Adjacency Matrix T o Represent Graph
Since yo u o nly need to kno w whether two statio ns are co nnected with each o ther, yo u co uld create a two dimensio nal array bo o le an m at rix[][] in which a given element m at rix[i][j] is t rue when there is a direct
co nnectio n between statio ns i and j. This array is symmetric so m at rix[i][j] equals m at rix[j][i].
Note
This representatio n do esn't include part o f the structure o f a subway system, namely the multiple
subway lines that may traverse the same link between two statio ns. Ho wever, fo r the purpo ses
o f this lesso n, it's o kay.
Create a new Java Pro ject named Graphs and assign it to the J ava6 _Le sso ns wo rking set.
In the /src fo lder o f the Graphs pro ject, create a subway package.
In the subway package o f the /src so urce fo lder, create a SubwayMat rix class. This class will represent a
graph using an adjacency matrix.
CODE TO TYPE: SubwayMatrix class
package subway;
public class SubwayMatrix {
final int n;
final boolean matrix[][];
public SubwayMatrix(int numStations) {
n = numStations;
matrix = new boolean[n+1][n+1];
}
}
The SubwayMat rix co nstructo r requires the to tal number o f vertices so it can co nstruct the m at rix[][] two dimensio nal array. Because statio ns (hence vertices) are numbered fro m 1 .. num Ve rt ice s, this array is o ne
number larger than it needs to be; this makes the co de easier to read and write.
The example subway system co uld be represented by auto -initializing the matrix in SubwayMat rix as
sho wn:
OBSERVE: po tential co mpiled initializatio n o f matrix fo r subway pro blem
boolean matrix[][] = new
/*
1
10 */
{false, false,
e, false},
/* 1 */ {false, false,
e, false},
/* 2 */ {false, false,
e, false},
/* 3 */ {false, false,
e, true},
/* 4 */ {false, true,
e, false},
/* 5 */ {false, false,
e, false},
/* 6 */ {false, false,
e, false},
/* 7 */ {false, false,
e, false},
/* 8 */ {false, false,
, false},
/* 9 */ {false, false,
e, true},
/* 10 */ {false, false,
, false}};
}
boolean[][] {
2
3
4
5
6
7
8
9
false, false, false, false, false, false, false, fals
false, false, true,
false, false, false, false, fals
false, false, true,
false, false, false, true,
fals
false, false, false, true,
false, false, false, fals
true,
false, false, false, fals
false, false, true,
false, true,
true,
false, false, false, false, fals
false, false, false, false, false, true,
false, false, false, false, true,
true,
false, fals
false, true,
false, false, false, false, true,
false, true
false, false, false, false, false, false, true,
false, true,
fals
fals
false, false, false, false, false, true
Instead o f defining the subway system in this way, add the metho d belo w to the end o f SubwayMat rix, which
will allo w yo u to update matrix dynamically, given an array o f int values in sequence:
CODE TO STYLE: Metho d to dynamically add statio ns in a line
public void addLine(int[] stations) {
for (int i = 1; i < stations.length; i++) {
matrix[stations[i-1]][stations[i]] = true;
matrix[stations[i]][stations[i-1]] = true;
}
}
We prefer this appro ach because yo u can co nstruct subway lines dynamically witho ut co mpilatio n.
Searching a Graph
To co mpute a path in a graph fro m any vertex X to ano ther vertex Y, make these assumptio ns:
The graph is co nnected; that is, it is po ssible to travel fro m any vertex to any o ther vertex by
fo llo wing the edges in the graph.
The path must no t visit the same vertex twice.
To find a ro ute fro m statio n 4 to 10 , fo r example, imagine that yo u have a co py o f the map that yo u can mark
up with a pencil. Start by shading statio n 4 in gray, and then co nsider traveling next to o ne o f its unvisited
neighbo rs, such as statio n 1. Draw an arro w co nnecting statio ns 4 and 1. Once yo u see that statio n 1 has no
neighbo ring statio n that yo u haven't visited, co lo r statio n 1 in black to indicate that there is no need to
co nsider that statio n again. No w, like enco untering a dead end in a maze, yo u have to "backtrack" to the
previo us statio n 4 to see if ano ther ro ute is po ssible:
Co ntinue the search by mo ving o n fro m statio n 4 to statio n 2, shading statio n 2 in gray:
Repeat this pro cedure with statio ns 8 , 7, and then 6 :
Statio n 6 is a dead end because there are no neighbo ring statio ns that yo u have no t already visited, so yo u
can co lo r statio n 6 black and backtrack to statio n 7. Yo u get he same result at statio n 7, so co lo r 7 black and
backtrack to statio n 8 .
Observe that statio n 8 still has an unvisited neighbo r (statio n 9 ), so head in that directio n and eventually yo u
will reach statio n 10 , yo ur destinatio n:
In the abo ve graphic, yo u can see that yo u have three different statio n co lo rs:
1. Black vertices have been visited and are fully pro cessed.
2. Gray vertices have been visited, but they may have an unvisited neighbor.
3. White vertices have no t been visited at all yet.
Instead o f sto pping when yo u reach the destinatio n vertex (at statio n 10 ), it's just to let the algo rithm explo re
the entire graph such that, when it's finished, all vertices are co lo red black. Yo u need to make o ne mo re
enhancement since this algo rithm is trying to find a path fro m the designated start vertex (statio n 4) to a
destinatio n vertex (statio n 10 ). In the images abo ve, yo u co nnected statio ns with arro ws in the directio n o f the
search. Ho wever, if instead yo u "flip" the arro ws so they reco rd where the search came from, yo u can recreate
the path fro m the start vertex to any o ther vertex in the graph by fo llo wing the arro ws in reverse:
Yo u can reco nstruct the path fro m the start vertex to any destinatio n vertex quickly by starting at a destinatio n
vertex and fo llo wing the black "previo us" arro ws all the way back to the start vertex. The co mputed path here
is 4 , 2, 8, 9 , 10 . This algo rithm is no t designed to co mpute the sho rtest path between two vertices. Fo r
example, altho ugh statio ns 4 and 5 are directly co nnected by the blue subway line, the co mputed path is 4 , 2,
8, 9 , 10 , 3, 5 .
This brief example highlights a Depth-First Search o ver a graph. When faced with that decisio n, try visiting
some vertex that you haven't yet visited; when yo u reach a dead end, backtrack to the previous vertex to see if
yo u missed a ro ute to an unvisited vertex. Co ntinue this appro ach until all vertices are visited.
It's time to apply these co ncepts to yo ur pro gram. Mo dify SubwayMat rix as sho wn:
CODE TO TYPE: SubwayMatrix class
package subway;
public class SubwayMatrix {
final static int White = 0;
final static int Gray = 1;
final static int Black = 2;
final int n;
final boolean matrix[][];
int src;
final int previous[];
final int color[];
public SubwayMatrix(int numStations) {
n = numStations;
matrix = new boolean[n+1][n+1];
previous = new int[n+1];
color = new int[n+1];
src = 0;
}
public void addLine(int[] stations) {
for (int i = 1; i < stations.length; i++) {
matrix[stations[i-1]][stations[i]] = true;
matrix[stations[i]][stations[i-1]] = true;
}
}
}
The SubwayMat rix co nstructo r requires the to tal number o f vertices so it can co nstruct the pre vio us and
co lo r arrays, as well as the m at rix[][] two -dimensio nal array. Because statio ns (hence vertices) are
numbered fro m 1 .. num Ve rt ice s (which makes the co de easier to read and write), these arrays are all o ne
size larger than they need to be. SubwayMatrix also sto res the so urce vertices, src, fro m which the desired
search is made. This is impo rtant because this algo rithm ultimately determines the path between the so urce
vertex, src, and every o ther vertex to which it is co nnected in the graph. Initially we see, src=0 , which means
that the algo rithm has no t yet executed.
To implement Depth-First Search o ver a graph, yo u need to kno w abo ut recursio n. Instead o f trying to so lve a
pro blem all at o nce, recursio n breaks a pro blem into smaller pieces. Fo r example, instead o f trying to find the
full path, start by visiting the start vertex. To visit a vertex u yo u shade it gray to remember that vertex u is no
lo nger unvisited. Then, recursively visit all neighboring vertices of u. Once these recursive tasks are do ne, yo u
shade u black to indicate that yo u are do ne with the vertex. This appro ach wo rks because yo u use the co lo r o f
the vertices to reco rd yo ur pro gress. When visiting a neighbo r v o f u, be sure to reco rd that pre vio us[v]=u
so yo u can reco nstruct the path fro m the so urce vertex to any o ther co nnected vertex in the graph. No te that in
a co nnected graph, after co mpleting the search, o nly the starting vertex has no co mputed pre vio us vertex.
The fo llo wing pseudo co de describes the Depth-First Search algo rithm:
OBSERVE: pseudo co de fo r Depth-First Search
dfsSearch(s)
foreach v in V do
color[v] = White
dfsVisit(s)
dfsVisit(u)
color[u] = Gray
foreach neighbor v of u do
if (color[v] = White) then
previous[v] = u
dfsVisit(v)
color[u] = Black
The algo rithm starts by co lo ring every vertex in the graph whit e befo re it visits the starting vertex, s. The visit
functio n, df sVisit (u), is a recursive functio n which invo kes df sVisit (v) o n each unvisited neighbo r v o f u.
As with previo us lesso ns, it is wo rth "stepping thro ugh" the executio n to make sure that it will wo rk pro perly.
In do ing so , yo u will see exactly ho w recursio n allo ws yo u to backtrack in yo ur search. Let's start by
graphically representing the state o f the algo rithm and its pro gress thro ugh the pseudo co de when
df sVisit (4 ) is called. Each executing pseudo co de statement is sho wn in red o n the right.
Once vertex 4 is co lo red gray, the f o re ach lo o p pro cesses each o f its neighbo rs; let's start with vertex 1.
Since its co lo r is Whit e , set pre vio us[1]=4 and recursively call df sVisit (1). When this call returns, the
f o re ach lo o p will co ntinue where it left o ff, and then pro cess the o ther two neighbo rs o f 4 (namely vertex 2
and 5). In o ther wo rds, the algo rithm will backtrack to vertex 4. This graphic sho ws the "call stack" and the
seco nd invo catio n o f df sVisit ():
The seco nd df sVisit (1) functio n first co lo rs vertex 1 gray. It then tries to find a neighbo r o f vertex 1 that is
white (indicating that it remains unvisited). Since there are no unvisited neighbo rs o f vertex 1, the functio n
co lo rs vertex 1 black and then returns. This is the key backtracking step—go ing back to an earlier po int in the
co mputatio n. The call stack sho ws that df sVisit (4 ) is still waiting fo r df sVisit (1) to co mplete so it can mo ve
o n to the o ther neighbo rs o f 4. Assuming that vertex 2 is visited next (after vertex 1), the next recursive call
(and co rrespo nding graph state) lo o ks like this:
Yo u can co ntinue this exercise as lo ng as yo u want, ultimately pro ducing the final graphic described earlier
where every vertex is co lo red black. With this pseudo co de in hand, yo u're ready to begin pro gramming.
Mo dify SubwayMat rix as sho wn to implement Depth-First Search o ver a graph represented using an
adjacency matrix representatio n:
CODE TO TYPE: Mo dificatio ns to SubwayMatrix
package subway;
import java.util.*;
public class SubwayMatrix {
final static int White = 0;
final static int Gray = 1;
final static int Black = 2;
final int n;
final boolean matrix[][];
int src;
final int previous[];
final int color[];
public SubwayMatrix(int numStations) {
n = numStations;
matrix = new boolean[n+1][n+1];
previous = new int[n+1];
color = new int[n+1];
src = 0;
}
public void dfsSearch(int s) {
for (int v = 1; v <= n; v++) {
color[v] = White;
previous[v] = 0;
}
dfsVisit(s);
src = s;
}
void dfsVisit(int u) {
color[u] = Gray;
for (int v = 1; v <= n; v++) {
if (matrix[u][v] && color[v] == White) {
previous[v] = u;
dfsVisit (v);
}
}
color[u] = Black;
}
public void addLine(int[] stations) {
for (int i = 1; i < stations.length; i++) {
matrix[stations[i-1]][stations[i]] = true;
matrix[stations[i]][stations[i-1]] = true;
}
}
}
This co de fo llo ws the pseudo co de fairly faithfully. Let's investigate mo re clo sely.
OBSERVE: Initializing and executing search
public void dfsSearch(int s) {
for (int v = 1; v <= n; v++) {
color[v] = White;
previous[v] = 0;
}
dfsVisit(s);
src = s;
}
The df sSe arch(int s) metho d first initializes the algo rithm's state by re se t t ing t he co lo r o f e ach ve rt e x
whit e and cle aring t he previous links. Using an array-based sto rage o f the graph allo ws yo u to write a
simple f o r lo o p to iterate o ver all vertices in the graph. Recall that o ne o f the fundamental tasks o f a depth first
search algo rithm is to identify the neighbo rs fo r a vertex u. With an array-based sto rage, yo u o nly need to use
a f o r (int v = 1; v <= n; v++) lo o p to lo cate the t rue m at rix[u][v] entries fo r different v values. Remember
that the vertices are numbered fro m 1 to n. Once the search is co mplete, it se t s t he src variable t o re co rd
t he so urce ve rt e x use d f o r t he se arch. No w let's investigate the recursive df sVisit (int u) metho d:
OBSERVE: Recursive dfsVisit metho d
void dfsVisit(int u) {
color[u] = Gray;
for (int v = 1; v <= n; v++) {
if (matrix[u][v] && color[v] == White) {
previous[v] = u;
dfsVisit (v);
}
}
color[u] = Black;
}
df sVisit (int u) must first co lo r ve rt e x u gray to signal that the vertex is no lo nger unvisited. Remember
that the algo rithm recursively visits all unvisited neighbo rs o f this vertex. With the array-based implementatio n,
yo u o nly need to it e rat e t hro ugh all po ssible ve rt ice s, v, to see if m at rix[u][v] is no n-ze ro (which
m e ans t he re is an e dge be t we e n u and v) and t hat co lo r[v] is Whit e (which means vertex v has no t
yet been visited). As its final act, dfsVisit(u) co lo rs u black to signal that vertex v is fully pro cessed. With
recursio n, yo u just have to trust that it will visit all vertices required. At this po int, yo u co uld insert a
mathematical pro o f to sho w that this algo rithm will terminate; instead, let me co nvince yo u no t to by sharing
two o bservatio ns. First, df sVisit starts by co lo ring a vertex gray. Seco nd, df sVisit recursively calls df sVisit
o nly o n vertices that are co lo red white. If yo u put these two o bservatio ns to gether, df sVisit will never be
called twice o n the same vertex. Since there is a fixed number o f vertices in the graph, eventually df sVisit will
run o ut o f unvisited vertices to pro cess.
To co mplete the implementatio n, add a pat h(d) metho d that returns a List o f integers representing the
statio ns between the o riginal so urce vertex and the given destinatio n vertex, d. This metho d traverses the
pre vio us links and prepends each vertex identifier to ensure pro per o rdering. Add this metho d to the end o f
SubwayMat rix:
CODE TO TYPE: Add path() metho d to SubwayMatrix
public List<Integer> path (int d) {
LinkedList<Integer> path = new LinkedList<Integer>();
if (src != 0 && src != d) {
while (d != 0) {
path.add(0, d);
d = previous[d];
}
}
return path;
}
To validate the abo ve co de, write so me perfo rmance tests:
Create a new pe rf o rm ance so urce fo lder:
Create a subway package in the /pe rf o rm ance so urce fo lder:
In the /pe rf o rm ance fo lder subway package, create a De m o nst rat e class as sho wn:
CODE TO TYPE: Demo nstrate class
package subway;
public class Demonstrate {
public static void main(String[] args) {
SubwayMatrix sm = new SubwayMatrix(10);
sm.addLine(new int[]{1, 4, 2, 8, 7, 6});
sm.addLine(new int[]{3, 5, 4, 2, 8, 9, 10});
sm.addLine(new int[]{3, 10});
sm.dfsSearch(4);
for (int i = 1; i <= 10; i++) {
System.out.println("4-" + i + " : " + sm.path(i));
}
}
}
Save and run it. This co de creates the subway described earlier and prints o ut the path o ne wo uld fo llo w
fro m statio n 4 to all o ther statio ns in the subway system:
OBSERVE: Sample Output Fro m Demo nstrate
4-1 : [4, 1]
4-2 : [4, 2]
4-3 : [4, 2, 8, 9, 10, 3]
4-4 : []
4-5 : [4, 2, 8, 9, 10, 3, 5]
4-6 : [4, 2, 8, 7, 6]
4-7 : [4, 2, 8, 7]
4-8 : [4, 2, 8]
4-9 : [4, 2, 8, 9]
4-10 : [4, 2, 8, 9, 10]
Yo u can verify that all o f these are valid paths in the subway system; this co de even handles the case where
the so urce and destinatio n statio ns are the same by pro ducing the empty path.
Practical Application
Let's put the Depth-First Search Algo rithm to use o n a related pro blem: generating a rectangular grid maze.
It's a related pro blem because yo u can frame the pro blem starting with a rectangular grid maze with every
interio r wall present. It's no t much o f a maze tho ugh since yo u can't mo ve thro ugh it. No w start with a cell o n
the to pmo st ro w o f the maze. If yo u can rando mly mo ve in o ne o f the (po tentially fo ur) valid directio ns, either
ho rizo ntally o r vertically, to a cell that has not yet been visited, then do so and remo ve the wall in between.
Repeat this pro cess until all cells have been visited.
In the /src so urce fo lder subway package, create a Maze Apple t class as sho wn:
CODE TO TYPE: MazeApplet class
package maze;
import javax.swing.*;
import java.awt.*;
import java.util.*;
public class MazeApplet extends JApplet {
int size
= 10, offset = 50;
int width = 500, height = 500;
final static int White = 0;
final static int Gray = 1;
final static int Black = 2;
int color[][];
LinkedList<Point> neighbors[][];
boolean hasEastWall[][];
boolean hasSouthWall[][];
void clearWall(int fromR, int fromC, int toR, int toC) {
if (fromC == toC) {
hasSouthWall[Math.min(fromR, toR)][fromC] = false;
} else {
hasEastWall[fromR][Math.min(fromC, toC)] = false;
}
}
public MazeApplet() {
hasEastWall = new boolean[height/size][width/size];
hasSouthWall = new boolean[height/size][width/size];
color
= new int[height/size][width/size];
neighbors
= new LinkedList[height/size][width/size];
for (int r = 0; r < height/size; r++) {
for (int c = 0; c < width/size; c++) {
hasEastWall[r][c] = true;
hasSouthWall[r][c] = true;
neighbors[r][c]
= new LinkedList<Point>();
if
if
if
if
(r
(r
(c
(c
!=
!=
!=
!=
0)
height/size-1)
0)
width/size-1)
{
{
{
{
neighbors[r][c].add(new
neighbors[r][c].add(new
neighbors[r][c].add(new
neighbors[r][c].add(new
Point(r-1, c));
Point(r+1, c));
Point(r, c-1));
Point(r, c+1));
Collections.shuffle(neighbors[r][c]);
}
}
dfsVisit(0,width/size/2);
hasSouthWall[height/size-1][width/size/2] = false;
}
void dfsVisit(int r, int c) {
color[r][c] = Gray;
while (!neighbors[r][c].isEmpty()) {
Point cell = neighbors[r][c].remove();
if (color[cell.x][cell.y] == White) {
clearWall(r,c, cell.x,cell.y);
dfsVisit(cell.x, cell.y);
}
}
color[r][c] = Black;
}
public void paint(Graphics g) {
g.drawLine(offset, offset, offset, offset+(height/size)*size);
}
}
}
}
g.drawLine(offset, offset, offset + (width/size/2)*size, offset);
g.drawLine(offset + size*(1+(width/size)/2), offset, offset+(width/size)*siz
e, offset);
for (int r = 0; r < height/size; r += 1) {
for (int c = 0; c < width/size; c += 1) {
if (hasSouthWall[r][c]) {
g.drawLine (offset+c*size, offset + (r+1)*size, offset+(c+1)*size, off
set + (r+1)*size);
}
if (hasEastWall[r][c]) {
g.drawLine (offset+(c+1)*size, offset + r*size, offset+(c+1)*size, off
set + (r+1)*size);
}
}
}
}
}
Save and run this applet and yo u'll see a windo w like this:
Let's take a clo ser lo o k at the co de. It has the skeletal structure o f Depth-First Search:
OBSERVE: dfsVisit(int r, int c) metho d
void dfsVisit(int r, int c) {
color[r][c] = Gray;
while (!neighbors[r][c].isEmpty()) {
Cell cell = neighbors[r][c].remove();
if (color[cell.row][cell.col] == White) {
clearWall(r,c, cell.row,cell.col);
dfsVisit(cell.row, cell.col);
}
}
color[r][c] = Black;
}
Here the co lo r array is two -dimensio nal because each cell is identified by a ro w and a co lumn. The co de
uses a java.awt .Po int class to reco rd a given cell po sitio n. First it marks the designated cell as Gray and
then, as lo ng as t he re is an unvisit e d ne ighbo r re m aining f o r t hat ce ll, it cle ars t he wall between
the cell r,c and the neighbo r ce ll.x, ce ll.y, befo re re cursive ly visit ing t hat ce ll. Once all recursio ns have
co mpleted, it sets co lo r[r][c] t o Black because it has co mpleted all pro cessing fo r that cell.
OBSERVE: clearWall metho d
void clearWall(int fromR, int fromC, int toR, int toC) {
if (fromC == toC) {
hasSouthWall[Math.min(fromR, toR)][fromC] = false;
} else {
hasEastWall[fromR][Math.min(fromC, toC)] = false;
}
}
The maze co ntains two arrays, hasSo ut hWall and hasEast Wall, that determine whether there is a wall at a
given ro w and co lumn. There is no need to wo rry abo ut no rth o r west walls, because the maze is symmetric
(meaning if yo u can get fro m cell u to v, yo u can get fro m v back to u). Using Mat h.m in, the abo ve co de
clears the so uth o r east walls as required. Drawing takes place like this:
OBSERVE: paint(Graphics) metho d
public void paint(Graphics g) {
g.drawLine(offset, offset, offset, offset+(height/size)*size);
g.drawLine(offset, offset, offset + (width/size/2)*size, offset);
g.drawLine(offset + size*(1+(width/size)/2), offset, offset+(width/size)*siz
e, offset);
for (int r = 0; r < height/size; r += 1) {
for (int c = 0; c < width/size; c += 1) {
if (hasSouthWall[r][c]) {
g.drawLine (offset+c*size, offset + (r+1)*size, offset+(c+1)*size, off
set + (r+1)*size);
}
if (hasEastWall[r][c]) {
g.drawLine (offset+(c+1)*size, offset + r*size, offset+(c+1)*size, off
set + (r+1)*size);
}
}
}
}
The f irst g.drawLine invo cat io n draws the vertical line representing the "western" vertical line o f the maze.
The ne xt t wo invo cat io ns o f g.drawLine leave a space at the to p o f the maze where the start cell exists—
exactly half-way thro ugh the first ro w o f the maze. The nested f o r lo o ps iterate o ver all po ssible cells in the
maze and draw the so uthern and/o r eastern walls fo r tho se cells if necessary.
OBSERVE: Create Maze
public MazeApplet() {
hasEastWall = new boolean[height/size][width/size];
hasSouthWall = new boolean[height/size][width/size];
color
= new int[height/size][width/size];
neighbors
= new LinkedList[height/size][width/size];
for (int r = 0; r < height/size; r++) {
for (int c = 0; c < width/size; c++) {
hasEastWall[r][c] = true;
hasSouthWall[r][c] = true;
neighbors[r][c]
= new LinkedList<Point>();
if
if
if
if
(r
(r
(c
(c
!=
!=
!=
!=
0)
height/size-1)
0)
width/size-1)
{
{
{
{
neighbors[r][c].add(new
neighbors[r][c].add(new
neighbors[r][c].add(new
neighbors[r][c].add(new
Point(r-1, c));
Point(r+1, c));
Point(r, c-1));
Point(r, c+1));
}
}
}
}
Collections.shuffle(neighbors[r][c]);
}
}
dfsVisit(0,width/size/2);
hasSouthWall[height/size-1][width/size/2] = false;
}
The Maze Apple t co nstructo r creates the co lo r sto rage array used fo r Depth-First Search. It also creates the
arrays fo r whether hasSo ut hWall and hasEast Wall exist. Finally, each cell has a number o f ne ighbo rs—
between 2 and 4, depending o n where that cell exists in the maze. There is a two -dimensio nal array,
ne ighbo rs, where each element is a LinkedList o f Cell o bjects. The ne st e d f o r lo o p inst ant iat e s t he
list o f all ne ighbo rs f o r e ach ce ll and then uses Co lle ct io ns.shuf f le to ensure that when dfsVisit(0,0)
executes, it will search thro ugh the maze rando mly.
The search starts in the middle o f the first ro w, at cell (0 , width/size/2). The end po int o f the maze is identified
by remo ving the so uth wall o f the cell in the middle o f the final ro w.
Lessons Learned
T wo -dim e nsio nal bo o le an m at rice s can capt ure re lat io nships be t we e n n e le m e nt s. A
simple graph is co mpo sed o f unique edges between any two elements in a set. The range o f the
matrix represents the vertices, and each value in the matrix is a bo o lean that represents the
existence o f an edge between two vertices in the graph.
A m at rix o f co m ple x t ype s can capt ure m e t adat a abo ut t he e dge s. Instead o f simply
reco rding the existence o f an edge, the value in matrix[u][v] can co ntain info rmatio n abo ut the
relatio nship, including real-wo rld pro perties such as distance o r co st.
De pt h-First Se arch is blind and ne e ds t o kno w t he t arge t de st inat io n so it kno ws whe n
it is do ne . Instead o f trying to co nduct an intelligent search, Depth-First Search tries each
available cho ice, relying o n recursio n and backtracking to ensure that the entire space will be
searched in pursuit o f the go al.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Graph Adjacency List and Shortest Path Algorithms
Lesson Objectives
After co mpleting this lesso n, yo u will be able to :
represent graphs using Adjacency Lists.
explain ho w Breadth-First Search uses a Queue to search a graph.
Searching For Optimal Paths
In the last lesso n, yo u applied a Depth-First Search algo rithm to traverse a graph. Ho wever, Depth-First Search will no t
help yo u co mpute the sho rt e st path between two vertices. In this lesso n, we'll learn ho w to co mpute the path with the
fewest number o f edge traversals between a given so urce and destinatio n vertex. In tackling this pro blem, yo u'll also
revise the way that graphs are sto red.
Representing Graph By Adjacency List
The SubwayMat rix class yo u designed in the prio r lesso n represents a graph using a two -dimensio nal
array kno wn as the adjacency matrix. An alternate representatio n fo r graphs is an adjacency list, which is a
mo re efficient data structure to use fo r sparse graphs. A graph with n vertices may po tential have n*(n-1)/2
edges (which demo nstrates quadratic gro wth), but a sparse graph has much fewer edges. Fo r example,
suppo se yo u want to use Breadth-First Search to determine the fewest number o f subway statio ns to visit in
the New Yo rk City subway system given a so urce and destinatio n statio n. Start by co nstructing a graph where
the vertices represent the 421 subway statio ns (46 8 if yo u individually co unt the subway statio ns that belo ng
to o ne o f the 32 statio n co mplexes). Theo retically, a graph with 421 vertices co uld have up to 421*420 /2, o r
8 8 ,410 , individual tracks co nnecting pairs o f statio ns. The actual number o f statio n pairings will be much
smaller, given the physical reality o f subway co nstructio n.
No w yo u'll develo p an adjacency list implementatio n that sto res a co llectio n o f neighbo ring vertices fo r each
vertex. As graphs beco me larger (and sparser) this fo rm o f representatio n will decrease the sto rage
requirements o f a graph significantly. Also , instead o f being fo rced to use a f o r lo o p to iterate o ver all
po ssible edges that might exist, co de using an adjacency list will o nly iterate o ver the existing kno wn
neighbo rs. The perfo rmance benefits will be negligible fo r small graphs, but when yo u tackle larger graphs,
yo u'll see the impro ved perfo rmance.
We'll co ntinue wo rking in the Graphs pro ject fo r this lesso n.
In the /src so urce fo lder subway package, create a SubwayList class. This class bo rro ws much o f the
implementatio n fro m SubwayMat rix:
CODE TO TYPE: SubwayList
package subway;
import java.util.*;
public class SubwayList {
final static int White = 0;
final static int Gray = 1;
final static int Black = 2;
final int n;
final Set<Integer>[] neighbors;
int src;
final int previous[];
final int color[];
public SubwayList(int numStations) {
n = numStations;
neighbors = new TreeSet[n+1];
for (int i = 1; i <= n; i++) {
neighbors[i] = new TreeSet<Integer>();
}
previous = new int[n+1];
color = new int[n+1];
src = 0;
}
public void addLine(int[] stations) {
for (int i = 0; i < stations.length-1; i++) {
neighbors[stations[i]].add(stations[i+1]);
neighbors[stations[i+1]].add(stations[i]);
}
}
public ArrayList<Integer> path (int d) {
ArrayList<Integer> path = new ArrayList<Integer>();
if (src != 0 && src != d) {
while (d != 0) {
path.add(0, d);
d = previous[d];
}
}
return path;
}
}
Let's take a clo ser lo o k at this class:
OBSERVE: SubwayList Structure
public class SubwayList {
final static int White = 0;
final static int Gray = 1;
final static int Black = 2;
final int n;
final Set<Integer>[] neighbors;
int src;
final int previous[];
final int color[];
public SubwayList(int numStations) {
n = numStations;
neighbors = new TreeSet[n+1];
for (int i = 1; i <= n; i++) {
neighbors[i] = new TreeSet<Integer>();
}
previous = new int[n+1];
color = new int[n+1];
src = 0;
}
...
}
Instead o f using a two -dimensio nal array, we use a single array, ne ighbo rs, to represent the Se t o f
neighbo ring vertices fo r each vertex in the graph. The index into ne ighbo rs is the vertex identifier (a number
fro m 1 .. n). No te that each element in the ne ighbo rs array is a Se t <Integer>. With this change, the addLine
metho d no w invo kes the add metho d to insert each vertex. Yo u do n't need to check whether two vertices are
already co nnected because the T re e Se t implements set-based semantics, so duplicates are prevented. The
same White, Gray, and Black co nstants are used, in additio n to the color and previous arrays.
Breadth-First Search
While Depth-First Search co mputes valid paths between two vertices in a co nnected graph, there is no
guarantee that the co mputed path is the sho rtest that exists. Yo u'll need to try ano ther appro ach. A BreadthFirst Search thro ugh a graph starts at a so urce vertex, s, then pro ceeds to visit all vertices that are o ne edge
away fro m s, then vertices no mo re than two edges away, then vertices no mo re than three edges away, and
so o n. The search pro ceeds metho dically fro m the so urce vertex, radiating o utwards until all vertices in the
co nnected graph are visited.
Using the same subway system fro m the previo us lesso n, let's co mpute the sho rtest path fro m statio n 4 to
all o ther statio ns in the system. Start by co lo ring vertex 4 Gray:
No w three statio ns are directly co nnected to statio n 4, so they are just o ne edge away. Co lo r statio n 4 Black
(to signal that it's do ne) and co lo r in Gray statio ns 1, 2, and 5. Reco rd the previo us statio n in the path (in this
case, statio n 4) using an arro w fo r each o f these statio ns:
At this po int, statio n 1 is a dead end because it has no unvisited neighbo rs. Ho wever, yo u can co ntinue to
extend the search o utwards fro m statio ns 2 and 5. Be sure to co lo r statio ns 2 and 5 Black because they are
no w fully pro cessed and update previo us links fo r statio ns 3 and 8 :
Co ntinue this pro cess until all vertices are co lo red Black and all previo us links are assigned. Trace a path
fro m any destinatio n statio n to statio n 4 (the so urce statio n fo r the search) and yo u wo n't be able to find a
sho rter path than the o ne co mputed by Breadth-First Search:
We've reused the co ncept o f co lo ring vertices develo ped in the last lesso n fo r Depth-First Search.
Specifically,:
black vertices have been visited and are fully pro cessed.
gray vertices have been visited but they may have an unvisited neighbor.
white vertices have no t been visited yet at all.
The fundamental questio n fo r implementing Breadth-First Search is ho w to keep track o f the state o f the
algo rithm as it pro gresses. Depth-First Search maintains o nly o ne "current vertex" as it searches thro ugh the
graph, backtracking to o verco me dead ends. Ho wever, Breadth-First Search needs to keep track o f the Gray
vertices that it has identified fo r explo ratio n. It also must make sure to pro cess the vertices in o rder. In the
subway system abo ve, the sho rtest path fro m statio n 4 to statio n 9 co ntains three edges (4 , 2, 8, 9 ); ano ther
lo nger path exists (4 , 5 , 3, 10 , 9 ).
To enable Breadth-First Search to keep track o f the Gray vertices, let's review the behavio r o f a First-in Firsto ut Queue, a versatile data structure that sto res an o rdered sequence o f items. Using the termino lo gy fro m
the Java Co llectio ns Framewo rk, a Queue is a Co llectio n that suppo rts this behavio r:
Items are added to the tail o f a Queue using the add o peratio n.
Items are remo ved fro m the head o f a Queue using the re m o ve o peratio n.
Breadth-First Search uses a Queue to maintain all Gray vertices, which represents the "bo undary" o f the
search radiating o utwards fro m the initial so urce vertex, s. While this Queue is no t empty, there may still be
o ther unvisited vertices to be pro cessed. This pseudo co de describes the Breadth-First Search algo rithm:
OBSERVE: pseudo co de fo r Breadth-First Search
bfsSearch(s)
foreach v in V do
previous[v] = 0
color[v] = White
color[s] = Gray
Q = empty Queue
add s to Q
while (Q is not empty) do
u = remove head of Q
foreach neighbor v of u do
if (color[v] = White) then
previous[v] = u
color[v] = Gray
add v to Q
color[u] = Black
In this lesso n, we'll co mplete the Breadth-First Search and Depth-First Search implementatio ns in bo th the
SubwayMat rix and SubwayList classes.
Mo dify the existing SubwayMat rix implementatio n described in the previo us lesso n, to co nvert this
pseudo co de to Java:
CODE TO TYPE: Mo dificatio ns to SubwayMatrix class
package subway;
import java.util.*;
public class SubwayMatrix {
final static int White = 0;
final static int Gray = 1;
final static int Black = 2;
final int n;
final boolean matrix[][];
int src;
final int previous[];
final int color[];
public SubwayMatrix(int numStations) {
n = numStations;
matrix = new boolean[n+1][n+1];
previous = new int[n+1];
color = new int[n+1];
src = 0;
}
public void addLine(int[] stations) {
for (int i = 1; i < stations.length; i++) {
matrix[stations[i-1]][stations[i]] = true;
matrix[stations[i]][stations[i-1]] = true;
}
}
public void dfsSearch(int s) {
for (int v = 1; v <= n; v++) {
color[v] = White;
previous[v] = 0;
}
dfsVisit(s);
src = s;
}
void dfsVisit(int u) {
color[u] = Gray;
for (int v = 1; v <= n; v++) {
if (matrix[u][v] && color[v] == White) {
previous[v] = u;
dfsVisit (v);
}
}
color[u] = Black;
}
public List<Integer> path (int d) {
LinkedList<Integer> path = new LinkedList<Integer>();
if (src != 0 && src != d) {
while (d != 0) {
path.add(0, d);
d = previous[d];
}
}
return path;
}
public void bfsSearch(int s) {
for (int v = 1; v <= n; v++) {
color[v] = White;
previous[v] = 0;
}
Queue<Integer> q = new LinkedList<Integer>();
color[s] = Gray;
q.add(s);
while (!q.isEmpty()) {
int u = q.remove();
for (int v = 1; v <= n; v++) {
if (matrix[u][v] && color[v] == White) {
previous[v] = u;
color[v] = Gray;
q.add(v);
}
}
color[u] = Black;
}
src = s;
}
}
Let's review this co de in mo re detail:
OBSERVE: Initializing Breadth-First Search
public void bfsSearch(int s) {
for (int v = 1; v <= n; v++) {
color[v] = White;
previous[v] = 0;
}
Queue<Integer> q = new LinkedList<Integer>();
color[s] = Gray;
q.add(s);
...
}
The f o r lo o p iterates o ver all vertices in the graph to reset their co lo r and pre vio us values. It then
co nst ruct s a Que ue o f int e ge rs st art ing wit h s as it s init ial e le m e nt . When analyzing an algo rithm,
it's helpful to identify so me invariants that are always true. Fro m the pseudo co de yo u saw earlier, o bserve
that any ve rt e x in t he Que ue is co lo re d Gray.
There are many classes in the Java Co llectio ns Framewo rk that implement the Queue interface; we cho se the
Linke dList class because it implements add (to the tail o f the queue) and re m o ve (fro m the head o f the
queue) efficiently. Also , o bserve a co mmo n idio m when using the Co llectio ns Framewo rk: referring to the
instantiated o bject, q, by its interface Que ue rather than the instantiating class Linke dList :
OBSERVE: Co mputing Breadth-First Search
while (!q.isEmpty()) {
int u = q.remove();
for (int v = 1; v <= n; v++) {
if (matrix[u][v] && color[v] == White) {
previous[v] = u;
color[v] = Gray;
q.add(v);
}
}
color[u] = Black;
}
src = s;
The algo rithm pro ceeds by re m o ving t he he ad e le m e nt , u, f ro m t he Que ue and adding t o t he t ail
t ho se unvisit e d ve rt ice s t hat are ne ighbo rs o f u.
As lo ng as there are vertices in the Queue that need to be pro cessed, the while lo o p will remo ve the head
vertex fro m the Queue. At so me po int the while lo o p must terminate because o nly the unvisited vertices
(co lo red White) are ever co nsidered fo r additio n to the Queue, and there are a finite number o f vertices in the
graph. No te that this co de maintains the invariant that o nly Gray vertices are added to the Queue. The add
metho d pro perly inserts the vertex at the tail to maintain pro per o rdering o f the vertices within the Queue. That
is, there is no o ther Gray o r White vertex in the graph that is clo ser to the so urce vertex, s.
To validate this implementatio n, write this perfo rmance co de:
In the /pe rf o rm ance so urce fo lder subway package, create a Co m pare class as sho wn:
CODE TO TYPE: Co mpare Class
package subway;
import java.util.*;
public class Compare {
public static void main(String[] args) {
SubwayMatrix sm = new SubwayMatrix(10);
sm.addLine(new int[]{1, 4, 2, 8, 7, 6});
sm.addLine(new int[]{3, 5, 4, 2, 8, 9, 10});
sm.addLine(new int[]{3, 10});
sm.dfsSearch(4);
List dfsPaths[] = new List[11];
for (int i = 1; i <= 10; i++) {
dfsPaths[i] = sm.path(i);
}
sm.bfsSearch(4);
List bfsPaths[] = new List[11];
for (int i = 1; i <= 10; i++) {
bfsPaths[i] = sm.path(i);
}
for (int i = 1; i <= 10; i++) {
if (bfsPaths[i].size() < dfsPaths[i].size()) {
System.out.println("4-" + i + " : " + bfsPaths[i] + " (instead of " + df
sPaths[i] + ")");
}
}
}
}
Save and run it. As yo u can see, this co de directly co mpares Breadth-First Search against Depth-First
Search o n the same subway system and prints o nly the paths that are sho rter when co mputed by BreadthFirst Search. The o utput belo w is co mputed and yo u can verify that it finds three sho rter paths in the graph:
OBSERVE: Co mpariso n o f Breadth-First and Depth-First Search o n Subway System
4-3 : [4, 5, 3] (instead of [4, 2, 8, 9, 10, 3])
4-5 : [4, 5] (instead of [4, 2, 8, 9, 10, 3, 5])
4-10 : [4, 5, 3, 10] (instead of [4, 2, 8, 9, 10])
No w yo u can co mplete SubwayList by implementing the Breadth-First Search algo rithm also :.
CODE TO TYPE: Mo dificatio ns to SubwayList class
package subway;
import java.util.*;
public class SubwayList {
final static int White = 0;
final static int Gray = 1;
final static int Black = 2;
final int n;
final Set<Integer>[] neighbors;
int src;
final int previous[];
final int color[];
public SubwayList(int numStations) {
n = numStations;
neighbors = new TreeSet[n+1];
for (int i = 1; i <= n; i++) {
neighbors[i] = new TreeSet<Integer>();
}
previous = new int[n+1];
color = new int[n+1];
src = 0;
}
public void addLine(int[] stations) {
for (int i = 1; i < stations.length; i++) {
neighbors[stations[i-1]].add(stations[i]);
neighbors[stations[i]].add(stations[i-1]);
}
}
public ArrayList<Integer> path (int d) {
ArrayList<Integer> path = new ArrayList<Integer>();
if (src != 0 && src != d) {
while (d != 0) {
path.add(0, d);
d = previous[d];
}
}
return path;
}
public void dfsSearch(int s) {
for (int v = 1; v <= n; v++) {
color[v] = White;
previous[v] = 0;
}
dfsVisit(s);
src = s;
}
void dfsVisit(int u) {
color[u] = Gray;
for (int v : neighbors[u]) {
if (color[v] == White) {
previous[v] = u;
dfsVisit(v);
}
}
color[u] = Black;
}
}
Because yo u can iterate o ver just the neighbo rs o f a given vertex, there is do n't have to use a f o r lo o p within
df sVisit to check fo r all po ssible edges that might exist like yo u did when the graph was represented as an
Adjacency Matrix.
To understand which representatio n o ptio n to cho o se (Adjacency Matrix o r Adjacency List), figure o ut which
types o f graphs yo u'll be pro cessing. In a dense graph, the number o f edges can gro w pro po rtio nal to the
square o f the number o f vertices. In a sparse graph, the number o f edges gro ws linearly with the number o f
vertices.
The fo llo wing perfo rmance co de generates stylized graphs (representing subway lines) o n which to test
these algo rithms. Specifically, these graphs have n=k2 +2 vertices and k3 -k2 +2k edges. The number o f edges
is ro ughly n 1.5 , where n is the number o f vertices. The fo llo wing is the example fo r k=4 which co ntains n=16
vertices and 56 edges (vertex 1 is the leftmo st vertex and vertex 18 is the rightmo st o ne):
In the /pe rf o rm ance so urce co de fo lder subway package, create a St ylize dDe m o nst rat io n class as
sho wn:
CODE TO TYPE: StylizedDemo nstratio n
package subway;
public class StylizedDemonstration {
static int Max = 20;
static int m = 1000000;
static SubwayMatrix mat;
static SubwayList list;
static int numVertices;
public static void main(String[] args) {
System.out.println("n\tMatrix\t\tList");
for (int k = 2; k <= 64; k *= 2) {
float totalMatrix = 0, totalList = 0;
for (int numTrials = 1; numTrials <= Max; numTrials++) {
generate(k);
System.gc();
long now = System.nanoTime();
mat.dfsSearch(1);
totalMatrix += (System.nanoTime()-now);
System.gc();
now = System.nanoTime();
list.dfsSearch(1);
totalList += (System.nanoTime()-now);
}
System.out.println(numVertices + "\t" + totalMatrix/Max/m + "\t" + totalLi
st/Max/m);
}
}
public static void generate(int k) {
int n = k*k;
numVertices = n+2;
mat = new SubwayMatrix(n+2);
list = new SubwayList(n+2);
int[] pairs;
for (int i = 2; i <= k+1; i++) {
pairs = new int[]{1,i};
list.addLine(pairs);
mat.addLine(pairs);
}
for (int i = n-k+2; i <= n+1; i++) {
pairs = new int[]{n+2,i};
list.addLine(pairs);
mat.addLine(pairs);
}
for (int i = 0; i < k-1; i++) {
for (int j = 0; j < k-1; j++) {
int u = 2 + i*k + j;
for (int m = 0; m < k; m++) {
pairs = new int[]{u, 2+(i+1)*k+m};
list.addLine(pairs);
mat.addLine(pairs);
}
}
}
}
}
Save and run it; yo u see o utput similar to this:
OBSERVE: Co mparing Adjacency List with Adjacency Matrix
n
6
18
66
258
1026
4098
Matrix
0.0047069504
0.0098863
0.033456147
0.24323966
3.4497294
52.31373
List
0.031076651
0.05587125
0.0704818
0.18596876
1.659615
26.572315
The Adjacency Matrix implementatio n initially o utperfo rms the Adjacency List implementatio n, but the situatio n
changes quickly, and the Adjacency Matrix implementatio n pro gresses twice as slo wly. No te that the
algo rithm has no t changed, but rather the structural representatio n o f the graph.
Lessons Learned
Re pre se nt at io n o f a dat a st ruct ure im pact s t he pe rf o rm ance f o r an algo rit hm . Even
when yo u have identified the pro per algo rithm to use, make sure that yo u are no t using a subo ptimal data structure. The Adjacency List is preferred fo r sparse graphs while Adjacency Matrix is
o ptimal fo r dense graphs. Only yo u kno w which types o f graphs that yo u intend to pro cess, so
cho o se wisely!
St acks suppo rt last -in, f irst -o ut while Que ue s suppo rt f irst -in, f irst -o ut . The difference
between Depth-First Search and Breadth-First Search can be traced directly to the data structures
used to represent the active search. Depth-First Search uses the call stack to sto re pro gress,
backtracking whenever it hits a dead end; Breadth-first Search uses a queue to metho dically search
a graph.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Priority Queues
Lesson Objectives
After co mpleting this lesso n yo u will be able to :
write yo ur o wn heap implementatio n using array-based sto rage.
describe two distinct implementatio ns o f prio rity queues.
co mpute the resulting heap structure after a number o f insertio ns and remo vals.
co mpute a Minimum Spanning Tree fo r a graph using Prim's Algo rithm.
Priority Queue Data Structure
A Queue is the data structure used when yo u need First-in, First-o ut behavio r as items are added to , and remo ved
fro m, a co llectio n. No rmally a queue is used to mo del a sequence o f items using the insertio n time as the co mparato r
between elements. The Priority Queue is a related structure that behaves like a queue, except that the items in the
queue all have an asso ciated priority value (typically an integer). In a prio rity queue, each item is added with an
asso ciated prio rity. When remo ving an element fro m the prio rity queue, the item with the smallest prio rity value is
remo ved first. That is, the mo st impo rtant item in the prio rity queue is the o ne with the smallest prio rity value. Typically,
prio rity values are no n-negative, so zero has the highest impo rtance while +Infinity has the lo west prio rity.
Note
If two o r mo re items have the same lo west prio rity value, either o ne may be returned when yo u request
the remo val o f an item fro m the prio rity queue.
To see a prio rity queue in o peratio n, let's intro duce the Pair class, which co ntains an integer key value and its
asso ciated integer prio rity.
Create a new Java Pro ject named Prio rit y and assign it to the J ava6 _Le sso ns wo rking set.
In the /src so urce fo lder, create a package named m st .
In the m st package, create a class named Pair as sho wn:
CODE TO TYPE: Pair class
package mst;
public class Pair {
int key;
int priority;
public Pair (int k, int p) {
key = k;
priority = p;
}
public String toString() {
return "(" + key + ",p=" + priority + ")";
}
}
The Pair class asso ciates a prio rity value with each key value. The class belo w co difies that smaller priority values
represent more important items in the priority queue.
In the m st package, create a class Prio rit yCo m parat o r as sho wn:
CODE TO TYPE: Prio rityCo mparato r class
package mst;
import java.util.Comparator;
public class PriorityComparator implements Comparator<Pair> {
public int compare(Pair first, Pair second) {
return first.priority - second.priority;
}
}
This Co m parat o r class determines ho w to co mpare two Pair o bjects in the Prio rit yQue ue . Pair o bjects with lo wer
prio rity values are co nsidered to be mo re impo rtant, so the co m pare metho d must return zero when the two o bjects
have the same prio rity and a negative number when the first o bject's prio rity value is smaller than the seco nd o bject's
prio rity value (in o ther wo rds, when the first o bject has higher impo rtance).
No w yo u can create a Prio rit yQue ue o bject to which yo u add Pair o bjects and fro m which yo u retrieve Pair o bjects,
in o rder o f their impo rtance:
In the m st package, create a class Sam ple Prio rit yQue ue as sho wn:
CODE TO TYPE: SamplePrio rityQueue class
package mst;
import java.util.*;
public class SamplePriorityQueue {
public static void main(String[] args) {
PriorityComparator comp = new PriorityComparator();
PriorityQueue<Pair> pq = new PriorityQueue<Pair>(10, comp);
pq.add(new
pq.add(new
pq.add(new
pq.add(new
Pair(1000,
Pair(2000,
Pair(3000,
Pair(4000,
5));
10));
7));
3));
while (!pq.isEmpty()) {
System.out.println(pq.remove());
}
}
}
Save and runit; yo u see this o utput.
OBSERVE: Output o f SamplePrio rityQueue
(4000,p=3)
(1000,p=5)
(3000,p=7)
(2000,p=10)
Yo ur first tho ught might be that a Prio rit yQue ue o nly so rts its elements by their respective prio rities. Ho wever, it isn't
mandato ry to so rt queue elements by their respective prio rity values. Yo u can make yo ur so rt mo re efficient by
remo ving the item with highest prio rity fro m the queue. Once again, efficiency will be based o n achieving O(log n)
perfo rmance o n bo th the add and re m o ve metho ds.
In this lesso n, yo u'll learn ho w to apply prio rity queues to co mpute a Minimum Spanning Tree (MST) o f a graph.
Co mputing the MST is central to many netwo rk pro blems, because it determines the lo west aggregate to tal fo r a set o f
edges that maintains the co nnected pro perty o f a graph. So lving MST is useful fo r chip design and the
teleco mmunicatio ns industry because they are o ften co ncerned with co mputing a co nnectivity scheme that uses the
lo west to tal length o f wire. In this lesso , yo u'll learn so me limitatio ns o f the existing Prio rit yQue ue implementatio n
as fo und in the Java Co llectio ns Framewo rk, and yo u'll develo p yo ur o wn prio rity queue class using a no vel data
structure, kno wn as a heap.
Minimum Spanning T ree
A co nnected graph allo ws yo u to go fro m any vertex in the graph to any o ther vertex in the graph o ver its
edges. Given a co nnected graph G=(V,E), it's po ssible that the graph will still be co nnected even if yo u discard
many o f its edges. A Spanning Tree fo r a graph is simply a graph STG=(V,SE) where SE is a subset o f the
o riginal edges in the graph such that the remo val o f any edge in SE fro m STG results in a disco nnected graph.
There are many such spanning trees fo r a given graph. If each edge in the graph is asso ciated with a po sitive
weight, yo u might want to find a minimum spanning tree fo r a graph who se accumulated edge weights is
minimum fo r all po ssible spanning trees.
To so lve this pro blem efficiently, yo u can't just generate all po ssible spanning trees and select the o ne who se
accumulated edge weights is minimum; there are simply to o many po ssible spanning trees. At the same
time, yo u are no t trying to find a unique spanning tree; there may be many spanning trees with accumulated
edge weights that all match the same minimum value.
Prim's Algorithm is an elegant so lutio n to co nstructing a minimum spanning tree (MST) fo r a given graph by
using a greedy appro ach in which each step o f the algo rithm makes fo rward pro gress to wards a so lutio n
witho ut reversing earlier decisio ns; That is, no backtracking is necessary.
This pseudo co de describes the algo rithm:
OBSERVE: pseudo co de fo r Prim's Algo rithm
computeMST(G)
MST = empty
S = some vertex in V
T = V - S
while (T is not empty) do
find edge (u,v) with lowest weight such that (u in S) and (v in T)
add (u,v) to MST
remove v from T
add v to S
return MST
At each step thro ugh the while lo o p, the algo rithm finds the edge with lo west weight that cro sses the
bo undary fro m the set S (representing the vertices in the MST) and T (representing the vertices still to be
pro cessed). This greedy appro ach will ensure that the accumulated weights o f the MST gro ws by the smallest
po ssible increment with each step, ultimately resulting in a minimum spanning tree fo r the graph.
Let's go o ver this pseudo co de o n a specific example to make sure it's designed pro perly. Here is a sample
graph fo r which yo u will co mpute a minimum spanning tree:
Start with vertex 0 as the initial vertex in set S, which means T = {1, 2, 3, 4}. The edge (0 ,1) highlighted in red is
the edge between S and T with the lo west edge weight.
The edge (0 ,1) is added to the MST and S and T are updated acco rdingly. In the middle graph abo ve, the edge
(1,2) is the edge between S and T with the lo west edge weight, so this edge is added to the MST, and S and T
are updated. In the third image abo ve, the edge (2,4) is the edge between S and T with the lo west edge weight,
so it is added to the MST.
The final step in the algo rithm belo w sho ws that edge (2,3) is the edge between S and T with the lo west edge
weight, so it is added to the MST and the algo rithm co mpletes. Yo u wo n't be able to find ano ther spanning
tree fo r this graph with accumulated weights that is lo wer than the 11 co mputed here.
What data structures sho uld yo u use to implement this algo rithm? Well, yo u can use Co lle ct io n o bjects to
represent sets S and T, but the mo st co stly o peratio n is within the while lo o p where yo u have to efficiently
find the edge with the lo west weight fo r all edges (u,v) where u belo ngs to S and v belo ngs to T. It seems like,
as S gro ws in size, it will be increasingly co mplicated to co mpute this edge. If yo u had to check each edge that
exists between S and T, the perfo rmance o f the algo rithm wo uld suffer.
Ho w can a prio rity queue be used to so lve this pro blem? The MST abo ve is co mputed by starting at vertex 0 ,
so the three edges being inspected first are tho se directly co nnected to vertex 0 : the edges to vertices 1, 3,
and 4. No te that vertex 2 is no t yet "o n the search ho rizo n" so the distance fro m vertex 0 to vertex 2 must be
co nsidered to be +Infinity. So , what if yo u were able to maintain a prio rity queue that co ntained all vertices with
a co mputed prio rity o f the current sho rtest distance fro m any ve rt e x in S? Fo r example, at the start yo u co uld
insert the vertices 1, 3, and 4 into the prio rity queue with prio rities 2, 8 , and 4 respectively; vertex 2 wo uld also
be in the prio rity queue, but its prio rity wo uld be +Infinity. The prio rity queue wo uld lo o k like this: (1, p=2) > (4 ,
p=4 ) > (3, p=8) > (2, p=INF). The o rdering in the prio rity queue is do ne acco rding to increasing distance, so
the first vertex to be remo ved fro m the prio rity queue wo uld be vertex 1. At this po int, yo u co uld review the
neighbo rs o f vertex 1 (in this case vertex 2) and determine to insert (2, p=3) into the prio rity queue; ho wever,
this vertex already exists in the prio rity queue. So meho w yo u need to decrease the prio rity value asso ciated
with a vertex that already exists in the prio rity queue. Technically, yo u want to find the key value associated with
vertex 2 in the queue and decrease its priority value from +INF to 3, such that the resulting prio rity queue is (2,
p=3) > (4 , p=4 ) > (3, p=8).
Co ntinuing fro m this prio rity queue, remo ve vertex 2 since it has the smallest prio rity value and yo u can
o bserve fro m the presence o f the edge (2,4) that yo u can co nnect to vertex 4 with a distance o f 1 instead of the
current distance of 4 as maintained in the priority queue. Similarly, with edge (2,3) yo u can co nnect to vertex 3
current distance of 4 as maintained in the priority queue. Similarly, with edge (2,3) yo u can co nnect to vertex 3
with a distance o f 4 instead of the current distance of 8 as shown in the priority queue. To do that, yo u need to
find the key value associated with these two vertices in the queue and decrease that key value so that the
resulting prio rity queue is (4 , p=1) > (3, p=5 ).
No w yo u're faced with a dilemma: lo cating a given element in a prio rity queue is po tentially an O(n) o peratio n.
Altering the prio rity asso ciated with an element in the prio rity queue seems like it can o nly be do ne safely by
remo ving the element first and then reinserting it with the new prio rity.
Start by creating a class to represent an edge in the co mputed minimum spanning tree.
In the m st package, create an Edge class as sho wn:
CODE TO TYPE: Edge class
package mst;
public class Edge {
int start;
int end;
public Edge (int s, int e) {
start = s;
end = e;
}
public String toString() {
return "" + start + "-" + end;
}
}
The co de belo w demo nstrates the naive use o f Prio rit yQue ue fro m the Java Co llectio ns Framewo rk. The
initial graph is represented as a two -dimensio nal adjacency matrix where the value o f graph[i][j] is the
weight asso ciated with the edge (i,j); if no such edge exists, then graph[i][j] = 0 .
In the m st package, create a Drive r class as sho wn:
CODE TO TYPE: Driver
package mst;
public class Driver {
public static void main (String[] args) {
int[][] graph = new int[][] {
{0, 2, 0, 8, 4},
{2, 0, 3, 0, 0},
{0, 3, 0, 5, 1},
{8, 0, 5, 0, 7},
{4, 0, 1, 7, 0}};
Edge[] mst = MST.compute(graph);
for (Edge e : mst) {
System.out.println(e + "(" + graph[e.start][e.end] + ")");
}
}
}
The abo ve co de represents the graph used in the earlier example by an adjacency matrix. It co mputes a
minimum spanning tree (using the MST class that yo u will write sho rtly). The returned array mst[k] represents
the n-1 edges in the co mputed minimum spanning tree.
In the m st package, create an MST class as sho wn:
CODE TO TYPE: MST class
package mst;
import java.util.*;
public class MST {
static Edge[] compute(int[][] graph) {
int n = graph.length;
Edge[] mst = new Edge[n-1];
PriorityQueue<Pair> pq = new PriorityQueue<Pair>(n, new PriorityComparator()
);
for (int i = 1; i < n; i++) {
pq.add(new Pair(i, Integer.MAX_VALUE));
mst[i-1] = new Edge(i, -1);
}
pq.add(new Pair(0, 0));
while (!pq.isEmpty()) {
int u = pq.remove().key;
for (int v = 0; v < n; v++) {
int weight = graph[u][v];
if (weight > 0) {
for (Pair pv : pq) {
if ((pv.key == v) && (weight < pv.priority)) {
mst[v-1].end = u;
pq.remove(pv);
pv.priority = weight;
pq.add(pv);
break;
}
}
}
}
}
return mst;
}
}
Save all o f the new files and run the Drive r class; yo u see o utput which co rrespo nds to the manual
co mputatio n fro m earlier. When given a co nnected graph o f n vertices, the co rrespo nding o utput will have n-1
edges:
INTERACTIVE SESSION: Output o f Driver
1-0(2)
2-1(3)
3-2(5)
4-2(1)
Let's lo o k at this co de mo re clo sely:
OBSERVE: MST initializatio n
int n = graph.length;
Edge[] mst = new Edge[n-1];
PriorityQueue<Pair> pq = new PriorityQueue<Pair>(n, new PriorityComparator());
for (int i = 1; i < n; i++) {
pq.add(new Pair(i, Integer.MAX_VALUE));
mst[i-1] = new Edge(i, -1);
}
pq.add(new Pair(0, 0));
This co de initializes the data structures used by the algo rithm. Prim's Algo rithm starts at so me vertex—fo r the
implementatio n, yo u will start at vertex 0 , so the prio rity queue will initially co ntain Pair o bjects fo r each o f the
o ther n-1 vertices with +Infinity as the co mputed minimum distance. The final initializatio n co de inse rt s
ve rt e x 0 int o t he prio rit y que ue with a prio rity o f 0 , which ensures that this Pair o bject will be the first o ne
remo ved fro m the prio rity queue.
All o f the lo gic is co ntained in this while lo o p:
OBSERVE: Main lo o p o f Prim's Algo rithm
while (!pq.isEmpty()) {
int u = pq.remove().key;
for (int v = 0; v < n; v++) {
int weight = graph[u][v];
if (weight > 0) {
for (Pair pv : pq) {
if ((pv.key == v) && (weight < pv.priority)) {
mst[v-1].end = u;
pq.remove(pv);
pv.priority = weight;
pq.add(pv);
break;
}
}
}
}
}
This co de implements Prim's Algo rithm. While t he prio rit y que ue pq is no t e m pt y, the Pair o bje ct
wit h lo we st prio rit y is re m o ve d and its ve rt e x u is ide nt if ie d. The first f o r lo o p checks all o ther
vertices, v, t o se e if t he re is a dire ct e dge co nne ct ing u and v. If a direct edge is fo und, we ight will be
a value gre at e r t han ze ro , so yo u have to determine if this new distance is smaller than the current
sho rtest distance to v already being maintained in pq. If it turns o ut that we ight is smaller, t he o ld pair pv
m ust be re m o ve d f ro m pq and re inse rt e d wit h t he lo we r prio rit y.
Yo u can evaluate the perfo rmance o f this implementatio n by reviewing the number o f lo o ps in the co de.
Essentially there is a triply nested lo o p, each o f which iterates o ver all n vertices. This gives the wo rst-case
estimate o f O(n 3 ) fo r ho w frequently elements in the prio rity queue are adjusted. The add and re m o ve
o peratio ns o n a Prio rityQueue perfo rm in O (log n), so the perfo rmance fo r the entire algo rithm is O (n 3 log n).
Surely we can do better!
Heap Data Structure
The previo us sectio n requires a data structure that returns the smallest element o f a co llectio n in co nstant
time; but it also needs to be able to lo cate a particular element in the co llectio n and reprio ritize it efficiently
(which means in O(log n) time).
There is an interesting data structure kno wn as a heap that can serve o ur purpo ses. A heap is a binary tree
with a structure that ensures two pro perties:
Shape pro pe rt y: A leaf no de at depth k > 0 can exist o nly if all 2 k- 1 no des at the previo us level k1 exist. Additio nally, no des at a partially filled level must be added fro m left to right.
He ap pro pe rt y: Each no de in the tree co ntains a value smaller than o r equal to either o f its two
He ap pro pe rt y: Each no de in the tree co ntains a value smaller than o r equal to either o f its two
children (if it has any).
The image belo w represents a sample heap o f 16 integer values fro m 0 to 15:
The heap co nsists o f a number o f levels. The value asso ciated with each no de in the heap is guaranteed to
be smaller than o r equal to bo th o f its children. Fo r this reaso n, the ro o t o f the heap always co ntains the
smallest value in the entire heap. No te that each level in the heap is fully filled befo re new elements are added
to the next level.
Given the rigid structure impo sed by the shape pro perty, a heap can be implemented efficiently within an array
A witho ut lo sing any o f its structural info rmatio n. The image belo w demo nstrates ho w a heap can be sto red in
an array by sto ring the element value fo r a no de in the array po sitio n identified by the no de's label. The o rder
o f the elements within A can be read fro m left to right as deeper levels o f the tree are explo red.
Develo p co de that allo ws yo u to create a heap efficiently. In this lesso n, we assume that yo u kno w the
maximum size o f the heap in advance when yo u co nstruct it. Let's get started.
In the m st package, create a new class named He ap as sho wn:
CODE TO TYPE: Heap class
package mst;
public class Heap {
int n = 0;
Pair[] elements;
public Heap(int n) {
elements = new Pair[n];
}
public boolean isEmpty() {
return (n == 0);
}
public void insert (int key, int priority) {
int idx = n++;
while (idx > 0) {
int parent = (idx-1)/2;
Pair p = elements[parent];
if (priority >= p.priority) { break; }
elements[idx] = p;
idx = parent;
}
elements[idx] = new Pair (key, priority);
}
}
Let's take a clo ser lo o k at this co de:
OBSERVE: Co nstruct Heap Sto rage
int n = 0;
Pair[] elements;
public Heap(int n) {
elements = new Pair[n];
}
public boolean isEmpty() {
return (n == 0);
}
The e le m e nt s array will sto re the Pair o bjects representing the elements in the prio rity queue. Attribute n
co unts the number o f elements in the prio rity queue.
This heap will be used as a prio rity queue so each element in the heap sto res a Pair o f values, where each
key has an asso ciated priority with lo wer integer values that represent greater impo rtance. To o nstruct a heap
o f a maximum size yo u just need to reserve ro o m fo r n po tential Pair elements. The heap is empty when its
n attribute is 0 .
To insert an element, reco gnize that the values in the heap are no t fully o rdered. Rather, the o nly glo bal
pro perty is that the values o n any path o f no des fro m the ro o t to a leaf stay the same o r increase. The heap
was created with sufficient space fo r all values that yo u will insert, so when it co mes time to insert a value, yo u
can place it in the "next" array lo catio n. In do ing so , yo u co ntinue to co nfo rm to the Shape Property o f the
heap. Ho wever, when inserting an element at this lo catio n, yo u may vio late the Heap Property, so yo u'll have
to make so me adjustments. The go o d news is that yo u do n't have to reo rder all elements in the heap; rather,
yo u need to fo cus o n the ancesto r no des o f the new element, go ing all the way back to the ro o t o f the heap.
Suppo se yo u insert "0 5" into the heap o f 16 elements sho wn earlier. First, it's placed in the 17th lo catio n:
Since 0 5 is smaller than 0 9 , the two values in the no des are swapped. Co ntinue up to the parent no de o n
level 2 to see if its co ntents are smaller than 0 5.
Once again, the two values in the no des are swapped. Keep go ing, checking with the parent no de o n level 1 to
see if its co ntents are smaller than 0 5.
Once yo u hit a no de with a value that's smaller than the newly added element, yo u're do ne. The heap is no w
guaranteed to have bo th its Heap and Shape pro perties. In the worst case, yo u o nly have to check and
po tentially swap O (log n) values in the heap, so inserting an element is O(log n) and co nstructing a heap o f n
elements in the is O(n log n).
Let's lo o k clo ser at the inse rt metho d:
OBSERVE: Insert (key, prio rity) into Heap
public void insert (int key, int priority) {
int idx = n++;
while (idx > 0) {
int parent = (idx-1)/2;
Pair p = elements[parent];
if (priority >= p.priority) { break; }
elements[idx] = p;
idx = parent;
}
elements[idx] = new Pair (key, priority);
}
The heap is aware that yo u are trying to add a key value with an asso ciated prio rity. The inse rt metho d first
incre m e nt s t he co unt o f e le m e nt s n and co nsiders the new Pair lo catio n at index lo catio n idx. Given
an index lo catio n in the heap o f idx, the pare nt no de is co m put e d as (idx-1)/2. If t he ne w prio rit y be ing
adde d is large r t han t his Pair's prio rit y, yo u're do ne , and the final line o f the metho d cre at e s a Pair
o bje ct in lo cat io n idx t o co nt ain t he asso ciat e d key and priority value s. If, ho wever, yo u still have to
swap no de values, mo ve the parent Pair p within the while lo o p into the child's lo catio n idx and repeat,
setting idx to the parent lo catio n to mo ve up a level to check o nce again fo r the Heap Property. If yo u co ntinue
to swap values all the way up to the ro o t (which has index lo catio n 0 ), the while lo o p will exit and the new
Pair will be placed there. In the wo rst case, the while lo o p requires O (log n) iteratio ns.
To use the heap data structure as a prio rity queue, yo u must be able to lo cate and remo ve the element with
the lo west prio rity. As stated earlier, the ro o t o f the heap will always co ntain the Pair o f the lo west prio rity.
Ho wever, yo u can't simply remo ve this no de because that wo uld vio late bo th the Heap and Shape pro perties.
Instead, remo ve the ro o t and replace it with the last Pair in the heap. In do ing so , yo u will maintain the Shape
pro perty, but no w yo u'll have to manipulate the heap to resto re the Heap pro perty, which states that each
no de must be smaller than bo th o f its children.
Add this metho d to the end o f the He ap class:
CODE TO TYPE: smallest() metho d fo r Heap
public int smallest () {
int key = elements[0].key;
Pair last = elements[--n];
elements[0] = last;
int idx = 0;
int child = 2*idx+1;
while (child <= n) {
Pair smaller = elements[child];
if (child < n) {
if (smaller.priority > elements[child+1].priority) {
smaller = elements[++child];
}
}
if (last.priority <= smaller.priority) { break; }
elements[idx] = smaller;
idx = child;
child = 2*idx+1;
}
elements[idx] = last;
return key;
}
Let's lo o k at this co de mo re clo sely.
OBSERVE: smallest() metho d fo r Heap
public int smallest () {
int key = elements[0].key;
Pair last = elements[--n];
elements[0] = last;
int idx = 0;
int child = 2*idx+1;
while (child <= n) {
Pair smaller = elements[child];
if (child < n) {
if (smaller.priority > elements[child+1].priority) {
smaller = elements[++child];
}
}
if (last.priority <= smaller.priority) { break; }
elements[idx] = smaller;
idx = child;
child = 2*idx+1;
}
elements[idx] = last;
return key;
}
It starts by re m e m be ring t he key o f t he Pair wit h lo we st prio rit y (that is, the ro o t o f the heap), since
that's the value being returned by the metho d. Then it takes the last Pair in t he he ap and m o ve s it int o
t he ro o t po sit io n. No w the while lo o p is similar to the lo o p within the inse rt metho d. The difference is
that yo u are starting fro m idx=0 (the ro o t) and wo rking do wn so me path in the heap, to find the smaller o f the
two children o f po sitio n idx. The first child is fo und at index lo catio n 2*idx+1 and the seco nd at idx*2+2. The
abo ve co de f inds t he child wit h t he lo we st prio rit y and bre aks o ut o f t he lo o p if t he He ap
pro pe rt y is st ill m aint aine d. Otherwise it swaps the smaller value with the parent lo catio n o f idx and
co ntinues do wn that child path. This while lo o p will never iterate mo re than log n times.
To test the heap in use, create a new class named He apDrive r in the m st package as sho wn:
CODE TO TYPE: HeapDriver class
package mst;
public class HeapDriver {
public static void main(String[] args) {
Heap heap = new Heap(16);
for (int i = 15; i >= 0; i--) {
heap.insert(i, i);
}
for (Pair p : heap.elements) {
System.out.print(p.priority + " ");
}
System.out.println();
while (!heap.isEmpty()) {
System.out.println(heap.smallest());
}
}
}
Save and run this class; it co nstructs a heap o f 16 values and repeatedly adds the integers fro m 15 do wn
to 0 to fill the heap. The pro gram prints o ut a representatio n o f the array-based sto rage o f the heap (sho wn
earlier) and then demo nstrates that it can return the smallest element in the heap, o ne at a time.
INTERACTIVE SESSION: Output o f HeapDriver
0 1 2 6 7 5 3 9 12 13 8 14 10 11 4 15
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Prim's Algorithm Implementation
To co mplete this lesso n, yo u need to mo dify the He ap to be able to suppo rt Prim's Algo rithm, which needs to
lo cate a value within the heap and decrease its prio rity value. To make this wo rk, yo u need to make a number
o f mo dificatio ns to He ap:
CODE TO TYPE: Mo dificatio ns to Heap
package mst;
public class Heap {
int n = 0;
Pair[] elements;
int[] positions;
public Heap(int n) {
elements = new Pair[n];
positions = new int[n];
}
public boolean isEmpty() {
return (n == 0);
}
public void insert (int key, int priority) {
int idx = n++;
while (idx > 0) {
int parent = (idx-1)/2;
Pair p = elements[parent];
if (priority >= p.priority) { break; }
elements[idx] = p;
positions[p.key] = idx;
idx = parent;
}
elements[idx] = new Pair (key, priority);
positions[key] = idx;
}
public int smallest () {
int key = elements[0].key;
Pair last = elements[--n];
elements[0] = last;
int idx = 0;
int child = 2*idx+1;
while (child <= n) {
Pair smaller = elements[child];
if (child < n) {
if (smaller.priority > elements[child+1].priority) {
smaller = elements[++child];
}
}
if (last.priority <= smaller.priority) { break; }
elements[idx] = smaller;
positions[smaller.key] = idx;
idx = child;
child = 2*idx+1;
}
elements[idx] = last;
positions[last.key] = idx;
return key;
}
void decreasePriority (int key, int newPriority) {
int size = n;
n = positions[key];
insert(key, newPriority);
n = size;
}
}
The essence o f the change is to be able to sto re the lo catio n in the array-based heap o f each key value in the
heap. This wo rks because the key values themselves are integers in the range [0, n). Every time a Pair o bject
p is inserted into elements[idx], there is a co rrespo nding positions[p.key] = idx to reco rd that fact.
The reaso n to maintain po sitio ns is evident in the final metho d being added to He ap, which decreases the
prio rity fo r a given key value fo und in the heap. In o rder to decrease prio rity o f a Pair, yo u need to reduce the
asso ciated prio rity value with the Pair. This metho d o nly wo rks if yo u are truly decreasing the prio rity
(increasing the prio rity wo uld break the Heap Pro perty). This metho d wo rks by reusing the inse rt metho d.
First it truncates the heap up to , but no t including the lo catio n where key is currently sto red in the heap (it do es
this by se t t ing n t o be t he ke y's lo cat io n in t he he ap). Then it invo ke s inse rt using t he o riginal key
value , but t he ne w prio rit y; as discussed earlier, this will reestablish the Heap Pro perty. Finally, since the
newPrio rity must be lo wer, yo u can e xpand t he size o f t he he ap back t o it s o riginal size and still have
a wo rking heap.
No w yo u're ready to write a revised Prim's Algo rithm that uses the heap data structure as a prio rity queue.
In the m st package, create a He apMST class as sho wn:
CODE TO TYPE: HeapMST class
package mst;
public class HeapMST {
PriorityComparator comp = new PriorityComparator();
static Edge[] compute(int [][] graph) {
int n = graph.length;
Edge[] mst = new Edge[n-1];
boolean inQueue[] = new boolean [n];
Heap heap = new Heap(n);
int[] priorities = new int[n];
heap.insert(0,0);
for (int i = 1; i < n; i++) {
priorities[i] = Integer.MAX_VALUE;
heap.insert(i, priorities[i]);
inQueue[i] = true;
mst[i-1] = new Edge(i, -1);
}
while (!heap.isEmpty()) {
int u = heap.smallest();
inQueue[u] = false;
for (int v = 0; v < n; v++) {
int weight = graph[u][v];
if (weight > 0 && inQueue[v]) {
if (weight < priorities[v]) {
mst[v-1].end = u;
priorities[v] = weight;
heap.decreasePriority(v, weight);
}
}
}
}
return mst;
}
}
The structure o f this co de is pro bably familiar to yo u fro m the MST class. Let's take a clo ser lo o k:
OBSERVE: Initializing fo r Prim's Algo rithm
static Edge[] compute(int [][] graph) {
int n = graph.length;
Edge[] mst = new Edge[n-1];
boolean inQueue[] = new boolean [n];
BinaryHeap heap = new BinaryHeap(n);
int[] priorities = new int[n];
heap.insert(0,0);
for (int i = 1; i < n; i++) {
priorities[i] = Integer.MAX_VALUE;
heap.insert(i, priorities[i]);
inQueue[i] = true;
mst[i-1] = new Edge(i, -1);
}
This implementatio n uses two additio nal arrays: prio rit ie s sto res the current best distance fro m any vertex
in set S to the vertices remaining in T. inQue ue determines whether a given vertex is currently in the prio rity
queue. Initially the heap is co nstructed with the de signat e d st art ve rt e x 0 be ing inse rt e d wit h gre at e st
im po rt ance (a prio rit y o f 0 ), while the o ther n-1 vertices are inserted with least impo rtance (maximum
prio rity). inQue ue is set to true fo r the n-1 vertices in the prio rity queue. The m st array will sto re the
co mputed edges fo r each vertex. Currently their end values are -1 to declare that they have yet to be
co mputed.
The real lo gic o ccurs in the while lo o p:
OBSERVE: Prim's Algo rithm Implementatio n
while (!heap.isEmpty()) {
int u = heap.smallest();
inQueue[u] = false;
for (int v = 0; v < n; v++) {
int weight = graph[u][v];
if (weight > 0 && inQueue[v]) {
if (weight < priorities[v]) {
mst[v-1].end = u;
priorities[v] = weight;
heap.decreasePriority(v, weight);
}
}
}
}
As lo ng as the heap is no t empty, it re t rie ve s t he ve rt e x u wit h t he sm alle st dist ance t o any ve rt e x in
se t S. No w that this vertex is o ut o f the queue, inQue ue [u] is se t t o f alse . The inner f o r lo o p still must
iterate o ver all the o ther n vertices to find if there is an edge (u,v) with a distance that's sho rter than previo usly
reco rded. The prio rit ie s array lets the co de determine this quickly. If so , the prio rity is adjusted in the
prio rit ie s array and the e le m e nt 's lo cat io n is adjust e d in t he prio rit y que ue using t he
de cre ase Prio rit y m e t ho d.
To evaluate the perfo rmance o f this algo rithm, assume there are n vertices and k edges in the graph. During
the initializatio n phase, each vertex is inserted into the prio rity queue fo r a to tal co st o f O(n log n). The
de cre ase Prio rit y metho d requires no less than O (log n) time. It can be called 2*k times at mo st since each
vertex is remo ved o nce fro m the prio rity queue and each edge in the graph is visited exactly twice. So , to tal
perfo rmance is O ((n+2*k) log n). If the graph is dense, k can be as high as n*(n-1)/2, so the wo rst case
perfo rmance is O(n 2 log n). If the graph is really sparse, then k is o n the o rder o f O(n) which results in O(n log
n) perfo rmance.
Evaluating Minimum Spanning T ree Implementations
Co mpare these two implementatio ns head to head o n the same graph.
In the m st package, create a Co m pariso n class as sho wn:
CODE TO TYPE: Co mpariso n class
package mst;
public class Comparison {
public static void main (String[] args) {
System.out.println("n\tHeapMST\t\tMST");
for (int n = 16; n <= 1024; n*= 2) {
int[][] graph = new int[n][n];
for (int i = 0; i < n-1; i++) {
for (int j = i+1; j < n; j++) {
int w = (int)(Math.random()*n);
graph[i][j] = w;
graph[j][i] = w;
}
}
System.gc();
long now = System.nanoTime();
Edge[] mst = MST.compute(graph);
long then = System.nanoTime();
System.gc();
Edge[] mst2 = HeapMST.compute(graph);
long last = System.nanoTime();
for (int i = 0; i < mst.length; i++) {
if ((mst[i].start != mst2[i].start) ||
(mst[i].end != mst[i].end)) {
System.err.println("ERROR");
System.exit(0);
}
}
float m = 1000000;
System.out.println(n + "\t" + (last-then)/m + "
\t" + (then-now)/m);
}
}
}
Save and run the co de:
INTERACTIVE SESSION: Co mpariso n Output
k
HeapMST
MST
16
4.720959
2.177451
32
3.310271
2.00772
64
3.199684
1.886284
128
4.528132
10.268524
256
5.046072
76.359505
512
5.648354
596.32855
1024 12.14046
4683.4565
Fo r small values o f n, the o riginal MST implementatio n o utperfo rms the He apMST implementatio n.
Ho wever, the true nature o f the heap implementatio n demo nstrates rapidly that it's incredible efficiency when
co mpared against the o bvio us co unterpart.
Lessons Learned
The Co llectio n Framewo rk o ffers a well-designed set o f classes that will be useful when yo u apply the
co mmo n data list, hash, and tree data structures. At the same time, the designers created a unifo rm interface
to all Co llectio n classes. In do ing so , they created metho ds that do no t behave as efficiently as o ther
specialized data structures. In particular, the Prio rityQueue Co llectio n class will pro vide the exact behavio r fo r
prio rity queues with elements that canno t change prio rity o nce they've been inserted into the queue. Prim's
Algo rithm demands this behavio r, so the default Prio rityQueue implementatio n will no t suffice. Yo u must
always read the do cumentatio n that acco mpanies the Co llectio n Framewo rk classes, because each class
pro vides info rmatio n abo ut the perfo rmance o f their impo rtant metho ds.
Yo u can impro ve perfo rmance by sto ring additio nal info rmatio n to reduce the number o f co mputatio ns
needed. The He apMST implementatio n is able to reduce perfo rmance time with o nly a mo dest investment in
sto rage. In yo ur o wn algo rithms, try to make this tradeo ff to achieve the same benefits.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Binary Tree Data Structure
Lesson Objectives
After co mpleting this lesso n, yo u will be able to :
describe the structure o f a Binary Search Tree (BST).
draw the BST after inserting a number o f elements in a specific o rder.
demo nstrate ho w to rebalance an AVL tree after inserting a no de.
Binary T ree Data Structure
A Binary Search Tree is a recursive data structure central to co mputer science. In earlier lesso ns we saw ho w Linked
Lists pro vide dynamic behavio r that impro ves o n co ntiguo us arrays. Ho wever, Linked Lists o nly pro vide O(n) behavio r
fo r determining whether an element is in the list.
Given an array o f so rted items, yo u can use Binary Array Search to determine efficiently whether the array co ntains a
given item in O(log n) time. This co de sho ws ho w to implement the algo rithm:
Create a new Java pro ject named BinaryT re e fo r this lesso n's wo rk, and assign it to the J ava6 _Le sso ns
wo rking set.
In the BinaryT re e pro ject's /src so urce fo lder, create a binary package.
In the binary package, create a BinaryArraySe arch class as sho wn:
CODE TO TYPE: BinaryArraySearch
package binary;
public class BinaryArraySearch {
public static void main (String[] args) {
int[] vals = new int [] { 2, 5, 8, 11, 15, 17 };
System.out.println("7 goes in position " + binarySearch (vals, 7));
System.out.println("2 found in position " + binarySearch (vals, 2));
}
public static int binarySearch(int[] A, int val) {
int low = 0;
int high = A.length-1;
while (low <= high) {
int mid = (low + high)/2;
if (val < A[mid]) {
high = mid-1;
} else if (val > A[mid]) {
low = mid + 1;
} else {
return mid;
}
}
return -(low + 1);
}
}
Save and run it:
OBSERVE: Sample executio n o f BinaryArraySearch
7 goes in position -3
2 found in position 0
Because 7 do es no t appear in the array, yo u can use the return value fro m binarySe arch to determine where in the
array 7 could be inserted to maintain the so rted o rder. When the return value is less than zero , negate it and subtract 1
to find the co rrect lo catio n to sto re the value in A. In this case, 7 sho uld be inserted at index lo catio n 2, which wo uld
place it between 5 and 8 as it sho uld be. The seco nd line o f o utput sho ws that the search is able to lo cate element 2 at
index lo catio n 0 .
Arrays are unable to delete and insert items efficiently while maintaining a specific o rdering o f elements. Ho wever,
Linked Lists can insert elements anywhere in the co llectio n, but then searching fo r a given item wil require O(n) time.
Binary Search Trees o ffer the impressive ability to maintain items in a structured o rder and o n average, it can suppo rt
adding, remo ving, and searching fo r items in O(log n) time.
In this lesso n, yo u will co nstruct a Binary Search Tree implementatio n fro m scratch. Thro ugh vario us perfo rmance
co de yo u'll see that the naive implementatio n can lead to wo rst-case perfo rmance o f O(n) fo r all key o peratio ns. At the
end o f the lesso n, yo u'll learn ho w to "balance" the tree to impro ve the average case perfo rmance.
Naive Binary T ree Implementation
A Binary Search Tree is a finite set o f no des where each no de sto res a typed value kno wn as the key fo r the
no de. A no n-empty BST co ntains a special root no de that is the ancesto r o f all o ther no des in the BST. Each
no de n in the BST refers to two binary search subtrees, left and right, and o beys the pro perty that if k is the key
fo r no de n, then all keys in left are <=k and all keys in right are >k. This pro perty is kno wn as the binary search
tree property. If subtrees left and right are null, the no de is called a leaf node.
Given a BST, the three primary o peratio ns are:
Add a new key.
Remo ve a key.
Determine if a key value exists.
There can be two o r mo re no des in the search tree with the same key value, but if yo u want to restrict the tree
to co nfo rm to Set-based semantics as defined in the Java Co llectio ns Framewo rk while ensuring that the
same implementatio n will wo rk, yo u need to prevent the insertio n o f duplicate keys. Fo r this lesso n, assume
that duplicate keys may exist in the BST.
In the /src so urce fo lder binary package, create a BinaryNo de class as sho wn:
CODE TO TYPE: BinaryNo de class
package binary;
public class BinaryNode<E extends Comparable<E>> {
final E key;
BinaryNode<E> left;
BinaryNode<E> right;
public BinaryNode(E k) {
this.key= k;
}
public int size() {
return 1 + size(left) + size(right);
}
int size(BinaryNode<E> n) {
if (n == null) { return 0; }
return n.size();
}
}
A BinaryNo de <E> class represents a no de in the BST with a co rrespo nding ke y value. BinaryNo de is a
generic class with the parameter, E, that determines the type o f the ke y attribute. The o nly restrictio n is that the
type o f the key must implement Co m parable , o therwise there will be no way to o rder the key values.
BinaryNo de defines left and right attributes to refer to the left and right subtrees, respectively. The size ()
metho d co unts the no des in a BST ro o ted at a given no de. Because a BST is a recursive data structure, the
implementatio n o f size () is also recursive. Thro ugho ut this lesso n, yo u see ho w to apply recursio n to
implement the required BST o peratio ns. The size (n) 'll helper metho d allo ws size () to be written in its
simplest fo rm. So , the size o f a BST ro o ted at a given no de n is 1 plus the respective sizes o f the left and right
subtrees o r n.
Because the BST is co mpo sed o f BinaryNo de o bjects representing keys in the BST, we need to write an
add(E) metho d that inserts a key into the BST ro o ted at a given no de. Add these metho ds to the end o f the
BinaryNo de class:
CODE TO TYPE: Mo dificatio ns to BinaryNo de
void add (E k) {
int rc = k.compareTo(key);
if (rc <= 0) {
left = add(left, k);
} else {
right = add(right, k);
}
}
BinaryNode<E> add(BinaryNode<E> parent, E k) {
if (parent == null) {
return new BinaryNode<E>(k);
}
parent.add(k);
return parent;
}
The binary search tree property states that all keys in the left subtree o f a no de are less than o r equal to the
no de's key, and all keys in the right subtree o f a no de are greater than the no de's key. Fo r this discussio n,
assume that add(k) is invo ked o n no de n where k is less than o r equal to n's key. There are two cases to
co nsider:
The no de n has no left subtree.
The no de n has a left subtree.
If there is no left subtree then a new no de co ntaining this key is created to be the left subtree o f n. This is the
case in add(pare nt ,k) when pare nt is null. If, ho wever, the left subtree do es exist, then add(pare nt ,k)
requests to add k recursively to that subtree. add(pare nt ,k) either returns the new no de which was created o r
the existing no de that no w has a descendant no de representing the newly added key. These two functio ns
present ano ther example o f do uble recursio n, where each functio n calls the o ther repeatedly until the
co mputatio n terminates. No te that the structure o f these two metho ds is similar to the size () metho d
presented earlier.
No w yo u can create a BinaryT re e class to take advantage o f this add capability.
In the binary package, create a BinaryT re e class as sho wn:
CODE TO TYPE: BinaryTree class
package binary;
public class BinaryTree<E extends Comparable<E>> {
BinaryNode<E> root = null;
public int size() {
if (root == null) { return 0; }
return root.size();
}
public void add (E k) {
if (root == null) {
root = new BinaryNode<E>(k);
return;
}
root = root.add(root,k);
}
}
Let's take a clo ser lo o k at a co uple o f things.
OBSERVE:
public int size() {
if (root == null) { return 0; }
return root.size();
}
public void add (E k) {
if (root == null) {
root = new BinaryNode<E>(k);
return;
}
root = root.add(root,k);
}
The BinaryNo de attribute ro o t represents the to p o f the BST; all no des in the BST are descendants o f ro o t .
Whe n ro o t == null, the BST is co nsidered to be empty.
The add(E k) co de demo nstrates an implementatio n style necessary fo r BinaryTree. If ro o t is null, all
metho ds have to handle the special case where the BST is empty. In this case, if ro o t is null, the ro o t o f
t he BST is se t t o a ne w BinaryNo de wit h t he give n ke y. Otherwise, t his ke y is adde d t o t he
subt re e ro o t e d at ro o t , using the add(pare nt ,k) metho d described earlier. Do ing so allo ws the ro o t to
be updated as needed.
Until yo u add a co nt ains(E k) metho d, there will be no easy way to validate that the abo ve metho ds wo rk.
Add these metho ds to the end o f BinaryT re e :
CODE TO TYPE: Mo dificatio ns to BinaryTree
public boolean contains (E k) {
return contains(root, k);
}
boolean contains (BinaryNode<E> parent, E k) {
if (parent == null) { return false; }
int rc = k.compareTo(parent.key);
if (rc == 0) {
return true;
} else if (rc < 0) {
return contains(parent.left, k);
} else {
return contains(parent.right, k);
}
}
public int height () {
if (root == null) { return 0; }
return height(root);
}
int height (BinaryNode<E> n) {
if (n == null) { return 0; }
return 1 + Math.max( height(n.left), height(n.right));
}
This metho d is placed in the BinaryT re e class, rather than BinaryNo de, because the no tio n o f "co ntainment"
is a pro perty o f the BST, no t an individual no de. Also , do ing so allo ws yo u to avo id co nstantly checking to find
o ut whether the le f t o r right subtree is empty. Specifically, co nt ains(pare nt ,k) returns f alse when pare nt
no de is null. This metho d recursively calls itself o n either the left subtree o r the right subtree if the no de's key
do esn't match the target key. Naturally, o nce a match is fo und, t rue is returned.
Yo u can demo nstrate pro per functio ning o f BinaryT re e with this JUnit test case:
Create a t e st so urce fo lder if it do esn't already exist.
Create a binary package.
in the /t e st so urce fo lder binary package, create a J Unit T e st Case named T e st BinaryT re e as
sho wn:
Note
Yo u may be pro mpted to cho o se JUnit 3 o r JUnit 4. If yo u're do n't kno w which to cho o se, go
with JUnit 3. Either is o kay, but the resulting file may lo o k slightly different fro m the o ne sho wn
here.
CODE TO TYPE: TestBinaryTree
package binary;
import java.util.*;
import junit.framework.TestCase;
public class TestBinaryTree extends TestCase {
public void testAdditions() {
int numToAdd = 100;
ArrayList<Integer> vals = new ArrayList<Integer>();
for (int i = 1; i < numToAdd; i += 2) {
vals.add(i);
}
Collections.shuffle(vals);
Integer[] add = vals.toArray(new Integer[]{});
BinaryTree<Integer> bst = new BinaryTree<Integer>();
for (int i : add) {
bst.add(i);
}
assertEquals (numToAdd/2, bst.size());
for (int i = 1; i < numToAdd; i++) {
if (i % 2 == 1) {
assertTrue (bst.contains(i));
} else {
assertFalse (bst.contains(i));
}
}
}
}
Launch this JUnit test case; it validates that the BST o nly co ntains the o dd numbers fro m 1 to 10 0 .
Evaluating Binary T ree Implementation
To determine the efficiency o f BinaryT re e , yo u need to identify the wo rst case and average case executio n o f
its metho ds. Yo u might be able to identify the wo rst case behavio r, that is, when keys are inserted into a BST
in increasing so rted o rder. Fo r example, co nsider adding the numbers fro m 1 to 10 into a BST. Each newly
inserted key beco mes the right-mo st no de in the BST. In fact, the structure mo re clo sely resembles a linked
list than a tree because no ne o f the no des in the BST have a left subtree.
To demo nstrate the perfo rmance o f the add(k) metho d in BinaryT re e , write this class:
In the BinaryT re e pro ject, create a /pe rf o rm ance so urce fo lder.
In the /pe rf o rm ance so urce fo lder, create a binary package.
In the binary package, create an Evaluat e class as sho wn:
CODE TO TYPE: Evaluate class
package binary;
import java.util.*;
public class Evaluate {
static int numTrials = 100;
public static void main(String[] args) {
System.out.println("N\tShuffled Stats & Time\tOrdered Stats & Time");
System.out.println("----\t---------------------\t--------------------");
for (int n = 128; n <= 65536; n *= 2) {
int totalShuffledHeight = 0;
int totalOrderedHeight = 0;
long totalShuffled = 0;
long totalOrdered = 0;
int min = n;
int max = 0;
for (int t = 0; t < numTrials; t++) {
ArrayList<Integer> vals = new ArrayList<Integer>();
for (int i = 1; i < n; i++) {
vals.add(i);
}
Integer[] ordered = vals.toArray(new Integer[]{});
Collections.shuffle(vals);
Integer[] shuffled = vals.toArray(new Integer[]{});
BinaryTree<Integer> bst = new BinaryTree<Integer>();
long now = System.nanoTime();
for (int i : shuffled) {
bst.add(i);
}
totalShuffled += (System.nanoTime() - now);
int h = bst.height();
if (h < min) { min = h; }
if (h > max) { max = h; }
totalShuffledHeight += h;
bst = new BinaryTree<Integer>();
now = System.nanoTime();
for (int i : ordered) {
bst.add(i);
}
totalOrdered += (System.nanoTime() - now);
totalOrderedHeight += bst.height();
}
System.out.println(n + "\t[" + min + "-" + max + ", avg:" +
totalShuffledHeight/numTrials + "] " +
totalShuffled/numTrials + "\t[avg:" + totalOrderedHeight/numTrials + "
] " +
totalOrdered/numTrials);
}
}
}
Evaluat e co nducts 10 0 rando m trials o f creating BSTs with n no des, ranging fro m n=128 to n=6 5536 ; n keys
(1 .. n) are inserted. Let's take a clo ser lo o k at this co de:
OBSERVE: Creating a BST fro m keys inserted in rando m o rder
ArrayList<Integer> vals = new ArrayList<Integer>();
for (int i = 1; i < n; i++) {
vals.add(i);
}
Integer[] ordered = vals.toArray(new Integer[]{});
Collections.shuffle(vals);
Integer[] shuffled = vals.toArray(new Integer[]{});
BinaryTree<Integer> bst = new BinaryTree<Integer>();
long now = System.nanoTime();
for (int i : shuffled) {
bst.add(i);
}
totalShuffled += (System.nanoTime() - now);
int h = bst.height();
if (h < min) { min = h; }
if (h > max) { max = h; }
totalShuffledHeight += h;
Two arrays o f N keys are created; o rde re d co ntains the keys in o rder while shuf f le d is created by using the
Co lle ct io ns.shuf f le metho d to distribute the keys rando mly. The co de co nducts 10 0 trials and reco rds the
to tal time (in nano seco nds) required to add all keys to the BST. The co de estimates the average co st o f
adding a rando m key to the BST by averaging the to tal time.
In additio n, the co de maintains statistics o n the height o f the BSTs generated during this pro cess. Fo r the
rando m BSTs, it reco rds the m in and m ax heights o f the BSTs and co mputes the to tal height so it can repo rt
o n the average height fo r n keys. Similar co de reco rds statistics fo r the BST generated fro m the o rdered
insertio n o f keys. Yo u want to kno w the average height o f the BST because that will determine the
perfo rmance o f the co nt ains metho d.
Here is sample o utput o f Evaluat e ; yo ur mileage may vary:
OBSERVE: evaluating BinaryTree Implementatio n
N Shuffled Stats & Time Ordered Stats & Time
---- --------------------- -------------------128 [11-18, avg:14] 26110 [avg:127] 90122
256 [14-23, avg:16] 35776 [avg:255] 326004
512 [16-25, avg:19] 83298 [avg:511] 1485010
1024 [19-26, avg:22] 168682 [avg:1023] 6234518
2048 [21-31, avg:25] 392221 [avg:2047] 26031424
Exception in thread "main" java.lang.StackOverflowError
at binary.BinaryNode.add(BinaryNode.java:24)
at binary.BinaryNode.add(BinaryNode.java:37)
at binary.BinaryNode.add(BinaryNode.java:28)
The minimum height o f a BST with n no des is log(n). The average height o f the BST created fro m the shuffled
values is abo ut twice the minimum. Also , the average time (in nano seco nds) to search fo r all n items gro ws
pro po rtio nally with n. Fo r example, when n gro ws fro m 128 to 512 (a fo ur-fo ld increase) the time to search fo r
all n numbers takes 3.19 times as lo ng. This is much different in the BST co nstructed fro m the o rdered keys.
The height o f the BST is n-1 (which means that it's really a linked list). Also , when n gro ws fro m 128 to 512, it
takes mo re than 16 times as lo ng to co mplete all n searches.
Typically yo u do no t have advance warning o f the o rder o f the elements being added into the BST, so yo u
need so me way to avo id po o r perfo rmance due to the elements being clo se to so rted when they were added
into the BST. In additio n, when the BST degenerates to a Linked List (because items are inserted in so rted
o rder) the recursive add metho d can cause a St ackOve rf lo wErro r, as sho wn abo ve.
Rebalancing Binary T rees
The smallest height fo r a tree with n elements is O(log n), which results in a perfectly balanced tree with the left
subtree o f the ro o t co ntaining ro ughly the same number o f values as the right subtree o f the ro o t. This
balanced pro perty sho uld apply recursively to all no des in the tree, no t just the ro o t. Yo u co uld try to rebuild
the entire tree after each insertio n to make sure that each no de is balanced, but that wo uld require way to o
much wo rk. Instead, find so me incremental strategy that adjusts the structure o f the tree o nly when it beco mes
unbalanced.
An AVL tree (named after its invento rs, Adelso n-Velskii, and Landis) is a self-balancing BST first described in
19 6 2. In the Co llectio ns Framewo rk, the T re e Map class is implemented using Red-Black trees, which are
ano ther fo rm o f self-balancing binary tree. After co mpleting this lesso n, yo u'll be able to co mpare these two
appro aches to determine which pro vides the best perfo rmance.
Let's define the co ncept o f height with AVL no des. The height o f a leaf no de is 0 since it has no children.
Recursively define the height o f an AVL no de to be 1 greater than the maximum o f the height values o f its 2
children no des (if at least 1 exists). To co mplete this definitio n, co nsider the height o f a no n-existent child
no de to be -1. The height difference fo r a no de is defined as height(left) - height(right), that is, the height o f the
left subtree minus the height o f the right subtree. An AVL must enfo rce the AVL Property in every no de,
namely, that the height difference fo r any no de is either -1, 0 o r 1.
In the /src so urce fo lder, create an avl package.
In the avl package, create an AVLBinaryNo de class as sho wn:
CODE TO TYPE: AVLBinaryNo de class
package avl;
public class AVLBinaryNode<E extends Comparable<E>> {
E key;
int height;
AVLBinaryNode<E> left;
AVLBinaryNode<E> right;
public AVLBinaryNode(E k) {
height = 0;
key = k;
}
void computeHeight (AVLBinaryNode<E> n) {
int height = -1;
if (n.left != null) {
height = Math.max(height, n.left.height);
}
if (n.right != null) {
height = Math.max(height, n.right.height);
}
n.height = height + 1;
}
int heightDifference(AVLBinaryNode<E> n) {
if (n == null) { return 0; }
int leftTarget = 0;
if (n.left != null) {
leftTarget = 1 + n.left.height;
}
int rightTarget = 0;
if (n.right != null) {
rightTarget = 1 + n.right.height;
}
return leftTarget - rightTarget;
}
}
To be as efficient as po ssible, each AVLBinaryNo de sto res its co mputed height in additio n to the expected
attributes o f key, left, and right. That is, rather than dynamically co mputing the height o f a no de when
requested, yo u o nly perfo rm this co mputatio n when a no de is added to the AVL tree. Finally, the
he ight Dif f e re nce metho d co mputes the height difference fo r a given no de, n. As with a regular binary tree,
yo u need to define an AVLBinaryT re e class.
In the avl package, create an AVLBinaryT re e class as sho wn:
CODE TO TYPE: AVLBinaryTree
package avl;
public class AVLBinaryTree<E extends Comparable<E>> {
AVLBinaryNode<E> root = null;
public void add (E k) {
if (root == null) {
root = new AVLBinaryNode<E>(k);
return;
}
root = root.add(root, k);
}
public boolean contains (E k) {
return contains(root, k);
}
boolean contains (AVLBinaryNode<E> parent, E k) {
if (parent == null) { return false; }
int rc = k.compareTo(parent.key);
if (rc == 0) {
return true;
} else if (rc < 0) {
return contains(parent.left, k);
} else {
return contains(parent.right, k);
}
}
}
This co de is nearly identical to its BinaryT re e co unterpart. The co de wo n't co mpile until yo u co mplete the
add metho d in AVLBinaryNo de —be careful with this metho d. It is po ssible after just three additio ns to have
the ro o t no de o f a binary tree vio late the AVL Pro perty. Co nsider an AVL tree with just two no des, co nstructed
by adding 50 and then 30 :
To demo nstrate that this tree suppo rts the AVL pro perty, yo u must co mpare the heights o f the children o f the
ro o t no de (which sto res the value 50 ). Ho wever, there is no right subtree fo r this no de. In this situatio n, the
height o f an empty child subtree is -1. The height difference fo r the ro o t no de is 0 - (-1) o r +1, which satisfies
the AVL pro perty.
No w insert the value 10 into the tree, which results in this structure:
First co nfirm that this binary tree is a BST by making sure that the value fo r each no de is greater than o r equal
to the value o f its left child, and smaller than the value o f its right child. The AVL pro perty is maintained by all
no des except for the root. The height difference fo r the ro o t is +2 because the left height is +1 while the
(missing) right child's height is -1. The difference vio lates the AVL Pro perty.
No w, co nsider this similar tree, which suppo rts the AVL Pro perty:
When the tree is ro o ted by the no de fo r 30 , each o f its subtrees is balanced.
After adding the value 10 to the o riginal AVL tree, it is po ssible to detect that o ne o f its ancesto r no des (the
o ne representing 50 ) is unbalanced. Yo u can recreate the balanced tree abo ve by perfo rming a rotate
o peratio n. Imagine "grabbing" the 30 no de in the o riginal tree and ro tating the tree to the right (o r clo ckwise),
pivo ting aro und the 30 no de to make 30 the ro o t, thereby creating the balanced tree abo ve. In do ing so , o nly
the height o f the 50 no de has changed (dro pping fro m 2 to 0 ) and the AVL Pro perty is resto red.
This o nly wo rks because the no de 30 in the o riginal tree had no right child. So , what if this tree had lo ts o f
o ther no des, each o f which was perfectly balanced and satisfied the AVL Pro perty? In the image belo w, each
o f the shaded triangles represents a po tential subtree o f the o riginal tree; each is labeled by its po sitio n, so
30 R is the subtree representing the right subtree o f no de 30 . The situatio n o n the left o ccurs immediately after
the 10 value is inserted into the tree. The ro o t is the o nly no de that do esn't suppo rt the AVL Pro perty. The
vario us heights in the tree are co mputed assuming that the new no de 10 has so me height k.
No w when yo u Rotate Right, yo u can re-attach the entire subtree 30 R so it beco mes the left subtree fo r no de
5 0 . This is po ssible because all o f these values are clearly smaller than 50 since the o riginal tree was a
Binary Search Tree. The resulting tree is balanced and all no des satisfy the AVL Pro perty.
Note
It is po ssible that the subtree 30 R had a height o f k in the tree o n the left. In this case, the new
no de 5 0 wo uld have a co mputed height o f k+1 and the ro o t no de 30 wo uld have a co mputed
height o f k+2. Ho wever, that the AVL Pro perty wo uld be pro perly maintained even in this case.
Add this co de to the end o f the AVLBinaryNo de class:
CODE TO TYPE: Mo dificatio ns to AVLBinaryNo de
AVLBinaryNode<E> rotateRight () {
AVLBinaryNode<E> newRoot = left;
AVLBinaryNode<E> grandson = newRoot.right;
left = grandson;
newRoot.right = this;
computeHeight(this);
return newRoot;
}
This co de is best described in the co ntext o f the specific example presented abo ve. Yo u invo ke ro t at e Right
o n the unbalanced no de, 5 0 , which is the this reference in the abo ve co de. newRoot is set to the 30 no de
while grandson is the subtree labeled 30 R. The co de le f t = grandso n sets the left child o f 5 0 to be the
subtree 30 R. The co de ne wRo o t .right = t his makes 5 0 the right child o f 30 . Once this manipulatio n is
co mplete, the height fo r the 5 0 no des is reco mputed, but the o riginal height o f the 10 no de is unaffected.
Finally, the new ro o t no de o f this subtree, 30 , is returned. Observe that its height has not yet been
reco mputed; that will be the respo nsibility o f the metho d that calls ro t at e Right .
Yo u've seen ho w to Rotate Right to rebalance an AVL tree in the left-left case, so named because the new
value being added (10 in this case) was added to left-child of the left-child o f the (no w-unbalanced) no de 5 0 .
Yes, that wo rd repetitio n is necessary! As yo u can imagine, there is also a Rotate Left o peratio n which can be
used to rebalance a tree that is unbalanced in the right-right case sho wn belo w, so named because the new
value being added (6 0 ) was added to the right-child of the right-child o f the (no w-unbalanced) no de 20 :
In similar fashio n, yo u can perfo rm this Rotate Left even when these no des have subtrees attached to them.
Add this co de to the end o f the AVLBinaryNo de class:
CODE TO TYPE: Mo dificatio ns to AVLBinaryNo de
AVLBinaryNode<E> rotateLeft () {
AVLBinaryNode<E> newRoot = this.right;
AVLBinaryNode<E> grandson = newRoot.left;
this.right = grandson;
newRoot.left = this;
computeHeight(this);
return newRoot;
}
There are two additio nal cases that have to be handled. Let's co nsider the left-right case, which suggests that
the newly added no de is added to the right child o f the left child o f the unbalanced no de. To create this AVL
tree, add 5 0 then 10 to an empty tree. Finally, add 30 :
Once again, the ro o t no de is unbalanced, ho wever this time yo u can't just Rotate Right to remedy the situatio n
because the "middle" no de, 10 canno t beco me the ro o t o f the tree because its value is smaller than bo th o f
the o ther two values. Fo rtunately, yo u can reso lve the issue by first co mpleting a Rotate Left o n the child no de
10 ; then yo u'll be able to perfo rm the Rotate Right step as described earlier. The image belo w demo nstrates
this situatio n o n a larger tree. The After the Rotate Left o peratio n, the tree is identical to the earlier tree o n
which the Rotate Right o peratio n was described.
This co de handles this left-right case. Add the fo llo wing changes to the end o f the AVLBinaryNo de class:
CODE TO TYPE: Mo dificatio ns to AVLBinaryNo de
AVLBinaryNode<E> leftRightRotation () {
AVLBinaryNode<E> child = left;
AVLBinaryNode<E> newRoot = child.right;
AVLBinaryNode<E> grand1 = newRoot.left;
AVLBinaryNode<E> grand2 = newRoot.right;
child.right = grand1;
left = grand2;
newRoot.left = child;
newRoot.right = this;
computeHeight(child);
computeHeight(this);
return newRoot;
}
In reference to the earlier left-right diagram, child is the 10 no de, newRoot is the 30 no de, grand1 is the 30 L
subtree and grand2 is the 30 R subtree. The fo ur sequential o peratio ns must take place in exactly the o rder as
sho wn in o rder to co mplete the Rotate Left then Rotate Right o peratio ns efficiently. Once that's do ne, the
heights o f child and this are reco mputed befo re newRoot is returned as the new ro o t o f this subtree. Once
again, it is the respo nsibility o f the calling metho d to reco mpute the height fo r newRoot.
In exactly the same way, the right-left case (no t sho wn here) wo uld first Rotate Right befo re co mpleting the
restructuring with a Rotate Left. The co de fo r this case is sho wn belo w; add this metho d to the end o f the
AVLBinaryNo de class:
CODE TO TYPE: Mo dificatio ns to AVLBinaryNo de
AVLBinaryNode<E> rightLeftRotation () {
AVLBinaryNode<E> child = right;
AVLBinaryNode<E> newRoot = child.left;
AVLBinaryNode<E> grand1 = newRoot.left;
AVLBinaryNode<E> grand2 = newRoot.right;
child.left = grand2;
right = grand1;
newRoot.left = this;
newRoot.right = child;
computeHeight(child);
computeHeight(this);
return newRoot;
}
All that remains no w is to write the appro priate add metho d in AVLBinaryNo de . Add this co de to the end o f
the AVLBinaryNo de class:
CODE TO TYPE: Mo dificatio ns to AVLBinaryNo de
void add (E k) {
int rc = k.compareTo(key);
if (rc <= 0) {
left = add(left, k);
} else {
right = add(right, k);
}
}
AVLBinaryNode<E> add(AVLBinaryNode<E> parent, E k) {
if (parent == null) {
return new AVLBinaryNode<E>(k);
}
parent = parent.add(k);
return parent;
}
To satisfy the binary search tree pro perty o f an AVL tree, the add metho d inserts the new value into either the
left o r right subtree. The abo ve co de is identical to the BinaryNo de we wro te earlier in this lesso n. Ho wever,
no w yo u must mo dify this co de to maintain the AVL Pro perty. In each o f the ro tatio ns described earlier,
o bserve ho w it was po ssible fo r the ro o t o f the tree to change during a ro tatio n. Fo r this reaso n, the add
metho d must change to return a po tentially new no de which beco mes the new ro o t o f a subtree. Mo dify
add(E k) as sho wn:
CODE TO TYPE: Mo dificatio ns to add(E k)
AVLBinaryNode<E>void add (E k) {
int rc = k.compareTo(key);
AVLBinaryNode<E> newRoot = this;
if (rc <= 0) {
left = add(left, k);
} else {
right = add(right, k);
}
computeHeight(newRoot);
return (newRoot);
}
By default, the new ro o t o f the subtree to which k is added will be this, which is the existing ro o t o f the subtree.
The abo ve co de prepares fo r the ro tatio ns by allo wing a new ro o t to be returned when a key is added to a
subtree. The height o f the new ro o t is co mputed prio r to the end o f this metho d; do ing so co mpletes each o f
the fo ur ro tatio n metho ds where it was made clear that the invo king metho d o f the ro tatio n wo uld be
respo nsible fo r co mputing the height o f newRoot.
After each invo catio n o f add(pare nt , ke y) it is po ssible that this has beco me unbalanced. Insert this new
co de which handles all fo ur cases:
CODE TO TYPE: Mo dificatio ns to AVLBinaryNo de
AVLBinaryNode<E> add (E k) {
int rc = k.compareTo(key);
AVLBinaryNode<E> newRoot = this;
if (rc <= 0) {
left = add(left, k);
if (heightDifference(this) == 2) {
if (k.compareTo(left.key) <= 0) {
newRoot = rotateRight();
} else {
newRoot = leftRightRotation();
}
}
} else {
right = add(right, k);
if (heightDifference(this) == -2) {
if (k.compareTo(right.key) > 0) {
newRoot = rotateLeft();
} else {
newRoot = rightLeftRotation();
}
}
}
computeHeight(newRoot);
return newRoot;
}
When yo u add a key to the left subtree fo r a no de, it's po ssible that the height difference fo r the parent no de
(that is, this) no lo nger ho no rs the AVL Pro perty, but this o nly happens o nce the difference is 2 (because it is
acceptable fo r this value to be -1, 0 , o r 1). When the height difference fo r the parent this no de is 2, a ro tatio n
must o ccur to bring this no de back into balance. Yo u need to determine whether a single Rotate Right is
needed (the left-left case) o r a Rotate Left and Rotate Right (the left-right case). Fo rtunately a simple co nditio n
can determine which is appro priate by co mparing the newly added key k with the key o f the left child. The leftleft is appro priate if k <= le f t .ke y, o therwise use the left-right case. In bo th cases, the ro tatio n invo catio n
returns the newRoot o f the subtree. Similar co de is used to handle unbalanced no des when inserting k to the
right subtree.
Similarly, when the height o f the right subtree fo r a no de exceeds the height o f the left subtree, the co mputed
height difference is negative; when the difference is -2, the add metho d determines whether the unbalanced
no de is a right-right o r right-left case by co mparing the newly added key with right.key.
Yo u are no w ready to try so me head-to -head co mpariso ns with the existing T re e Se t implementatio ns in the
Java Co llectio ns Framewo rk.
Using Collections T reeSet
It is co mmo nly accepted that AVL trees are easier to implement than the red-black self-balancing binary trees
implemented by TreeSet, altho ugh red-black o ffers better perfo rmance. Let's investigate and find o ut if this is
true. The perfo rmance co de belo w evaluates all three types o f binary trees—BinaryT re e , AVLBinaryT re e ,
and T re e Se t —against a single benchmark.
In the /pe rf o rm ance so urce fo lder, create an avl package.
In the avl package, create an Evaluat e class as sho wn:
CODE TO TYPE: Evaluate class
package avl;
import java.text.*;
import java.util.*;
import binary.*;
public class Evaluate {
static int numTrials = 100;
static double m = 1000000;
static NumberFormat nf;
public static void main(String[] args) {
nf = NumberFormat.getInstance();
nf.setMinimumFractionDigits(3);
System.out.println("N
\tB_Time\tB_Find\tA_Time\tA_Find\tT_Time\tT_Find");
System.out.println("----\t------\t------\t------\t------\t------\t------");
for (int n = 128; n <= 65536; n *= 2) {
long totalBSTCreate = 0;
long totalBSTFind = 0;
long totalAVLCreate = 0;
long totalAVLFind = 0;
long totalTreeSetCreate = 0;
long totalTreeSetFind = 0;
for (int t = 0; t < numTrials; t++) {
ArrayList<Integer> vals = new ArrayList<Integer>();
for (int i = 0; i < 2*n; i+=2) {
vals.add(i);
}
Collections.shuffle(vals);
Integer[] shuffled = vals.toArray(new Integer[]{});
System.gc();
AVLBinaryTree<Integer> avlTree = new AVLBinaryTree<Integer>();
long now = System.nanoTime();
for (int i : shuffled) {
avlTree.add(i);
}
totalAVLCreate += (System.nanoTime() - now);
System.gc();
now = System.nanoTime();
for (int i = 0; i < 2*n; i++) {
if (avlTree.contains(i) != (i%2 == 0)) {
System.err.println("Search fails for BST");
}
}
totalAVLFind += (System.nanoTime()-now);
System.gc();
BinaryTree<Integer> btree = new BinaryTree<Integer>();
now = System.nanoTime();
for (int i : shuffled) {
btree.add(i);
}
totalBSTCreate += (System.nanoTime() - now);
System.gc();
now = System.nanoTime();
for (int i = 0; i < 2*n; i++) {
if (btree.contains(i) != (i%2 == 0)) {
System.err.println("Search fails for BST");
}
}
totalBSTFind += (System.nanoTime()-now);
System.gc();
TreeSet<Integer> tree = new TreeSet<Integer>();
now = System.nanoTime();
for (int i : shuffled) {
tree.add(i);
}
totalTreeSetCreate += (System.nanoTime() - now);
System.gc();
now = System.nanoTime();
for (int i = 0; i < 2*n; i++) {
if (tree.contains(i) != (i%2 == 0)) {
System.err.println("Search fails for BST");
}
}
totalTreeSetFind += (System.nanoTime()-now);
}
System.out.println(n + "\t" +
nf.format(totalBSTCreate/numTrials/m) + "\t" +
nf.format(totalBSTFind/numTrials/m) + "\t" +
nf.format(totalAVLCreate/numTrials/m) + "\t" +
nf.format(totalAVLFind/numTrials/m) + "\t" +
nf.format(totalTreeSetCreate/numTrials/m) + "\t" +
nf.format(totalTreeSetFind/numTrials/m));
}
}
}
Save and run it.
OBSERVE: Output fro m Evaluate
N
B_Time B_Find A_Time A_Find T_Time T_Find
---- ------ ------ ------ ------ ------ -----128 0.031 0.043 0.054 0.042 0.045 0.036
256 0.044 0.063 0.051 0.050 0.037 0.046
512 0.091 0.129 0.105 0.106 0.076 0.095
1024 0.196 0.276 0.234 0.220 0.166 0.198
2048 0.440 0.616 0.514 0.480 0.359 0.430
4096 0.977 1.332 1.096 1.001 0.773 0.903
8192 2.173 2.994 2.417 2.159 1.682 1.979
16384 4.994 6.526 5.534 4.630 3.837 4.262
32768 11.769 14.662 12.753 10.062 8.844 9.381
65536 26.768 32.584 29.143 22.852 20.299 20.618
To reaffirm the need fo r rebalancing, co mment o ut the Co lle ct io ns.shuf f le (vals) line o f co de in Evaluat e
and reexecute. All to o so o n, the BinaryT re e implementatio n causes a stack o verflo w erro r, while the
AVLBinaryT re e and T re e Se t bo th co ntinue to functio n just fine.
OBSERVE: Reexecute co mpariso n when inserting elements in o rder
N
B_Time B_Find A_Time A_Find T_Time T_Find
---- ------ ------ ------ ------ ------ -----128 0.105 0.155 0.040 0.039 0.037 0.034
256 0.408 0.580 0.041 0.051 0.034 0.046
512 1.732 2.326 0.085 0.098 0.069 0.122
1024 6.858 9.623 0.191 0.206 0.160 0.198
2048 26.588 36.689 0.380 0.429 0.310 0.400
Exception in thread "main" java.lang.StackOverflowError
at binary.BinaryNode.add(BinaryNode.java:24)
...
If yo u plo t the searching results perfo rmance, be sure to do so using a lo garithmic scale o n the y-axis.
As the abo ve graph sho ws, the perfo rmance graph fo r all three binary tree structures is a straight line, with
naive BST perfo rming the wo rst. T re e Se t perfo rms best, but AVL trees are no t that far behind. This graph
pro vides further evidence that the searching behavio r is O(n log n).
Lessons Learned
So no w yo u kno w:
Co nstructing an AVL Binary Tree co nsumes the mo st time o f the three co nstructio ns; it's up to 50 %
slo wer than the T re e Se t implementatio n. This happens because AVL trees must co ntinually
rebalance to maintain the AVL Pro perty which is a stro ng co nstraint o n the structure o f the tree. By
co ntrast, the T re e Se t self-balancing strategy o nly ensures that the path fro m the ro o t to the
farthest leaf is no mo re than twice as long as the path fro m the ro o t to the nearest leaf. This relaxed,
self-balancing strategy turns o ut to be mo re efficient.
The T re e Se t co de pro vides the fastest average search times, altho ugh AVL trees are no t that
much slo wer.
The naive Binary Tree implementatio n perfo rms well o n rando mized data, which might mistakenly
lead yo u to use these BSTs as is fo r yo ur pro jects. Be warned that, when the data exhibits any
regularity, the co nstructio n and search times will rapidly degenerate into O(n) behavio r.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Multidimensional Algorithms
Lesson Objectives
When yo u finish this lesso n, yo u will be able to :
describe the structure o f a k-dimensio nal tree.
implement a preo rder traversal o n any recursive search tree.
co nstruct a k-dtree manually after the insertio n o f a number o f po ints.
A Data Structure For Multidimensional Algorithms
In an earlier lesso n, yo u saw ho w to use Binary Array Search to determine efficiently whether a so rted array co ntains a
given item in O(log n) time. Ho wever, if yo u have a co llectio n o f n Cartesian po ints (x,y), there is no co mparato r
functio n that completely orders the po ints within a o ne-dimensio nal array to enable Binary Array Search to lo cate a
given po int in O(log n) time. Arrays are simply no t po werful eno ugh to suppo rt efficient algo rithms when data has
multiple attributes o r dimensions.
Real-wo rld data is o ften represented in tabular fo rm, which makes it well-suited to being sto red in an Excel
spreadsheet o r a database table. Given such a table with n co lumns, each co lumn can be viewed as a dimensio n and
each ro w represents an n-dimensio nal po int. Unfo rtunately, efficiently pro cessing multidimensio nal data is challenging
because there is no way to o rder all o f the ro ws in a table co mpletely, using all dimensio ns simultaneo usly. Yo u've
already seen ho w Binary Trees (when balanced) can sto re n elements effectively to guarantee O(log n) perfo rmance fo r
searching. In this lesso n, we'll apply this co ncept to sto ring n elements, each o f which has k-dimensio ns o f
info rmatio n. In this lesso n we'll use k=2 so we can draw two -dimensio nal images mo re easily; this appro ach can also
be used fo r arbitrary dimensio ns higher than 2.
Assume yo u have a co llectio n o f n 2-dimensio nal po ints in the Cartesian plane. Here's an example where n=10 :
In past lesso ns, we demo nstrated the Divide and Conquer appro ach to so rt an array by dividing it into left and right
sub-arrays, which were then so rted. But can yo u split a two -dimensio nal set o f po ints into a left set and a right set? If
yo u draw a vertical line thro ugh Po int 1, yo u have fo ur po ints o n the left and five po ints o n the right—this lo o ks
pro mising:
The dashed rectangle enclo sing all po ints represents the infinite Cartesian plane [x_low = -Infinity, y_low = -Infinity,
x_high = +Infinity, y_high = +Infinity]. Let's asso ciate this regio n with Po int 1 and co nsider its left sub-regio n to be the
shaded vertical rectangle o n the left and the right sub-regio n to be the vertical rectangle o n the right. It do esn't seem
po ssible to co ntinue this divisio n pro cess by adding vertical lines. Ho wever, co nsider dividing these left and right subregio ns by adding two horizontal lines, o ne thro ugh Po int 2 and the o ther thro ugh Po int 6 . These ho rizo ntal lines do
no t extend acro ss the who le plane, but rather divide the respective sub-regio ns into quadrants:.
Each o f these fo ur quadrants co ntain just 1 o r 2 po ints. This pro cess o f using alternating vertical and ho rizo ntal lines to
subdivide the set o f po ints can be repeated recursively within each quadrant. Here yo u can see the final subdivisio n o f
these ten po ints:
Let's design a data structure to represent the partitio ned info rmatio n. A kd-tree (sho rt fo r k-dimensio nal tree) is a
recursive binary tree structure with n no des, each o f which co ntains:
a 2-dimensio nal po int in the co llectio n.
a partitio n directio n (either vertical o r horizontal).
an asso ciated rectangular subregio n o f the two -dimensio nal plane.
two child no de links (named below and above).
Given the abo ve 10 po ints, here is its co rrespo nding binary kd-tree:
Po int 1 partitio ns the maximum regio n into two halves. The child sub-tree ro o ted at the no de fo r Po int 2 co ntains all
po ints to the left o f the vertical line partitio ning the regio n thro ugh Po int 1. Similarly, the child sub-tree ro o ted at the
no de fo r Po int 3 co ntains all po ints to the right o f the vertical line partitio ning the regio n thro ugh Po int 1. Instead o f
using the terms "left" and "right" to refer to child no des (which wo uld o nly apply fo r vertical partitio ning), kd-trees use
the co ncept o f "belo w" and "abo ve." The no des in the child sub-tree ro o ted at the no de fo r Po int 2 all represent po ints
with an x-co o rdinate that is smaller than ("belo w") the x-co o rdinate o f Po int 1. Similarly, the no des in the child sub-tree
ro o ted at the no de fo r Po int 6 all represent po ints with an x-co o rdinate that is larger than o r equal to ("abo ve") the xco o rdinate o f Po int 1.
Structurally, the tree is a classic binary search tree, but each level in the tree alternates the partitio n directio ns o f the
no des in the tree.
Let's get started by defining a class to represent the regio ns partitio ned by the kd-tree. We can't just use the
java.awt .Re ct angle Class because that defines rectangles using widths and heights.
Create a new Java Pro ject named Mult idim e nsio n and assign it to the J ava6 _Le sso ns wo rking set.
In yo ur Mult idim e nsio n pro ject /src so urce fo lder, create a kd package.
In the kd package, create a Re gio n class as sho wn:
CODE TO TYPE: Regio n class
package kd;
public class Region {
int x_min;
int x_max;
int y_min;
int y_max;
public Region (int x1, int y1, int x2, int y2) {
x_min = x1;
y_min = y1;
x_max = x2;
y_max = y2;
}
public Region (Region r) {
this(r.x_min, r.y_min, r.x_max, r.y_max);
}
static final int
minValue = Integer.MIN_VALUE;
static final int
maxValue = Integer.MAX_VALUE;
static final Region max = new Region(minValue, minValue, maxValue, maxValue);
}
We assume all co o rdinate po ints are integer values and all attributes are accessible within the kd package to simplify
the pro gramming o f the algo rithm. The m ax regio n represents the maximal regio n po ssible. No w that we have a
definitio n fo r the regio ns, we can design the class to represent the no des in the kd-tree.
In the kd package, create a KDNo de class as sho wn:
CODE TO TYPE: KDNo de class
package kd;
import java.awt.Point;
public class KDNode {
final Point point;
final int direction;
Region region;
KDNode above;
KDNode below;
public static final int HORIZONTAL = 0;
public static final int VERTICAL = 1;
public KDNode(Point p, int dir, Region r) {
this.point = new Point (p);
this.direction = dir;
this.region = new Region(r);
}
public KDNode(Point p, int dir) {
this (p, dir, Region.max);
}
}
Each KDNo de o bject represents a no de in a kd-tree and sto res three pieces o f info rmatio n as described earlier: a
po int, a regio n, and a partitio n directio n. KDNo de defines two co nstants that differentiate between HORIZ ONT AL and
VERT ICAL partitio ning. By default, each KDNo de o bject is asso ciated with the maximum regio n available, as defined
by the Re gio n class. The values fo r HORIZ ONT AL and VERT ICAL are cho sen such that 1-d gives the o ppo site
directio n o f d.
Fo r this no de to define a recursive search tree, it must define children no des. In this case, each KDNo de reco rds two
children, o ne "belo w" the partitio ning line and the o ther "abo ve" the partitio ning line. The no tio n o f a child no de being
"abo ve" is relative to the directio n o f the KDNo de . When the partitio ning fo r a no de is HORIZ ONT AL, the child no de
"abo ve" a no de is fo und vertically abo ve the ho rizo ntal partitio ning line with a y-co o rdinate that divides the no de's
regio n. When the partitio ning fo r a no de is VERT ICAL, the child no de "belo w" a no de is fo und to the left o f the vertical
partitio ning line with an x-co o rdinate that divides the no de's regio n.
The kd-tree ro o ted at a KDNo de n defines a binary search tree because all po ints in the sub-tree ro o ted by the
"belo w" child will be "belo w" the partitio ning line fo r no de n, while all po ints in the sub-tree ro o ted by the "abo ve" child
will be "abo ve" the partitio ning line fo r no de n. To make this happen, yo u need to define so me helper metho ds. Add
these metho ds to the end o f KDNo de :
CODE TO TYPE: Helper metho ds to add to KDNo de
public boolean isBelow(Point p) {
if (direction == VERTICAL) {
return p.x < point.x;
} else {
return p.y < point.y;
}
}
public boolean isAbove(Point p) {
if (direction == VERTICAL) {
return p.x >= point.x;
} else {
return p.y >= point.y;
}
}
These metho ds help determine, fo r a given KDNo de , whether a po int p is "belo w" o r "abo ve" its partitio ning line.
Yo u'll need o ne mo re metho d that returns a pro perly co nfigured child no de fo r a given KDNo de . The trick is to
co mpute the regio n asso ciated with the child no de based o n the regio n asso ciated with its existing parent no de:
CODE TO TYPE: Helper metho d fo r to add to KDNo de
KDNode createChild (Point p, boolean below) {
Region r = new Region (region);
if (direction == VERTICAL) {
if (below) {
r.x_max = point.x;
} else {
r.x_min = point.x;
}
} else {
if (below) {
r.y_max = point.y;
} else {
r.y_min = point.y;
}
}
return new KDNode(p, 1-direction, r);
}
The child no de must have the o ppo site partio ning o f its parent; that's why 1-dire ct io n is used as the directio n o f the
child no de. The po int to asso ciate with the child, p is passed to the KDNo de co nstructo r. The challenge is to co mpute
the sub-regio n asso ciated with the newly created child no de. There are fo ur cases to co nsider as implemented in the
abo ve co de—we'll just explain o ne. If a no de is ho rizo ntal, its point partitio ns its rectangular regio n into a regio n
"abo ve" the y-co o rdinate o f its po int and a regio n "belo w" the y-co o rdinate. Invo king cre at e Child(p, f alse ) o n a
horizontal no de n means that the regio n fo r the child KDNo de must be "trimmed" to be a pro per subset o f the current
no de's regio n. To do this, the abo ve co de sets the y_m in o f the child's regio n r. The o ther three cases are similar.
No w we can implement a metho d to add a po int to a given kd-tree ro o ted at a KDNo de :
CODE TO TYPE: Create add(Po int) metho d in KDNo de
public void add (Point p) {
if (p.equals(point)) { return; }
if (isBelow(p)) {
if (below == null) {
below = createChild (p, true);
} else {
below.add(p);
}
} else {
if (above == null) {
above = createChild (p, false);
} else {
above.add(p);
}
}
}
The kd-tree implementatio n here abides by Set semantics, as described earlier in this co urse. This means that the
same po int canno t exist mo re than o nce in a given kd-tree. When the add metho d returns witho ut thro wing an
Exceptio n, the po int is guaranteed to be added to the kd-tree.
If the po int to be added is belo w its partitio ning line, the add metho d either creates a new no de to represent the
"belo w" child (if that no de do esn't already exist) o r adds the po int to the kd-tree ro o ted at the "belo w" child. The lo gic
fo r the "abo ve" case is similar.
We've co mpleted the KDNo de class; no w it's time to design the class to represent the kd-tree.
In the kd package, create a KDT re e class as sho wn:
CODE TO TYPE: KDTree class
package kd;
import java.awt.Point;
public class KDTree {
KDNode root;
public KDTree() {
root = null;
}
public void add (Point value) {
if (root == null) {
root = new KDNode(value, KDNode.VERTICAL);
} else {
root.add(value);
}
}
}
A KDT re e o bject is defined by a ro o t KDNo de . This class o ffers an add metho d to add po ints to the kd-tree. If the kdtree is empty, it creates a new ro o t no de who se partitio ning by default (arbitrarily) is VERT ICAL.
T raversing a kd-tree
Yo u have eno ugh co de written to co nstruct a kd-tree fro m a set o f po ints. The hard part is fuguring o ut
whether the co de is wo rking because the binary tree structure is sto red in memo ry and it can't simply be
printed o ut to the co nso le. Yo u need to write a traversal ro utine that walks thro ugh the kd-tree in a specific
o rder; if the kd-tree is co nstructed pro perly, the o utput will be co rrect. There are many ways to traverse a
recursive tree. Using the example presented at the beginning o f this lesso n, a pre-order traversal wo uld:
1. pro cess Po int 1.
2. recursively pro cess all po ints to the left o f the partitio ning line thro ugh Po int 1.
3. recursively pro cess all po ints to the right o f the partitio ning line thro ugh Po int 1.
The applet class belo w interactively draws a kd-tree whenever a po int is added because the mo use was
pressed. At last, yo u have so mething to run fo r yo ur effo rts!
In the kd package, create a KDApple t class as sho wn:
CODE TO TYPE: KDApplet class
package kd;
import java.awt.*;
import java.awt.event.*;
public class KDApplet extends java.applet.Applet {
KDTree tree = new KDTree();
int toAWT(int y) {
if (y == Region.maxValue) { return 0; }
int awty = getHeight();
if (y != Region.minValue) { awty -= y; }
return awty;
}
int toCartesian(int awty) { return getHeight() - awty; }
public void init() {
setSize(400,400);
addMouseListener (new MouseAdapter() {
public void mouseClicked(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
tree.add(pt);
repaint();
}
});
}
public void paint(Graphics g) {
if (tree.root == null) {
g.drawString("Click to add points", 150, 200);
} else {
visit(g, tree.root);
}
}
void drawPartition (Graphics g, Region r, Point p, int type) {
if (type == KDNode.VERTICAL) {
g.drawLine(p.x, toAWT(r.y_min), p.x, toAWT(r.y_max));
} else {
int xlow = r.x_min;
if (r.x_min == Region.minValue) { xlow = 0; }
int xhigh = r.x_max;
if (r.x_max == Region.maxValue) { xhigh = getWidth(); }
g.drawLine(xlow, toAWT(p.y), xhigh, toAWT(p.y));
}
g.fillRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
void visit (Graphics g, KDNode n) {
if (n == null) { return; }
drawPartition(g, n.region, n.point, n.direction);
visit (g, n.below);
visit (g, n.above);
}
}
Save and run it. Add po ints to the kd-tree by clicking the mo use at different places in the Applet windo w.
The partitio ning directio n alternates as each po int is added. We've anno tated the screensho t belo w using red
numbers to identify the o rder the po ints were added (these do n't appear in the running pro gram):
Let's take a clo ser lo o k at this co de:
OBSERVE: Setting Up The Applet
package kd;
import java.awt.*;
import java.awt.event.*;
public class KDApplet extends java.applet.Applet {
KDTree tree = new KDTree();
...
int toCartesian(int awty) { return getHeight() - awty; }
public void init() {
setSize(400,400);
addMouseListener (new MouseAdapter() {
public void mouseClicked(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
tree.add(pt);
repaint();
}
});
}
public void paint(Graphics g) {
if (tree.root == null) {
g.drawString("Click to add points", 150, 200);
} else {
visit(g, tree.root);
}
}
...
}
The Java graphics co o rdinate system is different fro m the Cartesian plane. Specifically, the upper-left co rner
o f the windo w is co o rdinate (0 ,0 ). Fro m left to right, the x-co o rdinate increases, as it do es with Cartesian
co o rdinates. Ho wever, when mo ving fro m to p to bo tto m, the y-co o rdinate increases, which is o ppo site o f
Cartesian co o rdinates. The t o Cart e sian helper metho d co nverts the y-co o rdinate o f an Abstract Windo wing
To o lkit (AWT) po int into Cartesian co o rdinates. This metho d is used within the Mo use Adapt e r that
respo nds to mo use-click events by adding a po int t o t he kd-t re e . After each po int is added, the apple t is
re paint e d. The applet repaints itself by traversing the kd-tree using the visit metho d.
OBSERVE: Pre-o rder traversal o f the kd-tree
void visit (Graphics g, KDNode n) {
if (n == null) { return; }
drawPartition(g, n.region, n.point, n.direction);
visit (g, n.below);
visit (g, n.above);
}
All no des in the tree will be visited by the abo ve metho d. This declares a pre-o rder traversal because it first
"visits" the given no de by drawing it s part it io ning line o n t he scre e n. Then it visit s it s child no de s,
first the o nes belo w it and then the o nes abo ve it. The base case o f the recursio n st o ps whe n aske d t o
visit a null no de .
The real drawing wo rk is do ne in drawPart it io n:
OBSERVE: Draw Partitio ning Line Fo r KDNo de
int toAWT(int y) {
if (y == Region.maxValue) { return 0; }
int awty = getHeight();
if (y != Region.minValue) { awty -= y; }
return awty;
}
void drawPartition (Graphics g, Region r, Point p, int type) {
if (type == KDNode.VERTICAL) {
g.drawLine(p.x, toAWT(r.y_min), p.x, toAWT(r.y_max));
} else {
int xlow = r.x_min;
if (r.x_min == Region.minValue) { xlow = 0; }
int xhigh = r.x_max;
if (r.x_max == Region.maxValue) { xhigh = getWidth(); }
g.drawLine(xlow, toAWT(p.y), xhigh, toAWT(p.y));
}
g.fillRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
To draw the partitio ning line fo r a vertical no de, o ne o nly needs to draw a ve rt ical line t hro ugh t he xco o rdinat e p.x using t he y-co o rdinat e s f ro m t he asso ciat e d re gio n f o r t he no de . Ho wever, since
these regio ns may reflect partially infinite regio ns in the plane, yo u need the t o AWT helper metho d that
co nverts a (po tentially infinite) Cartesian y-co o rdinate into its AWT co unterpart. If t he co o rdinat e is t he
m axim um allo we d value f o r a Re gio n, t he y-co o rdinat e is 0 , because that's the to pmo st co o rdinate in
the AWT co o rdinate system. If the Cartesian y-co o rdinate is the minimum allo wed value fo r a Regio n, the
co unterpart y-co o rdinate is simply t he he ight o f t he Apple t windo w. Otherwise, the t o AWT metho d
co nve rt s t he y-co o rdinat e base d o n it s dist ance f ro m t he bo t t o m o f t he windo w (the he ight o f
t he apple t windo w).
Using kd-trees to Search for Points
No w that we've demo nstrated the pro per co nstructio n o f a kd-tree, there are two kind o f queries we'd like to
suppo rt:
Co ntains—do es the kd-tree co ntain a given po int P.
Selectio n—select the po ints in the kd-tree that are co ntained within a query rectangle.
To find whether a given po int exists in the tree, yo u can use the partitio ning lines asso ciated with each no de to
direct the search, either to the child no de belo w the line o r the child no de abo ve the line. Add this co de to the
end o f KDT re e :
CODE TO TYPE: Add find metho ds to KDTree
public KDNode find(Point p) {
return find(root, p);
}
KDNode find (KDNode node, Point p) {
if (node == null) { return null; }
if (node.point.distance(p) < 5) { return node; }
if (node.isBelow(p)) {
return find(node.below, p);
} else {
return find(node.above, p);
}
}
The f ind(no de ,p) metho d must cho o se whether to investigate the child belo w o r abo ve, based o n the
partitio ning line. The recursive metho d will eventually terminate at a leaf no de o r when the no de's Euclidian
distance to p is smaller than 5 pixels.
To highlight the po int o ver which the curso r mo ves, mo dify KDApple t as sho wn:
CODE TO TYPE: Mo dificatio ns to KDApplet
package kd;
import java.awt.*;
import java.awt.event.*;
public class KDApplet extends java.applet.Applet {
KDTree tree = new KDTree();
KDNode match = null;
boolean redraw = false;
int toAWT(int y) {
if (y == Region.maxValue) { return 0; }
int awty = getHeight();
if (y != Region.minValue) { awty -= y; }
return awty;
}
int toCartesian(int awty) { return getHeight() - awty; }
public void init() {
setSize(400,400);
addMouseListener (new MouseAdapter() {
public void mouseClicked(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
tree.add(pt);
repaint();
}
});
addMouseMotionListener (new MouseAdapter() {
public void mouseMoved(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
match = tree.find(pt);
if (match != null) {
redraw = true;
Graphics g = getGraphics();
g.setColor(Color.RED);
g.fillRect(match.point.x - 4, toAWT(match.point.y) - 4, 8, 8);
g.dispose();
} else {
if (redraw) {
repaint();
redraw = false;
}
}
}
});
}
public void paint(Graphics g) {
if (tree.root == null) {
g.drawString("Click to add points", 150, 200);
} else {
visit(g, tree.root);
}
}
void drawPartition (Graphics g, Region r, Point p, int type) {
if (type == KDNode.VERTICAL) {
g.drawLine(p.x, toAWT(r.y_min), p.x, toAWT(r.y_max));
} else {
int xlow = r.x_min;
if (r.x_min == Region.minValue) { xlow = 0; }
int xhigh = r.x_max;
if (r.x_max == Region.maxValue) { xhigh = getWidth(); }
g.drawLine(xlow, toAWT(p.y), xhigh, toAWT(p.y));
}
g.fillRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
void visit (Graphics g, KDNode n) {
if (n == null) { return; }
drawPartition(g, n.region, n.point, n.direction);
visit (g, n.below);
visit (g, n.above);
}
}
Let's take a clo ser lo o k at the changes:
OBSERVE:
KDNode match = null;
boolean redraw = false;
...
addMouseMotionListener (new MouseAdapter() {
public void mouseMoved(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
match = tree.find(pt);
if (match != null) {
redraw = true;
Graphics g = getGraphics();
g.setColor(Color.RED);
g.fillRect(match.point.x - 4, toAWT(match.point.y) - 4, 8, 8);
g.dispose();
} else {
if (redraw) {
repaint();
redraw = false;
}
}
}
});
Two new fields are added. m at ch reco rds the last po int in the kd-tree matched by the curso r; re draw
determines when to redraw the image upo n matching the curso r.
The real lo gic o ccurs in the Mo use Mo t io nList e ne r implementatio n, which activates with each mo ve o f the
mo use. It co nve rt s t he m o use po int int o Cart e sian co o rdinat e s and then t rie s t o f ind t he po int
wit hin t he kd-t re e . If a po int is f o und, it is f ille d in re d. If t he m o use m o ve s and t he re is no
m at ching po int , the e nt ire kd-t re e m ust be re f re she d.
Save and run it; there is a flicker effect. We can use a technique called "do uble buffering" to eliminate mo st
o f the flickering. Make these changes:
CODE TO TYPE: Updates to KDApplet
package kd;
import java.awt.*;
import java.awt.event.*;
public class KDApplet extends java.applet.Applet {
KDTree tree = new KDTree();
KDNode match = null;
boolean redraw = false;
Image bufferImage;
Graphics bufferGraphics;
int toAWT(int y) {
if (y == Region.maxValue) { return 0; }
int awty = getHeight();
if (y != Region.minValue) { awty -= y; }
return awty;
}
int toCartesian(int awty) { return getHeight() - awty; }
public void init() {
setSize(400,400);
addMouseListener (new MouseAdapter() {
public void mouseClicked(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
tree.add(pt);
redraw();
repaint();
}
});
addMouseMotionListener (new MouseAdapter() {
public void mouseMoved(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
KDNode newMatch = tree.find(pt);
if (match != newMatch) {
match = newMatch;
redraw();
if (match != null) {
bufferGraphics.setColor(Color.RED);
bufferGraphics.fillRect(match.point.x - 4, toAWT(match.point.y) - 4,
8, 8);
bufferGraphics.setColor(Color.BLACK);
}
repaint();
}
match = tree.find(pt);
if (match != null) {
redraw = true;
Graphics g = getGraphics();
g.setColor(Color.RED);
g.fillRect(match.point.x - 4, toAWT(match.point.y) - 4, 8, 8);
g.dispose();
} else {
if (redraw) {
repaint();
redraw = false;
}
}
}
});
}
public void paint(Graphics g) {
if (bufferImage == null) {
bufferImage = createImage(getWidth(), getHeight());
bufferGraphics = bufferImage.getGraphics();
}
if (tree.root == null) {
g.drawString("Click to add points", 150, 200);
} else {
g.drawImage(bufferImage, 0, 0, this);
visit(g, tree.root);
}
}
void redraw() {
bufferGraphics.clearRect(0, 0, getWidth(), getHeight());
visit(bufferGraphics, tree.root);
}
void drawPartition (Graphics g, Region r, Point p, int type) {
if (type == KDNode.VERTICAL) {
g.drawLine(p.x, toAWT(r.y_min), p.x, toAWT(r.y_max));
} else {
int xlow = r.x_min;
if (r.x_min == Region.minValue) { xlow = 0; }
int xhigh = r.x_max;
if (r.x_max == Region.maxValue) { xhigh = getWidth(); }
g.drawLine(xlow, toAWT(p.y), xhigh, toAWT(p.y));
}
g.fillRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
void visit (Graphics g, KDNode n) {
if (n == null) { return; }
drawPartition(g, n.region, n.point, n.direction);
visit (g, n.below);
visit (g, n.above);
}
}
The key to flicker-free graphics is to have all drawing perfo rmed in an o ff-screen image and then have the
paint metho d draw that image to the screen. The o ffscreen image bufferImage is created the first time paint
is invo ked. A newly added re draw metho d perfo rms all drawing within the bufferGraphics o bject asso ciated
with this o ffscreen image:
OBSERVE:
public void mouseMoved(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
KDNode newMatch = tree.find(pt);
if (match != newMatch) {
match = newMatch;
redraw();
if (match != null) {
bufferGraphics.setColor(Color.RED);
bufferGraphics.fillRect(match.point.x - 4, toAWT(match.point.y) - 4,
8, 8);
bufferGraphics.setColor(Color.BLACK);
}
repaint();
}
The real lo gic o ccurs within the mo dified m o use Mo ve d metho d. If t he re is a ne w m at ch f o und t hat
dif f e rs f ro m t he last (no n-null) m at ch, t he po int is re drawn in re d in t he o f f scre e n buf f e r;
o therwise the re paint never o ccurs.
Save and run this co de; there's a no ticeable difference in perfo rmance. Yo u can no w mo ve the mo use
aro und rapidly o ver the po ints in the kd-tree and see the highlighted red po ints appear and disappear.
A kd-tree can suppo rt Rectangle Queries that efficiently return the set o f po ints co ntained within a two dimensio nal rectangle. A kd-tree can also suppo rt Nearest Neighbor Queries that lo o k fo r the clo sest po int in
the kd-tree to a query po int (x,y). Fo r mo re details o n kd-trees, yo u may want to refer to the Algo rithms in a
Nutshell bo o k.
Lessons Learned
Use a re cursive st ruct ure t o part it io n an n-dim e nsio nal se t o f po int s: In all examples so
far, yo u have seen recursio n that separates an aggregate into a "left" and a "right" side. By
alternating dimensio ns, the kd-tree co ncept can acco mo date n-dimensio nal data. This beco mes
really exciting with high-dimensio nal data because the rectangular and nearest neighbo r queries
can perfo rm efficiently.
Use re cursive t rave rsal t o visit e ve ry e le m e nt in a re cursive t re e : The pre-order traversal
is intro duced in this lesso n. The co ncept applies to any recursive data structure. In general, there
are three primary traversal o rderings: pre-o rder, po st-o rder, and in-o rder. Each fully traverses all
elements in the tree, but do es so in a different o rder.
Project
Mo dify the existing KDApple t class to display the pre-order number asso ciated with each po int. The preorder number is determined in a pre-o rder traversal o f a binary tree. If yo u refer to the sample screensho t
image sho wn earlier in the lesso n, the po ints are drawn with numbers in red signifying the pre-o rder
numbering. As new po ints are added, the numbers will change.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Mathematical Algorithms and Floating Point
Computations
Lesson Objectives
When yo u finish this lesso n, yo u will be able to :
explain the structure o f flo ating-po int numbers acco rding to the IEEE standard.
demo nstrate co de techniques to mitigate ro unding erro rs in flo ating po int co mputatio ns.
Mathematical Algorithms and Floating Point Computations
Co mputers are finite machines that are designed to perfo rm basic co mputatio ns o n values sto red in registers by a
Central Pro cessing Unit (CPU). The size o f these registers has evo lved as co mputer architectures have gro wn fro m
the po pular 8 -bit Intel pro cesso rs in the 19 70 s to to day's widespread acceptance o f 6 4-bit architectures.
Co mputatio ns o ver integer-based values (such as Bo o leans, 8 -bit sho rts, and 16 - and 32-bit integers) have
traditio nally been the mo st efficient co mputatio ns perfo rmed by the pro cesso r. Mo st CPUs to day are fully integrated
with a Flo ating Po int Unit (FPU) that suppo rts the IEEE Standard fo r Binary Flo ating-Po int Arithmetic (IEEE 754). This
means that the perfo rmance o f flo ating-po int co mputatio ns is o ften mo re efficient than their integer co unterparts.
A flo ating-po int number is a finite representatio n designed to appro ximate a real number with a binary representatio n
that may be infinite. As yo u begin to experiment with flo ating-po int numbers, yo u may be surprised at so me o f the
results.
Create a new Java Pro ject named Mat he m at ical and assign it to the J ava6 _Le sso ns wo rking set.
In yo ur Mat he m at ical pro ject /src so urce fo lder, create a num e ric package.
In the num e ric package, create a Flo at ing class:
CODE TO TYPE: Flo ating class
package numeric;
public class Floating {
public static void main(String[] args) {
float total = 3.9f;
while (total > 3.7) {
System.out.println(total);
total = total - 0.01f;
}
float f = 3.88f - 0.01f;
if (f == 3.87f) {
System.out.println ("Same");
}
int bits = Float.floatToIntBits(3.88f);
int signBit = 0;
if ((bits & 0x80000000) != 0) { signBit = 1; }
System.out.println ("3.88f is " + Integer.toHexString(bits));
System.out.println ("3.88f is " + Integer.toBinaryString(bits));
System.out.println ("[s][eeeeeeee][mmmmmmmmmmmmmmmmmmmmmmm]");
System.out.print ("[" + signBit + "]");
System.out.print ("[" + pad((bits & 0x7f800000) >> 23, 8) + "]");
System.out.print ("[" + pad((bits & 0x007fffff), 23) + "]");
}
static String pad (int value, int len) {
StringBuilder sb = new
StringBuilder(Integer.toBinaryString(value));
while (sb.length() < len) {
sb.insert(0, '0');
}
return sb.toString();
}
}
Save and run it. This co de prints a table o f values decreasing by 0 .0 1 each time:
OBSERVE: Flo ating o utput
3.9
3.89
3.88
3.8700001
3.8600001
3.8500001
3.8400002
3.8300002
3.8200002
3.8100002
3.8000002
3.7900002
3.7800002
3.7700002
3.7600002
3.7500002
3.7400002
3.7300003
3.7200003
3.7100003
3.7000003
3.88f is 407851ec
3.88f is 1000000011110000101000111101100
[s][eeeeeeee][mmmmmmmmmmmmmmmmmmmmmmm]
[0][10000000][11110000101000111101100]
Our pro gram starts as expected, but sho rtly thereafter so me o f the the co mputatio ns intro duce an erro r. What's so
special abo ut subtracting 0 .0 1 fro m 3.88? Let's take a clo ser lo o k at ho w Java represents 3.88, using the IEEE
Flo ating-Po int standard. The 32 bits used fo r this value are represented in hexadecimal no tatio n as 0 x4 0 7 85 1e c.
These bits are numbered fro m 31 (o n the left) to 0 (farthest to the right) and enco de this info rmatio n.
3.88f is represented in 32 bits as 4 0 7 85 1e c, as explained by the flo ating po int standard. Any number represented in
flo ating-po int is equal to m * 2 exp . Bit 31 (the bit that is selected by the mask 0 x80 0 0 0 0 0 0 ) indicates whether the value
is po sitive o r negative. Bits 30 -23 (the eight bits that are selected by the mask 0 x7 f 80 0 0 0 0 ) represent the expo nent,
exp. Bits 22-0 (the twenty-three bits that are selected by the mask 0 x0 0 7 f f f f f ) represent the mantissa, m, o f the
flo ating-po int number. In the o utput abo ve, Int e ge r.t o BinarySt ring prints o nly 31 binary characters and do esn't print
the sign bit. The 32 bits are arranged fro m left to right as sho wn abo ve, which pro duces the representatio n 4 0 7 85 1e c
as a 32-bit Java flo ating po int number.
Yo u can determine which po wer o f two to use by interpreting the expo nent bits as a po sitive number and then
subtracting a bias fro m the po sitive number. Fo r a flo at, the bias is 126 . Given the enco ding o f 0 x80 = 128, the
expo nent is 128 - 126 = 2.
To maintain the mo st precisio n in the representatio n, the mantissa is always no rmalized so its leftmo st digit is a 1; this
means that digit can actually be o mitted fro m the representatio n to increase the precisio n o f the final number by o ne bit.
To interpret the mantissa, remember that it is a binary fractio n co mputed as the sum o f fractio nal po wers o f two . Here
mantissa = .[1]11110 0 0 0 10 10 0 0 11110 110 0 , which sho ws that the first implied digit is a o ne. Expanding this
co mputatio n results in this:
mantissa = [1/2] + 1/4 + 1/8 + 1/16 + 1/32 + 1/10 24 + 1/40 9 6 + 1/6 5536 + 1/1310 72 + 1/26 2144 + 1/52428 8 + 1/20 9 7152 +
1/419 430 4.
If yo u co mpute the abo ve sum using a calculato r, it is exactly 0 .9 7 0 0 0 0 0 0 286 10 229 4 9 2187 5 .
The final value = 0 .9 7 0 0 0 0 0 0 286 10 229 4 9 2187 5 * 2 2 which equals the exact number
3.880 0 0 0 0 114 4 4 0 9 17 9 6 87 5 . So , the actual erro r o f this representatio n o f 3.88f is o n the o rder o f 0 .0 0 0 0 0 0 1 o r 1
in 10 ,0 0 0 ,0 0 0 .
Note
If the first bit in the mantissa is implied, ho w is the value 1/2 represented in flo ating po int? Well, the
mantissa must be zero (since the 1st bit is implied), the sign is 0 and the expo nent must be 0 , so yo u
need to add the bias o f 126 to see that the enco ding is 3F0 0 0 0 0 0 .
Wo rking with flo ating po int co mputatio ns can intro duce small rounding errors into yo ur so lutio ns. In this lesso n, yo u'll
so lve a co mmo n mathematical pro blem and learn ho w to manage ro unding erro rs in yo ur implementatio n.
Note
Java has two flo ating-po int number fo rmats: f lo at uses 32 bits, while do uble uses 6 4 bits. Fo r the rest
o f this lesso n, all co mputatio n will be do ne using do uble values.
Gauss Jordan Elimination
Given a set o f m linear equatio ns o f m variables each, is there a unique so lutio n fo r these variables? Fo r
example, let's say yo u are given these three equatio ns o ver the three variables: x, y, z:
x + 3y + 5z = 9
E1
2x + 7y + 2z = 2 E2
x + y + 4z = 2
E3
Yo u co uld use a trial and error appro ach, guessing values fo r these variables to see if they satisfy all
equatio ns simultaneo usly. Instead, co nsider an appro ach that systematically determines the values. Fo r
example, yo u co uld transfo rm the equatio ns by adding two o r mo re o f the equatio ns to gether, trying to
eliminate a variable.
If yo u subtract equatio n E1 twice fro m E2 and o nce fro m E3, the equatio ns beco me simpler because yo u are
able to eliminate the x variable fro m the seco nd and third equatio ns:
x + 3y + 5z = 9 E1
y - 8z = -16
- 2y - z = -7
E2
E3
No w just add equatio n E2 twice to equatio n E3:
x + 3y + 5z = 9 E1
y - 8z = -16
E2
-17z = -39
E3
Given these three equatio ns, yo u can so lve fo r z = 39/17. Insert this value back into the seco nd equatio n and
yo u can co mpute y = -16 + 312/17 = 40/17. Finally, insert these values o f y and z back into the first equatio n to
yield x = 9 - 120/17 - 195/17 = -162/17. So , the final so lutio n is (x = -162/17, y = 40/17, z = 39/17). Instead o f
substituting values, we co uld have repeated the eliminatio n steps such that each o f the three equatio ns abo ve
has a single variable. In fact, that's the Gauss Jordan Elimination algo rithm attempts to do that.
A pro blem instance is represented by an m*(m+1) matrix where there are m variables and m equatio ns. In the
abo ve example, m=3. No te that each equatio n has m+1 values because the last value is the co nstant value
equal to the sum o f the variable terms. There is no singular so lutio n when there are fewer than m equatio ns
with m variables. Yo u have eno ugh info rmatio n to write the pseudo co de fo r Gauss Jo rdan Eliminatio n o f a
set o f linear equatio ns represented by matrix A, where A[r][c] is the cth co efficient in the rth ro w. Ro w r is
defined in the range 0 .. m-1 and co lumn c is in the range 0 .. m because o f the co nstant value in each ro w.
This pseudo co de captures the appro ach used o n the pro blem instance sho wn earlier:
OBSERVE: pseudo co de fo r Gauss Jo rdan Eliminatio n
gaussJordan (A)
foreach base=0 to m-1 do
baseCoeff = A[base][base]
foreach row=0 to m-1 do
if (row != base) then
innerCoeff = A[row][base]
foreach column c=base to m do
A[row][column] -= (innerCoeff/baseCoeff)*A[base][column]
It's hard to understand triply-nested lo o ps just by lo o king at them. The best way to fo llo w this lo gic is to write
the co de and fo llo w the lo gic within the debugger.
In the num e ric package, create a GaussJ o rdan class as sho wn:
CODE TO TYPE: GaussJo rdan class
package numeric;
public class GaussJordan {
public static void gaussJordan(double[][] A) {
int m = A.length;
for (int base = 0; base < m; base++) {
double baseCoeff = A[base][base];
for (int row = 0; row < m; row++) {
if (row != base) {
double innerCoeff = A[row][base];
for (int c = base; c <= m; c++) {
A[row][c] -= (innerCoeff/baseCoeff)*A[base][c];
}
}
}
}
}
public static void main(String[] args) {
double [][]mat = {{1,3,5,9}, {2,7,2,2}, {1,1,4,2}};
gaussJordan(mat);
for (int i = 0; i < mat.length; i++) {
for (int j = 0; j < mat[0].length; j++) {
System.out.print(mat[i][j] + " ");
}
System.out.println();
}
}
}
Save and run it; it co mputes a so lutio n fo r the earlier pro blem instance:
OBSERVE: Sample executio n o f GaussJo rdan
1.0 0.0 0.0 -9.529411764705884
0.0 1.0 0.0 2.352941176470587
0.0 0.0 -17.0 -39.0
Yo u can co nfirm that these values co rrespo nd to the co mputed so lutio n earlier. That is, x = -16 2/17, y = 40 /17,
and z = 39 /17. Let's take a clo ser lo o k at the co de:
OBSERVE: Co mputing Gauss Jo rdan o n a matrix
public static void gaussJordan(double[][] A) {
int m = A.length;
for (int base = 0; base < m; base++) {
double baseCoeff = A[base][base];
for (int row = 0; row < m; row++) {
if (row != base) {
double innerCoeff = A[row][base];
for (int c = base; c <= m; c++) {
A[row][c] -= (innerCoeff/baseCoeff)*A[base][c];
}
}
}
}
}
base iterates o ver each ro w in the matrix, and base Co e f f sto res the co efficient o f the variable being
eliminated in each pass. Fo r every o ther ro w in the matrix (o t he r t han base ), t he inne rm o st lo o p
re duce s t he co e f f icie nt s o f t he se ro ws pro po rt io nal t o base Co e f f . inne rCo e f f is used to
no rmalize the adjustment which sho uld eliminate the co efficient o f the ro w base in all equatio ns.
OBSERVE:
public static void main(String[] args) {
double [][]mat = {{1,3,5,9}, {2,7,2,2}, {1,1,4,2}};
gaussJordan(mat);
for (int i = 0; i < mat.length; i++) {
for (int j = 0; j < mat[0].length; j++) {
System.out.print(mat[i][j] + " ");
}
System.out.println();
}
}
The final co de in GaussJ o rdan prints o ut the co ntents o f the matrix m at , which was mo dified in place by the
gaussJ o rdan metho d. To better understand this algo rithm's behavio r, run it within the debugger. These
image sho ws the GaussJ o rdan as it appears in Eclipse.
Set a breakpo int at the innermo st fo r lo o p (do this by do uble-clicking within the blue vertical bo rder o n the line
co rrespo nding to this lo o p) and run GaussJ o rdan using the Eclipse debugger (De bug As | J ava
Applicat io n). When the debugger sto ps the first time, select Windo w | Sho w Vie w | Ot he r | De bug |
Variable s to sho w the variables as fo llo ws (depending o n the current Eclipse versio n, yo ur screen may be
slightly different):
The matrix A appears in the debugger as three ro ws o f fo ur values each. This matches the equatio n set
described earlier exactly. No w use the debugger to o ls to co ntinue the executio n fo ur times. With each
co ntinuatio n, the values o f A change and ultimately beco me this:
The x variable has been eliminated fro m the seco nd ro w in A because the first value o f the seco nd gro up o f
numbers is 0 . Do this five mo re times; the values o f the third ro w in A also change to eliminate its co efficient
fo r x.
Rounding Errors
Is this implementatio n co mplete and co rrect? Go back to the matrix definitio n in GaussJ o rdan and change it
as sho wn:
CODE TO TYPE:
...
public static void main(String[] args) {
double [][]mat = {{1,3,5,9}, {2,7,2,2}, {1,1,4,210000,10000,40000,20000}};.
gaussJordan(mat);
...
No thing has changed mathematically because yo u have o nly multiplied the co efficients in the last ro w by
10 0 0 0 , but check the o utput:
OBSERVE: Output o f revised matrix
1.0 0.0 -3.552713678800501E-15 -9.529411764705884
0.0 1.0 0.0 2.3529411764705905
0.0 0.0 -170000.0 -390000.0
Because o f this small change, the co de was unable to eliminate the co efficient fo r z in the first equatio n. Yo u
might be tempted to use the java.m at h.BigDe cim al class to represent all values because it is designed to
sto re arbitrary-precisio n signed decimal numbers. Ho wever, the co mment belo w that appears in the
do cumentatio n fo r the divide metho d o f BigDe cim al states that, "if the exact quo tient canno t be represented
(because it has a no n-terminating decimal expansio n) an ArithmeticExceptio n is thro wn." So yo u can't use
BigDe cim al.
To learn abo ut o ther issues that pertain to flo ating-po int co mputatio ns, read the technical do cument, What
Every Co mputer Scientist Sho uld Kno w Abo ut Flo ating-Po int Arithmetic. This paper is the standard fo r
explaining challenges the yo u'll enco unter when wo rking with flo ating-po int. Fo r no w, let's fo cus o n so me
essential po ints o f flo ating-po int numbers.
Co mputatio ns perfo rmed o n flo ating numbers can pro duce infinitessimal differences in results, such as the
value abo ve, which is o n the o rder o f 10 -15 .
The usual strategy we use to deal with these very small numbers is to reco gnize that they typically o ccur o nly
thro ugh subtractio n and additio n, rather than multiplicatio n and divisio n. That's because in o rder to achieve
such lo w expo nents, yo u wo uld need to divide two numbers that are 15 o rders o f magnitude apart; this may
happen in rando m data, but it is unlikely to o ccur with mo st real-wo rld data.
Make these changes to GaussJ o rdan with the revised mat matrix definitio n in the m ain() metho d:
OBSERVE: Mo dificatio ns to GaussJo rdan
package numeric;
public class GaussJordan {
static final double epsilon = 1e-9;
static void gaussJordan(double[][] A) {
int m = A.length;
for (int base = 0; base < m; base++) {
double baseCoeff = A[base][base];
for (int row = 0; row < m; row++) {
if (row != base) {
double innerCoeff = A[row][base];
for (int c = base; c <= m; c++) {
A[row][c] -= (innerCoeff/baseCoeff)*A[base][c];
if (A[row][c] < epsilon && A[row][c] > -epsilon) {
A[row][c] = 0;
}
}
}
}
}
}
public static void main(String[] args) {
double [][]mat = {{1,3,5,9}, {2,7,2,2}, {10000,10000,40000,20000}};
gaussJordan(mat);
for (int i = 0; i < mat.length; i++) {
for (int j = 0; j < mat[0].length; j++) {
System.out.print(mat[i][j] + " ");
}
System.out.println();
}
}
}
Save and run it. The pro blem has been masked:
OBSERVE: Output with larger co efficients, tho ugh same so lutio n
1.0 0.0 0.0 -9.529411764705884
0.0 1.0 0.0 2.3529411764705905
0.0 0.0 -170000.0 -390000.0
The abo ve so lutio n reaches a nearly identical so lutio n (o nly the middle line is o ff by by a few fractio nal digits).
Yo u can use this appro ach when yo ur algo rithm depends o n detecting zero values in yo ur co mputatio ns.
Partial Input Data
There are o ther situatio ns that co uld cause the existing implementatio n to fail. Many o f the equatio ns are no t
mathematically independent, which means that o ne o f the equatio ns is equivalent to a sequence o f additio ns
(and subtractio ns) o f the o ther equatio ns. Fo r example, change the m at input matrix in the m ain() metho d as
sho wn:
CODE TO TYPE:
...
public static void main(String[] args) {
double [][]mat = {{1,3,5,9}, {2,7,2,2}, {10000,10000,40000,200002,6,10,18}};
.
gaussJordan(mat);
...
The third ro w is simply twice the first ro w. Rerun the class:
OBSERVE: Output when equatio ns are o nly partially so lvable
1.0 0.0 NaN NaN
0.0 1.0 NaN NaN
0.0 0.0 0.0 0.0
NaN stands fo r "No t a Number," which is defined in the Flo ating-Po int standard. This o ccurs typically when
dividing by zero . Java o nly thro ws an ArithmeticExceptio n when an integer divisio n causes a divide by zero ;
flo ating-po int co mputatio ns give no indicatio n that anything has go ne wro ng. To understand why the pro blem
happens, go back to the pseudo co de; yo u can see that there is no pro tectio n when base Co e f f is zero . Fix it
no w by treating any base Co e f f value sufficiently clo se to zero as zero ; in o ther wo rds, skip that co lumn:
CODE TO TYPE: Mo dificatio ns to gaussJo rdan metho d
static void gaussJordan(double[][] A) {
int m = A.length;
for (int base = 0; base < m; base++) {
double baseCoeff = A[base][base];
if (baseCoeff < epsilon && baseCoeff > -epsilon) { continue; }
for (int row = 0; row < m; row++) {
if (row != base) {
double innerCoeff = A[row][base];
for (int c = base; c <= m; c++) {
A[row][c] -= (innerCoeff/baseCoeff)*A[base][c];
if (A[row][c] < epsilon && A[row][c] > -epsilon) {
A[row][c] = 0;
}
}
}
}
}
}
Save and run it:
OBSERVE: Pro per o utput when equatio ns are o nly partially so lvable
1.0 0.0 29.0 57.0
0.0 1.0 -8.0 -16.0
0.0 0.0 0.0 0.0
While the matrix has no t been fully reduced to o ne variable per ro w, the o utput clearly demo nstrates that there
is no unique so lutio n fo r all variables, but rather a family o f so lutio ns.
Since yo u have just pro tected against zero co efficients, co nsider the set o f equatio ns belo w, in which no
equatio n uses mo re than two variables; no te that the co efficient o f x in the first equatio n is zero .
CODE TO TYPE:
...
public static void main(String[] args) {
double [][]mat = {{1,3,5,9}, {2,7,2,2}, {2,6,10,18}{0,2,1,7}, {1,2,0,3}, {2,
0,5,3}};.
gaussJordan(mat);
...
Save and run it:
OBSERVE: Invalid o utput when first co efficient is zero
0.0 0.0 0.0 3.4
1.0 2.0 0.0 3.0
2.0 0.0 5.0 3.0
Ho wever, if yo u reo rder the ro ws—which ultimately sho uld have no effect o n the so lutio n—a so lutio n can be
fo und. Change it again as sho wn:
CODE TO TYPE:
...
public static void main(String[] args) {
double [][]mat = {{0,2,1,7}, {1,2,0,3}, {2,0,5,3}{1,2,0,3}, {0,2,1,7}, {2,0,
5,3}};.
gaussJordan(mat);
...
This results in a valid so lutio n:
OBSERVE: Valid o utput when reo rganizing ro ws
1.0 0.0 0.0 -2.428571428571429
0.0 2.0 0.0 5.428571428571429
0.0 0.0 7.0 11.0
Yo u must mo dify the algo rithm slightly such that fo r each o f the first m co lumns, it finds a no n-zero pivot value
to use, rather than simply cho o sing the assigned co efficient in that co lumn. It turns o ut that there are
mathematical advantages if the pivo t is the co efficient in that co lumn o f greatest magnitude. The mo dified
pseudo co de lo o ks like this:
OBSERVE: pseudo co de fo r Gauss Jo rdan Eliminatio n
gaussJordan (A)
foreach base=0 to m-1 do
determine row r whose A[r][base] is highest magnitude; if zero skip and cont
inue
if r is different from base, swap rows r and base
baseCoeff = A[base][base]
foreach row=0 to m-1 do
if (row != base) then
innerCoeff = A[row][base]
foreach column c=base to m do
A[row][c] -= (innerCoeff/baseCoeff)*A[base][c]
The actual co de mo dificatio ns to the gaussJ o rdan metho d lo o k like this:
CODE TO TYPE: Mo dificatio ns to GaussJo rdan class
static void gaussJordan(double[][] A) {
int m = A.length;
for (int base = 0; base < m; base++) {
double pivot = 0;
int r = -1;
for (int k = base; k < m; k++) {
if (Math.abs(A[k][base]) > pivot) {
pivot = Math.abs(A[k][base]);
r = k;
}
}
if (pivot < epsilon && pivot > -epsilon) { continue; }
if (r != base) {
for (int c = base; c < m+1; c++) {
double tmp = A[base][c];
A[base][c] = A[r][c];
A[r][c] = tmp;
}
}
double baseCoeff = A[base][base];
if (baseCoeff < epsilon && baseCoeff > -epsilon) { continue; }
for (int row = 0; row < m; row++) {
if (row != base) {
double innerCoeff = A[row][base];
for (int c = base; c <= m; c++) {
A[row][c] -= (innerCoeff/baseCoeff)*A[base][c];
if (A[row][c] < epsilon && A[row][c] > -epsilon) {
A[row][c] = 0;
}
}
}
}
}
}
Save and run it; the o utput is co rrect again:
OBSERVE: Valid o utput when reo rganizing ro ws
2.0 0.0 0.0 -4.857142857142858
0.0 2.0 0.0 5.428571428571429
0.0 0.0 -3.5 -5.5
Matrix Determinant
There are a number o f useful algo rithms o ver square matrices that demo nstrate techniques yo u can use to
so lve wo rthwhile pro blems. In linear algebra, the determinant is a value asso ciated with a square matrix.
When the square matrix represents a system o f linear equatio ns, there will be a unique so lutio n fo r the
equatio ns if the determinant is no n-zero . Yo u can use the determinant to so lve fo r the linear system o f
equatio ns, much like the Gauss Jordan eliminatio n.
Given the abo ve two -dimensio nal matrix A, its determinant is:
Yo u can visualize this co mputatio n by subtracting the pro duct o f the No rtheast diago nal (2*5) fro m the
pro duct o f the So uthwest diago nal (3*4). To sho w the utility o f the determinant, let's so lve this system o f two
linear equatio ns:
3x + 5y = 7 E1
2x + 4y = 5 E2
If yo u lo o k at the 2x2 matrix fo rmed by just the co efficients o f the x and y variables, yo u'll reco gnize the earlier
2x2 matrix. Because this determinant has a no n-zero value, yo u kno w that there is a valid unique so lutio n. To
determine the so lutio n fo r x and y, we need to co mpute fo ur determinant values:
The deno minato rs o f these two fractio ns are the determinant when using the co efficients o f the x and y
variables. The numerato r o f the so lutio n fo r x is the determinant o f a 2x2 matrix that is fo rmed by replacing the
co lumn co ntaining the x co efficients with the co lumn o f the co nstant values. Similarly, the numerato r o f the
so lutio n fo r y is the determinant o f a 2x2 matrix fo rmed by replacing the co lumn co ntaining the y co efficients
with the co lumn o f the co nstant values. The value o f the x numerato r is 7*4 - 5*5 = 3 while the value o f the y
numerato r is 3*5 - 2*7 = 1. These results sho w that the so lutio n to these equatio ns is (x = 3/2 and y = 1/2).
Yo u can verify these values by plugging them back into either o f the o riginal equatio ns.
Do es this scale to equatio ns with mo re than 2 variables? Yes! Let's revisit the earlier set o f three equatio ns
used in the Gauss Jo rdan sectio n and use determinants to co mpute the so lutio n. Here are tho se equatio ns
again:
x + 3y + 5z = 9
E1
2x + 7y + 2z = 2 E2
x + y + 4z = 2
E3
Using the same determinant lo gic as the 2x2 case, the so lutio n to this equatio n is:
In the numerato rs, yo u replace the co lumn co rrespo nding to the variable with the co lumn o f co nstant values.
To so lve this, yo u need to be able to co mpute the determinant fo r a 3x3 matrix. Fo rtunately, yo u can co mpute
the determinant o f a 3x3 matrix by a co mputatio n invo lving the determinants o f 2x2 vertices. This image
sho ws ho w this is do ne:
To visualize this, take the to p ro w o f the 3x3 matrix which co ntains the values (1,9 ,5). Fo r each o f these
values, co mpute the determinants o f the three 2x2 matrices that remain when yo u remo ve the to p ro w and the
respective co lumn o f each o f these values. Then, sum the co mputatio n abo ve, alternating signs of the
constituent sub-parts. Instead o f do ing this o peratio n by hand, yo u need to write a pro gram (especially when
co mputing determinants fo r higher-o rder matrices).
In the num e ric package, create a Mat rix class as sho wn:
CODE TO TYPE: Matrix class
package numeric;
public class Matrix {
public static double det(double[][] m) {
switch (m.length) {
case 1:
return m[0][0];
case 2:
return m[0][0]*m[1][1] - m[0][1]*m[1][0];
case 3:
return m[0][0]*( m[1][1]*m[2][2] - m[2][1]*m[1][2]) m[0][1]*( m[1][0]*m[2][2] - m[2][0]*m[1][2]) +
m[0][2]*( m[1][0]*m[2][1] - m[2][0]*m[1][1]);
}
double result = 0;
for (int i = 0; i < m[0].length; i++) {
double temp[][] = new double[m.length - 1][m[0].length - 1];
for (int j = 1; j < m.length; j++) {
System.arraycopy(m[j], 0, temp[j-1], 0, i);
System.arraycopy(m[j], i+1, temp[j-1], i, m[0].length-i-1);
}
result += m[0][i] * Math.pow(-1, i) * det(temp);
}
return result;
}
}
Let's lo o k at this co de mo re clo sely. As a recursive implementatio n, there are three base cases to co nsider—
matrices o f size 1x1, 2x2, and 3x3:
OBSERVE: Determinant recursio n base cases
switch (m.length) {
case 1:
return m[0][0];
case 2:
return m[0][0]*m[1][1] - m[0][1]*m[1][0];
case 3:
return m[0][0]*( m[1][1]*m[2][2] - m[2][1]*m[1][2]) m[0][1]*( m[1][0]*m[2][2] - m[2][0]*m[1][2]) +
m[0][2]*( m[1][0]*m[2][1] - m[2][0]*m[1][1]);
}
The abo ve co des the co mputatio n as discussed earlier. Fo r matrices o f size n>=4, the co de must recreate n
sub-matrices o f size n-1 and recursively call de t with these sub-matrices, similar to the 3x3 example
described earlier.
OBSERVE: Recursive invo catio ns
double result = 0;
for (int i = 0; i < m[0].length; i++) {
double temp[][] = new double[m.length - 1][m[0].length - 1];
for (int j = 1; j < m.length; j++) {
System.arraycopy(m[j], 0, temp[j-1], 0, i);
System.arraycopy(m[j], i+1, temp[j-1], i, m[0].length-i-1);
}
result += m[0][i] * Math.pow(-1, i) * det(temp);
}
return result;
This co de pro cesses an nxn matrix by creating n smaller n-1 x n-1 sub-matrices in t e m p. The two arrayco py
invo catio ns co py the left and right side o f the smaller matrices, essentially skipping the ith co lumn with each
pass. Using the Mat h.po w(-1,i) statement, the re sult alternates between adding and subtracting the partial
co mputatio ns.
Add this metho d to the end o f the Mat rix class to so lve a linear system o f equatio ns represented by the m x
m+1 matrix used earlier:
CODE TO TYPE: Mo dificatio ns to Matrix class
public static double[] solve(double[][] mat) {
double[][] base = new double[mat.length][mat[0].length-1];
for (int i = 0; i < mat.length; i++) {
System.arraycopy(mat[i], 0, base[i], 0, mat[0].length-1);
}
double denom = det(base);
if (denom == 0) {
return null;
}
double[]solution = new double[mat.length];
for (int k = 0; k < mat.length; k++) {
for (int i=0, j=0; i < mat.length; i++, j++) {
System.arraycopy(mat[i], 0, base[i], 0, mat[0].length-1);
base[i][k] = mat[j][mat[0].length-1];
}
solution[k] = det(base)/denom;
}
return solution;
}
When yo u review implementatio ns o f mathematical algo rithms, yo u must beco me familiar with array indices
and nested lo o ps. One o f the qualities o f efficient mathematical co de is dense nested lo o ping lo gic. Let's take
a clo ser lo o k at this co de:
OBSERVE: Co mpute deno minato r determinant
double[][] base = new double[mat.length][mat[0].length-1];
for (int i = 0; i < mat.length; i++) {
System.arraycopy(mat[i], 0, base[i], 0, mat[0].length-1);
}
double denom = det(base);
if (denom == 0) {
return null;
}
The abo ve co de co nst ruct s a base array o f size m x m, where m is the number o f ro ws in the input m at
matrix. This matrix co ntains the variable co efficients o nly. If t he de t e rm inant o f t his m at rix is ze ro , there
is no unique so lutio n:
OBSERVE: Co mputing partial sums
double[]solution = new double[mat.length];
for (int k = 0; k < mat.length; k++) {
for (int i=0, j=0; i < mat.length; i++, j++) {
System.arraycopy(mat[i], 0, base[i], 0, mat[0].length-1);
base[i][k] = mat[j][mat[0].length-1];
}
solution[k] = det(base)/denom;
}
return solution;
The resulting so lutio n o f m variables is determined by co mputing the fractio ns o f determinants identified
earlier. The inne r f o r lo o p o ve r t he i variable creates an m x m matrix where t he k th co lum n in base is
re place d by t he co e f f icie nt s o f t he co nst ant co lum n, the rightmo st co lumn in the o riginal matrix,
m at .
To validate that this co de wo rks, add the m ain metho d to the end o f the Mat rix class as sho wn:
CODE TO TYPE: Main metho d to demo nstrate wo rking so lve
public static void main(String[] args) {
double[][] mat = {{1,3,5,9}, {2,7,2,2}, {1,1,4,2}};
double[] vals = solve(mat);
System.out.println("x=" + vals[0] + ", y=" + vals[1] + ", z=" + vals[2]);
}
Save and run it:
OBSERVE: So lutio n to linear equatio ns using determinants
x=-9.529411764705882, y=2.3529411764705883, z=2.2941176470588234
Co mpare these values with earlier values co mputed in the Gauss Jo rdan sectio n; they are nearly identical.
They o nly begin to differ in the last few digits. Yo u'll see this pheno meno n whenever yo u wo rk with flo ating
po int numbers.
Lessons Learned
Wo rking with flo ating-po int numbers can be suprising and challenging. Yo u need to understand the ways that
ro unding erro rs can be intro duced into co mputatio ns.
Use e psilo n-base d co ndit io nals whe n co m paring t o ze ro . When trying to co mpare a
flo ating-po int number with zero , yo u must be careful to take into acco unt the minute ro unding erro r
that o ften happens in co mputatio ns. Instead o f co mparing with equality, use two co nditio nals that
check if (x < e psilo n && x > -e psilo n) o r call Mat h.abs().
Ro unding Erro rs Can Accum ulat e : Even tho ugh ro unding erro rs by themselves are minute
values, they can rapidly accumulate thro ugh co mputatio ns, decreasing the accuracy o f yo ur
co mputatio ns if yo u do n't take time to review the co mputatio ns in yo ur co de. Since each
co mputatio n invo lves so me ro unding erro rs, try to minimize the number o f o peratio ns yo u
perfo rm. Fo r example, even tho ugh a*(b-c) is equal mathematically to a*b-a*c, the latter
co mputatio n requires three flo ating po int o peratio ns while the fo rmer co mputatio n o nly requires
two (and this wo uld be preferred in yo ur co de).
Flo at ing po int can st o re im po ssible num be rs: In flo ating po int, a co mputatio n co uld actually
divide by zero witho ut thro wing an Exceptio n. When bo th the numerato r and deno minato r are o f
type int , Java thro ws an java.lang.Arit hm e t icExce pt io n.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Brute Force Algorithms
Lesson Objectives
After co mpleting this lesso n, yo u will be able to implement brute-fo rce so lutio ns to permutatio n-style pro blems.
Using Brute Force T o Solve Permutation Problems
These pro blems have so mething in co mmo n:
What are the 5-letter wo rds that yo u can make using just the letters fo und in PALINDROME (witho ut
repetitio n)?
Generate all 4x4 magic squares using the numbers 1 thro ugh 16 where all ro ws, co lumns, and two lo ng
diago nals sum to the same value.
Ho w many ways can yo u place N queens o n an NxN chessbo ard such that no two queens attack each
o ther?
Each o f these pro blems relies o n so me co mbinato ric permutatio n o f input elements. There is a standard brute force
appro ach that can be used fo r tho se types o f pro blems, as lo ng as the size o f the pro blem instance isn't to o big. In
mathematics, a Permutation is defined fo r a set o f elements by impo sing so me particular o rder o f the elements. Fo r
example, given the set {"A", "B", "C"}, there are six permutatio ns: {"ABC", "ACB", "BAC", "BCA", "CAB", "CBA"}. To
so lve each o f the abo ve pro blems, yo u must so meho w co mpute the valid permutatio ns. In this lesso n, yo u'll see an
implementatio n that yo u can use as the structure fo r so lving such permutatio n pro blems.
Co nsider ho w to generate a 3x3 magic square using the digits 1 thro ugh 9 where all ro ws and co lumns and the two
lo ng diago nals sum to the same value. Start by writing in the numbers fro m 1 to 9 , three to a ro w fro m to p to bo tto m. Is
this a magic square? No pe. Here's an image o f six co nsecutive attempts.
These attempts were generated by backtracking fro m the initial attempt. Erase the 9 and 8 digit in the first attempt, and
instead try placing a 9 in the middle o f the bo tto m ro w. This seco nd attempt is no t a magic square either. Okay, so
backtrack and erase the 8 , 9 and 7 digits and instead place an 8 in the left co rner o f the matrix. At this po int, yo u can
generate the third attempt with "8 , 7, 9 " as the bo tto m ro w, which is still no t a so lutio n. No w erase the 9 and 7 digits
and write "8 , 9 , 7" as the bo tto m ro w o f the fo urth attempt—still no t a so lutio n, so backtrack by erasing the 7, 9 , and 8
digits and write a 9 in the left co rner o f the matrix. Yo u can generate the fifth attempt "9 , 7, 8 " as the bo tto m ro w, which
is no t a so lutio n. No w erase the 8 and 7 digits and write "9 , 8 , 7" as the sixth and final attempt—no so lutio n. This
certainly appears to be a tedio us exercise and yo u've o nly tried six o f the po ssible so lutio ns! Fo rtunately, a co mputer
pro gram can use this appro ach to try all such co nfiguratio ns.
Given ho w impo rtant backtracking is to that appro ach, expect to see recursio n play an impo rtant ro le in the algo rithm. If
yo u can divide yo ur pro blem into a finite number o f steps, the pseudo co de belo w describes a metho d so lve (int
st e p) which attempts to so lve the "next step" in a pro gressio n o f steps:
OBSERVE: pseudo co de fo r brute fo rce so lver
boolean solve(int step) {
if (step is past the final step) {
return isValid();
}
foreach possible value in step do
update state with value
if (solve(step+1)) then
return true
undo update of state
return false;
}
This recursive functio n first checks to see if t he give n st e p num be r is o ne m o re t han t he f inal st e p. If so , it's
finished, and re t urns whe t he r t he co m put e d at t e m pt is a valid so lut io n. If there are mo re steps left to be
executed, then f o r e ach po ssible value allo we d in a give n st e p, so lve () updat e s t he st at e and recursively
tries to so lve t he ne xt st e p. If so lve (st e p+1) ever returns true, then it has wo rked, and the algo rit hm st o ps
im m e diat e ly. If, ho wever, this recursive executio n returns false, the m o st re ce nt st at e updat e m ust be undo ne
so the next po ssible value at the given step can be applied. If the f o re ach lo o p co mpletes witho ut finding a so lutio n,
then the entire step fails, so the last st at e m e nt in t he pse udo co de re t urns f alse .
Each pro blem using this appro ach has its o wn isValid() metho d to determine whether the co mputed attempt so lves
the stated pro blem. Fo r the magic square pro blem, the final co de o nly needs to check that the sums o f each ro w,
co lumn, and lo ng diago nal equal the same target value.
Yo u may recall the definitio n o f the Facto rial functio n n! in mathematics, which co mputes n * (n-1) * (n-2) * ... * 2 * 1.
This functio n gro ws incredibly fast! While 9 ! is a manageable 36 2,880 , 16 ! is 20 ,9 22,7 89 ,888,0 0 0 . If yo u review the six
earlier attempts, yo u can see that this brute-fo rce appro ach is inefficient because it blindly tries all permutatio ns. It do es
have the benefit, ho wever, o f being relatively easy to write and it takes advantage o f the incredible po wer o f co mputers
to try hundreds o f tho usands o f po ssibilities per seco nd.
Create a new Java Pro ject named Brut e Fo rce and assign it to the J ava6 _Le sso ns wo rking set.
In yo ur Brut e Fo re c pro ject /src so urce fo lder, create a package pe rm ut e .
In the pe rm ut e package, create a MagicSquare class as sho wn:
CODE TO TYPE: MagicSquare
package permute;
public class MagicSquare {
final int square[][];
final boolean used[];
final int n;
public MagicSquare(int n) {
square = new int[n][n];
this.n = n;
used = new boolean[n*n+1];
}
boolean solve(int step) {
if (step == n*n) {
return isValid();
}
for (int val = 1; val <= n*n; val++) {
if (used[val]) { continue; }
used[val] = true;
square[step/n][step%n] = val;
if (solve(step+1)) {
return true;
}
square[step/n][step%n] = 0;
used[val] = false;
}
return false;
}
boolean validUpTo(int step) {
for (int r = 0; r < n; r++) {
if (step == (r+1)*n-1) {
int sum = 0;
for (int c = 0; c < n; c++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
for (int c = 0; c < n; c++) {
if (step == n*(n-1)+c) {
int sum = 0;
for (int r = 0; r < n; r++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
return true;
}
}
Let's lo o k clo ser:
OBSERVE:
public class MagicSquare {
final int square[][];
final boolean used[];
final int n;
public MagicSquare(int n) {
square = new int[n][n];
this.n = n;
used = new boolean[n*n+1];
}
The state o f the algo rithm is co ntained in two arrays: use d[n] reco rds whether the number n already appears
so mewhere in the magic square. square [r][c] sto res the number at the given ro w and co lumn index lo catio n. The
MagicSquare co nstructo r determines the desired pro blem size, n. The size o f use d[] is o ne larger than necessary
because the numbers in the magic square are fro m 1 to n*n.
OBSERVE:
boolean solve(int step) {
if (step == n*n) {
return isValid();
}
for (int val = 1; val <= n*n; val++) {
if (used[val]) { continue; }
used[val] = true;
square[step/n][step%n] = val;
if (solve(step+1)) {
return true;
}
square[step/n][step%n] = 0;
used[val] = false;
}
return false;
}
The real lo gic is in the so lve (int st e p) metho d. The first invo catio n o f this metho d must be so lve (0 ). Once all
numbers have been placed (when st e p is n*n o r o ne mo re than the number o f steps when co unting fro m zero ), the
metho d determines whether the state o f the magic square is a valid so lutio n using the isValid() metho d (which we'll
write sho rtly).
To determine "fo reach po ssible value in step," this co de re lie s o n t he use d[n] array so it do e sn't place t he
sam e num be r m ult iple t im e s in t he m agic square . To "update state with value," the co de reco rds
use d[val]=t rue and place s t he value in t he m agic square at t he appro priat e ro w and co lum n. This co de
uses a co mmo n idio m to co nvert a simple number into a two -dimensio nal ro w and co lumn placement. The integer
co mputatio n st e p/n pro perly truncates the step number to co mpute the ro w value, while st e p% n uses mo dulo
arithmetic to determine the pro per co lumn. Once the state is updated, it recursively calls so lve (st e p+1); if this metho d
returns t rue , a so lutio n has been fo und. Otherwise, it must undo t he st at e change (bo t h in use d and square )
befo re co ntinuing the f o r lo o p. If this lo o p co mpletes witho ut having fo und a so lutio n, the metho d re t urns f alse and
backtracks to the previo us step.
To co mplete the implementatio n o f MagicSquare , make the changes belo w. They'll take advantage o f the
mathematical fact that the target sum value fo r an n x n magic square is n*(n*n+1)/2. So , fo r a 3x3 magic square, the
sum o f each ro w, co lumn and diago nal is 3*(3*3+1)/2 = 3*10 /2 = 15 .
CODE TO TYPE: Mo dify MagicSquare
package permute;
public class MagicSquare {
final int square[][];
final boolean used[];
final int n;
final int magicSum;
public MagicSquare(int n) {
square = new int[n][n];
this.n = n;
used = new boolean[n*n+1];
magicSum = n*(n*n+1)/2;
}
boolean isValid() {
int sumD1 = 0;
int sumD2 = 0;
for (int i = 0; i < n; i++) {
int sumR = 0;
int sumC = 0;
sumD1 += square[i][i];
sumD2 += square[i][n-i-1];
for (int j = 0; j < n; j++) {
sumR += square[i][j];
sumC += square[j][i];
}
if (sumR != magicSum || sumC != magicSum) { return false; }
}
// diagonals
return (sumD1 == magicSum && sumD2 == magicSum);
}
boolean solve(int step) {
if (step == n*n) {
return isValid();
}
for (int val = 1; val <= n*n; val++) {
if (used[val]) { continue; }
used[val] = true;
square[step/n][step%n] = val;
if (solve(step+1)) {
return true;
}
square[step/n][step%n] = 0;
used[val] = false;
}
return false;
}
boolean validUpTo(int step) {
for (int r = 0; r < n; r++) {
if (step == (r+1)*n-1) {
int sum = 0;
for (int c = 0; c < n; c++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
for (int c = 0; c < n; c++) {
if (step == n*(n-1)+c) {
int sum = 0;
for (int r = 0; r < n; r++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
return true;
}
public void outputSolution () {
for (int r = 0; r < n; r++) {
for (int c = 0; c < n; c++) {
System.out.print(square[r][c]);
System.out.print(' ');
}
System.out.println();
}
System.out.println();
}
}
The o ut put So lut io n() metho d prints o ut the two -dimensio nal square using the values in square . The isValid()
metho d do es the real wo rk, using a nested f o r lo o p to co mpute the sum o f each ro w, co lumn, and lo ng diago nal. If
any sum fails to match m agicSum , the isValid() metho d returns f alse , which fo rces the so lve () metho d to backtrack
and try to find ano ther so lutio n.
To validate this so lutio n, write this perfo rmance co de:
In yo ur Brut e Fo rce pro ject, create a /pe rf o rm ance so urce fo lder.
In the /pe rf o rm ance so urce fo lder, create a pe rm ut e package.
In the pe rm ut e package, create a Main class as sho wn:
CODE TO TYPE: Main class
package permute;
public class Main {
public static void main(String[] args) {
MagicSquare m = new MagicSquare(3);
m.solve(0);
m.outputSolution();
}
}
Save and run Main.
OBSERVE: Output fro m Main fo r 3x3 magic square
2 7 6
9 5 1
4 3 8
This is a valid 3x3 magic square because all ro ws, co lumns, and lo ng diago nals sum to 15.
Befo re go ing further, sto p and think abo ut ho w many 3x3 magic square so lutio ns might exist. This is a natural
extensio n to the pro blem. Yo u can determine ho w many so lutio ns there are by adding a co unt() metho d that makes a
small mo dificatio n to the basic algo rithm. Mo dify MagicSquare as sho wn:
CODE TO TYPE: Updates to MagicSquare class
int total = 0;
...
void count(int step) {
if (step == n*n) {
if (isValid()) {
total++;
outputSolution();
}
return;
}
for (int val = 1; val <= n*n; val++) {
if (used[val]) { continue; }
used[val] = true;
square[step/n][step%n] = val;
if (validUpTo(step)) {
count(step+1);
}
square[step/n][step%n] = 0;
used[val] = false;
}
}
Let's lo o k clo ser:
OBSERVE:
int total = 0;
void count(int step) {
if (step == n*n) {
if (isValid()) {
total++;
outputSolution();
}
return;
}
for (int val = 1; val <= n*n; val++) {
if (used[val]) { continue; }
used[val] = true;
square[step/n][step%n] = val;
count(step+1);
square[step/n][step%n] = 0;
used[val] = false;
}
}
The co unt(step) metho d sto res the to tal number o f such magic squares fo und in an attribute, t o t al. The structure o f
this metho d is nearly identical to so lve (st e p), except that it do esn't sto p lo o king fo r so lutio ns when the first o ne is
fo und. Acco rdingly, this metho d is no w defined as vo id co unt (int st e p). So , when the final step is reached and
isValid() validates a so lutio n, the co unt () metho d incre m e nt s t he t o t al co unt and o ut put s t he so lut io n t o
t he scre e n. The recursive call always executes and the co de backtracks after every recursive invo catio n. To gether,
these changes ensure that all so lutio ns are inspected.
Mo dify Main as sho wn:
CODE TO TYPE: Main class
package permute;
public class Main {
public static void main(String[] args) {
MagicSquare m = new MagicSquare(3);
m.solve(0);
m.outputSolution();
m = new MagicSquare(3);
m.count(0);
System.out.println("There are " + m.total + " possible squares.");
}
}
Save and run it.
INTERACTIVE SESSION: Output fro m revised Main
2 7 6
9 5 1
4 3 8
2 7 6
9 5 1
4 3 8
2 9 4
7 5 3
6 1 8
4 3 8
9 5 1
2 7 6
4 9 2
3 5 7
8 1 6
6 1 8
7 5 3
2 9 4
6 7 2
1 5 9
8 3 4
8 1 6
3 5 7
4 9 2
8 3 4
1 5 9
6 7 2
There are 8 possible squares.
These magic squares are all really the same so lutio n ro tated (and flipped) to pro duce eight different versio ns o f the
same magic square.
It's amazing ho w quickly this pro gram co mputed these so lutio ns. Ho wever, if yo u change this to so lve a 4x4 magic
square, yo u'll have to wait a while lo nger fo r so lutio ns. Ho w lo ng? Well, a 4x4 magic square requires 16 ! permutatio ns,
o r 20 ,9 22,7 89 ,888,0 0 0 ; as a ro ugh estimate, it will take 5,76 5,76 0 times as lo ng to generate all 4x4 so lutio ns as it did
fo r the 3x3 so lutio n. Clearly, yo u have to o ptimize this appro ach, o r yo u'll never be able to co mpute even slightly larger
pro blems.
Fo rtunately, yo u can mo dify the so lve () metho d to search thro ugh the so lutio n set mo re intelligently. Basically,
instead o f blindly pursuing each recursive step, yo u can validate partial results o f the search befo re go ing fo rward.
Revise MagicSquare as sho wn:
CODE TO TYPE: Revised MagicSquare
package permute;
public class MagicSquare {
final int square[][];
final boolean used[];
final int n;
final int magicSum;
int total = 0;
public MagicSquare(int n) {
square = new int[n][n];
this.n = n;
used = new boolean[n*n+1];
magicSum = n*(n*n+1)/2;
}
// handles only rows and columns
boolean validUpTo(int step) {
for (int r = 0; r < n; r++) {
if (step == (r+1)*n-1) {
int sum = 0;
for (int c = 0; c < n; c++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
for (int c = 0; c < n; c++) {
if (step == n*(n-1)+c) {
int sum = 0;
for (int r = 0; r < n; r++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
return true;
}
boolean isValid() {
int sumD1 = 0;
int sumD2 = 0;
for (int i = 0; i < n; i++) {
int sumR = 0;
int sumC = 0;
sumD1 += square[i][i];
sumD2 += square[i][n-i-1];
for (int j = 0; j < n; j++) {
sumR += square[i][j];
sumC += square[j][i];
}
if (sumR != magicSum || sumC != magicSum ) { return false; }
}
// diagonals
return (sumD1 == magicSum
&& sumD2 == magicSum );
}
boolean solve(int step) {
if (step == n*n) {
return isValid();
}
for (int val = 1; val <= n*n; val++) {
if (used[val]) { continue; }
used[val] = true;
square[step/n][step%n] = val;
if (validUpTo(step) && solve(step+1)) {
return true;
}
square[step/n][step%n] = 0;
used[val] = false;
}
return false;
}
boolean validUpTo(int step) {
for (int r = 0; r < n; r++) {
if (step == (r+1)*n-1) {
int sum = 0;
for (int c = 0; c < n; c++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
for (int c = 0; c < n; c++) {
if (step == n*(n-1)+c) {
int sum = 0;
for (int r = 0; r < n; r++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
return true;
}
public void outputSolution () {
for (int r = 0; r < n; r++) {
for (int c = 0; c < n; c++) {
System.out.print(square[r][c]);
System.out.print(' ');
}
System.out.println();
}
System.out.println();
}
void count(int step) {
if (step == n*n) {
if (isValid()) {
total++;
outputSolution();
}
return;
}
for (int val = 1; val <= n*n; val++) {
if (used[val]) { continue; }
used[val] = true;
square[step/n][step%n] = val;
if (validUpTo(step)) {
count (step+1);
}
square[step/n][step%n] = 0;
used[val] = false;
}
}
}
Let's take a clo ser lo o k at the validUpT o () metho d:
OBSERVE: validUpTo metho d
// handles only rows and columns
boolean validUpTo(int step) {
for (int r = 0; r < n; r++) {
if (step == (r+1)*n-1) {
int sum = 0;
for (int c = 0; c < n; c++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
for (int c = 0; c < n; c++) {
if (step == n*(n-1)+c) {
int sum = 0;
for (int r = 0; r < n; r++) { sum += square[r][c]; }
return (sum == magicSum);
}
}
return true;
}
The algo rithm partially reviews its pro gress after filling each ro w and co lumn co mpletely. Given a st e p numbered fro m
0 to n*n-1, this means that whe ne ve r st e p e quals (r+1)*n-1 fo r so me ro w numbered 0 .. n-1, there is eno ugh
info rmatio n to determine if the sum to tal o f the ro w is m agicSum . Similarly, whe ne ve r st e p e quals n*(n-1)+c fo r
so me co lumn numbered 0 .. n-1, there is eno ugh info rmatio n to determine if the sum to tal o f the co lumn is
m agicSum .
With these changes in place, mo dify Main to co unt the number o f 4x4 magic squares, as sho wn:
CODE TO TYPE: Revised Main class
package permute;
public class Main {
public static void main(String[] args) {
MagicSquare m = new MagicSquare(34);
m.solve(0);
m.outputSolution();
m = new MagicSquare(34);
m.count(0);
System.out.println("There are " + m.total + " possible squares.");
}
}
Save and run it. The first 4x4 magic square appears almo st immediately:
OBSERVE: First co mputed magic square
1 2 15 16
12 14 3 5
13 7 10 4
8 11 6 9
If yo u let the pro gram run fo r abo ut three mo re minutes, it repo rts that 7 ,0 4 0 4x4 magic squares were fo und. Yo u kno w
that each magic square appears 8 times in this set (ro tated and flipped); this means there are 8 8 0 unique 4x4 magic
squares. The revised co de prints each o f these so lutio ns.
Of co urse, yo u can't use this appro ach fo r 5x5 magic squares (which have 1.5 x 10 25 po ssible so lutio ns). Yo u co uld
try to run this example o n yo ur o wn co mputer. After 41 ho urs o f co mputatio n o n the 5x5 so lutio n, this so lutio n sho ws
up:
INTERACTIVE SESSION: So lutio n fo r 5x5 fo und
Sun Sep 29 13:33:58 EDT 2013
1 2 13 24 25
3 22 19 6 15
23 16 10 11 5
21 7 9 20 8
17 18 14 4 12
Tue Oct 01 06:21:05 EDT 2013
Even so , this po werful technique can be used to so lve many small, "human-scale" pro blems in which yo u might be
interested.
Finding All Five-Letter words in PALINDROME
No w, use this same algo rithm to determine the five-letter wo rds that yo u can make using just the letters fo und in the
wo rd "PALINDROME." Perhaps yo u can already see ho w to apply the described algo rithm to so lve this pro blem.
Using the same pseudo co de fro m befo re, create this Wo rdFinde r class.
In the pe rm ut e package o f the /src so urce fo lder, create a Wo rdFinde r class as sho wn:
CODE TO TYPE: Wo rdFinder
package permute;
import java.util.*;
public class WordFinder {
final char[] letters;
final boolean[] used;
final int n;
char[] solution;
Set<String> results;
public WordFinder (String word) {
letters = word.toCharArray();
Arrays.sort(letters);
n = letters.length;
used = new boolean[n];
}
public void generate (int numChars) {
solution = new char[numChars];
results = new TreeSet<String>();
generate(numChars, 0);
}
void generate(int numChars, int step) {
if (step == numChars) {
results.add(new String(solution));
return;
}
for (int idx = 0; idx < n; idx++) {
if (used[idx]) { continue; }
used[idx] = true;
solution[step] = letters[idx];
generate(numChars, step+1);
solution[step] = 0;
used[idx] = false;
}
}
}
Let's break this so lutio n up into its co nstituent parts:
OBSERVE: Instantiating the Wo rdFinder pro blem
public class WordFinder {
final char[] letters;
final boolean[] used;
final int n;
char[] solution;
Set<String> results;
public WordFinder (String word) {
letters = word.toCharArray();
Arrays.sort(letters);
n = letters.length;
used = new boolean[n];
}
public void generate (int numChars) {
solution = new char[numChars];
results = new TreeSet<String>();
generate(numChars, 0);
}
...
}
The le t t e rs array co ntains the letters fro m the o riginal wo rd in so rted o rder. If the o riginal wo rd repeats a letter, that
letter will appear multiple times in this array. The use d array keeps track o f which letters have been already used, and
the re sult s o bject sto res the set o f all co mputed wo rds fo und.
The ge ne rat e (int num Chars) metho d allo ws yo u to use this o bject repeatedly to generate all wo rds that use a given
number o f characters. It se t s t he size o f t he so lut io n array base d o n t he de sire d num be r o f charact e rs, and
it instantiates re sult s using a T re e Se t o bject; this is do ne so it can pro duce the wo rds in so rted o rder rapidly when
requested. This metho d uses the recursive ge ne rat e (num Chars,st e p) metho d sho wn:
OBSERVE: Generating all wo rds using a given number o f characters
void generate(int numChars, int step) {
if (step == numChars) {
results.add(new String(solution));
return;
}
for (int idx = 0; idx < n; idx++) {
if (used[idx]) { continue; }
used[idx] = true;
solution[step] = letters[idx];
generate(numChars, step+1);
solution[step] = 0;
used[idx] = false;
}
}
This recursive functio n terminates when the st e p num be r is t he sam e as t he num be r o f de sire d charact e rs.
num Chars is passed thro ugh as an unchanged parameter to each o f the recursive invo catio ns. The so lut io n[] array
sto res the permutatio n o f letters. Whenever an appro priate wo rd is fo und, it is adde d t o t he re sult s se t . Let's try to
run this co de.
Create a MainWo rdFinde r class in the /pe rf o rm ance so urce fo lder:
CODE TO TYPE: MainWo rdFinder demo nstratio n class
package permute;
public class MainWordFinder {
public static void main(String[] args) {
WordFinder wf = new WordFinder("PALINDROME");
wf.generate(5);
System.out.println("There are " + wf.results.size() + " five letter words possible.
");
System.out.println("First ten are:");
int idx = 10;
for (String word : wf.results) {
System.out.println(word);
if (--idx == 0) { break; }
}
}
}
Save and run it.
INTERACTIVE SESSION: Output fro m MainWo rdFinder
There are 30240 five letter words possible.
The first ten words are:
ADEIL
ADEIM
ADEIN
ADEIO
ADEIP
ADEIR
ADELI
ADELM
ADELN
ADELO
N Queens Problem
As a final example, co nsider the N Queens problem, which asks yo u to place N queens o n an NxN chessbo ard such
that no two queens attack each o ther. Fo r N >= 4 the pro blem always has a so lutio n, but it may take yo u so me time to
determine that so lutio n. If yo u co uld o nly find so me way to co nvert this pro blem into a permutatio n pro blem, then yo u
co uld use the existing algo rithm to so lve it. Start by breaking the pro blem into N steps, placing a no n-attacking queen,
o ne at a time, into each o f the N co lumns o n the chessbo ard. With this appro ach, yo u need so me permutatio n o f
queen placements in each co lumn. Let's get started.
In the /src so urce fo lder pe rm ut e package, create an NQue e nsPro ble m class as sho wn:
CODE TO TYPE: NQueensPro blem
package permute;
public class NQueensProblem {
final int n;
final int solution[];
public NQueensProblem(int n) {
this.n = n;
solution = new int[n];
}
public void solve() {
solve(0);
outputSolution();
}
public int count() {
total = 0;
count(0);
return total;
}
int total = 0;
void count(int column) {
// TBA
}
boolean solve(int column) {
if (column == n) {
return true;
}
// TBA
return false;
}
public void outputSolution () {
for (int r = 0; r < n; r++) {
for (int c = 0; c < n; c++) {
if (solution[c] == r) {
System.out.print("Q");
} else {
if ((r-c) %2 == 0) {
System.out.print(" ");
} else {
System.out.print(".");
}
}
}
System.out.println();
}
System.out.println();
}
}
Using the mo del o f placing a queen in each co lumn, the so lut io n[i] array will reco rd the ro w value (0 .. n-1) o f each
queen placed in the i t h co lumn. Ho w many ways can yo u place N queens o n an NxN chess bo ard? Well, there are N*N
squares and fro m these yo u cho o se N squares. When cho o sing B elements fro m a larger set o f unique A elements,
the mathematical fo rmula to use is A!/(B!*(A-B)!). This number is actually far smaller than the to tals we were dealing
with earlier. Fo r example, given an 8 x8 chess bo ard o n which to place 8 queens, the abo ve fo rmula is 6 4 !/(8!*5 6 !) o r
6 4 *6 3*6 2*6 1*6 0 *5 9 *5 8*5 7 /8*7 *6 *5 *4 *3*2*1, which equals 4 ,4 26 ,16 5 ,36 8.
We can make this co de even mo re efficient by placing o nly no n-attacking queens at each step instead o f placing N
queens in the N co lumns, and only then checking whether any two queens attack. Do ing this efficiently can be a bit
tricky. This co de creates three arrays to keep track o f impo rtant state info rmatio n:
CODE TO TYPE:
...
public class NQueensProblem {
final int n;
final boolean usedRow[];
final boolean usedDiagonalNE[];
final boolean usedDiagonalNW[];
final int solution[];
public NQueensProblem(int n) {
this.n = n;
solution = new int[n];
usedRow = new boolean[n];
usedDiagonalNE = new boolean[2*n-1];
usedDiagonalNW = new boolean[2*n-1];
}
...
use dRo w[i] reco rds if a queen is placed in ro w i.
use dDiago nalNE[i] reco rds if a queen is placed in o ne o f the no rtheast diago nals o n the bo ard.
use dDiago nalNW[i] reco rds if a queen is placed in o ne o f the no rthwest diago nals o n the bo ard.
There are n elements in use dRo w, but 2*n-1 elements in each o f the diago nal arrays. When placing a queen at square
(co lum n, ro w), the co de reco rds that use dRo w[ro w] is true. Given the co o rdinates fo r co lum n and ro w, o bserve
that fo r all squares o n the i th no rtheast diago nal fo r i in the range 0 .. 2*n-1, the sum o f ro w+co lum n equals i. Fo r
squares o n the no rthwest diago nals, o bserve that the difference o f index values ro w-co lum n is in the range (-n+1) ..
(n-1), so to no rmalize this array, the co de uses ro w-co lum n+n-1 as the index values fo r i in the range 0 .. 2*n-1 into
use dDiago nalNW[i].
Revise the so lve (int co lum n) metho d as sho wn belo w. Yo u'll se that the recursive call is made o nly when it is clear
that placing the queen at (co lum n, ro w) do es no t attack any existing queen o n the bo ard. Once placed, these arrays
are updated prio r to the recursive invo catio n; they are reset after the invo catio n:
CODE TO TYPE: Revised so lve() metho d
boolean solve(int column) {
if (column == n) {
return true;
}
// TBA
for (int row = 0; row < n; row++) {
if (usedRow[row]) { continue; }
if (usedDiagonalNW[row-column+n-1]) { continue; }
if (usedDiagonalNE[row+column]) { continue; }
usedRow[row] = true;
usedDiagonalNW[row-column+n-1] = true;
usedDiagonalNE[row+column] = true;
solution[column] = row;
if (solve(column+1)) {
return true;
}
usedDiagonalNE[row+column] = false;
usedDiagonalNW[row-column+n-1] = false;
usedRow[row] = false;
}
return false;
}
The recursive metho d terminates when all co lumns have a queen. If there are mo re co lumns to pro cess, the f o r lo o p
runs thro ugh all po ssible ro w indices to find o ne that do esn't currently co ntain a queen (as determined by the
use dRo w array). Ho wever, two queens can attack diago nally, so two different arrays are used to reco rd whether there
is already a queen o n any o f the no rtheast diago nals o r no rthwest diago nals.
This co de co mpletes the co unt (int co lum n) implementatio n:
CODE TO TYPE: Mo dified co unt() metho d
void count(int column) {
// TBA
if (column == n) {
total++;
return;
}
for (int row = 0; row < n; row++) {
if (usedRow[row]) { continue; }
if (usedDiagonalNW[row-column+n-1]) { continue; }
if (usedDiagonalNE[row+column]) { continue; }
usedRow[row] = true;
usedDiagonalNW[row-column+n-1] = true;
usedDiagonalNE[row+column] = true;
solution[column] = row;
count(column+1);
usedDiagonalNE[row+column] = false;
usedDiagonalNW[row-column+n-1] = false;
usedRow[row] = false;
}
}
Write so me validating co de to demo nstrate the pro per executio n o f this metho d.
Create a MainNQue e ns class in the pe rm ut e package o f the /pe rf o rm ance so urce fo lder:
CODE TO TYPE: MainNQueens class
package permute;
public class MainNQueens {
public static void main(String[] args) {
for (int i = 8; i < 10; i++) {
NQueensProblem nqp = new NQueensProblem(i);
nqp.solve();
System.out.println("---------------------------------");
}
for (int i = 4; i < 14; i++) {
NQueensProblem nqp = new NQueensProblem(i);
System.out.println(i + ". " + nqp.count());
}
}
}
Save and run it.
OBSERVE: Output fro m MainNQueens
Q. . . .
. . . Q
. .Q. .
. . . .Q
Q . . .
. .Q. .
. . Q .
. Q . .
--------------------------------Q. . . .
. . Q . .
Q . . .
. . .Q. .
. . . .Q
. Q . . .
. . . Q
. .Q. . .
. . .Q.
--------------------------------4. 2
5. 10
6. 4
7. 40
8. 92
9. 352
10. 724
11. 2680
12. 14200
13. 73712
The table at the end o f the o utput reco rds the to tal number o f unique bo ards fo r different values o f n. Yo u can validate
that these are co rrect by co mparing against well-kno wn tables o f these values, like tho se fo und at the On-Line
Encyclo pedia o f Integer Sequences.
Lessons Learned
There are many permutatio n-style pro blems that can be so lved in small pro blem instances using a brutefo rce appro ach. While the technique do esn't scale to larger pro blem instances, it can be a useful "cure-all"
when no kno wn algo rithm exists, o r yo u just want to co nduct a quick search to see if so me so lutio n exists.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Path Finding for Single-Player Games
Lesson Objectives
After co mpleting this lesso n, yo u will be able to :
draw a search tree (o f a fixed depth) fo r a so litaire game.
design classes to represent the state in a so litaire game.
design mo ve classes to represent allo wable mo ves.
explain the difference between a Breadth First and Depth First search tree.
Path Finding For Single-Player Games
8 -puzzle is a so litaire game fo rmed using a three-by-three grid co ntaining eight square tiles numbered 1 to 8 and an
empty space that co ntains no tile. A tile adjacent (either ho rizo ntally o r vertically) to the empty space can be mo ved by
sliding it into the empty space. The aim is to start fro m a shuffled initial state and mo ve tiles to achieve a go al state (fo r
example, with the numbers in clo ckwise o rder fro m the upper left co rner, with the middle square being empty). There
are no co mpeting players taking alternate turns fo r these pro blems, but the behavio r is quite similar to game trees.
A search tree represents the set o f intermediate bo ard states as the path-finding algo rithm pro gresses. To be as
efficient as po ssible, the path-finding algo rithm must avo id visiting the same bo ard state twice, o therwise it might get
stuck in an infinite repetitio n o f useless mo ves. The result o f the co mputed search structure is a tree because the
algo rithm ensures that it do es no t visit a bo ard state twice. The algo rithm decides the o rder o f bo ard states to visit as it
attempts to reach its go al.
In o rder to write a pro gram to so lve 8 -puzzle, yo u need a class that represents the bo ard state.
Create a new Java Pro ject named Single Playe r and assign it to the J ava6 _Le sso ns wo rking set.
In the /src so urce fo lder, create a puzzle package.
In the puzzle package, create a Bo ard class as sho wn:
CODE TO TYPE: Bo ard class
package puzzle;
public class Board {
int[][] tiles;
public Board (int[][] initial) {
tiles = new int[3][3];
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
tiles[r][c] = initial[r][c];
}
}
}
public Board (Board b) {
tiles = new int[3][3];
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
tiles[r][c] = b.tiles[r][c];
}
}
}
public String toString() {
StringBuilder sb = new StringBuilder();
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
if (tiles[r][c] == 0) {
sb.append('-');
} else {
sb.append(tiles[r][c]);
}
}
sb.append('\n');
}
return sb.toString();
}
}
A two -dimensio nal array o f int values, t ile s, sto res the lo catio n o f each tile, using the number 0 to represent an
empty tile. The first co nstructo r passes in a sample state representatio n with three ro ws o f three co lumns each. The
seco nd co nstructo r makes a co py o f a Bo ard o bject. The t o St ring() metho d prepares a human-readable string
depicting the bo ard.
The mo st co mmo n o peratio n o n a bo ard state is to determine the valid mo ves where the player can slide a tile—
ho rizo ntally o r vertically—into the adjacent empty space. When the empty space is in the middle o f the bo ard, there are
fo ur po ssible mo ves, but when the empty space is in o ne o f the co rners o f the bo ard, o nly two mo ves are available.
If yo u have ever played a so litaire puzzle, yo u kno w ho w frustrating it can be to make (o ften rando m) lo ng sequences
o f mo ves witho ut kno wing whether yo u are actually making real pro gress to wards so lving the pro blem! Yo u co uld
prevent repeating a series o f mo ves if o nly yo u co uld remember whether yo u had visited a bo ard state previo usly. In
fact the algo rithm presented here demands this functio nality. The ability to detect whether a state has been visited is
directly analo go us to the ability to co lo r vertices in a graph during Depth-First Search.
Let's try to so lve a sample 8-puzzle pro blem instance. The initial state reading fro m left to right is: {{1,2,3} , {4 ,0 ,6 } ,
{7 ,5 ,8} } where 0 represents the empty square in the middle o f the bo ard. Let's make the final go al state {{1,2,3} ,
{8,0 ,4 } , {7 ,6 ,5 } } (highlighted in yello w belo w). The search tree belo w sho ws all six po ssible bo ard states reachable in
two o r fewer mo ves fro m the initial state. The no des o f a search tree are the bo ard states.
Fro m the initial state (depth 0 ), there are two po ssible mo ves that result in two new bo ard states o n depth 1. In each o f
these two bo ard states, there are three mo ves; ho wever, depth 2 has o nly two children states fo r each. That's because
the search tree never contains any board states that have already been visited. The go al o f this lesso n is to
demo nstrate ho w to auto mate this pro cess to determine the sequence o f mo ves that leads fro m the initial state to a
desired go al state.
We've already used Depth-First Search to search thro ugh a graph. In this case, the algo rithm co nstructs a search tree
by blindly explo ring ahead, cho o sing mo ves to make fro m the available set o f mo ves in each bo ard state. When using
Depth-First Search o n an actual graph, the algo rithm backtracks when it runs o ut o f new vertices to visit. Ho wever, the
search tree co ntinually expands as yo u execute mo ves to unco ver new bo ard states to explo re. To avo id having a
"runaway" DepthFirstSearch, let's intro duce a new parameter, m axDe pt h, which determines the maximum depth to
explo re a particular branch o f the search tree. If it do esn't reach the go al state by this maximum depth, it begins to
backtrack to try alternate sequences o f mo ves.
To keep track o f the so lutio n thro ugh the search tree, yo u need to make so me mo dificatio ns to the Bo ard class
structure:
CODE TO TYPE: Mo dificatio ns to Bo ard class
package puzzle;
import java.util.*;
public class Board {
int[][] tiles;
Board previous;
int depth;
...
}
The de pt h attribute reco rds the depth o f a Bo ard in the search tree, while pre vio us will be a link to the previo us Bo ard
in the search tree, which will always be the parent bo ard state fo r each bo ard state in the search tree.
This pseudo co de presents an appro ach to so lve so litaire puzzles like the 8 -puzzle:
OBSERVE: pseudo co de fo r so lving so litaire puzzles with depth restrictio n
search(initial, goal, maxDepth)
solution = {}
if initial is goal then return "Solution"
open = new Set
closed = new Set
insert (open, initial)
while open is not empty do
board = select state from open
insert (closed, board)
foreach valid move m at board do
next = board state after playing m
if closed doesn't contain next then
if next = goal then return "Solution"
else if not exceeded maxDepth then
insert (open, next)
return "No Solution"
The algo rithm maintains o pe n and clo se d sets to guide the search. o pe n represents the active ho rizo n o f the
search, and co ntains bo ard states that will be explo red. clo se d remembers the past bo ard states that were visited.
The algo rithm pro ceeds by se le ct ing a bo ard st at e f ro m o pe n t o e xplo re . Then it ge ne rat e s po t e nt ial bo ard
st at e s t o visit base d o n t he available m o ve s. If it hasn't e xce e de d it s m axim um de pt h and the ne wly
ge ne rat e d bo ard st at e s have n't be e n visit e d, the bo ard st at e s are inse rt e d int o t he o pe n co lle ct io n.
The behavio r o f the search algo rithm changes based o n the strategy used to decide the next bo ard state in o pe n to
pro cess. As yo u've seen in past lesso ns, if yo u use a Stack to sto re the o pen states, the algo rithm pursues a DepthFirst Search appro ach; if yo u use a Queue, the algo rithm implements Breadth-First Search.
To co mplete the implementatio n o f 8 -puzzle yo u need to create a class to represent a valid mo ve.
In the puzzle package, create a Slide Mo ve class as sho wn:
CODE TO TYPE: SlideMo ve class
package puzzle;
public class SlideMove {
final int fromR, fromC;
final int toR, toC;
public SlideMove (int fromR, int fromC, int toR, int toC) {
this.fromR = fromR;
this.fromC = fromC;
this.toR = toR;
this.toC = toC;
}
public boolean execute(Board b) {
if (!isValid(b)) { return false; }
b.swap(fromR, fromC, toR, toC);
return true;
}
public boolean isValid(Board b) {
if (fromR < 0 || fromR >= 3) { return false; }
if (fromC < 0 || fromC >= 3) { return false; }
if (toR < 0 || toR >= 3) { return false; }
if (toC < 0 || toC >= 3) { return false; }
return b.isAdjacentAndEmpty(fromR, fromC, toR, toC);
}
}
Let's lo o k at this class mo re clo sely. It wo n't co mpile just yet, but yo u'll so o n add the required new metho ds to the
Bo ard class.
OBSERVE:
public class SlideMove {
final int fromR, fromC;
final int toR, toC;
public SlideMove (int fromR, int fromC, int toR, int toC) {
this.fromR = fromR;
this.fromC = fromC;
this.toR = toR;
this.toC = toC;
}
public boolean execute(Board b) {
if (!isValid(b)) { return false; }
b.swap(fromR, fromC, toR, toC);
return true;
}
public boolean isValid(Board b) {
if (fromR < 0 || fromR >= 3) { return false; }
if (fromC < 0 || fromC >= 3) { return false; }
if (toR < 0 || toR >= 3) { return false; }
if (toC < 0 || toC >= 3) { return false; }
return b.isAdjacentAndEmpty(fromR, fromC, toR, toC);
}
}
A Slide Mo ve represents the mo vement o f a tile fro m a given (f ro m R, f ro m C) lo catio n to a destinatio n (t o R, t o C)
lo catio n. Such a mo ve is valid if t he inde x value s are all valid, t he re is an e m pt y square in t he (t o R, t o C)
lo cat io n, and t he t wo lo cat io ns are ne ighbo rs. The mo ve executes by swapping t he co nt e nt s o f t he t wo
lo cat io ns.
Add these two metho ds to the end o f the Bo ard class to enable Slide Mo ve to co mpile:
CODE TO TYPE: Add metho ds to end o f Bo ard class
public boolean isAdjacentAndEmpty(int fromR, int fromC, int toR, int toC) {
if (tiles[toR][toC] != 0) { return false; }
int dC = Math.abs(fromR-toR);
int dR = Math.abs(fromC-toC);
if ((dC == -1 && dR == 0) || (dC == +1 && dR == 0) ||
(dC == 0 && dR == -1) || (dC == 0 && dR == +1)) {
return true;
}
return false;
}
public void swap (int fromR, int fromC, int toR, int toC) {
int tmp = tiles[toR][toC];
tiles[toR][toC] = tiles[fromR][fromC];
tiles[fromR][fromC] = tmp;
}
OBSERVE:
public boolean isAdjacentAndEmpty(int fromR, int fromC, int toR, int toC) {
if (tiles[toR][toC] != 0) { return false; }
int dC = Math.abs(fromR-toR);
int dR = Math.abs(fromC-toC);
if ((dC == -1 && dR == 0) || (dC == +1 && dR == 0) ||
(dC == 0 && dR == -1) || (dC == 0 && dR == +1)) {
return true;
}
return false;
}
The swap metho d swaps the lo catio n o f two tiles in the bo ard. isAdjace nt AndEm pt y perfo rms so me calculatio ns to
determine if (t o R, t o C) represents an empty square neighbo ring ano ther lo catio n (f ro m R, f ro m C) either ho rizo ntally
o r vertically. Variables dC and dR co mpute the difference o f the indices and a ne ighbo r is just o ne co lum n o r ro w
away (but no t bo t h).
The algo rithm needs to kno w the valid mo ves at any given bo ard state. Add the attribute belo w and validMo ve s()
metho d to Bo ard to return a List o f the available sliding mo ves at that state:
CODE TO TYPE: Add attribute and metho d to the end o f the Bo ard class
static int deltas[][] = {{+1, 0}, {0, -1}, {-1, 0}, {0, 1}};
public List<SlideMove> validMoves() {
int br = -1, bc = -1;
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
if (tiles[r][c] == 0) {
br = r;
bc = c;
}
}
}
ArrayList<SlideMove> list = new ArrayList<SlideMove>();
for (int i = 0; i < deltas.length; i++) {
int dr = deltas[i][0];
int dc = deltas[i][1];
SlideMove sm = new SlideMove (br+dr, bc+dc, br, bc);
if (sm.isValid(this)) { list.add(sm); }
}
return list;
}
This metho d determines the ro w and co lumn o f the empty space and then co nstructs an ArrayList o bject that
represents the the available valid mo ves, using the Slide Mo ve class designed earlier.
All the pieces are no w ready to implement the Depth-First Search algo rithm fo r so lving the 8 -puzzle pro blem. To
implement a Depth-First Search yo u need to implement a search that uses a Stack to sto re the set o f o pe n states that
have no t yet been visited.
In the puzzle package, create a Se arch class as sho wn:
CODE TO TYPE: Search class
package puzzle;
import java.util.*;
public class Search {
public static Board depthFirst(Board initial, Board goal, int maxDepth) {
if (initial.equals(goal)) { return initial; }
Stack<Board> open = new Stack<Board>();
HashSet<Board> closed = new HashSet<Board>();
open.add(initial);
while (!open.isEmpty()) {
Board b = open.pop();
closed.add(b);
for (SlideMove sm : b.validMoves()) {
Board next = new Board(b);
sm.execute(next);
next.previous = b;
next.depth = b.depth + 1;
if (next.equals(goal)) { return next; }
if (!closed.contains(next)) {
if (next.depth < maxDepth) {
open.add(next);
}
}
}
}
return null;
}
}
Let's break this co de do wn:
OBSERVE: Initializing the Depth First Search algo rithm
public static Board depthFirst(Board initial, Board goal, int maxDepth) {
if (initial.equals(goal)) { return initial; }
Stack<Board> open = new Stack<Board>();
HashSet<Board> closed = new HashSet<Board>();
open.add(initial);
If t he init ial bo ard st at e is t he go al st at e , t he m e t ho d e xit s im m e diat e ly. Otherwise, it creates an o pe n
o bject, using the existing Stack class fro m the Java Co llectio ns Framewo rk. As we've seen in earlier lesso ns, the stack
is the fundamental structure to use when pursuing a depth-first algo rithm, because it can save states to which the
algo rithm can backtrack as necessary.
The clo se d o bject is a HashSet that represents states that have already been visited. This o bject canno t simply be an
ArrayList o r Linke dList because the size o f the clo se d set can be quite large and these classes o nly suppo rt O(n)
perfo rmance when checking whether the list co ntains an item. Also , yo u canno t use TreeSet immediately because that
demands that the elements being added all implement Co mparable; there is no immediate way to co mpare to bo ard
states to see which o ne is smaller (o r larger). We'll use HashSet, which o ffers O(1) co nstant time perfo rmance to
lo cate an element. This o bject will represent a set because the algo rithm will no t visit the same state twice in the search
tree.
Fo r HashSe t to wo rk pro perly, the Bo ard class must implement a specialized hashCo de metho d. Specifically, if two
Bo ard state o bjects represent the same state, the hashCo de metho d must return the same value. In an earlier
lesso n, we sho wed ho w the St ring class implements its hashCo de metho d efficiently by caching the co mputed hash
value. Here we take advantage o f the fact that o nce a Bo ard is added to the clo se d set, it never changes. When writing
classes that are used as key values in the Java Co llectio ns Framewo rk, yo u must pro vide bo th the hashCo de
metho d and a co mpatible e quals metho d.
Mo dify the Bo ard class as sho wn:
CODE TO TYPE: Mo dificatio ns to Bo ard
package puzzle;
import java.util.*;
public class Board {
int[][] tiles;
int hash;
Board previous;
int depth;
...
public List<SlideMove> validMoves() {
int br = -1, bc = -1;
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
if (tiles[r][c] == 0) {
br = r;
bc = c;
}
}
}
ArrayList<SlideMove> list = new ArrayList<SlideMove>();
for (int i = 0; i < deltas.length; i++) {
int dr = deltas[i][0];
int dc = deltas[i][1];
SlideMove sm = new SlideMove (br+dr, bc+dc, br, bc);
if (sm.isValid(this)) { list.add(sm); }
}
return list;
}
public boolean equals (Object o) {
if (o == null) { return false; }
if (!(o instanceof Board)) { return false; }
Board other = (Board) o;
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
if (tiles[r][c] != other.tiles[r][c]) { return false; }
}
}
return true;
}
public int hashCode() {
if (hash == 0) {
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
hash = 31*hash + tiles[r][c];
}
}
}
return hash;
}
}
Let's review this co de:
OBSERVE: equals metho d fo r Bo ard
public boolean equals (Object o) {
if (o == null) { return false; }
if (!(o instanceof Board)) { return false; }
Board other = (Board) o;
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
if (tiles[r][c] != other.tiles[r][c]) { return false; }
}
}
return true;
}
Two Bo ard o bjects are equal if they co ntain the same arrangement o f tiles. The structure is co mmo n to many e quals
metho ds yo u may have seen. First, it makes sure that the o bje ct o is no t null, because the Java co ntract fo r e quals
states that no o bject is ever equal to null. Seco nd, any at t e m pt t o co m pare e qualit y wit h a no n-Bo ard o bje ct
m ust f ail. Finally, this metho d iterates o ver all tiles to determine whether any two co rrespo nding lo catio ns co ntain
different tiles, re t urning f alse at t he f irst dif f e re nce be t we e n t he t wo Bo ard o bje ct s:
OBSERVE: hashCo de metho d fo r Bo ard
public int hashCode() {
if (hash == 0) {
for (int r = 0; r < 3; r++) {
for (int c = 0; c < 3; c++) {
hash = 31*hash + tiles[r][c];
}
}
}
return hash;
}
The hashCo de metho d uses the hash attribute to cache the co mputed hash value. This metho d co mputes the hash
by using the same multiplicative functio n fo und in the String class. Fo r example, the hash value co mputed fo r the go al
state is 27 4 86 9 24 4 .
Everything is in place to try so lving an initial bo ard state.
In the puzzle package, create a Main class as sho wn:
CODE TO TYPE: Main class fo r searching
package puzzle;
public class Main {
public static void printSolution(Board goal) {
if (goal == null) {
System.out.println("No Solution reached");
} else {
int count = -1;
while (goal != null) {
System.out.println(goal);
goal = goal.previous;
count++;
}
System.out.println(count + " total moves");
}
}
public static void main(String[] args) {
Board initial = new Board(new int[][]{{1,2,3}, {8,6,4}, {0,7,5}});
Board goal = new Board(new int[][]{{1,2,3}, {8,0,4}, {7,6,5}});
int maxDepth = 8;
Board result = Search.depthFirst(initial, goal, maxDepth);
printSolution(result);
}
}
Save and run it to co nfirm the so lutio n fo und earlier. The bo ards are displayed in reverse o rder, fro m the goal state
all the way back to the initial po sitio n:
OBSERVE: Sample Executio n o f DepthFirstSearch algo rithm o n search tree
123
8-4
765
123
864
7-5
123
864
-75
2 total moves
Breadth-First Search
With o nly marginal changes, yo u can create an implementatio n that explo res the search tree in Breadth-First
fashio n. Add this metho d to the end o f the Se arch class:
CODE TO TYPE: Mo dificatio ns to Search class
package puzzle;
import java.util.*;
public class Search {
public static Board depthFirst(Board initial, Board goal, int maxDepth) {
if (initial.equals(goal)) { return initial; }
Stack<Board> open = new Stack<Board>();
HashSet<Board> closed = new HashSet<Board>();
open.add(initial);
while (!open.isEmpty()) {
Board b = open.pop();
closed.add(b);
for (SlideMove sm : b.validMoves()) {
Board next = new Board(b);
sm.execute(next);
next.previous = b;
next.depth = b.depth + 1;
if (next.equals(goal)) { return next; }
if (!closed.contains(next)) {
if (next.depth < maxDepth) {
open.add(next);
}
}
}
}
return null;
}
public static Board breadthFirst(Board initial, Board goal) {
if (initial.equals(goal)) { return initial; }
Queue<Board> open = new LinkedList<Board>();
HashSet<Board> closed = new HashSet<Board>();
open.add(initial);
while (!open.isEmpty()) {
Board b = open.remove();
closed.add(b);
for (SlideMove sm : b.validMoves()) {
Board next = new Board(b);
sm.execute(next);
next.previous = b;
next.depth = b.depth + 1;
if (next.equals(goal)) { return next; }
if (!closed.contains(next)) {
open.add(next);
}
}
}
return null;
}
}
The co de is identical to de pt hFirst , except that it uses a Que ue (implemented by Linke dList ) to sto re the
open bo ard states, and it remo ves the next bo ard state fro m open using the re m o ve metho d. The behavio r is
very different fro m de pt hFirst tho ugh; it metho dically inspects all bo ard states in increasing distance fro m
the initial state, based o n the number o f mo ves that are used.
Mo dify Main to repo rt o n Breadth-First Search results as well:
CODE TO TYPE: Mo dificatio ns to Main class
package puzzle;
public class Main {
public static void printSolution(Board goal) {
if (goal == null) {
System.out.println("No Solution reached");
} else {
int count = -1;
while (goal != null) {
System.out.println(goal);
goal = goal.previous;
count++;
}
System.out.println(count + " total moves");
}
}
public static void main(String[] args) {
Board initial = new Board(new int[][]{{1,2,3}, {8,6,4}, {0,7,5}});
Board goal = new Board(new int[][]{{1,2,3}, {8,0,4}, {7,6,5}});
int maxDepth = 8;
Board result = Search.depthFirst(initial, goal, maxDepth);
printSolution(result);
Board bfsResult = Search.breadthFirst(initial, goal);
printSolution(bfsResult);
}
}
Save and run it.
OBSERVE: Co mpare DFS and BFS o n simple bo ard
123
8-4
765
123
864
7-5
123
864
-75
2 total moves
123
8-4
765
123
864
7-5
123
864
-75
2 total moves
When running o n a bo ard state that is o nly two mo ves remo ved fro m the so lutio n, bo th appro aches lo cate
the so lutio n quickly, but which o ne is mo re efficient? Yo u can judge efficiency in terms o f speed o f executio n,
but also evaluate the efficiency o f a search by co mparing the size o f the bo ard states visited (clo se d) as well
as the size o f the bo ard states yet to be pro cessed (o pe n). Yo u need to instrument the Se arch and Main
classes to reco rd this info rmatio n.
Make these changes to Se arch:
CODE TO TYPE: Mo dificatio ns to Search
package puzzle;
import java.util.*;
public class
static int
static int
static int
static int
Search {
numDFSOpen = 0;
numDFSProcessed = 0;
numBFSOpen = 0;
numBFSProcessed = 0;
public static Board depthFirst(Board initial, Board goal, int maxDepth) {
if (initial.equals(goal)) { return initial; }
Stack<Board> open = new Stack<Board>();
HashSet<Board> closed = new HashSet<Board>();
open.add(initial);
numDFSOpen=1;
numDFSProcessed=0;
while (!open.isEmpty()) {
Board b = open.pop();
numDFSOpen--;
numDFSProcessed++;
closed.add(b);
for (SlideMove sm : b.validMoves()) {
Board next = new Board(b);
sm.execute(next);
next.previous = b;
next.depth = b.depth + 1;
if (next.equals(goal)) { return next; }
if (!closed.contains(next)) {
if (next.depth < maxDepth) {
numDFSOpen++;
open.add(next);
}
}
}
}
return null;
}
public static Board breadthFirst(Board initial, Board goal) {
if (initial.equals(goal)) { return initial; }
Queue<Board> open = new LinkedList<Board>();
HashSet<Board> closed = new HashSet<Board>();
open.add(initial);
numBFSOpen=1;
numBFSProcessed=0;
while (!open.isEmpty()) {
Board b = open.remove();
numBFSOpen--;
numBFSProcessed++;
closed.add(b);
for (SlideMove sm : b.validMoves()) {
Board next = new Board(b);
sm.execute(next);
next.previous = b;
next.depth = b.depth + 1;
if (next.equals(goal)) { return next; }
if (!closed.contains(next)) {
numBFSOpen++;
open.add(next);
}
}
}
return null;
}
}
The changes update two co unts fo r each algo rithm, reco rding the number o f bo ard states in the o pe n state
and the number o f bo ard states that were pro cessed.
Make these changes to Main to display this info rmatio n fo r the individual runs:
CODE TO TYPE: Mo dificatio ns to Main class
package puzzle;
public class Main {
public static void printSolution(Board goal) {
if (goal == null) {
System.out.println("No Solution reached");
} else {
int count = -1;
while (goal != null) {
System.out.println(goal);
goal = goal.previous;
count++;
}
System.out.println(count + " total moves");
}
}
public static void main(String[] args) {
Board initial = new Board(new int[][]{{1,2,3}, {8,6,4}, {0,7,5}});
Board goal = new Board(new int[][]{{1,2,3}, {8,0,4}, {7,6,5}});
int maxDepth = 8;
Board result = Search.depthFirst(initial, goal, maxDepth);
printSolution(result);
System.out.println("DFSOpen=" + Search.numDFSOpen + ", DFSProcessed=" + Sear
ch.numDFSProcessed);
Board bfsResult = Search.breadthFirst(initial, goal);
printSolution(bfsResult);
System.out.println("BFSOpen=" + Search.numBFSOpen + ", BFSProcessed=" + Sear
ch.numBFSProcessed);
}
}
Save and run it to generate statistics fo r this initial test:
OBSERVE: Sample statistics fo r trivial bo ard state
123
8-4
765
123
864
7-5
123
864
-75
2 total moves
DFSOpen=1, DFSProcessed=2
123
8-4
765
123
864
7-5
123
864
-75
2 total moves
BFSOpen=2, BFSProcessed=3
In bo th cases, a so lutio n o f 2 mo ves was fo und, but Breadth-First Search had mo re states in its o pe n set to
be co nsidered fo r the future and it also pro cessed o ne mo re bo ard state than Depth-First Search.
Evaluating Search T ree Algorithms
To evaluate these search algo rithms pro perly, yo u have to generate initial bo ard states fro m which to search
fo r the go al state. Ho wever, yo u can't just rando mly assign the eight digits and the empty space to a Bo ard,
because there may be no way to slide the tiles to achieve the go al state fro m the rando mly selected initial
state. The so lutio n is to make a rando mly number o f mo ves fro m so me initial state, and then let the
algo rithms try to search their way back to the o riginal bo ard state. Let's get started o n that infrastructure no w.
In the puzzle package, create a Ge ne rat e class as sho wn:
CODE TO TYPE: Generate class
package puzzle;
import java.util.*;
public class Generate {
public static Board generate(int n) {
Board state = new Board(new int[][]{{1,2,3}, {8,0,4}, {7,6,5}});
Set<Board> visited = new HashSet<Board>();
visited.add(new Board(state));
for (int i = 0; i < n; i++) {
List<SlideMove> moves = state.validMoves();
Collections.shuffle(moves);
Board next = null;
for (SlideMove sm : moves) {
next = new Board(state);
sm.execute(next);
if (!visited.contains(next)) {
visited.add(next);
break;
}
}
if (state.equals(next)) {
System.err.println("Unable to generate " + n + " moves");
return null;
}
state = next;
}
return state;
}
}
Let's take a clo ser lo o k at this co de.
OBSERVE:
Board state = new Board(new int[][]{{1,2,3}, {8,0,4}, {7,6,5}});
Set<Board> visited = new HashSet<Board>();
visited.add(new Board(state));
for (int i = 0; i < n; i++) {
List<SlideMove> moves = state.validMoves();
Collections.shuffle(moves);
Starting fro m the go al state st at e , we instantiate a visit e d set to make sure that we pro perly generate a to tal
o f n steps witho ut revisiting a previo usly visited bo ard state. The f o r lo o p ge ne rat e s a list o f po t e nt ial
m o ve s f ro m t he give n st at e and shuf f le s t his list so the mo ves are investigated in rando m o rder.
OBSERVE: Making a rando m mo ve
Board next = null;
for (SlideMove sm : moves) {
next = new Board(state);
sm.execute(next);
if (!visited.contains(next)) {
visited.add(next);
break;
}
}
The co de abo ve t rie s e ach m o ve , o ne at a t im e , to se e if it ge ne rat e s a ne w bo ard st at e t hat has
no t alre ady be e n visit e d. With each pass thro ugh the lo o p, it cre at e s a co py o f t he st at e bo ard st at e
in ne xt so the mo ve is executed o n the co py witho ut affecting the o riginal st at e . If we find an unvisited bo ard
state, we bre ak o ut o f the f o r lo o p; o therwise, it repeats until all mo ves are exhausted. To co mplete the lo o p,
review this lo gic:
OBSERVE: Identifying when no t po ssible to generate bo ard
if (state.equals(next)) {
System.err.println("Unable to generate " + n + " moves");
return null;
}
state = next;
Whe n t his m e t ho d re t urns null, it means that it was unsuccessful in lo cating a bo ard state n mo ves away;
the o nly reaso n fo r failure is that all bo ard states with fewer number o f mo ves were visited. If the co mputed
ne xt state is no t the same as st at e tho ugh, it se t s st at e t o ne xt and advances to try ano ther mo ve. Once
the appro priate number o f mo ves has been applied, the metho d returns a sample bo ard state n mo ves away.
Demo nstrate the use o f this Ge ne rat e metho d by mo difying the Main class as sho wn:
CODE TO TYPE: Mo dificatio ns to Main class
package puzzle;
public class Main {
public static void printSolution(Board goal) {
if (goal == null) {
System.out.println("No Solution reached");
} else {
int count = -1;
while (goal != null) {
System.out.println(goal);
goal = goal.previous;
count++;
}
System.out.println(count + " total moves");
}
}
public static void main(String[] args) {
Board initial = new Board(new int[][]{{1,2,3}, {8,6,4}, {0,7,5}});
Board initial = Generate.generate(6);
Board goal = new Board(new int[][]{{1,2,3}, {8,0,4}, {7,6,5}});
int maxDepth = 89;
Board result = Search.depthFirst(initial, goal, maxDepth);
printSolution(result);
System.out.println("DFSOpen=" + Search.numDFSOpen + ", DFSProcessed=" + Sear
ch.numDFSProcessed);
Board bfsResult = Search.breadthFirst(initial, goal);
printSolution(bfsResult);
System.out.println("BFSOpen=" + Search.numBFSOpen + ", BFSProcessed=" + Sear
ch.numBFSProcessed);
}
}
Run the revised Main class; yo u'll get different results based o n the generated bo ard state. Here is a
sample run where a Depth-First appro ach finds an 8 -mo ve so lutio n (after pro cessing just 21 bo ard states),
while a Breadth-First appro ach finds a minimal 6 -mo ve so lutio n (after pro cessing 45 bo ard states).
OBSERVE: Sample run co mparing Breadth-First and Depth-First
123
8-4
765
123
-84
765
-23
184
765
2-3
184
765
283
1-4
765
283
-14
765
-83
214
765
8-3
214
765
83214
765
8 total moves
DFSOpen=5, DFSProcessed=21
123
8-4
765
1-3
824
765
-13
824
765
813
-24
765
813
2-4
765
8-3
214
765
83214
765
6 total moves
BFSOpen=32, BFSProcessed=45
Run it multiple times and co mpare the results to see lo w, high, and average results. In general, the BreadthFirst appro ach will co mpute the sho rtest number o f mo ves to achieve the go al state, but it will pro cess far
mo re states than the Depth-First appro ach. Ho wever, Depth-First is a blind algo rithm and will generate
so lutio ns with perhaps hundreds o r tho usands o f mo ves if yo u do n't set m axDe pt h—but there may no t be
an easy way to determine the pro per value to use fo r m axDe pt h because that implies that yo u have (mo re o r
less) an idea as to ho w many mo ves away yo u are.
Lessons Learned
In this lesso n yo u learned:
ho w to use a Que ue structure to impo se a Breadth-First appro ach when inserting and remo ving
states to search fro m the o pe n set.
ho w to use a St ack structure to impo se a Depth-First appro ach when inserting and remo ving
states to search.
that the HashSe t class fro m the Java Co llectio ns Framewo rk pro vides O(1) co nstant perfo rmance
fo r determining whether the set co ntains a given element.
ho w to avo id the need to implement an undo metho d in the mo ves by using a co nstructo r to co py
the existing bo ard state to which the mo ves execute their changes. This behavio r is distinctly
different fro m the game trees fro m the upco ming two -player lesso n.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Path Finding for Two-Player Games
Lesson Objectives
After co mpleting this lesso n yo u will be able to :
describe the structure o f a game tree fo r two -player games.
explain ho w Minimax co mputes best mo ve fo r player in a game tree.
explain ho w an evaluatio n functio n guides Minimax to cho o sing the best mo ve.
Path Finding For T wo-Player Games
Cho psticks is a two -player hand game. The versio n presented here is just o ne o ut o f many po ssible variatio ns. Each
player uses bo th hands, each o f which can represent the values 0 to 4 by the number o f extended fingers o n that hand
(thumbs are no t invo lved). To start, bo th players extend the index finger o n each hand. The players alternate turns, and
when it is his turn, a player may:
T ap
Re balance
To t ap, a player taps the hand o f an o ppo nent with o ne o f his hands. When this happens the player's "po ints" in the
tapping hand (represented by the number o f extended fingers) are added to the o ppo nent's tapped hand; the number o f
po ints in the player's hand do es no t change. Once a hand has five o r mo re po ints, the player clo ses the hand into a fist
and the hand is co nsidered to be "dead" with zero po ints.
To re balance , a player must have o ne "dead" hand, and o ne hand with an even number o f po ints. On his turn, the
player taps his o wn "dead" hand with his o ther hand, and the po ints are evenly split between bo th hands.
The go al o f each player is to fo rce his o ppo nent to have two "dead" hands.
This lesso n sho ws ho w to use a path finding technique fro m Artificial Intelligence to co mpute the best mo ve fo r a
player, that is, the mo ve that has the highest likeliho o d o f leading that player to victo ry. To so lve this kind o f pro blem
yo u have to frame the pro blem co mputatio nally and co rrectly. At any given mo ment, the state o f the game can be
represented by:
which player's turn it is (Player1 o r Player2).
Player1's left hand po ints (0 , 1, 2, 3, o r 4).
Player1's right hand po ints (0 , 1, 2, 3, o r 4).
Player2's left hand po ints (0 , 1, 2, 3, o r 4).
Player2's right hand po ints (0 , 1, 2, 3, o r 4).
So , there are 2x5 x5 x5 x5 = 1,25 0 unique states o f the game. Co nsider creating a game tree with no des that represent
valid states o f the cho psticks game as the players take their turns. Given a particular game state, there are no mo re
than five po ssible mo ves that can be made at any time:
Rebalance.
Use left hand to tap o ppo nent's left hand.
Use left hand to tap o ppo nent's right hand.
Use right hand to tap o ppo nent's left hand.
Use right hand to tap o ppo nent's right hand.
To co nstruct the game tree fro m a given starting po sitio n, create a ro o t no de that represents the initial game state o f
(Playe r1, 1, 1, 1, 1). In the graphic belo w, Player1 (the starting player) is depicted o n to p and his o ppo nent (Player2) is
o n the bo tto m. The two numbers, fro m left to right, represent the po ints o n the left and right hand, respectively. Because
it's Player1's turn (the to p player), his state is highlighted in o range. This is a co nvenient way to visualize the state
(Playe r1, 1, 1, 1, 1).
The game tree is expanded by adding child no des to the leaf no des in the tree (the ro o t no de o f the initial game tree is
a leaf no de). The game tree expands based upo n the allo wed mo ves fo r the active player. There are fo ur po ssible
tapping mo ves here, but these result in o nly two distinct game states: if Player1 uses his left (o r right) hand to tap the
left hand o f Player2, the resulting state is (Playe r2, 1, 1, 2, 1), which wo uld beco me the left child in the game tree. If
Player1 uses his left (o r right) hand to tap the right hand o f Player2, the resulting state is (Playe r2, 1, 1, 1, 2) which
wo uld beco me the right child in the game tree. No te that in each o f these two no des, it will be Player2's turn.
At this po int, o bserve that these two child no des are really equivalent. If a player has X po ints o n the left hand and Y
po ints o n the right hand, this is equivalent to having Y po ints o n the left hand and X po ints o n the right hand. So , yo u
can co mbine these child no des into o ne. Fo r co nsistency, a player's hand can be defined by two values (Lo w,High)
such that Lo w <= High. So , the game state is fully captured by (Playe r# , Playe r1Lo w, Playe r1High, Playe r2Lo w,
Playe r2High), where Playe r1Lo w <= Playe r1High and Playe r2Lo w <= Playe r2High.
The abo ve game tree has o ne level belo w the ro o t no de. Each successive level is kno wn as a ply and represents a set
o f game states asso ciated with the same active player. Fo r example, the ro o t no de (o ften called ply 0 ) represents the
active state fo r Player1, while the next level (ply 1) represents the active states fo r Player2. Let's expand the tree so
there are fo ur levels (ply 0 thro ugh ply 3):
The eight no des in level 3 o f this game tree represent all po ssible game states after three mo ves have been made.
The game tree is expanded by adding children to no des, so the resulting structure will have no cycles. The final level in
the tree represents the to tal number o f po ssible states after p=3 mo ves. As yo u can see, it's po ssible fo r Player2 to
have a "dead hand" after just three mo ves. Yo u can find this state abo ve because it uses the "*" character to indicate
that o ne o f the player's hands has no po ints. Each level (o r ply) o f the tree co ntains no des with an active player that is
the same. Even tho ugh there are five po ssible mo ves in each game state, no t every no de is expanded to have five
children no des. The size o f the game tree is ultimately determined by an expansion factor k which is the average
number o f child no des fo r any no de in the game tree; fo r cho psticks, k <= 5.
The abo ve image demo nstrates ho w large these trees can gro w, even fo r simple games. If we lo o ked eight mo ves
ahead (ply=8 ), the game tree wo uld co ntain up to 48 8 ,28 0 no des if every no de expanded by five children. The go al is
to select the best available mo ve fo r a player in a given game state. In the cho psticks game tree abo ve, Player1 can
make o nly o ne mo ve in the initial state, so there is really no decisio n to make. Ho wever, Player2 can try to fo rce
Player1 to make a bad decisio n that wo uld directly lead to Player2's victo ry. Let's expand o ne part o f the game tree
abo ve to sho w this situatio n:
The highlighted state abo ve is (Playe r2, 1, 3, 2, 4 ). It's Player2's turn in this state, so she can guarantee a victo ry by
using her hand with 4 fingers to tap Player1's hand with 1 finger. This results in the state (Playe r1, *, 3, 2, 4 ). Player1
wo uld then have o nly two po ssible mo ves: (1) tap Player2's hand with two fingers to create a dead hand; o r (2) tap
Player2's hand with 4 fingers to create a dead hand. Ho wever, in bo th o f these situatio ns, Player2 can simply tap
Player1's remaining hand with three fingers to make it a dead hand, thus fo rcing Player1 to have two dead hands, and
victo ry is guaranteed.
The game tree represents the full set o f po tential game states that result fro m a sequence o f valid mo ves fro m the
initial state; due to its size, it may never be co mputed fully. The go al o f a path-finding algo rithm is to determine fro m a
starting game state, the player's mo ve that maximizes (o r even guarantees) his chance o f winning the game. So we
transfo rm an intelligent set o f player decisio ns into a path-finding problem o ver the game tree. This appro ach wo rks fo r
games with small game trees, but it also can be scaled to so lve mo re co mplex pro blems.
Given a no de representing the current game state, the algo rithm co mputes the best mo ve fo r a player. Instead o f
co nsidering o nly the current game state and available mo ves at that state, the pro gram must co nsider any
co untermo ves that the o ppo nent will make after each mo ve. The pro gram must assume that the o ppo nent will select
his best mo ve cho ice and make no mistakes. To make this wo rk co mputatio nally, we need a functio n that can evaluate
a game state o bjectively and return an integer "sco re" value fo r that state. Smaller integer numbers (even negative
o nes) reflect weaker po sitio ns, while larger integer numbers (including po sitive infinity) represent stro nger po sitio ns
fo r the player.
Given a specific game state, it is easy to determine whether Player1 o r Player2 has lo st the game. Fo r example, o ne o f
the final states abo ve is (Playe r1, *, *, *, 4 ) so fro m Player1's perspective, this is a lo ss and the evaluatio n functio n
must rate this bo ard as -Inf init y. Ho wever, fro m Player2's po int o f view, this state is a win so the evaluatio n functio n
wo uld rate the same bo ard as +Inf init y. Clearly, it matters fro m who se perspective the bo ard is evaluated.
The first steps we'll take will be to write co de that represents the game state. Here are two ways we co uld acco mplish
that task:
Represent state with fo ur int attributes: playe rLo w, playe rHigh, o ppo ne nt Lo w, o ppo ne nt High.
Represent state with two int arrays: playe rPo int s[] and o ppo ne nt Po int s[].
The first o ptio n abo ve is inefficient, because yo u'd have to write special co de co nstantly to co mpare the fo ur different
sums that result fro m adding to gether the different po ssible values. The seco nd o ptio n uses arrays which is better
(because yo u co uld use nested f o r lo o ps to co mpute the sums), but yo u wo uld spend a lo t o f time keeping these two
values in so rted o rder to take advantage o f the earlier o bservatio n regarding game state.
Use an array o f T re e Se t o bjects to maintain the Se t o f po ints fo r the players; when a player has two hands with the
same value, o nly o ne value is sto red fo r that player's hand. This allo ws yo u to write simpler co de, altho ugh the co de
may lo o k slightly co mplicated the first time that yo u see it.
Create a new Java Pro ject named T wo Playe r and assign it to the J ava6 _Le sso ns wo rking set.
In yo ur T wo Playe r pro ject /src so urce fo lder, create a cho pst icks package.
In the cho pst icks package, create a Gam e St at e class as sho wn:
CODE TO TYPE: GameState class
package chopsticks;
import java.util.*;
public class GameState {
public static final int Player1 = 0;
public static final int Player2 = 1;
int player;
Set<Integer> values[] = new TreeSet[2];
public GameState (int player, int left1, int right1, int left2, int right2) {
this.player = player;
values[0] = new TreeSet<Integer>();
values[0].add(left1);
values[0].add(right1);
values[1] = new TreeSet<Integer>();
values[1].add(left2);
values[1].add(right2);
}
public boolean hasWon(int p) {
return values[1-p].size() == 1 && values[1-p].contains(0);
}
public String toString() {
Iterator<Integer> pValues
= values[0].iterator();
Iterator<Integer> oppValues = values[1].iterator();
StringBuilder sb = new StringBuilder("(Player").append(1+player).append(",");
int left1 = pValues.next();
sb.append(left1).append(",");
if (pValues.hasNext()) { sb.append(pValues.next()); } else { sb.append(left1); }
int left2 = oppValues.next();
sb.append(",").append(left2).append(",");
if (oppValues.hasNext()) { sb.append(oppValues.next()); } else { sb.append(left2);
}
return sb.append(")").toString();
}
}
Each game state must sto re the current playe r in that situatio n, using 0 fo r Player1 and 1 fo r Player2. These values
were cho sen to make it easier to index into the Se t <Int e ge r> value s[] array that sto res the T re e Se t o bjects. By
using sets, the game state uses the o ptimizatio n presented earlier where it o nly reco rds o ne value when bo th hands
have the same value. The hasWo n(p) metho d determines whether a player p has wo n in a game state; this happens
when that player's o ppo nent (1-p) has o nly o ne value (value s[1-p].size () == 1) and that value is 0 .
The t o St ring() helper metho d is used o nly during debugging. It builds up a string representing the game state by
iterating o ver the values in each hand. Fo r efficiency, it uses the St ringBuilde r class.
Aside fro m an o utright victo ry, ho w is it po ssible to co mpute a number that increases in value when a game state is
mo re likely to lead to victo ry fo r a given player? Yo u must develo p a heuristic based o n pro perties o f the game state.
Let's start with so me o bservatio ns:
Having o ne dead hand is bad.
Having hands with 4 po ints is mo re risky than having hands with 1 po int.
A player has a stro ng po sitio n when o ne o f her hands can make an o ppo nent's hand dead (and when the
o ppo nent can do this, the player's hand is weaker).
Instead o f including an evaluatio n metho d inside Gam e St at e , define an interface to be used by any evaluatio n
functio n. This lets yo u to experiment with different evaluatio n functio ns quickly.
In the /src so urce fo lder cho pst icks package, create an IEvaluat e interface as sho wn:
CODE TO TYPE: IEvaluate interface
package chopsticks;
public interface IEvaluate {
int evaluate(GameState state, int player);
}
In the cho pst icks package, create an Evaluat o r class that implements the heuristics defined earlier:
CODE TO TYPE: Evaluato r class
package chopsticks;
public class Evaluator implements IEvaluate {
public int evaluate(GameState s, int p) {
if (s.hasWon(p)) { return 10000; }
if (s.hasWon(1-p)) { return -10000; }
int sign = 1;
if (s.player == 1-p) { sign = -1; }
int value = 0;
// Having one dead hand is bad
if (s.values[s.player].contains(0)) { value -= sign*100; }
if (s.values[1-s.player].contains(0)) { value += sign*100; }
// Having hands with 4 points is more risky than having hands with 1 point (scale b
y 5 pts)
for (int pt : s.values[s.player]) {
value -= sign*pt*5;
}
for (int pt : s.values[1-s.player]) {
value += sign*pt*5;
}
// Player has strong position when one of his hands can make an opponent's hand dea
d
int numMakeDead = 0;
for (int pt1 : s.values[s.player]) {
for (int pt2 : s.values[1-s.player]) {
if (pt1 + pt2 >= 5) { numMakeDead++; }
}
}
value += sign * numMakeDead * 20;
return value;
}
}
The e valuat e (st at e ,p) metho d is symmetric; that is, e valuat e (st at e ,Playe r1) = -e valuat e (st at e ,Playe r2). The
go al o f this metho d is to make it po ssible to co mpare the co mputed evaluatio n o f two game states, gs1 and gs2, to
determine which is mo re favo rable to the current player. Acco rdingly, a victo ry is a clearly identified 10 0 0 0 (used in
place o f infinity since no number co mputed by the e valuat e metho d will ever be larger than this value). The co de
defines a sign variable that determines whether the resulting co mputatio n is negative (wo rse fo r the player p) o r
po sitive (better fo r the player p).
When develo ping heuristics, it is imperative that yo u write test cases so yo u can track changes to tho se heuristics.
Many o f the co nstants were cho sen arbitrarily and yo u will have to experiment with mino r tweaks, so test cases will
pro ve very useful.
In yo ur T wo Playe r pro ject, create a /t e st so urce fo lder.
In the /t e st so urce fo lder, create a cho pst icks package.
In the cho pst icks package, create a T e st Gam e St at e JUnit test case as sho wn:
CODE TO TYPE: TestGameState JUnit class
package chopsticks;
import junit.framework.TestCase;
public class TestGameState extends TestCase {
public void testWinning() {
GameState gs = new GameState(GameState.Player1, 0, 0, 0, 2);
assertTrue (gs.hasWon(GameState.Player2));
}
public void testNotWinning() {
GameState gs = new GameState(GameState.Player1, 1, 1, 1, 1);
assertFalse (gs.hasWon(GameState.Player1));
assertFalse (gs.hasWon(GameState.Player2));
}
public void testFourBoards() {
GameState[] states = {
new GameState(GameState.Player2, 1, 3, 1, 3),
new GameState(GameState.Player2, 1, 3, 0, 1),
new GameState(GameState.Player2, 1, 3, 2, 2)
};
GameState worst = new GameState(GameState.Player2, 1, 3, 2, 4);
IEvaluate eval = new Evaluator();
int worstRating = eval.evaluate(worst, GameState.Player1);
System.out.println("Worst State:" + worstRating + " " + worst);
System.out.println("Other Moves:");
for (GameState gs : states) {
int gsRating = eval.evaluate(gs, GameState.Player1);
System.out.println(gsRating + " " + gs);
assertTrue (eval.evaluate(gs, GameState.Player1) <= gsRating);
}
}
}
This test case ensures that the hasWo n(p) metho d wo rks pro perly. It also co mpares the fo ur game state children o n
the right side o f the game tree depicted earlier to validate that the wo rst state evaluates to a number that is sm alle st o f
the o ther three sibling states. In o ther wo rds, this sho uld identify that this state is the wo rst po ssible arrangement fro m
Player1's po int o f view. Yo u must be sure that yo u represent the bo ard states accurately, as well as the player fo r
who m the evaluatio n is being made . In t e st Fo urBo ards, all Gam e St at e o bjects are asso ciated with Player2 and
e valuat e () is called with Player1 as an argument, because the o riginating no de in the game tree is Player1.
Run the test case no w and check the o utput:
OBSERVE: Game State evaluatio n sco res
Worst State:-50 (Player2,1,3,2,4)
Other Moves:
-20 (Player2,1,3,1,3)
85 (Player2,1,3,0,1)
-30 (Player2,1,3,2,2)
When evaluating these fo ur game states fro m the perspective o f Player1, the (Playe r2, 1, 3, 2, 4 ) game state is rated
the lo west.
The game tree is expanded by co nsidering future game states after n mo ves have been made. Each level o f the tree
alternates between MAX levels (where the go al is to benefit the player by maximizing the evaluated sco re o f a game
state) and MIN levels (where the go al is to benefit the o ppo nent by minimizing the evaluated sco re o f a game state).
So , the levels alternate between MAX and MIN levels, which leads to an algo rithm kno wn as Minimax. In all cases, the
bo ard is evaluated from the point of view of the player making the original move in the game tree (that is, the active player
in ply 0 ).
The next game tree sho ws, in dashed bo xes, the sco re o f the evaluatio n functio n o n each leaf no de in a 2-ply game
tree starting fro m the (Playe r2, 1, 1, 1, 2) state where it is Player2's turn. Only leaf no des are evaluated. Interio r no des
o n levels marked Max receive the maximum sco re o f their children no des. Similarly, interio r no des o n levels marked
Min receive the minimum sco re o f their children no des.
Of the eight no des o n the ply-2 level, the state (Playe r2, 1, 3, 2, 4 ) mentio ned earlier is the highest-rated state fo r
Player2 with a rating o f 5 0 . Ho wever, the lo west rated state (Playe r2, 1, 3, *, 1) with a rating o f -85 is a sibling o f this
state. In the Min level in state (Playe r1, 1, 3, 1, 2), it is Player1's turn to make a mo ve. The algo rithm must assume that
an o ppo nent plays witho ut making mistakes; thus given the chance, Player1 wo uld fo rce Player2 into the lo west rated
state. Fo r this reaso n, the interio r no des o n the ply-1 level are rated as 5 and -85 respectively, representing the wo rst
po sitio ns that Player2 wo uld find himself in after Player1 mo ves. Finally, the ro o t no de o n ply 0 selects the mo ve that
maximizes the evaluatio n o f its children no des, thus the algo rithm wo uld cho o se the T ap mo ve that results in the state
(Playe r2, 1, 2, 1, 2).
Co nstructing the game tree abo ve do es no t include writing co de to auto mate this pro cess. This pseudo co de
describes that pro cess:
OBSERVE: pseudo co de fo r Minimax
bestMove (s, player)
original = player
[move, score] = minimax(s, ply, MaxLevel)
return move
minimax (s, ply, player, opponent)
best = [null, 0]
if (ply = 0 or no valid moves) then
score = evaluate s for original player
return [null, score]
foreach valid move m for player in state s do
execute move m on s
[move, score] = minimax(s, ply-1, not MaxLevel)
undo move m on s
if (player is original) then
if (score > best.score) then best = [m, score]
else
if (score < best.score) then best = [m, score]
return best
Because the game tree is a recursive structure, the Minim ax implementatio n also recursively identifies game states to
explo re. With each recursive call, the ply depth is decreased until ply=0 , at which case the state s is evaluated fro m the
perspective o f the o riginal player.
The f o re ach lo o p inside m inim ax evaluates the sco re fo r each o f the children no des fro m state s and remembers
the highest sco re if that level is a Max level (that is, player at that level is the o riginal player); alternatively, it remembers
the lo west sco re if that level is a Min level (i.e., no t a Maxlevel). Because the go al o f m inim ax is to return the best
mo ve fo r the o riginal player, it must return bo th the mo ve and the ultimate best sco re that the player can ho pe fo r in the
game tree when making that mo ve.
No w let's go write co de to match the Minimax pseudo co de.
Minimax Implementation
The go al o f Minimax is to identify a mo ve fo r a player in a given game state. To represent a valid mo ve, create
this interface.
In the /src so urce fo lder cho pst icks package, create an IMo ve interface as sho wn:
CODE TO TYPE: IMo ve
package chopsticks;
public interface IMove {
boolean valid(GameState state);
boolean make(GameState state);
boolean undo(GameState state);
}
Classes that claim to be a cho psticks mo ve must be able to execute that mo ve o n a Gam e St at e o bject,
changing its co ntents. All changes are made in place o n a Gam e St at e o bject, so a mo ve class must also be
able to undo that mo ve. Finally, the mo ve class must be able to determine if it is even valid fo r a given
Gam e St at e .
Define a class to represent the info rmatio n returned by m inim ax.
In the /src so urce fo lder cho pst icks package, create a Pair class as sho wn:
CODE TO TYPE: Pair class
package chopsticks;
public class Pair {
IMove move;
int score;
Pair (IMove move, int score) {
this.move = move;
this.score = score;
}
}
In the /src so urce fo lder cho pst icks package, create a Minim ax class as sho wn:
CODE TO TYPE: Minimax class
package chopsticks;
import java.util.*;
public class Minimax {
int ply;
int original;
IEvaluate eval;
public Minimax (int ply, IEvaluate ie) {
this.ply = ply;
this.eval = ie;
}
public IMove bestMove (GameState s, int player) {
original = player;
Pair bestMove = minimax (s, ply, true);
return bestMove.move;
}
Pair minimax (GameState s, int ply, boolean maxLevel) {
Collection<IMove> validMoves = null;
if (ply > 0) { validMoves = computeMoves(s); }
if (ply == 0 || validMoves.isEmpty()) {
int score = eval.evaluate(s, original);
return new Pair (null, score);
}
Pair best = null;
for (IMove m : validMoves) {
if (m.make(s)) {
Pair next = minimax(s, ply-1, !maxLevel);
next.move = m;
m.undo(s);
if (maxLevel) {
if (best == null || next.score > best.score) { best = next; }
} else {
if (best == null || next.score < best.score) { best = next; }
}
}
}
return best;
}
}
This co de is missing the co m put e Mo ve s() metho d (we'll get to that in a minute.) Let's take a clo ser lo o k at
this class:
OBSERVE: Co nstructing a Minimax instance
public class Minimax {
int ply;
int original;
IEvaluate eval;
public Minimax (int ply, IEvaluate ie) {
this.ply = ply;
this.eval = ie;
}
public IMove bestMove (GameState s, int player) {
original = player;
Pair bestMove = minimax (s, ply, true);
return bestMove.move;
}
...
}
Minimax st o re s t he IEvaluat e im ple m e nt at io n used to evaluate game states, as well as the ply
representing the maximum depth o f the game tree to expand. To find the best mo ve fo r a given state s, call the
Minimax metho d be st Mo ve (s, p), which then st o re s t he o riginal playe r t o use when evaluating game
states. All o f the interesting actio n happens in the m inim ax metho d:
OBSERVE: minimax recursive metho d
Pair minimax (GameState s, int ply, boolean maxLevel) {
Collection<IMove> validMoves = null;
if (ply > 0) { validMoves = computeMoves(s); }
if (ply == 0 || validMoves.isEmpty()) {
int score = eval.evaluate(s, original);
return new Pair (null, score);
}
Pair best = null;
for (IMove m : validMoves) {
if (m.make(s)) {
Pair next = minimax(s, ply-1, !maxLevel);
next.move = m;
m.undo(s);
if (maxLevel) {
if (best == null || next.score > best.score) { best = next; }
} else {
if (best == null || next.score < best.score) { best = next; }
}
}
}
return best;
}
Assuming that ply is greater than zero , minimax it e rat e s t hro ugh all o f t he valid m o ve s o ne by o ne .
After applying each mo ve to the game state, minimax() re cursive ly invo ke s m inim ax at a de pt h o f ply-1
and ne gat e s t he m axLe ve l t o alt e rnat e be t we e n Min and Max levels. When the recursive call ends,
t he m o ve is undo ne , minimax() reco rds the maximum (o r minimum) sco re o f the children no des o f s, and
it asso ciates that mo ve with the co mputed sco re. This metho d re t urns t he be st co m put e d m o ve .
When the recursive minimax metho d re ache s ply o f 0 , it has reached the final depth (fo r co mpleteness.
There may be so me game trees where a player has no mo re available mo ves earlier than that depth; that
case is treated in the same way). minimax e valuat e s t he gam e st at e f ro m t he pe rspe ct ive o f t he
o riginal playe r and re t urns t he e valuat e d sco re wit hin a Pair o bje ct that currently has no mo ve
asso ciated with it. The invo king metho d will asso ciate the appro priate mo ve o bject.
No w yo u just need to implement the co m put e Mo ve s(s) metho d that returns a co llectio n o f mo ve o bjects
that represent the available mo ves at that state. First, yo u need to create a class that represents a T ap mo ve.
In the /src so urce fo lder cho pst icks package, create a T apMo ve class as sho wn:
CODE TO TYPE: TapMo ve class
package chopsticks;
public class TapMove implements IMove {
final int fromPoints;
final int toPoints;
int
newValue;
public TapMove (int fromPoints, int toPoints) {
this.fromPoints = fromPoints;
this.toPoints = toPoints;
}
public String toString () {
return "Tap " + toPoints + " with " + fromPoints;
}
public boolean valid(GameState s) {
if (!s.values[s.player].contains(fromPoints)) { return false; }
if (!s.values[1-s.player].contains(toPoints)) { return false; }
return true;
}
public boolean make(GameState s) {
if (!valid(s)) { return false; }
newValue = fromPoints + toPoints;
if (newValue >= 5) { newValue = 0; }
if (s.values[1-s.player].size() == 2) {
s.values[1-s.player].remove(toPoints);
}
s.values[1-s.player].add(newValue);
s.player = 1-s.player;
return true;
}
public boolean undo(GameState s) {
s.player = 1-s.player;
if (s.values[1-s.player].size() > 1) {
s.values[1-s.player].remove(newValue);
}
s.values[1-s.player].add(toPoints);
return true;
}
}
Let's take a clo ser lo o k.
TapMo ve class
public class TapMove implements IMove {
final int fromPoints;
final int toPoints;
int
newValue;
public TapMove (int fromPoints, int toPoints) {
this.fromPoints = fromPoints;
this.toPoints = toPoints;
}
public String toString () {
return "Tap " + toPoints + " with " + fromPoints;
}
public boolean valid(GameState s) {
if (!s.values[s.player].contains(fromPoints)) { return false; }
if (!s.values[1-s.player].contains(toPoints)) { return false; }
return true;
}
...
}
A TapMo ve o bject represents the Tap mo ve with a tapping hand that has f ro m Po int s and with a tapped
hand that co ntains t o Po int s. To determine whether a given mo ve is valid in Gam e St at e s, the valid
metho d o nly needs to determine if t he value s asso ciat e d wit h t he curre nt playe r (s.playe r) co nt ain
f ro m Po int s. Similarly, the mo ve is o nly valid if t he hand o f t he o ppo ne nt (1-s.playe r) co nt ains
t o Po int s. The ne wValue value is co mputed within the m ake mo ve to allo w undo to wo rk.
The real lo gic o ccurs within m ake ():
OBSERVE: TapeMo ve make() metho d
public boolean make(GameState s) {
if (!valid(s)) { return false; }
newValue = fromPoints + toPoints;
if (newValue >= 5) { newValue = 0; }
if (s.values[1-s.player].size() == 2) {
s.values[1-s.player].remove(toPoints);
}
s.values[1-s.player].add(newValue);
s.player = 1-s.player;
return true;
}
If t he m o ve is no t valid in t he st at e s, t he n it re t urns f alse ; o therwise it determines the ne wValue to
use fo r the o ppo nent's hand. If ne wValue is f ive o r gre at e r, the player has a dead hand. The o nly tricky
lo gic is to decide ho w to update the po ints fo r the o ppo nent's hand. If t he o ppo ne nt 's hand alre ady
co nt aine d t wo dist inct value s (as de t e rm ine d by s.value s[1-s.playe r].size ()), yo u must re m o ve
t he t o Po int s value because it is being replaced with ne wValue . Ho wever, if the hand has two fingers with
the same value (the set s.value s[1-s.playe r] o nly has o ne value), yo u o nly have to add ne wValue t o t he
se t . Finally, o nce the mo ve is made, t he playe r asso ciat e d wit h t he st at e f lips t o t he o t he r playe r.
To co mplete the T apMo ve class, there needs to be an undo () implementatio n.
OBSERVE: TapMo ve undo metho d
public boolean undo(GameState s) {
s.player = 1-s.player;
if (s.values[1-s.player].size() > 1) {
s.values[1-s.player].remove(newValue);
}
s.values[1-s.player].add(toPoints);
return true;
}
The undo metho d is invo ked o nly after a successful mo ve. Its o peratio ns reverse the effect o f the m ake
metho d. It first swit che s t he act ive playe r o f t he st at e , then re place s t he ne wValue value in t he
o ppo ne nt 's se t wit h t he o riginal t o Po int s value . . If t he o ppo ne nt has t wo dist inct value s,
ne wValue can be remo ved safely.
With T apMo ve available, yo u can no w go back to the Minim ax class and add the
co m put e Mo ve s(Gam e St at e s) metho d:
CODE TO TYPE: Adding co mputeMo ves to Minimax
static Collection<IMove> computeMoves (GameState s) {
ArrayList<IMove> set = new ArrayList<IMove>();
for (int to : s.values[1-s.player]) {
if (to == 0) { continue; }
boolean alreadyOver = false;
for (int from : s.values[s.player]) {
if (from == 0) { continue; }
if (!alreadyOver) {
set.add(new TapMove (from, to));
}
if (from + to >= 5) {
alreadyOver = true;
}
}
}
return set;
}
This metho d checks the fo ur po ssible T ap mo ves by iterating o ver all the po ints in the player's hands, and
trying to fo rm T apMo ve o bjects with the po ints in each o f the o ppo nent's hands. No te that it must avo id dead
hands with no po ints. This metho d also adds o ne mo re o ptimizatio n that eliminates duplicate mo ves. Fo r
example, in the state (Playe r1, 2, 3, 3, 4 ), Player1 has fo ur T ap mo ves (2 o n 3, 3 o n 3, 2 o n 4, 3 o n 4).
Ho wever, there are really o nly two po tential states that can result fro m these mo ves: (Playe r2, 2, 3, *, 3) and
(Playe r2, 2, 3, *, 4 ). The alre adyOve r variable is set to t rue to avo id co mputing redundant T apMo ve
o bjects.
No w yo u're ready to put everything to gether! Write the co de belo w to determine the best mo ve fo r Player2
within the (Playe r2, 1, 1, 1, 2) state.
In yo ur T wo Playe r pro ject, create a /pe rf o rm ance so urce fo lder.
In the /pe rf o rm ance so urce fo lder, create a cho pst icks package.
In the cho pst icks package, create a Print Gam e T re e class as sho wn:
COE TO TYPE: PrintGameTree class
package chopsticks;
public class PrintGameTree {
public static void main(String[] args) {
GameState gs = new GameState(GameState.Player2, 1, 1, 1, 2);
IEvaluate eval = new Evaluator();
int ply = 2;
Minimax m = new Minimax(ply, eval);
IMove move = m.bestMove(gs, GameState.Player2);
System.out.println("best move: " + move);
}
}
Save and run it. The o utput is be st m o ve : T ap 1 wit h 1. This means that the best mo ve is go ing to be to
the left o f the game tree presented earlier. So , ho w can yo u make sure that the co de is wo rking co rrectly?
Make these co de changes to o utput info rmatio n as the algo rithm executes:
CODE TO TYPE: Changes to Minimax to expo se info rmatio n as it pro cesses
package chopsticks;
import java.util.*;
public class Minimax {
int ply;
int original;
IEvaluate eval;
StringBuffer padding;
public Minimax (int ply, IEvaluate ie) {
this.ply = ply;
this.eval = ie;
}
public IMove bestMove (GameState s, int player) {
padding = new StringBuffer();
original = player;
Pair bestMove = minimax (s, ply, true);
return bestMove.move;
}
Collection<IMove> computeMoves (GameState s) {
ArrayList<IMove> set = new ArrayList<IMove>();
for (int to : s.values[1-s.player]) {
boolean alreadyOver = false;
for (int from : s.values[s.player]) {
if (!alreadyOver) {
set.add(new TapMove (from, to));
}
if (from + to >= 5) {
alreadyOver = true;
}
}
}
return set;
}
Pair minimax (GameState s, int ply, boolean maxLevel) {
System.out.print(padding.toString() + s + " ");
Collection<IMove> validMoves = null;
if (ply > 0) { validMoves = computeMoves(s); }
if (ply == 0 || validMoves.isEmpty()) {
int score = eval.evaluate(s, original);
System.out.println(" [" + score + "]");
return new Pair (null, score);
}
System.out.println();
Pair best = null;
for (IMove m : validMoves) {
if (m.make(s)) {
padding.append(" ");
Pair next = minimax(s, ply-1, !maxLevel);
next.move = m;
padding.setLength(padding.length()-2);
m.undo(s);
if (maxLevel) {
if (best == null || next.score > best.score) { best = next; }
} else {
if (best == null || next.score < best.score) { best = next; }
}
}
}
System.out.println(padding.toString() + "
, " + best.score);
return best;
}
}
returning best: " + best.move + "
No w when yo u run it, yo u see this:
OBSERVE: Trace o f the Minimax algo rithm
(Player2,1,1,1,2)
(Player1,1,2,1,2)
(Player2,1,2,2,2) [5]
(Player2,1,2,2,3) [10]
(Player2,1,2,1,3) [15]
(Player2,1,2,1,4) [30]
returning best: Tap 1 with 1, 5
(Player1,1,3,1,2)
(Player2,1,3,2,2) [30]
(Player2,1,3,2,4) [50]
(Player2,1,3,1,3) [20]
(Player2,1,3,0,1) [-85]
returning best: Tap 2 with 3, -85
returning best: Tap 1 with 1, 5
best move: Tap 1 with 1
This o utput reflects the evaluatio n values presented in the earlier game tree. The indentatio n reflects the depth
in the game tree, and the evaluatio n o f each no de (fro m the perspective o f Player2) appears in brackets at the
end o f each ro w.
Let's see if this algo rithm can find the winning mo ve described earlier fo r Player2 in the state (Playe r2, 1, 3, 2,
4 ). Mo dify Print Gam e T re e as to expand o nly o ne level:
CODE TO TYPE: Mo dified PrintGameTree
package chopsticks;
public class PrintGameTree {
public static void main(String[] args) {
GameState gs = new GameState(GameState.Player2, 1, 3, 2, 41, 1, 1, 2);
IEvaluate eval = new Evaluator();
int ply = 12;
Minimax m = new Minimax(ply, eval);
IMove move = m.bestMove(gs, GameState.Player2);
System.out.println("best move:" + move);
}
}
Run it. It determines that the best mo ve is to tap the o ppo nent's hand with 3 fingers to fo rce a dead hand
fo r Player1:
OBSERVE: Co mputed 1-ply Minimax search o n (Player2, 1, 3, 2, 4)
(Player2,1,3,2,4)
(Player1,3,3,2,4) [-55]
(Player1,0,3,2,4) [45]
(Player1,0,1,2,4) [55]
returning best: Tap 3 with 2, 55
best move: Tap 3 with 2
Minimax wo rks best when it explo res sufficient levels o f the tree. Change the ply to 2 and o bserve that
Minimax no w finds a better mo ve:
OBSERVE: Co mputed 2-ply Minimax search o n (Player2, 1, 3, 2, 4)
(Player2,1,3,2,4)
(Player1,3,3,2,4)
(Player2,3,3,0,4) [-85]
(Player2,3,3,0,2) [-75]
returning best: Tap 2 with 3, -85
(Player1,0,3,2,4)
(Player2,0,3,0,4) [15]
(Player2,0,3,0,2) [25]
returning best: Tap 2 with 3, 15
(Player1,0,1,2,4)
(Player2,0,1,3,4) [90]
(Player2,0,1,0,2) [-5]
returning best: Tap 4 with 1, -5
returning best: Tap 1 with 4, 15
best move: Tap 1 with 4
In fact, yo u can increase ply to larger values, but the extra searches are redundant no w that a victo ry has been
fo und. There is a mo re efficient path-finding algo rithm called Alpha/Beta that can reduce the size o f game
trees dramatically by intelligently pruning redundant searches. Yo u can read abo ut this algo rithm in the
Algo rithms In A Nutshell co mpanio n bo o k.
Lessons Learned
Much o f the success o f Minim ax is derived fro m its ability to mo del bo th the game state and the available
mo ves in the game efficiently. Fo r cho psticks, there were three po tential ways to represent the game state.
Design the game state with care because each po tential mo ve class must implement three metho ds—valid,
make, and undo . If yo u select an o verly co mplicated mo deling structure, yo u will waste precio us time
debugging the mo ve classes. Yo u must cho o se a design that o ffers the greatest benefits to the mo st mo ve
classes.
Do no t skip the step where the IEvaluat e interface was defined. The success o f Minimax ultimately depends
o n having accurate and relevant evaluatio n classes to estimate the "strength" o f a bo ard fro m a player's po int
o f view. Crafting these heuristics is almo st an art fo rm and yo u'll want to experiment with a number o f
po tential evaluatio n classes.
When implementing an algo rithm, be sure to keep the lo gic o f the co re algo rithm fully encapsulated within its
o wn set o f classes. Use interfaces to identify the user-specified classes cleanly fo r the actual game pro blem
being so lved. That way, yo u can use the Minimax co de as an engine fo r multiple two -player games.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Algorithms On Sound Data
Lesson Objectives
When yo u finish this lesso n, yo u will be able to :
demo nstrate ho w to invo ke Fast Fo urier Transfo rm o n a third-party library.
generate so und wave fo rms to play using Java's Audio Fo rmat class.
co nvert a frequency into its no te equivalent o n a piano keybo ard.
Signal Processing Algorithms
The algo rithms in this co urse fo cus mo stly o n human-readable real-wo rld data, such as string values, integers,
flo ating-po int numbers, and Cartesian po ints. Wo uldn't it be great to be able to pro cess so und data, fo r example, to
detect the pitch o f a no te—o r even a cho rd o f no tes—being played? In this lesso n, yo u'll learn ho w to create and
pro cess so und data using the Fast Fourier Transform (FFT), which is o ften co nsidered o ne o f the mo st impo rtant
numerical algo rithms o f the 20 th century. Yo u will learn ho w to pro cess Wavefo rm Audio File Fo rmat (WAV) data
co ntaining unco mpressed audio enco ded using a linear pulse co de mo dulatio n (LPCM) fo rmat.
So und is a traveling lo ngitudinal wave which is an o scillatio n o f pressure. An individual wave is defined by its perio d
(the distance in time between two high po ints) and amplitude (the to tal distance vertically fro m the highest po int to the
lo west po int). The amplitude represents the energy o f the wave o r its "lo udness." Fo r this lesso n, we will assume that
all wave fo rms are no rmalized between [-1, 1] because the fo cus is o n frequency analysis.
The human ear interprets a so und wave by co nverting it into a musical pitch (o r no te). Each musical no te co rrespo nds
to a specific frequency which is measured in hertz, o r the number o f co mplete cycles per seco nd o f a perio dic
pheno meno n (in this case, the so und wave). Studies have demo nstrated that the range o f hearing fo r an infant child is
20 Hertz to 20 ,0 0 0 Hertz. The middle C o n a piano keybo ard is tuned to the frequency o f 26 1.6 26 Hertz, which is well
within this range (fo r additio nal frequency values, see the Wikipedia entry o n piano frequencies). If yo u were to sample
this so und wave 44,10 0 times per seco nd, yo u wo uld co mpute 44,10 0 individual values—the first 450 are sho wn
belo w in the blue time series. The ho rizo ntal axis (t-axis) represents time, while the vertical y-axis represents the
energy co ntained in the wave at time t.
To interpret the abo ve blue so und wave, yo u need to kno w the sampling rate and the time when the blue so und wave
co mpletes a full perio d. The blue wave perio d is abo ut 16 9 time units. Since there are 44,10 0 to tal samples, the
co mputed frequency o f the blue wave is 44100/169 o r 26 0 .9 5—clo se to the middle C frequency we mentio ned earlier.
The red so und wave abo ve represents the to ne when playing the E key just abo ve middle C. Based o n a perio d o f 135
time units, its frequency is co mputed to be 326 .6 6 —clo se to its actual value o f 329 .6 28 .
Pulse Co de Mo dulatio n (PCM) demo nstrates ho w to represent the co ntinuo us pro perties o f the wave fo rm discretely.
PCM represents an audio wavefo rm as a sequence o f amplitude values reco rded at a sequence o f times. LPCM is
PCM with linear quantizatio n. The standard audio file fo rmat fo r CDs, fo r example, is LPCM-enco ded with two channels
o f 44,10 0 samples per seco nd. Each sample is reco rded as an unsigned 16 -bit integer value.
To begin o ur investigatio n into so und data, we'll write a small pro gram that generates an 8 -bit quality so und wave fo rm
that plays a middle C no te.
Create a new Java Pro ject named So undFile s and assign it to the J ava6 _Le sso ns wo rking set.
In yo ur So undFile s pro ject /src so urce fo lder, create an f f t package.
In the f f t package, create a Ge ne rat e class as sho wn:
CODE TO TYPE: Generate class
package fft;
import javax.sound.sampled.*;
public class Generate {
public static void main(String[] args) throws Exception {
float sampleRate = 44100;
double f = 261.626;
double a = 0.5;
double twoPiF = 2*Math.PI*f;
double[] buffer = new double [44100];
for (int sample = 0; sample < buffer.length; sample++) {
double time = sample / sampleRate;
buffer[sample] = a * Math.sin(twoPiF*time);
}
final byte[] byteBuffer = new byte[buffer.length];
int idx = 0;
for (int i = 0; i < byteBuffer.length; ) {
int x = (int) (buffer[idx++]*127);
byteBuffer[i++] = (byte) x;
}
boolean bigEndian = false;
boolean signed = true;
int bits = 8;
int channels = 1;
AudioFormat format = new AudioFormat(sampleRate,bits,channels,signed,bigEndian);
DataLine.Info info = new DataLine.Info(SourceDataLine.class, format);
SourceDataLine line = (SourceDataLine) AudioSystem.getLine(info);
line.open(format);
line.start();
long now = System.currentTimeMillis();
line.write(byteBuffer, 0, byteBuffer.length);
line.close();
long total = System.currentTimeMillis() - now;
System.out.println(total + " ms.");
}
}
Save and run it. Yo u hear a to ne that so unds like the middle C no te o n the piano . To create this so und, this class
generates a o ne-seco nd wave fo rm using 8 -bit so und enco ding. Let's take a clo ser lo o k:
OBSERVE: Creating Wave Fo rm
float sampleRate = 44100;
double f = 261.626;
double a = 0.5;
double twoPiF = 2*Math.PI*f;
double[] buffer = new double[44100];
for (int sample = 0; sample < buffer.length; sample++) {
double time = sample / sampleRate;
buffer[sample] = a * Math.sin(twoPiF*time);
}
So und data can be represented using a time series co mputed using the trigo no metric Sine functio n. In a time series,
the value t represents the time unit, which increments fro m 0 to increasing po sitive numbers. The variable f specifies
the desired frequency (in this case, Middle C). The variable a represents a scaling facto r (between 0 and 1) to apply to
the amplitude o f the wave. The wave will be at its "lo udest" when a is 1.0 , and so fter with decreasing values o f a. The
variable t wo PiF pre-co mputes the co nstant value used within the f o r lo o p fo r o ptimizatio n. buf f e r co ntains the
sequence o f 44,10 0 flo ating po int values representing the wave fo rm. With each pass thro ugh the f o r lo o p, t im e
represents the t-co o rdinate fo r which the wave fo rm is co mputed using the fo rmula a*Sin(2*PI*f*t).
Once buf f e r is co mputed, it must be co nverted into its co rrespo nding byte enco ding. Fo r 8 -bit so und quality, there are
256 different values that can be generated. Naturally, 16 -bit so und quality is able to mo re accurately mo del a so und
wave fo rm because it allo ws fo r a to tal o f 6 5,536 different values:
OBSERVE: Create byte buffer fro m flo ating-po int buffer
final byte[] byteBuffer = new byte[buffer.length];
int idx = 0;
for (int i = 0; i < byteBuffer.length; ) {
int x = (int) (buffer[idx++]*127);
byteBuffer[i++] = (byte) x;
}
The flo ating po int values are "scaled" to beco me signed byte values (-128 to 127). The co de actually o nly co mputes
255 po ssible values (fro m -127 to 127) because o f the co nversio n fro m int to byt e , but that's acceptable when
digitizing a Sine wave. The image belo w charts the sample byte values o f this buffer. The values o nly range fro m -6 3 to
+6 3 because the amplitude o f the generated wave fo rm, a, is scaled at 0 .5 :
The size o f byteBuffer is the same as the o riginal buffer. The JDK plays the 8 -bit enco ded byte buffer:
OBSERVE: Playing bytes as so und
boolean bigEndian = false;
boolean signed = true;
int bits = 8;
int channels = 1;
AudioFormat format = new AudioFormat(sampleRate,bits,channels,signed,bigEndian);
DataLine.Info info = new DataLine.Info(SourceDataLine.class, format);
SourceDataLine line = (SourceDataLine) AudioSystem.getLine(info);
line.open(format);
line.start();
long now = System.currentTimeMillis();
int written = line.write(byteBuffer, 0, byteBuffer.length);
line.close();
System.out.println(written + " bytes written.");
long total = System.currentTimeMillis() - now;
System.out.println(total + " ms.");
The Audio Fo rm at o bject represents the enco ding used so the underlying audio so ftware can interpret the bytes it
receives pro perly. As yo u can imagine, there are numero us enco ding styles and hardware devices available to
pro cess these enco dings. Here the co de specifies that:
the bytes are enco ded in little-Endian o rder, fro m least significant bit to greatest. This o nly matters fo r 16 -bit
and higher enco dings.
the bytes are signed (where negative numbers are "belo w" the x-axis).
there are 8 bits in each enco ding, thus the audio hardware will read 8 bits at a time.
there is just a single channel o f o utput.
An Audio Fo rmat o bject co uld have multiple streams o f input, called channels. A stereo audio so urce, fo r example,
wo uld have two channels (left and right). A mo no so urce wo uld have o nly a single channel.
Once the fo rmat is declared, the co de creates a So urce Dat aLine o bject to manage the transfer o f bytes. First, the
line is o pe ne d wit h t he de clare d Audio Fo rm at o bje ct . Then the st art m e t ho d is invo ke d to declare that it
must be ready to receive the data, which is writ t e n as a single blo ck writ e . Finally, the line is clo se d and f inal
st at ist ics are o ut put .
Run this co de again; yo u might be surprised to hear that the so und seems to play fo r much less than o ne seco nd.
The o utput sho ws that the pro gram ran in abo ut 1/2 seco nd:
OBSERVE: Generate o utput
44100 bytes written.
565 ms.
What might be go ing wro ng? Well, no rmally all so und info rmatio n to be played is buffered prio r to being delivered to
the hardware. To determine the size o f the buffer, mo dify Ge ne rat e as sho wn:
CODE TO TYPE: Detect size o f so und buffer
...
long now = System.currentTimeMillis();
System.out.println("buffer size:" + line.available());
int written = line.write(byteBuffer, 0, byteBuffer.length);
...
Run it again; yo u see that the buffer size is 22,0 50 , which suppo rts abo ut 1/2 seco nd o f audio . The pro gram
co mpletes befo re the audio hardware finishes playing the so und. The simplest way to fix this is to make sure that all
bytes sent to the audio hardware are drained befo re it can be clo sed. This means calling drain() blo cks until all data
has been played.
CODE TO TYPE: Pro perly drain buffer to play entire so und
...
int written = line.write(byteBuffer, 0, byteBuffer.length);
line.drain();
line.close();
...
No w in additio n to playing the so und fo r a full seco nd, the o utput sho ws so mething like this:
OBSERVE: Pro per executio n o f Generate
buffer size:22050
44100 bytes written.
1035 ms.
Yo u can have lo ts o f fun with so und wave fo rms. The next change generates a stereo signal o f middle C being played,
but the amplitude changes in the left and right sides to pro vide the illusio n o f so und depth o ver a two -seco nd perio d:
CODE TO TYPE: Mo dify Generate to generate stereo o utput
package fft;
import javax.sound.sampled.*;
public class Generate {
public static void main(String[] args) throws Exception {
float sampleRate = 44100;
double f = 261.626;
double a = .5;
double twoPiF = 2*Math.PI*f;
double[] buffer = new double[44100*4];
for (int sample = 0; sample < buffer.length; sample++) {
double time = (sample/2) / sampleRate;
double a1 = a*Math.sin(Math.PI*time);
double a2 = a*Math.cos(Math.PI*time);
buffer[sample++] = a1 * Math.sin(twoPiF*time);
// channel 1
buffer[sample] = a2 * Math.sin(twoPiF*time);
// channel 2
}
byte[] byteBuffer = new byte[buffer.length];
int idx = 0;
for (int i = 0; i < byteBuffer.length; ) {
int x = (int) (buffer[idx++]*127);
byteBuffer[i++] = (byte) x;
}
boolean bigEndian = false;
boolean signed = true;
int bits = 8;
int channels = 2;1;
AudioFormat format = new AudioFormat(sampleRate,bits,channels,signed,bigEndian);
DataLine.Info info = new DataLine.Info(SourceDataLine.class, format);
SourceDataLine line = (SourceDataLine) AudioSystem.getLine(info);
line.open(format);
line.start();
long now = System.currentTimeMillis();
System.out.println("buffer size:" + line.available());
int written = line.write(byteBuffer, 0, byteBuffer.length);
System.out.println(written + " bytes written.");
line.drain();
line.close();
long total = System.currentTimeMillis() - now;
System.out.println(total + " ms.");
}
}
Experiment with the co de so me mo re. Fo r example, change the frequency, f , to determine the lo west o r highest pitch
that yo u can hear.
To be able to pro cess actual so und files co ntaining reco rded music, yo u need to wo rk with 16 -bit so und data. Mo dify
Ge ne rat e as sho wn to recreate a 16 -bit mo no enco ding o f just middle C:
CODE TO TYPE: Mo dificatio ns to Generate class
package fft;
import javax.sound.sampled.*;
public class Generate {
public static void main(String[] args) throws Exception {
float sampleRate = 44100;
double f = 261.626;
double a = .5;
double twoPiF = 2*Math.PI*f;
double[] buffer = new double [44100*42];
for (int sample = 0; sample < buffer.length; sample++) {
double time = (sample/2) / sampleRate;
double a1 = a*Math.sin(Math.PI*time);
double a2 = a*Math.cos(Math.PI*time);
buffer[sample++] = a1 * Math.sin(twoPiF*time); // channel 1
buffer[sample] = a2 * Math.sin(twoPiF*time);
// channel 2
}
byte[] byteBuffer = new byte[buffer.length*2];
int idx = 0;
for (int i = 0; i < byteBuffer.length; ) {
int x = (int) (buffer[idx++]*25532767);
byteBuffer[i++] = (byte) x;
byteBuffer[i++] = (byte) (x >>> 8);
}
boolean bigEndian = false;
boolean signed = true;
int bits = 8;16;
int channels = 2;1;
AudioFormat format = new AudioFormat(sampleRate,bits,channels,signed,bigEndian);
DataLine.Info info = new DataLine.Info(SourceDataLine.class, format);
SourceDataLine line = (SourceDataLine) AudioSystem.getLine(info);
line.open(format);
line.start();
long now = System.currentTimeMillis();
System.out.println("buffer size:" + line.available());
int written = line.write(byteBuffer, 0, byteBuffer.length);
System.out.println(written + " bytes written.");
line.drain();
line.close();
long total = System.currentTimeMillis() - now;
System.out.println(total + " ms.");
}
}
Yo u pro bably wo n't detect any audible difference, since it still plays a middle C to ne fo r just abo ut two seco nds.
We've o nly made mino r changes to the earlier 8 -bit mo no versio n. Let's take a clo ser lo o k at the way byteBuffer is
co nstructed:
OBSERVE: Create byte buffer with byte pairs
byte[] byteBuffer = new byte[buffer.length*2];
int idx = 0;
for (int i = 0; i < byteBuffer.length; ) {
int x = (int) (buffer[idx++]*32767);
byteBuffer[i++] = (byte) x;
byteBuffer[i++] = (byte) (x >>> 8);
}
To sto re 16 -bit data, yo u need a byte buffer t wice as large as t he 8-bit so lut io n. In additio n, the byte values are no
lo nger simply drawn fro m the range o f 256 values (o r 2 8 ). Instead yo u need to represent 6 5,536 (o r 2 16 ) different
values. To do this, the value is enco ded into two neighbo ring byte values. This co de will generate o nly 6 5,535
po ssible enco dings, but that's acceptable. Using little-Endian enco ding, the lo wer 8 bits o f the enco ded value, x, is
writ t e n o ut f irst , then the value o f x is shif t e d 8 bit s t o t he right befo re it is written o ut. Finally, the number o f
bits in the Audio Fo rmat o bject is changed fro m 8 to 16 and the number o f channels is set to 1. To represent the audio
in big-Endian enco ding, yo u wo uld simply swap the o rder o f these two bytes in byteBuffer.
Composed Wave Forms
Let's return to the o riginal plo t at the start o f this lesso n, which co ntained a blue wave fo rm representing
middle C and a red wave representing the E abo ve middle C. Instead o f depicting these separately, the image
belo w represents the combined sound wave o f these two no tes playing simultaneo usly. The wave data is
no rmalized so its values all remain within the [-1, 1] range:
Where befo re the so und waves sho wed clear signs o f perio dicity, this wave fo rm seems unintelligible.
Ho wever, if yo u plo t mo re samples, the perio dic structure beco mes visible:
Mo dify the Ge ne rat e class as sho wn:
CODE TO TYPE: Mo dificatio ns to Generate
package fft;
import javax.sound.sampled.*;
public class Generate {
public static void main(String[] args) throws Exception {
float sampleRate = 44100;
double f1 = 261.626;
double f2 = 329.628;
double a = .5;
double twoPiF1 = 2*Math.PI*f1;
double twoPiF2 = 2*Math.PI*f2;
double[] buffer = new double [44100*2];
for (int sample = 0; sample < buffer.length; sample++) {
double time = sample / sampleRate;
buffer[sample] = a * (Math.sin(twoPiF1*time) + Math.sin(twoPiF2*time))/2;
}
byte[] byteBuffer = new byte[buffer.length*2];
int idx = 0;
for (int i = 0; i < byteBuffer.length; ) {
int x = (int) (buffer[idx++]*32767);
byteBuffer[i++] = (byte) x;
byteBuffer[i++] = (byte) (x >>> 8);
}
boolean bigEndian = false;
boolean signed = true;
int bits = 16;
int channels = 1;
AudioFormat format = new AudioFormat(sampleRate,bits,channels,signed,bigEndi
an);
DataLine.Info info = new DataLine.Info(SourceDataLine.class, format);
SourceDataLine line = (SourceDataLine) AudioSystem.getLine(info);
line.open(format);
line.start();
long now = System.currentTimeMillis();
System.out.println("buffer size:" + line.available());
int written = line.write(byteBuffer, 0, byteBuffer.length);
System.out.println(written + " bytes written.");
line.drain();
line.close();
long total = System.currentTimeMillis() - now;
System.out.println(total + " ms.");
}
}
Save and run it. Yo u hear two no tes playing simultaneo usly (Middle C at a frequency o f 26 1.6 26 and the E
no te just abo ve it at a frequency o f 329 .6 38 ). The o nly real difference with earlier co de is the co nstructio n o f
the flo ating-po int buffer. Let's review this co de:
OBSERVE: Create co mpo sed So und Wave
float sampleRate = 44100;
double f1 = 261.626;
double f2 = 329.628;
double a = 0.5;
double twoPiF1 = 2*Math.PI*f1;
double twoPiF2 = 2*Math.PI*f2;
double[] buffer = new double [44100*2];
for (int sample = 0; sample < buffer.length; sample++) {
double time = sample / sampleRate;
buffer[sample] = a * (Math.sin(twoPiF1*time) + Math.sin(twoPiF2*time))/2;
}
The values in buf f e r are the co mpo sitio n o f two so unds being played. The values in buf f e r must be in the
range [-1, 1]. When adding two Sine values to gether, t he co de divide s by 2 to ensure that the resulting value
remains within this range.
Analyzing Composed Wave Forms
To analyze a co mpo sed wave fo rm, yo u want to identify the individual so und wave frequencies that represent
the do minant co mpo nents o f the co mpo sitio n. In mathematics, there is a Discrete Fourier Transform (DFT)
that can co nvert a sampled functio n into a finite co mbinatio n o f co mplex sinuso idal functio ns o rdered by their
frequencies that has the same sample values. There are three impo rtant co ncepts presented in this o ne
sentence:
Co nve rt a Sam ple d Funct io n: Yo u may kno w that yo u can determine a line uniquely by just two
po ints. That is, given just two pairs o f (x,y) values o n a line, yo u can determine its equatio n. By
analo gy, the sampled so und frequencies are being treated like individual po ints, this time with a tcoordinate representing the time o f that sample and a y-coordinate representing the amplitude o f
the wave at that time unit. Based so lely o n this info rmatio n, yo u're trying to determine a functio n f(t)
that satisfies all o f these po ints.
A f init e co m binat io n o f sinuso idal f unct io ns: In the co mpo sed wave fo rm example, the
resulting cho rd is co mputed by adding to gether two sinuso idal functio ns. In general, yo u canno t
kno w in advance ho w many sinuso idal functio ns are present in any co mplex wave fo rm, but yo u
can assume that yo u are lo o king fo r o nly a finite number.
Co m ple x sinuso idal f unct io ns: The DFT is defined o ver the set o f complex numbers, which are
numbers that can be expressed in the fo rm a + bi, where a and b are real numbers and i is the
imaginary unit, defined as i2 = 1. Every real number is already a co mplex number (with the
imaginary part o f b=0). Seco nd, we are co ncerned with the magnitude o f the co mplex numbers
being pro cessed. Fo r a co mplex number o f the fo rm a + bi, its magnitude is the square ro o t o f (a*a
+ b*b).
Here is the single fo rmula that "explains" the DST:
Ok, that's a bit much. Fo r this lesso n yo u do n't need to understand ho w this fo rmula was derived, but I'll sho w
yo u ho w to implement it in Java. Co mplex numbers use the special variable i to represent the imaginary unit;
because yo u can see i in the abo ve fo rmula, yo u kno w that it relies o n co mputatio ns o ver co mplex numbers.
Yo u can reduce the right half o f the abo ve fo rmula by co nverting the expo nential value o f e like this:
Yo u accumulate X(k) by x(t) times the abo ve, making sure to deal with the co mplex values that result fro m the
co mputatio n pro perly. Yo u are given n, which is the number o f sampled values, x(t). Yo u o nly need to
determine the range o f frequencies fo r k.
The term x(t) represents the sample value fo r time unit t; there are n sample values in all. Yo u want to
determine X(k), which represents the signal level fo r frequency k. No w, fo r which values o f k are yo u go ing to
co mpute X? In any input sample, the highest frequency is 1/2 the to tal number o f samples (based o n the
co ncept o f Nyquist Frequency). Ho wever, this still leaves yo u with n*n/2 co mputatio ns, o r O(n 2 ) co mputatio ns.
Yo u can likely execute DFT o n o nly a small number o f samples befo re it beco mes to o co stly to execute.
Once the values o f X(k) are co mputed, yo u can investigate them to find tho se maximal values which directly
co rrelate to the existence o f a wave fo rm in the input with frequency k.
Let's write so me co de to co mpute DFT. Yo u can reuse the cho rd generatio n co de abo ve. Generally, DFT is
meant to pro cess co mplex values as input; ho wever, yo ur input co nsists o f real valued samples, so the co de
is a bit simpler than the generic DFT.
In the f f t package, create a DFT class as sho wn:
CODE TO TYPE: DFT class
package fft;
public class DFT {
static void dft(double[] inR, double[] outR, double[] outI) {
for (int k = 0; k < inR.length; k++) {
for (int t = 0; t < inR.length; t++) {
outR[k] += inR[t]*Math.cos(2*Math.PI * t * k / inR.length);
outI[k] -= inR[t]*Math.sin(2*Math.PI * t * k / inR.length);
}
}
}
public static void main(String[] args) throws Exception {
float sampleRate = 44100;
double f1 = 261.626;
double f2 = 329.628;
double a = .5;
double twoPiF1 = 2*Math.PI*f1;
double twoPiF2 = 2*Math.PI*f2;
double[] bufferR = new double [2048];
for (int sample = 0; sample < bufferR.length; sample++) {
double time = sample / sampleRate;
bufferR[sample] = a * (Math.sin(twoPiF1*time) + Math.sin(twoPiF2*time))/2;
}
double[] outR = new double[bufferR.length];
double[] outI = new double[bufferR.length];
dft(bufferR, outR, outI);
double results[] = new double[outR.length];
for (int i = 0; i < outR.length; i++) {
results[i] = Math.sqrt(outR[i]*outR[i] + outI[i]*outI[i]);
}
java.io.PrintStream ps = new java.io.PrintStream("Sample.txt");
for (double d : results) {
ps.println(d);
}
ps.close();
}
}
Let's lo o k at this co de mo re clo sely. The first half o f the m ain metho d is identical to earlier co de that
co nstructs a co mpo sed wave fo rm fro m playing two no tes (C and E):
OBSERVE: Invo king DFT o n the wave fo rm
double[] outR = new double[bufferR.length];
double[] outI = new double[bufferR.length];
dft(bufferR, outR, outI);
double results[] = new double[outR.length];
for (int i = 0; i < outR.length; i++) {
results[i] = Math.sqrt(outR[i]*outR[i] + outI[i]*outI[i]);
}
The dft metho d co mputes t wo buf f e rs, o ut R and o ut I, which co ntain the n co mputed co mplex values o f
X(k). These are co m po se d int o a single re sult s array by determining the magnitude o f the co mplex
number. The magnitude o f the co mplex number a + bi is co mputed as the square ro o t o f a*a + b*b.
The actual DFT co mputatio n is perfo rmed in the df t metho d, which is simplified because the input co ntains
o nly real numbers, no t co mplex numbers:
OBSERVE: DFT implementatio n
static void dft(double[] inR, double[] outR, double[] outI) {
for (int k = 0; k < inR.length; k++) {
for (int t = 0; t < inR.length; t++) {
outR[k] += inR[t]*Math.cos(2*Math.PI * t * k / inR.length);
outI[k] -= inR[t]*Math.sin(2*Math.PI * t * k / inR.length);
}
}
}
Given n sam ple s in inR, this lo o p perfo rms n*n o peratio ns, ultimately accumulating the pro per co mplex
number result in o ut R and o ut I. When this metho d co mpletes, these two arrays co ntain the co mplex values
o f X(k).
Save and run DFT; this will create a file "Sample.txt" in the current Eclipse pro ject. To see this file, select the
enclo sing pro ject and right-click to select Re f re sh.
Because there are 44,10 0 sample values in just o ne seco nd o f so und data, the DFT is inefficient fo r o ur
purpo ses. Here, o nly 20 48 samples are used to enable the co mputatio n to co mplete in under a seco nd.
Ho wever, ho w do yo u kno w that the result is accurate? Retrieve the values fro m "Sample.txt" and plo t them
using a pro gram such as Excel.
This graph is symmetric. The x-axis represents a frequency index, which divides the 44,10 0 (the sample rate)
po ssible frequencies into 2,0 48 (the number o f samples being used fo r DFT) discrete o nes. This graph
further demo nstrates that yo u o nly need to co nsider the first half o f these frequencies. Let's fo cus o n the first
35 values:
We are co ncerned with o nly the magnitude o f these values and the two highest po ints that o ccur are the 13th
and 16 th po ints. These indices are based o n co unting fro m zero , so these are actually frequency indices o f 12
and 15. Yo u can co nvert these frequency indices into actual frequencies like this:
12*4410 0 /20 48 = 258 .39 8 4375
15*4410 0 /20 48 = 322.9 9 8 0 46 8 75
These two values are really clo se to the frequencies o f the C and E no tes in the co mpo sed wave fo rm. Think
abo ut what this co de has acco mplished! Given a buffer co ntaining a co mpo sed wave fo rm, the DFT was
so meho w able to iso late the two do minant frequencies. No w let's add so me quick and dirty pro cessing co de
to identify these maximum peaks within the results array o f a DFT, which will allo w yo u to detect these
frequencies in the co mpo sed wave fo rm. Fo r mo re co mplex wave fo rms, yo u will need a mo re nuanced
appro ach, but this gives yo u an idea o f what's po ssible. Mo dify DFT as sho wn:
CODE TO TYPE: Mo dificatio ns to DFT
package fft;
import java.util.*;
public class DFT {
static void dft(double[] inR, double[] outR, double[] outI) {
for (int k = 0; k < inR.length; k++) {
for (int t = 0; t < inR.length; t++) {
outR[k] += inR[t]*Math.cos(2*Math.PI * t * k / inR.length);
outI[k] -= inR[t]*Math.sin(2*Math.PI * t * k / inR.length);
}
}
}
public static void main(String[] args) throws Exception {
float sampleRate = 44100;
double f1 = 261.626;
double f2 = 329.628;
double a = .5;
double twoPiF1 = 2*Math.PI*f1;
double twoPiF2 = 2*Math.PI*f2;
double[] bufferR = new double [2048];
for (int sample = 0; sample < bufferR.length; sample++) {
double time = sample / sampleRate;
bufferR[sample] = a * (Math.sin(twoPiF1*time) + Math.sin(twoPiF2*time))/2;
}
double[] outR = new double[bufferR.length];
double[] outI = new double[bufferR.length];
dft(bufferR, outR, outI);
double results[] = new double[outR.length];
for (int i = 0; i < outR.length; i++) {
results[i] = Math.sqrt(outR[i]*outR[i] + outI[i]*outI[i]);
}
java.io.PrintStream ps = new java.io.PrintStream("Sample.txt");
for (double d : results) {
ps.println(d);
}
ps.close();
List<Float> found = process(results, sampleRate, bufferR.length, 4);
for (float freq : found) {
System.out.println("Found: " + freq);
}
}
static List<Float> process(double results[], float sampleRate, int numSamples,
int sigma) {
double average = 0;
for (int i = 0; i < results.length; i++) {
average += results[i];
}
average = average/results.length;
double sums = 0;
for (int i = 0; i < results.length; i++) {
sums += (results[i]-average)*(results[i]-average);
}
double stdev = Math.sqrt(sums/(results.length-1));
ArrayList<Float> found = new ArrayList<Float>();
double max = Integer.MIN_VALUE;
int maxF = -1;
for (int f = 0; f < results.length/2; f++) {
if (results[f] > average+sigma*stdev) {
if (results[f] > max) {
max = results[f];
maxF = f;
}
} else {
if (maxF != -1) {
found.add(maxF*sampleRate/numSamples);
max = Integer.MIN_VALUE;
maxF = -1;
}
}
}
return (found);
}
}
The pro ce ss metho d co mputes the average and standard deviatio n o f the re sult s array and it eliminates
fro m co nsideratio n any frequency index f with a results[f] that is smaller than average + 4*stdev which sho uld
eliminate 9 9 .73% o f the frequency indices fro m co nsideratio n. If a magnitude fo r a particular index is higher
than this thresho ld, it warrants further co nsideratio n. No w sweeping f fro m 0 to n/2 where n is the number o f
co mputed values in re sult s the f o r lo o p seeks to find a lo cal maximum, max, and its co rrespo nding
frequency index value, maxF. Then it co mputes the detected frequency by multiplying the frequency index,
maxF, by the sampleRate and dividing by numSamples.
Save and run it; these frequencies are detected in the o utput:
OBSERVE: Execute DFT
Found: 258.39844
Found: 322.99805
To validate that this co de functio ns as expected, change the f 1 and f 2 values to two different frequencies in
the range 27 to 4,18 6 , which represent the full range o f the 8 8 keys o n the keybo ard. The co mputatio ns wo n't
identify the frequency precisely because the accuracy o f the co mputatio n is limited to 4410 0 /20 48 o r 21.5
hertz. The o nly way to increase accuracy is to increase the number o f samples pro cessed by DFT. Ho wever,
do ing so will dramatically slo w the co mputatio n because o f the O(n 2 ) behavio r. If yo u rerun the abo ve co de
using different samples, 40 9 6 and 8 19 2 respectively, yo u get these results in ro ughly the identified time:
40 9 6 : (3 seco nds) 258 .39 8 44, 333.76 46 5
8 19 2: (10 seco nds) 26 3.78 174, 328 .38 135
There is a mo re efficient versio n kno wn as the Fast Fourier Transform (FFT). In this lesso n, yo u will learn ho w
to use FFT, rather than implement it, because o f the numerical co mplexity o f the algo rithm. There are a
number o f freely available implementatio ns.
To get this library, right-click this link and save the file in yo ur wo rkspace.
Then, add the co m m o ns-m at h3-3.2.jar library to be part o f the build path in Eclipse. Right-click o n yo ur
pro ject ico n within the wo rkspace and select the Pro pe rt ie s entry; o n the left side, select J ava Build Pat h.
Yo u see this dialo g:
Click the Add J ARs... butto n o n the right and use the pro vided windo w to bro wse in yo ur pro ject to the libs
fo lder where the co m m o ns-m at h3-3.2.jar file exists. Select it and click OK. No w yo u are ready to start using
the FFT co de which is part o f this JAR file.
The best way to learn FFT is to use it. Mo dify the DFT class to execute FFT o n the buffer o f do uble values that
it creates. The o nly requirement that FFT has is that the input size is a perfect po wer o f 2. Otherwise, it
pro duces exactly the same result fo rmat as DFT:
CODE TO TYPE: Mo dificatio ns to DFT
package fft;
import java.util.*;
import org.apache.commons.math3.complex.Complex;
import org.apache.commons.math3.transform.*;
public class DFT {
static void dft(double[] inR, double[] outR, double[] outI) {
for (int k = 0; k < inR.length; k++) {
for (int t = 0; t < inR.length; t++) {
outR[k] += inR[t]*Math.cos(2*Math.PI * t * k / inR.length);
outI[k] -= inR[t]*Math.sin(2*Math.PI * t * k / inR.length);
}
}
}
public static void main(String[] args) throws Exception {
float sampleRate = 44100;
double f1 = 261.626;
double f2 = 329.628;
double a = .5;
double twoPiF1 = 2*Math.PI*f1;
double twoPiF2 = 2*Math.PI*f2;
double[] bufferR = new double [2048];
for (int sample = 0; sample < bufferR.length; sample++) {
double time = sample / sampleRate;
bufferR[sample] = a * (Math.sin(twoPiF1*time) + Math.sin(twoPiF2*time))/2;
}
double[] outR = new double[bufferR.length];
double[] outI = new double[bufferR.length];
dft(bufferR, outR, outI);
FastFourierTransformer fft = new FastFourierTransformer(DftNormalization.STA
NDARD);
Complex resultC[] = fft.transform(bufferR, TransformType.FORWARD);
double results[] = new double[outR.length];
for (int i = 0; i < outR.length; i++) {
results[i] = Math.sqrt(outR[i]*outR[i] + outI[i]*outI[i]);
}
for (int i = 0; i < resultC.length; i++) {
double real = resultC[i].getReal();
double imaginary = resultC[i].getImaginary();
results[i] = Math.sqrt(real*real + imaginary*imaginary);
}
List<Float> found = process(results, sampleRate, bufferR.length, 4);
for (float freq : found) {
System.out.println("Found: " + freq);
}
}
static List<Float> process(double results[], float sampleRate, int numSamples,
int sigma) {
double average = 0;
for (int i = 0; i < results.length; i++) {
average += results[i];
}
average = average/results.length;
double sums = 0;
for (int i = 0; i < results.length; i++) {
sums += (results[i]-average)*(results[i]-average);
}
double stdev = Math.sqrt(sums/(results.length-1));
ArrayList<Float> found = new ArrayList<Float>();
double max = Integer.MIN_VALUE;
int maxF = -1;
for (int f = 0; f < results.length/2; f++) {
if (results[f] > average+sigma*stdev) {
if (results[f] > max) {
max = results[f];
maxF = f;
}
} else {
if (maxF != -1) {
found.add(maxF*sampleRate/numSamples);
max = Integer.MIN_VALUE;
maxF = -1;
}
}
}
return (found);
}
}
Save and run it; yo u get the same result as befo re, but much mo re quickly. FFT makes it practical to
accurately pro cess co mpo sed wave fo rms. The next step, naturally, is to try to execute FFT o n actual reco rded
audio samples. Let's get started.
Using FFT on WAV file samples
Yo ur pro ject sho uld co me with so me existing WAV reso urces fo r yo u to use. Once yo u finish this lesso n,
make yo ur o wn so und reco rdings to see if yo u can replicate the pro cessing do ne here. Wo rking with real
so und files intro duces a number o f additio nal issues. Let's see ho w to lo ad up a WAV so und file co ntaining
16 -bit enco ded data that needs to be co nverted into do uble values; this sequence is essentially the reverse
o f the so und generatio n yo u did at the beginning o f this lesso n.
There are five so und files to be pro cessed:
CFA_Majo rCho rd.wav: 7 seco nds o f a C-F-A cho rd played o n a regular piano
CMajo rCho rd.wav: 7 seco nds o f a C-E-G majo r cho rd played o n a regular piano
Clavino vaCMajo rCho rd.wav: 7 seco nds o f a C-E-G majo r cho rd played o n an electric Yamaha
Clavino va
CSeventhCho rd.wav: 7 seco nds o f a C-E-G-Bb cho rd (Majo r C-7th) played o n a regular piano
CrystalGlass.wav: 4 seco nds o f the ringing o f a crystal glass
Let's pro cess the clearest signal—the crystal glass.
In the f f t package, create a WAVPro ce ssing class as sho wn:
CODE TO TYPE: WAVPro cessing
package fft;
import
import
import
import
import
java.io.*;
java.util.*;
javax.sound.sampled.*;
org.apache.commons.math3.complex.Complex;
org.apache.commons.math3.transform.*;
public class WAVProcessing {
public static void main(String[] args) throws Exception {
File fileIn = new File("chords\\CrystalGlass.wav");
AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(fileIn);
System.out.println(audioInputStream.getFormat());
int size = audioInputStream.available();
byte[] bytesIn = new byte[size];
audioInputStream.read(bytesIn);
AudioFormat format = audioInputStream.getFormat();
float rate = format.getFrameRate();
int numChannels = format.getChannels();
double[] buffer = new double [1048576];
int idx = 0;
for (int i = 0; i < bytesIn.length && idx < buffer.length; i += 2) {
byte blow = bytesIn[i];
byte bhigh = bytesIn[i+1];
buffer[idx++] = (blow & 0xFF | bhigh << 8)/32767;
if (numChannels == 2) { i += 2; }
}
FastFourierTransformer fft = new FastFourierTransformer(DftNormalization.STA
NDARD);
Complex resultC[] = fft.transform(buffer, TransformType.FORWARD);
double[] results = new double[resultC.length];
for (int i = 0; i < resultC.length; i++) {
double real = resultC[i].getReal();
double imaginary = resultC[i].getImaginary();
results[i] = Math.sqrt(real*real + imaginary*imaginary);
}
List<Float> found = DFT.process(results, rate, resultC.length, 7);
HashMap<String,Float> keys = new HashMap<String,Float>();
System.out.println("Found:" + found);
for (float freq : found) {
keys.put(closestKey(freq), freq);
}
for (String note : keys.keySet()) {
System.out.println("Found: " + note + " @ freq=" + keys.get(note));
}
}
static String[] notes = {"A", "A#", "B", "C", "C#", "D", "D#", "E", "F", "F#",
"G", "G#"};
public static String closestKey(double freq) {
int key = closestKeyIndex(freq);
if (key <= 0) { return null; }
int range = 1+(key-1)/notes.length;
return notes[(key-1)%notes.length] + range;
}
public static int closestKeyIndex(double freq) {
return 1+(int)((12*Math.log(freq/440)/Math.log(2) + 49) - 0.5);
}
}
Save and run it:
OBSERVE: CrystalGlass Analysis
PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian
Found:[1645.6919, 1646.7013, 1647.5845, 1651.4116, 1664.2811, 3217.153]
Found: G#6 @ freq=1664.2811
Found: G7 @ freq=3217.153
The co de may have tro uble distinguishing frequencies at the higher o ctaves o n the piano . Ideally the so und o f
crystal glass wo uld have o nly o ne harmo nic subto ne o ne o ctave abo ve the base to ne o f G# 6 . Yo u can see
that the identified frequences are nearly do uble each o ther.
Mo st o f this co de is familiar to yo u by no w. Let's review the new additio ns:
OBSERVE: Co nverting Frequency into a piano no te
static String[] notes = {"A", "A#", "B", "C", "C#", "D", "D#", "E", "F", "F#",
"G", "G#"};
public static String closestKey(double freq) {
int key = closestKeyIndex(freq);
if (key <= 0) { return null; }
int range = 1+(key-1)/notes.length;
return notes[(key-1)%notes.length] + range;
}
public static int closestKeyIndex(double freq) {
return 1+(int)((12*Math.log(freq/440)/Math.log(2) + 49) - 0.5);
}
A number o f frequencies were detected by FFT. The co de we write next co nso lidates these frequencies into
distinct pitches using a HashMap to asso ciate the detected frequency with the clo sest key as co mputed
abo ve.
The no t e s static field reco rds the 12 distinct no tes as fo und o n a piano . Each to ne o ccurs at a given octave
number. The lo west no te o n the 8 8 -key piano keybo ard is key number 1 (A0 ); the highest no te is key number
8 8 (C8 ). The clo se st Ke yInde x metho d takes a frequency and returns the co rrespo nding key number o n the
piano in the range fro m 1-8 8 . This fo rmula is derived fro m the lo garithmic nature o f the frequencies. The
clo se st Ke y functio n co nverts this number into a human-readable string representing the no te o n the piano
that mo st clo sely co rrespo nds to the given frequency:
OBSERVE: co nso lidate frequencies into pitches
List<Float> found = DFT.process(results, rate, resultC.length, 7);
HashMap<String,Float> keys = new HashMap<String,Float>();
System.out.println("Found:" + found);
for (float freq : found) {
keys.put(closestKey(freq), freq);
}
Frequencies that are "clo se to gether" beco me co nso lidated in the HashMap, so o nly two detected no tes
appear in the o utput.
The no tes o n an ideal piano range fro m a lo w frequency o f 27.5 to a high o f 418 6 .0 1. Instead o f being evenly
spaced, the no tes are arranged in o ctaves that are multiples o f each o ther. Fo r example, middle C is the
frequency 26 1.6 26 , while the C o ne o ctave higher is 523.251. The clo se st Ke yInde x metho d co mputes the
piano key index with 1 being the lo west key o n the piano and 8 8 being the highest key. If the frequency is
lo wer than the lo west key o n the piano , this metho d returns a number smaller than 1; that's why the
clo se st Ke y metho d pro tects against this situatio n.
Lessons Learned
So und wave dat a has a st ruct ure t hat yo u can m anipulat e : So und data is enco ded in bytes
to represent the wave fo rms.
Re al-wo rld so und dat a is no t pe rf e ct : The so und data yo u generate has a near-perfect
representatio n as sinuso iudal wave fo rms. Reco rded so unds rarely have this structure, so the FFT
results are indicative o f existing frequencies, rather than clear and definitive.
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.
Conclusion
Lesson Objectives
When yo u finish this lesso n, yo u will be able to :
implement a data structure that co nfo rms to the Se t interface.
write co de to remo ve an element fro m an unbalanced binary search tree.
explain why kd-trees are unable to easily suppo rt element remo val.
explain ho w to rebalance a self-balancing binary tree after deleting values.
Concluding Lesson For Algorithms
In this final lesso n, we'll go o ver remo ving elements fro m a co llectio n. Thro ugho ut the co urse, yo u have learned ho w
to use a variety o f data structures to represent info rmatio n. In all cases, the presentatio n fo cused o n ho w to co nstruct
entire representatio ns fro m the beginning. Ho wever, it is also impo rtant to be able to describe ho w to remo ve an
element fro m a co llectio n that yo u have spent so much time co nstructing. We'll co nduct this to pic in these co ntexts:
Arrays
Binary Search Trees
AVL Binary Search Trees
kd-trees
This lesso n pro vides a capstone experience by pulling to gether mo st o f the impo rtant data structures yo u've seen and
explaining new functio nality that yo u might expect to see.
Removing Elements From a Sorted Array
Given a so rted array o f elements, yo u can use an Binary Array Search to lo cate an element in the array in
O(log n) time. To remo ve an element, ho wever, yo u have two cho ices:
Allo cate a new array to co ntain all elements fro m the o riginal set minus the o ne being remo ved.
Shift elements do wn within the existing array and maintain an additio nal value, num be r, that
reco rds the number o f elements in the array (no te that number < length).
Past lesso ns have demo nstrated bo th o f these o ptio ns, and neither leads to an efficient implementatio n.
Specifically, inserting an element into —o r remo ving an element fro m—an array-based structure requires O(n)
in the wo rst case because yo u have to co py n-1 elements within the same array. Neither o f these cho ices lets
yo u amo rtize the co sts acro ss multiple remo ve o r add requests.
Yo u might co nsider a third cho ice, using an ArrayList o bject to sto re the so rted elements, but then yo u
beco me respo nsible fo r inserting new values at their pro per lo catio ns. This can be less difficult to implement
than either o f the first two cho ices. When yo u use ArrayList , make sure that So rt e dSe t co nfo rms to the
Se t interface o f the Java Co llectio ns Framewo rk.
Create a new Java pro ject named Co nclusio n and assign it to the J ava6 _Le sso ns wo rking set.
Co py the packages and pro grams fro m yo ur BinaryT re e pro ject into yo ur Co nclusio n pro ject.
In yo ur Co nclusio n pro ject /src so urce fo lder, binary package, create a So rt e dSe t class as sho wn:
CODE TO TYPE: So rtedSet class
package binary;
import java.util.*;
public class SortedSet<E extends Comparable<E>> implements Set<E> {
ArrayList<E> list = new ArrayList<E>();
int binarySearch(E e) {
int low = 0;
int high = list.size()-1;
while (low <= high) {
int mid = (low + high)/2;
int rc = e.compareTo(list.get(mid));
if (rc < 0) {
high = mid - 1;
} else if (rc > 0) {
low = mid + 1;
} else {
return mid;
}
}
return -(low + 1);
}
public boolean add(E e) {
int idx = binarySearch(e);
if (idx >= 0) { return false; }
list.add(-(idx+1), e);
return true;
}
public boolean remove(Object o) {
int idx = binarySearch((E)o);
if (idx < 0) { return false; }
list.remove(idx);
return true;
}
}
The So rt e dSe t class uses an ArrayList o bject to sto re a set o f elements in so rted o rder; the set co ntains
no duplicates.
The co de wo n't co mpile just yet because there are still so me metho ds that yo u have to write to satisfy the
Se t interface. Let's take a clo ser lo o k at the initial functio nality:
OBSERVE: binarySearch o n a so rted ArrayList
int binarySearch(E e) {
int low = 0;
int high = list.size()-1;
while (low <= high) {
int mid = (low + high)/2;
int rc = e.compareTo(list.get(mid));
if (rc < 0) {
high = mid - 1;
} else if (rc > 0) {
low = mid + 1;
} else {
return mid;
}
}
return -(low + 1);
}
The binarySearch metho d assumes the underlying list ArrayList sto res its items in o rder. The co de is similar
to the binarySearch implemented in an earlier lesso n; the o nly difference is that it must access each element
in list using the ge t () metho d. binarySe arch returns a no n-negative value (that is, greater than o r equal to
zero ), when it finds the desired element e in the ArrayList . When binarySe arch returns a negative number x,
element e sho uld be inserted at -(x+1). Fo r example, when x=-1 is returned, element e is to be inserted at
po sitio n 0. Yo u can see this behavio r in the co de fo r add:
OBSERVE: Metho ds to add element to and remo ve element fro m so rted ArrayList
public boolean add(E e) {
int idx = binarySearch(e);
if (idx >= 0) { return false; }
list.add(-(idx+1), e);
return true;
}
public boolean remove(Object o) {
int idx = binarySearch((E)o);
if (idx < 0) { return false; }
list.remove(idx);
return true;
}
In the co ntract defined by the Java Co llectio ns Framewo rk, the add metho d in the Se t interface m ust re t urn
t rue whenever its co ntents have changed. The re m o ve metho d similarly re t urns t rue o nly when its
co ntents have changed. To co nfo rm to the Se t co ntract, the re m o ve metho d takes a generic Obje ct as its
parameter.
The Se t interface defines a co nt ains metho d that yo u can add to the end o f the So rt e dSe t class no w:
CODE TO TYPE: Add metho d to the end o f So rtedSet
...
public boolean contains(Object o) {
return (binarySearch((E)o) >= 0);
}
}
The Co llectio ns Framewo rk defines a number o f bulk operations to perfo rm o n a set. Add these metho ds to
the So rt e dSe t class.
CODE TO TYPE: Add bulk o peratio n metho ds to end o f class
...
public boolean addAll(Collection<? extends E> c) {
boolean changed = false;
for (E e : c) {
changed |= add(e);
}
return changed;
}
public boolean removeAll(Collection<?> c) {
boolean changed = false;
for (E e : (Collection<E>)c) {
changed |= remove(e);
}
return changed;
}
public boolean containsAll(Collection<?> c) {
for (E e : (Collection<E>)c) {
if (binarySearch(e) < 0) { return false; }
}
return true;
}
public boolean retainAll(Collection<?> c) {
boolean changed = false;
for (int idx = list.size() - 1; idx >= 0; idx--) {
if (!c.contains(list.get(idx))) {
list.remove(idx);
changed = true;
}
}
return changed;
}
}
The addAll(c) and re m o ve All(c) metho ds iterate o ver elements in the Co lle ct io n parameter c and add o r
remo ve that element fro m the ArrayList sto rage.
The co nt ainsAll(c) metho d iterates o ver every element, e, in c to determine if the So rt e dSe t co ntains e,
returning f alse immediately when a no n-member element, e, is detected. If all elements in c belo ng to the
So rt e dSe t , it returns t rue .
The re t ainAll(c) metho d demands a mo re co mplicated implementatio n. Specifically, this metho d remo ves
all elements in So rt e dSe t that do no t exist within c; it do es so by iterating thro ugh its elements in reverse
o rder, remo ving each element that do es no t exist in c. If the re t ainAll(c) metho d changes the set in any way,
it must return t rue , based o n the co ntract fo r the Se t interface.
To co mplete the implementatio n o f the necessary metho ds required by Se t , add the fo llo wing metho ds to the
end o f the So rt e dSe t class:
CODE TO TYPE: Co mplete So rtedSet implementatio n
...
public
public
public
public
public
public
}
int size() { return list.size(); }
Object[] toArray() { return list.toArray(); }
<T> T[] toArray(T[] a) { return list.toArray(a); }
void clear() { list.clear(); }
boolean isEmpty() { return list.isEmpty(); }
Iterator<E> iterator() { return list.iterator(); }
In each case, the required metho d delegates each request to the internal list ArrayList o bject.
Even tho ugh the So rt e dSe t class no w co mpiles, yo u still have to implement so me metho ds to co nfo rm to
the Java Co llectio ns Framewo rk. Specifically, fo r So rt e dSe t to truly satisfy the Se t interface, its hashCo de
metho d must be implemented to return the sum o f the hashCo de o f the values it co ntains. Add the fo llo wing
metho d to the end o f So rt e dSe t :
CODE TO TYPE: Add hashCo de metho d to So rtedSet
...
public int hashCode() {
int hash = 0;
for (int i = 0; i < list.size(); i++) {
hash += list.get(i).hashCode();
}
return hash;
}
}
The final change is to ensure that the e quals(o ) metho d returns t rue if and o nly if o is a set, the two sets
have the same size, and every member o f o is co ntained in this set. Add this metho d to the end o f the
So rt e dSe t class:
CODE TO TYPE: Add equals metho d to So rtedSet
...
public boolean equals(Object o) {
if (o == null) { return false; }
if (!(o instanceof Set)) { return false; }
Set<E> s = (Set<E>) o;
if (s.size() != list.size()) { return false; }
for (E e : s) {
if (binarySearch(e) < 0) { return false; }
}
return true;
}
}
Once the e quals metho d determines that it is co mparing against ano ther Se t o bject, s, it iterates thro ugh
each element, e in s, to determine whether the So rt e dSe t co ntains e, returning f alse at the first difference.
Once all elements are determined to be co ntained within the So rt e dSe t , it can safely return t rue .
Co ngratulatio ns! Yo u have co mpleted yo ur first Se t implementatio n! Write the St re ssT e st class to
demo nstrate its functio nality and co mpare its perfo rmance with the T re e Se t implementatio n.
In yo ur Co nclusio n pro ject /src so urce fo lder, binary package, create a St re ssT e st class as sho wn:
CODE TO TYPE: StressTest class
package binary;
import java.util.*;
public class StressTest {
final static double AddProb
final static double ContainsProb
final static int
SetSize
final static int
TrialSize
final static String[] Types
=
=
=
=
=
0.20;
0.70;
5000;
50000;
{"Add", "Contains", "Remove" };
static void fail(String err) {
System.err.println("Failed on:" + err);
System.exit(-1);
}
public static void main(String[] args) {
TreeSet<Integer> base = new TreeSet<Integer>();
SortedSet<Integer> set = new SortedSet<Integer>();
double baseCount[] = new double[3];
double setCount[] = new double[3];
int counts[] = new int[3];
long start;
boolean b,s;
for (int t = 0; t < TrialSize; t++) {
int n = (int)(Math.random()*SetSize);
double choice = Math.random();
if (choice < AddProb) {
start = System.nanoTime();
b = base.add(n);
baseCount[0] += (System.nanoTime() - start);
start = System.nanoTime();
s = set.add(n);
setCount[0] += (System.nanoTime() - start);
if (b != s) { fail(Types[0]); }
counts[0]++;
} else if (choice < ContainsProb) {
start = System.nanoTime();
b = base.contains(n);
baseCount[1] += (System.nanoTime() - start);
start = System.nanoTime();
s = set.contains(n);
setCount[1] += (System.nanoTime() - start);
if (b != s) { fail(Types[1]); }
counts[1]++;
} else {
start = System.nanoTime();
b = base.remove(n);
baseCount[2] += (System.nanoTime() - start);
start = System.nanoTime();
s = set.remove(n);
setCount[2] += (System.nanoTime() - start);
if (b != s) { fail(Types[2]); }
counts[2]++;
}
}
for (int i = 0; i < counts.length; i++) {
if (counts[i] != 0) {
baseCount[i] /= counts[i];
setCount[i] /= counts[i];
}
System.out.println(Types[i] + " base=" + (int)baseCount[i]+ " set=" + (int
)setCount[i]);
}
}
}
Run it to co mpare the behavio r o f So rt e dT e st against T re e Se t when 50 0 ,0 0 0 rando m o peratio ns are
perfo rmed o n each co llectio n where 20 % o f the time a rando m integer (up to 50 0 0 ) is added, 50 % o f the time
a co ntains query is executed, and 30 % o f the time a rando m integer (up to 50 0 0 ) is requested to be deleted.
Fine-grained timing statistics are reco rded fo r each o peratio n o n the two data structures. The sample o utput
belo w sho ws that, o n average, So rt e dSe t is always slo wer than T re e Se t (almo st three times slo wer fo r
add and remove). Yo ur perfo rmance results will likely vary fro m tho se sho wn:
OBSERVE: Output fro m StressTest executio n
Add base=255 set=856
Contains base=215 set=283
Remove base=245 set=659
Ultimately the T re e Se t class will o utperfo rm So rt e dSe t because o f the extra co st o f gro wing the array that
co ntains the elements o f the So rt e dSe t ; still this was a wo rthwhile exercise.
No w let's lo o k at ho w to handle the remo val o f elements fro m highly structured data structures.
Removing Elements From Binary Search T rees
Binary Search Trees (BSTs) o ffer the first data structure fo r which remo ving an element sho uld be an O(log n)
o peratio n; after all, it takes O(log n) perfo rmance to determine whether the BST co ntains the element in the first
place. Yo u need to identify a deterministic way to reconstruct the BST after remo ving an element. If yo u are
remo ving a leaf no de, the BST already is pro perly fo rmed; ho wever, what if yo u remo ve an element that has
o ne o r mo re children? Co nsider this small BST with seven elements fro m which yo u wo uld like to remo ve the
element 4 :
As yo u can see, this value is currently the ro o t no de o f the tree. What can be do ne? Yo u co uld always recreate
a new BST by starting fro m scratch and inserting all n-1 elements, but that wo uld be inefficient; in fact, that
single o peratio n wo uld require o n the o rder o f O(n log n) o peratio ns. Instead, co mpare the fo llo wing sixelement BST with the earlier seven-element BST.
Yo u can o bserve that the right sub-trees are the same in bo th BSTs and that the seco nd tree co ntinues to
suppo rt the Binary Search Tree Property. The seco nd tree was no t fo rmed by deleting the no de 4 . Rather, the
value asso ciated with this no de was replaced with the largest element in its left sub-tree, in this case, 3. There
is o ne small caveat; if the no de yo u want to delete has no left sub-tree, it can be replaced by the entire right
sub-tree o f the no de to be deleted, as sho wn in the image belo w, which describes the result o f remo ving the
value 6 fro m a sample BST:
Given a no de n with value that is to be remo ved fro m the tree, the largest element in its left sub-tree is exactly
the right-most descendant of the left child of n. As yo u can see, this value is smaller than all o f the values in the
right sub-tree o f the no de being remo ved (because the tree is a BST). This value is also larger than all o f the
o ther no des in the BST ro o ted by the left child o f the no de being remo ved. Once yo u find X, the right-mo st
descendant o f the left child o f the no de being remo ved, yo u can Well, yo u can swap its value with the no de
being remo ved. No w, what if the no de fo r X has any child no des? Then it canno t have any right children,
o therwise it wo uld no t be the right-mo st descendant o f the left child o f the no de being remo ved. Ho wever, X
might have a left child; indeed, it might have an entire sub-tree ro o ted by that left child. Fo rtunately, as with the
ro tatio ns described earlier in the AVL lesso n, yo u can "lift" up X's left sub-tree to replace X in the BST, and the
BST pro perties will o nce again ho ld. The image belo w sho ws the transfo rmatio n o f the BST when requested
to remo ve the value 5 0 fro m the BST. X=30 , is the largest element in the left sub-tree o f 5 0 . The inner no de
10 has o nly its right sub-tree changed to be the entire sub-tree 30 L, as sho wn:
Make these changes to the BinaryT re e class in the binary package:
CODE TO TYPE: Mo dificatio ns to BinaryTree
package binary;
public class BinaryTree<E extends Comparable<E>> {
BinaryNode<E> root = null;
public int size() {
if (root == null) { return 0; }
return root.size();
}
public int height () {
if (root == null) { return 0; }
return height(root);
}
int height (BinaryNode<E> n) {
if (n == null) { return 0; }
return 1 + Math.max( height(n.left), height(n.right));
}
public void add (E k) {
if (root == null) {
root = new BinaryNode<E>(k);
return;
}
root = root.add(root, k);
}
public boolean contains (E k) {
return contains(root, k);
}
boolean contains (BinaryNode<E> parent, E k) {
if (parent == null) { return false; }
int rc = k.compareTo(parent.key);
if (rc == 0) {
return true;
} else if (rc < 0) {
return contains(parent.left, k);
} else {
return contains(parent.right, k);
}
}
public void remove (E k) {
if (root == null) { return; }
root = remove(root, k);
}
BinaryNode<E> remove (BinaryNode<E> parent, E k) {
if (parent == null) { return null; }
int rc = k.compareTo(parent.key);
if (rc == 0) {
return parent.updateNodes();
} else if (rc < 0) {
parent.left = remove(parent.left, k);
} else {
parent.right = remove(parent.right, k);
}
return parent;
}
}
The structure o f the new re m o ve metho d is similar to the co nt ains metho d—to delete an element, yo u must
first determine whether it exists in the BST. Once k.co m pare T o (pare nt .ke y) returns 0 , yo u have fo und the
no de that co ntains the value to be remo ved. At this po int, yo u need to write an additio nal metho d in
BinaryNo de that pro perly updates the no de.
Mo dify BinaryNo de as sho wn:
CODE TO TYPE: Mo dificatio ns to BinaryNo de
package binary;
public class BinaryNode<E extends Comparable<E>> {
final E key;
BinaryNode<E> left;
BinaryNode<E> right;
public BinaryNode(E k) {
this.key = k;
}
public int size() {
return 1 + size(left) + size(right);
}
int size(BinaryNode<E> n) {
if (n == null) { return 0; }
return n.size();
}
void add (E k) {
int rc = k.compareTo(key);
if (rc <= 0) {
left = add(left, k);
} else {
right = add(right, k);
}
}
BinaryNode<E> add(BinaryNode<E> parent, E k) {
if (parent == null) {
return new BinaryNode<E>(k);
}
parent.add(k);
return parent;
}
public BinaryNode<E> updateNodes() {
if (left == null && right == null) { return null; }
if (left == null) { return right; }
if (right == null) { return left; }
BinaryNode<E> child = left;
BinaryNode<E> grandChild = child.right;
if (grandChild == null) {
left = child.left;
key = child.key;
} else {
while (grandChild.right != null) {
child = grandChild;
grandChild = grandChild.right;
}
key = grandChild.key;
child.right = grandChild.left;
}
return this;
}
}
The updat e No de s metho d is called o n a no de that has been targeted fo r deletio n. The value returned must
be the (po ssibly new) no de that will take the place o f this no de in the BST and which may have its o wn left and
right sub-trees.
The first three if statements in the updat e No de s metho d handle the fo llo wing cases (in this o rder):
The first three if statements in the updat e No de s metho d handle the fo llo wing cases (in this o rder):
1. The no de being deleted is a leaf no de, in which case it can be remo ved entirely.
2. The no de being deleted has o nly a right child, in which case that child is returned.
3. The no de being deleted has o nly a left child, in which case that child is returned.
If the no de to be deleted has bo th left and right children, the mo re co mplicated lo gic must be fo llo wed. The
go al is to find the right-mo st descendant o f the left child o f the no de being deleted (t his). To start, child is set
to the left child and grandChild is the right child (if it exists) o f child. If child has no right child (that is,
grandChild is null), then child itself is the right-mo st descendant o f t his. The image belo w describes this
case:
The co de "lifts" the left sub-tree o f child to beco me the left sub-tree o f t his and the key value asso ciated with
t his is set to the child's key value. Ho wever, if child has a right child, the co de seeks to find the right-mo st
descendant by traversing the right links co ntinually until grandChild.right is null (in o ther wo rds,
grandChild is kno wn to be the right-mo st child). The image belo w describes this case. Here the no de
identified as z is the right-mo st descendant o f child. It has no right child o f its o wn.
In the resulting mo dified BST, the key o f the no de with the value that was remo ved has been changed to z and
this will maintain the BST pro perty o f the o verall tree; in additio n, the left sub-tree o f z (if it exists) has been
"lifted up" to be the right child o f y, which was z's fo rmer parent.
Note
Yo u co uld also have selected the left-mo st descendant o f the right child o f the no de being
remo ved and the lo gical results wo uld have been the same.
With this mo dificatio n, yo u have fully implemented the Binary Search Tree. Mo dify the St re ssT e st co de yo u
have just written to co mpare BinaryT re e against T re e Se t . BinaryT re e do es no t enfo rce set semantics, so
St re ssT e st makes sure to co nvert add requests fo r elements already in the set into contains requests:
CODE TO TYPE: StressTest class
package binary;
import java.util.*;
public class StressTest {
final static double AddProb
final static double ContainsProb
final static int
SetSize
final static int
TrialSize
final static String[] Types
=
=
=
=
=
0.20;
0.70;
5000;
50000;
{"Add", "Contains", "Remove" };
static void fail(String err) {
System.err.println("Failed on:" + err);
System.exit(-1);
}
public static void main(String[] args) {
TreeSet<Integer> base = new TreeSet<Integer>();
SortedSetBinaryTree<Integer> set = new SortedSetBinaryTree<Integer>();
double baseCount[] = new double[3];
double setCount[] = new double[3];
int counts[] = new int[3];
long start;
boolean b,s;
for (int t = 0; t < TrialSize; t++) {
int n = (int)(Math.random()*SetSize);
double choice = Math.random();
if (choice < AddProb && !base.contains(n)) {
start = System.nanoTime();
b = base.add(n);
baseCount[0] += (System.nanoTime() - start);
start = System.nanoTime();
s = set.add(n);
setCount[0] += (System.nanoTime() - start);
if (b != s)if (base.contains(n) != set.contains(n)) { fail(Types[0]); }
counts[0]++;
} else if (choice < ContainsProb) {
start = System.nanoTime();
b = base.contains(n);
baseCount[1] += (System.nanoTime() - start);
start = System.nanoTime();
s = set.contains(n);
setCount[1] += (System.nanoTime() - start);
if (b != s) { fail(Types[1]); }
counts[1]++;
} else {
start = System.nanoTime();
b = base.remove(n);
baseCount[2] += (System.nanoTime() - start);
start = System.nanoTime();
s = set.remove(n);
setCount[2] += (System.nanoTime() - start);
if (b != s)if (base.contains(n) != set.contains(n)) { fail(Types[2]); }
counts[2]++;
}
}
for (int i = 0; i < counts.length; i++) {
if (counts[i] != 0) {
baseCount[i] /= counts[i];
setCount[i] /= counts[i];
}
System.out.println(Types[i] + " base=" + (int)baseCount[i]+ " set=" + (int
)setCount[i]);
}
}
}
Save and run St re ssT e st again; the results are much mo re favo rable, altho ugh T re e Se t still
o utperfo rms all o peratio ns. Yo ur results will likely vary:
OBSERVE: Output o f revised StressTest
Add base=252 set=330
Contains base=202 set=258
Remove base=247 set=302
Successive additio ns and remo vals will o ften result in an unbalanced BST. Since AVL trees are a selfbalancing structure, yo u must no w add the remo val functio nality to AVL trees so they can rebalance
themselves after the remo val o f an element.
Removing Elements From AVL T rees
We can reuse the same lo gic fo r deleting a value fro m an AVL tree to replace its value with the value o f the
right-mo st descendant o f the left child o f the no de being remo ved. Once this actio n is do ne, yo u may have to
rebalance a number o f o ther no des, alo ng the path between the parent o f the right-mo st descendant to the
ro o t.
Add this metho d to the end o f the AVLBinaryT re e class in the avl package:
CODE TO TYPE: Mo dificatio ns to AVLBinaryTree
public void remove (E k) {
if (root == null) { return; }
root = root.remove(root, k);
}
All o f the real wo rk takes place in the AVLBinaryNo de class. Yo u'll need to add these metho ds to the end o f
the class:
CODE TO TYPE: Mo dificatio ns to AVLBinaryNo de
AVLBinaryNode<E> remove(AVLBinaryNode<E> parent, E k) {
if (parent == null) { return null; }
return parent.remove(k);
}
AVLBinaryNode<E> remove (E k) {
int rc = k.compareTo(key);
AVLBinaryNode<E> newRoot = this;
if (rc == 0) {
if (left == null) {
return right;
}
AVLBinaryNode<E> child = left;
while (child.right != null) {
child = child.right;
}
E childKey = child.key;
left = remove(left, childKey);
key = childKey;
if (heightDifference(this) == -2) {
if (heightDifference(right) <= 0) {
newRoot = this.rotateLeft();
} else {
newRoot = this.rightLeftRotation();
}
}
} else if (rc < 0) {
left = remove(left, k);
if (heightDifference(this) == -2) {
if (heightDifference(right) <= 0) {
newRoot = this.rotateLeft();
} else {
newRoot = this.rightLeftRotation();
}
}
} else {
right = remove(right, k);
if (heightDifference(this) == 2) {
if (heightDifference(left) >= 0) {
newRoot = this.rotateRight();
} else {
newRoot = this.leftRightRotation();
}
}
}
computeHeight(newRoot);
return newRoot;
}
}
This is a lo t to take in! Let's start with the first helper metho d, re m o ve (pare nt ,k), which remo ves the value k
fro m the sub-tree ro o ted at pare nt . The real wo rk o ccurs within the re m o ve (k) metho d. This metho d has
nearly the same structure as the add(k) metho d that already exists (and is repeated belo w):
OBSERVE: Existing add metho d in AVLBinaryNo de
AVLBinaryNode<E> add (E k) {
int rc = k.compareTo(key);
AVLBinaryNode<E> newRoot = this;
if (rc <= 0) {
left = add(left, k);
if (heightDifference(this) == 2) {
if (k.compareTo(left.key) <= 0) {
newRoot = rotateRight();
} else {
newRoot = leftRightRotation();
}
}
} else {
right = add(right, k);
if (heightDifference(this) == -2) {
if (k.compareTo(right.key) > 0) {
newRoot = rotateLeft();
} else {
newRoot = rightLeftRotation();
}
}
}
computeHeight(newRoot);
return newRoot;
}
The key po int to o bserve is that whe ne ve r t he he ight Dif f e re nce o f a no de e xce e ds t he allo we d
t hre sho lds, a ro tatio n o ccurs. The re m o ve metho d will have three cases to handle; the two cases sho wn
belo w explain ho w ro tatio ns take place whenever the remo val o f an element fro m a no de's left sub-tree (o r
right sub-tree) causes that no de to beco me unbalanced).
OBSERVE: remo ve metho d structure
AVLBinaryNode<E> remove (E k) {
int rc = k.compareTo(key);
AVLBinaryNode<E> newRoot = this;
if (rc == 0) {
// perform the deletion
} else if (rc < 0) {
left = remove(left, k);
if (heightDifference(this) == -2) {
if (heightDifference(right) <= 0) {
newRoot = this.rotateLeft();
} else {
newRoot = this.rightLeftRotation();
}
}
} else {
right = remove(right, k);
if (heightDifference(this) == 2) {
if (heightDifference(left) >= 0) {
newRoot = this.rotateRight();
} else {
newRoot = this.leftRightRotation();
}
}
}
computeHeight(newRoot);
return newRoot;
}
Let's go o ver ho w to perfo rm the deletio n:
OBSERVE: Perfo rm deletio n o f no de in AVL tree
if (rc == 0) {
if (left == null) {
return right;
}
AVLBinaryNode<E> child = left;
while (child.right != null) {
child = child.right;
}
E childKey = child.key;
left = remove(left, child.key);
key = childKey;
if (heightDifference(this) == -2) {
if (heightDifference(right) <= 0) {
newRoot = this.rotateLeft();
} else {
newRoot = this.rightLeftRotation();
}
}
}
This co de immediately checks whe t he r t he re is e ve n a le f t child f o r t he no de be ing de le t e d; if no t,
the right sub-tree is "lifted" to take its place.
If the left child is present, the co de lo cat e s t he right -m o st de sce ndant quickly, child. The metho d then
uses do uble recursio n to invo ke re m o ve (le f t , child.ke y) to remo ve the child.ke y value fro m the sub-tree
ro o ted at le f t and replace the key fo r the no de being deleted with child.ke y. Once this task is co mplete, yo u
kno w that the sub-tree ro o ted at le f t is balanced pro perly. Then o ur co de che cks t o se e if any ro t at io n is
ne e de d; because yo u o nly lo o ked fo r the right-mo st descendant o n the left side o f the tree, yo u o nly need to
co nsider two ro tatio n cases, which are co mplementary to the cases in the add metho d (except that yo u are
remo ving elements, no t adding them).
As with the add metho d, the rebalancing may o ccur at any time between the o riginal lo catio n o f the value
being deleted and the path fro m that no de to the ro o t in the tree. So , there may be a to tal o f O(log n) ro tatio ns
whenever yo u remo ve an element fro m an AVL tree. Fo r this reaso n, the Red-Black Tree implementatio n o f
T re e Se t in the Java Co llectio ns Framewo rk is mo re efficient than an AVL implementatio n when inserting
elements into the tree. At the same time, the AVL tree is mo re co mpact than the Red-Black Tree
implementatio n, which means the contains queries are go ing to be faster. Review the abo ve co de to make
sure yo u understand ho w each o f the co nstituent elements wo rks to self-balance the tree auto matically.
In yo ur Co nclusio n pro ject /src so urce fo lder, avl package, create an AVLSt re ssT e st class as sho wn.
This class is co mplicated because it co ntains co de to validate that the AVL Property o f the AVLBinaryT re e is
no t vio lated after any additio n o r deletio n:
CODE TO TYPE: AVLStressTest class
package avl;
import java.util.*;
public class AVLStressTest {
final static double AddProb
final static double ContainsProb
final static int
SetSize
final static int
TrialSize
final static String[] Types
=
=
=
=
=
0.20;
0.70;
5000;
50000;
{"Add", "Contains", "Remove" };
static void fail(String err) {
System.err.println("Failed on:" + err);
System.exit(-1);
}
static int height (AVLBinaryNode<?> n) {
if (n == null) { return 0; }
return 1 + Math.max( height(n.left), height(n.right));
}
public static int height (AVLBinaryTree<?> tree) {
if (tree.root == null) { return -1; }
return height(tree.root);
}
static boolean validateAVLProperty (AVLBinaryNode<?> n) {
if (n == null) { return true; }
int leftHeight = 0;
if (n.left != null) { leftHeight = height(n.left); }
int rightHeight = 0;
if (n.right != null) { rightHeight = height(n.right); }
int diff = leftHeight - rightHeight;
if (diff < -1 || diff > 1) { return false; }
return validateAVLProperty (n.left) && validateAVLProperty (n.right);
}
static boolean validateAVLProperty(AVLBinaryTree<?> tree) {
if (tree.root == null) { return true; }
return validateAVLProperty(tree.root);
}
public static void main(String[] args) {
TreeSet<Integer> base = new TreeSet<Integer>();
AVLBinaryTree<Integer> set = new AVLBinaryTree<Integer>();
double baseCount[] = new double[3];
double setCount[] = new double[3];
int counts[] = new int[3];
long start;
boolean b,s;
for (int t = 0; t < TrialSize; t++) {
int n = (int)(Math.random()*SetSize);
double choice = Math.random();
if (choice < AddProb && !base.contains(n)) {
start = System.nanoTime();
b = base.add(n);
baseCount[0] += (System.nanoTime() - start);
start = System.nanoTime();
set.add(n);
setCount[0] += (System.nanoTime() - start);
if (!validateAVLProperty(set)) { fail(Types[0]); }
if (base.contains(n) != set.contains(n)) { fail(Types[0]); }
counts[0]++;
} else if (choice < ContainsProb) {
start = System.nanoTime();
b = base.contains(n);
baseCount[1] += (System.nanoTime() - start);
start = System.nanoTime();
s = set.contains(n);
setCount[1] += (System.nanoTime() - start);
if (b != s) { fail(Types[1]); }
counts[1]++;
} else {
start = System.nanoTime();
b = base.remove(n);
baseCount[2] += (System.nanoTime() - start);
start = System.nanoTime();
set.remove(n);
setCount[2] += (System.nanoTime() - start);
if (!validateAVLProperty(set)) { fail(Types[2]); }
if (base.contains(n) != set.contains(n)) { fail(Types[2]);
counts[2]++;
}
}
}
for (int i = 0; i < counts.length; i++) {
if (counts[i] != 0) {
baseCount[i] /= counts[i];
setCount[i] /= counts[i];
}
System.out.println(Types[i] + " base=" + (int)baseCount[i]+ " set=" + (int
)setCount[i]);
}
}
}
Save and run it. It will take lo nger to co mplete because o f the validatio n co de. The validat e AVLPro pe rt y
metho d validates that the height difference fo r every no de in the AVL binary tree is within the required
to lerance as demanded by the AVL Property.
OBSERVE: AVLStressTest o utput
Add base=305 set=464
Contains base=270 set=227
Remove base=360 set=411
When reviewing this result, o bserve that the baseline T re e Se t implementatio n still o utperfo rms AVL binary
trees when it co mes to adding and removing elements. Ho wever, the contains query is no w almo st 20 % faster
in the AVL binary tree, because o f its mo re co mpact structure. It's always satisfying when empirical evidence
suppo rts the expected results.
Removing Elements From KD-trees
Given the success we've had deleting no des fro m binary trees, yo u'd expect to be able to do the same with
the kd-tree. Unfo rtunately, this will be impo ssible because the kd-tree alternates its ho rizo ntal and vertical
partitio ning levels within its structure. That is, while the structure o f a kd-tree resembles a Binary Search Tree,
yo u canno t simply "lift" no des o ne level up as yo u have been able to do fo r the BST and AVL trees described
earlier in this lesso n.
Since yo u can't remo ve elements fro m a kd-tree easily, what can yo u do ? Instead o f rebuilding the kd-tree
with each deletio n, co nsider a strategy that marks elements as deleted (which takes O(log n) time) and then
the kd-tree can reco nsitute itself auto matically whenever the ratio o f deleted no des to present no des in the
tree exceeds so me thresho ld.
There is a co mparative precedent fo r this behavio r in hashtables, such as HashMap, to auto matically resize
themselves when the number o f entries exceeds so me inner thresho ld based o n the load capacity o f the
sto rage.
Co py the kd package fro m yo ur Mult idim e nsio n pro ject into yo ur Co nclusio n pro ject /src fo lder. In the kd
package, mo dify the KDNo de class as sho wn:
CODE TO TYPE: Mo dificatio ns to KDNo de
package kd;
import java.awt.Point;
public class KDNode {
final Point point;
final int direction;
Region region;
KDNode above;
KDNode below;
boolean deleted;
public static final int HORIZONTAL = 0;
public static final int VERTICAL = 1;
public KDNode(Point p, int dir, Region r) {
this.point = new Point (p);
this.direction = dir;
this.region = new Region(r);
}
public KDNode(Point p, int dir) {
this (p, dir, Region.max);
}
public boolean isBelow(Point p) {
if (direction == VERTICAL) {
return p.x < point.x;
} else {
return p.y < point.y;
}
}
public boolean isAbove(Point p) {
if (direction == VERTICAL) {
return p.x >= point.x;
} else {
return p.y >= point.y;
}
}
public boolean isDeleted() { return deleted; }
public voidboolean add (Point p) {
if (p.equals(point)) {
if (deleted) {
deleted = false;
return true;
}
return false;
}
if (isBelow(p)) {
if (below == null) {
below = createChild (p, true);
return true;
} else {
return below.add(p);
}
} else {
if (above == null) {
above = createChild (p, false);
return true;
} else {
return above.add(p);
}
}
}
KDNode createChild (Point p, boolean below) {
Region r = new Region (region);
if (direction == VERTICAL) {
if (below) {
r.x_max = point.x;
} else {
r.x_min = point.x;
}
} else {
if (below) {
r.y_max = point.y;
} else {
r.y_min = point.y;
}
}
return new KDNode(p, 1-direction, r);
}
}
The first mo dificatio n is to asso ciate a de le t e d attribute with each KDNo de o bject. If this value is t rue fo r a
no de, its asso ciated po int no lo nger belo ngs in the set, but it remains within the kd-tree to pro vide the
necessary structure. An isDe le t e d metho d is pro vided to determine the status o f a KDNo de o bject.
In the initial implementatio n, the add metho d had no thing to return. No w yo u need to kno w whether the set
changed as a result o f the invo catio n. This is the same behavio r designed by the Java Co llectio ns
Framewo rk. The co de canno t return f alse if the po int being added already exists within the kd-tree because,
after all, it may have previo usly been deleted, in which case the co de flips the de le t e d attribute value to f alse
befo re returning t rue to signal that the set has changed.
We'll make mo re significant changes in the KDT re e class no w:
CODE TO TYPE: Mo dificatio ns to KDTree
package kd;
import java.awt.Point;
public class KDTree {
KDNode root;
int deletedCount;
int totalCount;
float loadFactor;
static float DEFAULT_LOAD_FACTOR = 0.5f;
public KDTree() {
this (DEFAULT_LOAD_FACTOR);
}
public KDTree(float factor) {
root = null;
loadFactor = factor;
}
public booleanvoid add (Point value) {
if (root == null) {
root = new KDNode(value, KDNode.VERTICAL);
totalCount = 1;
return true;
} else {
if (root.add(value);) {
totalCount++;
return true;
}
return false;
}
}
void recreate() {
KDNode oldRoot = root;
root = null;
int remaining = totalCount - deletedCount;
totalCount = deletedCount = 0;
if (remaining == 0) {
return;
}
fill(oldRoot);
}
void fill(KDNode n) {
if (n == null) { return; }
if (!n.deleted) {
add(n.point);
}
fill(n.below);
fill(n.above);
}
public boolean remove (Point p) {
KDNode exist = find(p);
if (exist != null && !exist.deleted) {
exist.deleted = true;
deletedCount++;
if (deletedCount*1.0/totalCount >= loadFactor) {
recreate();
}
return true;
}
return false;
}
public KDNode find(Point p) {
return find(root, p);
}
KDNode find (KDNode node, Point p) {
if (node == null) { return null; }
if (node.point.distance(p) < 5) { return node; }
if (node.isBelow(p)) {
return find(node.below, p);
} else {
return find(node.above, p);
}
}
}
Let's lo o k at the co de mo re clo sely:
OBSERVE: KDTree Mo dified State
int deletedCount;
int totalCount;
float loadFactor;
static float DEFAULT_LOAD_FACTOR = 0.5f;
public KDTree() {
this (DEFAULT_LOAD_FACTOR);
}
public KDTree(float factor) {
root = null;
loadFactor = factor;
}
Each KDT re e o bject maintains a t o t alCo unt o f values in the kd-tree, as well as the de le t e dCo unt o f
po ints that have been remo ved. Whenever the ratio o f deleted po ints to to tal po ints is greater than o r equal to
lo adFact o r, the kd-tree is recreated to co ntain o nly the no n-deleted po ints. The user can specify a lo ad
f act o r o n co nstructio n; if they do n't specify, t he de f ault is 0 .5 .
OBSERVE: Mo dified add metho d
public boolean add (Point value) {
if (root == null) {
root = new KDNode(value, KDNode.VERTICAL);
totalCount = 1;
return true;
} else {
if (root.add(value)) {
totalCount++;
return true;
}
return false;
}
}
The add metho d is changed to update the t o t alCo unt o f values in the kd-tree; no w it also re t urns t rue to
reflect a change to its underlying set.
The new functio nality is co ntained in the re m o ve metho d:
OBSERVE: Remo ve metho d added to KDTree
public boolean remove (Point p) {
KDNode exist = find(p);
if (exist != null && !exist.deleted) {
exist.deleted = true;
deletedCount++;
if (deletedCount*1.0/totalCount >= loadFactor) {
recreate();
}
return true;
}
return false;
}
The re m o ve metho d re t urns t rue when the set has changed. Acco rdingly, it must first check to see if t he
po int e ve n e xist s wit hin t he t re e ; if it do es, it must make sure that t he asso ciat e d no de has no t
alre ady be e n de le t e d. Assuming it has no t, t he no de is m arke d as be ing de le t e d and t he
co rre spo nding de le t e dCo unt is incre m e nt e d fo r the kd-tree. At this po int it is po ssible that t he rat io
o f de le t e d no de s t o t o t al no de s e xce e ds t he loadFactor t hre sho ld, at which po int the entire kd-tree
is reco nstructed to co ntain o nly the no n-deleted no des. This is acco mplished in the re cre at e and f ill
metho ds:
OBSERVE: Co de to reco nstruct kd-tree fro m no n-deleted no des
void recreate() {
KDNode oldRoot = root;
root = null;
int remaining = totalCount - deletedCount;
totalCount = deletedCount = 0;
if (remaining == 0) {
return;
}
fill(oldRoot);
}
void fill(KDNode n) {
if (n == null) { return; }
if (!n.deleted) {
add(n.point);
}
fill(n.below);
fill(n.above);
}
The re cre at e metho d checks to see if all po int s have be e n de le t e d. If so , it can re t urn after se t t ing
ro o t t o null. If there are any po ints left tho ugh, all o f t he no n-de le t e d po int s in t he o ldRo o t are
pro ce sse d by f ill.
The f ill metho d perfo rms a pre-o rder traversal o f the kd-tree; f o r all no n-de le t e d no de s, the asso ciat e d
n.point is inse rt e d int o t he ne w kd-t re e . The recursive call makes sure to traverse bo th the be lo w and
abo ve sub-trees fo r each no de.
To demo nstrate the new capability in actio n, write the fo llo wing applicatio n, which allo ws the user to add
po ints to the kd-tree by clicking with the left butto n and to remo ve po ints by clicking with the right butto n.
The KDT re e class makes an impo rtant design decisio n to enable the f ind metho d to return the asso ciated
KDNo de fo r the requested po int being searched. This is impo rtant because it is up to the caller to determine
whether the no de represents a deleted po int in the kd-tree o r a valid po int. In do ing so , the co de is able to
draw deleted no des with an "X" while no n-deleted no des are filled in squares.
In the /src so urce fo lder kd package, create a KDApple t De le t e class as sho wn:
CODE TO TYPE: KDAppletDelete class
package kd;
import java.awt.*;
import java.awt.event.*;
public class KDAppletDelete extends java.applet.Applet {
KDTree tree = new KDTree();
KDNode match = null;
Image bufferImage;
Graphics bufferGraphics;
int toAWT(int y) {
if (y == Region.maxValue) { return 0; }
int awty = getHeight();
if (y != Region.minValue) { awty -= y; }
return awty;
}
int toCartesian(int awty) { return getHeight() - awty; }
public void init() {
setSize(400,400);
addMouseListener (new MouseAdapter() {
public void mouseClicked(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
if (me.getButton() == MouseEvent.BUTTON3) {
KDNode match = tree.find(pt);
if (match != null) {
tree.remove(match.point);
redraw();
drawNode(bufferGraphics, match.point, true, true);
repaint();
}
} else {
tree.add(pt);
redraw();
repaint();
}
}
});
addMouseMotionListener (new MouseAdapter() {
public void mouseMoved(MouseEvent me) {
Point pt = new Point (me.getX(), toCartesian(me.getY()));
KDNode newMatch = tree.find(pt);
if (match != newMatch) {
match = newMatch;
redraw();
if (match != null) {
drawNode(bufferGraphics, match.point, true, match.deleted);
}
repaint();
}
}
});
}
void drawNode(Graphics g, Point p, boolean selected, boolean deleted) {
if (selected) {
g.setColor(Color.RED);
g.clearRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
if (deleted) {
g.drawRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
g.drawLine(p.x - 4, toAWT(p.y) - 4, p.x + 4, toAWT(p.y) + 4);
g.drawLine(p.x - 4, toAWT(p.y) + 4, p.x + 4, toAWT(p.y) - 4);
} else {
g.fillRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
g.setColor(Color.BLACK);
}
public void paint(Graphics g) {
if (bufferImage == null) {
bufferImage = createImage(getWidth(), getHeight());
bufferGraphics = bufferImage.getGraphics();
}
if (tree.root == null) {
g.drawString("Click to add points", 150, 200);
} else {
g.drawImage(bufferImage, 0, 0, this);
}
}
void redraw() {
bufferGraphics.clearRect(0, 0, getWidth(), getHeight());
visit(bufferGraphics, tree.root);
}
void drawPartition (Graphics g, Region r, Point p, int type, boolean deleted)
{
if (type == KDNode.VERTICAL) {
g.drawLine(p.x, toAWT(r.y_min), p.x, toAWT(r.y_max));
} else {
int xlow = r.x_min;
if (r.x_min == Region.minValue) { xlow = 0; }
int xhigh = r.x_max;
if (r.x_max == Region.maxValue) { xhigh = getWidth(); }
g.drawLine(xlow, toAWT(p.y), xhigh, toAWT(p.y));
}
drawNode(g, p, false, deleted);
}
void visit (Graphics g, KDNode n) {
if (n == null) { return; }
drawPartition(g, n.region, n.point, n.direction, n.deleted);
visit (g, n.below);
visit (g, n.above);
}
}
Much o f this co de is similar to the applets yo u wro te fo r an earlier lesso n, but there are a so me differences.
Let's lo o k at the co de mo re clo sely:
OBSERVE: draw Metho d
void drawNode(Graphics g, Point p, boolean selected, boolean deleted) {
if (selected) {
g.setColor(Color.RED);
g.clearRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
if (deleted) {
g.drawRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
g.drawLine(p.x - 4, toAWT(p.y) - 4, p.x + 4, toAWT(p.y) + 4);
g.drawLine(p.x - 4, toAWT(p.y) + 4, p.x + 4, toAWT(p.y) - 4);
} else {
g.fillRect(p.x - 4, toAWT(p.y) - 4, 8, 8);
}
g.setColor(Color.BLACK);
}
The draw metho d draws a no de accurately, whether it is marked fo r deletio n o r selected by the user. The
m o use Clicke d, m o use Mo ve d, and drawPart it io n metho ds all invo ke draw as needed. The mo use
handlers o perate as befo re, but no w right mo use clicks (as designated by Mo use Eve nt .BUT T ON3) are
used to remo ve po ints fro m the kd-tree.
Run KDApple t De le t e and add ten po ints using the left mo use butto n. Then select five different po ints fo r
deletio n. As yo u select the first fo ur po ints, the applet redraws tho se po ints using a small "x" as sho wn:
Once yo u select the fifth po int fo r deletio n, the kd-tree will reassemble itself auto matically with o nly five po ints
because the ratio n o f deleted po ints to actual po ints has hit the predetermined ratio o f 50 %. The image belo w
sho ws the resulting kd-tree o nce reco nstructed:
The resulting reco nstructed kd-tree is no t likely balanced, but there is an algo rithm, described in the Algorithms
in a Nutshell bo o k, which enables yo u to create a balanced kd-tree fro m any selectio n o f po ints.
Lessons Learned
Co m plicat e d dat a st ruct ure s have invariant s t hat m ust be m aint aine d unde r addit io n
and re m o val.
Me t ho ds t hat re t urn vo id m iss an o ppo rt unit y t o re t urn use f ul inf o rm at io n: Co nsider
the add metho d in the Co llectio ns Framewo rk and ho w it returns t rue when the co llectio n changes
but f alse o therwise. This bit o f info rmatio n is extremely helpful in several algo rithms.
Divide and Co nque r is an e xt re m e ly po we rf ul st rat e gy: The algo rithms that deliver O(n log
n) perfo rmance o ften do so by using this technique to divide a pro blem into two (o r mo re) smaller
subpro blems, who se results are co mbined to pro duce the appro priate answer. Yo u have seen this
in Me rge So rt .
Copyright © 1998-2014 O'Reilly Media, Inc.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
See http://creativecommons.org/licenses/by-sa/3.0/legalcode for more information.