implements vs extends
When would you use one over the other? Here are some notes about each (non-exhaustive
list):
Interfaces (implements)
● Defines rules (methods) to be followed, doesn’t say how
● All variables and methods are public by default (Any variables are also static and final)
● Can define default methods
● One class can implement multiple interfaces
Classes (extends)
● Can define variables, methods, and constructors
● State-holding variables and methods are inherited when the class is being extended
● Can leave methods abstract
● One class can extend only one other class
When should we use them?
Then what’s the difference between classes (that can be abstract) and interfaces
(that can use default methods)?
●
Classes can pass on state-holding variables. Interfaces only have static + final
variables, so the only variables one can work with in an interface belong to a
class.
●
One class can implement many interfaces, but it can only extend at most one
class. Choose wisely!
●
Class inheritance is useful when your classes are really similar, and end up
sharing a lot of implementation. Interface inheritance is useful when you have
totally different ways of supporting the same behavior.
Interface vs. Implementation Inheritance
Interface Inheritance (a.k.a. what):
●
Allows you to generalize code in a powerful, simple way.
Implementation Inheritance (a.k.a. how):
●
Allows code-reuse: Subclasses can rely on superclasses or interfaces.
○ Gives another dimension of control to subclass designers: Can
decide whether or not to override default implementations.
Important: In both cases, we specify “is-a” relationships, not “has-a”.
●
●
Good: Dog implements Animal, LinkedListDeque implements Deque,
RotatingLinkedListDeque extends LinkedListDeque.
Bad: Cat implements Claw, Set implements SLList.
Overriding versus
Overloading
Method Overriding
If a “subclass” has a method with the exact same signature as in the
“superclass”, we say the subclass overrides the method.
public interface Deque<T> {
public void addLast(T item);
...
public class ArrayDeque<T> implements Deque<T>{
...
public void addLast(T item) {
...
ArrayDeque overrides addLast(T)
Method Overriding
public class Animal {
public void makeNoise() {
System.out.print(“hi!”);
}
}
public class Pig extends Animal {
public void makeNoise() {
System.out.print(“oink”);
}
}
A Pig might always makeNoise
a little differently than your
average animal.
makeNoise is overriden.
Method Overriding
public class Animal {
public void makeNoise() {
System.out.print(“hi!”);
}
}
But maybe Dogs say “hi” to
everyone, but when greeted with
another Dog, they react differently.
Now, is Dog overriding makeNoise?
public class Pig extends Animal {
public void makeNoise() {
System.out.print(“oink”);
}
}
public class Dog
extends Animal
{
public void makeNoise(Dog friend) {
System.out.print(“woof!”);
}
}
Method Overriding vs. Overloading
Methods with the same name but different signatures are overloaded.
● This applies to both from a subclass to a superclass, as well within one class.
public class Dog extends Animal {
public void makeNoise(Dog x)
}
makeNoise is overloaded
{…}
public class Math {
public int
abs(int a) {...}
public double abs(double a) {...}
}
abs is overloaded
Note: Adding the @Override Annotation
In 61BL, we’ll always mark every overriding method with the @Override annotation.
● Example: Mark ArrayDeque.java’s overriding methods with @Override.
● Even if you don’t write @Override, subclass still overrides the method if it’s written
correctly. It’s just an optional reminder that you’re overriding.
Why use @Override?
● Main reason: Protects against typos.
○ If you say @Override, but the method isn’t actually overriding anything, compile
error.
○ e.g. public void addLats(T item)
● Reminds programmer that method definition came from somewhere higher up in the
inheritance hierarchy.
public class ArrayDeque<T> implements Deque<T>{
...
@Override
public void addLast(T item) {
...
ArrayDeque
Deque
LinkedListDeque
Static vs Dynamic Types
and Casting
Is-A
Recall: Inheritance lets us define hierarchies.
●
●
●
A dog “is-a” canine.
A canine “is-a” carnivore.
A carnivore “is-an” animal.
animal
omnivore
herbivore
carnivore
feline
canine
dog
Is-A
So could we declare a variable like below:
canine A = new Dog();
animal
Or could a method with the signature
public void feed(carnivore c) {...}
omnivore
herbivore
carnivore
be called on a Dog instance?
canine
dog
feline
The Magic Box Analogy
I like to imagine Java as a big factory. When you declare a variable, you’re
setting aside a magic memory box. The boxes only hold what you say it can
hold.
●
If I have a box that can hold Fruit, it can certainly hold an Apple, right? This
is the beauty of inheritance.
Once the Java compiler puts the value inside the box, it forgets exactly what’s
in there. It just reads the label on the box–which is whatever you declared it
as.
●
●
This “label” that the compiler knows is the static type of a variable.
The type of whatever is actually inside the box is the dynamic type.
Static Type vs. Dynamic Type
Every variable in Java has a “compile-time type”, a.k.a. “static type”.
● This is the type specified at declaration. Never changes!
● This is the type checked during compilation.
Variables also have a “run-time type”, a.k.a. “dynamic type”.
● This is the type specified at instantiation (e.g. when using new).
● Equal to the type of the object being pointed at.
● This is the type we start with during runtime.
public static void main(String[] args) {
LivingThing lt1;
lt1 = new Fox();
Animal a1 = (Animal) lt1;
Fox h1 = new Fox();
lt1 = new Squid();
}
Static Type
Dynamic Type
LivingThing
lt1
LivingThing
null
Static Type vs. Dynamic Type
Every variable in Java has a “compile-time type”, a.k.a. “static type”.
● This is the type specified at declaration. Never changes!
● This is the type checked during compilation.
Variables also have a “run-time type”, a.k.a. “dynamic type”.
● This is the type specified at instantiation (e.g. when using new).
● Equal to the type of the object being pointed at.
● This is the type we start with during runtime.
public static void main(String[] args) {
LivingThing lt1;
lt1 = new Fox();
Animal a1 = (Animal) lt1;
Fox h1 = new Fox();
lt1 = new Squid();
}
Static Type
Dynamic Type
LivingThing
lt1
LivingThing
Fox
Static Type vs. Dynamic Type
Every variable in Java has a “compile-time type”, a.k.a. “static type”.
● This is the type specified at declaration. Never changes!
● This is the type checked during compilation.
Variables also have a “run-time type”, a.k.a. “dynamic type”.
● This is the type specified at instantiation (e.g. when using new).
● Equal to the type of the object being pointed at.
● This is the type we start with during runtime.
public static void main(String[] args) {
LivingThing lt1;
lt1 = new Fox();
Animal a1 = (Animal) lt1;
Fox h1 = new Fox();
lt1 = new Squid();
}
Static Type
Dynamic Type
LivingThing
lt1
LivingThing
Fox
Animal
Fox
Animal
a1
Static Type vs. Dynamic Type
Every variable in Java has a “compile-time type”, a.k.a. “static type”.
● This is the type specified at declaration. Never changes!
● This is the type checked during compilation.
Variables also have a “run-time type”, a.k.a. “dynamic type”.
● This is the type specified at instantiation (e.g. when using new).
● Equal to the type of the object being pointed at.
● This is the type we start with during runtime.
public static void main(String[] args) {
LivingThing lt1;
lt1 = new Fox();
Animal a1 = (Animal) lt1;
Fox h1 = new Fox();
lt1 = new Squid();
}
Static Type
Dynamic Type
LivingThing
lt1
LivingThing
Fox
Animal
Fox
Fox
Fox
Animal
a1
Fox
h1
Static Type vs. Dynamic Type
Every variable in Java has a “compile-time type”, a.k.a. “static type”.
● This is the type specified at declaration. Never changes!
● This is the type checked during compilation.
Variables also have a “run-time type”, a.k.a. “dynamic type”.
● This is the type specified at instantiation (e.g. when using new).
● Equal to the type of the object being pointed at.
● This is the type we start with during runtime.
public static void main(String[] args) {
LivingThing lt1;
lt1 = new Fox();
Animal a1 = (Animal) lt1;
Fox h1 = new Fox();
lt1 = new Squid();
}
Static Type
Dynamic Type
LivingThing
lt1
LivingThing
Squid
Animal
Fox
Fox
Fox
Animal
a1
Fox
h1
An interesting question...
What if we tried doing something like Animal a = new Car();?
Compile time checking
The Java compiler is like the diligent factory inspector, making sure that we only
put items in boxes that they can fit in.
●
The compiler also catches syntax errors or undefined references
When we make an assignment, var = value, the compiler asks, “Can a memory
box of var’s type hold something of value’s type?”.
Similarly, if we were to call a method, var1.myMethod(var2), the compiler asks,
“Can an item of var1’s type call a method with the name myMethod that could take
in an item of var2’s type?”. If it can, then we’ll give that method our seal of approval
and write it into our records.
Dynamic Method Selection For Overridden Methods
public class Animal {
public void makeNoise() {
System.out.print(“hi!”);
}
}
public class Pig extends Animal {
public void makeNoise() {
System.out.print(“oink”);
}
}
Animal p = new Pig();
p.makeNoise()
What happens here? Let’s ask…
● What is the static type of p?
● What is the dynamic type of p?
● Which type does the compiler see?
● What is the signature that the
compiler gives the seal of approval
to?
● What method gets run at runtime?
Dynamic Method Selection For Overridden Methods
Suppose we call a method of an object using a variable with:
●
●
compile-time type X
run-time type Y
Then if Y overrides the method in X, Y’s method is used instead.
●
This is known as “dynamic method selection”. Well, at least, at Berkeley!
Dynamic Method Selection Puzzle
Suppose we have classes defined below. Try to predict the results.
public interface Animal {
default void greet(Animal a) {
public class Dog implements Animal {
print("hello animal"); }
void sniff(Animal a) {
default void sniff(Animal a) {
print("dog sniff animal"); }
print("sniff animal"); }
default void flatter(Animal a) { void flatter(Dog a) {
print("u r cool dog"); }
print("u r cool animal"); }
}
}
Animal a = new Dog();
Dog d = new Dog();
a.greet(d);
a.sniff(d);
d.flatter(d);
a.flatter(d);
Dynamic Method Selection Puzzle
Suppose we have classes defined below. Try to predict the results.
public interface Animal {
default void greet(Animal a) {
public class Dog implements Animal {
print("hello animal"); }
void sniff(Animal a) {
default void sniff(Animal a) {
print("dog sniff animal"); }
print("sniff animal"); }
default void flatter(Animal a) { void flatter(Dog a) {
print("u r cool dog"); }
print("u r cool animal"); }
}
}
Animal a = new Dog();
Dog d = new Dog();
a.greet(d);
// "hello animal"
a.sniff(d);
d.flatter(d);
a.flatter(d);
Dynamic Method Selection Puzzle
Suppose we have classes defined below. Try to predict the results.
public interface Animal {
default void greet(Animal a) {
public class Dog implements Animal {
print("hello animal"); }
void sniff(Animal a) {
default void sniff(Animal a) {
print("dog sniff animal"); }
print("sniff animal"); }
default void flatter(Animal a) { void flatter(Dog a) {
print("u r cool dog"); }
print("u r cool animal"); }
}
}
Animal a = new Dog();
Dog d = new Dog();
a.greet(d);
// "hello animal"
a.sniff(d);
// "dog sniff animal"
d.flatter(d);
a.flatter(d);
Dynamic Method Selection Puzzle
Suppose we have classes defined below. Try to predict the results.
public interface Animal {
default void greet(Animal a) {
public class Dog implements Animal {
print("hello animal"); }
void sniff(Animal a) {
default void sniff(Animal a) {
print("dog sniff animal"); }
print("sniff animal"); }
default void flatter(Animal a) { void flatter(Dog a) {
print("u r cool dog"); }
print("u r cool animal"); }
}
}
Animal a = new Dog();
Dog d = new Dog();
a.greet(d);
// "hello animal"
a.sniff(d);
// "dog sniff animal"
d.flatter(d); // "u r cool dog"
a.flatter(d);
Dynamic Method Selection Puzzle
Suppose we have classes defined below. Try to predict the results.
public interface Animal {
default void greet(Animal a) {
public class Dog implements Animal {
print("hello animal"); }
void sniff(Animal a) {
default void sniff(Animal a) {
print("dog sniff animal"); }
print("sniff animal"); }
default void flatter(Animal a) { void flatter(Dog a) {
print("u r cool dog"); }
print("u r cool animal"); }
}
}
Animal a = new Dog();
Dog d = new Dog();
a.greet(d);
// "hello animal"
flatter is
a.sniff(d);
// "dog sniff animal"
overloaded, not d.flatter(d); // "u r cool dog"
overridden!
a.flatter(d); // “u r cool animal”
The Method Selection Algorithm
Consider the function call var1.func(var2), where var1 has static
type class1, and var2 has static type class2.
At compile time, the compiler verifies that class1 has a method named func
that can handle class2. It then records the signature of this method.
●
Note: If there are multiple methods that can handle class2, the compiler
records the “most specific” one. For example, if class2=Dog, and class1
has func(Dog) and func(Animal), it will record func(Dog).
At runtime, if var1’s dynamic type overrides the recorded signature, use the
overridden method. Otherwise, use class1’s version of the method.
Casting
Let’s say Dog also has a method, Bark(), that Animal does not have. Do the
two lines below work?
Animal A = new Dog()
A.bark()
Casting
Let’s say Dog also has a method, Bark(), that Animal does not have. Do the
two lines below work?
X
Animal A = new Dog()
A.bark()
Compiler says no! >:(
Casting
Java has a special syntax for specifying the compile-time type of any expression.
●
●
Put desired type in parenthesis before the expression.
Examples:
○
Compile-time type Animal:
○
Compile-time type Dog:
Animal fido = new Dog();
(Dog) fido;
Tells compiler to pretend it sees a particular type. Believe us, O might compiler!
Before and after: casting makeover
Fruit f = new Apple();
Apple a = f;
f.appleJuice();
Fruit f = new Apple();
Apple a = (Apple) f;
((Apple) f).appleJuice();
Lecture 3
CS 61BL Summer 2021
●
Announcements
July 8 2021
●
●
●
●
Midterm 1 today 6-8PM.
Alternate exams have been
scheduled via email.
Gitlet released!
No quiz tomorrow
Lab 9 tomorrow
Partner repo submissions
only
Last week recap
Last week...
●
●
●
●
Linked lists were our first deep dive into data structures.
○ Introduced in lecture, lab5 and lab6
○ Solidified in Proj1
We expanded from Linked Lists to consider the concept of a “list” in general.
○ Interfaces in lab7
○ Collections: list, set, map
○ Interface inheritance
Inheritance and dynamic method selection
○ Lab8
Discussion of error handling
○ What to do when something goes wrong or we don’t support some functionality?
○ Exception handling.
○ Example: Iterators and the NoSuchElementException
We’re ready to take on bigger projects!
Software Engineering
Motivation for Today
In some ways, we have misled you about what programming entails.
● 61A: Fill in the function.
● 61B(L): Implement the class according to our spec.
Always working at a “small scale” introduces habits that will cause you great
pain later.
In this lecture, we’ll try to give you a sense of how to deal with the “large scale”
● Projects 2 and 3 will give you a chance to encounter the issues yourself.
● We hope these lectures help with projects 2 and 3.
Credits
This lecture is a based on Josh Hug’s Spring 2019 SWE lectures.
His lectures were heavily inspired by “A Philosophy of Software Design” by
John Ousterhout.
● It’s cheap and very good!
● https://books.google.com/books?id=pD6-swEACAAJ&dq=philosophy+soft
ware+design&hl=en&sa=X&ved=0ahUKEwj9sZDmisvhAhXN6Z4KHcY6AYo
Q6AEIKjAA
● https://www.amazon.com/Philosophy-Software-Design-John-Ousterhout/
dp/1732102201
Complexity Defined
The Power of Software
Unlike other engineering disciplines, software is effectively unconstrained by
the laws of physics.
● Programming is an act of almost pure creativity!
The greatest limitation we face in building systems is being able to understand
what we’re building! Very unlike other disciplines, e.g.
● Chemical engineers have to worry about temperature.
● Material scientists have to worry about how brittle a material is.
● Civil engineers have to worry about the strength of concrete.
Complexity, the Enemy
Our greatest limitation is simply understanding the system we’re trying to
build!
As real programs are worked on, they gain more features and complexity.
● Over time, it becomes more difficult for programmers to understand all
the relevant pieces as they make future modifications.
Tools like IntelliJ, JUnit tests, the IntelliJ debugger, the visualizer all make it
easier to deal with complexity.
● But our most important goal is to keep our software simple.
Dealing with Complexity
There are two approaches to managing complexity:
● Making code simpler and more obvious.
○
●
Eliminating special cases, e.g. sentinel nodes.
Encapsulation into modules.
○
In a modular design, creators of one “module” can use other modules without knowing
how they work.
The Nature of Complexity
What is complexity exactly? Ousterhout defines it as:
● “Complexity is anything related to the structure of a software system that
makes it hard to understand and modify the system.”
Takes many forms:
● Understanding how the code works.
● The amount of time it takes to make small improvements.
● Finding what needs to be modified to make an improvement.
● Difficult to fix one bug without introducing another.
“If a software system is hard to understand and modify, then it is complicated.
If it is easy to understand and modify, then it is simple”.
Complexity
Note: Our usage of the term
“complex” in these lecture is not
synonymous with “large and
sophisticated”.
● It is possible for even short
programs to be complex.
Example from a former CS10
student.
What makes this complex?
Complexity and Importance
Complexity also depends on how often a piece of a system is modified.
● A system may have a few pieces that are highly complex, but if nobody
ever looks at that code, then the overall impact is minimal.
Ousterhout’s book gives a crude mathematical formulation:
● C = sum(c_p * t_p) for each part p.
○
○
c_p is the complexity of part p.
t_p is the time spent working on part p.
Symptoms and Causes of
Complexity
Symptoms of Complexity
Ousterhout describes three symptoms of complexity:
● Change amplification: A simple change requires modification in many
places.
● Cognitive load: How much you need to know in order to make a change.
○
●
Note: This is not the same as number of lines of code. Often MORE lines of code actually
makes code simpler, because it is more narrative.
Unknown unknowns: The worst type of complexity. This occurs when it’s
not even clear what you need to know in order to make modifications!
○
Common in large code bases.
Obvious Systems
In a good design, a system is ideally obvious.
In an obvious system, to make a change a developer can:
● Quickly understand how existing code works.
● Come up with a proposed change without doing too much thinking.
● Have a high confidence that the change should actually work, despite not
reading much code.
Complexity Comes Slowly
Every software system starts out beautiful, pure, and clean.
As they are built upon, they slowly twist into uglier and uglier shapes. This is
almost inevitable in real systems.
● Each complexity introduced is no big deal, but: “Complexity comes about
because hundreds or thousands of small dependences and obscurities
build up over time.”
● “Eventually, there are so many of these small issues that every possible
change is affected by several of them.”
● This incremental process is part of what makes controlling complexity so
hard, thus Ousterhout recommends a zero tolerance philosophy.
Strategic vs. Tactical
Programming
Tactical Programming
Much (or all) of the programming that you have done, Ousterhout would
describe as “tactical”.
● “Your main focus is to get something working, such as a new feature or
bug fix.” (or to get an autograder test to pass)
Tactical Programming
Much (or all) of the programming that you have done, Ousterhout would
describe as “tactical”.
● “Your main focus is to get something working, such as a new feature or
bug fix.” (or to get an autograder test to pass)
This may seem like a silly criticism. Clearly working code is good.
Tactical Programming
The problem with tactical programming:
● You don’t spend problem thinking about overall design.
● As a result, you introduce tons of little complexities, e.g. making two
copies of a method that do something similar.
● Each individual complexity seems reasonable, but eventually you start to
feel the weight.
○
○
Refactoring would fix the problem, but it would also take time, so you end up introducing
even more complexity to deal with the original ones.
Projects 2 and 3 will give you a chance to feel this!
The end result is misery.
Strategic Programming
“The first step towards becoming a good software designer is to realize that
working code isn’t enough.”
● “The most important thing is the long term structure of the system.”
● Adding complexities to achieve short term time gains is unacceptable.
Strategic programming requires lots of time investment.
● And as novice programmers, it’ll seem quite mysterious and hard. On
projects 2 and 3, try to plan ahead, but realize your system is very likely
going to be horrible looking when you’re done.
Suggestions for Strategic Programming
For each new class/task:
● Rather than implementing the first idea, try coming up with (and possibly
even partially implementing) a few different ideas.
● When you feel like you have found something that feels clean, then fully
implement that idea.
● In real systems: Try to imagine how things might need to be changed in
the future, and make sure your design can handle such changes.
Strategic Programming is Very Hard
No matter how careful you try to be, there will be mistakes in your design.
● Avoid the temptation to patch around these mistakes. Instead, fix the
design.
○
○
●
Example: Don’t add a bunch of special cases! Instead, make sure the system gracefully
handles the cases you didn’t think about.
Specific example: Adding sentinel nodes to SLLists.
Indeed, it is often impossible to design large software systems entirely in
advance.
Tactical vs. Strategic Programming Case Study
As a startup, Facebook embraced tactical programming.
● “Move fast and break things.”
● Common for new engineers to push changes to the live site within their
first week.
○
○
●
Facebook was very successful, but its codebase was a mess:
○
●
Very rapid development process in the short term.
Felt empowering!
“incomprehensible, unstable, few comments or tests, and painful to work with.”
Eventually, motto became “Move fast with stable infra.”
Note: Arguably Facebook’s general attitude has done great harm.
Tactical vs. Strategic Programming Case Study
By contrast Google and VMware are known as highly strategic organizations.
● “Both companies placed a heavy emphasis on high quality code and good
design.”
● “Both companies built sophisticated products that solved complex
problems with reliable software systems.”
● “The companies’ strong technical cultures became well known in Silicon
Valley. Few other companies could compete with them to hire the top
technical talent.”
Real world projects and companies succeed with either approach!
● … but Ousterhout says it’s probably more fun to work somewhere with a
nice code base.
Modular Design
Inspired partially by “A Philosophy of Software Design” chapters 4 and 5.
Hiding Complexity
One powerful tool for managing complexity is to design your system so that
programmer is only thinking about some of the complexity at once.
● By using helper methods and helper classes, you can hide complexity.
Modular Design
In an ideal world, each system would be broken down into modules, where
every module is totally independent.
● Here, “module” is an informal term referring to a class, a package, or
other unit of code.
● Not possible for modules to be entirely independent, because code from
each module has to call other modules.
○
e.g. need to know signature of methods to call them.
In modular design, our goal is to minimize dependencies between modules.
Interface vs. Implementation
As we’ve seen, there is an important distinction between Interface and
Implementation.
● Deque is an interface.
● LinkedListDeque, ArrayDeque, TreeDeque, etc. are implementations.
Deque
LinkedListDeque
ArrayDeque
TreeDeque
Interface vs. Implementation
Ousterhout: “The best modules are those whose interfaces are much simpler
than their implementation.” Why?
● A simple interface minimizes the complexity the module can cause
elsewhere. If you only have a getNext() method, that’s all someone can
do.
● If a module’s interface is simple, we can change an implementation of that
module without affecting the interface.
○
Silly example: If List had an arraySize method, this would mean you’d be stuck only
being able to build array based lists.
Interface
A Java interface has both a formal and an informal part:
● Formal: The list of method signatures.
● Informal: Rules for using the interface that are not enforced by the
compiler.
○
○
○
○
Example: If your code requires methods to be called in a particular order to work
properly, that is an informal part of the interface.
Example: If your add method throws an exception on null inputs, that is an informal part
of the interface.
Example: Runtime for a specific method, e.g. add in ArrayList.
Can only be specified in comments.
Be wary of any informal rules of your modules as you build projects 2 and 3.
Information Hiding
Ousterhout: “The best modules are those that provide powerful functionality
yet have simple interfaces. I use the term deep to describe such modules.”
The most important way to make your modules deep is to practice
“information hiding”.
● Embed knowledge and design decision in the module itself, without
exposing them to the outside world.
Reduces complexity in two ways:
● Simplifies interface.
● Makes it easier to modify the system.
Information Leakage
The opposite of information hiding is information leakage.
● Occurs when design decision is reflected in multiple modules.
○
○
Any change to one requires a change to all.
Information is embodied in two places, i.e. it has “leaked”.
Ousterhout:
● “Information leakage is one of the most important red flags in software
design.”
● “One of the best skills you can learn as a software designer is a high level
of sensitivity to information leakage.”
Summary and Suggestions
Some suggestions as you embark on future large scale projects:
● Build classes that provide functionality needed in many places in your
code.
● Create “deep modules”, e.g. classes with simple interfaces that do
complicated things.
● Be strategic, not tactical.
● Most importantly: Hide information from yourself when unneeded!
Hydration is key
AFAB: Drink ~ 11 cups a day
AMAB: Drink ~ 16 cups a day
Sources: cnn and harvard
TLDR: you are probably not drinking enough water.
Extra: Teamwork
Teamwork
This class is team based with teams of 2-3 students.
Two main reasons:
● Get practice working on a team.
● Learn from your partner(s) and come up with creative solutions together.
Ancillary reason: Also reduces programming workload per person. This course
could potentially be made a solo experience but we have decided against that.
Slides again adapted from Josh Hug. Some material for this section of lecture
drawn from www.teamingxdesign.com.
Teamwork
In the real world, some tasks are much too large to be handled by a single
person.
It’s important to learn how to work well in groups now! This is a real world
skill.
Group Intelligence
In the famous “Evidence for a Collective Intelligence Factor in the Performance
of Human Groups”, Woolley et. al investigated the success of teams of
humans on various tasks.
They found that performance on a wide variety of tasks is correlated, i.e.
groups that do well on any specific task tend to do very well on the others.
● This suggests that groups do have “group intelligence”
Group Intelligence
In the famous “Evidence for a Collective Intelligence Factor in the Performance
of Human Groups”, Woolley et. al investigated the success of teams of
humans on various tasks.
Studying individual group members, Woolley et. al found that:
● Collective intelligence is not significantly correlated with average or max
intelligence of each group.
Group Intelligence
In the famous “Evidence for a Collective Intelligence Factor in the Performance
of Human Groups”, Woolley et. al investigated the success of teams of
humans on various tasks.
Studying individual group members, Woolley et. al found that:
● Instead, collective intelligence was correlated with three things:
Group Intelligence
In the famous “Evidence for a Collective Intelligence Factor in the Performance
of Human Groups”, Woolley et. al investigated the success of teams of
humans on various tasks.
Studying individual group members, Woolley et. al found that:
● Instead, collective intelligence was correlated with three things:
○
Average social sensitivity of group members as measured using the “Reading the Mind in
the Eyes Test” (this is really interesting).
Group Intelligence
In the famous “Evidence for a Collective Intelligence Factor in the Performance
of Human Groups”, Woolley et. al investigated the success of teams of
humans on various tasks.
Studying individual group members, Woolley et. al found that:
● Instead, collective intelligence was correlated with three things:
○
○
Average social sensitivity of group members as measured using the “Reading the Mind in
the Eyes Test” (this is really interesting).
How equally distributed the group was in conversational turn-taking, e.g. groups where
one person dominated did poorly.
Group Intelligence
In the famous “Evidence for a Collective Intelligence Factor in the Performance
of Human Groups”, Woolley et. al investigated the success of teams of
humans on various tasks.
Studying individual group members, Woolley et. al found that:
● Instead, collective intelligence was correlated with three things:
○
○
○
Average social sensitivity of group members as measured using the “Reading the Mind in
the Eyes Test” (this is really interesting).
How equally distributed the group was in conversational turn-taking, e.g. groups where
one person dominated did poorly.
Percentage of females in the group (paper suggests this is due to correlation with greater
social sensitivity).
Teamwork
So, your success as a team isn’t about just letting the “smartest” person do all
the work.
Instead, it’s about effectively collaborating and communicating to build your
group intelligence.
●
●
●
●
Treat each other with respect.
Be open and honest with each other.
Make sure to set clear expectations.
… and if those expectations are not met, confront this fact head on.
Reflexivity
Important part of teamwork is “reflexivity”.
● “A group’s ability to collectively reflect upon team objectives, strategies,
and processes, and to adapt to them accordingly.”
● Recommended that you “cultivate a collaborative environment in which
giving and receiving feedback on an ongoing basis is seen as a
mechanism for reflection and learning.”
○
It’s OK and even expected for you and your partner to be a bit unevenly matched in terms
of programming ability.
You might find this description of best practices for team feedback useful,
though it’s targeted more towards larger team projects.
● Some key ideas from this document follow.
Feedback is Hard: Negativity
Most of us have received feedback from someone which felt judgmental or in
bad faith.
● Thus, we’re afraid to give even constructive negative feedback for fear
that our feedback will be misconstrued as an attack.
● And we’re conditioned to watch out for negative feedback that is
ill-intentioned.
Feedback is Hard: Can Seem Like a Waste of Time
Feedback also can feel like a waste of time:
● You may find it a pointless exercise to reflect on one other during the
project. What does that have to do with a programming class?
● In the real world, the same thing happens. Your team has limited time to
figure out “what” to do, so why stop and waste time reflecting on “how”
you’re working together?
Feedback is Hard: Coming Up With Feedback is Tough
Feedback can simply be difficult to produce.
● You may build incredible technical skills, but learning to provide useful
feedback is hard!
● Without confidence in your ability to provide feedback, you may wait until
you are forced to do so by annual reviews or other structured time to
provide it.
○
If you feel like your partnership could be better, try to talk about it without waiting. Both
parties will probably feel better if you share how you are feeling.
Gitlet Introduction
Some Advice
•
Gitlet is the best
•
Start early!
•
Complete lab 12
•
Read the spec
Gitlet Introduction
java gitlet.Main init
git init
Working Directory
Gitlet Repository
Staging Area
.
Commits
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Commits
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Commits
Commit 0
Metadata
null
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Commits
Commit 0
Metadata
null
message = “initial commit”
timestamp = 00:00:00 UTC, Thursday, 1 January 1970
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Commits
Commit 0
Metadata
null
Working Directory
Gitlet Repository
.
Staged for addition
Master
Commit 0
Metadata
null
HEAD
Staged for removal
Staging Area
Commits
nano Hello.txt
Working Directory
Gitlet Repository
.
Staged for addition
Master
Commit 0
Metadata
null
HEAD
Staged for removal
Staging Area
Commits
java gitlet.Main add Hello.txt
git add Hello.txt
Working Directory
Gitlet Repository
.
Staged for addition
Master
HEAD
Staged for removal
Staging Area
Commits
Commit 0
Metadata
null
Blobs
Working Directory
Gitlet Repository
.
Staged for addition
Master
Staged for removal
Staging Area
Commits
HEAD
Commit 0
Metadata
null
Blobs
Blob 0
“Hello”
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Hello.txt → Blob 0
Master
HEAD
Commits
Commit 0
Metadata
null
Blob 0
“Hello”
Blobs
java gitlet.Main commit “Created Hello.txt”
git commit -m “Created Hello.txt”
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Hello.txt → Blob 0
Master
HEAD
Commits
Commit 0
Metadata
null
Blob 0
“Hello”
Blobs
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Hello.txt → Blob 0
Master
Commits
HEAD
Commit 0
Commit 1
Metadata
Metadata
null
null
Blobs
Blob 0
“Hello”
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
Hello.txt → Blob 0
Master
Commits
HEAD
Commit 0
Commit 1
Metadata
Metadata
null
null
Blob 0
“Hello”
message = “Created Hello.txt”
timestamp = 17:10:00 PST, Monday, 13 April 2020
Blobs
Working Directory
Gitlet Repository
.
Staged for addition
Master
Staged for removal
Commits
HEAD
Commit 0
Commit 1
Metadata
Metadata
null
Staging Area
Hello.txt
Blobs
Blob 0
“Hello”
Working Directory
Gitlet Repository
.
Staged for addition
Master
Commit 1
Metadata
Metadata
Blob 0
“Hello”
Staging Area
Commits
HEAD
Commit 0
null
Staged for removal
Hello.txt
Blobs
Working Directory
Gitlet Repository
.
Staged for addition
Master
Staged for removal
HEAD
Commit 0
Commit 1
Metadata
Metadata
null
Staging Area
Commits
Hello.txt
Blobs
Blob 0
“Hello”
nano World.txt
java gitlet.Main add World.txt
java gitlet.Main add Hello.txt
java gitlet.Main commit “Created World.txt and
tried to add Hello.txt”
Working Directory
Gitlet Repository
.
Staged for addition
Master
Staged for removal
Commit 1
Metadata
Metadata
null
Commits
HEAD
Commit 0
Staging Area
Hello.txt
Blobs
Blob 0
“Hello”
Working Directory
Gitlet Repository
.
Staged for addition
Master
Staged for removal
HEAD
Commit 0
Commit 1
Metadata
Metadata
null
Blob 0
“Hello”
Staging Area
Commits
Hello.txt
Blobs
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
Staging Area
World.txt → Blob 1
Master
Commit 0
Commit 1
Metadata
Metadata
null
Commits
HEAD
Hello.txt
Blob 0
Blob 1
“Hello”
“World”
Blobs
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
HEAD
Master
Commit 0
Commit 1
Commit 2
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
Blob 0
Blob 1
“Hello”
“World”
Staging Area
Commits
World.txt
Blobs
nano Hello.txt
nano World.txt
java gitlet.Main add World.txt
java gitlet.Main add Hello.txt
java gitlet.Main commit “Modified World.txt
and Hello.txt”
Working Directory
Gitlet Repository
.
Staged for addition
Staged for removal
HEAD
Master
Commit 0
Commit 1
Commit 2
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
Blob 0
Blob 1
“Hello”
“World”
Staging Area
Commits
World.txt
Blobs
Working Directory
Gitlet Repository
.
Staged for addition
Staging Area
Staged for removal
Hello.txt → Blob 2
World.txt → Blob 3
Commit 0
Commit 1
Commit 2
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
Commits
HEAD
Master
World.txt
Blobs
Blob 0
Blob 1
Blob 2
Blob 3
“Hello”
“World”
“Hello there”
“World!!!!!”
Working Directory
Gitlet Repository
.
Staged for addition
Staging Area
Staged for removal
HEAD Commits
Master
Commit 0
Commit 1
Commit 2
Commit 3
Metadata
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
World.txt
Hello.txt
World.txt
Blob 0
Blob 1
Blob 2
Blob 3
“Hello”
“World”
“Hello there”
“World!!!!!”
Blobs
java gitlet.Main init
java gitlet.Main log
//Read from.gitlet all the files you need to run
….
//Write onto disk (.gitlet folder) all the new objects you created / modified
Diving Into the
Technical Details
Working Directory
Gitlet Repository
.
Staged for addition
Staging Area
Staged for removal
HEAD Commits
Master
Commit 0
Commit 1
Commit 2
Commit 3
Metadata
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
World.txt
Hello.txt
World.txt
Blobs
Blob 0
Blob 1
Blob 2
Blob 3
“Hello”
“World”
“Hello there”
“World!!!!!”
Working Directory
Gitlet Repository
.
Staged for addition
Staging Area
Staged for removal
HEAD Commits
Master
Commit 0
Commit 1
Commit 2
Commit 3
Metadata
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
World.txt
Hello.txt
World.txt
Blob 0
Blob 1
Blob 2
Blob 3
“Hello”
“World”
“Hello there”
“World!!!!!”
Blobs
HEAD Commits
Master
Commit 0
Commit 1
Commit 2
Commit 3
Metadata
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
World.txt
Hello.txt
World.txt
Blob 0
Blob 1
Blob 2
Blob 3
“Hello”
“World”
“Hello there”
“World!!!!!”
HEAD Commits
Master
Commit 0
Commit 1
Commit 2
Commit 3
Metadata
Metadata
Metadata
Metadata
null
Hello.txt
Hello.txt
World.txt
Hello.txt
Blobs
World.txt
Blob 0
Blob 1
Blob 2
Blob 3
“Hello”
“World”
“Hello there”
“World!!!!!”
Blobs
HEAD
Master
Commit 3 Commit 3
Hello.txt
Blob 0
Commit 3
Metadata
Hello.txt
World.txt
Blob 0
Blob 1
Commit 2
null
Metadata
Commit 1
Metadata
Commit 2
Commit 1
Commit 0
Commit 0
Metadata
Hello.txt
World.txt
Blob 2
Blob 3
Blob 0
Blob 1
Blob 2
Blob 3
“Hello”
“World”
“Hello there”
“World!!!!!”
commit 0
commit 1
Commits
commit 2
commit ?
commit 3
Blobs
commit 0
commit 1
commit 2
commit ?
commit 3
SHA1(
) = 8e95382d558549303e6
HEAD
Master
5e6194cbf 5e6194cbf
Hello.txt
a699ba6
5e6194cbf
Metadata
Hello.txt
World.txt
a699ba6
9639c673b
71b40c70
null
Metadata
9639c673b
Metadata
71b40c70
9639c673b
f3af0eb43
f3af0eb43
Commits
Metadata
Hello.txt
World.txt
0e93cac
0e84daf
a699ba6
9639c673b
0e93cac
0e84daf
“Hello”
“World”
“Hello there”
“World!!!!!”
Blobs
HEAD
Master
5e6194cbf 5e6194cbf
a699ba6
Hello.txt
a699ba6
9639c673b
5e6194cbf
Metadata
Hello.txt
World.txt
a699ba6
9639c673b
0e93cac
71b40c70
null
Metadata
9639c673b
Metadata
71b40c70
9639c673b
f3af0eb43
f3af0eb43
Commits
Metadata
Hello.txt
World.txt
0e93cac
0e84daf
0e84daf
001001010
111001010
111011010
111001111
byte[ ]
byte[ ]
byte[ ]
byte[ ]
Blobs
Lecture 4
CS 61BL Summer 2021
●
●
●
Announcements
7/12/21
●
●
●
●
Midterm grades out
Midterm regrade requests open
tomorrow 7/13, close in a week.
Grades tab in Beacon is out. Some
assignments (extra credit practice
exam, surveys, midterm) have yet
to be imported.
Lab Tues-Friday (Tues is a
Gradescope assignment, Friday is a
Proj2 workday)
Gitlet Checkpoint Friday
Quiz 5 and 6 this Wednesday and
Friday
Incident report form and shoutout
form both on front page of site.
Asymptotic Analysis
61B(L): Writing Efficient Programs
“An engineer will do for a dime what any fool will do for a dollar.”
Efficiency comes in two flavors:
●
Programming cost (course to date. Will also revisit later).
○
○
How long does it take to develop your programs?
How easy is it to read, modify, and maintain your code?
■ More important than you might think!
■ Majority of cost is in maintenance, not development!
61B(L): Writing Efficient Programs
“An engineer will do for a dime what any fool will do for a dollar.”
Efficiency comes in two flavors:
●
Programming cost (course to date. Will also revisit later).
○
○
●
How long does it take to develop your programs?
How easy is it to read, modify, and maintain your code?
■ More important than you might think!
■ Majority of cost is in maintenance, not development!
Execution cost (from today until end of course).
○
○
How much time does your program take to execute?
How much memory does your program require?
Example of Algorithm Cost
Objective: Determine if a sorted array contains any duplicates.
●
Given sorted array A, are there indices i and j where A[i] = A[j]?
-3
-1
2
4
4
8
10
12
Silly algorithm: Consider every possible pair, returning true if any match.
●
Are (-3, -1) the same? Are (-3, 2) the same? ...
Better algorithm?
Example of Algorithm Cost
Objective: Determine if a sorted array contains any duplicates.
●
Given sorted array A, are there indices i and j where A[i] = A[j]?
-3
-1
2
4
4
8
10
12
Silly algorithm: Consider every possible pair, returning true if any match.
●
Are (-3, -1) the same? Are (-3, 2) the same? ...
Better algorithm?
●
Today’s goal: Introduce formal
technique for comparing
algorithmic efficiency.
For each number A[i], look at A[i+1], and return true the first time you
see a match. If you run out of items, return false.
Intuitive Runtime Characterizations
How Do I Runtime Characterization?
Our goal is to somehow characterize the runtimes of the functions below.
● Characterization should be simple and mathematically rigorous.
● Characterization should demonstrate superiority of dup2 over dup1.
public static boolean dup1(int[] A) {
for (int i = 0; i < A.length; i += 1) {
for (int j = i + 1; j < A.length; j += 1) {
dup2
if (A[i] == A[j]) {
public static boolean dup2(int[] A) {
return true;
for (int i = 0; i < A.length - 1; i += 1) {
}
if (A[i] == A[i + 1]) {
}
return true;
}
}
return false;
}
}
return false;
dup1
}
Techniques for Measuring Computational Cost
Technique 1: Measure execution time in seconds using a client program.
● Tools:
○
○
○
Physical stopwatch.
Unix has a built in time command that measures execution time.
Princeton Standard library has a Stopwatch class.
public static void main(String[] args) {
int N = Integer.parseInt(args[0]);
int[] A = makeArray(N);
// record times for each function call
dup1(A);
dup2(A);
}
Time Measurements for dup1 and dup2
N
dup1
dup2
10000
0.08
0.08
50000
0.32
0.08
100000
1.00
0.08
200000
8.26
0.1
400000
15.4
0.1
seconds
N
Time to complete (in seconds)
Techniques for Measuring Computational Cost
Technique 1: Measure execution time in seconds using a client program.
● Good: Easy to measure, meaning is obvious.
● Bad: May require large amounts of computation time. Result varies with
machine, compiler, input data, etc.
public static void main(String[] args) {
int N = Integer.parseInt(args[0]);
int[] A = makeArray(N);
// record times for each function call
dup1(A);
dup2(A);
}
Techniques for Measuring Computational Cost
Technique 2A: Count possible operations for an array of size N = 10,000.
● Good: Machine independent. Input dependence captured in model.
● Bad: Tedious to compute. Array size was arbitrary. Doesn’t tell you actual
time.
operation
count, N=10000
for (int i = 0; i < A.length; i += 1) {
for (int j = i+1; j < A.length; j += 1) {
if (A[i] == A[j]) {
return true;
}
}
}
return false;
i=0
1
j=i+1
1 to 10000
less than (<)
2 to 50,015,001
increment (+=1)
0 to 50,005,000
equals (==)
1 to 49,995,000
array accesses
2 to 99,990,000
The counts are tricky to compute. Work not shown.
best case
worst case
Techniques for Measuring Computational Cost
Technique 2B: Count possible operations in terms of input array size N.
● Good: Machine independent. Input dependence captured in model. Tells you
how algorithm scales.
● Bad: Even more tedious to compute. Doesn’t tell you actual time.
for (int i = 0; i < A.length; i += 1) {
for (int j = i + 1; j < A.length; j += 1)
{
if (A[i] == A[j]) {
return true;
}
}
}
return false;
operation
symbolic count
count, N=10000
i=0
1
1
j=i+1
1 to N
1 to 10000
less than (<)
2 to (N2+3N+2)/2
2 to 50,015,001
increment (+=1)
0 to (N2+N)/2
0 to 50,005,000
equals (==)
1 to (N2-N)/2
1 to 49,995,000
array accesses
2 to N2-N
2 to 99,990,000
Techniques for Measuring Computational Cost
Your turn: Try to come up with rough estimates for the symbolic and
exact counts for at least one of the operations.
● Tip: Don’t worry about being off by one. Just try to predict the rough
magnitudes of each.
for (int i = 0; i < A.length - 1; i += 1){
if (A[i] == A[i + 1]) {
return true;
}
}
return false;
operation
sym.
count
count,
N=10000
i=0
1
1
less than (<)
increment (+=1)
equals (==)
array accesses
Techniques for Measuring Computational Cost
Your turn: Try to come up with rough estimates for the symbolic and
exact counts for at least one of the operations.
for (int i = 0; i < A.length - 1; i += 1) {
if (A[i] == A[i + 1]) {
return true;
}
}
return false;
Especially observant folks may notice we didn’t count
everything, e.g. “- 1” and “+ 1” operations. We’ll see
why this omission is not a problem very shortly.
operation
symbolic
count
count,
N=10000
i=0
1
1
less than (<)
0 to N
0 to 10000
increment (+=1)
0 to N - 1
0 to 9999
equals (==)
1 to N - 1
1 to 9999
array accesses
2 to 2N - 2
2 to 19998
If you did this exercise but were off by one, that’s fine. The exact numbers aren’t that important.
Comparing Algorithms
Which algorithm is better? Why?
dup1
dup2
operation
symbolic count
count, N=10000
i=0
1
1
j=i+1
1 to N
1 to 10000
less than (<)
2 to (N2+3N+2)/2
2 to 50,015,001
increment (+=1)
0 to (N2+N)/2
0 to 50,005,000
equals (==)
1 to (N2-N)/2
1 to 49,995,000
array accesses
2 to N2-N
2 to 99,990,000
operation
symbolic
count
count,
N=10000
i=0
1
1
less than (<)
0 to N
0 to 10000
increment (+=1)
0 to N - 1
0 to 9999
equals (==)
1 to N - 1
1 to 9999
array accesses
2 to 2N - 2
2 to 19998
Comparing Algorithms
Which algorithm is better? dup2. Why?
● Fewer operations to do the same work [e.g. 50,015,001 vs. 10000 operations].
● Better answer: Algorithm scales better in the worst case. (N2+3N+2)/2 vs. N.
● Even better answer: Parabolas (N2) grow faster than lines (N).
dup1
dup2
operation
symbolic count
count, N=10000
i=0
1
1
j=i+1
1 to N
1 to 10000
less than (<)
2 to (N2+3N+2)/2
2 to 50,015,001
increment (+=1)
0 to (N2+N)/2
0 to 50,005,000
equals (==)
1 to (N2-N)/2
1 to 49,995,000
array accesses
2 to N2-N
2 to 99,990,000
operation
symbolic
count
count,
N=10000
i=0
1
1
less than (<)
0 to N
0 to 10000
increment (+=1)
0 to N - 1
0 to 9999
equals (==)
1 to N - 1
1 to 9999
array accesses
2 to 2N - 2
2 to 19998
Asymptotic Behavior
In most cases, we care only about asymptotic behavior, i.e. what happens
for very large N.
● Simulation of billions of interacting particles.
● Social network with billions of users.
● Logging of billions of transactions.
● Encoding of billions of bytes of video data.
Algorithms which scale well (e.g. look like lines) have better asymptotic runtime
behavior than algorithms that scale relatively poorly (e.g. look like parabolas).
Parabolas vs. Lines
Suppose we have two algorithms that zerpify a collection of N items.
● zerp1 takes 2N2 operations.
● zerp2 takes 500N operations.
For small N, zerp1 might be faster, but as dataset size grows, the parabolic
algorithm is going to fall farther and farther behind (in time it takes to
complete).
zerp2
zerp1
Scaling Across Many Domains
We’ll informally refer to the “shape” of a runtime function as its
order of growth (will formalize soon).
● Effect is dramatic! Often determines whether a problem can be solved at all.
(from Algorithm Design: Tardos, Kleinberg)
Duplicate Finding
Our goal is to somehow characterize the runtimes of the functions below.
Characterization should be simple and mathematically rigorous.
Characterization should demonstrate superiority of dup2 over dup1.
dup1: parabolic, a.k.a. quadratic
dup2: linear
operation
symbolic count
operation
i=0
1
symbolic
count
j=i+1
1 to N
i=0
1
less than (<)
2 to (N2+3N+2)/2
less than (<)
0 to N
increment (+=1)
0 to (N2+N)/2
increment (+=1)
0 to N - 1
equals (==)
1 to (N2-N)/2
equals (==)
1 to N - 1
array accesses
2 to N2-N
array accesses
2 to 2N - 2
Worst Case
Order of Growth
Duplicate Finding
Our goal is to somehow characterize the runtimes of the functions below.
● Characterization should be simple and mathematically rigorous.
operation
count
i=0
1
j=i+1
1 to N
operation
count
i=0
1
less than (<)
0 to N
increment (+=1)
0 to N - 1
equals (==)
1 to N - 1
array accesses
2 to 2N - 2
2
less than (<)
2 to (N +3N+2)/2
increment (+=1)
0 to (N2+N)/2
2
equals (==)
1 to (N -N)/2
array accesses
2 to N2-N
Let’s be more careful about what we mean when we say the left function is “like” a
parabola, and the right function is “like” a line.
Intuitive Simplification 1: Consider Only the Worst Case
Simplification 1: Consider only the worst case.
for (int i = 0; i < A.length; i += 1) {
for (int j = i+1; j < A.length; j += 1) {
if (A[i] == A[j]) {
return true;
}
}
}
return false;
operation
count
i=0
1
j=i+1
1 to N
less than (<)
2 to (N2+3N+2)/2
increment (+=1)
0 to (N2+N)/2
equals (==)
1 to (N2-N)/2
array accesses
2 to N2-N
Intuitive Simplification 1: Consider Only the Worst Case
Simplification 1: Consider only the worst case.
● Justification: When comparing algorithms, we often care only about the
worst case [but we will see exceptions in this course].
for (int i = 0; i < A.length; i += 1) {
for (int j = i+1; j < A.length; j += 1) {
if (A[i] == A[j]) {
return true;
}
}
}
return false;
We’re effectively focusing on the case where there are no
duplicates, because this is where there is a performance
difference.
operation
worst case count
i=0
1
j=i+1
N
less than (<)
(N2+3N+2)/2
increment (+=1)
(N2+N)/2
equals (==)
(N2-N)/2
array accesses
N2-N
Intuitive Order of Growth Identification:
Consider the algorithm below. What do you expect will be the
order of growth of the runtime for the algorithm?
A.
B.
C.
D.
N
N2
N3
N6
[linear]
[quadratic]
[cubic]
[sextic]
operation
count
less than (<)
100N2 + 3N
greater than (>)
2N3 + 1
and (&&)
5,000
In other words, if we plotted total runtime vs. N, what shape would we expect?
Intuitive Order of Growth Identification
Consider the algorithm below. What do you expect will be the
order of growth of the runtime for the algorithm?
A.
B.
C.
D.
N [linear]
N2 [quadratic]
N3 [cubic]
N6 [sextic]
operation
count
less than (<)
100N2 + 3N
greater than (>)
2N3 + 1
and (&&)
5,000
Argument:
● Suppose < takes α nanoseconds, > takes β nanoseconds, and && takes γ
nanoseconds.
Extremely
2
3
important point.
● Total time is α(100N + 3N) + β(2N + 1) + 5000γ nanoseconds.
Make sure you
● For very large N, the 2βN3 term is much larger than the others.
understand it!
Intuitive Simplification 2: Restrict Attention to One Operation
Simplification 2: Pick some representative operation to act as a proxy
for the overall runtime.
There are other good choices.
● Good choice: increment.
● Bad choice: assignment of j = i + 1.
operation
worst case count
We call our choice the “cost model”.
for (int i = 0; i < A.length; i += 1) {
for (int j = i+1; j < A.length; j += 1) {
if (A[i] == A[j]) {
return true;
}
}
}
return false;
i=0
1
j=i+1
N
less than (<)
(N2+3N+2)/2
increment (+=1)
(N2+N)/2
equals (==)
(N2-N)/2
array accesses
N2-N
cost model = increment
Intuitive Simplification 3: Eliminate low order terms.
Simplification 3: Ignore lower order terms.
for (int i = 0; i < A.length; i += 1) {
for (int j = i+1; j < A.length; j += 1) {
if (A[i] == A[j]) {
return true;
}
}
}
return false;
operation
worst case
increment (+=1)
(N2+N)/2
Intuitive Simplification 4: Eliminate multiplicative constants.
Simplification 4: Ignore multiplicative constants.
● Why? It has no real meaning. We already threw away information when
we chose a single proxy operation.
for (int i = 0; i < A.length; i += 1) {
for (int j = i+1; j < A.length; j += 1) {
if (A[i] == A[j]) {
return true;
}
}
}
return false;
operation
worst case
increment (+=1)
N2/2
Simplification Summary
Simplifications:
1. Only consider the worst case.
2. Pick a representative operation (a.k.a. the cost model).
3. Ignore lower order terms.
4. Ignore multiplicative constants.
operation
count
i=0
1
j=i+1
1 to N
less than (<)
2 to (N2+3N+2)/2
These three simplifications are OK because we only
care about the “order of growth” of the runtime.
2
increment (+=1)
0 to (N +N)/2
equals (==)
1 to (N2-N)/2
array accesses
2 to N2-N
operation
worst case o.o.g.
increment (+=1)
N2
Worst case order of growth of runtime: N2
Big-Theta
Formalizing Order of Growth
Given a function R(N), we can apply our last two simplifications
(ignore lower order terms and multiplicative constants) to yield the order of
growth.
● Example: R(N) = 3N3 + N2
● Order of growth: N3
Let’s move to a more formal notation called Big-Theta.
● The math might seem daunting at first.
● … but the idea is exactly the same! Using “Big-Theta” instead of “order of
growth” does not change the way we analyze code at all.
Order of Growth Exercise
Consider the functions below.
● Informally, what is the “shape” of each function for very large N?
●
In other words, what is the order of growth of each function?
function
order of growth
N3 + 3N4
1/N + N3
1/N + 5
NeN + N
40 sin(N) + 4N2
Order of Growth Exercise
Consider the functions below.
● Informally, what is the “shape” of each function for very large N?
●
In other words, what is the order of growth of each function?
function
order of growth
N3 + 3N4
N4
1/N + N3
N3
1/N + 5
1
NeN + N
NeN
40 sin(N) + 4N2
N2
Big-Theta
Suppose we have a function R(N) with order of growth f(N).
● In “Big-Theta” notation we write this as R(N) ∈ Θ(f(N)).
● Examples:
○
○
○
○
○
N3 + 3N4 ∈ Θ(N4)
1/N + N3 ∈ Θ(N3)
1/N + 5 ∈ Θ(1)
NeN + N ∈ Θ(NeN)
40 sin(N) + 4N2 ∈ Θ(N2)
function R(N)
order of growth
N3 + 3N4
N4
1/N + N3
N3
1/N + 5
1
NeN + N
NeN
40 sin(N) + 4N2
N2
Big-Theta: Formal Definition (Visualization)
means there exist positive constants k1 and k2 such that:
for all values of N greater than some N0.
i.e. very large N
2
2
Example: 40 sin(N) + 4N ∈ Θ(N )
● R(N) = 40 sin(N) + 4N2
● f(N) = N2
● k1 = 3
● k2 = 5
Big-Theta: Intuitive Definition
means that R(N) will always take f(N) time. In other
words, every time we execute the R(N) program, we are
guaranteed it will take f(N) time.
Big-Theta: Intuitive Definition
means that R(N) will always take f(N) time. In other
words, every time we execute the R(N) program, we are
guaranteed it will take f(N) time.
for (int i = 0; i < N; i += 1) {
System.out.println(i);
}
The function to the left always takes N time to run, so we
can say it is runs in Θ(N)
Big-Theta and Runtime Analysis
Using Big-Theta doesn’t change anything about runtime analysis
(no need to find k1 or k2 or anything like that).
● The only difference is that we use the Θ symbol anywhere we would have
said “order of growth”.
operation
worst case count
i=0
1
j=i+1
Θ(N)
operation
worst case count
increment (+=1)
Θ(N2)
2
less than (<)
Θ(N )
increment (+=1)
Θ(N2)
equals (==)
Θ(N2)
array accesses
Θ(N2)
Big O Notation
Worst case runtime: Θ(N2)
Big Theta
We used Big Theta to describe the order of growth of a function.
function R(N)
order of growth
N3 + 3N4
Θ(N4)
1/N + N3
Θ(N3)
1/N + 5
Θ(1)
NeN + N
Θ(NeN)
40 sin(N) + 4N2
Θ(N2)
We also used Big Theta to describe the rate of growth of the runtime
of a piece of code.
Big O
Whereas Big Theta can informally be thought of as something
like “equals”, Big O can be thought of as “less than or equal”.
Example, the following are all true:
● N3 + 3N4 ∈ Θ(N4)
● N3 + 3N4 ∈ O(N4)
● N3 + 3N4 ∈ O(N6)
● N3 + 3N4 ∈ O(N!)
● N3 + 3N4 ∈ O(NN!)
Big O
Whereas Big Theta can informally be thought of as something
like “equals”, Big O can be thought of as “less than or equal”.
Example, the following are all true:
● N3 + 3N4 ∈ Θ(N4)
● N3 + 3N4 ∈ O(N4)
Note that this is the tightest big O bound
● N3 + 3N4 ∈ O(N6)
● N3 + 3N4 ∈ O(N!)
● N3 + 3N4 ∈ O(NN!)
Big O: Formal Definition (Visualization)
means there exists a positive constant k2 such that:
for all values of N greater than some N0.
i.e. very large N
Big O: Intuitive Definition
means that the function R(N) will never be worse, or
slower, than this bound.
A function is “bad” if it
takes a long time to run
Big O: Intuitive Definition
means that the function R(N) will never be worse, or
slower, than this bound.
N is A.length
dup2
for (int i = 0; i < A.length - 1; i += 1) {
if (A[i] == A[i + 1]) {
return true;
}
}
return false;
What is the runtime of dup2, with
a tight big O bound?
Big O: Intuitive Definition
means that the function R(N) will never be worse, or
slower, than this bound.
N is A.length
dup2
for (int i = 0; i < A.length - 1; i += 1) {
if (A[i] == A[i + 1]) {
return true;
}
}
return false;
Big Omega Notation
What is the runtime of dup2, with
a tight big O bound?
Answer: O(N)
Big Omega
Whereas Big O can informally be thought of as something
like “less than or equal”, Big Omega Ω can be thought of as “greater
than or equal”.
Example, the following are all true:
● N3 + 3N4 ∈ Ω(N4)
● N3 + 3N4 ∈ Ω(N3)
● N3 + 3N4 ∈ Ω(N2)
● N3 + 3N4 ∈ Ω(N)
● N3 + 3N4 ∈ Ω(1)
Are all functions Ω(1)?
Big Omega
Whereas Big O can informally be thought of as something
like “less than or equal”, Big Omega Ω can be thought of as “greater
than or equal”.
Example, the following are all true:
● N3 + 3N4 ∈ Ω(N4)
Note that this is the tightest big Ω bound
● N3 + 3N4 ∈ Ω(N3)
● N3 + 3N4 ∈ Ω(N2)
● N3 + 3N4 ∈ Ω(N)
● N3 + 3N4 ∈ Ω(1)
Are all functions Ω(1)?
Big Omega: Formal Definition
means there exists a positive constant k1 such that:
for all values of N greater than some N0.
i.e. very large N
Big Omega: Intuitive Definition
means that the function R(N) will never be better, or
faster, than this bound.
Big Omega: Intuitive Definition
means that the function R(N) will never be better, or
faster, than this bound.
N is A.length
dup2
for (int i = 0; i < A.length - 1; i += 1) {
if (A[i] == A[i + 1]) {
return true;
}
}
return false;
What is the runtime of dup2, with
a tight big Ω bound?
Big Omega: Intuitive Definition
means that the function R(N) will never be better, or
faster, than this bound.
N is A.length
dup2
for (int i = 0; i < A.length - 1; i += 1) {
if (A[i] == A[i + 1]) {
return true;
}
}
return false;
What is the runtime of dup2, with
a tight big Ω bound?
Answer: Ω(1)
Summary: Big Theta vs. Big O vs. Big Omega
Big Theta
Θ(f(N))
Big O
O(f(N))
Big Omega
Ω(f(N))
Informal meaning:
Family
Family Members
Order of growth is
f(N).
Θ(N2)
N2/2
2N2
N2 + 38N + N
Order of growth is
less than or equal
to f(N).
O(N2)
N2/2
2N2
lg(N)
Order of growth is
greater than or
equal to f(N).
Ω(N2)
N2/2
2N2
8N4
iPad Demo (if time permits)
Problem 1 from this discussion
Citations
Slides revamped from Professor Hug’s Spring 2019 CS 61B lecture
An introduction to Trees & Traversals
What is a tree?
Trees naturally represent recursively defined, hierarchical objects with more
than one recursive subpart for each instance
A tree consists of:
●
●
A set of nodes.
A set of edges that connect those nodes.
○
Constraint: There is exactly one path between any two nodes.
Rooted Trees
A rooted tree is a tree where we’ve chosen one node as the “root”.
●
●
Every node N except the root has exactly one parent, defined as the first
node on the path from N to the root.
A node with no child is called a leaf.
A
For each of these:
B
● A is the root.
● B is a child of A. (and C of B)
C
● A is a parent of B. (and B of C)
A
A
B
C
C
B
C
Tree traversal orderings: level
Level Order
●
Visit top-to-bottom, left-to-right (like reading in English):
D
B
A
F
E
C
G
Tree traversal orderings: level
Level Order
●
Visit top-to-bottom, left-to-right (like reading in English): DBFACEG
D
B
A
F
C
E
G
Tree traversal orderings: depth first
Depth First Traversals
●
●
●
3 types: Preorder, Inorder, Postorder
Basic (rough) idea: Traverse “deep nodes” (e.g. A) before shallow ones (e.g.
F).
Note: Traversing a node is different than “visiting” a node. See next slide.
D
B
A
F
E
C
G
Depth first: preorder
Preorder: “Visit” a node, then traverse its children:
preOrder(BSTNode x) {
if (x == null) return;
print(x.key)
preOrder(x.left)
preOrder(x.right)
}
D
B
A
F
C
E
G
Depth first: preorder
Preorder: “Visit” a node, then traverse its children:
preOrder(BSTNode x) {
if (x == null) return;
print(x.key)
preOrder(x.left)
preOrder(x.right)
}
D B A C F E G
D
B
A
F
E
C
G
Depth first: inorder
Inorder traversal: Traverse left child, visit, then traverse right child:
inOrder(BSTNode x) {
if (x == null) return;
inOrder(x.left)
print(x.key)
inOrder(x.right)
}
D
B
A
F
C
E
G
Depth first: inorder
Inorder traversal: Traverse left child, visit, then traverse right child:
inOrder(BSTNode x) {
if (x == null) return;
inOrder(x.left)
print(x.key)
inOrder(x.right)
}
A B C DE F G
D
B
A
F
E
C
G
Depth first: postorder
Postorder traversal: Traverse left, traverse right, then visit:
postOrder(BSTNode x) {
if (x == null) return;
postOrder(x.left)
postOrder(x.right)
print(x.key)
}
D
B
A
F
C
E
G
Depth first: postorder
Postorder traversal: Traverse left, traverse right, then visit: ACBEGFD
postOrder(BSTNode x) {
if (x == null) return;
postOrder(x.left)
postOrder(x.right)
print(x.key)
}
D
B
F
A
C
E
G
A Useful Visual Trick (for Humans, Not Algorithms)
●
●
●
Preorder traversal: We trace a path around the graph, from the top going
counter-clockwise. “Visit” every time we pass the LEFT of a node.
Inorder traversal: “Visit” when you cross the bottom of a node.
Postorder traversal: “Visit” when you
cross the right a node.
1
2
Example: Post-Order Traversal
●
478529631
4
3
6
5
7
8
9
What Good Are All These Traversals?
Example: Preorder Traversal for printing directory listing:
sc2APM
directOverlay
directIO
DXHookD3D11.cs
directO.suo
Injector.cs
Where do we go from here?
●
●
●
BFS: breadth first search
Graphs
Graph algorithms
notes
directO.sln
python
printAPM.py
Disjoint Sets
What problems are we curious about?
●
●
●
●
Dynamic Connectivity and the Disjoint Sets Problem
Quick Find
Quick Union
Weighted Quick Union
Meta-goals of the Coming Lectures
Data Structure Refinement:
Next couple of weeks: Deriving classic solutions to interesting problems, with
an emphasis on how sets, maps, and priority queues are implemented.
Today: Deriving the “Disjoint Sets” data structure for solving the “Dynamic
Connectivity” problem. We will see:
How a data structure design can evolve from basic to sophisticated.
How our choice of underlying abstraction can affect asymptotic runtime (using
our formal Big-Theta notation) and code complexity.
The Disjoint Sets Data Structure
The Disjoint Sets data structure has two operations:
●
connect(x, y): Connects x and y.
●
isConnected(x, y): Returns true if x and y are connected. Connections can
be transitive, i.e. they don’t need to be direct.
Example:
●
●
●
●
●
●
●
connect(China, Vietnam)
connect(China, Mongolia)
isConnected(Vietnam, Mongolia)? true
connect(USA, Canada)
isConnected(USA, Mongolia)? false
connect(China, USA)
isConnected(USA, Mongolia)? true
Hype
rloop
The Disjoint Sets Data Structure
The Disjoint Sets data structure has two operations:
●
connect(x, y): Connects x and y.
isConnected(x, y): Returns true if x and y are connected. Connections can
be transitive, i.e. they don’t need to be direct.
Useful for many purposes, e.g.:
●
●
Percolation theory:
○
●
Computational chemistry.
Implementation of other algorithms:
○
Kruskal’s algorithm.
Hype
rloop
Disjoint Sets on Integers
To keep things simple, we’re going to:
●
Force all items to be integers instead of arbitrary data (e.g. 8 instead of
USA).
●
Declare the number of items in advance, everything is disconnected at
start.
ds = DisjointSets(7)
ds.connect(0, 1)
ds.connect(1, 2)
ds.connect(0, 4)
ds.connect(3, 5)
ds.isConnected(2, 4): true
ds.isConnected(3, 0): false
0
4
1
2
3
5
6
Disjoint Sets on Integers
To keep things simple, we’re going to:
●
Force all items to be integers instead of arbitrary data (e.g. 8 instead of
USA).
●
Declare the number of items in advance, everything is disconnected at
start.
0
1
2
3
ds = DisjointSets(7)
ds.connect(0, 1)
ds.connect(1, 2)
ds.connect(0, 4)
ds.connect(3, 5)
ds.isConnected(2, 4): true
ds.isConnected(3, 0): false
ds.connect(4, 2)
4
5
6
Disjoint Sets on Integers
To keep things simple, we’re going to:
●
Force all items to be integers instead of arbitrary data (e.g. 8 instead of
USA).
●
Declare the number of items in advance, everything is disconnected at
start.
ds = DisjointSets(7)
ds.connect(0, 1)
ds.connect(1, 2)
ds.connect(0, 4)
ds.connect(3, 5)
ds.isConnected(2, 4): true
ds.isConnected(3, 0): false
ds.connect(4, 2)
ds.connect(4, 6)
0
1
2
4
3
5
6
Disjoint Sets on Integers
To keep things simple, we’re going to:
●
Force all items to be integers instead of arbitrary data (e.g. 8 instead of
USA).
●
Declare the number of items in advance, everything is disconnected at
start. ds = DisjointSets(7)
ds.connect(0, 1)
ds.connect(1, 2)
ds.connect(0, 4)
ds.connect(3, 5)
ds.isConnected(2, 4): true
ds.isConnected(3, 0): false
ds.connect(4, 2)
ds.connect(4, 6)
ds.connect(3, 6)
0
4
1
2
3
5
6
Disjoint Sets on Integers
●
●
Force all items to be integers instead of arbitrary data (e.g. 8 instead of
USA).
Declare the number of items in advance, everything is disconnected at
start.
ds = DisjointSets(7)
ds.connect(0, 1)
ds.connect(1, 2)
ds.connect(0, 4)
ds.connect(3, 5)
ds.isConnected(2, 4): true
ds.isConnected(3, 0): false
ds.connect(4, 2)
ds.connect(4, 6)
ds.connect(3, 6)
ds.isConnected(3, 0): true
The Disjoint Sets Interface
0
1
2
4
3
5
6
getBack()
connect(int p, int q)
deleteBack()
isConnected(int
p, int q)
get(int
i)
public interface DisjointSets {
/** Connects two items P and Q. */
void connect(int p, int q);
/** Checks to see if two items are connected. */
boolean isConnected(int p, int q);
}
Goal: Design an efficient DisjointSets implementation.
● Number of elements N can be huge.
● Number of method calls M can be huge.
● Calls to methods may be interspersed (e.g. can’t assume it’s only connect
operations followed by only isConnected operations).
The Naive Approach
Naive approach:
●
●
Connecting two things: Record every single connecting line in some data
structure.
Checking connectedness: Do some sort of (??) iteration over the lines to
see if one thing can be reached from the other.
0
4
1
2
3
6
5
A Better Approach: Connected Components
Rather than manually writing out every single connecting line, only record
the sets that each item belongs to.
{0}, {1}, {2}, {3}, {4}, {5}, {6}
{0, 1}, {2}, {3}, {4}, {5}, {6}
{0, 1, 2}, {3}, {4}, {5}, {6}
{0, 1, 2, 4}, {3}, {5}, {6}
{0, 1, 2, 4}, {3, 5}, {6}
connect(0, 1)
connect(1, 2)
connect(0, 4)
connect(3, 5)
isConnected(2, 4): true
isConnected(3, 0): false
{0, 1, 2, 4}, {3, 5}, {6}
connect(4, 2)
{0, 1, 2, 4, 6}, {3, 5}
connect(4, 6)
{0, 1, 2, 3, 4, 5, 6}
connect(3, 6)
isConnected(3, 0): true
A Better Approach: Connected Components
For each item, its connected component is the set of all items that are
connected to that item.
●
●
Naive approach: Record every single connecting line somehow.
Better approach: Model connectedness in terms of sets.
How things are connected isn’t something we need to know.
Only need to keep track of which connected component each item belongs to.
○
○
0
4
1
2
3
5
{ 0, 1, 2, 4 }, {3, 5}, {6}
Up next: We’ll consider how to do track set membership in Java.
Quick Find
6
Challenge: Pick Data Structures to Support Tracking of Sets
Before connect(2, 3) operation:
0
1
2
4
3
After connect(2, 3) operation:
0
6
1
2
4
5
{ 0, 1, 2, 4 }, {3, 5}, {6}
3
6
5
{ 0, 1, 2, 4, 3, 5}, {6}
Assume elements are numbered from 0 to N-1.
Challenge: Pick Data Structures to Support Tracking of Sets
Before connect(2, 3) operation:
0
4
1
2
3
After connect(2, 3) operation:
6
5
{ 0, 1, 2, 4 }, {3, 5}, {6}
0
1
2
4
5
{ 0, 1, 2, 4, 3, 5}, {6}
Map<Integer, Integer> -- first number represents set and
second represents item
● Slow because you have to iterate to find which set
something belongs to.
Assume elements are numbered from 0 to N-1.
3
6
Challenge: Pick Data Structures to Support Tracking of Sets
Before connect(2, 3) operation:
0
1
2
4
3
After connect(2, 3) operation:
0
6
1
2
4
5
{ 0, 1, 2, 4 }, {3, 5}, {6}
3
6
5
{ 0, 1, 2, 4, 3, 5}, {6}
Map<Integer, Integer> -- first number represents the item, and
the second is the set number.
● More or less what we get to shortly, but less efficient for
reasons I will explain.
Assume elements are numbered from 0 to N-1.
Challenge: Pick Data Structures to Support Tracking of Sets
Before connect(2, 3) operation:
0
4
1
2
3
After connect(2, 3) operation:
6
5
0
4
1
2
3
5
{ 0, 1, 2, 4 }, {3, 5}, {6}
{ 0, 1, 2, 4, 3, 5}, {6}
Idea #1: List of sets of integers, e.g. [{0, 1, 2, 4}, {3, 5}, {6}]
● In Java: List<Set<Integer>>.
● Very intuitive idea.
6
Challenge: Pick Data Structures to Support Tracking of Sets
If nothing is connected:
0
1
2
3
4
5
6
Idea #1: List of sets of integers, e.g. [{0}, {1}, {2}, {3}, {4}, {5}, {6}]
● In Java: List<Set<Integer>>.
● Very intuitive idea.
● Requires iterating through all the sets to find anything. Complicated and slow!
○ Worst case: If nothing is connected, then isConnected(5, 6) requires
iterating through N-1 sets to find 5, then N sets to find 6. Overall runtime
of Θ(N).
Performance Summary
Implementation
constructor
connect
isConnected
ListOfSetsDS
Θ(N)
O(N)
O(N)
Constructor’s runtime has order of
growth N no matter what, so Θ(N).
Worst case is Θ(N), but other
cases may be better. We’ll say
O(N) since O means “less than
or equal”.
ListOfSetsDS is complicated and slow.
● Operations are linear when number of connections are small.
○ Have to iterate over all sets.
● Important point: By deciding to use a List of Sets, we have doomed
ourselves to complexity and bad performance.
My Approach: Just Use a Array of Integers
Before connect(2, 3) operation:
0
1
2
After connect(2, 3) operation:
3
4
6
{ 0, 1, 2, 4 }, {3, 5}, {6}
4 4 4 5 4 5 6
0
1
2
3
1
2
3
4
5
int[] id
0
4
5
6
5
{ 0, 1, 2, 4, 3, 5}, {6}
int[] id
6
5 5 5 5 5 5 6
0
1
2
3
4
5
6
Idea #2: list of integers where ith entry gives set number (a.k.a. “id”) of item i.
●
connect(p, q): Change entries that equal id[p] to id[q]
QuickFindDS
public class QuickFindDS implements DisjointSets {
private int[] id;
Very fast: Two array accesses: Θ(1)
public boolean isConnected(int p, int q) {
return id[p] == id[q];
}
Relatively slow: N+2 to 2N+2 array accesses: Θ(N)
public void connect(int p, int q) {
int pid = id[p];
public QuickFindDS(int N) {
int qid = id[q];
id = new int[N];
for (int i = 0; i < id.length; i++) {
for (int i = 0; i < N; i++)
if (id[i] == pid) {
id[i] = i;
id[i] = qid;
}
}
}
}...
Performance Summary
Implementation
constructor
connect
isConnected
ListOfSetsDS
Θ(N)
O(N)
O(N)
QuickFindDS
Θ(N)
Θ(N)
Θ(1)
QuickFindDS is too slow for practical use: Connecting two items takes N time.
● Instead, let’s try something more radical.
Quick Union
Improving the Connect Operation
Approach zero: Represent everything as boxes and lines. Overly complicated.
0
1
2
3
6
??? in Java instance variables.
4
5
ListOfSets: Represent everything as connected components. Represented
connected components as list of sets of integers.
{ 0, 1, 2, 4 }, {3, 5}, {6}
[{0, 1, 2, 4}, {3, 5}, {6}]
List<Set<Integer>>
QuickFind: Represent everything as connected components. Represented
connected components as a list of integers, where value = id.
{ 0, 1, 2, 4 }, {3, 5}, {6}
[2, 2, 2, 3, 2, 3, 6]
int[]
Improving the Connect Operation
QuickFind: Represent everything as connected components. Represented
connected components as a list of integers where value = id.
● Bad feature: Connecting two sets is slow!
{ 0, 1, 2, 4 }, {3, 5}, {6}
[2, 2, 2, 3, 2, 3, 6]
int[]
Next approach (QuickUnion): We will still represent everything as connected
components, and we will still represent connected components as a list of
integers. However, values will be chosen so that connect is fast.
Improving the Connect Operation
Hard question: How could we change our set representation so that combining
two sets into their union requires changing one value?
0
1
2
3
4
0
6
{ 0, 1, 2, 4 }, {3, 5}, {6}
1
2
3
3
4
5
6
5
{ 0, 1, 2, 4, 3, 5}, {6}
0 0 0 3 0 3 6
0
2
4
5
id
1
3 3 3 3 3 3 6
id
6
0
1
2
3
4
5
6
Improving the Connect Operation
Hard question: How could we change our set representation so that combining
two sets into their union requires changing one value?
●
Idea: Assign each item a parent (instead of an id). Results in a tree-like
shape.
parent
-1
0
1
-1
0
3
-1
0
1
2
3
4
5
6
○
An innocuous sounding, seemingly arbitrary solution.
○
Unlocks a pretty amazing universe of math that we won’t discuss.
Note: The optional textbook
has an item’s parent as itself
instead of -1 for root items.
0
0 is the “root” of
this set.
6
3
1
5
4
{0, 1, 2, 4}
2
{3, 5}
{6}
Improving the Connect Operation
connect(5, 2)
●
How should we change the parent list to handle this connect operation?
○
If you’re not sure where to start, consider: why can’t we just set id[5] to 2?
parent
-1
0
1
-1
0
3
-1
0
1
2
3
4
5
6
0
0 is the “root” of
this set.
4
6
3
1
5
2
{0, 1, 2, 4}
{3, 5}
{6}
Improving the Connect Operation (Your Answer)
connect(5, 2)
●
One possibility, set id[3] = 2
●
Set id[3] = 0
parent
-1
0
1
2
0
2
-1
0
1
2
3
4
5
6
0
0 is the “root” of
this set.
4
2
{0, 1, 2, 4}
6
3
1
5
{3, 5}
{6}
Improving the Connect Operation
connect(5, 2)
●
Find root(5). // returns 3
●
Find root(2). // returns 0
●
Set root(5)’s value equal to root(2).
parent
-1
0
1
-1
0
3
-1
0
1
2
3
4
5
6
0
0 is the “root” of
this set.
4
6
3
1
5
2
{0, 1, 2, 4}
{3, 5}
{6}
Improving
the
Connect
Operation
connect(5, 2)
●
●
●
Find root(5). // returns 3
Find root(2). // returns 0
Set root(5)’s value equal to root(2).
parent
-1
0
1
0
0
3
-1
0
1
2
3
4
5
6
0
0 is the “root” of
this set.
4
2
{0, 1, 2, 4}
6
3
1
5
{3, 5}
{6}
Set Union Using Rooted-Tree Representation
connect(5, 2)
●
Make root(5) into a child of root(2).
parent
-1
0
1
0
0
3
-1
0
1
2
3
4
5
6
What are the potential performance issues with this approach?
●
Compared to QuickFind, we have to climb up a tree.
0
0 is the “root” of
this set.
4
6
3
1
5
2
{0, 1, 2, 4,
3, 5}
{6}
Set Union Using Rooted-Tree Representation
connect(5, 2)
●
Make root(5) into a child of root(2).
parent
-1
0
1
0
0
3
-1
0
1
2
3
4
5
6
What are the potential performance issues with this approach?
●
Tree can get too tall! root(x) becomes expensive.
0
0 is the “root” of
this set.
4
5
2
{0, 1, 2, 4,
6
3
1
3, 5}
{6}
The Worst Case
If we always connect the first item’s tree below the second item’s tree, we can
end up with a tree of height M after M operations:
0
●
●
●
●
connect(4, 3)
connect(3, 2)
connect(2, 1)
connect(1, 0)
1
2
3
4
For N items, what’s the worst case runtime…
For connect(p, q)?
● For isConnected(p, q)?
●
The Worst Case
If we always connect the first item’s tree below the second item’s tree, we can
end up with a tree of height M after M operations:
●
●
●
●
connect(4, 3)
connect(3, 2)
connect(2, 1)
connect(1, 0)
0
1
2
3
For N items, what’s the worst case runtime…
● For connect(p, q)? Θ(N)
● For isConnected(p, q)? Θ(N)
4
QuickUnionDS
public class QuickUnionDS implements DisjointSets {
private int[] parent;
public QuickUnionDS(int N) {
parent = new int[N];
for (int i = 0; i < N; i++)
{ parent[i] = -1; }
For N items, this means a worst case runtime of Θ(N).
}
private int find(int p) {
int r = p;
while (parent[r] >= 0)
{ r = parent[r]; }
return r;
}
Here the find operation is the same as
the “root(x)” idea we had in earlier slides.
public boolean isConnected(int p, int q) {
return find(p) == find(q);
}
{
int i = find(p);
int j = find(q);
parent[i] = j;
}
Performance Summary
Implementation
Constructor
connect
isConnected
ListOfSetsDS
Θ(N)
O(N)
O(N)
QuickFindDS
Θ(N)
Θ(N)
Θ(1)
QuickUnionDS
Θ(N)
O(N)
O(N)
Using O because runtime can
be between constant and linear.
QuickFindDS defect: QuickFindDS is too slow: Connecting takes Θ(N) time.
QuickUnion defect: Trees can get tall. Results in potentially even worse
performance than QuickFind if tree is imbalanced.
● Observation: Things would be fine if we just kept our tree balanced.
Weighted Quick Union
A Choice of Two Roots
Suppose we are trying to connect(2, 5). We have two choices:
A. Make 5’s root into a child of 2’s root.
B. Make 2’s root into a child of 5’s root.
0
1
Which is the better choice?
4
Height: 2
3
2
5
0
1
4
+
2
3
3
0
5
4
5
Height: 3
1
2
A Choice of Two Roots
Suppose we are trying to connect(2, 5). We have two choices:
A. Make 5’s root into a child of 2’s root.
B. Make 2’s root into a child of 5’s root.
0
1
Which is the better choice?
4
Height: 2
3
2
5
0
1
+
4
3
3
0
5
2
5
4
Height: 3
1
2
A Choice of Two Roots
Suppose we are trying to connect(2, 5). We have two choices:
A. Make 5’s root into a child of 2’s root.
B. Make 2’s root into a child of 5’s root.
0
1
Which is the better choice?
4
0
3
2
2
5
4
3
+
0
1
5
Height: 2
3
4
5
Height: 3
1
2
Improvement #1: Weighted QuickUnion
Modify quick-union to avoid tall trees.
●
●
Track tree size (number of elements).
New rule: Always link root of smaller tree to larger tree.
New rule: If we call connect(3, 8), which entry (or entries) of parent[] changes?
A.
B.
C.
D.
0
parent[3]
parent[0]
parent[8]
parent[6]
parent
1
2
3
4
Note: The rule I picked is based on weight,
not height. We’ll talk about why soon.
-1
0
0
0
0
0
0
6
6
8
0
1
2
3
4
5
6
7
8
9
Implementing WeightedQuickUnion
Minimal changes needed:
●
●
●
Use parent[] array as before.
isConnected(int p, int q) requires no changes.
connect(int p, int q) needs to somehow keep track of sizes.
○
○
See the Disjoint Sets lab for the full details.
Two common approaches:
■ Use values other than -1 in parent array for root nodes to track size.
■
parent
Create a separate size array.
-6
0
0
0
0
0
-4
6
6
8
0
1
2
3
4
5
6
7
8
9
size
10
1 1 1 1 1 4 1 2 1
0
1
2
3
4
5
6
7
8
9
6
5
7
8
9
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
0
1
N
H
1
0
2
1
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
0
2
1
3
N
H
1
0
2
1
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
0
N
H
1
0
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
0
1
2
3
N
H
1
0
2
1
4
2
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
4
0
1
2
N
H
1
0
2
1
4
2
5
3
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
4
0
1
2
3
5
N
H
1
0
2
1
4
2
6
7
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
4
0
1
2
5
3
N
H
1
0
2
1
4
2
6
7
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
0
1
2
4
3
5
6
7
N
H
1
0
2
1
4
2
8
3
Weighted Quick Union Performance
Let’s consider the worst case where the tree height grows as fast as possible.
●
N
H
1
0
2
1
4
2
8
3
16
4
Worst case tree height is Θ(log N).
0
1
2
4
3
5
8
6
7
9
10
12
11
13
14
15
Performance Summary
Implementation
Constructor
connect
isConnected
ListOfSetsDS
Θ(N)
O(N)
O(N)
QuickFindDS
Θ(N)
Θ(N)
Θ(1)
QuickUnionDS
Θ(N)
O(N)
O(N)
WeightedQuickUnionDS
Θ(N)
O(log N)
O(log N)
QuickUnion’s runtimes are O(H), and WeightedQuickUnionDS height is given
by H = O(log N). Therefore connect and isConnected are both O(log N).
By tweaking QuickUnionDS we’ve achieved logarithmic time performance.
●
Fast enough for all practical problems.
Why Weights Instead of Heights?
We used the number of items in a tree to decide upon the root.
●
Why not use the height of the tree?
○
Worst case performance for HeightedQuickUnionDS is asymptotically the same! Both are
Θ(log(N)).
○
Resulting code is more complicated with no performance gain.
0
1
2
3
1
6
4
5
+
7
2
0
3
8
9
(Briefly) Path Compression
4
5
6
7
8
9
What We’ve Achieved
Implementation
Constructor
connect
isConnected
ListOfSetsDS
Θ(N)
O(N)
O(N)
WeightedQuickUnionDS
Θ(N)
O(log N)
O(log N)
Performing M operations on a DisjointSet object with N elements:
●
●
●
For our naive implementation, runtime is O(MN).
For our best implementation, runtime is O(N + M log N).
For N = 109 and M = 109, difference is 30 years vs. 6 seconds.
○
●
Key point: Good data structure unlocks solutions to problems that could otherwise not
be solved!
Good enough for all practical uses, but could we theoretically do better?
Suppose we have a ListOfSetsDS implementation of Disjoint Sets.
Suppose that it has 1000 items, i.e. N = 1000.
Suppose we perform a total of 150 connect operations and 212 isConnected
operations.
●
M = 150 + 212 = 362 operations
So when we say O(NM), we’re saying it’ll take no more than 1000 * 362 units of
time (in some arbitrary unit of time).
●
This is a bit informal. O is really about asymptotics, i.e. behavior for very
large N and M, not specific N and Ms that we pick.
170 Spoiler: Path Compression: A Clever Idea
Below is the topology of the worst case if we use WeightedQuickUnion.
●
Clever idea: When we do isConnected(15, 10), tie all nodes seen to the root.
○
Additional cost is insignificant (same order of growth).
0
1
5
11
2
6
12
7
8
13
3
9
4
10
14
15
Path Compression: Theoretical Performance (Bonus)
Path compression results in a union/connected operations that are very very close to
amortized constant time.
●
●
●
M operations on N nodes is O(N + M lg* N).
A tighter bound: O(N + M α(N)), where α is the inverse Ackermann function.
The inverse ackermann function is less than 5 for all practical inputs!
N
○ See “Efficiency of a Good But Not Linear Set Union Algorithm.”
○ Written by Bob Tarjan while at UC Berkeley in 1975.
1
0
15
11
5
1
12
6
2
7
8
3
9
10
4
α(N)
0
...
1
...
2
...
3
...
4
5
13
14
170 Spoiler: Path Compression: A Clever Idea
Below is the topology of the worst case if we use WeightedQuickUnion
●
Clever idea: When we do isConnected(15, 10), tie all nodes seen to the root.
○
Additional cost is insignificant (same order of growth).
0
1
5
11
6
12
2
7
8
13
3
9
4
10
14
15
Path Compression: Another Clever Idea
Below is the topology of the worst case if we use WeightedQuickUnion
● Clever idea: When we do isConnected(15, 10), tie all nodes seen to the root.
○ Additional cost is insignificant (same order of growth).
0
15
11
5
1
12
6
13
2
7
8
14
3
9
10
4
Path Compression: Another Clever Idea
Draw the tree after we call isConnected(14, 13).
0
15
11
5
1
12
6
2
7
8
13
3
10
4
3
10
4
9
14
Path Compression: Another Clever Idea
Draw the tree after we call isConnected(14, 13).
0
15
11
5
1
12
6
13
2
7
8
14
9
Path Compression: Another Clever Idea
Draw the tree after we call isConnected(14, 13).
0
15
11
5
1
12
6
2
7
8
13
3
10
4
3
10
4
9
14
Path Compression: Another Clever Idea
Draw the tree after we call isConnected(14, 13).
0
15
11
5
12
1
13
6
7
14
8
2
9
170 Spoiler: Path Compression: A Clever Idea
Path compression results in a union/connected operations that are very very
close to amortized constant time (amortized constant means “constant on
average”).
●
●
M operations on N nodes is O(N + M lg* N) - you will see this in CS170.
N
lg* N
lg* is less than 5 for any realistic input.
1
0
0
15
11
5
12
1
6
2
7
8
3
4
10
9
2
13
2
1
4
2
16
3
65536
4
265536
5
16
14
Path Compression: Theoretical Performance (Bonus)
Path compression results in a union/connected operations that are very very close to
amortized constant time.
●
●
●
M operations on N nodes is O(N + M lg* N).
A tighter bound: O(N + M α(N)), where α is the inverse Ackermann function.
The inverse ackermann function is less than 5 for all practical inputs!
N
○ See “Efficiency of a Good But Not Linear Set Union Algorithm.”
○ Written by Bob Tarjan while at UC Berkeley in 1975.
1
0
15
11
5
1
12
6
2
7
8
3
9
10
4
α(N)
0
...
1
...
2
...
3
...
4
5
13
14
A Summary of Our Iterative Design Process
And we’re done! The end result of our iterative design process is the standard way
disjoint sets are implemented today - quick union and path compression.
The ideas that made our implementation efficient:
●
Represent sets as connected components (don’t track individual connections).
○
ListOfSetsDS: Store connected components as a List of Sets (slow, complicated).
○
QuickFindDS: Store connected components as set ids.
○
QuickUnionDS: Store connected components as parent ids.
■
WeightedQuickUnionDS: Also track the size of each set, and use size to decide on new
tree root.
●
WeightedQuickUnionWithPathCompressionDS: On calls to connect and
isConnected, set parent id to the root for all items seen.
Performance Summary
Implementation
Runtime
ListOfSetsDS
O(NM)
QuickFindDS
Θ(NM)
QuickUnionDS
O(NM)
WeightedQuickUnionDS
O(N + M log N)
WeightedQuickUnionDSWithPathCompression
O(N + M α(N))
Runtimes are given assuming:
● We have a DisjointSets object of size N.
● We perform M operations, where an operation is defined as either a call to
connected or isConnected.
Citations
Hilf Sp20 and Hug Fa20 slides :)
The proof of the inverse ackermann runtime for disjoint sets is given here:
http://www.uni-trier.de/fileadmin/fb4/prof/INF/DEA/Uebungen_LVA-Ankuendi
gungen/ws07/KAuD/effi.pdf
as originally proved by Tarjan here at UC Berkeley in 1975.
Are you getting enough sleep?
●
●
●
●
●
●
1 in 3 adults do not get enough sleep.
This leads to long term diseases and lower cognitive function, while tired.
Teen (13–18 years old) should be getting 8–10 hours per 24 hours
Adult (18–60 years old) should be getting 7 or more hours per night
It is important to maintain consistency in a bed time (even on the
weekends) to ensure good sleep quality.
Source: https://www.cdc.gov/sleep/features/getting-enough-sleep.html
Lecture 5
CS61BL Summer 2021
-
GitBugs for debugging
-
Announcements
Monday, July 19th
-
See resources/GitBugs guide
Gitlet due this Friday
Redemption quiz 5 today
Redemption tutoring for quiz
6 today
Redemption quiz 6 tomorrow
Quiz 7 on Wednesday, Quiz 8
on Friday
Lab as normal
Binary Search
Our goal:
-
You have a group of items with some natural ordering (e.g. numbers or
letters). In other words, the items are comparable.
You want to be able to quickly add items to the structure. You also want
to be able to quickly check if your structure contains a given item.
Idea 1: An ordered list, starting at the front
A
B
D
L
M
X
Z
Contains(M)
Contains(G)
Insert(Y)
Idea 1: An ordered list, starting at the front
A
Contains() is in O(n)
Insert() is also in O(n).
Can we do better?
B
D
L
M
X
Z
Idea 2: An ordered list, starting at the middle
A
B
D
L
M
X
Z
Contains(M)
Contains(G)
Insert(Y)
Idea 2: An ordered list, starting at the middle
A
B
D
L
M
X
Z
Because we start at the middle, we can cut out half of the list.
However, O(n/2) is the same as O(n)! So we’re still linear… Can we continue improving?
Trees
Binary Search Trees
-
Last week, we learned what a tree is, in general
Now, let’s take those ideas and look at a specific type of tree: a binary
search tree!
We’ll apply the idea we saw in our first improvement: narrow down your
search and continually cut down the amount of items we have to look at.
Idea 3: A tree!
A
B
D
L
M
X
Z
L
B
X
D
A
M
Z
Idea 3: A tree!
Contains(M)
L
Contains(G)
B
A
X
D
M
Insert(Y)
Z
We’ll formally analyze the runtime of
these operations in a moment.
Binary Search Trees
Invariants:
○ Left child’s value must be less than root’s value
○ Right child’s value must be greater than root’s value
Insert(Item):
○ Start at root. If your item is greater than current node’s item, go right. If it’s
less, go left. Keep doing this until you hit the end!
Contains(Item):
○ Start at root. If your item is greater than current node’s item go right. If it’s
less, go left. Keep going until you find your item that you’re looking for. If you
reach a leaf and haven’t found it, its not present.
Let’s look at some examples
●
●
●
●
Examples
X
2
0
1
1
0
4
2
0
2
0
3
2
0
1
1
1
0
2
0
6
3
1
2
8
5
9
7
Question for the chat: are these both valid trees?
0
1
0
3
1
2
6
3
5
4
Binary Search Trees
● What does inserts runtime depend on?
○ Answers:
● What does contains runtime depend on?
○ Answers:
● Best case? Worst case? Average case?
○ What do these trees look like?
2
4
5
6
Question for the chat: which one would you prefer?
0
1
0
3
1
2
6
3
2
4
6
5
5
4
Tree Height
0
1
-
-
Insert and contains both depend on the height of
the tree. If both trees have N items, how do their
heights compare?
For the spindly tree, we see that each layer has a
single item. Then with N items, we have N layers in
our tree.
3
2
6
5
4
0
1
3
2
4
5
6
Tree Height
For the bushy tree:
- Node 0 is the root a tree with N items. Node 1 is
the root of a subtree with roughly N/2 items. Node
2 is the same.
- For each node, its rooting a subtree that is half the
size of the subtree at the layer before.
- We start with N, and each layer we divide by 2.
Then a node at layer k is the root of a subtree of
size N/2ᵏ.
- How many layers are there until we reach a leaf?
- In other words, how many times do we have to
divide N by 2 until we reach 1 (a leaf is a root of a
subtree with only 1 element)
- N/2ᵏ = 1
- N = 2ᵏ
- k = log₂(N)
Tree Height
-
0
1
3
2
6
5
4
0
1
2
3
4
6
5
0
1
In conclusion, an extremely spindly tree would
have a height bounded by O(N)
A perfectly bushy tree would have a height
bounded by O(logN)
3
2
6
How does this affect the best/worst case analysis of our
methods like insert() and contains()?
5
4
0
1
3
2
4
5
6
Best case analysis: contains()
0
Which of the following describes the best
case?
1
A)
3
B)
2
6
1
5
3
4
C)
0
D)
2
4
5
6
The best case is when the tree only
has one element, and we get Θ(1)
runtime no matter what.
The best case is when the element
we’re looking for is the root, and we
get Θ(1) runtime
The best case is when the element
is not present in the tree, and we
get Θ(1) runtime.
The best case is when the element
is not present in the tree, and we
get Θ((log(N)) runtime for the tree
on the right, and Θ(N) for the tree
on the left.
Worst case analysis: contains()
0
What describes the worst case?
1
A)
3
B)
2
0
6
1
5
4
3
C)
2
4
5
6
The worst case is when the element
is not present, and we have to look
at every node in the tree to see if its
the one we want. This is Θ(n) for
both trees.
The worst case is when there are a
LOT of elements in the tree, so it
takes longer to look through them.
The worst case is when the element
is not present, and we have to
traverse the entire height of the
tree. This is Θ(N) for the left tree,
and Θ(logN) for the right.
..
.
... oops
What’s the fundamental issue?
● Binary trees are “lazy” about where to insert data
○ The shape of the tree is highly dependent on the order
of insertions.
● Why don’t we just randomize before we insert?
○ Get Θ(log(n)) height!! Great! Right?
○ We often do not have all the data before starting the
program.
● How can we do better?
Self Balancing Growth
B-Trees
● Solution attempt number one:
○ Overstuff the nodes! (to a certain extent)
○ The tree can never be imbalanced
7
2
5
7
10
5
8
12
7
3
2
10
3
8
10
3
12 13
2
5
8
12 13 15
B-Trees
B-Trees are trees that can have more than one item per node (and more
than two children per node). They are most popular in two specific contexts:
Order refers to the maximum number of children
● Small order (3 or 4):
a single node can have.
○ Used as a conceptually simple balanced search tree (as today).
● Very large order (say thousands)
○ Used in practice for databases and filesystems (i.e. systems with
very large records).
■ See CS186
Note: Since the small order tree isn’t practically useful (as we’ll see in a few
slides), people usually only use the name B-tree to refer to the very large
order case.
2-3 Trees
● A B-Tree of order 3 is called a 2-3 tree
● The name comes from the invariant that a single node can
have 2 or 3 children
○ Each node can have 1-2 entries
● Every non-leaf node must have one more child than it has
entries
The origin of "B-tree" has never been explained by the authors. As we shall
see, "balanced," "broad," or "bushy" might apply. Others suggest that the
"B" stands for Boeing. Because of his contributions, however, it seems
appropriate to think of B-trees as "Bayer"-trees.
- Douglas Corner (The Ubiquitous B-Tree)
2-3 Tree Structure
20
10 1018
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: get
● V get(K k):
○ At each node, check for a key matching k.
○ If not found, move down to the appropriate child by comparing k
against the keys in the node.
○ Continue until k is found, or at a leaf not containing k.
get(3)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: get
● V get(K k):
○ At each node, check for a key matching k.
○ If not found, move down to the appropriate child by comparing k
against the keys in the node.
○ Continue until k is found, or at a leaf not containing k.
get(3)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: get
● V get(K k):
○ At each node, check for a key matching k.
○ If not found, move down to the appropriate child by
comparing k against the keys in the node.
○ Continue until k is found, or at a leaf not containing k.
get(3)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: get
● V get(K k):
○ At each node, check for a key matching k.
○ If not found, move down to the appropriate child by comparing k
against the keys in the node.
○ Continue until k is found, or at a leaf not containing k.
get(3)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: get
● V get(K k):
○ At each node, check for a key matching k.
○ If not found, move down to the appropriate child by comparing k
against the keys in the node.
○ Continue until k is found, or at a leaf not containing k.
get(3)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
10 10 18
1
3
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
40
We’ve reached a leaf
node. Add the item here!
10 10 18
1
3
17
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
10 10 18
1
3
8
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by
sending the middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
40
This node has too many
items. Let’s split the node!
10 10 18
1
8
3
17
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by
sending the middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
3
1
10 10 18
8
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
20
40
This node has too many
items. Let’s split the node!
3
10 10 18
8
1
17
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
10
3
1
20
18
8
17
40
55
26
19
22
39
41
66
2-3 Tree Operations: add
● add(K k, V v): (inserting key value pair)
○ Walk down the tree like we’re doing get(k).
○ When reaching a leaf, add the key to its ordered position.
○ If the resulting leaf node has 3 entries, “split” the node by sending the
middle element up.
○ Repeat if this creates more 3-entry-nodes.
insert(8)
10
20
40
This node has too many
items. Let’s split the node!
3
18
8
1
17
55
26
19
22
39
41
66
2-3 Tree Operations: add
insert(8)
20
Balanced! :-)
10
3
1
40
18
8
17
55
26
19
22
39
41
66
Perfect Balance
Observation: 2-3 Trees have perfect balance.
● If we split the root, every node gets pushed down by exactly one level.
● If we split a leaf node or internal node, the height doesn’t change.
● Height increases only if the root is split
We will never get unbalanced trees that
look like the ones below! We don’t have
time to go over why, but I suggest you
try creating some imbalanced trees on
your own and see what they end up
looking like.
7
2
1
10 13
5
8
12
G
15 17
AB
G
X
B
C
A
X
C
2-3 Tree Runtimes
● Because of this balance, the height H of a 2-3 Tree is between
log3(N) and log2(N)
● Get: Traverse down the tree to find item. Worst case: Θ(H) or Θ(log
N)
● Insert: Traverse down the tree and put the item in the correct leaf.
Max number of times we split nodes per insert is ~H, so worst case
is still Θ(2H) or Θ(log N)
● Runtimes are pretty good!
7 13
10
3
Max 2 items per node.
Max 3 non-null children per node.
2
5
8
11
16 20
15
17
21
But there’s a slight problem...
●
Though the runtimes are good, 2-3 Trees can be a real pain to implement
○
●
The remove() operation (omitted) can be hard to code
“Beautiful algorithms are, unfortunately, not always the most useful” Knuth
Lets look at some other alternatives.
BST Structure and
Tree Rotation
BSTs
Suppose we have a BST with the numbers 1, 2, 3. There are five possible BSTs.
● The specific BST you get is based on the insertion order.
● More generally, for N items, there are Catalan(N) different BSTs.
1
1
3
3
2
2
3
1
1
3
2
3
2
2
1
Given any BST, it is possible to move to a different configuration using
“rotation”.
Tree Rotation Definition
rotateLeft(G): Let x be the right child of G. Make G the new left child of x.
● Preserves search tree property. No change to semantics of tree.
● Can think of as temporarily merging G and P, then sending G down
and left.
G
C
P
A
k
B
P
GP
j
r
l
G
C
A
k
B
j
C
r
l
For this example rotateLeft(G) increased height of the tree!
r
A
k
j
B
l
Your Turn
rotateRight(P): Let x be the left child of P. Make P the new right child of x.
● Can think of as temporarily merging G and P, then sending P down
and right.
P
G
r
C
k
A
j
l
B
Your Turn
rotateRight(P): Let x be the left child of P. Make P the new right child of x.
● Can think of as temporarily merging G and P, then sending P down
and right.
● Note: k was G’s right child. Now it is P’s left child.
P
G
G
r
C
A
k
j
C
P
A
l
k
B
j
r
l
B
For this example rotateRight(P) decreased height of tree!
Rotation for Balance
Rotation:
● Can shorten (or lengthen) a tree.
● Preserves search tree property.
D
B
rotateRight(D)
B
D
>D
<B
<B
rotateLeft(B)
> B and < D
> B and < D
>D
Rotation for Balance
Rotation:
● Can shorten (or lengthen) a tree.
● Preserves search tree property.
D
B
rotateRight(D)
B
D
>D
<B
<B
> B and < D
rotateLeft(B)
> B and < D
Can use rotation to balance a BST: Demo
● Rotation allows balancing of a BST in O(N) moves.
>D
Rotation for Balance
Rotation:
● Can shorten (or lengthen) a tree.
● Preserves search tree property.
D
B
rotateRight(D)
B
D
>D
<B
<B
> B and < D
rotateLeft(B)
Red-Black Trees
> B and < D
>D
Search Trees
There are many types of search trees:
● Binary search trees: Can balance using rotation, but we have no
algorithm for doing so (yet).
● 2-3 trees: Balanced by construction, i.e. no rotations required, but very
hard to implement.
Let’s try something clever, but strange.
Our goal: Build a BST that is structurally identical to a 2-3 tree.
● Since 2-3 trees are balanced, so will our special BSTs.
Red-Black trees
● We can use red-black trees to help us more easily implement this
idea of 2-3 and 2-3-4 trees.
● Generally, red-black trees are binary search trees that color edges
red or black in order to provide some form of organization to the
nodes.
○
○
Red: the nodes connected by this edge in our red-black tree are in the same node
in the corresponding B-Tree
Black: regular node
● Red-black trees have a correspondence with 2-3-4 trees, while
left-leaning red-black trees have a 1-1 correspondence with 2-3 trees.
○
We will be discussing left-leaning red black trees for the remainder of this lecture.
Representing a 2-3 Tree as a BST
A 2-3 tree with only 2-nodes is trivial.
● BST is exactly the same!
m
m
o
e
b
g
n
o
e
p
b
g
n
p
What do we do about 3-nodes?
m
b
e
????
o
df
g
n
p
Dealing with 3-Nodes
Possibility 1: Create dummy “glue” nodes.
m
m
o
o
df
d
b
e
g
n
f
n
p
b
e
g
Result is inelegant. Wasted link. Code will be ugly.
df
d
f
p
Dealing with 3-Nodes
Possibility 2: Create “glue” links with the smaller item off to the left.
m
m
b
e
f
o
df
g
n
p
o
d
b
n
g
e
Idea is commonly used in practice (e.g. java.util.TreeSet).
f
df
d
For convenience, we’ll mark glue links as “red”.
Left-Leaning Red Black Binary Search Tree (LLRB)
A BST with left glue links that represents a 2-3 tree is often called a “Left
Leaning Red Black Binary Search Tree” or LLRB.
●
●
●
LLRBs are normal BSTs!
There is a 1-1 correspondence between an LLRB and an equivalent 2-3 tree.
The red is just a convenient fiction. Red links don’t “do” anything special.
p
Left-Leaning Red Black Binary Search Tree (LLRB)
Draw the LLRB corresponding to the 2-3 tree shown below.
uw
as
v
xy
Left-Leaning Red Black Binary Search Tree (LLRB)
Draw the LLRB corresponding to the 2-3 tree shown below.
w
uw
u
as
v
y
xy
s
a
v
x
Left-Leaning Red Black Binary Search Tree (LLRB)
Draw the LLRB corresponding to the 2-3 tree shown below.
w
uw
u
as
y
xy
v
s
v
x
a
Searching an LLRB tree for a
key is easier than a 2-3 tree.
● Treat it exactly like any
BST.
LLRB Height
Suppose we have a 2-3 tree of height H.
● What is the maximum height of the corresponding LLRB?
L
DE
B
A
P
G
C
F
J
H
I
SU
N
K
M O
QR
T
VW
LLRB Height
Suppose we have a 2-3 tree of height H.
● What is the maximum height of the corresponding LLRB?
○
Total height is H (black) + H + 1 (red) = 2H + 1.
Worst case would be if these
were both 3 nodes.
L
DE
B
A
P
G
C
F
J
H
I
SU
N
K
M O
QR
T
VW
Left-Leaning Red Black Tree (LLRB) Properties
Some handy LLRB properties:
● No node has two red links [otherwise it’d be analogous to a 4 node, which
are disallowed in 2-3 trees].
● Every path from root to null reference has same number of black links
[because 2-3 trees have the same number of links to every leaf]. LLRBs
are therefore balanced.
Left-Leaning Red Black Tree (LLRB) Properties
Some handy LLRB properties:
● No node has two red links [otherwise it’d be analogous to a 4 node, which
are disallowed in 2-3 trees].
● Every path from root to null reference has same number of black links
[because 2-3 trees have the same number of links to every leaf]. LLRBs
are therefore balanced.
G
C
G
G
X
B
B
A
A
Invalid, B has two red links.
B
X
C
Invalid, not
black balanced.
A
G
X
C
Invalid, not
black balanced.
B
A
X
C
Valid
Left-Leaning Red Black Tree (LLRB) Properties
One last important question: Where do LLRBs come from?
● Typically would not make sense to build a 2-3 tree, then convert. Even
more complex.
● Instead, it turns out we implement an LLRB insert as follows:
○
○
Insert as usual into a BST.
Use zero or more rotations to maintain the 1-1 mapping.
Maintaining 1-1
Correspondence Through
Rotations
The 1-1 Mapping
There exists a 1-1 mapping between:
● 2-3 Tree
● LLRB
Implementation of an LLRB is based on maintaining this 1-1 correspondence.
● When performing LLRB operations, pretend like you’re a 2-3 tree.
● Preservation of the correspondence will involve tree rotations.
Design Task #1: Insertion Color
Should we use a red or black link when inserting?
S
(E)
d
ad
E
S
ad
d(E
S
)
E
LLRB World
add(E)
S
ES
World 2-3
Design Task #1: Insertion Color
Should we use a red or black link when inserting?
S
(E)
d
ad
E
S
ad
d(E
S
)
E
LLRB World
S
World 2-3
add(E)
ES
Use red! In 2-3 trees new
values are ALWAYS added to
an existing leaf node (at first).
Design Task #2: Insertion on the Right
Suppose we have leaf E, and insert S with a red link. What is the problem below,
and what do we do about it?
B
A
B
add(S)
E
A
E
S
LLRB World
B
A
B
add(S)
A
E
ES
World 2-3
Design Task #2: Insertion on the Right
Suppose we have leaf E, and insert S with a red link. What is the problem below,
and what do we do about it: Red right links aren’t allowed, so rotateLeft(E).
B
A
B
add(S)
E
rotateLeft(E)
A
E
A
S
LLRB World
B
A
World 2-3
B
add(S)
E
A
B
ES
S
E
New Rule: Representation of Temporary 4-Nodes
We will represent temporary 4-nodes as BST nodes with two red links.
● This state is only temporary (more soon), so temporary violation of
“left leaning” is OK.
B
B
A
Represents
temporary 4 nodes.
Temporarily violates
“no red right links”.
add(Z)
S
A
S
E
E
Z
LLRB World
B
B
add(Z)
A
Temporarily violates
“no 4 nodes”.
A
ES
ESZ
World 2-3
Design Task #3: Double Insertion on the Left
Suppose we have the LLRB below and insert E. We end up with the wrong
representation for our temporary 4 node. What should we do so that the
temporary 4 node has 2 red children (one left, one right) as expected?
B
B
add(E)
A
Z
A
Z
S
S
E
LLRB World
B
A
World 2-3
B
add(E)
SZ
A
ESZ
Design Task #3: Double Insertion on the Left
What should we do so that the temporary 4 node has 2 red children (one left,
one right) as expected: Rotate Z right.
B
B
B
add(E)
A
Z
A
rotateRight(Z)
Z
S
A
S
S
E
Z
E
LLRB World
B
B
add(E)
A
A
SZ
ESZ
World 2-3
Design Task #4: Splitting Temporary 4-Nodes
Suppose we have the LLRB below which includes a temporary 4 node. What
should we do next?
● Try to figure this one out! It’s a particularly interesting puzzle.
G
B
A
X
C
Hint: Ask yourself “What Would
2-3 Tree Do?” WW23TD?
LLRB World
G
World 2-3
ABC
BG
split(A/B/C)
X
A
C
X
Design Task #4: Splitting Temporary 4-Nodes
Suppose we have the LLRB below which includes a temporary 4 node. What
should we do next?
● Flip the colors of all edges touching B.
G
G
flip(B)
B
A
X
B
C
A
X
C
Note: This doesn’t change the
BST structure/shape!
LLRB World
G
ABC
World 2-3
BG
split(A/B/C)
X
A
C
X
… and That’s It!
Congratulations, you just invented the red-black BST.
● When inserting: Use a red link.
● If there is a right leaning “3-node”, we have a Left Leaning Violation.
○
●
If there are two consecutive left links, we have an Incorrect 4 Node
Violation.
○
●
Rotate left the appropriate node to fix.
Rotate right the appropriate node to fix.
If there are any nodes with two red children, we have a Temporary 4 Node.
○
Color flip the node to emulate the split operation.
One last detail: Cascading operations.
● It is possible that a rotation or flip operation will cause an additional
violation that needs fixing.
Optional Exercises
Cascading Balance Example
Inserting Z gives us a temporary 4 node.
● Color flip yields an invalid tree. Why? What’s next?
B
B
A
add(Z)
S
B
A
flip(S)
S
E
E
A
S
Z
E
Z
LLRB World
B
A
World 2-3
B
add(Z)
ES
A
BS
split(E/S/Z)
ESZ
A
E
Z
Cascading Balance Example
Inserting Z gives us a temporary 4 node.
● Color flip yields an invalid tree. Why? What’s next?
● We have a right leaning 3-node (B-S). We can fix with rotateLeft(b).
B
S
rotateLeft(B)
A
S
E
B
Z
A
Z
E
LLRB World
BS
World 2-3
A
E
Z
Insertion of 7 through 1
To get an intuitive understanding of why all this works, try inserting the 7, 6,
5, 4, 3, 2, 1, into an initially empty LLRB.
● You should end up with a perfectly balanced BST!
To check your work, see this demo.
4
2
1
6
3
5
7
LLRB Runtime and
Implementation
LLRB Runtime
The runtime analysis for LLRBs is simple if you trust the 2-3 tree runtime.
● LLRB tree has height O(log N).
● Contains is trivially O(log N).
● Insert is O(log N).
○
○
O(log N) to add the new node.
O(log N) rotation and color flip operations per insert.
We will not discuss LLRB delete.
● Not too terrible really, but it’s just not interesting enough to cover. See
optional textbook if you’re curious (though they gloss over it, too).
LLRB Implementation
●
Using what we have learned, turning a BST into an LLRB requires
additional code to handle the rotations and color flips.
This is exactly what you have done in Lab!
○
●
One other note on red links vs. red nodes:
In this lecture we used the idea of a red link which glued nodes together.
However in lab we used red nodes and no colored links.
In implementation it is relatively easy to add coloring to each node, and relatively more
complicated to add coloring to the links.
As such we will typically implement the version with red nodes.
We can think of all of the following trees as equivalent.
○
○
○
○
○
S
S
BS
B
A
E
2-3 Tree
Z
A
B
Z
E
LLRB w/ red links
A
Z
E
LLRB w/ red nodes
Multidimensional Data
Yet Another Type of Map/Set
We’ve touched on two fairly general implementations of sets and maps (we
still have not discussed how Hash Tables work, that will be next week!):
● Hash Table: Requires that keys can be hashed.
● Search Tree: Requires that keys can be compared.
Both of these implementations require that data is one-dimensional.
Today, we’ll discuss one very important special case: Multi-dimensional keys.
One-dimensional Data
So far our data has been one dimensional. Some examples:
● Integers: 1 is smaller than 3; 5 is bigger than 2
● Doubles: 1.0 is smaller than 3.0; 5.0 is bigger than 2.0
● Strings: “apple” is “smaller” than “cat”, “dog” is bigger than “banana”
There is a single
axis, and there is a
clear interpretation
of which of two
given elements is
bigger.
Two-dimensional Data
What if our data lies in two dimensions?
● (Integer, Integer) Points: e.g. (1,2) or (0,4)
● (Double, Double) Points: e.g. (1.0, 2.0) or (0.0, 4.0)
● (String, String) tuples: e.g. (“apple”, “cat”) or (“dog”, “banana”)
There are now two
axes that our data
lies on, the x-axis
and the y-axis.
Two-dimensional Data
Out of the points (1,2) and (0,4) which is “bigger”? Why?
Two-dimensional Data
Out of the points (1,2) and (0,4) which is “bigger”? Why?
We could impose some
kind of relative ordering on
these points, but inherently
by saying one point is
larger than the other we
are compressing the data
into one-dimension.
For multidimensional data
the question of which is
items are “bigger” is a
somewhat flawed question.
Two-dimensional Data
Out of the points (1,2) and (0,4) which is “bigger”? Why?
We could impose some
kind of relative ordering on
these points, but inherently
by saying one point is
larger than the other we
are compressing the data
into one-dimension.
For multidimensional data
the question of which is
items are “bigger” is a
somewhat flawed question.
(0, 4)
(1, 2)
(1, 2)
X-Based Tree
(0, 4)
Y-Based Tree
Suppose we chose the X-based tree, what
happens if we want to look up a point
based on it’s Y-value? It could be anywhere!
k-dimensional Data
What if our data lies in k dimensions?
● (Integer, Integer, . . . ,Integer) Points: e.g. (1,2,...,0) or (0,4,...,8)
● (Double, Double, . . . , Double) Points: e.g. (1.0,2.0,...,0.0) or (0.0,4.0,...,8.0)
● (String, String, . . . ,String) tuples: e.g. (“apple”, “cat”,...”monkey”) or (“dog”,
“banana”,...,”orange”)
With k-dimensions we have the same problems!
● In this class we will only deal with one-dimensional data and
two-dimensional data, but it can be useful to consider the generalization
of this to higher k-dimensional data.
k-d Trees
Binary Search Trees
BSTs had some nice properties:
● The ability to compare one-dimensional data is leveraged organize items.
● This organization can lead to logarithmic operations for checking if an item
exists in the tree or adding an element to the tree.
○
To guarantee this we need some form of balancing.
We’ll exploit similar properties to organize spatial (multidimensional) data. This
will allow us to:
● Have similarly fast add and contains operations
○
●
Again we need to have some form of guarantee that our tree is balanced.
Define new operations like nearest which make more sense for spatial data.
○
○
More on nearest later.
Nearest will be one of the key reasons we will study and implement k-d trees.
Key Insight
The multidimensional data that we have is comprised of elements which can be
compared on one dimension! We saw this before:
(0, 4)
(1, 2)
(1, 2)
X-Based Tree
(0, 4)
Y-Based Tree
Instead of choosing to use strictly an X-based approach or a Y-based approach,
we can combine both of them.
k-d Tree: An X and Y Based Tree
Instead of choosing to use strictly an X-based approach or a Y-based approach,
we can combine both of them.
●
●
We switch between comparing on X values and comparing on Y values.
At each level we apply the same binary search tree principal for the
dimension we are comparing on.
k-d Tree: An X and Y Based Tree
It can be helpful to visualize the data both as a tree, and also spatially on the
coordinate plane.
●
●
●
The red lines correspond to when we split the space into two spaces by x.
The blue lines correspond to when we split the space into two spaces by y.
As we proceed further down the tree spaces get even further divided into
pieces.
Exercise: Where to look
If we have the following k-d tree, which nodes should we consider when looking
for the point (2,3)? Can we skip any like with a BST?
Exercise: Where to look
If we have the following k-d tree, which nodes should we consider when looking
for the point (2,3)? Can we skip any like with a BST?
Yes! We can skip the nodes above which have been struck out. We should
exclude them for the same ways that we will exclude nodes in a BST.
k-d Trees: Operations
k-d Trees Insertion / Construction
Let’s see an example of how to build a k-d tree.
●
k-d tree insertion demo.
(4, 5)
(1, 5)
C
E
(3, 3)
A
F
(4, 4)
D
(2, 3)
B
(4, 2)
K-d Trees and Nearest Neighbor
k-d trees support an efficient nearest method.
● Optimization: Do not explore subspaces that can’t possible have a better
answer than your current best.
Example: Find the nearest point to (0, 7).
● Answer is (1, 5).
● k-d tree nearest demo.
(4, 5)
(1, 5)
(0, 7)
C
E
(3, 3)
A
F
(4, 4)
D
(2, 3)
B
(4, 2)
Nearest Pseudocode
nearest(Node n, Point goal, Node best):
● If n is null, return best
● If n.distance(goal) < best.distance(goal), best = n
● If goal < n (according to n’s comparator):
○
●
●
goodSide = n.”left”Child
badSide = n.”right”Child
goodSide = n.”right”Child
badSide = n.”left”Child
best = nearest(goodSide, goal, best)
If bad side could still have something useful
○
●
■
■
else:
■
■
Nearest is a helper method that returns
whichever is closer to goal out of the
following two choices:
1. best
2. all items in the subtree starting at n
best = nearest(badSide, goal, best)
return best
This is our pruning rule.
Inefficient Nearest Pseudocode
nearest(Node n, Point goal, Node best):
● If n is null, return best
● If n.distance(goal) < best.distance(goal), best = n
● best = nearest(n.leftChild, goal, best)
● best = nearest(n.rightChild, goal, best)
● return best
Here we do no pruning so
we will search the whole
tree.
Consider implementing this inefficient version first.
● Easy to implement.
● Once you’ve verified this is working, you can implement the more efficient
version on the previous slide.
○
The key difference is that you would have to add pruning.
Today’s PSA
● ALWAYS backup your files.
● Why?
○ Theft
○ Water Accident (maybe don’t watch netflix in the bathtub)
○ rm -rf ~ (Do Not try this at home)
○ Computer cracks in half/corrupted/etc
● There are many methods to do this, some free and some paid
○ Google Drive/GitHub/Bitbucket (and other free file storage)
○ Physical Hard Drive (one time purchase)
○ Online backup (subscription based)
Lecture 6: Heaps
and Graphs
CS 61BL Summer 2021
●
BYOW released today
○
○
●
●
Announcements
Monday, July 26
●
●
●
Do Lab 16 (tomorrow’s lab)
before starting
Two AG phases, see spec
Midterm 2 today
No redemption quiz or
tutoring for quiz 8
Quiz 9 on Wednesday, Quiz
10 on Friday
Lab as normal
DO NOT discuss the exam on
Ed until we release you.
○
This would be considered
academic dishonesty. See the
syllabus for more
Priority Queue
The Priority Queue Interface
/** (Min) Priority Queue: Allowing tracking and removal of the
* smallest item in a priority queue. */
public interface MinPQ<Item> {
/** Adds the item to the priority queue. */
public void add(Item x);
/** Returns the smallest item in the priority queue. */
public Item getSmallest();
/** Removes the smallest item from the priority queue. */
public Item removeSmallest();
/** Returns the size of the priority queue. */
public int size();
}
Useful if you want to keep track of the “smallest”, “largest”, “best” etc. seen so far.
Usage Example: Unharmonious Texts
Suppose you want to monitor the text messages of the citizens to make sure
that they are not having any unharmonious conversations.
Each day, you create a list of the M messages that seem most unharmonious
using the HarmoniousnessComparator.
Naive approach: Create a list of all messages sent for the entire day. Sort it
using your comparator. Return the M messages that are largest.
Naive Implementation: Store and Sort
public List<String> unharmoniousTexts(Sniffer sniffer, int M) {
ArrayList<String> allMessages = new ArrayList<String>();
for (Timer timer = new Timer(); timer.hours() < 24; ) {
allMessages.add(sniffer.getNextMessage());
}
Comparator<String> cmptr = new HarmoniousnessComparator();
Collections.sort(allMessages, cmptr, Collections.reverseOrder());
return allMessages.sublist(0, M);
}
Potentially uses a huge amount of memory Θ(N), where N is number of texts.
● Goal: Do this in Θ(M) memory using a MinPQ.
MinPQ<String> unharmoniousTexts = new HeapMinPQ<Transaction>(cmptr);
Better Implementation: Track the M Best
public List<String> unharmoniousTexts(Sniffer sniffer, int M) {
Comparator<String> cmptr = new HarmoniousnessComparator();
MinPQ<String> unharmoniousTexts = new HeapMinPQ<Transaction>(cmptr);
for (Timer timer = new Timer(); timer.hours() < 24; ) {
unharmoniousTexts.add(sniffer.getNextMessage());
if (unharmoniousTexts.size() > M)
{ unharmoniousTexts.removeSmallest(); }
}
ArrayList<String> textlist = new ArrayList<String>();
while (unharmoniousTexts.size() > 0) {
textlist.add(unharmoniousTexts.removeSmallest());
}
return textlist;
}
Can track top M transactions using only M memory. API for MinPQ also makes code
very simple (don’t need to do explicit comparisons).
How Would We Implement a MinPQ?
Some possibilities:
● Ordered Array
● Bushy BST: Maintaining bushiness is annoying. Handling duplicate priorities is
awkward.
● HashTable: No good! Items go into random places.
Ordered Array
Bushy BST
Hash Table
add
Θ(N)
Θ(log N)
Θ(1)
getSmallest
Θ(1)
Θ(log N)
Θ(N)
removeSmallest
Θ(N)
Θ(log N)
Θ(N)
Caveats
Dups tough
Worst Case Θ(·) Runtimes
Heap
Heaps
Introducing the Heap
BSTs would work, but need to be kept bushy and duplicates are awkward.
Binary min-heap: Binary tree that is complete and obeys min-heap property.
●
●
Min-heap: Every node is less than or equal to both of its children.
Complete: Missing items only at the bottom level (if any), all nodes are as far
left as possible.
0
5
8
5
1
8
0
0
0
6
2
8
5
1
8
6
8
0
1
8
Incomplete
5
2
8
1
4
6
Lacks min-heap property
Heap Comprehension Test
Which of these are min heaps?
8
8
8
8
8
8
4
5
2
6
7
0
0
8
7
5
3
1
9
Heap Comprehension Test Solutions
8
8
8
8
8
4
5
8
2
6
7
0
8
Incomplete
0
7
3
5
1
9
Lacks min-heap property
What Good Are Heaps?
Heaps lend themselves very naturally to implementation of a priority queue.
Hopefully easy question:
●
How would you support getSmallest()?
0
5
8
1
8
6
How Do We Add To A Heap?
1
Challenge: Come up with an algorithm for add(x).
●
How would we insert 3?
5
1
6
Runtime must be logarithmic.
7
6
5
7
8
?
Bonus: Come up with an algorithm for removeSmallest().
Solution: See https://goo.gl/wBKdFQ for an animated demo.
3
Heap Operations Summary
Given a heap, how do we implement PQ operations?
●
●
●
getSmallest() - return the item in the root node.
add(x) - place the new employee in the last position, and promote as high
as possible.
removeSmallest() - assassinate the president (of the company), promote
the rightmost person in the company to president. Then demote repeatedly,
always taking the ‘better’ successor.
See https://goo.gl/wBKdFQ for an animated demo.
Remaining question: How would we do all this in Java?
Heap Representations
How should we represent the Heap?
Approach 1:
●
Similar to what we did for BSTs
class Node {
int value; // e.g. 0
Node left;
Node right;
...
0
5
8
1
6
8
How should we represent the Heap?
Approach 1:
●
Similar to what we did for BSTs
class Node {
int value; // e.g. 0
Node left;
Node right;
...
0
5
8
1
8
6
This works but doesn’t leverage the complete nature of the tree...
How should we represent the Heap?
Approach 2: Store keys in an array. Don’t store structure anywhere.
- To interpret array: Simply assume tree is complete.Obviously only works
for “complete” trees.
public class Heap<Key> {
Key[] keys;
...
0
1
3
7
e
2
4 g
b
a
k
d
8
f
9
5 p
j
10
m
11
Key[] keys
v
6
r
12
y
k
e
v
b
g
p
y
a
d
f
j
m r
0
1
2
3
4
5
6
7
8
9
10 11 12 13
x
13
A Deep Look at Approach 2
w
Challenge: Write the parent(k) method for approach 2.
x
public void swim(int k) {
if (keys[parent(k)] ≻ keys[k]) {
swap(k, parent(k));
swim(parent(k));
}
}
0
1
3
7
a
d
8
2
f
9
5 p
j
10
Key[] keys
z
y
w
x
y
z
0
1
2
3
Key[] keys
k
e
4 g
b
x
m
11
v
6
r
12
x
13
y
k
e
v
b
g
p
y
a
d
f
j
m r
0
1
2
3
4
5
6
7
8
9
10 11 12 13
public class Tree<Key> {
Key[] keys;
...
x
A Deep Look at Approach 2
w
Challenge: Write the parent(k) method for approach 2.
x
z
y
public void swim(int k) {
if (keys[parent(k)] ≻ keys[k]) {
Key[] keys w x y z
swap(k, parent(k));
0 1 2 3
swim(parent(k));
public int parent(int k) {
}
return (k - 1) / 2;
}
}
0
1
3
7
e
2
4 g
b
a
Key[] keys
k
d
8
f
9
v
5 p
j
10
m
11
6
r
12
k
e
v
b
g
p
y
a
d
f
j
m r
x
0
1
2
3
4
5
6
7
8
9
10 11 12 13
public class Tree<Key> {
Key[] keys;
...
y
x
13
Slight improvement: Leaving One Empty Spot
Approach 2b: Store keys in an array. Offset everything by 1 spot.
● Same as 2, but leave spot 0 empty.
● Makes computation of children/parents “nicer”.
○
○
○
leftChild(k) = k*2
rightChild(k) = k*2 + 1
parent(k) = k/2
1
2
4
8
a
e
3
5 g
b
d
9
Key[] keys
k
f
10
6 p
j
11
m
12
v
7
r
13
x
14
y
-
k
e
v
b
g
p
y
a
d
f
j
m r
x
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14
Heap Implementation of a Priority Queue
Ordered Array
Bushy BST
Hash Table
Heap
add
Θ(N)
Θ(log N)
Θ(1)
Θ(log N)
getSmallest
Θ(1)
Θ(log N)
Θ(N)
Θ(1)
removeSmallest
Θ(N)
Θ(log N)
Θ(N)
Θ(log N)
Graphs
Graph Definition
A graph consists of:
● A set of vertices.
● A set of zero or more edges, each of which connects two vertices.
Green structures below are graphs.
● Note, all trees are graphs!
Graph Definition
A simple graph is a graph with:
● No edges that connect a vertex to itself, i.e. no “loops”.
● No two edges that connect the same vertices, i.e. no “parallel edges”.
Green graph below is simple, pink graphs are not.
Graph Definition
A simple graph is a graph with:
●
●
No edges that connect a vertex to itself, i.e. no “loops”.
No two edges that connect the same vertices, i.e. no “parallel edges”.
In 61BL, unless otherwise explicitly stated, all graphs will be simple.
●
In other words, when we say “graph”, we mean “simple graph.”
Graph Terminology
●
●
●
●
Graph:
○ Set of vertices, a.k.a. nodes.
○ Set of edges: Pairs of vertices.
○ Vertices with an edge between are
adjacent.
A path is a sequence of vertices connected by
edges.
○ A simple path is a path without repeated
vertices.
A cycle is a path whose first and last vertices are
the same.
○ A graph with a cycle is ‘cyclic’.
Two vertices are connected if there is a path
between them. If all vertices are connected, we
say the graph is connected.
Figure from Algorithms 4th Edition
Graph Types
Undirected
Directed
b
Acyclic:
a
d
a
d
c
c
b
Cyclic:
a
e
b
b
d
c
a
d
c
Graph Example: The Paris Metro
This schematic map of the Paris Metro is a graph:
●
●
●
●
Undirected
Connected
Cyclic (not a tree!)
Vertex-labeled (each has a color).
Directed Graph Example
Edge captures ‘is-a-type-of’ relationship. Example: descent is-a-type-of movement.
Depth-First Traversal
Not a tree!
● Two paths from
group_action
to event.
s-t Connectivity
Let’s solve a classic graph problem called the s-t connectivity problem.
●
Given source vertex s and a target vertex t, is there a path between s and
t?
3
Requires us to traverse the graph somehow.
0
s
1
6
4
2
7
5
t
8
s-t Connectivity
Let’s solve a classic graph problem called the s-t connectivity problem.
●
Given source vertex s and a target vertex t, is there a path between s and
t?
3
Requires us to traverse the graph somehow.
0
s
Come up for an algorithm for connected(s, t)
1
6
4
2
5
7
t
8
s-t Connectivity
One possible recursive algorithm for connected(s, t).
●
●
●
Does s == t? If so, return true.
Otherwise, if connected(v, t) for any neighbor v of s, return true.
Return false.
3
What is wrong with the algorithm above?
0
s
1
6
4
7
5
2
t
8
s-t Connectivity
One possible recursive algorithm for connected(s, t).
● Does s == t? If so, return true.
● Otherwise, if connected(v, t) for any neighbor v of s, return true.
● Return false.
What is wrong with it? Can get caught in an infinite loop. Example:
● connected(0, 7):
3
○
○
●
Does 0 == 7? No, so...
if (connected(1, 7)) return true;
connected(1, 7):
○
○
Does 1 == 7? No, so…
If (connected(0, 7)) … ← Infinite loop.
0
s
1
6
4
2
5
7
t
8
s-t Connectivity
One possible recursive algorithm for connected(s, t).
●
●
●
Does s == t? If so, return true.
Otherwise, if connected(v, t) for any neighbor v of s, return true.
Return false.
3
What is wrong with it? Can get caught in an infinite loop.
●
How do we fix it?
0
s
1
6
4
7
5
2
t
8
s-t Connectivity
One possible recursive algorithm for connected(s, t).
●
●
●
●
Mark s.
Does s == t? If so, return true.
Otherwise, if connected(v, t) for any unmarked neighbor v of s, return
true.
3
Return false.
Basic idea is same as before, but visit each
vertex at most once.
0
s
1
2
●
Marking nodes prevents multiple visits.
●
Demo: Recursive s-t connectivity.
6
4
5
7
t
8
Depth First Traversal
This idea of exploring a neighbor’s entire subgraph before moving on to the
next neighbor is known as Depth First Traversal.
●
●
Example: Explore 1’s subgraph completely before using the edge 0-3.
Called “depth first” because you go as deep as possible.
s
0
1
3
2
4
8
7
t
5
6
Depth First Traversal
This idea of exploring a neighbor’s entire subgraph before moving on to the
next neighbor is known as Depth First Traversal.
●
●
Example: Explore 1’s subgraph completely before using the edge 0-3.
Called “depth first” because you go as deep as possible.
s
1
2
5
6
0
3
4
8
7
t
Entirely possible for 1’s subgraph to include 3!
● It’s still depth first, since we’re not using the
edge 0-3 until the subgraph is explored.
From: https://xkcd.com/761/
The Power of Depth First Search
DFS is a very powerful technique that can be used for many types of graph
problems.
Another example:
●
Let’s discuss an algorithm that computes a path to every vertex.
●
Let’s call this algorithm DepthFirstPaths.
●
Demo: DepthFirstPaths.
Tree vs. Graph Traversals
Tree Traversals
D
There are many tree traversals:
●
●
●
●
Preorder: DBACFEG
Inorder: ABCDEFG
Postorder: ACBEGFD
Level order: DBFACEG
B
A
F
C
E
G
Graph Traversals
D
There are many tree traversals:
●
●
●
●
Preorder: DBACFEG
Inorder: ABCDEFG
Postorder: ACBEGFD
Level order: DBFACEG
B
F
A
E
C
G
What we just did in DepthFirstPaths is called “DFS Preorder.”
●
●
DFS Preorder: Action is before DFS calls to neighbors.
○ Our action was setting edgeTo.
○ Example: edgeTo[1] was set before DFS calls to
neighbors 2 and 4.
One valid DFS preorder for this graph: 012543678
○ Equivalent to the order of dfs calls.
3
0
s
1
6
4
7
5
2
8
Graph Traversals
D
There are many tree traversals:
●
●
●
●
Preorder: DBACFEG
Inorder: ABCDEFG
Postorder: ACBEGFD
Level order: DBFACEG
B
F
A
E
C
G
Could also do actions in DFS Postorder.
●
●
●
●
DFS Postorder: Action is after DFS calls to neighbors.
Example: dfs(s):
○ mark(s)
○ For each unmarked neighbor n of s, dfs(n)
○ print(s)
Results for dfs(0) would be: 347685210
Equivalent to the order of dfs returns.
3
0
s
1
6
4
2
5
8
7
Graph Traversals
D
Just as there are many tree traversals:
●
●
●
●
Preorder: DBACFEG
Inorder: ABCDEFG
Postorder: ACBEGFD
Level order: DBFACEG
B
F
A
E
C
So too are there many graph traversals, given some source:
● DFS Preorder: 012543678 (dfs calls).
● DFS Postorder: 347685210 (dfs returns).
0
s
G
3
1
6
4
7
5
2
8
Graph Traversals
D
Just as there are many tree traversals:
●
●
●
●
Preorder: DBACFEG
Inorder: ABCDEFG
Postorder: ACBEGFD
Level order: DBFACEG
B
A
So too are there many graph traversals, given some source:
● DFS Preorder: 012543678 (dfs calls).
● DFS Postorder: 347685210 (dfs returns).
0
● BFS order: Act in order of distance from s.
s
○ BFS stands for “breadth first search”.
○ Analogous to “level order”. Search is wide, not deep.
○ 0 1 24 53 68 7
F
E
C
G
3
1
6
4
2
5
8
7
Shortest Paths Challenge
2
Goal: Given the graph above, find the shortest path
from s to all other vertices.
s
0
1
●
Give a general algorithm.
●
Hint: You’ll need to somehow visit vertices in BFS
order.
●
Hint #2: You’ll need to use some kind of data
structure.
●
Hint #3: Don’t use recursion.
5
3
4
6
7
BFS Answer
Breadth First Search.
● Initialize a queue with a starting vertex s and mark that vertex.
○
○
●
A queue is a list that has two operations: enqueue (a.k.a. addLast) and dequeue (a.k.a.
removeFirst).
Let’s call this the queue our fringe.
Repeat until queue is empty:
○
○
A queue is the opposite of a stack. Stack
Remove vertex v from the front of the queue.
has push (addFirst) and pop (removeFirst).
For each unmarked neighbor n of v:
■ Mark n.
■ Set edgeTo[n] = v (and/or distTo[n] = distTo[v] + 1).
■ Add n to end of queue.
Demo: Breadth First Paths
Do this if you want to track distance value.
Citations
Slides adapted from Josh Hug’s Spring 2021 iteration of CS 61B
Lecture 7
CS61BL Summer 2021
●
●
Announcements
●
Monday, August 2nd 2021
●
●
●
Phase 1 Partner Review form
will be released on ed at the
end of lecture.
Midterm 2 Grades released,
congrats!
Regrade requests open today
at 6PM, close next week.
Redemption quiz 9 today
Redemption tutoring for quiz
10 today
Week 6 Survey due
tomorrow!
Agenda
-
Dijkstra’s
A*
MSTs
Agenda
-
Dijkstra’s
A*
MSTS
Note that we won’t be going over sorting
today for the sake of time, but we will
link comprehensive videos in the lab
specs and it is in scope for the final.
Finding the shortest path
Suppose we’re trying to get from s to t. Can we use BFS?
s
t
The reason the places are in Chinese is
because Professor Hug took this diagram from
his Chinese language version of 61B.
Breadth First Search for Mapping Applications
BFS yields the wrong route from s to t.
● BFS yields a route of length ~190 instead of ~110.
● We need an algorithm that takes into account edge distances, also known
as “edge weights”!
s
80
10
s
20
20
40
40
40
40
t
80
10
40
40
70
Correct Result
t
70
BFS Result
Dijkstra’s Algorithm
Problem: Single Source Single Target Shortest Paths
Goal: Find the shortest path from source vertex s to some target vertex t.
3
11
1
3
2
s
5
0
1
2
4
1
1
2
15
6
5
4
1
5
t
Challenge: Try to find the shortest path from town 0 to town 5.
● Each edge has a number representing the length of that road in miles.
Problem: Single Source Single Target Shortest Paths
Goal: Find the shortest paths from source vertex s to some target vertex t.
3
Best path from 0 to 5 is
● 0 -> 2 -> 4 -> 5.
● Total length is 9 miles.
11
1
3
2
s
5
0
1
2
4
1
1
2
6
5
4
15
The path 0 -> 2 -> 5 only
involves three towns, but total
length is 16 miles.
1
5
Problem: Single Source Single Target Shortest Paths
Goal: Find the shortest paths from source vertex s to some target vertex t.
3
11
1
2
3
2
s
5
0
1
4
1
1
2
15
6
5
4
1
v
0
1
2
3
4
5
6
distTo[]
0.0
2.0
5.0
9.0
-
5
Observation: Solution will always be a path with no cycles (assuming
non-negative weights).
edgeTo[]
0→1
1→4
4→5
-
Problem: Single Source Shortest Paths
Goal: Find the shortest paths from source vertex s to every other vertex.
3
11
1
3
2
s
5
0
1
2
4
1
1
2
6
5
4
15
1
5
Challenge: Try to write out the solution for this graph.
● You should notice something interesting.
Problem: Single Source Shortest Paths
Goal: Find the shortest paths from source vertex s to every other vertex.
3
11
1
2
3
2
s
5
0
1
4
1
1
2
15
6
5
4
1
v
0
1
2
3
4
5
6
distTo[]
0.0
2.0
1.0
11.0
5.0
9.0
10.0
5
Observation: Solution will always be a tree.
● Can think of as the union of the shortest paths to all vertices.
edgeTo[]
0→1
0→1
6→3
1→4
4→5
4→6
SPT Edge Count
If G is a connected edge-weighted graph with V vertices and E edges, how many
edges are in the Shortest Paths Tree (SPT) of G? (assume every vertex is
reachable)
3
1
6
s
4
0
2
5
SPT Edge Count
If G is a connected edge-weighted graph with V vertices and E edges, how many
edges are in the Shortest Paths Tree (SPT) of G? (assume every vertex is
reachable)
3
V: 7
Number of edges in SPT
is 6
1
6
s
4
0
2
5
Always V-1:
● For each vertex,
there is exactly one
input edge (except
source).
Finding a Shortest Paths Tree (By Hand)
What is the shortest paths tree for the graph below? Note: Source is A.
B
2
5
s
2
A
1
1
D
5
C
Finding a Shortest Paths Tree (By Hand)
What is the shortest paths tree for the graph below? Note: Source is A.
● Annotation in magenta shows the total distance from the source.
2
B
0
s
2
5
2
A
1
1
D
5
C
1
4
Creating an Algorithm
Let’s create an algorithm for finding the shortest paths tree.
Will start with an incorrect algorithm and then successively improve it.
● Algorithm begins in state below. All vertices unmarked. All distances
∞
infinite. No edges in the SPT.
B
s
2
5
∞
2
A
∞
1
D
1
5
C
∞
Finding a Shortest Paths Tree Algorithmically (Incorrect)
Bad algorithm #1: Perform a depth first search. When you visit v:
● For each edge from v to w, if w is not already part of SPT, add the edge.
5
dfs(A):
dfs(B):
∞
5
Add A->B to SPT
Add B->D to SPT
B
Add A->C to SPT
0
s
2
5
3
A
∞
1
5
1
1
C
0
2
5
3
A
1
1
1
7
D
5
C
1
∞
D
5
B
∞1
s
3
A
5
C
2
5
0
s
D
1
dfs(C):
B already in SPT.
D already in SPT.
B
A already in SPT.
7
Finding a Shortest Paths Tree Algorithmically (Incorrect)
Bad algorithm #2: Perform a depth first search. When you visit v:
●
For each edge from v to w, add edge to the SPT if that edge yields shorter
distance.
We’ll call this
5
dfs(A):
A->B is 5, < than ∞
A->C is 1, < than ∞
0
s
dfs(B):
B->D is 5 + 2 = 7, better than ∞.
B->A is 5 + 3 = 8, worse than 0.
∞
B
2
5
3
A
∞
1
D
1
5
1
dfs(C):
C->B is 1 + 1 = 2, better than 5.
C->D is 1 + 5 = 6, better than 7.
7
D
5
2
3
A
∞
C
5
0
s
1
B
∞1
process “edge
relaxation”.
2
3
A
52
C
B
5
0
s
5
7
1
6
1
D
1
5
Improvements:
● Use better edges if found.
C
1
Finding a Shortest Paths Tree Algorithmically
Dijkstra’s Algorithm: Perform a best first search (closest first). When you visit v:
● For each edge from v to w, relax that edge.
5
A has lowest dist, so dijkstras(A):
C has lowest dist, so dijkstras(C):
∞
5 2
C->B is 1 + 1 = 2, better than 5.
A->B is 5, < than ∞
A->C is 1, < than ∞
0
s
C->D is 1 + 5 = 6, better than ∞.
B
2
5
3
A
∞
1
1
5
1
1
0
3
A
1
1
1
D
6
4
1
D
5
C
6
C
2
5
∞
5
B
∞1
B has lowest dist, so dijkstras(B):
B->A is 2 + 3 = 5, worse than 0.
B->D is 2 + 2 = 4, better than 6.
3
A
2
C
s
s
2
5
0
D
B
Improvements:
● Use better edges if found.
● Traverse “best first”.
Dijkstra’s Algorithm Demo
Insert all vertices into fringe Priority Queue (PQ), storing vertices in order of
distance from source.
Repeat: Remove (closest) vertex v from fringe, and relax all edges pointing
from v.
3
Dijkstra’s Algorithm Demo Link
∞
11
∞
1
0
s
3
2
5
0
1
∞
∞
1
15
6
5
4
∞
2
1
2
4
1
∞
5
Dijkstra’s Pseudocode and
Runtime
Dijkstra’s Algorithm Pseudocode
Dijkstra’s:
● PQ.add(source, 0)
● For other vertices v, PQ.add(v, infinity)
● While PQ is not empty:
○
○
p = PQ.removeSmallest()
Relax all edges from p
Relaxing an edge p → q with weight w:
● If distTo[p] + w < distTo[q]:
○
○
○
distTo[q] = distTo[p] + w
edgeTo[q] = p
PQ.changePriority(q, distTo[q])
Key invariants:
● edgeTo[v] is the best known
predecessor of v.
● distTo[v] is the best known total
distance from source to v.
● PQ contains all unvisited vertices
in order of distTo.
Important properties:
● Always visits vertices in order of
total distance from source.
● Relaxation always fails on edges
to visited (white) vertices.
Guaranteed Optimality
Dijkstra’s Algorithm:
● Visit vertices in order of best-known distance from source. On visit, relax
every edge from the visited vertex.
Dijkstra’s is guaranteed to return a correct result if all edges are non-negative.
Guaranteed Optimality
Dijkstra’s Algorithm:
● Visit vertices in order of best-known distance from source. On visit, relax
every edge from the visited vertex.
Dijkstra’s is guaranteed to return a correct result if all edges are non-negative.
Proof sketch: Assume all edges have non-negative weights.
● At start, distTo[source] = 0, which is optimal.
● After relaxing all edges from source, let vertex v1 be the vertex with
minimum weight, i.e. that is closest to the source. Claim: distTo[v1] is
optimal, and thus future relaxations will fail. Why?
○
○
distTo[p]
≥ distTo[v1] for all p, therefore
distTo[p] + w ≥ distTo[v1]
Can use induction to prove that this holds for all vertices after dequeuing.
Negative Edges
Dijkstra’s Algorithm:
● Visit vertices in order of best-known distance from source. On visit,
relax every edge from the visited vertex.
Dijkstra’s can fail if graph has negative weight edges. Why?
● Relaxation of already visited vertices can succeed.
14
101
34
1
82
33
-67
Negative Edges
Dijkstra’s Algorithm:
● Visit vertices in order of best-known distance from source. On visit,
relax every edge from the visited vertex.
Dijkstra’s can fail if graph has negative weight edges. Why?
● Relaxation of already visited vertices can succeed.
14
34
1
101
34
82
33
-67
Even though vertex 34 has greater distTo at
the time of its visit, it is still able to modify the
distTo of a visited (white) vertex.
Dijkstra’s Algorithm Runtime
Priority Queue operation count, assuming binary heap based PQ:
● add: V, each costing O(log V) time.
○
●
●
Note that we since each priority is initially infinity, each insertion really takes O(1) time
removeSmallest: V, each costing O(log V) time.
changePriority: E, each costing O(log V) time.
Overall runtime: O(V*log(V) + V*log(V) + E*logV).
● Assuming E > V, this is just O(E log V) for a connected graph.
# Operations
Cost per operation
Total cost
PQ add
V
O(log V)
O(V log V)
PQ removeSmallest
V
O(log V)
O(V log V)
PQ changePriority
E
O(log V)
O(E log V)
A* Algorithm
Single Target Dijkstra’s
Is this a good algorithm for a navigation application?
● Will it find the shortest path?
● Will it be efficient?
The Problem with Dijkstra’s
Dijkstra’s will explore every place within nearly two thousand miles of Denver
before it locates NYC.
The Problem with Dijkstra’s
Dijkstra’s will explore every place within nearly two thousand miles of Denver
before it locates NYC.
Why am I in Portland
if I’m trying to find
shortest path to New
York...
The Problem with Dijkstra’s
We have only a single target in mind, so we need a different algorithm. How
can we do better?
How can we do Better?
Explore eastwards first?
Introducing A*
Compared to Dijkstra’s which only considers d(source, v).
Big idea:
● Visit vertices in order of d(Denver, v) + h(v, goal), where h(v, goal) is a
guess at the distance from v to our goal NYC.
● In other words, look at some location v if:
○ We know already know the fastest way to
reach v.
○ AND we suspect that v is also the fastest way to
NYC taking into account the time to get to v.
Example: Henderson is farther than Englewood, but
probably overall better for getting to NYC.
We set this as the priority of
a vertex in the fringe in A*
A* Demo, with s = 0, goal = 6.
Insert all vertices into fringe PQ, storing vertices in order of d(source, v) + h(v, goal).
Repeat: Remove best vertex v from PQ, and relax all edges pointing from v.
3
A* Demo Link
∞
11
∞
1
#
0
1
2
3
4
5
6
h(v, goal)
0
1
s 0
3
15
2
Heuristic h(v, goal)
5
estimates that
∞
distance from 2 to 6
is 15.
0
3
2
5
1
∞
∞
1
15
4
1
∞
5
6
5
4
∞
2
1
2
A* Heuristic Example
How do we get our estimate?
●
●
Estimate is an arbitrary heuristic h(v, goal).
Doesn’t have to be perfect!
For the map to the right, what could we use?
A* Heuristic Example
How do we get our estimate?
● Estimate is an arbitrary heuristic h(v, goal).
● Doesn’t have to be perfect!
For the map to the right, what could we use?
● As-the-crow-flies distance to NYC.
/** h(v, goal) DOES NOT CHANGE as algorithm runs. */
public double h(v, goal) {
return computeLineDistance(v.latLong, goal.latLong);
}
A* vs. Dijkstra’s Algorithm
http://qiao.github.io/PathFinding.js/visual/
Note, if edge weights are all equal (as here),
Dijkstra’s algorithm is just breadth first search.
This is a good tool for understanding distinction
between order in which nodes are visited by the
algorithm vs. the order in which they appear on the
shortest path.
● Unless you’re really lucky, vastly more nodes are
visited than exist on the shortest path.
Minimum Spanning Trees
A new type of problem
We’ve already got a handful of graph-related party tricks: we can traverse in
depth first order and breadth first order, find a topological ordering, and find
the shortest path from one vertex to another in a weighted graph.
Let’s now discuss a totally different type of graph problem: finding a minimum
spanning tree.
Spanning Trees
Given an undirected graph, a spanning tree T is a subgraph of G, where T:
● Is connected.
These two properties
make it a tree.
● Is acyclic.
● Includes all of the vertices. This makes it spanning.
Example:
● Spanning tree is the black edges and vertices.
A minimum spanning tree is a spanning tree of minimum total edge weight.
● Example: Directly connecting buildings by power lines.
Spanning Trees
Which are valid spanning trees?
A)
B)
C)
Which are valid spanning trees?
Not a spanning
tree! Has two
disconnected
components.
Not a spanning
tree! Has a cycle.
Yes, this is a
valid spanning
tree!
MST Applications
●
●
●
●
●
●
Design of networks, water supply networks, electrical grids.
Subroutine in algorithms for other problems (e.g. traveling salesman
problem)
Handwriting and mathematical expression recognition
Image segmentation
Describe financial markets
And many more!
MST
Find the MST for the graph.
B
s
3
A
C
2
2
D
2
MST
Find the MST for the graph.
B
s
A
C
2
2
3
D
2
A Useful Tool for Finding the MST: Cut Property
●
●
A cut is an assignment of a graph’s nodes to two non-empty sets.
A crossing edge is an edge which connects a node from one set to a node
from the other set.
Cut property: Given any cut, minimum weight crossing edge is in the MST.
Question for you:
●
For rest of today, we’ll assume edge weights are unique.
Which edge is the minimum weight edge crossing the cut {2, 3, 5, 6}?
0-7
2-3
1-7
0-2
5-7
1-3
1-5
2-7
4-5
1-2
4-7
0-4
6-2
3-6
6-0
6-4
0.16
0.17
0.19
0.26
0.28
0.29
0.32
0.34
0.35
0.36
0.37
0.38
0.40
0.52
0.58
0.93
Question for you:
Which edge is the minimum weight edge crossing the cut {2, 3, 5, 6}?
0-2 must be
part of the MST!
0-7
2-3
1-7
0-2
5-7
1-3
1-5
2-7
4-5
1-2
4-7
0-4
6-2
3-6
6-0
6-4
0.16
0.17
0.19
0.26
0.28
0.29
0.32
0.34
0.35
0.36
0.37
0.38
0.40
0.52
0.58
0.93
Cut Property Proof
Suppose that for some cut, the minimum crossing edge e is not in the MST.
● Adding e to the MST creates a cycle.
● Some other edge f that is in the MST must also be a crossing edge in order for both
sides of the cut to be connected.
● Removing f and adding e is a lower weight spanning tree.
● Contradiction!
Generic MST Finding Algorithm
Start with no edges in the MST.
● Find a cut that has no crossing edges already in the MST.
● Add smallest crossing edge to the MST.
● Repeat until V-1 edges are in the MST.
This should work, but we need some way of finding a cut with no crossing
edges!
● Random isn’t a very good idea.
Prim’s Algorithm
Prim’s Algorithm
Start from some arbitrary start node, add this to your MST under construction.
● Repeatedly add shortest edge that connects a node in the MST to one not yet
included
● Repeat until all vertices are included.
Let’s walk through a demo that shows how
Prim’s works conceptually.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
leads to a node that is
unvisited.
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
leads to a node that is
unvisited.
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
3
7
D
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Algorithm Demo
Prim’s Algorithm
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Start with any node.
Add that node to the set of nodes in
the MST.
While there are still nodes not in
the MST:
Add the lightest edge that
connects a visited node to an
unvisited one (sprawl outward)
Add the new node to the set of
nodes in the MST.
Prim’s Correctness and the Cut Property
Now that we’ve gotten a feel for how Prim’s works, we can see that Prim’s algorithm is
an implementation of the generic MST solving algorithm we came up with earlier!
Start with no edges in the MST.
●
Find a cut that has no crossing edges already in the MST.
●
Add smallest crossing edge to the MST.
●
Repeat until V-1 edges are in the MST.
Remember, the problem with the generic algorithm was figuring out how to find that
cut.
Prim’s Correctness and the Cut Property
A
5
1
4
B
6
0
C
7
D
3
F
E
2
Instead of randomly selecting a cut,
we methodically build outward from
some start vertex. The two sides of
the cut are the vertices already
included, and those yet to be
included.
This way, we always have a cut that
has no crossing edges already in the
MST (because all the edges that are in
the MST are in one side of the cut).
Prim’s Algorithm Implementation
So, we walked over a conceptual example of how Prim’s works. But how do we actually
do it? The natural implementation of the conceptual version of Prim’s algorithm is highly
inefficient.
● Example: Iterating over purple edges shown is unnecessary and slow.
3
Can use some cleverness and a PQ to speed things up.
11
1
3
2
s
5
0
1
2
4
1
1
2
15
6
3
4
1
5
Prim’s vs. Dijkstra’s
While they solve totally different problems, Prim’s and Dijkstra’s algorithms are nearly
identical in implementation. Dijkstra’s considers “distance from the source”, and Prim’s
considers “distance from the tree.”
Visit order:
● Dijkstra’s algorithm visits vertices in order of distance from the source.
● Prim’s algorithm visits vertices in order of distance from the MST under construction.
Relaxation:
● Relaxation in Dijkstra’s considers an edge better based on distance to source.
● Relaxation in Prim’s considers an edge better based on distance to tree.
Prim’s vs Dijkstra’s
Realistic Implementation Demo (Link)
● Very similar to Dijkstra’s!
Prim’s Runtime
Exactly like Dijkstra’s runtime:
● Insertions: V, each costing O(log V) time.
● Pop min from queue: V, each costing O(log V) time.
● UpdatePriority: E, each costing O(log V) time.
Overall runtime, assuming E > V, we have O(E log V) runtime.
Note: this should be a reasonable assumption as we typically run MST algorithms on connected graphs.
Operation
Number of Times
Time per Operation
Total Time
Insert
V
O(log V)
O(V log V)
Delete minimum
V
O(log V)
O(V log V)
Update priority
E
O(log V)
O(E log V)
Kruskal’s Algorithm
Kruskal’s Algorithm
The awesome thing about data structures and algorithms is that there are
often many ways to solve the same problem.
Now, let’s see another approach: Kruskal’s algorithm.
High-level overview
Prim’s algorithm approaches finding an MST by choosing a start vertex, and
slowly crawling out, incorporating more and more vertices into the MST by
always picking the next edge that connects the incorporated set into the
unincorporated.
Kruskal’s will work by sorting all the edges from lightest to heaviest. Then, we’ll
repeatedly add the next edge that doesn’t cause a cycle in the MST.
Kruskal’s Algorithm
Initially, the MST is empty.
● Consider edges in increasing order of weight.
● Add edge to MST unless doing so creates a cycle.
● Repeat until V-1 edges.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the new node to the set of
nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
3
B
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Algorithm Demo
A
5
1
4
B
3
F
Kruskal’s Algorithm
6
0
C
7
D
E
2
While there are still nodes not in
the MST:
Add the lightest edge that
doesn’t create a loop.
Add the endpoints of that edge
to the set of nodes in the MST.
Kruskal’s Correctness and the Cut Property
Why does Kruskal’s work? Special case of generic MST algorithm.
● Suppose we add edge e = v->w.
● Side 1 of cut is all vertices connected to v, side 2 is everything else.
● No crossing edge is black (since we don’t allow cycles).
● No crossing edge has lower weight (consider in increasing order).
Realistic Kruskal’s
Just as we saw with Prim’s, the conceptual version we went through is simple
enough to think about (and probably what you’d want to do if asked to run
Kruskal’s on an exam).
However, how do we implement Kruskal’s in reality?
●
●
Like Prim’s, we can use a priority queue for easily getting the next lightest
edge (or we could sort the list and iterate down the list).
But what about cycle detection?
Design question:
Let’s say we’re considering adding the next lightest edge e, that connects
vertices v and w. In code, how do we determine if adding e creates a loop?
Design question:
Let’s say we’re considering adding the next lightest edge e, that connects
vertices v and w. In code, how do we determine if adding e creates a loop?
●
Throwback to Disjoint Sets! We can just the disjoint sets data structure
and start with each edge in its own set. Whenever we add an edge (a, b),
we connect(a, b). To check if an e would form a cycle, just see if v and w
are already connected with isConnected()!
Kruskal’s Algorithm Realistic Implementation Demo (Link)
Prim’s vs. Kruskal’s
Step by step visualization for you to play around with (not shown in lecture)
Kruskal’s Runtime
For all intents and purposes, we can consider the runtime of Kruskal’s to be O(ElogV) (same as Prim’s)
Fun fact: This is possible because of “bottom-up
heapification”.
Operation
Insert
Number of Times
Time per Operation
Total Time
E
O(log E)
O(E)
Delete minimum
O(E)
O(log E)
O(E log E)
union
O(V)
O(log* V)
O(V log* V)
isConnected
O(E)
O(log* V)
O(E log*V)
Note: If we use a pre-sorted list of edges (instead of a PQ), then we can simply iterate through the list in
O(E) time, so the second row is no longe relevant.
Though we are assuming E>V (like we can in most scenarios w connected graphs), we also know E is limited to
at most V² . Thus, E*logE is the same as E logV² = E 2log(V) = E log(V)
170 Spoiler: Faster Compare-Based MST Algorithms
year
worst case
discovered by
1975
E log log V
Yao
1984
E log* V
Fredman-Tarjan
1986
E log (log* V)
Gabow-Galil-Spencer-Tarjan
1997
E α(V) log α(V)
Chazelle
2000
E α(V)
Chazelle
2002
optimal (link)
Pettie-Ramachandran
???
E ???
???
(Slide Courtesy of Kevin Wayne, Princeton University)
PSA: joy pottery particles purple
Your passwords protect your online identity.
Don’t reuse your passwords: https://xkcd.com/792/
Some password schemes are dumb: https://xkcd.com/936/
XKCD was not entirely correct about Correct Horse Battery Stapler:
https://security.stackexchange.com/questions/62832/is-the-oft-cited-xkcd-scheme-no-longer-good-advice
XKCD Password generator: https://preshing.com/20110811/xkcd-password-generator/