The Great Puzzles of
Computer Science
Chandler Gregg
5/28/13
Math 7 – Spring 2013
Professor Winkler
Chandler Gregg
The Great Puzzles of Computer Science
2
Computer science is one of the most intriguing endeavors of mathematics and
science, but most people do not know that the subject is much older than personal
computers and the Internet. A mix of mathematics, engineering, and logic, computer
science encompasses many fields of study and is applicable in many more, from economics
to psychology. Primitive computers, first conceived during the Second World War to crack
codes, took up massive space yet contained much less computing power than a smart
phone or even a modern car’s internal computer. With the advent of transistors,
semiconductors, efficient memory, and microchips, computing has reached an incredible
threshold. Because programming languages tend to separate people from what they want
to accomplish and how to implement that into a computer, most people feel that computer
science is inaccessible. However, coding is not the only facet of the broad world of
computing; computer science has deep foundations in mathematics and logic. Simple
computing puzzles and riddles are easily accessible to the public, entertaining, challenging,
and do an amazing job of conveying important computing ideas.
An issue that is often brought up in introductory computer science courses is how to
search for names in a list, which turns out to be trickier than one might expect. Sure, any
human can look for a name by leafing through the Yellow Pages and eventually arriving at a
desired entry, but computers are only as smart as the instructions they are given. As it
turns out, there are three main ways for a computer to search through a directory. What
are they?
Two of the ways to search a directory are fairly obvious. A computer could simply
pick a random entry in the directory and see if it matches the desired entry. Given ten
entries, a computer is almost certain to find the proper entry with enough time, but it is not
Chandler Gregg
The Great Puzzles of Computer Science
3
certain whether the computer will arrive at the desired entry. The best case for this
random search method is one step: the computer arrives at the proper entry on the first
guess. However, the worst-case is that the computer never arrives at the entry. We can nix
the random search from our list of ideas. The second strategy would be to cycle through all
of the entries, starting at the beginning and finishing at the end, which is called a “linear
search”. The best-case scenario is the same as with random search and occurs if the desired
entry is the first one. The worst case improves significantly. No matter what, a linear search
will arrive at any given entry, even if it means cycling through every single entry to arrive
at ‘Zach Zebra’ at the end. Linear search provides us with a surefire technique for searching
a directory, but can we do better?
The third option is more obscure than the first two, but it is far more efficient and
elegant in its design and implementation. Let’s step back from the problem for a second to
evaluate the conditions we have been given and how we can use those conditions to
develop a better search. Note that a directory is sorted alphabetically, meaning that each
entry is further down the alphabet than the one prior. Couldn’t we use this fact to narrow
down the directory with each additional step? If we pick a random value, we can check if it
is less than or greater than our desired entry and then remove the section that contains the
superfluous entries. (My computer science teacher made this process incredibly dramatic
by tearing out whole sections of the Yellow Pages and throwing them across the room.) One
issue with choosing a random value at first is that our two sections will be lopsided,
reducing efficiency. We can just split the dictionary in half to ensure that both subsections
contain the same number of entries. Let’s implement it: we split the dictionary in two,
Chandler Gregg
The Great Puzzles of Computer Science
4
search the subsection that contains our entry, split that section in two, and repeat until we
find the desired entry. It works!
Binary search, as it is called, is the most efficient way to search a sorted list, and the
reason has to do with the underlying mathematics.1 Random search has a worst-case
runtime (time that it takes a computer to run a program) of infinity and linear search has a
worst-case runtime of n, (given n entries). Binary search has a worst-case runtime of log 2 𝑛,
the inverse of exponential growth. Because binary search divides the list it searches by a
factor of two for each step, doubling the size of the list being searched only requires one
additional step. The easiest implementation of binary search is via recursion, a concept that
will be explored later on. A powerful aspect about binary search is that it requires no
knowledge of computer code, yet it conveys a solution to minimizing runtime efficiency of a
search. Binary search is a very useful technique that allows databases, lists, and the
Internet to be searched with incredible speed.
How can binary search be applied outside of the world of computer science?
Consider the following problem, a Google interview question that is popular among puzzle
blogs:
You are in the lobby of a 100-story building and you are given
two identical eggs of unknown shell strength. Your goal is to
determine the highest floor from which you can drop either
one of the eggs without it breaking. You are only allowed to
1
(Stoimen)
Chandler Gregg
The Great Puzzles of Computer Science
5
break two eggs. What is the minimum number of egg drops
required?2
While it may not seem obvious at first glance, this problem involves the same issues as the
searching techniques explored above. In essence, the problem asks the solver to search for
a floor with the property that an egg drop from that floor will succeed, but one made from
one floor higher will cause the egg to break. We could employ binary search because it is
the most efficient search algorithm. However, if we don’t find the desired floor in two steps,
we have broken both of our eggs. Binary search is not a good technique given two eggs.
It looks as though our next best option is linear search. We could start from floor 1,
but it would be more efficient to start at a higher floor, say floor 10 and if the first egg
breaks, we only need 9 more steps to check for the proper floor. Although we are trying to
minimize the number of steps, linear search is a more exhaustive approach and will take a
considerable number of steps. If the egg doesn’t break at floor 10, we could try at floor 20.
If the egg broke at 20, we would need 9 steps to test floor 11 through 19. This will go on
until we reach floor 100. But what is the smallest number of steps that it will take? The idea
is to start at some floor p and search intervals of floors that decrease by 1 each time we
drop a new egg. We can generalize this as 𝑝 + (𝑝 − 1) + (𝑝 − 2) + ⋯ + 2 + 1. We start at p,
then increase to floor 𝑝 + (𝑝 − 1) if we haven’t found the desired floor, then increase to
floor 𝑝 + (𝑝 − 1) + (𝑝 − 2) and so on until we reach the number 1.3 This is simply p’s
triangular number, represented as (𝑝(𝑝 + 1))/2 (if you don’t know what a triangular
2
3
(Google Interview Puzzle : 2 Egg Problem)
(Google Interview Puzzle : 2 Egg Problem)
Chandler Gregg
The Great Puzzles of Computer Science
6
number is, consult my previous paper.) To solve the problem, we need to set (𝑝(𝑝 + 1))/2
greater than or equal to 100 (greater than to account for rounding up) because in this case
we are searching 100 floors. A bit of simple algebra, employing the quadratic formula, leads
to a p value of 14, which is the correct answer. But why did a modified linear search work?
The most intriguing part of the Two Egg Problem is that although it relates to
computer science, its answer is a trick; many computer science students would begin to
solve the problem with the intuition of binary search, only to arrive at the same dead end
that we did. By asking this problem, Google not only tests its applicants’ intuition, but their
ability to discern appropriate applications of computing techniques. Computer science is
rife with tradeoffs and the tradeoff in this problem is one of runtime and accuracy. Binary
search would run in log 2 (100) time, equal to around 7, but that would require 7 eggs. One
wrong choice in this problem is bound to happen no matter what, but two leads to failure.
Binary search may be the better choice 95% of the time, but in the off chance of 5%, you
will end up with two broken eggs and none the wiser. The Two Egg Problem demonstrates
tradeoffs in computer science without any explicit reference to the subject.
Before we attempt the next puzzle, it is of interest that there is a group of Brahmin
priests that have been trying to solve a 64-disc version of the problem for many years, and
when they succeed, “the world will vanish”.4 Thankfully, we will show that in order to solve
the problem, it will take 264 − 1 seconds, or 585 billion years to happen.
The Towers of Hanoi is a well-known puzzle that consists of three upright pegs, on
one of which rests a certain number of discs of increasing radius from top to bottom. The
goal is to move all the discs from one peg to another -- the only rules are that any disc may
4
(Stone)
7
Chandler Gregg
The Great Puzzles of Computer Science
not rest on a disc of smaller radius and discs must be moved one at a time. Games of four or
five discs take a bit of time and are reasonable to solve, but the more discs that are added,
the exponentially harder it gets. Here is a figure:
Figure 1
In order to understand the optimal solution to the Towers of Hanoi, it is essential to
understand the concept of recursion. Recursion is a phenomenon of repeating smaller and
smaller iterations of the same thing. A day-to-day example is the recursion of the Land-OLakes butter logo, which contains a picture of a lady who is holding another box with a logo
and so on. If you’ve never seen a box, here is one:
Figure 2
The way that recursion is used to solve the Towers of Hanoi is to call a function that solves
the problem and then call that same function inside itself, only on a smaller part of the
problem. To solve the Towers of Hanoi with 3 discs, the program could solve it with 2 discs,
Chandler Gregg
The Great Puzzles of Computer Science
8
move the 3rd disc into the end position, and then move the other 2 discs on top of the 3rd
one. How would you solve for 2 discs? You run a function recursively on the 2 discs, and
end up moving 1 disc, which is trivial. The trivial instance is called the “base case” and
every recursive program has one. The recursion approach seems incredibly repetitive and
disorganized, but computers are very good at repetitive tasks. The downside with
recursion is that it requires a massive amount of memory to store temporary data. In the
end, however, recursion provides a correct solution to the Towers of Hanoi.
What about the Brahmin priests who are patiently solving the Towers of Hanoi until
the world crumbles? We must figure out the runtime of the recursive Towers of Hanoi
program to figure out why we have plenty of time (585 billion years) until the world ends
by the hands of the Brahmin priests. Binary search ran in log 2 (𝑛) time because for each
step added, the program could handle twice as much data. The recursive Towers of Hanoi
program does the opposite. For each additional disc added, the function has to run through
its recursive sub-functions again, on top of the functions it just ran. The runtime of the
recursive Towers of Hanoi function is 2𝑛 − 1.5 What’s with the minus 1? It is a paltry
amount compared to 2𝑛 .
The Traveling Salesman Problem is a very well known computer science problem
whose optimal solution is attached to a significant financial reward. To celebrate the new
millennium, the Clay Mathematics Institute in Cambridge, Massachusetts established seven
prizes (of $1,000,000) for solutions to seven unsolved problems in mathematics and
computer science. One of the Clay Millennium Problems is the P vs. NP problem. P
programs are simple to execute while NP programs “require an impossibly long time” to
5
(Farid)
Chandler Gregg
The Great Puzzles of Computer Science
9
execute.6 The Traveling Salesman Problem is a famous NP-complete problem related to
graph theory and has yet to be solved. The problem goes as follows:
“Given a collection of cities connected by highways, what is the
shortest route that visits every city and returns to the starting
place?”7
The runtime of the correct Traveling Salesman problem solution is somewhere around 𝑛!,
which is far, far slower than the Towers of Hanoi program.
The reason that the Traveling Salesman Problem is so complex is because it
stipulates that each city must be visited just once, the first city must be the endpoint of the
journey, and on top of that, distance must be minimized. These factors combine to make a
solution for large numbers nearly impossible to reach and this corroborates the case for
NP-complete problems. To definitively solve the problem, a brute force method is required
and that method would run in factorial time, making the Towers of Hanoi problem seem
easy as pie. However, computer scientists and graph theorists have devised algorithms that
approximate solutions to the problem and these algorithms are called “heuristic”.8
Heuristics in computer science are tradeoffs between speed and accuracy. Although the
heuristic “nearest neighbor algorithm” is not optimal, it comes very close and because it is
much faster than the brute force method, it can calculate traveling salesman paths
containing a much greater number of cities. Heuristic algorithms are incredibly helpful in
(Clay Mathematics Institute)
(Klarreich)
8 (Travelling salesman problem)
6
7
Chandler Gregg
The Great Puzzles of Computer Science
10
applications of the Traveling Salesman Problem because they provide a solution whose
difference from the optimal solution is negligible. In fact, solutions to the problem are used
in the manufacture of circuit boards, scheduling, and even genetics.9 The fact that the
Traveling Salesman Problem has not been optimized but that it is used for a variety of
applications makes the problem much more compelling to study.
An important distinction to be made about the problem is that progress is being
made in the speed and accuracy of solutions because of improved algorithms and improved
computers. Although computers are always increasing in memory and processing speed,
they come nowhere near what the large scale Traveling Salesman Problem solution
requires of them. Computers have limitations on space that will keep them from ever
optimizing solutions to the Traveling Salesman problem. The progress being made on the
problem is coming from people devising smarter and better heuristic algorithms. This
brings up the question that if the Traveling Salesman Problem is an NP-complete problem
and NP-complete problems end up being proven to be unsolvable, what is the point of
tackling the problem in the first place? For a pure mathematician, the answer would be that
there is no point, but for an applied mathematician, results of the Traveling Salesman
Problem provide a whole host of solutions to real-world problems.
One last intricacy of the Traveling Salesman Problem is its striking comparability to
the Seven Bridges of Konigsberg, the founding problem of graph theory, which was devised
by Leonhard Euler. The Seven Bridges of Konigsberg problem asks if there is a certain path
9
(Klarreich)
Chandler Gregg
The Great Puzzles of Computer Science
11
on the following figure that traverses each bridge once and only once.10 Here is the
arrangement of the bridges and land:
Figure 3
As it turns out, the problem has no solution. To summarize Euler’s proof, he expressed the
landmasses as dots (called “nodes” in graph theory) and the bridges as edges (edges
connect nodes). He noticed that in order to fulfill a “Eulerian Tour,” (visiting each edge once
and only once) a graph must have either zero or two nodes with an odd number of edges.
Nodes of any other odd degree besides two will cause a traveler to become stuck at an odd
node.
The comparison between the Traveling Salesman Problem and the Eulerian Tour of
the Seven Bridges of Konigsberg draws computer science closer to graph theory. Both
problems have very similar looking graphs. Both have nodes and edges and solutions
traverse all the nodes and edges of the graphs. However, the Traveling Salesman Problem
asks for what is called a “Hamiltonian circuit,” which is a graph circuit that visits each node
once and only once, and the problem asks for the distance of the circuit to be minimized.11
Because distance plays no role in the Seven Bridges of Konigsberg, it is a featured puzzle in
10
11
(Weisstein, Königsberg Bridge Problem)
(Weisstein, Hamiltonian Cycle)
12
Chandler Gregg
The Great Puzzles of Computer Science
topology as well as graph theory. Despite their differences, both problems require an
understanding of graphs. The inclusion of graph theory in computer science allows for
computer science to solve a much broader, more complex range of problems and
applications.
The Dining Philosophers is a problem found in computer science that deals with the
issues of concurrent processes competing for scarce resources.12 Compared to the
Traveling Salesman Problem and Towers of Hanoi, the Dining Philosophers problem is
much simpler in computational execution, but it is no less interesting and introduces to the
solver a variety of concepts in computer science. The problem goes as follows:
Five philosophers, who spend their lives eating or thinking, are
seated at a circular table that has a bottomless plate of pasta in
the middle. Between each philosopher is a chopstick. In order
to eat, any philosopher must have two chopsticks (one from
the left, one from the right). Communication is not allowed
between philosophers and once a philosopher is done eating,
they place both their chopsticks at their sides and proceed to a
thinking state. How do all of the philosophers successfully
maintain their lifestyle without dying of starvation?13
Figure 4
12
13
(Wilensky)
(Shene)
Chandler Gregg
The Great Puzzles of Computer Science
13
The problem is peculiar because it is hard to state an answer for how the philosophers
should proceed without errors occurring. This is due to a variety of factors that will be
explained later on, but for now let’s consider a reasonable strategy for the philosophers.
The figure below explains the strategy that each philosopher should take; when the
philosopher is finished thinking, he or she should pick up the left chopstick as soon as it is
available and then pick up the right chopstick as soon as that one is available. The
philosopher will eat for as long as it needs or wants, replace the chopsticks on the table,
and proceed to a thinking state until he or she is hungry again. If every philosopher abides
by this strategy, it seems reasonable that each one will be properly fed. What is the point of
the problem?
As it turns out, things can and do go wrong. Some of the diners
have a fair chance of dying by starvation. Consider the possibility
that at the beginning of the simulation, each philosopher is hungry
and picks up the chopstick to the left of him or her. Now, every
diner has a left chopstick but there are no chopsticks remaining on
the table. Communication is forbidden (although it wouldn’t matter,
because these philosophers don’t talk) so nobody can negotiate the
release of one chopstick. This is called a “deadlock” in computer
science and is a serious problem because it prevents any forward
progress.14 The program will be stuck on the five diners, each
Figure 5
holding a chopstick to no avail. Eventually, the philosophers will die
and the program will terminate.
14
(Shene)
Chandler Gregg
The Great Puzzles of Computer Science
14
Another problem with the program is that the timeframe with which each
philosopher eats and thinks is not specified. It is not out of the question that one or more
philosophers wish to eat indefinitely or think indefinitely. In the case of two philosophers
eating indefinitely, the other three philosophers will have no access to chopsticks and will
eventually die.15 Conversely, if this situation occurs and one of the philosophers without a
chopstick thinks indefinitely, that philosopher won’t die and equilibrium between three of
the philosophers will occur. The problem could also be modified and further analyzed;
what if there were ten philosophers, or what if a particular philosopher only ate when he
could reach a salt shaker? The combinations are truly endless, making the problem so
interesting to tackle from multiple angles.
I see the Dining Philosophers problem more as a behavioral simulation than a
problem because the conditions are very vague, there are many variables, and the problem
keeps running until starvation occurs (which may be never). In order to appreciate the
problem, it is essential to take it for what it is; a theoretical model. Theoretical models
made using computer science are incredibly helpful, from predicting bacterial growth to
the spread of a disease to predicting the erratic motion of financial markets. Without
models, the weatherman would lose his job and without computers, models would be less
powerful phenomena. Models strive to create a basic understanding of a dynamic system,
but they are rarely 100% accurate. The Dining Philosophers problem mirrors this
unpredictability. Every time we start a new simulation of our five philosophers, we don’t
know whether Marx, Socrates, Hobbes, Descartes, and Freud will live in solitary peace or
whether one or more of them will pass away from starvation. Adding more philosophers
15
(Shene)
Chandler Gregg
The Great Puzzles of Computer Science
15
and more conditions further complicates the simulation, just as real-world variables
complicate models. The Dining Philosophers problem succeeds in bridging the gap between
an innocent puzzle and a field in which computers work to predict the conditions of the
world in which we live.
While it is difficult to segregate mathematical puzzles into different categories,
puzzles in computer science seem to occupy a niche category that blurs the very definition
of puzzle and synthesizes various subjects, both mathematical and beyond. It is because
computer science puzzles lie in this category that they are so intriguing. Like the P vs. NP
puzzles, computer science puzzles relate to each other very well. Closely related puzzles
are even more interesting to solve because a solution to one can lead to a solution of
another, a complication, or a flaw in a proof. The fact that Google and Microsoft ask
interviewees to answer puzzles like these is a testament to the puzzles’ difficulty and
importance in computing. Puzzles like the Dining Philosophers and the Towers of Hanoi
may have been invented for different reasons, but they can be used as powerful tools to
teach computing techniques and definitions without overtly mentioning code or
computing. Computer science tends to be shunned by the general public because of the
perception of coding and programming as incredibly complex, but this type of puzzle helps
to unravel the layers of complexity. Computer science puzzles have yet to proliferate to the
degree that other mathematical puzzles have, but if given the chance to tackle a
computational puzzle, the public would see computer science as a much more accessible,
friendly, and entertaining subject.
16
Chandler Gregg
The Great Puzzles of Computer Science
Works Cited
Clay Mathematics Institute. P vs NP Problem. 2013. 20 May 2013
<http://www.claymath.org/millennium/P_vs_NP/>.
Farid, Hany. Recursion. 2013. 20 May 2013
<http://www.cs.dartmouth.edu/~cs1/lectures/lecture14/lecture14.html>.
Google Interview Puzzle : 2 Egg Problem. 5 December 2006. 19 May 2013 <http://classic
-puzzles.blogspot.com/2006/12/google-interview-puzzle-2-egg-problem.html>.
Klarreich, Erica. Computer Scientists Find New Shortcuts for Infamous Traveling Salesman
Problem. 30 January 2013. 20 May 2013
<http://www.wired.com/wiredscience/2013/01/traveling-salesman-problem/>.
Shene, C.K. "ThreadMentor: The Dining Philosophers Problem." 13 March 2011.
Multithreaded Programming with ThreadMentor: A Tutorial. 20 May 2013
<http://www.cs.mtu.edu/~shene/NSF-3/e-Book/MUTEX/TM-example-philos
-1.html>.
Stoimen. Computer Algorithms: Binary Search. 26 December 2011. 20 May 2013
<http://www.stoimen.com/blog/2011/12/26/computer-algorithms-binarysearch/>.
Stone, Kevin. Tower of Hanoi. 2013. 20 May 2013
<http://www.brainbashersgames.com/games/tower0.asp>.
"Travelling salesman problem." 20 May 2013
<http://fds.oup.com/www.oup.com/pdf/oxed/D2.pdf>.
Weisstein, Eric W. "Hamiltonian Cycle." 2013. Wolfram MathWorld. 20 May 2013
<http://mathworld.wolfram.com/HamiltonianCycle.html>.
Chandler Gregg
The Great Puzzles of Computer Science
17
"Königsberg Bridge Problem." 2013. Wolfram MathWorld. 20 May 2013
<http://mathworld.wolfram.com/KoenigsbergBridgeProblem.html>.
Wilensky, U. Dining Philosophers. 2003. Center for Connected Learning and ComputerBased Modeling, Northwestern Institute on Complex Systems, Northwestern
University. 20 May 2013
<http://ccl.northwestern.edu/netlogo/models/DiningPhilosophers>.
Figures courtesy of:
Figure 1: http://mathforum.org/mathimages/imgUpload/10_Ring_Hanoi.jpg
Figure 2:
http://www.landolakesinc.com/stellent/groups/public/@lolinc/documents/web_
content/ecmp2-0169204.jpg
Figure 3: http://mathworld.wolfram.com/KoenigsbergBridgeProblem.html
Figures 4 and 5: http://www.cs.mtu.edu/~shene/NSF-3/e-Book/MUTEX/TM
-example-philos-1.html