Chapter 7
Inefficiency and Intractability
CS 345
Winter Quarter, 2016
Towers of Hanoi Revisited
• We discovered that the time complexity of
the Towers of Hanoi problem is 2N -1
• The original problem had 64 rings. If one
ring could be moved every 10 seconds, it
would take over 5 trillion years to solve the
problem.
• Even moving 1 million rings a second takes
over 1/2 million years
The Monkey Puzzle
• See pages 160-162 for an explanation of the
puzzle
• Given N cards, a brute-force solution takes
N! steps.
• Arranging 1 million cards a second still
takes over 490 billion years for 25 cards.
Reasonable vs. Unreasonable
Time
• The factorial function grows very rapidly.
• Factorial grows even faster than 2N.
• One of the fastest growing functions we
know is Ackerman’s Function.
• For comparison, 2 slides that follow
reproduce figures 7.3 (page 163) and 7.5
(page 167)
Ackerman’s Function
A(1, j) = 2j, for j  1
A(i, 1) = A(i-1, 2) for i  2
A(i, j) = A( i -1, A( i, j-1)) for i, j  2
How fast does this function grow? Note that
A(i, j+1) > A( i,j) ; A(i+1,j) > A(i,j) and
A(3,4) = 2^2^2 …^2 (a stack of 65,536 2’s)
65,536
N
20
60
100
300
1000
100
300
500
1500
5000
86
354
665
2469
9966
N2
400
3,600
10,000
90,000
1 mil
N3
8,000
216,000
1 mil
27 mil
1 bil
31 digit
91 digit
Func.
5N
N LogN
2N
1048576 19 digit
N!
19 digit
161 digit
623 digit
NN
27 digit 107 digit 201 digit
744 digit
82 digit
302 digit
big
real big
Figure 7.5, page 164
assume 1 instruction per nanosecond
N
f(N)
20
40
60
N2 1/2500ms 1/625ms 1/278ms
N3
1/300s
2N
1/1000s
NN
3.3bilY
1/10s
18.3m
78/100s
36.5Y
46digC 89digC
100
300
1/100ms
1/11ms
10s
40.5m
400bilC
72digC
182digC
725digC
Tractable Algorithms
• Examining the previous two slides indicate
that some algorithms might be reasonable
and others not reasonable.
• Algorithms whose run time is bounded from
above by a polynomial function of N are
called polynomial-time algorithms, are said
to be reasonable, and are called tractable.
Intractable Algorithms
• An algorithm which has an exponential or
super-exponential upper bound is said to be
unreasonable and is called intractable.
• Note that an intractable algorithm has a
solution. It just takes an unreasonably long
time to find it.
Decision Problems
• Perhaps part of the long running time of an
algorithm is that we are looking for a
solution.
• What if we just asked, “Does a solution
exist?”
• A decision problem is one which has a yes
or no answer.
• Most problems have a decision problem
version.
• For example: What is the minimum grade I
can get on the final and still get an A in the
class?
• The decision problem version is: If I get a
(some number) on the final, will I still get
an A in the class?
More on the Monkey Puzzle
Page 166
1. Computers are becoming faster by the
week. Over the last 10 years or so
computer speed has increased roughly by a
factor of 50. Perhaps obtaining a practical
solution to the problem is just a question of
awaiting an additional improvement in
computer speed.
2. Doesn’t the fact that we have not found a
better algorithm for this problem indicate
our incompetence at devising efficient
algorithms? Shouldn’t computer scientists
be working at trying to improve the
situation rather than spending their time
writing books about it?
3. Haven’t people tried to look for an
exponential-time lower bound on the
problem, so that we might have a proof that
no reasonable algorithm exists?
4. Maybe the whole issue is not worth the
effort, as the monkey puzzle problem is just
one specific problem. It might be a colorful
one, but it certainly doesn’t look like a very
important one.
• Reply to point 1: Examine figure 7.7 on
page 166. Given an exponential algorithm,
an improvement in computer speed by a
constant factor--however large--will make
only an insignificant improvement overall.
• Reply to point 4: It so happens that the
“Monkey Puzzle” problem is not alone.
• There are close to 1000 diverse algorithmic
problems which have intractable solutions.
But, none are known not to have tractable
solutions.
• In many cases, the best-known lower
bounds are (N). That is, these problems
have very large algorithmic gaps
The Class NP
• We denote these problems as NP.
• These are problems that have no known
tractable solution, but we cannot prove that
a tractable solution does not exist.
Some NP Problems
• Generalized jigsaw or arrangement
• Graph problems
–
–
–
–
–
Clique
TSP (the Decision Version)
Hamiltonian cycle and path
Eulerian Path: Königsberg bridges, 1736
Graph Colorings
Nondeterminism and Cook’s
Theorem
• The theory of NP began with a remarkable
theorem of Cook in 1971.
• The notion of nondeterminism is rather nonintuitive.
• One should think of a nondeterministic
algorithm as an abstract notion, not as a
realistic goal.
• Nondeterminism is more important to the
development of the theory and the
explanation of the existence of this class of
problems than it is to the techniques for
using the theory.
• A nondeterministic algorithm has, in
addition to all the regular operations of a
deterministic algorithm, a very powerful
oracle.
• Problems solved by nondeterministic
algorithms have the property that they can
take an unacceptably long time to produce a
solution. If a solution is arrived at,
however, there is an easy way to convince
someone that it is a solution.
• Nondeterministic algorithms are very
powerful, but their power is not unlimited.
Not all problems can be solved efficiently
by nondeterministic algorithms.
• The class of problems that have a
deterministic, polynomial time algorithm
that solves them is called P.
• The class of problems that have a
nondeterministic algorithm that can be
checked in polynomial time is called NP.
• Are nondeterministic algorithms more
powerful than deterministic ones????
• Its clear that P NP, but is P NP ???
• No one has yet demonstrated an algorithm
that is in NP but not in P
• In contrast, to prove P = NP requires us to
show that every problem in NP can be
solved by a deterministic polynomial time
algorithms. Nobody has done this either
(and very few believe it is true)
• The question does P = NP? (first asked in
1971) is the most difficult open question in
theoretical Computer Science.
Polynomial Reduction
• Algorithm X is reducible to Algorithm Y if
there is a polynomial time algorithm which
transforms an input to X into an input to Y
in such a manner that Y always gives the
same answer ( “yes” or “no”) using the
transformed input as X does with the
original input.
• If Alg X is reducible to Alg Y then, if Y can
be solved with a polynomial time
deterministic algorithm, so can X.
• This means that X is no harder than Y
• Definition: A problem X is called an NPhard problem if every problem in NP is
reducible to X. This means that no
problem in NP is harder than X.
• Definition: A problem X is called an NPcomplete problem if (1) it belongs to NP
and (2) it is NP-hard.
• Note that from the definition of NP-hard, if
any NP-hard problem is shown to be in P,
then P = NP.
• Cook’s remarkable theorem proved that
there exists NP-complete problems. In
particular, he demonstrated the SAT was
NP-complete.
• Cook’s theorem makes showing a problem
is NP-complete easier.
• Definition: A problem X is an NP-complete
problem if (1) X belongs in NP and (2)
some problem Y reduces to X where Y is
already known to be in NP-complete.
• Because all problems in NP reduce to each
problem in NP-complete, if one NPcomplete problem has a polynomial-time
deterministic algorithm, then P = NP.
Imperfect Solutions to NPComplete Problems
• NP-complete problems are usually taken as
the decision problem (yes/no) version of
optimization problems. E.g. TSP
• But, humans manage to solve these
problems all the time in everyday life, and it
doesn’t take centuries. Also, the human
brain is orders of magnitudes slower than
today’s computers.
• The human brain is massively parallel
• Approximation and Probabilistic
Algorithms
• Heuristics
Provably Intractable Problems
• We are not able to prove that problems in
NP have no tractable solution
• There are problems, however, such as the
Towers Of Hanoi, that can be shown to have
an exponential lower bound and that have
no certificate.
• These problems belong to classes that are
even harder than NP.
Complexity Classes
• Theoretical Computer Scientists talk about a
hierarchy of complexity classes.
• These classes are for both time and space
requirements.
• Dual classes or co- classes (co-NP): the yes
or no answers are not the same. Is it true
that p is a prime number? / Is it true that p
is a composite number?
Research on Complexity Classes
• Some theorists work with questions such as:
Does P = NP? or Does NP = co-NP? or
Can we refine the complexity classes?
• Others work to develop approximate
solutions or heuristic solutions or alternate
models of computation: genetic algorithms,
neural networks, etc..