Computability
Tractable, Intractable and
Non-computable functions
1
Efficiency

A computer program should be totally correct,
but it should also



execute as quickly as possible (time-efficiency)
use memory wisely (storage-efficiency)
How do we compare programs (or algorithms
in general) with respect to execution time?



various computers run at different speeds due to different
processors
compilers optimize code before execution
the same algorithm can be written differently depending on
the programming language used
2
Order of Complexity

For very large n, we express the number of
operations as the order of complexity.

Order of complexity for worst-case behavior is often
expressed using Big-O notation:
Number of operations
Order of Complexity
n
O(n)
Usually doesn't
n/2 + 6
O(n)
matter what the
constants are...
2n + 9
O(n)
we are only
concerned about
the highest power
of n.
3
O(n) ("Linear")
2n + 9
Number of
Operations
n
n/2 + 6
n
(amount of data)
4
Order of Complexity
Number of operations Order of Complexity
n2
O(n2)
2n2 + 7
O(n2)
n2/2 + 5n + 2
O(n2)
Usually doesn't
matter what the
constants are...
we are only
concerned about
the highest power
of n.
5
O(n2) ("Quadratic")
n2
2n2 + 7
n2/2 + 5n + 2
Number of
Operations
n
(amount of data)
6
Order of Complexity
Number of operations
log2n
log10n
2(log2n) + 5
Order of Complexity
O(log n)
O(log n)
O(log n)
The logarithm base
is not written in
big O notation
since all that matters
is that the function
is logarithmic.
7
O(log n) ("Logarithmic")
2(log2 n) + 5
Number of
Operations
log2 n
log10 n
n
(amount of data)
8
Comparing Big O Functions
Number of
Operations
O(2n)
O(n2)
O(n log n)
O(n)
O(log n)
O(1)
n
(amount of data)
9
Searching & Sorting
WORST CASE Order Of Complexity on N data elements

Linear Search
O(N)

Binary Search
O(log N)

Selection Sort
O(N2)

Bubble Sort
O(N2)

Merge Sort
O(N log N)

Quick Sort
O(N2)

Sort + Binary Search
O(N log N) + O(log N)
= O(N log N)
10
Comparing Algorithms

Assume an algorithm processes n data values. If each
operation takes 1 s to execute, how many s will it
take to run the algorithm on 100 data values if the
algorithm has the following number of computations?
Number of Computations
n
n • log2 n
n2
n3
2n
n!
Execution Time
100 s
665 s
10,000 s
1,000,000 s = 1 sec
> 1030 s
> 10160 s
11
Decision Problems



A specific set of computations are classified as
decision problems.
An algorithm describes a decision problem if
its output is simply YES or NO, depending on
whether a certain property holds for its input.
Example:
Given a set of n shapes,
can these shapes be
arranged into a rectangle?
12
Monkey Puzzle Problem

Given:




A set of n square cards whose sides are imprinted
with the upper and lower halves of colored
monkeys.
n is a square number, such that n = m2.
Cards cannot be rotated.
Problem:

Determine if an arrangement of the n cards in an
m X m grid exists such that each adjacent pair of
cards display the upper and lower half of a monkey
of the same color.
Source: www.dwheeler.com (2002)
13
Example
1
4
7
2
5
8
3
6
9
Images from: Simonas Šaltenis, Aalborg University, simas@cs.auc.dk
14
Analysis
Simple algorithm:

Pick one card for each cell of m X m grid.

Verify if each pair of touching edges make a full
monkey of the same color.

If not, try another arrangement until a solution is
found or all possible arrangements are checked.

Answer "YES" if a solution is found. Otherwise,
answer "NO" if all arrangements are analyzed and
no solution is found.
15
Analysis
If there are n = 9 cards (m = 3):
To fill the first cell, we have 9 card choices.
To fill the second cell, we have 8 card
choices left.
To fill the third cell, we have 7 card choices
remaining.
etc.
The total number of unique arrangements for n = 9 cards is:
9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 362,880
16
Analysis
For n cards, the number of arrangements to examine
is n! (n factorial)
If we can analyze one arrangement in a microsecond:
n
Time to analyze all arrangements
9
362,880 s = 0.36288 s
16
20,922,789,888,000 s
≈ 242 days
25
15,511,210,043,330,985,984,000,000 s
≈ 491,520,585,955 years
17
Map Coloring



Given a map of n territories, can the map be
colored using k colors such that no two
adjacent territories are colored with the
same color?
k=4: Answer is always yes.
k=2: Only if the map contains no point that
is the junction of an odd number of
territories.
18
Map Coloring
19
Map Coloring



Given a map of 48 territories, can the map be colored
using 3 colors such that no two adjacent territories are
colored with the same color?

Pick a color for California (3 choices)

Pick a color for Nevada (3 choices)

...
There are 348 = 79,766,443,076,872,509,863,361
possible colorings.
No one has come up with a better algorithmic solution
that works in general for any map, so far.
20
Classifications

Algorithms that are O(nk) for some
fixed k are polynomial-time algorithms.



O(1), O(log n), O(n), O(n log n), O(n2)
reasonable, tractable
All other algorithms are
super-polynomial-time algorithms.


O(2n), O(n!), O(nn)
unreasonable, intractable
21
Traveling Salesperson


Given: a weighted graph of nodes
representing cities and edges representing
flight paths (weights represent cost)
Is there a route that takes the salesperson
through every city and back to the starting
city with cost no more than k?

The salesperson can visit a city only once
(except for the start and end of the trip).
22
Traveling Salesperson
12
B
10
7
8
C
6
7
F
4
D
9
A
7
E
Is there a route with cost at most 52?
Is there a route with cost at most 48?
3
5
G
11
YES (Route above costs 50.)
YES? NO?
23
Traveling Salesperson



If there are n cities, what is the maximum number
of routes that we might need to compute?
Worst-case: There is a flight available between
every pair of cities.
Compute cost of every possible route.





Pick a starting city
Pick the next city (n-1 choices remaining)
Pick the next city (n-2 choices remaining)
...
how to
build a
route
Maximum number of routes: (n-1)! = O(n!)
24
P and NP


The class P consists of all those decision
problems that can be solved on a deterministic
sequential machine (e.g. a computer) in an
amount of time that is polynomial with respect to
the size of the input
The class NP consists of all those decision
problems whose positive solutions can be verified
in polynomial time given the right information.
from Wikipedia
25
NP Complete

The class NPC consists of all those problems in NP
that are least likely to be in P.



Each of these problems is called NP Complete.
Monkey puzzle, Traveling salesperson, Hamiltonian
path, map coloring, satisfiability are all in NPC.
Every problem in NPC can be transformed to
another problem in NPC.

If there were some way to solve one of these problems
in polynomial time, we should be able to solve all of
these problems in polynomial time.
26
Complexity Classes
NP Problems
P Problems
NP Complete
Problems
We know that P < NP, since
any problem that can be solved
in polynomial time can certainly
have a solution verified in
polynomial time.
But does P = NP?
If P ≠ NP, then all decision problems can be broken
down into this classification scheme.
If P = NP, then all three classes are one and the same.
The Clay Mathematics Institute is offering a $1M prize
for the first person to prove P = NP or P ≠ NP.
(http://www.claymath.org/millennium/P_vs_NP/)
27
28
It gets worse...

Tractable Problems


Intractable Problems


Problems that have reasonable, polynomialtime solutions
Problems that have no reasonable, polynomialtime solutions
Noncomputable Problems

Problems that have no algorithms at all to solve
them
29
Noncomputability and
Undecidability

An algorithmic problem that has no algorithm is
called noncomputable.
If the noncomputable algorithm requires a
yes/no answer, the problem is called
undecidable.

Example:



Given any set of any number of different tile designs
(examples shown above), with an infinite number of
each type of tile, can we tile any area with these
tiles so that like colored edges touch?
This problem is undecidable!
30
Tiling Problem
YES
Note the periodicity in the tiling.
31
Tiling Problem
NO
For this 3 X 3 room, if we try all 39
tiling configurations, no tiling works.
32
Tiling Problem

Possible algorithm:



If we find a repeating pattern, report YES.
If we find a floor we cannot tile, report NO.
BUT: there are some tilings which have no
repeating pattern!
33
Tiling Problem


The only way to know if this set of tiles
can tile every finite-sized floor is to evaluate
every possible floor.
BUT: there are an infinite number of finite-sized
floors!


So we could never answer YES in this case.
This problem is undecidable.
34
Another Undecidable Problem:
The Barber Paradox
Suppose there is a town with one
male barber; and that every man in
the town keeps himself clean-shaven:
some shave themselves and some
are shaved by the barber. Only the
barber can shave another man. The
barber shaves all and only those men
who do not shave themselves.
Does the barber shave himself?
35
Program Termination


Can we determine if a program will terminate
given a valid input?
Example:
1. Input x
2. While x is not equal to 1, do the following:
(a) Subtract 2 from x.


Does this algorithm terminate when x = 15105?
Does this algorithm terminate when x = 2008?
36
Program Termination

Another Example:
1. Input x
2. While x is not equal to 1, do the following:
(a) If x is even, divide x by 2.
(b) Otherwise, Set x to 3x + 1.



Does this algorithm terminate for x = 15?
Does this algorithm terminate for x = 105?
Does this algorithm terminate for any positive x?
37
The Halting Problem

Can we write a general program Q that takes
as its input any program P and an input I and determines
if program P will terminate (halt) when run with input I?



It will answer YES if P terminates successfully on input I.
It will answer NO if P never terminates on input I.
This computational problem is undecidable!




No such general program Q can exist!
It doesn’t matter how powerful the computer is.
It doesn’t matter how much time we devote to the computation.
The proof of this involves contradiction.
38
Contradiction
start
no
Does this
algorithm end?
yes
end
39
Contradiction isn't just for
computer scientists...
40