IN210 − lecture 12
Randomized computing
Machines that can toss coins (generate
random bits/numbers)
• Worst case paradigm
• Always give the correct (best) solution
Autumn 1999
1 of 12
IN210 − lecture 12
Randomized algorithms
Idea: Toss a coin & simulate
non-determinism
Example 1: Proving polynomial
non-identities
?
(x + 2) 6= x2 + 2xy + y 2
2
?
6= x2 + y 2
• What is the “classical” complexity of the
problem?
• Fast, randomized algorithm:
— Guess values for x and y and compute
left-hand side (LHS) and right-hand side
(RHS) of equation.
— If LHS 6= RHS, then we know that the
polynomials are different.
— If LHS = RHS, then we suspect that the
polynomials are identical, but we don’t
know for sure, so we repeat the
experiment with other x and y values.
• Idea works if there are many witnesses.
Autumn 1999
2 of 12
IN210 − lecture 12
1 kr
Mynt
f (n)
witnesses
Let f (n) be a polynomial in n and let the
probability of success after f (n) steps/coin
tosses be ≥ 12 . After f (n) steps the algorithm
either
• finds a witness and says “Yes, the
polynomials are different”, or
• halts without success and says “No, maybe
the polynomials are identical”.
This sort of algorithm is called a Monte Carlo
algorithm.
Note: The probability that the Monte
Carlo algorithm succeeds after f (n)
steps is independent of input (and
dependent only on the coin tosses).
• Therefore the algorithm can be repeated
on the same data set.
• After 100 repeated trials, the probability of
failure is ≤ 2−100 which is smaller then the
probability that a meteorite hits the
computer while the program is running!
Autumn 1999
3 of 12
IN210 − lecture 12
Def. 1 A probabilistic Turing machine (PTM)
is a canonical NTM modified as follows:
• At every step computation the machine
chooses one of the two possible transitions
uniformly at random.
• The machine has two final states: qY and
qN .
Note: An algorithm on a PTM doesn’t have to
toss coins explicitly at every step, there can be
implicit coin tosses with the two possible
transitions being to the same state.
Def. 2 A PTM M is said to be a Monte Carlo
algorithm for language L if there is a
polynomial f (n) such that for all inputs x M
halts in f (|x|) steps and
• if x 6∈ L then M halts in qN .
• if x ∈ L then Pr (M halts in qY ) ≥ 12 .
Notes:
• The probability 12 can in principle be
replaced by any fixed probability > 0,
because of the technique of repeated trials.
Autumn 1999
4 of 12
IN210 − lecture 12
Notes (continued):
• The probability 12 for choosing the right
answer for positive instances must hold in
the worst case, meaning even for the most
tricky input.
• A Monte Carlo algorithm never gives
incorrect positive answers, but it can give
false NO-answers (“maybe NO”).
• Only languages that are in N P or in
Co-N P can have Monte Carlo algorithms.
Def. 3 A pair of Monte Carlo algorithms, one
for language L and one for Lc, together
compose a Las Vegas algorithm for L.
Notes:
• Only problems in N P ∩ Co-N P can have
Las Vegas algorithms. Example:
P RIMALITY.
• After 100 repetitions of a Las Vegas
algorithm on the same input, we are
“pretty sure” to get either a definite
YES-answer (from the L-algorithm) or a
definite NO-answer (from the
Lc-algorithm).
Autumn 1999
5 of 12
IN210 − lecture 12
Randomized omplexity classes
RP (randomized P)
A language belongs to class RP if it has a
Monte Carlo algorithm.
ZPP (zero error probability P)
A language belongs to class ZPP if it has a Las
Vegas algorithm.
BPP (bounded probability of error P)
A language L belongs to class BPP if it is
recognized in polynomial time by a PTM M
such that
• x ∈ L ⇒ Pr (M answers ’YES’) ≥
2
3
• x 6∈ L ⇒ Pr (M answers ’NO’) ≥
2
3
Autumn 1999
6 of 12
IN210 − lecture 12
PSPACE
Co
NP
NP
BPP
P
ZPP
RP
• The map of complexity classes augmented
with randomized classes.
• We don’t know whether the inclusions are
proper (e.g. we don’t know whether RP (
N P or RP = N P).
• It is unknown whether BPP is contained
in N P, so it has a stippled line on the map.
Applictations
Cryptography, number-theoretic comp.,
pattern matching, etc.
Autumn 1999
7 of 12
IN210 − lecture 12
Example: Combining randomizing
and average-case analysis
A “random walk” algorithm for
H AMILTONICITY with good average-case
performance:
• Idea: Try to construct a Hamiltonian path
by adding nodes at random:
— Pick one of the neighbors, x, of the
LAST-node at random.
— If x is not in the path, add x as the new
LAST-node.
— If x is already in the path, change the
path in the following way:
x
LAST
y
LAST
• The algorithm is very simple, but the
analysis is not.
• Algorithm is based upon coupon collector
pattern:
— How many coupons does it take on
average to collect n different ones?
2
O (n), O (n log n), O n , O (2n)
Autumn 1999
8 of 12
IN210 − lecture 12
Parallel computing
?
= 3×
• some problems can be efficiently
parallelized
• some problems seems inherently
sequential
Parallel machine models
• Alternating TMs
• Boolean Circuits
output
∨
∧
¬
"time" = 3
¬
inputs: x1
x2
x3
— Boolean Circuit complexity: “time”
(length of longest directed path) and
hardware (# of gates)
Autumn 1999
9 of 12
IN210 − lecture 12
• Parallell Random Access Machines
(PRAMs)
...
m0 m1 m2 m3
P0
P1
P2
...
— Read/Write conflict resolution strategy
— PRAM complexity: time (# of steps) and
hardware (# of processors)
Example: Parallel summation in time
O (log n)
m0:3 m1:5 m2:2 m3:7 m4:6 m5:1 m6:2 m7:5
P0
P1
m0 : 8
m1 : 9
time
log2 n
P2
P3
m2 : 7
m3 : 7
P0
P1
m0 : 17
m1 : 14
P0
m0 : 31
Result: Boolean Circuit complexity = PRAM
complexity.
Autumn 1999
10 of 12
IN210 − lecture 12
Limitations to parallel computing
Good news
parallel time ↔ sequential space
Example: H AMILTONICITY can easily be
solved in parallel polynomial time:
• On a graph with n nodes there are at most
n! possible Hamiltonian paths.
• Use n! processors and let each of them
check 1 possible solution in polynomial
time.
• Compute the the OR of the answers in
parallel time O (log(n!)) = O (n).
Bad news
Theorem 1 With polynomial many processors
parallel poly. time = sequential poly. time
Proof:
• 1 processor can simulate one step of m
processors in sequential
time
t1(m) = O mk , for a constant k
• Let t2(n) be the polynomial parallel time of
the computation. If m is polynomial then
t1(m) · t2(n) = polynomial.
Autumn 1999
11 of 12
IN210 − lecture 12
Parallel complexity classes
Def. 4 A language is said to be in class N Cifit
is recognized in polylogarithmic, O logk (n) ,
parallel time with polynomial hardware.
P-hard, Ex: C IRCUIT VALUE
P
NC
?
• P = NC
Autumn 1999
12 of 12