CSE 200 Computability and Complexity
Monday, May 20, 2013
Lecture 15: Randomized Computation (cont.)
Instructor: Professor Shachar Lovett
1
Scribe: Dongcai Shen
Random Walk Algorithms for k-SAT
1.1
A random walk algorithm for 2-SAT
2-SAT. φ(x) = (x1 ∨ ¬x2 ) ∧ (x3 ∨ x1 ) ∧ · · · . φ is satisfiable if there exists an assignment a s.t. φ(a) = 1.
Algorithm 1 Random Walk for 2-SAT [2]
1: Choose r ∈ {0, 1}n randomly.
2: If φ(r) = 1, done.
3: otherwise, exists clause (e.g.) C = xi ∨ xj for which ri = rj = 0.
4: choose either i, j randomly, flip ri or rj .
5: Go to 2
Theorem 1 (Papadimitriou [2]) If φ is satisfiable, then whp after O(n2 ) steps r reaches a satisfiable
assignment.
Proof Fix a satisfying assignment a. Define the distance of r to a, denoted dist(r, a), as the number
of coordinates where ri 6= ai (this is called Hamming distance). Let r = r1 , r2 , r3 , · · · be the assignments
generated by the algorithm, and let di = dist(ri , a). Note that as we only change one bit of the assignment,
di+1 = di + ∆i where ∆i ∈ {−1, 1}. We claim that Pr[∆i = −1] ≥ 1/2. Once this is established, we have a
random walk of d1 , d2 , . . . on the range {0, 1, 2, . . . , n}, which decreases in each step with probability at least
1/2. It can be shown that such a random walk will reach 0 after O(n2 ) steps whp.
To finish, we need to prove that Pr[bi = −1] ≥ 1/2. To see that, assume the value of ri on the current
two bits is (α1 , α2 ) and the value of a is (β1 , β2 ), where α1 , α2 , β1 , β2 ∈ {0, 1} and (α1 , α2 ) 6= (β1 , β2 ) since
ri does not satisfy φ. Then, it can be verified that
• If α1 = β1 , α2 6= β2 then Pr[bi = −1] = 1/2.
• If α1 6= β1 , α2 = β2 then Pr[bi = −1] = 1/2.
• If α1 6= β1 , α2 6= β2 then Pr[bi = −1] = 1.
0
1
2
3
4
5
···
Figure 1: Random walk
This algorithm runs in O(n2 ) expectation time.
15-1
···
n
1.2
A random walk algorithm for 3-SAT
What about 3-SAT? Same algorithm. r ∈ {0, 1}n randomly. If φ(r) 6= 1, find an arbitrary clause C and flip
one of the variables of r in C. a = (0, 0, 1) → r = (0, 1, 1).
(
−1 w. prob. 31
dist time t + 1 = dist time t+
(1)
+1 w. prob. 23
1
With some careful analysis, it resolves to ( 34 )n . k-SAT’s such algorithm runs in time 2n(1−O( k )) . This is the
best known algorithm for k-SATup to the constants in the O(·).
2
List Coloring
Definition 2 (A simple path in graphs)
• Input: An undirected graph G = (V, E) and a number k.
• Question: Does G have a simple path of length k. A simple path is a sequence of vertices v1 , v2 , · · · , vk ∈
V s.t. (vi , vi+1 ) ∈ E ∀i ∈ [k − 1] and vi 6= vj ∀i 6= j.
Algorithm 2 A trivial algorithm
1: Try all combinations.
Algorithm 2’s running time is O(nk ). The following randomized algorithm (Algorithm 2) runs in time
n
· 2k .
def
Let χ : V → [k] be a random coloring. Let p = v1 v2 · · · vk be a simple path in G. We say χ is good if
χ(v1 ), χ(v2 ), · · · , χ(vk ) are all different (e.g., take all k possible colors).
O(1)
Claim 3 Pr [χ is good] ≈ e−k .
Claim 4 If χ is good, we can find a simple path in time nO(1) · 2k .
Proof of 3:
( k )k
k!
Pr [v1 , · · · , vk get all k different colors] = k ≈ e k =
k
k
k
1
e
where Stirling approximation [3] was applied.
Proof of 4:
For A ⊂ [k], define
def
SA = {v ∈ V : there is a simple path of length |A|, ends in v and takes colors in A}.
We will compute SA using dynamic programming, and then the result can be derived from S[k] .
Say we want to compute SA with |A| = a. We know already SA0 for all sets A0 of size |A0 | < a. Let
def
A = {c1 , · · · , ca }. A vertex v ∈ SA of color c ∈ A iff v is a neighbor of a vertex u ∈ SA\c . Hence we can
compute
[
SA =
v : χ(v) = c, v neighbor of u ∈ SA\{c} .
c∈A
Computing SA from {SA\{a} } takes time nO(1) . So, to compute S[k] takes 2k · nO(1) time.
15-2
3
Relations Between Randomized Complexity Classes and Other
Complexity Classes
Theorem 5 (Adelman [1]) BPP ⊆ P/poly.
Proof Suppose a language L ∈ BPP. There is a poly-time machine M (x, r) s.t.
• x ∈ L ⇒ Prr [M (x) = 1] ≥ 2/3.
• x 6∈ L ⇒ Prr [M (x) = 1] ≤ 2/3.
By amplification (majority of nO(1) independent runs), we get a new TM M 0 such that
Pr [M 0 (x, r) = L(x)] > 1 − 2−2n .
r
So there exists a fixing of the randomness r∗ such that
Pr
x∈{0,1}n
[M 0 (x, r∗ ) = L(x)] ≥ 1 − 2−2n ⇒ M 0 (x, r∗ ) = L(x) ∀x ∈ {0, 1}n
def
Let C(x) = M 0 (x, r∗ ) where r∗ is hard-wired to C.
Question 6 BPP ⊆ NP?
Theorem 7 (Sipser-Gács [4]) BPP ⊆ Σ2 ∩ Π2 .
Proof If suffices to prove that BPP ⊆ Σ2 . Apply a similar argument at the beginning of the proof of
Theorem 5, Assume there exists a TM M s.t. Prr [M (x, r) = L(x)] > 1 − 2−2n .
def
Let Sx = {r ∈ {0, 1}m : M 0 (x, r) = 1} where m = nO(1) . Then
• x ∈ L ⇒ |Sx | > (1 − 2−2n ) · 2m .
• x 6∈ L ⇒ |Sx | < 2−2n · 2m .
Claim 8 Set k = m
n.
(1) If x ∈ L, there exists y1 · · · yk ∈ {0, 1}m s.t. ∪ki=1 (Sx + yi ) = {0, 1}m .
(2) If x 6∈ L, for every y1 · · · yk ∈ {0, 1}m , ∪ki=1 (Sx + yi ) 6= {0, 1}m .
Proof of Claim 8:
Pk
(2) x 6∈ L, |Sx | < 2m−2n . | ∪ki=1 (Sx + yi )| ≤ i=1 |Sx + yi | ≤ k · |Sx | ≤ k · 2m−2n < 2m .
m
2n
(1) x ∈ L, |Sx | > 2 (1 − 2 ). Choose y1 , · · · , yk ∈ {0, 1}m randomly. Fix z ∈ {0, 1}m .
Pr z 6∈ ∪ki=1 (Sx + yi )
y1 ,··· ,yk
= Pr [z 6∈ y1 + Sx , z 6∈ y2 + Sx , · · · , z 6∈ yk + Sx ]
=
Pr
y1 ,··· ,yk
≤
[y1 , · · · , yk 6∈ Sx + z]
2m − |Sx + z|
2m
k
≤ 2−2nk = 2−2m .
Hence,
Pr ∃z ∈ {0, 1}m , z 6∈ ∪ki−1 (Sx + yi )
y1 ,··· ,yk
X
≤
Pr [z 6∈ ∪(Sx + yi )]
z∈{0,1}m
y1 ,··· ,yk
≤2m · 2−2m = 2−m .
15-3
Now,
x ∈ L ⇔ ∃y1 , · · · , yk ∈ {0, 1}m , ∀z ∈ {0, 1}m ,
k
_
i=1
M (x, yi + z) = 1
{z
}
|
yi +z∈Sx
Therefore, L ∈ Σ2 .
4
Probabilistic Constructions
Probabilistic constructions are very useful to show the existence of various combinatorial structures. We will
illustrate this with codes. An (n, k, d) binary code is a subset C ⊆ {0, 1}n with |C| = 2k where for any
x 6= y ∈ C, dist(x, y) ≥ d.
A code is considered good if k = αn, d = βn, for some constants α, β > 0. We will prove good codes exist
by a simple probabilistic argument.
def
def
Theorem 9 For any α, β > 0, small enough, there exists (n, k = αn, d = βn) codes for all large enough n.
Proof
Let C ⊂ {0, 1}n of size |C| = 2k be chosen uniformly. Then
Pr [∃x, y ∈ C, d(x, y) ≤ d]
X
≤
Pr [x, y ∈ C]
x,y∈{0,1}n
dist(x,y)≤d
d k 2
X
n
2
≤2n
2n
i
i=0
k = αn, d = βn, H(α) = α log
1
1
(1 − α) log
α
1−β
=2n · 2(H(α)+o(1))·n · 22β−2n
=2(H(α)+2β+o(1)−1)n
When α, β are small enough, H(α) + 2β < 1, almost all (n, k, d) codes are good (for large enough n).
References
[1] Leonard M. Adleman. Two theorems on random polynomial time. In FOCS, pages 75–83, 1978.
[2] Christos H. Papadimitriou. On selecting a satisfying truth assignment (extended abstract). In FOCS,
pages 163–169, 1991.
[3] Herbert Robbins. A remark on stirling’s formula. The American Mathematical Monthly, 62(1):pp. 26–29,
1955.
[4] Michael Sipser. A complexity theoretic approach to randomness. In STOC, pages 330–335, 1983.
15-4