Introduction To Computational
Compelxity
http://wisdom.weizmann.ac.il/~oded/teaching.html
(please join the google group!)
We will talk about P, NP, and completeness, what is an efficient algorithm?
Search Problems – A binary relation: 𝑅 ⊆ {0,1}∗ × {0,1}∗
(
⏟
𝑥
,
𝑦
⏟
𝑝𝑟𝑜𝑏𝑙𝑒𝑚 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛
“Given instance 𝑥, find a 𝑦 such that (𝑥, 𝑦) ∈ 𝑅
Or say that none exist.
Natural Restrictions – The solution is not too long.
𝑅 is polynomially bounded.
TODO: Missing text! Fill in from analog notebook!
Topics that will be discussed
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Next lecture on 26/11
Join the Google group
Recalling last lecture
Poly-time
Reductions
Cook reductions
Karp & Levin-reductions
Self-reducibility
NP-Completeness
Self-Reducibility of NP-complete problems
𝑅 ⊆ {0,1}∗ × {0,1}
𝑅(𝑋) = {𝑦: (𝑥, 𝑦) ∈ 𝑅}
𝑆𝑅 = {𝑥: 𝑅(𝑥) ≠ 𝜙}
𝑃 = 𝑁𝑃 ↔ 𝑃𝐶(𝐹𝑁𝑃) ⊆ 𝑃𝐹(𝐹𝑃)
𝑆𝑅′ = {⟨𝑥, 𝑦 ′ ⟩: ∃𝑦 ′′ 𝑠. 𝑡. (𝑥, 𝑦 ′ , 𝑦 ′′ ) ∈ 𝑅}
)∈𝑅
Reduction of 𝐵 to 𝐴:
Any procedure that performs an efficient algorithm that solves 𝐴 to an efficient algorithm that
solves 𝐵.
Cook Reductions
A cook reduction from 𝐵 to 𝐴 –An efficient algorithm that solves 𝐵 using a black box for solving
𝐴. (Informal definition)
An oracle machine 𝑀 has oracle access to a function 𝑓.
Denoted: 𝑀 𝑓 .
Formal definition:
A polynomial time Turing machine 𝑀 𝐴 that solves 𝐵.
Clique
Clique= {⟨⟨𝐺, 𝑘⟩, 𝐶⟩||𝐶| ≥ 𝑘, 𝐶 𝑖𝑠 𝑎 𝑐𝑙𝑖𝑞𝑢𝑒}
Given a graph 𝐺 = (𝑉, 𝐸) a clique 𝐶 is a subset of 𝑉 such that all vertices have edges between
each other.
INDEP-SET={⟨⟨𝐺, 𝑘⟩, 𝐼⟩||𝐼| ≥ 𝑘, 𝐼 𝑖𝑠 𝑎𝑛 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑠𝑒𝑡}
An independent set, is a subset of vertices such that no pair have an edge between them.
Karp Reductions
Given two sets 𝑆1 , 𝑆2 ⊆ {0,1}∗, a Karp reduction of 𝑆1 to 𝑆2 is a polynomial time computable
𝑓: {0,1}∗ → {0,1}∗ s.t. ∀𝑥: 𝑥 ∈ 𝑆1 ↔ 𝑓(𝑥) ∈ 𝑆2
Levin Reductions
Levin Reductio: 𝑅1 , 𝑅2 ⊆ {0,1}∗ × {0,1}∗.
A levin rediction of 𝑅1 to 𝑅2 is 𝑓: {0,1}∗ → {0,1}∗ , 𝑔: {0,1}∗ × {0,1}∗.
Such that
𝑅1 (𝑥) ≠ 𝜙 ↔ 𝑅2 (𝑓(𝑥)) ≠ 𝜙
𝑔(𝑥, 𝑦) ∈ 𝑅1 (𝑥) ↔ 𝑦 ∈ 𝑅2 (𝑓(𝑥))
Self Reducibility
𝑅 ⊆ {0,1}∗ × {0,1}∗ is self reducible if 𝑅 is cook reducible to the corresponding decision
problem.
Example – SAT
𝑋1 , … , 𝑋𝑛 - Boolean variables
⊢,∧,∨ - operators
SAT = {(𝜙, 𝑎)|𝜙(𝑎) = 𝑡𝑟𝑢𝑒} where a is an assignment for all variables.
CNF formulas are formula’s of the form:
𝐶1 ∧ 𝐶2 , … ,∧ 𝐶𝑘
𝐶𝑖 = (𝑙1 ∨ 𝑙2 ∨ … ∨ 𝑙𝑛 )
TODO: Write how to self-reduce SAT
NP Hardness
A decision problem 𝑆 ⊂ {0,1}∗ is NP-Hard iff every 𝑆 ′ ∈ 𝑁𝑃 reduces to it.
NP-Completeness: 𝑆 is NP-Complete if it is both in NP and it is NP-Hard.
Bounded-Halting
𝑆 ∈ 𝑁𝑃 ↔ ∃𝑝𝑜𝑙𝑦𝑡𝑖𝑚𝑒 𝑀 and 𝑝𝑜𝑙𝑦𝑛𝑜𝑚𝑖𝑎𝑙 𝑝: ℕ → ℕ
𝑠 = {𝑥|∃𝑦 ∈ {0,1}∗ , |𝑦| ≤ 𝑝(|𝑥|), 𝑀(𝑥, 𝑦) 𝑎𝑐𝑐𝑒𝑝𝑡𝑠}
An instance of bounded halting could be a Turing machine 𝑀, a string 𝑥, and a string 1𝑡 .
Solution: 𝑦 𝑠. 𝑡. 𝑀 𝑎𝑐𝑐𝑒𝑝𝑡𝑠 (𝑥, 𝑦), |𝑦| ≤ 𝑡, in ≤ 𝑡 steps.
Reduction: 𝑥 → ⟨𝑀𝑆 , 𝑥, 1𝑡𝑠 (|𝑥|+𝑃𝑆(|𝑥|)) ⟩
Claim: ∀𝑅 𝑠. 𝑡. 𝑆𝑅 is NP-Complete, 𝑅 is cook-reducible to 𝑆𝑅′ .
Proof: 𝑅 cook reduces to 𝑆𝑅′ , 𝑆𝑅′ ∈ 𝑁𝑃 → 𝑆𝑅′ reduces to 𝑆𝑅 .
----- end of lesson 2
Theorem: There exists NP-Complete problems.
𝑅𝑈 = {(𝑀, 𝑥, 1𝑡 ), 𝑦|𝑀 𝑎𝑐𝑐𝑒𝑝𝑡𝑠 (𝑥, 𝑦) 𝑖𝑛 𝑎𝑡 𝑚𝑜𝑠𝑡 𝑡 𝑠𝑡𝑒𝑝𝑠, |𝑦| ≤ 𝑡} ∈ 𝑃𝐶
𝑡 < |(𝑀, 𝑥, 1𝑡 )|
𝑆𝑢 = {(𝑚,
⏟ 𝑥, 1𝑡 ) |∃𝑦 (𝑥̅ , 𝑦) ∈ 𝑅𝑢 } ∈ 𝑁𝑃
𝑥̅
Let 𝑅 ∈ 𝑃𝐶 → 𝑆𝑅
Let
𝑃𝑅 - polynomially bound solution length
𝑀𝑅 - Recognizing machine
𝑡𝑅 - Polynomially bounded number true of 𝑀𝑅
∆
𝑥 → 𝑓𝑅 (𝑥) = (𝑀𝑅 , 𝑥, 1𝑡𝑅(|𝑥|←𝑃𝑅(|𝑥|))
We shall prove: ∀(𝑥, 𝑦) ∈ 𝑅 ⇔ (𝑓𝑅 (𝑥), 𝑦) ∈ 𝑅𝑢
Proposition 1: CSAT is NP-Complete.
∆
∀𝑅 ∈ 𝑃𝐶 ∃Levin Reduction of 𝑅 to 𝑅𝐶𝑆𝐴𝑇 = {(𝐶, 𝑦)|𝐶(𝑦) = 1}
Proposition 2: If 𝑅1 is PC-Complete and 𝑅1 Levin Reduces to 𝑅2 ∈ 𝑃𝐶, then 𝑅2 is NP-Complete.
By the first part of the condition: 𝑃𝐶 ∝ 𝑅1 ∝ 𝑅2 , ∀𝑅 ∈ 𝑃𝐶
Proposition 3:
(1) 𝑅𝑆𝐴𝑇 is in PC
(2) 𝑅𝐶𝐴𝑆𝑇 ∝ 𝑅𝑆𝐴𝑇
Proof of proposition 1:
𝑅 ∈ 𝑃𝐶 → 𝑃𝑅 , 𝑀𝑅 , 𝑡𝑅
𝑥 → 𝐶𝑥
(𝐶𝑥 , 𝑦) ∈ 𝑅𝐶𝑆𝐴𝑇
∀𝑥, 𝑦 (𝑥, 𝑦) ∈ 𝑅 ⇔ ⏟
𝐶𝑥 (𝑦)=1
Just as a mental thought, consider the computation of 𝑀𝑅 (𝑥, 𝑦) up to 𝑡𝑅 (|𝑥| + 𝑝𝑅 (𝑥)) steps
TODO: draw drawing…
∆
𝑅𝑆𝐴𝑇 = {(𝜑, 𝜏)|𝜑(𝜏) = 1}
𝑅𝐶𝑆𝐴𝑇 ∝ 𝑅𝑆𝐴𝑇
Promise Problems
Promise is easy to verify.
Promise = set of instances.
1. Make invalid instances easy to solve by introducing a trivial solution.
2. Map it to an arbitrary valid instance
∆
𝑅𝑆𝐴𝑇 = {(𝜑, 𝜏)|𝜑(𝜏) = 1}
𝑅 ∈ 𝑃𝐶,
𝑆𝑅
Candid search problem of 𝑅:
Promise: Instance 𝑥 ∈ 𝑆𝑅
Task: Find 𝑦 ∈ 𝑅(𝑥) ≠ 0
Theorem: For every 𝑅 ∈ 𝑃𝐶, ∃polynomial 𝑝 and an algorithm 𝐴0 that solves the candid search
problem or 𝑅 such that ∀ algorithm 𝐴 that solves the candid search problem of 𝑅
∀𝑥 𝑡𝐴0 (𝑥) = 𝑂̃(𝑡𝐴 (𝑥) + 𝑝(|𝑥|))
For every 𝑗 = 1 …
For every 𝑖 = 1, … , √2𝑗
𝑗
Enumerate the I’th machine for at most [2 ⁄(𝑖
+ 1)2 ]
Suppose 𝐴 solves the candid search problem in time 𝑇(𝐴)
∆
Let 𝑖𝐴 = 𝑖𝑛𝑑𝑒𝑥 𝑜𝑓 𝐴 𝑖𝑛 𝑡ℎ𝑒 𝑒𝑛𝑢𝑚𝑒𝑟𝑎𝑡𝑖𝑜𝑛 ≈ 2|𝐴|
1
(𝑖𝐴 + 1)2
The𝑂̃ hides a constant that depends on 𝐴! And probably exponential of 𝐴.
----- End of lesson 3
Circuit Complexity
∗
∗
Infinite family of circuits {𝐶𝑛 }∞
𝑛=1 computes 𝑓: {0,1} → {0,1}
If ∀𝑥 it holds that 𝐶|𝑥| (𝑥) = 𝑓(𝑥)
{𝐶𝑛 }∞
𝑛=1 has size 𝑆(𝑛) ⇔ ∀𝑛. |𝐶𝑛 | ≤ 𝑆(𝑛)
We want to prove: 𝑁𝑃 ⊈ 𝑃⁄𝑃𝑜𝑙𝑦
Prove that 𝑃⁄𝑃𝑜𝑙𝑦 can even solve undecidable problems.
𝑆 ′ = {1𝑛 |𝐵𝑖𝑛𝑎𝑟𝑦 𝑒𝑥𝑝𝑎𝑛𝑠𝑖𝑜𝑛 𝑜𝑓 𝑛 𝑖𝑠 𝑖𝑛 𝑆}
{𝐶𝑛 }∞
𝑛=1
Just build the infinite series that for every 1𝑛 in 𝑆 returns one and otherwise returns 0.
∗
Turing machine 𝑀 that takes advice {𝑎𝑛 }∞
𝑛=1 ⊆ {0,1} computes 𝑓 if ∀𝑥 it holds that
𝑀(𝑥, 𝑎|𝑥| ) = 𝑓(𝑥) Advice length 𝑙(𝑛) if ∀𝑛 |𝑎𝑛 | ≤ 𝑙(𝑛)
𝑃⁄
𝑃𝑜𝑙𝑦 – 𝑆 that are decided by a circuit family of polynomial size.
Claim: 𝑃⁄𝑃𝑜𝑙𝑦 that are decided by poly-time machines that take advice of polynomial length.
𝑆 has poly-size family {𝐶𝑛 }𝑛
Advice {𝐶𝑛 }𝑛
And the machine would be:
𝑀(𝑥, 𝐶|𝑥| ) = 𝐶|𝑥| (𝑥)
Suppose 𝑆 has poly-time 𝑀 that takes poly-length {𝑎𝑛 }𝑛
{𝐷𝑛 } 𝐷𝑛 (𝑦) = 𝑀(𝑦)
{𝐶𝑛 }𝑛 𝐶𝑛 (𝑥) = 𝐷(|𝑥| +|𝑎 ) (𝑥, 𝑎|𝑥| )
|𝑥|
{𝐶𝑛 }∞
𝑛=1 is uniform, if ∃𝑀 that on input 𝑛 outputs 𝑐𝑛 in time polynomial in the size of 𝑐𝑛 .
Describe 𝑃⁄𝑙 - a class of Turing machines that take advice of length 𝑙.
∀𝑆 ⊆ {0,1}∗
Denote 𝑆 = {0,1}∗ \𝑆
∀𝐶 (complexity class), denote by 𝑐𝑜𝐶 the class of complementary set:
𝑐𝑜𝐶 = {𝑆̅|𝑆 ∈ 𝐶}
𝑆 ∈ Σ2 ⇔ ∃poly time 𝑉 such that 𝑆 = {𝑥|∃𝑦1 ∀𝑦2 𝑣(𝑥, 𝑦, 𝑦2 ) 𝑎𝑐𝑐𝑒𝑝𝑡𝑠}
𝑆 ∈ Π2 ⇔ ∃poly time 𝑉 such that 𝑆 = {𝑥|∀𝑦1 ∃𝑦2 𝑣(𝑥, 𝑦, 𝑦2 ) 𝑎𝑐𝑐𝑒𝑝𝑡𝑠}
𝑠 ∈ Σ𝑘 ⇔ ∃poly time 𝑣 such that 𝑆 = {𝑥|∃𝑦1 ∀𝑦2 … 𝑄𝑘 𝑦𝑘 𝑣(𝑥, 𝑦1 , … , 𝑦𝑘 )}
𝑠 ∈ Π𝑘 ⇔ ∃poly time 𝑣 such that 𝑆 = {𝑥|∀𝑦1 ∃𝑦2 … 𝑄𝑘 𝑦𝑘 𝑣(𝑥, 𝑦1 , … , 𝑦𝑘 )}
𝑃 = Σ0 = Π0
𝑁𝑃 = Σ1
𝑐𝑜𝑁𝑃 = Π1
Π𝑘 = 𝑐𝑜Σ𝑘
Conjecture:
∀𝑘 Σ𝑘 ⊊ Σ𝑘+1
∀𝑘 Π𝑘 ⊊ Σ𝑘+1
PH = ∪𝑘 Σ𝑘 =∪𝑘 Π𝑘
Σ𝑘 -Complete Problem:
Given 𝜑𝑦1 , … , 𝑦𝑘
Σ𝑘 − 𝑆𝐴𝑇 − ∃𝑦1 ∀𝑦2 … 𝑄𝑘 𝑦𝑘 𝜑(𝑦1 , … , 𝑦𝑘 ) = 1
𝑆 ∈ Σ𝑘+1 ⇔ ∃𝑆 ′ ∈ Π𝑘 𝑆 = {𝑥|∃𝑦(𝑥, 𝑦) ∈ 𝑆}
Σ𝑘 = Π𝑘 ⇒ ∀𝑘 ′ ≥ 𝑘Σ𝑘 ′ = Σ𝑘 ′ +1
𝑆 ∈ 𝑁𝑃𝑐
∃poly time 𝑣 ∃𝑆 ′ ∈ 𝐶 such that
′
𝑆 = {𝑥|∃𝑦𝑣 𝑆 (𝑥, 𝑦) 𝑎𝑐𝑐𝑒𝑝𝑡𝑠}
Σ𝑘+1 = 𝑁𝑃Σ𝑘 = 𝑁𝑃Π𝑘
Karp-Lipton Theorem
Under the assumption that Σ2 ≠ Π2 ⇒ 𝑁𝑃 ⊈ 𝑃⁄𝑃𝑜𝑙𝑦
We will prove the other way: 𝑁𝑃 ⊆ 𝑃⁄𝑃𝑜𝑙𝑦 ⇒ Σ2 = Π2
Suppose 𝑁𝑃 ⊆ 𝑃⁄𝑃𝑜𝑙𝑦
Take Π2 − 𝑆𝐴𝑇 = {𝜑|∀𝑦1 ∃𝑦2 𝜑(𝑦1 , 𝑦2 ) = 1}
∃𝑉 ′ Π2 − 𝑆𝐴𝑇 = {𝜑|∃𝑦1′ ∀𝑦2′ 𝑉 ′ (𝜑, 𝑦1′ , 𝑦2′ )}
𝑆 = {⟨𝜑, 𝑦1 ⟩|∃𝑦2 𝜑(𝑦1 , 𝑦2 )}
𝑆 ∈ 𝑁𝑃 ⇒ 𝑆 ∈ 𝑃⁄𝑃𝑜𝑙𝑦 ⇒ {𝐶𝑛 }∞
𝑛−1 decides 𝑆.
Π2 − 𝑆𝐴𝑇 = {𝜑|∀𝑦1 𝐶𝑛 (𝜑, 𝑦1 ) = 1}
A second attempt:
Π2 − 𝑆𝐴𝑇 = {𝜑|∃𝐶∀𝑦1 𝐶(𝜑, 𝑦1 ) = 1}
But this is not true since can get any circuit.
We know 𝑆𝐴𝑇 ∈ 𝑁𝑃 ⇒ 𝑆𝐴𝑇 ∈ 𝑃⁄𝑃𝑜𝑙𝑦 ⇒ {𝐷𝑛 }𝑛 deciding 𝑆𝐴𝑇.
Denote {𝐷𝑛′ } find satisfying assignments to 𝑆𝐴𝑇 or output ⊥(since 𝑆𝐴𝑇 is self reducible).
So now:
Π2 − 𝑆𝐴𝑇 = {𝜑|∃𝐷′ ∀𝑦, 𝐷 ′ (𝜑𝑌1 =𝑦1 ) = 𝑦2 ∧ 𝜑(𝑦1 , 𝑦2 ) = 1}
⇔ 𝑣 ′ (𝜑, 𝐷 ′ , 𝑦1 ) = 1
Π2 − 𝑆𝐴𝑇 = {𝜑|∃𝐷′ ∀𝑦1 . 𝑣 ′ (𝜑, 𝐷 ′ , 𝑦1 ) 𝑎𝑐𝑐𝑝𝑒𝑡𝑠}
---- end of lesson 4
𝐷𝑇𝑖𝑚𝑒(𝑡) - Class of sets decidable in 𝑡 steps by a 2-tape TM
Is 𝐷𝑇𝑖𝑚𝑒(𝑛) ⊊ 𝐷𝑇𝑖𝑚𝑒(2𝑛 )?
Is 𝐷𝑇𝑖𝑚𝑒(𝑡) ⊊ 𝐷𝑇𝑖𝑚𝑒(2𝑡 ) for ∀𝑡: ℕ → ℕ?
For a nice 𝑡 (nice will be defined soon): 𝐷𝑇𝑖𝑚𝑒(𝑡) ⊊ 𝐷𝑇𝑖𝑚𝑒(𝑂(𝑡 log 𝑡))
A function 𝑡: ℕ → ℕ is time-constructible, if it can be computed in time 𝑡.
For circuits all functions are nice.
Time Hierarchy Theorem
∀ Time constructible 𝑡: ℕ → ℕ, 𝐷𝑇𝑖𝑚𝑒(𝑡) ⊊ 𝐷𝑇𝑖𝑚𝑒(𝑡 ∙ log 𝑡)
We will prove a weaker version: 𝐷𝑇𝑖𝑚𝑒(𝑡) ⊊ 𝐷𝑇𝑖𝑚𝑒(𝑛 ∙ 𝑡 ∙ log 𝑡)
Proof:
Denote 𝑋𝑀 - Description of 𝑀
𝑆 = {𝑋𝑀 |𝑀(𝑋𝑀 ) 𝑟𝑒𝑗𝑒𝑐𝑡𝑠 𝑖𝑛 t(|X M |) steps}
Suppose 𝑆 is decided by machine 𝑀 in time 𝑡.
What is the result of invoking 𝑀(𝑋𝑀 )?
If 𝑀(𝑋𝑀 ) accepts ⇒ 𝑋𝑀 ∉ 𝑆
If 𝑀(𝑋𝑀 ) rejects ⇒ 𝑋𝑀 ∈ 𝑆
A general fact: The universal machine can simulate a machine M for 𝑡 steps in time
𝑂(𝑋𝑀 ∙ 𝑡 ∙ log 𝑡)
Time Gap Theorem
For every computable non-decreasing 𝑔: ℕ → ℕ there exists a computable non decreasing and
non-time constructible 𝑡: ℕ → ℕ such that 𝐷𝑇𝑖𝑚𝑒(𝑡) = 𝐷𝑇𝑖𝑚𝑒(𝑔(𝑡))
We can choose 𝑔 to be 𝑔(𝑡) = 2𝑡
Fix 𝑔, Let 𝑀1 , 𝑀2 , … be all Turing Machines
𝑡𝑖 (𝑛) - the time bound of 𝑀𝑖 on inputs of length 𝑛.
𝑡(𝑛) =
max
{𝑡𝑖 (𝑛), 𝑡(𝑛 − 1)} + 1
1≤i≤n,ti (n)≠∞
Let 𝑆 ∈ 𝐷𝑇𝑖𝑚𝑒(𝑔(𝑡)), so 𝑆 is decided by some 𝑀𝑖 in time 𝑔(𝑡)
∀𝑛 ≥ 𝑖, it holds that 𝑡(𝑛) > 𝑡𝑖 (𝑛)
So ∀(𝑛 ≥ 𝑖), 𝑀 decides 𝑆 in time 𝑡.
∃𝑀 that decides 𝑆 in time 𝑡 for ∀n
Oded steps in:
There exists a problem 𝑆.
∀ algorithm that solves 𝑆, there exists an algorithm that solves 𝑆 much faster
𝑀𝑎𝑥𝐶𝑙𝑖𝑞𝑢𝑒(𝐺, 𝑘) = is maximum clique size 𝑘?
is maximum clique size ≥ 𝑘 is 𝑁𝑃 complete!
Is maximum clique size ≤ 𝑘 is 𝑐𝑜 − 𝑁𝑃 complete!
Space Complexity
Space complexity ~ the number of cells of work device visited during computation.
But since you can increase the alphabeit, we usually multiply it by log 2|Σ| (and we denote it as
binary space).
We also add log 2|𝑄| but this is just a constant…
Focus: Log-Space
Question: What’s the smallest amount of space that is more powerful then constant space?
There is a set: 𝑆 = {𝑤𝑘 |𝑘 ∈ ℕ} 𝑤𝑘 = 2𝑘 strings of length 𝑘 𝑤1 = 0 ∗ 1 𝑤2 = 00 ∗ 01 ∗ 10 ∗ 11
The machine only needs log 𝑘 to check the input is of the form. But 𝑘 = log 𝑛. So the machine
needs log log 𝑘. Also it checks whether the given string is of the right form and if not is uses only
log log 𝑘 space as well.
Time VS Space
In time, if you have a reduction, you can simply execute the reduction, then execute the second
function on the reduction. However, when we’re talking about space, we need to store the
damn thing somewhere.
However, we can recycle space!!
A non obvious ideal:
TODO: A lot of drawings. Baaahhhh…
A connection between space and time:
𝐷𝑆𝑝𝑎𝑐𝑒(𝑠) ⊆ 𝐷𝑇𝑖𝑚𝑒(𝑛 ∙ 2𝑠(𝑛) )
Corollary: Define 𝐿 as what can be done in log space – 𝐿 ⊆ 𝑃
Theorem: For any algorithm 𝐴 that halts on each input, it’s time complexity is at most 𝑛 ∙
2𝑠(𝑛)+log 𝑛+log 𝑠(𝑛)
Proof: Fix 𝑥, What is needed to store for the configuration?
𝑠(𝑛) + log 𝑛 + log 𝑠(𝑛)
--- End of lesson 5
Theorem: CircuitEval is P-Complete.
Proof: 𝑆 ∈ 𝑃 → 𝑀𝑠 , 𝑡𝑠
Draw Pseudo Cook reduction for circuits
𝑖𝑛 log 𝑠𝑝𝑎𝑐𝑒
Reduct: 𝑥 →
𝑝𝑜𝑙𝑦 𝑡𝑖𝑚𝑒
{𝐶𝑛 }𝑛∈ℕ 𝑛 →
(𝐶|𝑥| , 𝑥)
𝐶𝑛
∀𝑆 ∈ 𝑃 ∃ Uniform family of poly-size circuits.
NSpace VS NP
𝑆 ∈ 𝑁𝑃 ⇔ ∃polynomial 𝑝 and ∃ poly-time algorithm 𝐴 such that
∀𝑥 ∈ 𝑆 ⇔ ∃𝑦 ∈ {0,1}𝑃(𝑥) 𝐴(𝑥, 𝑦) = 1
𝐴 runs in time polynomial in |𝑥|
𝑆 ∈ 𝑁𝑆𝑝𝑎𝑐𝑒[𝑥]
Basic model is: Main input 𝑥, aux. input, or “witness”/”proof” 𝑦.
Off-line model: free(read) access to 𝑦 or bidirectional access.
Are these walks (free read access) helpful?
They are very helpful at times.
For instance, look at the following language: 𝑆 = {𝑥 ∗ 𝑥|𝑥 ∈ {0,1}∗}
On-line model: There is a unidirectional access to 𝑦.
If we want it or not, this divides the world into two non-deterministic space groups:
𝑁𝑆𝑝𝑎𝑐𝑒𝑂𝐹𝐹 (𝑠) and 𝑁𝑆𝑝𝑎𝑐𝑒𝑂𝑁 (𝑠)
𝑁𝑆𝑝𝑎𝑐𝑒𝑂𝑁 (𝑠) ⊆ 𝑁𝑆𝑝𝑎𝑐𝑒𝑂𝐹𝐹 (𝑠)
∀𝑠 at least linear: 𝑁𝑆𝑝𝑎𝑐𝑒𝑂𝑁 (𝑠) = 𝑁𝑠𝑝𝑎𝑐𝑒𝑂𝐹𝐹 (𝜃(log 𝑠))
Theorem: 𝑁𝑆𝑝𝑎𝑐𝑒𝑂𝑁 (𝑠) ⊆ 𝐷𝑇𝑖𝑚𝑒(2𝑠 ∙ 𝑠 ∙ 𝑛)
𝑑𝑒𝑓
𝑁𝐿 = 𝑁𝑆𝑝𝑎𝑐𝑒𝑂𝑁 (𝑂(log 𝑛)) ⊆ 𝑃
Directed Connectivity is 𝑁𝐿 complete.
Undirected Connectivity ∈ 𝐿!
𝐷𝐶𝑜𝑛𝑛 ≈ 𝑁𝐿
Input: Graph 𝐺 and pair or vertices 𝑠, 𝑡
Question: Is there a directed path between 𝑠 and 𝑡?
Theorem: 𝑠𝑡 − 𝑐𝑜𝑛𝑛 is NL-Complete (under log space reductions)
𝑙𝑜𝑘𝑔 𝑠𝑝𝑎𝑐𝑒
∀𝑆 ∈ 𝑁𝐿,
𝑆→
𝑀 on-line log space
𝑆𝑇 − 𝐶𝑂𝑁𝑁
𝑥 ∈ 𝑆 ⇒ ∃ computation of 𝑀(𝑥) that accepts.
Theorem: 𝑁𝐿 ⊆ 𝐷𝑆𝑝𝑎𝑐𝑒(log 2 𝑛)
Will show 𝑆𝑇 − 𝐶𝑂𝑁𝑁 ∈ 𝐷𝑆𝑝𝑎𝑐𝑒(log 2 𝑛)
Input: Grah 𝐺, vertices 𝑠, 𝑡
Question: Is there a directed path from 𝑠 to 𝑡.
Length of the path is always smaller of equal to the number of vertices in the graph.
𝑑𝑒𝑓
𝜙𝐺 (𝑢, 𝑣, 𝐿) = ∃ a path of length < 𝐿 from 𝑢 to 𝑣.
𝜑𝐺 (𝑢, 𝑣2𝑙) = ⋁ 𝜙𝐺 (𝑢, 𝑤, 𝑙) ∧ 𝜙𝐺 (𝑤, 𝑣, 𝑙)
𝑤
For every 𝑤 ∈ 𝑉
𝜎 ← 𝜙𝐺 (𝑢, 𝑤, 𝑙) - a recursive call
𝜎 ← 𝜎 ∧ 𝜙𝐺 (𝑤, 𝑣, 𝑙)
𝑆𝑝𝑎𝑐𝑒(2𝑙) = log|𝑉| + 1 + 𝑆𝑃𝐴𝐶𝐸(𝑙)
So we get that we can do it in log 2 𝑛
The general statement is: 𝑁𝑆𝑃𝐴𝐶𝐸(𝑠) ⊆ 𝑁𝑆𝑃𝐴𝐶𝐸(𝑠 2 )
𝑁𝐿 = 𝑐𝑜𝑁𝐿 – {{0,1}∗ \𝑆|𝑆 ∈ 𝑁𝐿}
Equivalent: ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
𝑆𝑇 − 𝐶𝑂𝑁𝑁 log space reduced to 𝑆𝑇 − 𝐶𝑂𝑁𝑁.
Notion: Non-Deterministically computing a function.
There exists an algorithm 𝐴 such that:
∀𝑥:
1) ∀𝑦 𝐴(𝑥, 𝑦) ∈ {𝑓(𝑥), ⊥} - ⊥∉ {0,1}∗
2) ∃𝑦 𝐴(𝑥, 𝑦) = 𝑓(𝑥)
Property: 𝑁𝐿 = 𝑐𝑜𝑁𝐿 iff ∀𝑆 ∈ 𝑁𝐿 such that 𝜒𝑆 is non-deterministically computable in log space.
Proof:
(1)
𝑁𝐿 = 𝑐𝑜𝑁𝐿 → 𝑆 ∈ 𝑁𝐿 → 𝑅, 𝑥 ∈ 𝑆 𝑖𝑓𝑓 ∃𝑦. (𝑥, 𝑦) ∈ 𝑅
𝑆̅ ∈ 𝑁𝐿 → 𝑅0 𝑥 ∉ 𝑆 𝑖𝑓𝑓 ∃𝑦 (𝑥, 𝑦) ∈ 𝑅
Algorithm (x,𝜎, 𝜔)= if (𝑥, 𝜔) ∈ 𝑅𝜎 then output 𝜎. Otherwise output ⊥.
(3) 𝐴 non-deterministically computes 𝜒𝑆 → 𝑆 ∈ 𝑁𝐿 ∩ 𝑐𝑜𝑁𝐿
Theorem: Log-Space is non-deterministically computing of 𝜒𝑆𝑇−𝐶𝑂𝑁𝑁
Reachability: Input is a directed graph 𝐺(𝑉, 𝐸) and a vertex 𝑆.
Output = the number of vertices reachable from 𝑆 in the graph 𝐺.
Log space reduction to counting the number of reachable vertices.
Fix 𝐺 and 𝑆. Also denote 𝑛 = |𝑉|.
𝑅𝑖 = {𝑣|∃ 𝑑𝑖𝑟𝑒𝑐𝑡𝑒𝑑 𝑝𝑎𝑡ℎ 𝑜𝑓 𝑙𝑒𝑛𝑔𝑡ℎ 𝑖 𝑓𝑟𝑜𝑚 𝑠 𝑡𝑜 𝑣}
𝑅0 = {𝑆}
We want to know |𝑅𝑛 |
For 𝑖 = 1 to 𝑛 = |𝑁|
- Guess the size of 𝑅𝑖 , denote my guess by 𝑔.
- Verify that |𝑅𝑖 | ≥ 𝑔 - Loop all vertices and check that it is reachable by 𝑖 steps.
- Verify that |𝑅𝑖 | ≤ 𝑔 ⇔ |𝑉 − 𝑅𝑖 | ≥ 𝑛 − 𝑔 - “enumerate 𝑅𝑖−1 and for each 𝑤 in 𝑅𝑖−1
I verify the following – 𝑢 ≠ 𝑤 and that ⟨𝑒, 𝑢⟩ ∉ 𝐸
𝑁𝑃 ⊆ 𝑃𝑆𝑃𝐴𝐶𝐸,
𝑐𝑜𝑁𝑃 ⊆ 𝑃𝑆𝑃𝐴𝐶𝐸
We also know 𝑃𝐻 ⊆ 𝑃𝑆𝑃𝐴𝐶𝐸
𝑆 ∈ Σ𝑘 take the polytime V:
𝑆 = {𝑥: ∃𝑦1 ∀𝑦2 … 𝑄𝑘 𝑦𝑘 𝑉(𝑥, 𝑦1 , … , 𝑦𝑘 ) accepts}
𝑄𝐵𝐹 − quantified Boolean formula.
Input: 𝑄1 𝑥1 𝑄2 𝑥2 … 𝑄𝑚 𝑥𝑚 𝜑(𝑥1 , … , 𝑥𝑚 )
Output: Yes ⇔ this is a true statement
Theorem: 𝑄𝐵𝐹is PSPACE-complete
Let 𝑆 ∈ 𝑃𝑆𝑃𝐴𝐶𝐸. Let 𝑀 be the machine that decides 𝑆. Let 𝑥 ∈ {0,1}𝑛
𝐺 = 𝐺𝑀,𝑋 - Configuration graph of 𝑀 on 𝑥.
𝜙𝐺 (𝑢, 𝑣, 𝑙) – true ⇔ ∃ a path of length ≤ 𝑙 from 𝑢 to 𝑣.
𝐿 = 2⌈log2 |𝑉𝜙 |⌉
𝑙
𝑙
𝜙𝐺 = ∃𝑤 ∈ 𝑉𝐺 . 𝜙 (𝑢, 𝑤, ) ∧ 𝜙𝐺 (𝑤, 𝑣, )
2
2
𝑙
𝜙(𝑢, 𝑣, 𝑙) = ∃𝑤 ∈ 𝑉𝐺 ∀(𝑢′ , 𝑣 ′ ) ∈ {(𝑢, 𝑤), (𝑤, 𝑣)}𝜙𝐺 (𝑢′ , 𝑣 ′ , )
2
𝑙
∃𝑤 ∈ 𝑉𝐺 ∀𝑏 ∈ {0,1}∃𝑢′ 𝑣 ′ ∈ 𝑉𝐺 ∧ 𝜙𝐺 (𝑢′ , 𝑣 ′ , )
𝑠
[(𝑏 = 0) ∧ (𝑢′ = 𝑢) ∧ (𝑣 ′ = 𝑤) ∨ (𝑏 = 1) ∧ (𝑢′ = 𝑤) ∧ (𝑢′ = 𝑣]
Games
∃𝑥1 ∀𝑥2 … 𝑄𝑛 𝑋𝑛 𝜑(𝑥1 , … , 𝑥𝑚 )
--- end of lesson
Standard random variables take real numbers. We will allow ourselves to do whatever we want.
𝑈𝑛 – The random variable that takes random strings of length 𝑛 and is uniform.
Δ(𝑋1 , 𝑋2 ) = max{Pr[𝑋1 ∈ 𝑆] − Pr[𝑋2 ∈ 𝑆]}
S
𝑋1 , 𝑋2 are independent iff ∀𝑋1 , 𝑋2 Pr[𝑋1 = 𝑥1 , 𝑋2 = 𝑥2 ] = Pr[𝑋1 = 𝑥1 ] ∙ Pr[𝑋2 = 𝑥2 ]
𝑋1 , … , 𝑋𝑛 are 𝑘-wise independent iff every 𝑘 variables of this set are independent.
If 𝑋1 , … , 𝑋𝑛 are pair-wise independent, then 𝑉[𝑋1 + ⋯ + 𝑋𝑛 ] = 𝑉[𝑋1 ] + ⋯ + 𝑉[𝑋2 ]
𝐸[𝑋 𝑘 ] is called the 𝑘’th moment of 𝑋.
1
1000 𝑤. 𝑝. 2
𝑋={
1:
−1000 𝑤. 𝑝. 2
Markov Inequality: If 𝑋 is non-negative, then:
1
Pr[𝑥 > 𝑘𝐸[𝑥]] <
𝑘
Or
𝐸[𝑋]
Pr[𝑋 > 𝑘] <
𝑘
𝐸[𝑋] = ∑ 𝑥 ∙ Pr[𝑋 = 𝑥] > ∑ 𝑘 ∙ Pr[𝑋 = 𝑥] = 𝑘 ∙ Pr[𝑥 > 𝑘]
𝑥
𝑥>𝑘
Chebyshev Inequality
Pr[|𝑋 − 𝐸[𝑋]| > 𝜖] ≤
𝑉[𝑋]
𝜖2
Proof: Pr[|𝑋 − 𝐸[𝑋]| > 𝜖] = Pr[(𝑋 − 𝐸[𝑋])2 > 𝜖 2 ]
𝐵𝑦 𝑀𝑎𝑟𝑘𝑜𝑣
≤
𝐸[(𝑋−𝐸[𝑋])2 ]
𝜖2
=
𝑉[𝑋]
𝜖2
𝑋1 , … , 𝑋𝑛 all have expectations 𝜇 and variance 𝜎 2
1
Pr [| ∑ 𝑋𝑖 − 𝜇| > 𝜖] =?
𝑛
𝑖
If 𝑋1 , … , 𝑋𝑛 are pair-wise independent. Then:
1
𝜎
Pr [| ∑ 𝑋𝑖 − 𝜇| > 𝜖] ≤ 2
𝑛
𝜖 ∙𝑛
𝑖
1
1
1
Proof: The variables are pair-wise independent. Thus, 𝑉 [𝑛 ∑ 𝑋𝑖 ] = 𝑛2 ∑𝑖 𝑉[𝑋𝑖 ] = 𝑛2 ∙ 𝑛 ∙ 𝜎 2 =
Follows by chebyshev.
𝜎2
𝑛
Chernoff/Hoefding Inequality
Let 𝑋1 , … , 𝑋𝑛 totally independent all have expectation 𝜇 and take values in the segment [𝑎, 𝑏].
2
2𝜖
1
−
∙𝑛
Pr [| ∑ 𝑋𝑖 − 𝜇| > 𝜖] < 2𝑒 (𝑏−𝑎)2
𝑛
𝑖
Let 𝑓: {0,1}𝑛 → [0,1]. A (𝛿, 𝜖)-sampler 𝐴 is a random algorithm that has a black-box access to 𝑓
and Pr[|𝐴 𝑓 − 𝐸[𝑓(𝑈𝑛 )]| > 𝜖] < 𝛿
Sample Complexity – how many queries are made to 𝑓?
How many random bits are used.
Deterministic 2𝑛 samples, Randomness 0 samples.
Totally independent: 𝑞 = log
1
𝛿
𝜖2
( )
Choose totally independent uniformly distributed 𝑋1 , … , 𝑋𝑞 ∈ {0,1}𝑛
For 𝑞 ≈ log
1
𝑠
𝜖2
( )
1
and output 𝑞 ∑𝑖 𝑓(𝑋𝑖 )
1
Just like before, but use pairwise independent samples: 𝑞 = 𝜖2 ∙𝛿
For the two-wise independent we will have
1
𝜖 2 ∙𝛿
but we will gain a lot in the randomness.
Let 𝑛 ∈ 𝑁, 𝑆 ⊆ {0,1}𝑛 |𝑆| ≪ 2𝑛
Hash table of 2𝑛 entries.
Idenfity entries with {0,1}𝑛
Choose some function ℎ: {0,1}𝑛 → {0,1}𝑚
Store 𝑥 ∈ 𝑆 in entry 𝑦 = ℎ(𝑥)
One way to do it is choose ℎ to be a random function.
1
∀𝑥1 , 𝑥2 ∈ {0,1}𝑛 Pr[ℎ(𝑥1 ) = ℎ(𝑥2 )] = 2𝑚
Really want: Efficiently computable ℎ that informally “behaves like a random function”.
Pairwise independent hash functions.
𝑛
𝐻𝑚
- A family of functions from {0,1}𝑛 to {0,1}𝑚
𝑛
𝐻𝑚
is pair-wise independent if it satisfies the following:
∀𝑥1 , 𝑥2 ∈ {0,1}𝑛 ,
∀𝑦1 , 𝑦2 ∈ {0,1}𝑚 ,
Pr[ℎ(𝑥1 ) = 𝑦1 ∧ ℎ(𝑥2 ) = 𝑦2 ] =
1
(2𝑚 )2
𝑛
𝑛
Can construct 𝐻𝑚
such that each function in the family ℎ ∈ 𝐻𝑚
can be represented using 2𝑛 bits
and computed in time poly(n).
|𝑆|
𝐸[#𝑐𝑜𝑙𝑙𝑖𝑠𝑖𝑜𝑛𝑠] = ( ) ∙ 2−𝑚
2
𝑚 ≈ log|𝑆|2
Construction: Suffices to construct 𝐻𝑛𝑛
Let 𝐹2𝑛 the finite field of size 2𝑛
It’s elements can be identified with strings of length 𝑛.
ℎ = ℎ𝑎,𝑏 ∀𝑥 ∈ {0,1}𝑛 , ℎ𝑎,𝑏 (𝑥) = 𝑎 ∙ 𝑥 + 𝑏
𝑎, 𝑏 ∈ 𝐹2𝑛
Prove it is pairwise independent.
𝑆 ⊆ {0,1}𝑛 , 𝑦 ∈ {0,1}𝑚 what is |ℎ−1 (𝑥) ∩ 𝑆|
|𝑆|
If ℎ was totally random, expect 2𝑚
𝑅
𝑛
On the other hand, if ℎ ∈ 𝐻𝑚
: Pr [|ℎ−1 (𝑦) ∩ 𝑆| ∈ [(1 − 𝜖)
𝐸[𝑋𝑖 ] =
1
,
2𝑚
|𝑆|
2𝑚
, (1 + 𝜖)
|𝑆|
2𝑚
1
2𝑚
𝑉[𝑋𝑖 ] = 𝐸[𝑋𝑖2 ] − (𝐸[𝑋𝑖 ]2 ) ≤
Let 𝑆 = {𝑥1 , … , 𝑥|𝑆| }
𝑋𝑖 - Random Variable that is 1 if ℎ(𝑥𝑖 ) = 𝑦 and 0 otherwise.
1
|𝑆|
|𝑆|
∑ 𝑋𝑖
1
1
2𝑚
𝑚
2
Pr [|∑ 𝑋𝑖 − 𝑚 | > 𝜖 ∙ 𝑚 ] ≤ Pr [|
− 𝑚| > 𝜖 ∙ 𝑚] ≤ 𝜖
= 2
|𝑆| 2
2
2
2
|𝑆| 𝜖 ∙ |𝑆|
2𝑚 ∙
Mixing: ∀𝑆 ⊆ {0,1}𝑛 , 𝑇 ⊆ {0,1}𝑚
Pr [|ℎ−1 (𝑇) ∩ 𝑆| ∈ [(1 − 𝜖)
|𝑆|∙|𝑇|
2𝑚
, (1 + 𝜖)
2𝑚
|𝑆|∙|𝑇|
2𝑚
]] ≥ 1 − 𝜖2 ∙|𝑆|∙|𝑇|
Another property:
𝑅
3
1
𝑛
Leftover hash lemma: ∀𝑆 ⊆ {0,1}𝑛 , ℎ ∈ 𝐻𝑚
, 𝜖 = √2𝑚
∀𝛼 ∈ (0,1),
Pr [Δ(ℎ(𝑈𝑛 ), 𝑈𝑚 ) ≤
2∙𝜖
]≥ 1−𝛼
𝛼
2𝑚
]] > 1 − 𝜖2 |𝑆|