Lecture 9. Circuit Complexity
From this lecture we start to prove lower bounds in the circuit model. As we said, the task is
too hard for general circuits. So people studied special types of circuits. One important class
contains circuits with small depth.
Recall that a circuit is a DAG with each node associated with a gate that computes a basic
function, such as AND, OR and NOT. The fanin of a gate can be arbitrary. (Namely, the
AND and OR gates are not just binary.) We will draw a circuit in the top-down manner, s.t.
the output gate is the top layer and gates in the bottom layer connect to the input variables.
1.
Depth 2: DNF and CNF
This section considers depth-2 circuits. If the top gate is AND, then the circuit is essentially a
CNF or DNF. Recall:
𝑘
𝑖
CNF (conjunctive normal form): 𝑓(𝑥1 … 𝑥𝑛 ) = ⋀𝑚
𝑖=1 ⋁𝑗=1 𝑙𝑖𝑗 , where each 𝑙𝑖𝑗 is either a
𝑘
𝑖
variable or the negation of a variable. Each 𝑙𝑖𝑗 is called a literal, and each ⋁𝑗=1
𝑙𝑖𝑗 is a
called a clause.
𝑘
𝑖
DNF (Disjunctive normal form): 𝑓(𝑥1 … 𝑥𝑛 ) = ⋁𝑚
𝑖=1 ⋀𝑗=1 𝑙𝑖𝑗 , where each 𝑙𝑖𝑗 is a literal, and
𝑘
𝑖
each ⋀𝑗=1
𝑙𝑖𝑗 is a called a monomial.
Any function can be written as a CNF and DNF. (Convince yourself.) We’d like to find an
explicit function 𝑓 s.t. if we write it as a CNF or DNF, then the number of
clauses/monomials is very large. Such a function is not hard to find, actually the Parity
function serves as such an example. (Recall: Parity(𝑥) = 𝑥1 ⊕ … ⊕ 𝑥𝑛 .)
Theorem 1.1. Any depth-2 circuit that computes Parity needs at least 2n−1 gates.
Can you figure out a proof? (It’s not complicated; only one or two lines. And it’ll be used
later.)
2.
Depth 3
Now we consider depth-3 circuits. If the top gate of a circuit is AND, then we call it a
Π3 -circuit.
Let’s first prove that if the circuit has a small top fanin, then it has to have a large size.
Theorem 2.1. ([Tsa01]) If a Π3 -circuit computing the Parity function has fanout 1 and top
fanin 𝑡, then it has at least 𝑡2(𝑛−1)/𝑡 AND gates at the bottom layer.
Proof. The idea is to switch the AND and OR of the two top layers using de Morgan rule, and
then the problem is reduced to the depth 2 case. Suppose that the i-th OR gate on the middle
layer has fanin 𝑠𝑖 . Let ℎ𝑖𝑗 be the 𝑗-th AND of the 𝑖-th OR gate of the top AND gate. Use
de Morgan rule, we can change the depth-3 circuit to a depth-2 one, which has the top gate
OR with fanin 𝑠1 𝑠2 … 𝑠𝑡 . Note that each AND can accept at most one input, so 𝑠1 𝑠2 … 𝑠𝑡 ≥
2𝑛−1. Since each gate has fanout 1, the number of AND dates at the bottom layer is
𝑠1 + 𝑠2 + ⋯ + 𝑠𝑛 ≥ 𝑡(𝑠1 𝑠2 … 𝑠𝑡 )1/𝑡 ≥ 𝑡2(𝑛−1)/𝑡 .
□
From the bound, you can see that when t is large, then we need new method. Next we
introduce the k-limit method.
Suppose that 𝐴 ⊆ 𝑓 −1 (1) and 𝐵 ⊆ 𝑓 −1 (0). An input 𝑦 ∈ 𝐴 is a lower k-limit for B if for
any 𝑆 ⊆ [𝑚], |𝑆| = 𝑘, there is an input 𝑥 ∈ 𝐵 s.t. 𝑥𝑆 = 𝑦𝑆 and 𝑥 ≤ 𝑦. Recall that 𝑥 ≤ 𝑦
means each 𝑥𝑖 ≤ 𝑦𝑖 .
We will use the concept to prove an exponential lower bound for (the negation of) Majority.
Formally, define
¬Maj𝑛 (𝑥) = 1 iff 𝑥1 + ⋯ + 𝑥𝑛 < 𝑛/2.
The result was proved in [HJP95].
Theorem 2.2. Any Π3 -circuit computing the ¬Maj𝑛 function needs 𝑙 ≥ 2Ω(√𝑛) fanout.
Proof. The parameters in the following proof:
𝑚=
𝑛
+ 𝑟,
2
𝑟=
√𝑚
,
4
𝑘 = 2√𝑚.
Solving it gives the following approximation:
𝑚≈
𝑛 √𝑛/2
+
,
2
4
𝑟≈
√𝑛/2
,
4
𝑘 ≈ 2√𝑛/2.
We need to define two sets:
𝐴 = {𝑥 ∈ {0,1}𝑚 : |𝑥| < 𝑟},
𝐵 = {𝑥 ∈ {0,1}𝑚 : |𝑥| = 𝑟}.
Now we’ll take three steps:
1. Set 𝑛 − 𝑚 variables to be 1, s.t. after this, each bottom AND gate has at most k negated
variables. Then we are left with a function 𝑓: {0,1}𝑚 → 1 with
𝑛
𝑓(𝑥) = 1 ⇔ |𝑥| < 2 − (𝑛 − 𝑚) = 𝑟.
(We will use Lemma 2.3 to prove this.)
2. Infer that there is an OR gate 𝑔 in the middle level s.t.
𝐵𝑔′ = {𝑥 ∈ {0,1}𝑚 : |𝑥| = 𝑟, 𝑔(𝑥) = 0}
𝑚
has |𝐵𝑔′ | ≥ ( ) /𝑙, and 𝐵𝑔′ does not have a lower k-limit in A.
𝑟
(We will use Lemma 2.4 to prove this.)
3. Conclude that
𝑚
( )
𝑟
𝑙
𝑟
≤ 𝑘 , and thus 𝑙 ≥
𝑚
( )
𝑟
𝑘𝑟
𝑚 𝑟
≥ (𝑘𝑟) = 2Ω(√𝑛) .
(We will use Lemma 2.5 to prove this.) □
Next we prove the three lemmas needed, and show how to use them in the above proof of
Theorem 2.2.
𝑛 𝑘/2
Lemma 2.3. If 𝐹 ⊆ {𝑆 ⊆ [𝑛]: |𝑆| > 𝑘} has |𝐹| ≤ (𝑚)
, then there is a 𝑇 ⊆ [𝑛] with
|𝑇| = 𝑛 − 𝑚 to intersect all 𝑆 ∈ 𝐹.
Proof. Since each 𝑆 ∈ 𝐹 has size more than k, at least one 𝑥𝑖1 belong to at least 𝑘/𝑛
fraction of sets in F. Put it in T. Remove those sets 𝑆 ∈ 𝐹 containing 𝑥𝑖1 . This leaves (1 −
𝑘/𝑛) fraction of F. Continue this. We claim that 𝑛 − 𝑚 steps kill all 𝑆 ∈ 𝐹. Indeed,
𝑘
𝑘
𝑘
𝛼 = (1 − ) (1 −
) … (1 −
)
𝑛
𝑛−1
𝑚+1
𝑘
𝑘
𝑘
≤ 𝑒𝑥𝑝 (− −
−⋯−
)
𝑛 𝑛−1
𝑚+1
= 𝑒𝑥𝑝(−𝑘(𝐻𝑛 − 𝐻𝑚 ))
𝑛 −𝑘/2
≤( )
𝑚
canceling the upper bound of |𝐹| in the assumption of the lemma.
□
To use this lemma, we take F to be the collection of bottom AND gates with more than k
negated variables. By the lemma, there is a 𝑇 ⊆ [𝑛] intersecting all of them. Set all variables
in T to be 1, then these AND gates are eliminated. Thus each of the remaining AND gates
contains at most k negated variables.
Lemma 2.4. Suppose that we have a Π3 -circuit computing 𝑓, with top fanin at most 𝑙 and
each bottom AND gate having at most k negated variables. Then there is an OR gate 𝑔 in
the middle level s.t. the set
𝐵𝑔′ = {𝑥 ∈ {0,1}𝑚 : |𝑥| = 𝑟, 𝑔(𝑥) = 0}
has
𝑚
|𝐵𝑔′ | ≥ ( ) /𝑙 and 𝐵𝑔′ does not have any lower k-limit in A.
𝑟
Proof. Recall the set
𝐵 = {𝑥 ∈ {0,1}𝑚 : |𝑥| = 𝑟}.
Note that 𝐵𝑔′ ⊆ 𝐵 ⊆ 𝑓 −1 (0). Since the top gate is AND, there is at least one OR gate in the
middle level evaluating to 0 for inputs in B. Thus there is an OR gate 𝑔 s.t.
𝑚
|𝐵𝑔′ | ≥ ( ) /𝑙.
𝑟
Suppose the set has a lower k-limit 𝑦 ∈ 𝐴, namely 𝑔(𝑦) = 1 and for any 𝑆 ⊆ [𝑚], |𝑆| = 𝑘,
there is an input 𝑥 ∈ 𝐵𝑔′ s.t. 𝑥𝑆 = 𝑦𝑆 . Consider each AND gate h feeding in 𝑔:
ℎ(𝑥) = ⋀ 𝑥̅𝑖 ∧ ⋀ 𝑥𝑗
𝑖∈𝑆
𝑗∈𝑇
We claim that ℎ(𝑦) = 0. Indeed, since ℎ(𝑥) = 0, there is an 𝑥𝑗 = 0 or an 𝑥𝑖 = 1. If the
former, then 𝑦𝑗 ≤ 𝑥𝑗 = 0 and thus ℎ(𝑦) = 0. If the latter, then since 𝑥𝑆 = 𝑦𝑆 , we also have
yi = 𝑥𝑖 = 1, which again implies that ℎ(𝑦) = 0.
Since this holds for each h, we know that 𝑔(𝑦) = 0 as well, violating the assumption that
𝑦 ∈ 𝐴.
□
Lemma 2.5. For any set 𝐵 ′ ⊆ {𝑥 ∈ {0,1}𝑚 : |𝑥| = 𝑟} with |𝐵 ′ | > 𝑘 𝑟 , there is a lower k-limit
𝑦 ∈ 𝐴 for 𝐵’.
Proof. Induction on 𝑟.
𝑟 = 1: 0𝑚 ∈ A is a lower k-limit for 𝐵’. First, it is lower to any other vector, namely 0𝑚 ≤
𝑥 for any x. Second, for any 𝑆 ⊆ [𝑚] with |𝑆| = 𝑘, since |𝐵 ′ | > 𝑘 when 𝑟 = 1, we have
some element 𝑥 ∈ 𝐵′ s.t. the only 1 in x is not in 𝑆. This element serves the lower k-limit
definition.
Assuming 𝑟 − 1, consider 𝑟: If 0𝑚 ∈ A is not a lower k-limit for 𝐵’, then there is a set 𝑆 ⊆
[𝑚] with |𝑆| = 𝑘 s.t. every 𝑥 ∈ 𝐵′ has at least one 1 in 𝑆. Take the most “popular” 𝑖 ∈ 𝑆
at which at least 1/𝑘 fraction of 𝐵′ have 1 at i-th position. Call the set of these elements
𝐶 ′ ⊆ 𝐵′. Flip these 1 to 0 and call the resulting set 𝐷′, then they each have 𝑟 − 1 ones. Note
that |𝐷′ | = |𝐶 ′ | ≥
|𝐵′ |
𝑘
> 𝑘 𝑟−1. Use induction hypothesis to get a 𝑦 ∈ 𝐴 with less than 𝑟 −
1 ones s.t. y is lower k-limit for 𝐷’. Note that 𝑦𝑖 = 0 since y is lower than an element in 𝐷′.
Flip 𝑦𝑖 to 1, and this element 𝑦 ′ ∈ 𝐴 since |𝑦 ′ | = |𝑦| < 𝑟 − 1 + 1 = 𝑟. We claim that this
𝑦′ is a lower k-limit for 𝐵′. The “lower” part is easy. For the “k-limit” part: for any 𝑇 ⊆ [𝑚]
with |𝑇| = 𝑘, there is a 𝑥 ∈ 𝐷′ with 𝑦 ≤ 𝑥 and 𝑦𝑇 = 𝑥𝑇 . Note that flipping the i-th bit
from 0 to 1 makes 𝑦 to 𝑦′, and also make 𝐷′ to 𝐶′ ⊆ 𝐵′. This implies that 𝑦𝑇′ = 𝑥𝑇𝑖 where
𝑥 𝑖 is the string obtained from x by flipping the i-th bit. Note that 𝑥 𝑖 ∈ 𝐶 ′ ⊆ 𝐵′. Thus 𝑦 ′ is a
lower k-limit for 𝐵′.
□
𝑚
After the second step of the proof of Theorem 2.2, we get a set 𝐵𝑔′ with |𝐵𝑔′ | ≥ ( ) /𝑙, and
𝑟
𝐵𝑔′
does not have a lower k-limit in A. Thus by the above lemma, we know that
𝑚
( )
𝑟
𝑙
≤
|𝐵𝑔′ | ≤ 𝑘 𝑟 .
References
[Tsa01] Shi-Chun Tsai, A depth 3 circuit lower bound for the parity function, Journal of
Information Science and Engineering 17(5), 857–860, 2001.
[HJP95] J. Hastad, S. Jukna, and P. Pudlak, Top-down lower bounds for depth-three
circuits, Computational Complexity 5, 99–112, 1995.