ISIT 2009, Seoul, Korea, June 28 - July 3, 2009
The Robustness of Stochastic Switching Networks
Po-Ling Loh
Hongchao Zhou
Jehoshua Bruck
Department of Mathematics
California Institute of Technology
Pasadena, CA 91125
Email: loh@caltech.edu
Department of Electrical Engineering
California Institute of Technology
Pasadena, CA 91125
Email: hzhou@caltech.edu
Department of Electrical Engineering
California Institute of Technology
Pasadena, CA 91125
Email: bruck@caltech.edu
Abstract-Many natural systems, including chemical and biological systems, can be modeled using stochastic switching
circuits. These circuits consist of stochastic switches, called
pswitches, which operate with a fixed probability of being open
or closed. We study the effect caused by introducing an error
of size E to each pswitch in a stochastic circuit. We analyze two
constructions-simple series-parallel and general series-parallel
circuits-and prove that simple series-parallel circuits are robust
to small error perturbations, while general series-parallel circuits
are not. Specifically, the total error introduced by perturbations
of size less than E is bounded by a constant multiple of E in
a simple series-parallel circuit, independent of the size of the
circuit. However, the same result does not hold in the case of
more general series-parallel circuits. In the case of a general
stochastic circuit, we prove that the overall error probability is
bounded by a linear function of the number of pswitches.
I. INTRODUCTION
Stochastic switching circuits have widespread applications
in many fields of modem science, including neuroscience
and chemical networks. In neuroscience, stochastic circuits
are used to model neural networks, since the propagation of
electrical impulses between neurons depends probabilistically
on environmental and physical factors [3]. In seeking to model
the human brain, it is informative to identify classes of circuits
which appear to emulate the behavior of natural systems.
Another application of stochastic switching circuits is found
in the framework of chemical reactions [5]. In this context,
the probabilities assigned to stochastic switches represent
relative concentrations of chemical species after each reaction
in a chemical network. Developing algorithms to synthesize
stochastic circuits with certain probabilities of being closed
may be applied to areas such as drug delivery, where the
desired result is a product species with a fixed concentration.
Furthermore, since concentrations of chemicals are difficult
to control precisely, results on the robustness of stochastic
circuits are particularly relevant in the context of chemical
networks.
Formally, a stochastic switching circuit C with two terminals is composed of stochastic switches known as pswitches.
Each pswitch is assigned a Bernoulli random variable with
parameter 0 < P < 1, where 1 indicates that the switch
is closed and 0 indicates that the switch is open. The set
S of probability parameters for the pswitches in a circuit is
called the pswitch set. We denote by P( C) the probability
that the two terminals of C are connected, and call P( C)
978-1-4244-4313-0/09/$25.00 ©2009 IEEE
-
® }
1
2 ----f""'*+
(a) An sp circuit.
2
(b) An ssp circuit.
Fig. 1. Examples of sp and ssp circuits. Note that (a) is not ssp.
the probability of c. We can realize the probability x using
a pswitch set S if and only if there exists a circuit C, with
pswitch probabilities from S, such that x == P (C).
As with resistor circuits [2], we can connect two switching
circuits C 1 and C 2 in series by connecting one terminal of C 1
to one terminal of C 2 . Then the probability of the resulting
circuit is PIP2, where PI == P(C1 ) and P2 == P(C2). We can
connect C 1 and C 2 in parallel by connecting the corresponding
terminals of both circuits. The probability of the resulting
circuit is 1- (1-Pl)(1-P2) == PI +P2 -PIP2. A series-parallel
(sp) circuit is either (1) a single pswitch, or (2) a series or
parallel combination of two sp circuits. Simple series-parallel
(ssp) switching circuits are a special case of sp circuits, and
are either (1) a single pswitch, or (2) an ssp circuit with an
additional pswitch added in series or parallel. See Figure 1.
In [6], the authors proved that any probability of the form
where 0 < a < 2n , can be realized with at most n
switches, using the pswitch set S ==
However, in natural
systems, the states of pswitches may depend on many factors,
so we cannot fix their probabilities at specific values.
In this paper, we analyze further properties of stochastic switching circuits, where the probabilities of individual
pswitches are taken from a fixed pswitch set, but given an error
allowance of E; i.e., the error probabilities of the pswitches are
bounded by Eo For a stochastic circuit with multiple pswitches,
the "error probability" of the circuit is the absolute difference
between the probability that the circuit is closed when error
probabilities of pswitches are included, and the probability that
the circuit is closed when error probabilities are neglected. We
show that ssp circuits are robust to small error perturbations,
but the error probability of a general sp circuit may be
amplified with additional pswitches. In particular, we have the
following two theorems:
2066
ISIT 2009, Seoul, Korea, June 28 - July 3, 2009
Theorem (Robustness of ssp circuits). Given a pswitch set S,
if the error probability of each pswitch is bounded by E, then
the total error probability ofan ssp circuit is bounded by . E,
where
m == min{min{S}, 1 - max{S}}.
Theorem (Unbounded error of sp circuits). Given a pswitch
set S, if the error probability of each pswitch is bounded by
E, then for any 0 < x < 1, there exists an sp circuit with error
probability close to x.
These theorems demonstrate the advantage of ssp circuits
over general sp circuits in designing engineering systems.
For a general stochastic switching circuit, we also have the
following error bound:
Theorem (Error bound of a circuit with n pswitches). Given
a pswitch set S, if the error probability of each pswitch is
bounded by E, then the total error probability of a stochastic
switching circuit with n pswitches is bounded by ne.
The remainder of this paper is organized as follows: In
Section II, we discuss the error bound for ssp circuits. In
Section III, we prove that a similar error bound does not hold
in the case of sp circuits. In Section IV, we provide an error
bound for general stochastic switching circuits.
By the Triangle Inequality and the induction hypothesis,
le11
< IE21 +
Theorem 1. Given a pswitch set S, if the error probability of
each pswitch is bounded by E, then the total error probability
of an ssp circuit is bounded by
. E, where
m == min{min{S}, 1 - max{S}}.
Proof: We induct on the number of pswitches. If we
have just one pswitch, the result is trivial. Suppose the result
holds for n pswitches, and note that for an ssp circuit with
n + 1 pswitches, the last pswitch will either be added in series
or in parallel with the first n pswitches. By the induction
hypothesis, the circuit constructed from the first n pswitches
has probability p + E1 of being closed, where E1 is the error
probability introduced by the first n pswitches and IE11 ::; Eo
The (n + 1)st pswitch has probability t + E2 of being closed,
where t E Sand IE21 ::; E.
If the (n + 1)st pswitch is added in series, then the new
circuit (with errors) has probability
+ (1)(t + (2) == tp + E2(p + (1) + tEl
of being closed. Without considering the error probability of
each pswitch, the probability of the new circuit is tp.
Hence, the overall error probability of the circuit is
==
E2 (p
+ E1) + tEl·
1
-E
m
< --·E.
Note that m
+t
m
::; (1 - max{ S} ) + max{ S}
== 1, so
1
le11 ::; - . E,
m
completing the induction.
Similarly, if the (n + 1)st pswitch is added in parallel, then
the new circuit (with errors) has probability
(p + E1) + (t + E2) - (p + E1)(t + E2)
== (p + t - tp) + (E1 + E2 - tEl - pE2 - E1E2)
of being closed. Without considering the error probability of
each pswitch, the probability that the circuit is closed is p +
t - tp.
Hence, the overall error probability of the circuit with n + 1
pswitches is e2 == E1 (1 - t) + E2(1 - P - (1)' Again using the
induction hypothesis and the Triangle Inequality, we have
le21
< (1 - t)IE11 + IE2(1 -
p -
(1)1
1-t
< --E+ IE21
We begin by analyzing the susceptibility of ssp circuits to
small error perturbations in individual pswitches. Instead of
assigning a pswitch a probability of p, the pswitch may be
assigned a probability between p - E and p + E, where E is a
fixed error allowance.
e1
t·
m+t
II. SIMPLE SERIES-PARALLEL CIRCUITS
(p
< IE211 (p + E1)I + tiE11
<
m
1-t+m
1
m
E
< -E,
m
since m == min{min{S}, 1 - max{S}} ::; t. This completes
the proof.
•
As an application, note that if S ==
and each pswitch is
given an error allowance of E, then the overall error probability
of any ssp circuit with pswitch probabilities from S is bounded
by 2E.
III. GENERAL SERIES-PARALLEL CIRCUITS
In the last section, we proved that for a given pswitch set
S, the overall error probability of an ssp circuit is bounded
by a constant multiple of E, where E is a fixed error allowance
for each pswitch. We want to know that whether this property
holds for all sp circuits. In this section, we will show that even
though the error probability of each pswitch is still bounded
by E, the overall error probability of a given circuit may be
unbounded as the number ofpswitches increases. We have two
main results, summarized in Theorems 2 and 3.
Although the result of Theorem 2 may be deduced from
Theorem 3, both theorems provide insight about the nature
of robustness in stochastic switching circuits. Theorem 2
is inspired by an iterative construction where two identical
circuits are composed on each step. It shows that regardless of
the starting circuit (even if it were not sp), we can always build
a circuit with an (asymptotic) error probability greater than any
constant multiple of Eo Rather than using a component-wise
2067
ISIT 2009, Seoul, Korea, June 28 - July 3,2009
construction, Theorem 3 constructs a non-ssp circuit which
realizes any specified error probability by repeatedly using a
single pswitch with error probability E. In this construction,
the overall probability that the circuit is closed converges to
I. We are not concerned with the overall probability of the
circuits in Theorem 2, since we only analyze the asymptotic
behavior of the error as the number of iterations increases.
For the iterative construction, see Figure 2. Beginning with
any arbitrary circuit, we calculate the probability that the
circuit is closed, and write it as a + be + 0(E2) . If a
we build the next circuit by taking the previous circuit and
putting it in series with itself. If instead a <
we take the
previous circuit and put it in parallel with itself. Then we
iterate the process.
Log Growthof Epsilon Coefficient
y = 0.21· x - 0.17
10
Fig. 3.
-1
a+bc
H
(b) a
Fig. 2.
We see that if a
circuit is closed is
20
25
30
The growth of the error coefficient.
Proof Simple algebraic calculations show that for a
starting circuit probability of a + E, we have the following
four cases:
a+bc
(a) a
15
Iteration
<
Iterative construction.
then the probability that the next
Hence, the error term is Zabe + 0(E2 ) . Since a
we have
2ab b, so the coefficient of the E term grows.
Similarly, if a <
then the probability that the next circuit
is closed is
• 0 :::; a :::; 1In this case, we put the circuit in parallel
with itself, then put the new circuit in parallel: "parallel,
followed by parallel."
• 1:::; a :::;
In this case, we have "parallel, followed
by series."
•
a :::;
In this case, we have "series, followed by
parallel."
•
a :::; 1. In this case, we have "series, followed by
series."
After two steps, the first case yields the error coefficient
4(1 - a)3, the second case yields 4a(1 - a)(2 - a), the third
case yields 4a(1 - a 2 ) , and the fourth case yields 4a 3. So the
error coefficient b can be written as
4(1 -a) 3,
4a(1 - a)(2 - a) ,
4a(1 - a 2 ) ,
4a 3 ,
Again, since a <
we have 2(1 - a)b b, so the coefficient
of E in the error term grows. We are effectively using a
"greedy" approach in this construction, where the coefficient
of E goes from b to 2 max{ a, 1 - a}b on each step.
We now study the rate of growth of the coefficient of E in
the error term. Using simple Matlab code , we generate the
graph in Figure 3, which is illustrative of the behavior of the
growth of the error coefficient under general initial conditions.
(In this graph, we used the initial circuit probability 0.6, and
scaled the coefficient of E to a starting coefficient of 1.)
In general, the points appear in pairs, with an approximate
slope of 0.21 in the linear regression. This corresponds to the
error coefficient being multiplied by an average factor of 1.23
on each step.
We now provide a lower bound on the growth of the
coefficient of E in two consecutive steps. We assume E is small
enough that we can ignore the 0(E2) term.
Lemma 1. Assume the initial circuit is closed with probability
a + E. Then the coefficient of E is multiplied by a factor of at
least J2 in two consecutive steps.
'0'-
0.2
04
O <a
-<l -...L
v'2
l -...L
v'2 -<a <.!.
- 2
.!.2 -<a <...L
- v'2
a :::; 1.
0.6
0.8
Fig. 4. The growth of the error coefficient of E as a function of a, after two
steps.
2068
We graph the growth of the error coefficient in Figure 4,
ISIT 2009, Seoul, Korea, June 28 - July 3, 2009
and we can check algebraically that the absolute minimum of
J2 occurs at 1/J2 .707 and 1 - 1/J2 .293.
•
This proves the unboundedness theorem presented in the
introduction:
m strings
Theorem 2. Given a pswitch set S, if the error probability
of each pswitch is bounded by E (E --* 0), then there exists
no constant c such that the error probability of any sp circuit
with pswitches from S is bounded by CEo
,
n pswitches
One may conjecture from the preceding discussion that the
greedy procedure will always yield the largest error coefficient
after an arbitrary number of steps, since we have seen this to
be true after two consecutive steps. However, using a starting
probability of .51 + E, the greedy algorithm does not yield a
maximum error coefficient on the third step. Indeed, the greedy
algorithm provides the following series of terms:
Fig. 5.
The construction of a noisy sp circuit.
Note that
lim m· log(l _ pn)
.
-logx·log(l- pn)
hm
pn
-logx.
. _pn logp
.51 + E, .26 + 1.02E, .45 + 1.51E, .70 + 1.65E,
pn logp
corresponding to the sequence "series, parallel, parallel." However, the sequence "parallel, series, series" yields the terms
.51+E, .76+.98E, .58+1.49E, .33+1.72E.
Comparing the coefficients of E in the final terms of each
sequence, we see that the non-greedy result actually provides
a larger error coefficient, disproving the conjecture. However,
we have shown that the greedy algorithm already yields a
construction where the error coefficient grows exponentially,
so using a non-greedy procedure would merely cause the error
coefficient to grow even more rapidly.
We also have an upper bound on the rate of growth of the
error coefficient in two steps:
logx.
Hence, the probability that the circuit will be closed converges
to
1 - e 10 g x == 1 - x
as n --* 00.
Now suppose we introduce an error of E to each pswitch,
so the probability that each pswitch in the circuit is closed is
q == p + E. Then the probability that the circuit is closed is
1 - (1 - qn)m, and we compute
lim m .1og(1 _ qn)
Lemma 2. Assume the initial circuit is closed with probability a + E. After an initial number of steps (at most
log21ogmax{a,1-a}
the coefficient of E is multiplied by a
1.688 in two consecutive steps.
factor of at most
.m -logx ·log(l - qn)
11
pn
-logx.
. _qn logq
lim
l-q
p" logp
q
q)
log x . log
1.
(
- - - - . 1m logp
p
n
Note that since 0 < p, q < 1, we have
< o. So for
E > 0, we have
> 1, implying that the limit is -00. Hence,
the probability that the circuit is closed converges to 1- 0 == 1.
However, we still need to take into consideration the fact
n J. (We need to make this modification
that
m = l-log xTheorem 3. Given a pswitch set S, if the error probability of
each pswitch is bounded by E, then for any 0 < x < 1, there in order to ensure that m is an integer.) Note that adding the
floor function changes m by at most 1. Then the quantity
exists an sp circuit with error probability close to x.
(1 - pn)m (or (1 - qn)m) changes by at most a factor of
Proof: Suppose pES. We construct an sp circuit by (1 - pn). As n increases, this factor approaches 1. Hence, we
taking a string of n pswitches of probability p, then connecting can constrain n such that (1 - pn) is close enough to 1, and
m of these strings in parallel, where m = l- log x .
n J,
the probabilities that the circuits are closed can still be made
arbitrarily close to 1 - x and 1.
•
as shown in Figure 5.
Clearly, the probability that the circuit will be closed is
IV. GENERAL STOCHASTIC SWITCHING CIRCUITS
1 - (1 _ pn)m.
In this section, we extend our discussion to the case of
general stochastic switching circuits. We have the following
To simplify our computations, we let m =
log x .
n.
theorem, which clearly also holds for sp and ssp circuits:
Then
For the proof of Lemma 2, we refer the reader to the
Appendix of [1].
In fact, using a different construction, we can obtain a
stronger version of Theorem 2:
lim (1 - pn)m == lim emolog(l-pn).
0)
Theorem 4. Given a general stochastic switching circuit
with n pswitches taken from a finite pswitch set S, if each
2069
ISIT 2009, Seoul, Korea, June 28 - July 3,2009
pswitch has error probability bounded by E, then the total
error probability of the circuit is bounded by ne.
Proof:
•
Note that this is the best bound we can afford in the case
of a general pswitch set S and any arbitrary E. Indeed, given
a value of n, choose p close to 1 and E << p. Putting n
pswitches of probability p - E in series, we have probability
(p _ E)n
pn _ npn-I E
that the circuit is closed. Without errors, the probability of the
circuit is p"; so the overall error is
n. pn-I E•
Choosing p sufficiently close to 1, we can make the error
probability of the circuit arbitrarily close to tie.
V.
Fig. 6.
An example of a general stochastic switching circuit.
We first index all the pswitches in the circuit C as shown in
Figure 6. If Xi is the probability that the i t h pswitch is closed,
we write C (Xl, X2, ..., Xn) to denote the n-pswitch circuit. For
each i, we write Xi == a.; + Ei, where a.; E Sand Ei is the error
probability (so lEi I ::; E).
Let pCk) denote the probability that C is closed when we
only take into account the error probabilities of the first k
pswitches; i.e.,
p(k)
== P(C(XI, ... , Xk, ak+l, ... , an)).
The overall error of the circuit can then be written as
E
pen) _ p(D)
(pCn) _ pCn-I))
... + (pCI)
_
+ (pCn-l)
_ p Cn-2))
+
pCD)).
We need to prove that IP(k) - pCk-l) I ::; E for all L'< k ::; n.
We write
IPCk) _ p(k-l) I
IP(C(XI,
-P(C(XI,
, Xk, ak+l,···, an))
, Xk-l, ak,···, an))1
IXkP(C(XI,
, Xk-l, 1, ak+l,···, an))
In this paper, we have analyzed the effects caused by small
error perturbations in stochastic switching circuits and shown
that ssp circuits are robust to small errors of size E, while
general sp circuits are not. We have also provided a linear
bound for the error in a general stochastic switching circuit,
when each pswitch has an error probability of at most E.
Our result on the robustness of ssp circuits supports the hypothesis that ssp circuits provide a better model for biological
systems than general sp circuits. This is consistent with the
observation that the inductive construction of an ssp circuit
resembles the synthesis of natural systems through biological
or evolutionary growth. Further directions of research include
analyzing the case where the probabilities assigned to the
pswitches are not discrete, but continuous (perhaps timedependent probabilities), and considering other systems where
the rules of composition for series and parallel are analogous
to the rules of composition for electrical circuits.
ACKNOWLEDGMENT
This work was supported in part by the NSF Expeditions in
Computing Program under grant CCF-0832824. The authors
would also like to thank the Caltech Summer Undergraduate
Research Fellowship (SURF) program for its support in funding this research, and Dan Wilhelm for his comments and
suggestions in revising the paper.
+(1 - Xk)P(C(XI, ... , Xk-l, 0, ak+l,···, an))
-akP(C(xI, ... , Xk-l, 1, ak+l,···, an))
-(1 - ak)P(C(XI, ... , Xk-l, 0, ak+l,···, an))1
/Ek . [P(C(XI, ... , Xk-l, 1, ak+l,···, an))
-P(C(XI, . . . ,Xk-l, 0, ak+l,· .. ,an))] I
< IEkl
< E.
Therefore, we have
E
< IP(n) _ pCn-l) I + IP(n-l) _ p(n-2) I +
... + IPCI)
< tie,
_ pCD) I
CONCLUSION
REFERENCES
[1] P. Loh, H. Zhou and J. Bruck, "The Robustness of Stochastic Switching
Networks." Appears as Caltech Paradise Electronic Techinical Report at
http://www.paradise.caltech.edu/papers/etr092.pdf.
[2] P.A. Macmahon, "The combinations of resistances," The Electrician, vol.
28, pp. 601-602, 1892. (Reprinted in: Discr. Appl. Math., 54:225-228,
1994.)
[3] F. Rieke, D. Warland, R.R. van Steveninck, W. Bialek, "Spikes: exploring
the neural code," MIT Computational Neuroscience series, 1999.
[4] C. Shannon, "A symbolic analysis of relay and switching circuits," Trans.
AlEE, vol. 57, pp. 713-723, 1938.
[5] D. Soloveichik, M. Cook, E. Winfree, 1. Bruck, "Computation with finite
stochastic chemical reaction networks," Natural Computing 7(4), pp. 615633,2008.
[6] D. Wilhelm and 1. Bruck, "Stochastic switching circuit synthesis," IEEE
International Symposium on Information Theory, Toronto, Canada, July
2008. Also appears as Caltech Paradise Electronic Technical Report at
http://www.paradise.caltech.edu/papers/etr089.pdf.
as wanted.
2070