I. INTRODUCTION
Closed-Form Expression for the
Poisson-Binomial Probability
Density Function
The binomial pdf describes the numbers of
successes in N independent trials when the individual
probabilities of success are constant across trials. If
these probabilities are allowed to vary, as is the case
in most practical applications, the resulting pdf is
known as a Poisson-binomial. It is an extraordinarily
useful model that can be encountered in all kinds of
applications across vastly differing fields; here are
some examples, together with references showing
where to find related cases.
MANUEL FERNÁNDEZ, Member, IEEE
STUART WILLIAMS
Lockheed Martin
The Poisson-binomial probability density function (pdf)
describes the numbers of successes in N independent trials,
when the individual probabilities of success vary across trials.
Its use is pervasive in applications, such as fault tolerance, signal
detection, target tracking, object classification/identification,
multi-sensor data fusion, system management, and performance
characterization, among others. We present a closed-form
expression for this pdf, and we discuss several of its advantages
regarding computing speed and implementation and in
simplifying analysis, with examples of the latter including
the computation of moments and the development of new
trigonometric identities for the binomial coefficient and the
binomial cumulative distribution function (cdf). Finally we
also pose and address the inverse Poisson-binomial problem;
that is, given such pdf, how to find (within a permutation) the
probabilities of success of the individual trials.
Manuscript received December 1, 2006; revised September 27,
2007, May 29, 2008, and November 18, 2008; released for
publication January 13, 2009.
IEEE Log No. T-AES/46/2/936818.
Refereeing of this contribution was handled by W. Koch.
Authors’ current addresses: M. Fernandez, Lockheed Martin,
MS2, 497 Electronics Pkwy., Liverpool, NY 13088, E-mail:
(manuel.f.fernandez@Imco.com); S. Williams, Sensis Corporation,
East Syracuse, NY 13057.
c 2010 IEEE
0018-9251/10/$26.00 °
1) Reliability Theory/Fault Tolerance [1]: If a
manufacturing process fails when at least M out of
N subprocesses fail, find the probability of failure
when the individual probability of failure of the nth
subprocess is pn .
2) Target Tracking [2—4]: A target track is initiated
when at least M detections are declared, by the
given sensor, in N consecutive, independent “look
opportunities.” Given pn , the (varying) probability of
detection per look, determine the probability of track
initiation.
3) Pattern Identification/Decision Theory [5]: If the
nth expert, diagnosing whether a particular condition
is present or not, does so correctly in pn percent of
the cases, how many such independent experts should
coincide in their diagnosis to achieve an overall
success rate that exceeds some desired percentage
value.
4) Educational Examination Design: Given the
percentages of students answering correctly on at
least n out of the N equally-weighted questions
of an standardized test, n = 0, 1, : : : , N, it may be
of interest to determine (under assumptions of
test and test-taker independence) the percentage
of students that correctly answered each of the
individual questions so as to establish whether the
questions exhibit a desired “spread of difficulty.”
(This is an inverse Poisson-binomial problem in
that we are trying to obtain the probabilities of
success (the percentages in this case) of the individual
trials. Since, as is seen, the Poisson-binomial
model is independent of the order in which the
trials take place, this can only be done up to a
permutation; in other words one can determine that
x percent of the students got a question right, but
one may not be able to determine which question it
was.)
5) Multi-Sensor Fusion [5]: Given a net of N
sensors whose detection/no-detection outputs are to
be combined through a voting scheme, what should
each sensor’s individual probability of false alarm be
so as to achieve a specified M-out-of-N “fused” false
alarm probability.
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2
APRIL 2010
803
6) Project Management/Resource Allocation
[1]: Given that rk resources have already been
allocated to each of K workstations, the probability
of each reaching its respective production
quota is pk = f(rk ). Given the availability of
L additional workstations, and assuming the
invertibility of function f, what resources should
be allocated to the new workstations so that
at least M-out-of-the-(K + L) workstations
achieve their production quota with a specified
probability P.
A methodology for obtaining closed-form
representations for Poisson-binomial expressions of
common use, such as the probability density function
(pdf) and the cumulative distribution function (cdf)
is presented. These closed-form expressions can be
applied directly to solve most of the examples just
presented.
We believe, however, that the main contribution
of this paper is not necessarily the presentation
of the closed-form expressions, but rather the
description of the methodology itself since it enables
the development of algorithms for addressing
other applications of the Poisson-binomial model.
The paper is thus intended to be tutorial in tone,
striving to derive, from basic principles, nearly
all the results invoked, so as to be understandable
to people from across as many disciplines as
possible.
The breakdown of the paper is as follows.
Section I introduces the relationship between
the probabilities of success of individual trials
and the Poisson-binomial pdf. Sections III
and IV use numerical techniques (polynomial
interpolation and discrete Fourier transform (DFT)
methods) to derive closed-form formulas for the
Poisson-binomial pdf and cdf, and Sections V and
VI then demonstrate the use of these expressions
by obtaining new representations of the binomial
coefficient, the binomial cdf, and the Poisson-binomial
moments. Section VII illustrates the use of the
various techniques presented in this paper by
applying them to solve the problem in example
6 of this Introduction. Section VIII again uses
polynomial methods, together with matrix-theoretic
techniques, to shed some light on the inverse
Poisson-binomial problem (namely, given the
Poisson-binomial pdf or cdf, how to obtain, up
to a permutation, the probabilities of success of
the individual trials); this section also sounds
a cautionary note regarding numerical stability
when using polynomials. Finally Section IX
presents some comments and a summary of the
paper.
804
II.
GROUNDWORK
Consider N independent trials with probabilities
of success and failure, for the kth trial, equal to pk
and 1 ¡ pk , respectively (“Poisson trials”). The
number Y of successes can be written as the sum Y =
X1 + X2 + ¢ ¢ ¢ + XN of N mutually independent random
variables Xk with the distribution vectors [PrfXk = 0g
PrfXk = 1g] = [1 ¡ pk pk ], where Prfug denotes
“probability of u” (see Feller [6] for example).
The distribution of the sum Y of these random
variables, the Poisson-binomial pdf (a.k.a.
Bernoulli-Poisson pdf), is then given by the linear
convolution of the distributions of the Xk s; that is
[PrfY = 0g PrfY = 1g ¢ ¢ ¢ PrfY = Ng]
= [1 ¡ p1
p1 ] ¤ [1 ¡ p2
p2 ] ¤ ¢ ¢ ¢ ¤ [1 ¡ pN
pN ]:
(1)
1
Taking the Z-transform of each side of (1) yields
two versions of the generating function of the
Poisson-binomial pdf,
P0 + P1 z + P2 z 2 + ¢ ¢ ¢ + PN z N
= (1 ¡ p1 + p1 z)(1 ¡ p2 + p2 z) ¢ ¢ ¢ (1 ¡ pN + pN z)
(2)
where Pn was used instead of PrfY = ng to simplify
the notation.
A minor manipulation of the right-hand side of (2)
yields
P0 + P1 z + P2 z 2 + ¢ ¢ ¢ + PN z N
= ®(z ¡ s1 )(z ¡ s2 ) ¢ ¢ ¢ (z ¡ sN )
where
®=
N
Y
pk
(3a)
(3b)
k=1
and
sk = ¡(1 ¡ pk )=pk :
(3c)
Notice, thus, that the values comprising the
Poisson-binomial pdf and the pdfs of the individual
trials are merely the parameters of two different
representations of the same polynomial. The former
is in expanded form, while the latter is factored;
hence, given one we may extract the other (i.e., it
is an invertible mapping). Notice also that, since the
1 In
the modern engineering literature, the Z-transform is usually
defined as the representation of a sequence of numbers as
coefficients of a polynomial in z ¡1 (i.e., the nth element of the
sequence, n = 0, 1, : : : , N ¡ 1, becomes the coefficient of the
polynomial’s (z ¡1 )n term); however, since for our purposes
such representation is far simpler when defined in terms of z
rather than z ¡1 , we have done so. (Some authors try to avert this
definitional conflict by using x instead of z ¡1 and by dubbing it
the “X-transform” (see [7], for example), but we decided to avoid
introducing more esoterica.)
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2 APRIL 2010
Fig. 1. MATLAB code for mapping (a) from individual probabilities of success in N independent trials to Poisson-binomial pdf and
(b) vice versa.
Q
polynomial N
k=1 (z ¡ sk ) on the right-hand side of (3)
is monic, the coefficient of the largest order term of
the polynomial in expanded form PN satisfies PN = ®.
As shown in Fig. 1, using a computing language
such as MATLAB, with its built-in functions “poly”
and “roots,” it is very easy to quickly write a script
for, given the probabilities of the individual trials (i.e.,
the pk ), obtaining the entries of the Poisson-binomial
pdf (i.e., the Pn ), and vice versa. All these two routines
produce, however, are merely lists of numbers which,
albeit extremely useful in many applications, do
not easily lead to new insight into further uses or
manipulations of the Poisson-binomial pdf. (From
the perspective of implementation complexity,
using the routine of Fig. 1(a) to obtain the values
of the Poisson-binomial pdf by direct expansion
of the right-hand side of (3a) via the fast method
implemented in MATLAB’s “poly” function is a very
efficient procedure, requiring only on the order of
N log2 N operations.2 )
III. A CLOSED-FORM EXPRESSION FOR THE
POISSON-BINOMIAL PDF
Returning to (2) let us recall that our objective
is to, given the pk , obtain an expression yielding
the Pn . One such expression can be obtained by
using the Vandermonde polynomial interpolation
method; namely by evaluating the right-hand side
of (2) at N + 1 different values of z and then by
finding the coefficients of the Nth-order polynomial
2 Other
methods for obtaining the Poisson-binomial pdf include the
use of “fast convolution” techniques to obtain the right-hand side of
(1) (see [1] and [8]), as well as a variety of recursive approaches
(see, for example, [1], [9], and [10], all of which also provide
interesting examples of the applicability of the Poisson-binomial
density across seemingly dissimilar fields).
FERNÁNDEZ & WILLIAMS: CLOSED-FORM EXPRESSION FOR THE POISSON-BINOMIAL PDF
805
exactly traversing the results. In matrix-vector form,
this method involves solving, for vector P, the linear
system of equations
2 3
P0
21 a
2
N3
a0 ¢ ¢ ¢ a0 6 7
0
7 6 P1 7
61 a
7
a21 ¢ ¢ ¢ aN
1
6
1 76
7 6 P2 7
6
7
6 ..
..
..
.. 7 6
6 7
4.
.
.
. 5 6 .. 7
4 . 5
1 aN a2N ¢ ¢ ¢ aN
N
|
{z
} PN
| {z }
V
2
P
N
Y
3
fpk a0 + (1 ¡ pk )g 7
6
7
6 k=1
7
6
7
6 N
7
6Y
6
fpk a1 + (1 ¡ pk )g 7
7
6
7
=6
7
6 k=1
7
6
..
7
6
7
6
.
7
6
7
6Y
N
5
4
fpk aN + (1 ¡ pk )g
or
|
k=1
{z
r
(4a)
e¡j2¼nm=(N+1)
P(m) =
n=0
k=1
(N + 1),
fpk ej2¼n=(N+1) + (1 ¡ pk )g
m = 0, 1, : : : , N
(5)
a closed-form expression for the Poisson-binomial
pdf.
As already stated many algorithms exist for
efficiently computing
),
(N
Y
j2¼n=(N+1)
fpk e
+ (1 ¡ pk )g
(N + 1) (6)
DFT
k=1
which is what (5) entails. Further simplifications can
be achieved by considering instead
),
(N
Y
(ej2¼n=(N+1) ¡ sk )
(N + 1)
(7)
®DFT
}
k=1
VP = r:
(4b)
Matrices with a structure as that of V are known as
“Vandermonde” matrices.
Theoretically speaking square Vandermonde
matrices are nonsingular; hence we should be able to
obtain P = V¡1 r. In practice, however, these matrices
are particularly unstable numerically in the sense that,
in general, as N increases, the unbalancing in the
structure of the matrix (i.e., relatively large-magnitude
numbers concentrating on certain zones of the matrix),
coupled to computer round-off, cause enough error to
perturb the matrix into singularity. And even if this
were not so, the problem of having to compute the
actual inverse of V still would remain since otherwise
the closed-form expression for P would not provide
much insight.
Fortunately both issues (stability and inversion)
can be easily solved by choosing the an as roots
of unity; that is, as an = ej2¼n=(N+1) . Such a choice
clearly thwarts magnitude growth with N, and it also
solves the inversion problem by reason of the fact
that the resulting Vandermonde matrix is unitary; that
is, VH V = VVH = (N + 1)I, where I is an (N + 1) £
(N + 1) identity matrix and where the superscripted
H is used to denote the complex-conjugate transpose.
Hence, P = VH r=(N + 1), or, taking advantage of the
fact that in this particular case matrix V happens to
be symmetric, P = V¤ r=(N + 1), where the asterisk
represents the operation of conjugating the entries
of V.
The matrix V¤ , for our particular choice of an
(let’s call it matrix F so as to distinguish it from
806
other possibilities), is known as the DFT matrix,
and countless fast Fourier transform algorithms exist
for computing products of the form Fr or, as more
commonly expressed, for computing DFTfr(n)g.
More importantly for us the entries of P can now be
expressed in terms of a summation; namely as
(
),
N
N
X
Y
with ® and sk as defined in (3b) and (3c) so as
to reduce the number of multiplications. Even
further one may exploit the fact that, since the
Poisson-binomial pdf is real-valued,
the argument
Q
j2¼n=(N+1)
(e
¡ sk ),
of the DFT, that is, the values N
k=1
n = 0, 1, : : : , N, must posses an “even” real part and
an “odd” imaginary part (see Bracewell [11] for
example), which means that we only need to compute
half of the entries.
Leaving to others the details of efficiently
implementing the Poisson-binomial pdf formula,
let’s now entertain some of the possibilities
that are opening before us by possessing such a
closed-form expression. We consider the formula
for the Poisson-binomial cdf, the derivation of a
new identity for the binomial coefficient, and new
expressions for the moments of the Poisson-binomial
distribution.
IV. THE POISSON-BINOMIAL CUMULATIVE
DISTRIBUTION FUNCTION
The Poisson-binomial pdf provides the
probabilities of exactly m successes out of N Poisson
trials (m = 0, 1, : : : , N). Perhaps of more practical
use, however, is the complement of its cdf (its
“survival function”–call it Q(m)), which provides
the probabilities of at least m successes in N Poisson
trials; that is
Q(m) =
N
X
P(t),
m = 0, 1, : : : , N
(8)
t=m
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2 APRIL 2010
to maximize the likelihood of success in at least M
future contests.
or equivalently
8
m¡1
X
>
<1 ¡
P(t),
Q(m) =
t=0
>
:
1,
m = 1, 2, : : : , N
:
V.
m=0
(9)
Substituting (5) in for P(t) in (9), and exchanging the
summations, yields
Q(0) = 1
(10a)
Q(m) = 1 ¡
("m¡1
N
X
X
n=0
e¡j2¼nt=(N+1)
t=0
#
N
Y
fpk ej2¼n=(N+1)
k=1
),
+ (1 ¡ pk )g
(10b)
We can now simplify the expression in brackets
in (10b) to obtain a closed-form expression for
the complement of the Poisson-binomial cdf;
namely
(
N
X
(1 ¡ e¡j2¼nm=(N+1) )=(1 ¡ e¡j2¼n=(N+1) )
n=0
¢
),
N
Y
k=1
fpk ej2¼n=(N+1) + (1 ¡ pk )g
(N + 1)
= 1 ¡ m=(N + 1)+
¡
N
X
n=1
(
(1 ¡ e¡j2¼nm=(N+1) )=(1 ¡ e¡j2¼n=(N+1) )
¢
N
Y
k=1
),
fpk ej2¼n=(N+1) + (1 ¡ pk )g
In this section we illustrate the use of the
expressions and techniques presented so far by
using them to derive new representations of the
binomial coefficient and the binomial cdf. These new
expressions are based solely on sums of sinusoidals
and completely avoid the use of factorials.
Consider the well-known expression for the
binomial pdf
μ ¶
B(k j N, p) =
(N + 1),
m = 1, : : : , N:
Q(m) = 1 ¡
A NEW FORMULA FOR THE BINOMIAL
COEFFICIENT
(N + 1)
N
k
pk (1 ¡ p)N¡k
(11)
Clearly, since the relationship between a cdf (call it
C(m)) and its complement is C(m) = 1 ¡ Q(m), one
can also extract from (11) a closed-form expression
for the Poisson-binomial cdf. The outcomes of such
expression (i.e., the C(m), m = 0, 1, : : : , N) would
represent the probability of success in (strictly) less
than m trials out of N.
Applications of the complement of the
Poisson-binomial cdf are countless, from determining
the probability of overall system failure due to faults
in at least M components [1] and that of initiating a
target track after N sensor scans [2] to inferring the
probability of correctly classifying a pattern after
fusing (e.g., via “majority voting”) the declarations
of multiple independent experts [5], or estimating
what resources to allocate to a current action so as
k = 0, 1, : : : , N
(12)
where (N k) = N!=((N ¡ k)!k!) is the “binomial
coefficient.” This coefficient, also known as the “given
N, choose k” function, tells us the number of ways of
choosing k out of N objects without regard of their
order, i.e., the number of combinations.
The binomial pdf provides the probability of
having k successes out of N independent trials, when
the probability of success of each individual trial
has the same value p. It is thus a special case of the
Poisson-binomial pdf, and we can equate (5) and
(12) in such instances. We do so for the special case
p = 1=2, and we change the notation slightly (using
“m” instead of “k” in (12) above) so as to match the
notation of (5).
Setting p = 1=2 in both (12) and (5) and
simplifying and equating both expressions yields
μ ¶
N
X
N
¡N
2
fe¡j2¼nm=(N+1) (ej2¼n=(N+1) + 1)N g=
= 2¡N
m
n=0
(N + 1),
for m = 0, 1, : : : , N:
for
m = 0, 1, : : : , N:
(13)
(Remark: Notice that the left-hand side of (13) is
merely the binomial coefficient function scaled by
the constant 2¡N . In other words one can think of
the binomial coefficient, when considered a function
of m, as a pdf scaled by 2N . This fact can be used to
simplify its computation.)
Factoring the term in parentheses in (13) as
ej2¼n=(N+1) + 1 = ej¼n=(N+1) (ej¼n=(N+1) + e¡j¼n=(N+1) ), and
using the identity ejÁ + e¡jÁ = 2 cos(Á), results in
μ ¶
N
X
N
= 2N
m
n=0
fej¼n(N¡2m)=(N+1) cosN (¼n=(N + 1))g=(N + 1),
m = 0, 1, : : : , N
(14)
or equivalently
μ ¶
N
m
= 2N DFTfej¼nN=(N+1) cosN (¼n=(N + 1))g=(N + 1),
FERNÁNDEZ & WILLIAMS: CLOSED-FORM EXPRESSION FOR THE POISSON-BINOMIAL PDF
m = 0, 1, : : : , N:
(15)
807
Finally exploiting the fact that the left-hand side is
real (meaning that the imaginary part of the right-hand
side must vanish), we can further simplify things to
obtain
μ ¶
N
X
N
= 2N
fcos(¼n(N ¡ 2m)=(N + 1))
m
n=0
¢ cosN (¼n=(N + 1))g=(N + 1),
m = 0, 1, : : : , N:
(16)
This new identity3 expresses the binomial
coefficient in terms of a weighted sum of cosines
rather than as a ratio of factorials, with the
weights being the factors 2N cosN (¼n=(N + 1))=
(N + 1). Since these weights solely depend on N,
they need to be computed only once, and they can be
prestored.4
However, although (16) is perhaps a “cleaner”
expression, writing the identity as in (15) may be of
more general utility. For example consider obtaining
an identity for the power of a cosine: Taking the
inverse DFT (the “IDFT”) of both sides of (15) results
in
μ ¶¾ Á
N ½
X
N
ej2¼nm=(N+1)
(N + 1)
m
m=0
= 2N fej¼nN=(N+1) cosN (¼n=(N + 1))g=(N + 1):
(17)
Solving for cosN (¼n=(N + 1)) yields
μ ¶¾
N ½
X
N
cosN (¼n=(N + 1)) = 2¡N
e¡j¼n(N¡2m)=(N+1)
m
m=0
yielding the well-known trigonometric identity for the
power of a cosine.
Expression (15) can also be used to obtain a
new formulation for the binomial cdf. Roughly
speaking this cdf depicts the probability of success
in at most M out of N trials, when each trial has an
equal probability p of success; it can be portrayed
as
μ ¶¾
M ½
X
N
m
(N¡m)
,
p (1 ¡ p)
m
m=0
M = 0, 1, : : : , N:
(20)
Assuming that p is strictly less than 1 (i.e., 0 · p < 1),
we can perform the manipulation
where
pm (1 ¡ p)(N¡m) = (1 ¡ p)N tm
(21a)
t = p=(1 ¡ p):
(21b)
Substituting this in (20), together with (15) for the
binomial coefficient, yields
μ ¶¾
M ½
X
N
m
(N¡m)
p (1 ¡ p)
m
m=0
= 2N (1 ¡ p)N =(N + 1)
( N
M
X
X
(ej¼nN=(N+1)
tm
¢
m=0
n=0
¡j2¼nm=(N+1)
N
¢ cos (¼n=(N + 1))e
(18)
and, realizing that both sides of the equation must be
real,
μ ¶¾
N ½
X
N
N
¡N
cos (Á) = 2
cos(Á(N ¡ 2m))
:
m
m=0
(19)
)
) :
(22)
By exchanging summations the right-hand side of (22)
becomes
(
N
X
N
N
ej¼nN=(N+1) cosN (¼n=(N + 1))
2 (1 ¡ p) =(N + 1)
n=0
At this particular stage Á = ¼n=(N + 1); however,
realizing that Á doesn’t depend on m allows
generalizing it to represent any desired angle,5 thus
¢
M
X
(te¡j2¼n=(N+1) )m
)
(23a)
m=0
or, after simplifying,
3 An
anonymous reviewer suggests [12] for a current compilation of
binomial identities.
4 Several symmetries can be exploited to reduce the number of
operations required to compute (16), the most obvious being that
of the binomial coefficient about m = N=2 and that of the portion in
braces, for the values of n > 0, about n = (N + 1)=2 (when n = 0 the
portion in braces is always equal to 1, regardless of m).
5 Another way to visualize this is to consider the convolution of
both sides of (19) with a shift-inducing delta function of the
form ±(n ¡ '), with ' chosen such that ¼'=(N + 1) equals the
desired Á.
808
2N (1 ¡ p)N =(N + 1)
N
X
fej¼nN=(N+1) cosN (¼n=(N + 1))
n=0
(1 ¡ tM+1 e¡j2¼n(M+1)=(N+1)) )=
(1 ¡ te¡j2¼n=(N+1) )g:
(23b)
By realizing that (23b), being a cumulative
distribution function, it must be a real number (that
is, its imaginary part must vanish), we obtain
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2 APRIL 2010
M ½
X
m
p (1 ¡ p)
m=0
(N¡m)
μ ¶¾
N
m
N
N
= 2 (1 ¡ p) =(N + 1)
(
T+
N
X
n=1
fcosN (Ãn)[(1 ¡ t) cos(ÃnN) ¡ tM+1 (cos(Ãn(2(M + 1) + 1)N)
2
)
¡t cos(Ãn(2(M + 1) ¡ 1)N))]=(1 ¡ 2t cos(2Ãn) + t )g
with
à = ¼=(N + 1)
(24b)
and
8
(1 ¡ tM+1 )=(1 ¡ t)
>
<
for 0 · p < 1, p 6= 0:5 (i.e., t 6= 1)
T=
>
:
M +1
for p = 0:5 (i.e., t = 1)
(24c)
where L’Hopital’s rule is used to obtain the value of
T for the case where p = 0:5. (Note that, for p = 1,
the binomial cdf equals zero, except when M = N, in
which case it equals 1. Using (24a) directly would be
cumbersome for such case, which would again require
the use of L’Hopital’s rule as t would be undefined.)
Expressions (24a)—(24c) provide a new
representation of the binomial cdf, which is solely
based on trigonometric functions. These, together
with (16)–the new representation for the binomial
coefficient–are, as a reviewer points out, maybe
somewhat surprising, before one realizes that they
are just simple consequences of the binomial theorem
and the properties of the direct and inverse DFT. The
very surprise that the binomial coefficient (a ratio of
factorials) can be formulated as a sum of sinusoidals,
the fact that this formulation is nothing more–and
nothing less!–than a result of the properties of the
DFT, and the fact that different formulations could
have been obtained by selecting other entries for
the Vandermonde matrix in (4) are perhaps the main
contribution of this section as they open opportunities
for obtaining new representations for existing tools
and for recognizing them when they appear under a
different guise.
VI. POISSON-BINOMIAL MOMENTS
The Lth Poisson-binomial moment is the scalar
EfmL P(m)g
=
N
X
m=0
=
fmL P(m)g
N
X
m=0
(
L
m DFT
(N
Y
k=1
j2¼n=(N+1)
fpk e
+ (1 ¡ pk )g
))Á
(N + 1)
Using the well-known identity (e.g., see Bracewell
[11]) that the Lth moment of the DFT of an N + 1
element sequence r(n) satisfies
¯
N
L
X
¯
L
L d
m DFTfr(n)g = ((N + 1)=(2¼j))
(r(n))¯¯
L
dn
n=0
m=0
yields the following expression for the Lth
Poisson-binomial moment:
dL
EfmL P(m)g = (N + 1)L¡1 =(2¼j)L L
dn
ÃN
!¯
¯
Y
¯
¢
fpk ej2¼n=(N+1) + (1 ¡ pk )g ¯
¯
k=1
(26)
n=0
(27)
which mainly involves taking the Lth derivative of the
generating function.
An alternate expression can be achieved, however,
that obviates the need for taking such derivatives.
It is more easily obtained if we represent (25) in
matrix-vector form:
N
X
L
Efm P(m)g =
mL DFTfr(n)g=(N + 1) = gTL Fr
m=0
(28)
QN
j2¼n=(N+1)
+ (1 ¡ pk )g, F is
where r(n) = k=1 fpk e
the DFT matrix, r = [r(0) r(1) ¢ ¢ ¢ r(N)]T with the
superscripted T denoting transposition, and where
gTL = [0 1 2L ¢ ¢ ¢ N L ]=(N + 1).
By taking advantage of the symmetry of the
DFT matrix F, one can express the product gTL F as
the transpose (without conjugation) of the vector
that results from taking the DFT of gL ; that is,
EfmL P(m)g = GTL r, where vector GL = DFTfgL g.
Since gL is independent of the probabilities of success
of the independent trials, GL can be precomputed
for the desired values of N and L. Calculation of the
corresponding moment thus corresponds to a weighted
sum of the results of evaluating the Poisson-binomial
generating function at the values ej2¼n=(N+1) , n =
0, 1, : : : , N.
We now rederive this result for those who prefer
summations to matrices.
We return to (28) and express the DFT in all its
glory;
(
)Á
N
N
X
X
EfmL P(m)g =
(25)
(24a)
mL
m=0
n=0
fe¡j2¼nm=(N+1) r(n)g
where use is made of (5) and (6) for P(m).
FERNÁNDEZ & WILLIAMS: CLOSED-FORM EXPRESSION FOR THE POISSON-BINOMIAL PDF
(N + 1):
(29)
809
TABLE I
Values of the Arithmetic Series Divided by N + 1 and Functions Resulting after Evaluating Lth Derivative of u(x) at x = n, for L = 1, 2,
3, and 4
N
X
L
¯
¯
dL
u(x)¯
¯
dxL
mL =(N + 1)
m=1
1
N=2
2¼j=(1 ¡ e¡j2¼n=(N+1) )
2
N(2N + 1)=6
(4¼ 2 =(1 ¡ e¡j2¼n=(N+1) ))f1 + 2e¡j2¼n=(N+1) =[(1 ¡ e¡j2¼n=(N+1) )(N + 1)]g
3
N 2 (N + 1)=4
(¡8¼ 3 j=(1 ¡ e¡j2¼n=(N+1) ))f1 + 3(N + 2)e¡j2¼n=(N+1) =[(1 ¡ e¡j2¼n=(N+1) )(N + 1)2 ] +
6e¡j4¼n=(N+1) =[(1 ¡ e¡j2¼n=(N+1) )2 (N + 1)2 ]g
4
N(6N 3 + 9N 2 + N ¡ 1)=30
(¡16¼ 4 =(1 ¡ e¡j2¼n=(N+1) ))f1 + (2e¡j2¼n=(N+1) =[(1 ¡ e¡j2¼n=(N+1) )(N + 1)3 ])f2N 2 + 7N + 7 +
6(N + 3)e¡j2¼n=(N+1) =(1 ¡ e¡j2¼n=(N+1) ) + 12e¡j4¼n=(N+1) =(1 ¡ e¡j2¼n=(N+1) )2 gg
By noticing that r(0) = 1 and by exchanging
summations,
( N
)
X
L
L
Efm P(m)g =
m =(N + 1)
m=0
+
N
N X
X
[mL e¡j2¼nm=(N+1) ]r(n)=(N + 1):
(30)
n=1 m=0
The quantity in brackets canP
be expressed as
¡j2¼xm=(N+1)
)jx=n
[(N + 1)=(¡2¼j)]L (dL =dxL )( N
m=0 e
or equivalently as [(N + 1)=(¡2¼j)]L (dL =dxL )
f(1 ¡ e¡j2¼x )=(1 ¡ e¡j2¼x=(N+1) )gjx=n ; hence
( N
)
X
EfmL P(m)g =
mL =(N + 1)
m=1
+ (N + 1)
L¡1
=(¡2¼j)
L
N ½
X
n=1
¯
¯
dL
r(n) L u(x)¯¯
dx
x=n
¾
(31)
where
u(x) = (1 ¡ e¡j2¼x )=(1 ¡ e¡j2¼x=(N+1) ):
Since u(x) does not depend on the probabilities of
success of the individual trials, the function resulting
after evaluating its Lth derivative at x = n may be
worked out and tabulated beforehand (likewise with
the first term of (31)).
Table I shows such functions of n for the values of
L from 1 to 4. One can obtain further simplifications
by comparing the expressions for the first and
second moments obtained using (31) to those that
would have been obtained using basic principles
(i.e., EfmP(m)g = 1T p, and Efm2 P(m)g = pT (1 ¡ p)
+(1T p)2 , where 1 is a vector of ones and p is a
vector comprised of the individual probabilities of
success–see [6]).
VII. EXAMPLE
For purposes of illustrating the techniques
of this paper, in this section we address one of
810
x=n
the application examples mentioned in Section I,
namely, case 6. In summarized form: given that
each of K existing workstations have a probability
pk of achieving their production goals, what
probability of success do we need to attain in each
of L additional workstations if we are to ensure a
specified probability Ps that at least M-of-the-(K +
L) workstations achieve their production quotas.
(We assume here, for purposes of simplification,
that the probability of success for each of the new
workstations is equal–call it pc .)
First let us briefly further abstract and simplify
this example in an attempt to obtain a more intuitive
understanding of the underlying problem and its
solution (and hopefully open the door to more ideas
about its applicability). Namely let us consider the
problem where we are given the not-necessarily-equal
probabilities of success of each of N individual
trials, and we are then told that one additional trial
is allowed. What probability of success pc would
we need to assign to that extra trial so as to attain a
desired probability Ps of “at least M successes out of
the N + 1 trials”?
Clearly in such a case, the desired probability Ps
may be expressed as
Ps = Pr(“at least M out of N”)
+ pc ¢ Pr(“exactly M ¡ 1 out of N”):
(32)
Notice that this is a first-order polynomial in pc . In
solving for pc we obtain the value
pc = (Ps ¡ Pr(“at least M out of N”))=
Pr(“exactly M ¡ 1 out of N”)
(33)
as the estimate of the probability of success needed to
achieve exactly the desired Ps at the fatidic (N + 1)th
try. (Notice, however, that the above expression
for Ps has no safeguards to ensure that pc is a valid
probability; hence for it to be of practical use, and
depending on the application, the resulting value is
usually either discarded or at least clipped, e.g., set
to zero if negative or truncated when exceeding a
user-selected threshold.)
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2 APRIL 2010
By extending this example to the case where two
new trials are allowed, we would obtain
Ps = Pr(“at least M out of N + 1”)
+ pc ¢ Pr(“exactly M ¡ 1 out of N + 1”)
= (Pr(“at least M out of N”)
+ pc ¢ Pr(“exactly M ¡ 1 out of N”))
+ pc ¢ (pc ¢ Pr(“exactly M ¡ 2 out of N”)
+ (1 ¡ pc ) ¢ Pr(“exactly M ¡ 1 out of N”))
= qp2c + 2pc ¢ Pr(“exactly M ¡ 1 out of N”)
+ Pr(“at least M out of N”),
(34a)
where
q = Pr(“exactly M ¡ 2 out of N”)
¡ Pr(“exactly M ¡ 1 out of N”):
(34b)
The solution to this quadratic polynomial in pc is, of
course, of the form
pc = [¡Pe (M ¡ 1) § fPe (M ¡ 1)2
+ (Ps ¡ Pr(“at least M out of N”))qg1=2 ]=q
(35)
with the notation Pe (L) meaning Pr(“exactly L out
of N”). Again neither pc may be a valid probability,
and, depending on the application, it may also have to
be discarded or clipped.
Continuing the drill any further simply complicates
the formulation of the problem and its solution, hence
the advantage of using the methodology of this paper.
Nonetheless hopefully some insight may be obtained
out of the exercise and maybe even some inspiration
for new application ideas.
By returning now to the example and
appropriately tailoring (3a) to it, we can see that
the Poisson-binomial pdf for this particular case
corresponds to the coefficients of the polynomial
This provides, as one of its outputs, the probability
that at least M-of-the-(K + L) workstations will
achieve their production quotas. By exploiting the
monotonicity of the Poisson-binomial cdf and, hence,
of its complement (the probability of at least M
successes out of N trials), one can then use a method
such as bisection, or equivalently MATLAB’s “fzero”
function, to iteratively converge to the solution.
(Fig. 2 presents a MATLAB version of this approach.)
Notice that one only needs to perform this guessing
game if, when setting pc = 0, one doesn’t meet
or exceed the desired Ps , but, when setting pc = 1,
one does. In the first case (pc = 0), the production
quota is achieved without the need of any additional
workstations, while, in the second (pc = 1), the
quota is not reached with the addition of only L of
them.
Even though, numerically speaking, the function
of Fig. 2 is more than sufficient to address our
example, it may still be of interest to go through the
development of the expressions implicitly contained
therein.
Letting N = K + L and using (11) yields, for
m = M,
Q(M) = Ps = 1 ¡ M=(N + 1)
¡
N
X
(
(1 ¡ e¡j2¼nM=(N+1) )=(1 ¡ e¡j2¼n=(N+1) )
n=1
N
Y
¢
k=1
j2¼n=(N+1)
fpk e
)Á
+ (1 ¡ pk )g
(N + 1)
= 1 ¡ M=(N + 1)
¡
N
X
n=1
(
(1 ¡ e¡j2¼nM=(N+1) )=(1 ¡ e¡j2¼n=(N+1) )
¢ (pc ej2¼n=(N+1) + (1 ¡ pc ))L
¢
p(z) = ®(z ¡ r)L (z ¡ s1 )(z ¡ s2 ) ¢ ¢ ¢ (z ¡ sK )
K
Y
k=1
j2¼n=(N+1)
fpk e
+ (1 ¡ pk )g
)Á
(N + 1)
(37)
(36a)
where
® = pLc
N
Y
pk
(36b)
k=1
r = ¡(1 ¡ pc )=pc
(36c)
sk = ¡(1 ¡ pk )=pk :
(36d)
and
An anonymous reviewer suggested the following very
simple, practical, and stable numerical approach for
solving the example problem using these results.
Guess a value for pc , and feed the vector pT =
[pc 1TL p1 p2 ¢ ¢ ¢ pK ], where 1L denotes a vector of
L “ones,” to function “ProbMofN” of Fig. 1(a).
where use is made of the assumption that pc , the
probability of reaching the production quota, is the
same for all the new workstations.
Alternatively
(N + 1)(1 ¡ Ps ) ¡ M
(
N
X
(1 ¡ e¡j2¼nM=(N+1) )=(1 ¡ e¡j2¼n=(N+1) )
n=1
¢ (pc ej2¼n=(N+1) + (1 ¡ pc ))L
)
K
Y
j2¼n=(N+1)
¢ fpk e
+ (1 ¡ pk )g
(38)
k=1
FERNÁNDEZ & WILLIAMS: CLOSED-FORM EXPRESSION FOR THE POISSON-BINOMIAL PDF
811
Fig. 2. Given vector with non-zero probabilities of success of each of K individual, independent trials, this MATLAB code computes
probability of success that would be needed in each of L additional trials to obtain desired probability of success in at least M out of
K + L trials. (This function can be used directly to solve the problem of Section VII.)
where we have collected the terms outside the
summation, and the scale factor (N + 1), on the
left-hand side of the expression.
Since the right-hand side of (38) is a polynomial
in pc , we may follow again the strategy of expanding
this polynomial by using the approach of Section III
“correcting” the zeroth-order term by subtracting from
it the left-hand side of (38), and then solving for its
zeros by picking as pc any such zero that is also a
valid probability, if it exists.
Solving for the coefficients of the right-hand size
of (38) by using the techniques of Section III involves
substituting for pc values of the form ej2¼l=(N+1) ,
l = 0, 1, : : : , L, computing the L + 1 element sequence
of values resulting from such substitutions, and then
taking the DFT of the sequence and scaling it by
1=(L + 1). In other words the coefficients of the
C(u) =
N
X
(
n=1
¢
812
(1 ¡ e
" L
X
`=0
¡j2¼nM=(N+1)
polynomial of the right-hand side of (38) are given by
C(u), u = 0, 1, : : : , L, where
C(u) =
e¡j2¼lu=(N+1)
`=0
à N (
X
¢
(1 ¡ e¡j2¼nM=(N+1) )=(1 ¡ e¡j2¼n=(N+1) )
n=1
¢ (ej2¼(n=(N+1)+l=(L+1)) + (1 ¡ ej2¼l=(N+1) ))L
)! Á
K
Y
j2¼n=(N+1)
¢ fpk e
+ (1 ¡ pk )g
(L + 1),
k=1
u = 0, 1, : : : , L:
(39)
Exchanging summations and simplifying yields
¡j2¼n=(N+1)
)=(1 ¡ e
L
X
K
Y
) (pk (ej2¼n=(N+1) ¡ 1) + 1)
k=1
e¡j2¼lu=(N+1) (ej2¼l=(L+1)) (ej2¼n=(N+1) ¡ 1) + 1)L
#Á
(L + 1),
)
u = 0, 1, : : : , L:
(40)
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2 APRIL 2010
By using matrix notation one may condense (40):
observe that the factor in brackets is equivalent to
taking the DFTs of the columns of an (L + 1) £ N
matrix (call it “E”) with element E(l + 1, n) defined
by
E(l + 1, n) = (ej2¼l=(L+1) (ej2¼n=(N+1) ¡ 1) + 1)L ,
l = 0, 1, : : : , L,
n = 1, 2, : : : , N:
(41)
Also notice that the portion in braces of (40) can be
expressed as the element-by-element (“Hadamard”)
product of N-element vectors d and r, with elements
given by
d(n) = (1 ¡ e¡j2¼nM=(N+1) )=(1 ¡ e¡j2¼n=(N+1) )
(42a)
and
r(n) =
K
Y
(pk (ej2¼n=(N+1) ¡ 1) + 1),
n = 1, 2, : : : , N
k=1
(42b)
or, alternatively, letting the d(n) represent the nth
diagonal element of a diagonal matrix D, said product
may also be represented as the matrix-vector product
Dr. In combining these results (40) can be expressed
in matrix/vector form as
C = FEDr=(L + 1)
(43)
where F is the DFT matrix.
As an aside notice that evaluating (43) by
proceeding from right-to-left is efficient in that
it involves a Hadamard product, followed by
matrix-vector product and an “FFT”. However notice,
also, that since the product FED does not depend on
the pk , it can be precomputed for a given (K, L, M)
triad and then used to obtain C for multiple sets of pk .
Having obtained the coefficients of the polynomial
that comprise the right-hand side of (38), we may now
subtract the left-hand side from the zeroth-order term
(i.e., from C(0) or, equivalently, from the first element
of vector C; that is
q = C ¡ ((N + 1)(1 ¡ Ps ) ¡ M)e1
(44)
where e1 denotes the (L + 1)-element vector
[1 0 ¢ ¢ ¢ 0]T .
The zeros of the polynomial q(L + 1)xL +
q(L)xL¡1 + ¢ ¢ ¢ + q(2)x + q(1) should include the
solution to our example, if such solution exists. These
zeros can be obtained by using the companion matrix
approach described in Section VIII; however, as
discussed in all numerical analysis textbooks, it is well
known that the roots of a polynomial are not easily
found.6 It is, thus, more sensible to exploit the fact
that since the entries of q are all real numbers and
6 Reference [13], suggested by an anonymous reviewer, presents an
excellent, concise description of the difficulties inherent to obtaining
polynomial roots.
the zero of interest (if it exists) must lie between 0
and 1, we can find it using an iterative zero-finding
approach, such as a combination of a Newton-type
method and a bisection algorithm (or equivalently
MATLAB’s “fzero” function).
Some comments regarding this section now follow.
First notice that by setting the pk k = 1, 2, : : : , K equal
to zero in the expressions above, our formulation
also solves example 5 of Section I; in other words it
also addresses a particular class of “inverse” binomial
problem.
Also (even though we do not show it here), by
treading through the developments of this section
for the case where the pk are zero, one can derive
an alternative expression for the probability of
“at least M successes out of N Bernoulli (i.e.,
equal-probability) trials” which is already in expanded
polynomial form; namely for M > 0,
μ ¶
N
X
N
n
N¡n
p (1 ¡ p)
n
n=M
"
¶!#
μ ¶ ÃX
μ
N
M
X
N
n
n
n
m
=
(¡1)
p (¡1)
n
m¡1
n=M
m=1
(45)
where the left-hand side denotes the classical
form of expressing such probability and where the
right-hand side represents it in expanded polynomial
form. Notice that, since all the coefficients can be
precomputed, solving a bounded single-variable
problem of the form
"
¶!#
μ ¶ ÃX
μ
N
M
X
N
n
n
n
m
Ps =
(¡1)
pc (¡1)
n
m¡1
n=M
m=1
(46)
can again be done using an iterative zero-finding
algorithm.
VIII. THE INVERSE POISSON-BINOMIAL PROBLEM
The practical definition of the “inverse” of a
Poisson-binomial pdf usually lies in the eye of the
practitioner, which means that there are as many
variants and complications as there are applications
for this problem. Nonetheless one such definition
may be simple enough: given an (N + 1)-element
Poisson-binomial pdf, find (within a permutation) the
probabilities of success of the N independent Poisson
trials that gave rise to it. Theoretically these individual
probabilities should be obtainable using either the
MATLAB routine of Fig. 1(b) or, adding more detail,
by using the companion matrix-based approach
described shortly (both are equivalent). Unfortunately
as will be seen, numerical stability issues can
render the algorithm of Fig. 1(b) useless even for
moderately-sized problems (e.g., N > 30). This section
FERNÁNDEZ & WILLIAMS: CLOSED-FORM EXPRESSION FOR THE POISSON-BINOMIAL PDF
813
thus merely sketches some general hurdles, describes
what may be meant by “inverse” under different
circumstances, and references applications where such
inverses and hurdles may be encountered. A more
detailed explanation of some of these problems and
their solutions will be the subject of a future paper.
As can be ascertained from the expressions in
Section II it is always possible to associate any
N positive numbers that do not exceed unity to a
Poisson-binomial pdf. Practical issues regarding
the sensitivity of this pdf’s entries to errors in
the estimates of the probabilities of success of
the individual trials, whether the trials are or not
independent or whether the pdf matches or not
our particular expectations, are immaterial from
the perspective that the result exhibits, under all
viewpoints, the characteristics of a Poisson-binomial
pdf.
The converse, however, is not so clear-cut. Of
course any N + 1 nonnegative numbers not exceeding
unity, and adding up to 1 (i.e., a “pdf”), do not, in
general, represent a Poisson-binomial pdf and, as
such, we shouldn’t expect to be able to factorize its
generating function into terms of the form of (3).
But the problem is a little more insidious than that:
even when a pdf’s probabilities indeed proceed from
a Poisson-binomial process, we may still not be able
to properly factorize the pdf’s generating function
due to errors in the estimates (or “measurements”)
of those probabilities as well as to inherent numerical
instabilities.
An approach for handling the case where there are
errors in the pdf entries is to attempt to solve, instead,
for the roots of the polynomial that qualifies as a
generating function for a Poisson-binomial pdf and
is closest, per some norm, to the generating function
of the given, erroneous pdf data. (Experimental
Presults
seem to indicate that the l1 norm (i.e., kxk1 = k jxk j)
may be generally better suited than the quadratic for
addressing the inverse Poisson-binomial problem.)
Such a polynomial should satisfy the constraints
that its coefficients should constitute a valid pdf
and that all of its roots (i.e., the sk of (3)) should be
nonpositive real numbers.
Another perspective can be gained by formulating
the problem as that of finding the eigenvalues of a
matrix that is a companion to the pdf’s generating
function in the sense that the latter’s generating
function equals, within a scale factor, the characteristic
polynomial of the former. For example monic
polynomials xN + aN¡1 xN¡1 + ¢ ¢ ¢ + a1 x + a0 can be
associated to “companion matrices” of the form
¸
· T
¡a ¡a0
(47)
Ac =
I
0
where aT = [aN¡1 aN¡2 ¢ ¢ ¢ a1 ], I is an (N ¡ 1) £
(N ¡ 1) identity matrix, and 0 is an (N ¡ 1) element
zero vector. It can be easily verified that the
polynomial above is also the characteristic polynomial
814
of this matrix; hence the roots of the polynomial are
the eigenvalues of the companion matrix. (This is the
approach followed in the MATLAB function “roots”.)
Since, theoretically, a polynomial’s roots are
not affected by factoring out the coefficient of the
polynomial’s highest order term, one can, in principle,
use this eigenvalue method to solve for the roots
of any polynomial. One can conceptually exploit
such a fact to, under ideal circumstances, solve
for the sk on the right-hand side of (3); then, since
sk = ¡(1 ¡ pk )=pk , one can obtain the probabilities of
success in the individual, independent Poisson trials as
pk = 1=(1 ¡ sk ):
(48)
Unfortunately in practice this is an extremely
numerically-unstable problem [13]. Depending on
the values of the Poisson-binomial pdf and of the
individual-trial probabilities that implicitly spawned it,
even minute errors introduced by computer round-off
may result in large eigenvalue displacements, and
hence, in estimates for individual-trial probabilities
that do not meet requirements. For example Fig. 3
shows the outcome of the simple test of running,
in succession, the two routines of Fig. 1. First a
list of N = 50 uniform random numbers between
0 and 1 is generated, and the corresponding
Poisson-binomial pdf is computed by using the first
routine. The second routine is then run using the
output of the first as the input. Fig. 3 shows the
results of this final step. The circles represent the
randomly-generated individual-trial probabilities,
while the Xs represent the output of the inverse
Poisson-binomial process (i.e., the estimates of
those individual-trial probabilities when given their
Poisson-binomial pdf). As can be seen even with
MATLAB’s fantastic precision, about half of those
estimates are complex numbers.
Reprising: perturbations of the elements of the
Poisson-binomial pdf (i.e., of the coefficients of its
generating function) can be viewed as perturbations of
selected elements of its companion matrix which, even
when infinitesimally small, can cause large eigenvalue
excursions. The problem is, thus, how to solve for
the probabilities of success of the individual trials
when, due to measurement or experimental error,
the perturbations of the pdf elements are anything
but infinitesimal. Experiments suggest that a way of
approaching this problem may be by determining and
operating on a “more appropriate” companion matrix
than the one defined by (47). Such a matrix could
be constrained to correspond to a pdf’s generating
function, have nonpositive real eigenvalues (e.g., it
could be designed as Ac = ¡AP , where AP is a real,
positive semi-definite symmetric matrix), and ensue
from a “minimal” perturbation of the original data.
That is, the inverse Poisson-binomial problem, as
stated, would involve, in practice, the formulation
(and solution) of a highly structured (and constrained)
problem.
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2 APRIL 2010
Fig. 3. Complex plot resulting from running in succession routines of Fig. 1 on an input consisting of list of N = 50 uniform random
numbers between 0 and 1.
But, albeit very interesting, the “inverse problem”
just presented is not the one of most practical concern
now, as one can seldom determine or specify all
the entries of a particular pdf. Namely estimating
these entries through experimentation may not be
viable and, on the other hand, specifying them most
certainly produce a pdf which is not of the persuasion
desired. Thus its use in estimating/evaluating the
probability of success of individual trials and/or
in determining whether the underlying process is
indeed Poisson-binomial may be mainly limited
to experimental settings with tightly controlled
conditions (e.g., see [5]).
Most inverse problems of current interest involve
the determination of some of the individual-trial
probabilities of success that would be needed to
achieve specific entries of either the Poisson-binomial
pdf or of its cdf’s complement. For example in many
applications, the “inverse problem” of interest is the
one of Section VII: some of the individual-event
probabilities are assumed known, and the objective
is to estimate the probabilities (usually assumed
constant) that should be achieved in the future so as
to obtain a desired probability of “at least M successes
out of N trials” for specified values of M and N (see
[1], for example). As we see the techniques sketched
in this paper can be used to attack this problem.
Another inverse problem, very popular in
reliability modeling circles (see [1]) and in radar
engineering applications (e.g., for the binary
integration of received pulses [3—4]), involves the
determination of the optimal values of M and, at
times, N. For example an important problem in
reliability modeling involves solving for the M and N
that, given the “reliabilities” (akin to the probabilities
of success) of individual components (or “events”),
minimize an objective function that accounts for
the cost of each component and the cost of system
failure. Regardless of the application of this particular
“inverse” problem, however, and again for purposes of
simplification, the individual probabilities of success
are usually assumed to be equal across events.
Finally since as part of an “inversion” operation
one may need to obtain a pdf when given a cdf or its
complement, we now present, for completeness’ sake,
such relationship in simple matrix form. Beginning
with the mapping from the pdf into/onto the cdf’s
complement, we obtain
HP = Q
P = H ¡1 Q
and/or
(49)
where (N + 1)-element vectors P and Q contain,
respectively, the entries of the pdf and of the
complement of the cdf. (For the Poisson-binomial
case, the entries of these vectors would be arranged as
follows. The nth entry of P represents the probability
of exactly n ¡ 1 successes in N trials, while the nth
entry of Q represents the probability of at least n ¡ 1
successes in N trials.)
H and its inverse are (N + 1) £ (N + 1) matrices of
the form
21 1 1 ¢¢¢ 1 13
60 1 1 ¢¢¢ 1 17
6
7
6
7
6
7
0
0
1
¢
¢
¢
1
1
H=6
7
6 .. .. ..
.. .. 7
4 . . . ¢¢¢ . . 5
0
0 0
¢¢¢
0
1
2 1 ¡1 0 ¢ ¢ ¢ 0 0 3
6 0 1 ¡1 ¢ ¢ ¢ 0 0 7
6
7
6. .
..
..
.. 7
¡1
6
7
.
.
H =6.
.
.
¢¢¢ .
. 7:
6
7
40 0
0 ¢ ¢ ¢ 1 ¡1 5
0
0
FERNÁNDEZ & WILLIAMS: CLOSED-FORM EXPRESSION FOR THE POISSON-BINOMIAL PDF
0
¢¢¢
0
(50)
1
815
Considering now the into/onto mapping between the
pdf and its cdf, we have that since C = 1 ¡ Q, where
1 represents an (N + 1)-element vector of ones, and
the entries of C, the (N + 1)-element cdf vector, are
ordered (in the Poisson-binomial case) so that the
nth entry corresponds to the probability of strictly
less than n ¡ 1 successes in N trials, the relationship
between P and C becomes
1 ¡ HP = C
and/or
P = H ¡1 (1 ¡ C): (51)
REFERENCES
[1]
IX. SUMMARY
This paper presents a methodology for obtaining
closed-form representations for a number of
Poisson-binomial expressions of common practical
use. In a nutshell since these expressions can all
be traced back to the sum of random variables
drawn from binary distributions (Bernoulli-Poisson
trials), they can be formulated in terms of the linear
convolution of N two-element sequences. In turn
the output of such convolution can be related to the
coefficients of the polynomial that results from the
product of monomials associated to the two-element
sequences being convolved. By exploiting the
Vandermonde polynomial interpolation approach,
all these operations can then be written as part of
a single expression that consists of a finite sum
of trigonometric products. Particular expressions
derived in this paper using this methodology are the
closed-form formulations for the Poisson-binomial
pdf and for the complement of its cdf ((5) and (11),
respectively); the new trigonometric identities for the
binomial coefficient and the binomial cdf ((16) and
(24a)—(24c), respectively); and (31), which provides
the Poisson-binomial moments.
Furthermore since going from the binary
distributions to the Poisson-binomial expressions can
be conceptualized as the product of monomials, it is
shown that the binary distributions of the individual
trials can be obtained from the Poisson-binomial
expressions by solving for the monomials’ coefficients
or, equivalently, for the roots of the polynomial
that correspond to the Poisson-binomial expression
of interest. The MATLAB routine of Fig. 1(b) is
provided to solve this inverse Poisson-binomial
problem for relatively small-sized cases (less than 25
individual trials), and stability issues are discussed,
which suggests that such an inverse problem could
be made more robust by formulating it in terms of
finding the eigenvalues of a real, symmetric, positive
semi-definite companion matrix.
816
The just-described procedures, which
implementation-wise are very attractive, can be used
as tools for signal detection and tracking [2—4] and
for evaluating the fusion of the outcomes of different
processing strings [5]. And even a cursory look
suffices to show their all-encompassing applicability
to the various fields of reliability optimization
theory [1].
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
Kuo, W., and Zuo, M.
Optimal Reliability Modeling.
Hoboken, NJ: Wiley, 2003.
Blackman, S.
Multiple-Target Tracking with Radar Application.
Norwood, MA: Artech House, 1986.
Shnidman, D.
Binary integration for Swerling target fluctuations.
IEEE Transactions on Aerospace and Electronics Systems,
34, 3 (1998), 1043—1053.
Frey, T.
An approximation for the optimum binary integration
threshold for Swerling II targets.
IEEE Transactions on Aerospace and Electronics Systems,
32, 3 (1996), 1181—1185.
Fernández, M., and Aridgides, A.
Measures for evaluating sea-mine identification
processing performance and the enhancements provided
by fusing multisensor/multiprocess data via an
M-out-of-N voting scheme.
Proceedings of SPIE, vol. 5089, 2003.
Feller, W.
An Introduction to Probability Theory and Its Applications,
vol. I.
Wiley Series in Probability and Mathematical Statistics,
New York: Wiley, 1968.
McClellan, J., and Rader, C.
Number Theory in Digital Signal Processing.
Upper Saddle River, NJ: Prentice-Hall, 1979.
Belfore, L.
An O(n(log2 (n))2 ) algorithm for computing the reliability
of k-out-of-n : G & k-to-l-out-of-n : G systems.
IEEE Transactions on Reliability, 44, 1 (1995), 132—136.
Radke, G., and Evanoff, J.
A fast recursive algorithm to compute the probability of
M-out-of-N events.
In Proceedings of the Annual Reliability and
Maintainability Symposium, 1994, 114—117.
Chen, S., and Liu, J.
Statistical applications of the Poisson-binomial and
conditional Bernoulli distributions.
Statistica Sinica, 7 (1997), 875—892.
Bracewell, R.
The Fourier Transform and Its Applications.
Columbus, OH: McGraw-Hill, 1999.
http://binomial.csuhayward.edu/Identities.html.
http://en.wikipedia.org/wiki/Wilkinson’s polynomial.
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 46, NO. 2 APRIL 2010
Manuel Fernández (M’98) obtained his B.S.E.E. from the Georgia Institute
of Technology, Atlanta, in 1979 and his M.S.E.E. and Eng.E.E. degrees from
Stanford University, Stanford, CA, in 1980 and 1982.
In 1983 he joined General Electric’s Electronic Laboratory in Syracuse,
NY, as a research engineer, and in 1990 he became part of the GE Aerospace
Advanced Engineering department (currently Lockheed Martin—MS2 Strategic
Research and Technology Development group). His areas of interest include,
among others, linear and nonlinear adaptive processing, signal detection and
classification, and mission and sensor management.
Stuart Williams received his BSEE from John Brown University, Siloam Springs,
AR, in 1981 and his M.S.E.E. from Syracuse University, Syracuse, NY, in 1984.
He held a variety of radar and sonar System Engineering positions for
General Electric Company Aerospace from 1981—1992, primarily pursuing new
business opportunities for the Advanced Development Engineering group. He
worked as a Senior System Engineer for Lockheed Martin Corporation from
1992—2007, with primary emphasis on ground based radar applications, including
the development of sensor management techniques. Since 2007 he has worked for
Sensis Corporation in the area of long range expeditionary radar.
FERNÁNDEZ & WILLIAMS: CLOSED-FORM EXPRESSION FOR THE POISSON-BINOMIAL PDF
817