Contents
Page
1 Discrete Probability Spaces
1.1 Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Set Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Discrete Probability Theory . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 The Relative Frequency of an Event and Equiprobable Probability Spaces . .
1.5 The Calculus of Discrete Probability Spaces . . . . . . . . . . . . . . . . . .
1.6 Independent Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.7 Conditional Probability and Baye’s Theorem . . . . . . . . . . . . . . . . . .
1.8 Chapter 1 Homework Exercises . . . . . . . . . . . . . . . . . . . . . . . . .
1
1
5
13
20
25
29
34
39
2 Elementary Combinatorics and Arrangements of Sets
2.1 The Addition and Multiplication Rules . . . . . . . . . . . . . . . . . . . . .
2.2 The Definition of an Arrangement and r-Orderings . . . . . . . . . . . . . .
2.3 r-Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 r-Combinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5 The Binomial Theorem and the Principal of Inclusion and Exclusion . . . . .
2.6 The Multinomial Theorem and Ordered Partitions . . . . . . . . . . . . . . .
2.7 r-Unorderings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.8 Chapter 2 Homework Exercises . . . . . . . . . . . . . . . . . . . . . . . . .
44
44
52
54
57
62
67
70
73
3 Discrete Random Variables
76
3.1 Partitions qua Equivalence Relations and Quotient Sets . . . . . . . . . . . . 76
3.2 Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.3 The Histogram and The Cumulative Distribution Function . . . . . . . . . . 87
3.4 Transformations, Generating Functions, and the Law of the Unconscious Statistician or the Transformed Expected Value Rule . . . . . . . . . . . . . . . . 89
3.5 Discrete Random Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.6 The Expected Value E(X) and Variance V (X) of a Discrete Random Variable
X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.7 Independent Random Variables and the Coefficient of Correlation . . . . . . 103
3.8 Chebyshev’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.9 Chapter 3 Homework Exercises . . . . . . . . . . . . . . . . . . . . . . . . . 112
4 Parametric Families of Discrete Random Variables
115
4.1 Bernoulli Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.2 The Binomial Random Variable B(n, p) . . . . . . . . . . . . . . . . . . . . . 117
4.3 The Negative Binomial Random Variable Q(r, p) . . . . . . . . . . . . . . . . 122
4.4 The Geometric Random Variable G(p) . . . . . . . . . . . . . . . . . . . . . 128
4.5 The Hypergeometric Random Variable H(M, N, k) . . . . . . . . . . . . . . 131
4.6 The Poisson Random Variable π(λ) . . . . . . . . . . . . . . . . . . . . . . . 138
4.7 The Multinomial Random Variable M (n; xi ; pi ) . . . . . . . . . . . . . . . . 144
1
4.8
4.9
The Multivariate Hypergeometric Random Variable . . . . . . . . . . . . . . 147
Chapter 4 Homework Exercises . . . . . . . . . . . . . . . . . . . . . . . . . 148
5 Markov Chains
154
5.1 Naive Category Theory and the Category of Automata CA . . . . . . . . . . 154
5.2 Elementary Introduction to Graphs and Trees . . . . . . . . . . . . . . . . . 156
5.3 Matrix Representations of Graphs . . . . . . . . . . . . . . . . . . . . . . . . 160
5.4 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5.5 Markov Processes and Markov Chains . . . . . . . . . . . . . . . . . . . . . . 169
5.6 Communication Classes of Markov Chains and Simple Random Walks . . . . 177
5.7 Random Walks on Finite Graphs . . . . . . . . . . . . . . . . . . . . . . . . 181
5.8 Regular Markov Chains and the Page Rank Algorithm . . . . . . . . . . . . 186
5.9 Chapter 5 Homework Exercises . . . . . . . . . . . . . . . . . . . . . . . . . 190
6 Markov Decision Processes
193
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
6.2 Finite Horizon Markov Decision Processes . . . . . . . . . . . . . . . . . . . 194
6.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
7 Quantum Probability with an Eye Toward Quantum Computing
2
197
1
Discrete Probability Spaces
In this chapter we introduce the mathematical objects encoding both the events observed
in an experiment and the likelihood of their occurrence. Such objects are characterized by
triples (Ω, β, P ), consisting of a countable set Ω, the set of all of its subsets β, and a set
function P : β → [0, 1] satisfying a list of three axioms. Accordingly, we have occasion here
to begin our study of discrete probability theory by introducing the aspects of set theory
required to understand this object. The reader may object such generality is unnecessary;
however, such objections might be naive, for they neglected to consider the necessity of such
generality in several both major and modern applications.
1.1
Set Theory
In this section, we introduce enough set theory to study the main ideas of this text.
Definition 1. We say a collection of elements, denoted S, is a set. Furthermore, we say
any not necessarily proper subcollection of elements, say T , is a subset. We denote the subset
relation among sets by ⊂ and write T ⊂ S. We write |S| for the number of elements in S,
and refer to this number as its cardinality.
In terms of elements, the subset relation is satisfied by the condition that every element t
of T , or in notation, t ∈ T , that we also have t ∈ S. The subset relation between sets allows
us to define the equality of these objects.
Definition 2. Let S and T be sets, then we say S is equal to T and write S = T if and only
if both T ⊂ S and S ⊂ T are true.
Given this definition, one must demonstrate both conditions logically equivalent to the
equality of two sets hold in order to demonstrate that they are equal. We shall practice this
requirement in both examples and exercises.
Next, we shall introduce the implements at a mathematician’s disposal for generating
sets from those that they are given, in much the same way that one is able to generate
sets of numbers from a subset of the same by means of arithmetic operations. Indeed, we
shall introduce set operations for this purpose. They are named according to the arithmetic
operations that they generalize.
Definition 3. Let S and T be sets. Then we say the set of elements in either S or T, denoted
S ∪ T , is their union. Furthermore, we say the set of elements in both S and T, denoted
S ∩ T , is their intersection.
There is other notation common in the literature used to describe these operations. One
reads
S ∪ T = { x | x ∈ S OR x ∈ T }
as elements x ’such that’ for ’|’ x is an element of either S or T . This elemental notation
easily generalizes to other sets and we shall use it throughout the text. Indeed, S ∩ T =
{ x | x ∈ S AND x ∈ T }. One might notice the curious capitalization of the conjunctions
1
’or’ and ’and’ that we used in the defining conditions aspect of our set notation. The reason
for this is to emphasize that these conditions are defined logically. We shall review what we
need of formal logic below and its relation to set theory, however, the relation may perhaps
already be clear to the learned reader.
In terms of a comparison to arithmetic operations, these set operations we have defined
correspond to the sum and product operations on the set of integers, and for that reason,
are occasionally referred to as the logical sum and logical product, respectively. If one should
wonder whether there are operations corresponding to the arithmetic inverses, namely differences and quotients, this is a rather more delicate question, especially in the second case.
Because it is so, we shall postpone any mention of the latter, other than to say that we shall
examine this point sufficiently for our purposes with respect to discrete probability theory
in chapter 4. On the other hand, the analogue of arithmetic differences is neatly expressed
in set theory as the incarnation of logical negation.
Definition 4. Let S and T be sets such that T ⊂ S, then we say the complement of T in S
is the set consisting of elements in S that are not in T, denoted S \ T .
Again, the so-called logical difference defined above can also be described in the ubiquitous elemental notation by S \ T = {x | x ∈ S AND x ∈
/ T }, where we read the symbol ∈
/
as ’not in,’ whence the role of logical negation. These operations for forming new sets from
given ones satisfy properties recognizable as properties that the addition and multiplication
of integers satisfy. In point of fact, consider the following proposition.
Proposition 1. Let S, T, and U, be arbitrary sets, then the following statements are true.
1. Union of sets is commutative, that is S ∪ T = T ∪ S
2. Intersection of sets is commutative, that is S ∩ T = T ∩ S
3. Union is associative, that is (S ∪ T ) ∪ U = S ∪ (T ∪ U )
4. Intersection is associative, that is (S ∩ T ) ∩ U = S ∩ (T ∩ U )
5. Union distributes across intersection, that is S ∪ (T ∩ U ) = (S ∪ T ) ∩ (S ∪ U )
6. Intersection distributes across union, that is S ∩ (T ∪ U ) = (S ∩ T ) ∪ (S ∩ U )
We shall prove statement 6 of the proposition as an illustration of the definition of set
equality. So, let us show first that S ∩ (T ∪ U ) ⊂ (S ∩ T ) ∪ (S ∩ U ). In order to do this, we
must show every element of the former set is an element of the latter set. As such, we have,
for x ∈ S ∩ (T ∪ U ) that both x ∈ S and x ∈ T ∪ U . Consequently, x ∈ T or x ∈ U and so,
x ∈ S ∩ T or x ∈ S ∩ U . Conversely, suppose x ∈ (S ∩ T ) ∪ (S ∩ U ), then either x ∈ S ∩ T
or x ∈ S ∩ U , so x ∈ S and either x ∈ T or x ∈ U , as desired.
One might notice that, under the analogy between sets and integers, statement 5 is rather
unusual, as it would correspond to a distribution law for sums across products. This would
mean that, for integers s, t, and u, that s+t·u = (s+t)·(s+u) which is of course, in general,
utter nonsense; yet, in this new world or category of sets, one can now easily demonstrate
statement 5 as the dual of statement 6. We shall not digress to explain exactly what we
2
mean by dual statement beyond indicating it means that if a statement is true of a union
and intersection of sets, a corresponding statement is true of the intersection and union of
their complements. This powerful way of thinking is used throughout this text and emerges
from the following famous precept of logic.
Proposition 2. Let S and T be subsets of a set Ω, and let S c = Ω \ S. A similar notation
is used for T c , (S ∩ T )c , etc. Then the following statements are true.
1. (S ∩ T )c = S c ∪ T c
2. (S ∪ T )c = S c ∩ T c
Statements 1 and 2 of the proposition are known collectively as DeMorgan’s laws. To
prove statement 2, suppose x ∈ (S ∩ T )c , then x ∈
/ S or x ∈
/ T since x ∈
/ S and x ∈
/ T.
Therefore, x ∈ S c ∪ T c , as desired. The converse is similar. We shall not prove statement 1.
Moreover, applying DeMorgan’s laws to statement 6 implies that statement 5, the so-called
dual, is also true.
The following is a digression on notation that is important and useful. All of the above
operations can be carried out over indexing sets, which are sets whose elements are intended
only to list how other sets are to be substituted into some set theoretic operation. Specifically,
let us consider some indexing set I, but most often, we may take I = N or some finite subset
thereof. Then a sequence of sets indexed by I is written as {Si }i∈I . The manner in which
the various set operations extend to indexed sets is hopefully obvious. For example, we shall
consider
[
Si
i∈I
and say it is the I-fold union of the sets Si . The usefulness of such general notation is
obvious, as often one wishes to appeal to concepts more complex than ones that may only
depend upon the union of two or three sets. The corresponding description of the I-fold
intersection of sets is the obvious one.
Before proceeding to the final concept in this section, let us consider what the additive
identity for the logical sum would be according to the analogy we have sketched between set
theoretic operations and numerical ones. In other words, what set object would correspond
to the number zero? It seems plain that the answer would be the complement of an arbitrary
set in itself, as zero is defined as the numerical element satisfying the equation n − n = 0 for
an arbitrary number n ∈ N. This discussion motivates the following definition.
Definition 5. Let S be a set. Then we say S \ S is the null or empty set, denoted ∅
Indeed, one should notice that the relation S ∪ ∅ = S holds for sets as the relation
n + 0 = n holds for integers. Similarly, if we regard the intersection of sets as a kind of
product, then the relation S ∩ ∅ = ∅ holds in sets, as the relation n · 0 = 0 also holds in
integers.
The final set theoretic concept in this section should be equivalent to fractions or rational numbers with respect to the loose analogy we have developed between sets and integers.
However, as mentioned above, this point is nettlesome and will be postponed. Nonetheless,
3
the utility of the concept is immediate and independent of its explanation of the correspondence between operations on sets and operations on integers.
The reason for defining the null set before defining the concept of partition below is
because the concept of a partition depends on the concept of disjointedness. Indeed, for any
sets S and T , we say they are disjoint if their intersection is the null set. This will occur
when the overlap of two events in an experiment is logically absurd, and in that context,
we shall refer to mutually exclusive events. No matter what, disjoint sets are fundamental
throughout this course, as many ideas that follow are statements under the hypothesis, either
explicitly or implicitly, that we can use disjoint sets to partition a problem into cells that
are easier to examine than the initial set we are given on the whole.
Definition 6. Let {Ai }i∈I be a sequence of subsets of some set S indexed by I. Assume that
elements of this sequence are pair-wise disjoint, that is,
Ai ∩ Aj = ∅
for all i, j ∈ I such that i ̸= j. Then we say the sequence {Ai }i∈I is a partition of S and
that the Ai are its cells if
[
Ai = S
i∈I
Moreover, we say the partition is of length r if |I| = r.
1.1 WORKED EXAMPLES
1. 1.1 Let R be the set of all rectangles with integer-valued length and width. Consider
R(l, w) ⊂ R the subset of rectangles with either even integer-valued length l or even
integer-valued width w. Furthermore, consider R(A) ⊂ R the subset of rectangles
whose area is an even integer.
(a) Prove that R(l, w) = R(A)
We proceed by the definition of the equality of sets in this section. That is, S = T if
and only if both S ⊂ T and T ⊂ S. Observe,
Solution (a) Let R ∈ R(l, w), then Area(R) = l · w. Since either l or w is an even
integer by hypothesis, so is l · w. Therefore, R ∈ R(A). As R was arbitrary, we have
R(l, w) ⊂ R(A). Conversely, let R′ ∈ R(A). Write Area(R′ ) = 2m, then 2m = l · w.
Since 2 is prime, 2 divides either l or w. Therefore, either l or w is even. Thus, as R′
was arbitrary, we have R(A) ⊂ R(l, w). Therefore, by the definition of set equality,
R(l, w) = R(A).
2. 1.1 Simplify the complements of the following sets by DeMorgan’s laws.
(a) ({x ≥ 5} ∪ {x ≤ 4})c
4
(b) Let S, T, U be sets, then simplify (S ∩ T ∪ U )c
We proceed by DeMorgan’s laws, which indicate how to compose the complement of
sets with their unions and intersections.
Solution (a) We have ({x ≥ 5} ∪ {x ≤ 4})c = {x ≥ 5}c ∩ {x ≤ 4}c by DeMorgan’s
laws. We compute the complement of these intervals by definition, so that
{x ≥ 5}c ∩ {x ≤ 4}c = {x < 5} ∩ {x > 4} = (4, 5).
Solution (b) Proceed by both the distribution law and DeMorgans. Observe,
S ∩ T ∪ U = S ∩ T ∪ S ∩ U so that
(S ∩ T ∪ U )c = (S ∩ T )c ∩ (S ∩ U )c = S c ∪ T c ∩ S c ∪ U c = S c ∪ T c ∩ U c
3. 1.1 Consider a group of four individuals named Alan, Baker, Carmen and Duran. Let
S be the set of arrangements of these individuals lined-up in order from first to last.
For example, ABCD would represent the order Alan is first in line, Baker is second
in line, etc. Partition S in a manner that depends upon the position of the individuals.
(a) Partition S in a manner that depends upon the position of the individuals.
We proceed by definition of a partition of a set S to define cells that depends upon the
position of the individuals.
Solution (a) There are several options available to us according to which position
we choose. We shall choose the first position for ease. Accordingly, we may partition
S into cells consisting of which of the four individuals are first in the line-up. So
define A1 = subset of line-ups with Alan first, . . . , A4 =subset of line-ups with Duran
first. Hence S = ∪4i=1 Ai since each line-up must be led by an individual and
Ai ∩ Aj = ∅ since different individual in first position determine different line-ups.
1.2
Set Functions
In this section we define functions between sets, which should be familiar to the reader
as a generalization of real-valued functions introduced and studied carefully in any standard
calculus track. Regarding Rn , for n ≥ 0, as sets, the functions one is familiar with in calculus
f :Rn → R
(x1 , x2 , . . . , xn ) 7→ y = f (x1 , x2 , . . . , xn )
are examples of the following definition. More generally, the following definition is also a
particular example of morphisms between objects in a category. We shall pursue this naive
categorical language whenever there is occasion for its reference throughout these notes. An
analogy under the auspices of this naive language of the following definition is that of linear
transformation with respect to a pair a vector spaces, say T : V → W . Accordingly, set
functions are the morphisms in the category of sets.
5
Definition 7. Let S and T be sets. Then we say a well-defined assignment of elements in S
to elements in T is a function, denoted
f :S → T
s 7→ t = f (s)
Well-defined in the above definition is intended in the same sense as one intends its
use in the aforementioned calculus track. That is, well-defined means that the function f
assigns elements in S uniquely to elements in T . One must be careful as nowadays the
word "unique" is abusively used to mean "rare," whereas the mathematician still intends
its lexical definition. Moreover, well-defined does not preclude that f may assign several
distinct elements of S to the same element of T , for such an assignment is still a unique one
with respect to elements of S, which is all the definition requires.
As usual, there is a notion of graph attached to a set function that we shall use to relate
experiments to planar regions later on. To describe this, we need another concept from set
theory, that of the Cartesian product of sets.
Definition 8. Given sets S and T , one writes S × T = {(s, t) | s ∈ S, t ∈ T } for the set of
ordered pairs of elements in S and T . We say that S × T is the Cartesian product of S and
T.
Categorically speaking, just as set functions are in morphisms, Cartesian products are a
special case of the limit of sets. We shall not investigate categorical generality, other than to
mention it occasionally in passing as a convenient language or to point out structural aspects
of our subject until explaining it a little more precisely in chapter 5. Returning to our remark
above, given a set function f : S → T one defines Γf ⊂ S × T where Γf = {(s, t)| t = f (s)},
to be graph of f . One can rephrase the defining condition of a set function f in terms of
the graph Γf by the assertion that there exists a unique t ∈ T such that (s, t) ∈ Γf for any
s ∈ S.
Beside the graph of a set function, all of the remaining features of real-valued functions
may be expressed in the generality of the category of sets. A helpful remark is that injectivity
generalizes the familiar concept of a 1-1 function and that surjectivity generalizes an onto
function in the context of real-valued functions. The following multiple part definition recalls
these features.
Definition 9. Let f : S → T be a function between sets S and T. Then we say
1. S is the domain of f and T is the codomain or range of f
2. For A ⊂ S, one says that f (A) = { t | t = f (s) for some s ∈ A} ⊂ T is the image of A
under f .
3. For B ⊂ T , one says that f −1 (B) = { s | f (s) ∈ B } ⊂ S is the pre-image of B under
f.
4. f is injective if s1 ̸= s2 implies f (s1 ) ̸= f (s2 ) for all si ∈ S
5. f is surjective if T = f (S)
6
6. f is a bijection if it is both injective and surjective, and we say S and T are in bijective
correspondence, denoted S ∼
=T
What follows next are some structural features set functions impart to their domain
and codomain. This treatment is foundational in our subject. For example, later we shall
define a partition of a σ-algebra with respect to the equivalence classes induced by the preimages of a random variable defined on a discrete probability space in order to define the
probability of the event "a function equals a number." We continue on with the development
of this definition to demonstrate later that so-called stochastic processes satisfy Kolmogorov’s
axioms, the axioms we invoke below to define the probability of an event in an experiment
rigorously. Such exotic notions require this degree of abstraction to manage.
Definition 10. Let f : S → T be a function between sets S and T . Then we say the
pre-image of t ∈ T under f is the fibre of f over t.
The reason to isolate this special case of the definition of pre-image as an important
structural feature of sets is because it emphasizes an important conceptual perspective on
what it is functions are. Under the auspices of fibres, one may regard the data of a function
of sets f as a tool to partition the domain of a function into fibres over points indexed by
its image. Indeed, the very definition of a function as that of a well-defined rule mandates
this concept, for the fibres over points t ∈ f (S) partition S by its definition. That is,
[
S∼
f −1 (t)
=
t∈f (S)
Different functions with the same domain and codomain induce in general different partitions of the domain by their fibres. As such, one may conceptualize functions as a sort of
organizing principle insofar as a given function arranges the elements of the domain into its
fibres. As stated already, we introduce this perspective at this earlier stage in the text to
anticipate its usage in chapter 3 in the presence of random variables.
Let us next discuss the composition of functions in the generality of sets. Our exposition
upon the composition of set functions is the obvious generalization of the composition of
real-valued functions.
Definition 11. Let f : S → T and g : T → U be a pair of set functions, then the function
g ◦ f :S → U
s 7→ u = g ◦ f (s) = g(f (s))
is said to be their composition. Diagrammatically, we have
f
S
T
g
g◦f
U
7
We can understand the commutativity of this diagram in terms of fiberings of the domain,
S. Indeed, for every u1 , u2 ∈ U such that (g◦f )−1 (u1 ) ̸= (g◦f )−1 (u2 ) we have f −1 (g −1 (u1 )) ̸=
f −1 (g −1 (u2 )), for the fibering of S with respect to the function g ◦ f : S → U uniquely
determines the fibering of T induced by g and then the fibering of the S induced by the
pre-images of the fibres of g under f . Accordingly, we read the diagram as asserting that the
function g ◦ f is the same as the function f followed by g in the sense they induce the same
fibering of the domain, S. Next we enumerate how composition preserves either injectivity
and surjectivity under various configurations of these properties.
Proposition 3. Let f : S → T and g : T → U be set functions. Then the following
statements are true.
1. If both f and g are surjective, then so is their composition g ◦ f
2. If both f and g are injective, then so is their composition g ◦ f
3. If both f and g are bijective, then so is their composition g ◦ f
4. If f is surjective and g ◦ f is surjective as well, then so is g
5. If g is injective and g ◦ f is injective as well, then so is f
One can see that item 2 follows from the fact that both f −1 and g −1 are well-defined.
Accordingly, their fibres are singletons, and so, their composition is an injection. Item 4
is required later to show that the image of a function is in bijective correspondence with
its quotient set with respect to a specific relation on the domain induced by the fibration
with respect to a function described above. Thus, to prove this, one undertakes a so-called
diagram chase using the diagram illustrated in the definition of composition and applying
the hypotheses to show the fibre over a point in U is non-empty for any point u ∈ U .
We continue this section with general properties that describe the compatibilities of the
images and pre-images of a set function f with respect to set operations.
Proposition 4. Let f : S → T be a set function, and C, C1 , C2 ⊂ S, D, D1 , D2 ⊂ T . Then
the following statements are true.
1. C ⊂ f −1 ◦ f (C)
2. f ◦ f −1 (D) ⊂ D
3. f (C1 ∩ C2 ) ⊂ f (C1 ) ∩ f (C2 )
4. f (C1 ∪ C2 ) = f (C1 ) ∪ f (C2 )
5. f −1 (D1 ∩ D2 ) = f −1 (D1 ) ∩ f −1 (D2 )
8
6. f −1 (D1 ∪ D2 ) = f −1 (D1 ) ∪ f −1 (D2 )
7. f −1 (T \ D) = S \ f −1 (D)
Furthermore, suppose that f is an injection, then 1. is an equality, f a surjection, then 2.
is an equality, and if f is an injection, then 3. is an equality.
We shall not prove each of these statements, but rather prove only item 3 in order to once
more exhibit the notion of set equality in an example. To this end, let t ∈ f (C1 ∩ C2 ), then
by the definition of image there exists s ∈ C1 ∩C2 such that f (s) = t for t, as above. As such,
s ∈ C1 and s ∈ C2 , but then f (s) ∈ f (C1 ) and f (s) ∈ f (C2 ), Thus, f (s) = t ∈ f (C1 )∩f (C2 ).
As t was arbitrary, this argument demonstrates the containment.
To complete the illustration of set equality we intend, let us now suppose that f is
injective. Given this, suppose t ∈ f (C1 ) ∩ f (C2 ). Then there exists s1 ∈ C1 and s2 ∈ C2 such
that both f (s1 ) = t and f (s2 ) = t. Since f is assumed to be injective, we have by definition
that as f (s1 ) = f (s2 ), therefore s1 = s2 . Hence s1 ∈ C1 ∩ C2 and f (s1 ) = t ∈ f (C1 ∩ C2 ),
which gives the reverse containment, since t was arbitrary. Therefore, when f is injective, 3
is an equality, as desired.
Composition allows us to describe both injectivity and surjectivity in terms of formulas.
To do this, we require the concept of a function’s inverse, which, again, is the straightforward generalization of that concept in the context of real-valued functions. We define a set
function’s inverse formally as the well-defined assignment of elements from its codomain to
its domain induced by the definition of pre-image applied to the codomain. This definition
relies upon the following two rules obtained from f : S → T . The first is that of the left inverse of f , which is the function fL−1 : T → S such that fL−1 ◦ f = IdS , where IdS : S → S is
the identity function of S. The second one defines is the right inverse of f mutatis mutandi.
These concepts combine to proffer the following definition of the inverse of a set function in
terms of composition.
Definition 12. Let f : S → T be a function. Then we say f −1 is the inverse of f if f −1 is
both the left and right inverse of f .
The following proposition then expresses the fundamental properties of functions in terms
of the existences of left and right inverses.
Proposition 5. Let f : S → T be a function. Then the following statements are true.
1. f is injective if and only if there exists a left inverse fL−1
2. f is surjective if and only if there exists a right inverse fR−1
3. f is bijective if and only if there exists an inverse f −1
The first item follows from the fact that, if f is injective, its fibres are all singletons, thus,
its pre-image determines a well-defined assignment from elements in its image to elements in
its pre-image. This defines fL−1 given that f is an injection together with the stipulation that
all set functions map the empty set to the empty set. Conversely, assuming a left inverse
9
fL−1 exists, suppose that for s1 , 22 ∈ S we have f (s1 ) = f (s2 ). Applying fL−1 to this element
in its domain we have, by definitino of the same, that fL−1 ◦ f (s1 ) = fL−1 ◦ f (s1 ) which implies
s1 = s2 since f composes with its left inverse to the identity on the domain, S, by definition.
This finishes our argument. A dual argument proves item 2 in conjunction with the axiom
of choice, so we omit it from the text. Therefore, item 3 follows from the compatibility of
composition and injectivity and surjectivity, for f −1 is a bijection if it is both a left and a
right inverse by items 1 and 2, above.
One reason in the author’s view for all of this formalism is to define several key concepts
in this book, least among them being the adjective of its title, "discrete". In order to
mathematically define this adjective in the present context of set functions, we will explain
how the cardinality of various sets are related to each other, for the adjective is one that
indicates a particular range of cardinalities for a set.
Of course, in the most elementary approach, one could state a relation between |S| to |T |
by simply counting their elements and then comparing the numbers. However, this assumes
there are finitely many elements in both sets, and furthermore, even finite, not too many to
practically count. Accordingly, this naive approach is insufficient for our purposes even in
this modest regard, doubly insofar as we could use such an approach for defining ”discrete.”
The following more sophisticated approach will explain what we mean by the word "discrete"
and in general clarify what we mean when we attempt to relate the cardinalities of various
sets. The approach relies upon an important notion of a "near equivalence" of non-identical
sets, which even distinguishes a set in terms of how its elements are arranged in order, that
of a bijection defined above. So to the problem of comparing the sizes of arbitrary sets can
be settled by the following proposition.
Proposition 6. Suppose S and T are sets and f : S → T is a set function, then |S| = |T |
if, and only if f is a bijection.
This more sophisticated approach extends to arbitrary cardinalities and does not rely
upon the act of counting elements of sets in order to determine its number of elements. We
rely upon this proposition to obtain the definition of "discrete" we shall use throughout this
text. Often one finds this proposition being used in proofs by constructing an inverse to
a given function in order to establish the equality of two cardinalities. This strategy is a
consequence of the previous proposition.
Definition 13. Let S be a set. We say S is countable if S there exists a bijection c : S →
T ⊂ Z onto a not-necessarily proper subset of the set of integers. Accordingly, there is a
distinction between sets in bijective correspondence between proper and improper subsets of
Z. In the former case, we say S is finite. In the latter case, we say S is countably infinite.
Below we shall define discrete probability spaces in terms of this definition. Specifically,
they are probability spaces whose sample spaces are countable. In the present section,
however, this definition may require some context as it is itself rather unusual, for it suggestsfamously-the existence of orders of infinity. That is to say, implicitly, it is conceivable there
are so-called uncountable infinities, and indeed there are. The following theorem is due to
Cantor and emphasizes this point.
Theorem 1. There is no bijection between Z and R.
10
In terms of our definition, one could say, following our italics, that the set of real numbers
R is uncountable, and indeed, this would be correct. The proof of the above theorem is
usually proven by Cantor’s famous diagonal argument which illustrates there is no bijection
between Q and R. In terms of the theorem, for Cantor’s argument to be equivalent, it would
follow that |Z| = |Q|, which is surprisingly true!
Proposition 7. Let Q be the set of rational numbers. Then |Z| = |Q|, that is, Q is a
countable set.
The proof of this proposition amounts to demonstrating the existence of a bijection
between the set of integers and the set of rational numbers. Ideally proving such a seemingly
counter-intuitive fact inspires the reader’s belief in the necessity of our sophisticated approach
to reckoning the cardinalities of sets. So, let us construct such a bijection. First, define Q±
to be the sets of positive, respectively negative, rational numbers. Define c+ : Q+ → Z
m
by c+ ( ) = 2m 3n . The fundamental theorem of arithmetic implies that c is injective, so
n
that the image c+ (Q+ ) ⊂ Z is a well-ordered subset. Appealing to the well-ordering on the
image of c+ , one constructs a second bijection onto N. One recalls that the set of natural
numbers is itself countable, thus demonstrating that Q+ is countable via composition. A
similar argument shows that Q− is countable. As the union of countable sets is countable
and Q = Q+ ∪ Q− ∪ {0}, we have that the set of rational numbers Q is countable, as desired.
The proof of this proposition is not entirely self-contained in this text, as it is only
intended to illustrate the initial breadth of our new notion of size, that is, reckoning size
according to the existence of a bijection. The interested reader is referred to standard
introductory texts on Real Analysis. Let us now consider a few examples of the material in
this section.
1.2 WORKED EXAMPLES
1. 1.2 Let us prove the following statement from this section. Let S and T be sets, and
f : S → T a set function. Show that {f −1 (t)}t∈f (S) is a partition of S.
Solution We must show that the fibres of f are the cells of a partition of S. To this
end, we first establish that, for t ̸= t′ ∈ T , f −1 (t) ∩ f −1 (t′ ) = ∅. Suppose
s ∈ f −1 (t) ∩ f −1 (t′ ), then by definition of fibre, both f (s) = t and f (s) = t′ -absurd, as
f is well-defined, by hypothesis. Thus, f −1 (t) ∩ f −1 (t′ ) = ∅. Secondly,
S = ∪t∈f (S) f −1 (t) since S is the domain of f . Therefore, the fibres of f partition S.
One may interpret this result by thinking of f as dividing S into the of the fibres
of f . Later on, viewing set functions in this manner shall play an important role in
conceptualizing the role of random variables in discrete probability theory.
2. 1.2 Let us now provide an example of the statement in the above section of how
different functions between the same sets impart distinct partitions of their shared
domain. Consider S = Z the set of integers and T = [10] the set of integers from 0
through 10. Consider the following functions.
11
(a) f : Z → [1] where m 7→ r2 (m), that is, f maps the integer m to its remainder
upon division by 2.
(b) g : Z → [2] where m 7→ r3 (m), that is, g maps the integer m to its remainder
upon division by 3.
(c) Demonstrate that f and g induce distinct partitions of Z.
Solution (a) To determine the partition Z induced by f , we merely must consider
the image of f , for the partition corresponding to f by the first worked example is
indexed by the fibres of f . Therefore, the image of f is self evidently {0, 1} ⊂ T , for
every integer has either remainder 0 or 1 when divided by 2. Therefore, Z = 0 ∪ 1,
where 0 = f −1 (0), 1 = f −1 (1). Notice that the partition induced by f is in bijective
correspondence with the set of integers {0, 1}. Readers familiar with binary notation
should recognize this construction.
Solution (b) Similarly, to determine the partition Z induced by g, we merely must
consider the image of g, for the partition corresponding to g by the first worked
example is indexed by the fibres of g. Therefore, the image of g is self evidently
{0, 1, 2} ⊂ T , for every integer has either remainder 0, 1, or 2 when divided by 3.
Therefore, Z = 0 ∪ 1 ∪ 2, where 0 = f −1 (0), 1 = f −1 (1) and 2 = f −1 (2). Notice that
the partition induced by g is in bijective correspondence with the set of integers
{0, 1, 2}.
Solution (c) To conclude the example, we note that there is no bijection between
the sets [1] and [2] for their cardinalities are different. As such, f and g impart
different partitions to their domain.
3. 1.2 In this example, let us work out the composition of functions together with the
partition of its domain.
(a) Let S = {(i, j)}1≤i,j≤6 be the set of pairs of integers from 1 through 6 and
T = {i + j}1≤i,j≤6 be the set of sums of integers from 1 through 6. Furthermore,
let f : S → T be the obvious addition function.
(b) Let T be as above and U = [4] be the set of integers from 0 through 4. Let
g : T → U be the function that maps i + j to it remainder upon division by 5.
Below, let us first compute the composition of functions and second the partition of
the domain of the composition.
Solution (a) The composition of f and g can be obtained by describing its value in
terms of an element of its domain. Specifically, g ◦ f ((i, j)) = r5 (i + j), that is, the
remainder after division of 5 into i + j for 1 ≤ i, j ≤ 6. Accordingly, g ◦ f : S → U .
12
Solution (b) To compute the fibres of g ◦ f , we proceed in two steps. First, we
compute the fibres of g and f . Second, we compute the fibre of g ◦ f by recognizing it
as a union of the fibres f over the fibres of g. Observe, g −1 (0) = {5, 10} and
f −1 (5) = {(1, 4), (2, 3), (3, 2), (4, 1)}, f −1 (10) = {(4, 6), (5, 5), (6, 4)}. Therefore,
g ◦ f −1 (0) = {(1, 4), (2, 3), (3, 2), (4, 1), (4, 6), (5, 5), (6, 4)} or f −1 (5) ∪ f −1 (10). The
fibres over other points of the image are computed in a similarly straightforward
manner.
1.3
Discrete Probability Theory
In this section we define the first of the three fundamental concepts in this text-the other
two being that of a discrete random variable and that of a Markov chain. The fundamental
object that we define here is the discrete probability space. It is a triple consisting of the data
(Ω, β, P ), described below, subject to certain axioms. It is intended to model the events of a
real-world experiment and the likelihoods of the same. Instantiating experiments as mathematical objects allows experimenters to bring to bear the clarity of rigorous mathematics
upon the object of their study.
In order to obtain this object from a real-world experiment, we must first review some
basic logic. In this text, a real-world experiment is defined to be the set of logically consistent
statements about an actual or idealized experiment. We shall encode these logical statements
in set theory by relating them to a set of their predicates. It is under the auspices of such an
encoding that one obtains our mathematical model of the events themselves of a real-world
experiments. The the likelihood of events is treated subsequently by considering the images
of the sets of predicates under a specific set-function required to satisfy certain axioms that
are intended to reflect our shared notion of what probability is.
Let O be the collection of all atomic sentences describing events in a real-world experiment. Recall that an atomic sentence is a logically true or false declarative statement that
is irreducible with respect to the disjunction or conjunction of other such statements. The
logical consistency of these sentences of course means no contradictions are allowed. Accordingly, O consists of atomic sentences a and b but does not contain their conjunctions and
disjunctions, say a ∧ b, a ∨ b, but does contain a, read as a AND b, a OR b, and NOT a,
respectively. The truth values of the connected atomic sentences and the negative or "not"
statement are given by the following truth tables, with respect to the above order.


AND a ∧ b

T T T



F
F
T
1. 



T F F
F F F

OR





a
T
F
T
F
13
∨
T
T
T
F

b
T

T

F
F
and a is F if and only if a is T. DeMorgan’s laws can be expressed in this notation by
a ∧ b = a ∨ b and a ∨ b = a ∧ b. Notice that negation is idempotent, or a = a. To gloss the
experimental language this digression is intended for, we shall say an atomic sentence is a
simple statement.
We relate the collection O of simple statements to a set, say Ω, to begin translating the
logical description of a real-world experiment into mathematics. In general, we say a set
A verifies a simple statement a if there exists u ∈ A such that u(a) is a predicate of a.
Recall that a predicate is a symbol designating a property or relation of the sentence a. As
such, we assign the simple statements describing a real-world experiment uniquely to a set
of predicates, called simple events, in our notation, this assignment is a 7→ u. We formalize
this encoding in the following definition.
Definition 14. Let O be all the simple statements about a real-world experiment germane
to an experimenter. Then we say a set of their unique predicates, denoted Ω, is the sample
space of an experiment. We say such predicate symbols u ∈ Ω are simple events. More
generally, we say any subset A ⊂ Ω is an event.
The manner in which we express the likelihood or equivalently the probability of a simple
event u ∈ Ω will be as the image of u under a set function P into the unit interval [0, 1].
Indeed, we intuitively understand the probability of an event to be a real number between
0 and 1 when thought of as a percentage. Hence, in terms of simple events u ∈ Ω we model
this intuition rigorously by declaring its probability to be its value under a set function
P : Ω → [0, 1], where [0, 1] is the unit interval in the set of real numbers R. Namely, for
u ∈ Ω, one has 0 ≤ P (u) ≤ 1 or P (u) ∈ [0, 1], imparting rigor to our intuition that the
likelihood of a statement about an experiment is some value between 0 %-likely and 100
%-likely.
Although creative, this definition is not general enough, for it does not include arbitrary
events. Strictly speaking, as a domain for P , we can only evaluate P on the simple events
in the above formulation. To obviate this problem, we introduce the general concept of a
σ-algebra and that of a power set as our only operative example. In this way, one can model
the likelihood of any statement about a real-world experiment as an event.
Observe that the extension of the assignment of simple statements to simple events to
conjunctions, disjunctions, and negations, assigns to these connectives the events corresponding to the intersection, union, and complement of the subsets of Ω verifying the same. In
notation, the extension of the assignment of simple statements to simple events is
a ∧ b 7→ A ∩ B
a ∨ b 7→ A ∪ B
a 7→ Ac
where A, B ∈ β, where β shall be our notation for the set of all events in Ω, verifying a
and b. Although the assignment extends to arbitrary events, it is important to emphasize
that, by definition, Ω does not contain these. Now we will turn our attention to expressing
the likelihood of these and arbitrary events in an experiment by determining an appropriate
domain for our putative P .
Definition 15. Let S be a set. Then we say a set of subsets β of S such that the following
closure axioms are satisfied is a σ-algebra.
14
1. β is closed under complements, that is, if A ∈ β, then Ac ∈ β.
2. β is closed under the empty set, that is ∅ ∈ β
3. β is closed under arbitrary unions, that is, if {Ai }i∈I ∈ β, then ∪i∈I Ai ∈ β
Notice that DeMorgan’s laws imply that, for any σ-algebra β, we also have S ∈ β for
S c = ∅ as β is closed under complements. Similarly, β is closed under arbitrary intersections,
for if ∪i∈I Ai ∈ β then so is (∪i∈I Aci )c = ∩i∈I Ai , by DeMorgan’s laws. Taken together, for
reasons beyond the scope of this book, β has the structure of a mathematical object called
an algebra, whence its name.
Setting up for the appropriate domain for P above, we mention the definition of a power
set.
Definition 16. Let S be a set. Then we define the power set of S to be the set of all subsets
of S, denoted 2S .
So far in our discussion of probability theory we have not involved a hypothesis that
distinguishes our subject-that of discrete probability theory. We finally involve such a distinguishing feature according to the following proposition.
Proposition 8. Let S be a countable set, then its power set 2S is a σ-algebra
The proof is trivial by definition of power set. Hereafter, we shall only consider real-world
experiments such that the corresponding sample space Ω is countable in the sense of the
previous section. At first such a hypothesis might seem unnecessary, however, a background
in elementary statistics should convince the reader there are commonplace scenarios in which
this hypothesis is absent. In any case, we have the following proposition which will rightly
characterize the power set of a countable set as the domain for P .
Observe that replacing the sample space Ω in our naive definition of P above by its
power set 2Ω = β obviates the problem it had encountered, for one may now evaluate P on
an arbitrary event. To wit
P : β → [0, 1]
is defined for all events A ∈ β corresponding to logical not-necessarily simple statements
about a real-world experiment. The structure of the σ-algebra upon the power set of the
sample space guarantees this observation.
We are in a position to rigorously define the mathematical models of real-world experiments. We remark that the definition that follows specifies a class of set functions P with
respect to set operations, as discussed in section 2. So, for example, we shall require that the
value of P (A ∪ B) which is equal to P (A) ∪ P (B), in general, to be equal to P (A) + P (B)
when A ∩ B = ∅. We require this restriction on the class of set functions whose domain is
β and codomain [0, 1] for, as we have already seen, arbitrary set functions would not distinguish between the images of unions of events and the images of disjoint unions. Hence, we
emphasize that the union of the images of events under P is not, in general, computed as
the sum of their individual values. This accords with our intuition that in an experiment the
likelihood of one event or another that share no simple events or outcomes, speaking more
15
experimentally, should be, as the size of an event in an experiment, the sum of their sizes,
for there is no overlap. More generally, when events do share outcomes, we recognize how
experimentally this must be accounted for when computing the probability of their union.
We address this below as an application of the principle of inclusion-exclusion.
Definition 17. Let Ω be the sample space of a real-world experiment and β = 2Ω its power
set together with its σ-algebra structure, then we say any set function P : β → [0, 1] such
that following axioms are satisfied is a probability function.
1. P (Ω) = 1
2. P (A) ≥ 0 for any A ∈ β
3. Let {Ai }i∈I be a sequence of events in Ω that are pair-wise disjoint, i.e. Ai ∩ Aj = ∅
for i ̸= j, then
P(
[
Ai ) =
i∈I
X
P (Ai )
i∈I
The three axioms in this definition are quite famous and should be committed to memory.
They are known as Kolmogorov’s axioms and we shall have many occasions throughout this
text to appeal to them. These axioms allow us to define the first of the three fundamental
objects in this text. This object summarizes the above discussion for the remainder of the
text as it is our mathematical object that models a real-world experiment.
Definition 18. Let Ω be a countable sample space of a real-world experiment, then we say the
triple (Ω, β, P ) where β = 2Ω is the canonical σ-algebra associated to Ω, and P is a probability
function satisfying Kolmogorov’s axioms, is a discrete probability space. Moreover, we say
the probability of an event A ∈ β is its value under P .
So, with this definition at hand, one defines discrete probability theory as the aspect
of probability theory germane to discrete probability spaces. Specifically, to studying the
likelihood of statements about experiments with but countably many simple sentences describing them. We conclude with a canonical definition for P itself, our axiomatization of
the likelihood of a statement or equivalently the probability of an event. Throughout the
text this definition is assumed unless explicitly stated otherwise.
Theorem 2. Let (Ω, β, P ) be a discrete probability sample space such that |Ω| = n is finite.
In terms of its simple events, write Ω = {u
P1n, u2 , . . . , un }. Given real numbers pi ∈ [0, 1], for
1 ≤ i ≤ n, such that both pi = P (ui ) and i=1 pi = 1. If one defines
X
P (A) =
pi
i | ui ∈A
for an arbitrary event A ∈ β, then P is a probability function.
16
The proof of this theorem is simply a matter of verifying that the definition of P satisfies
Kolmogorov’s axioms givenP
the hypotheses. So, for A ∈ β, it is obvious that P (A) ≥ 0 since
pi ≥ 0 for each ui ∈ A. As ni=1 pi = 1 and Ω = ∪ni=1 ui it follows that P (Ω) = 1 by the last
axiom, as ui ∩ uj = ∅. To demonstrate the last axiom holds, for {A1 , A2 , . . . , Ak } ∈ β such
that the Ai are pair-wise disjoint, we have


!
k
[
X
P
Ai =P 
pj 
i=1
j|uj ∈∪ki=1 Ai
=
k
X

X

i=1
=

k
X
pj 
j|uj ∈Ai
P (Ai )
i=1
where the middle equality is obtained by the hypothesis that the Ai are pair-wise disjoint.
We conclude by emphasizing the corollary of the theorem for countably infinite Ω is true
if one is replaces the n−tuple {pi }ni=1 inP
the theorem by a countable sequence of real numbers
{pi }∞
such
that
corresponding
series
i=1
i pi converges to 1. We require this generality later
when we discuss parametric families of discrete random variables in chapter 4. As this
formula works in either the finite or countably infinite cases, we shall treat the formula for
P given by this theorem as the canonical probability function in this text. All examples
discrete probability spaces in the remainder of the text rely upon it for their formulation.
1.3 WORKED EXAMPLES
1. 1.3 Let us consider an example of an abstract discrete probability space, which is to
say a discrete probability space whose events have no experimental significance. To
wit, let Ω = {u1 , u2 , u3 } be the set of simple events so that β = {∅, u1 , u2 , u3 , A =
u1 ∪ u2 , B = u1 ∪ u3 , C = u2 ∪ u3 , Ω}. The values p1 = .25, p2 = .5, and p3 = .25
satisfy Kolmogorov’s axioms when assigned to u1 , u2 and u3 under P , respectively. In
this manner, P (A = u1 ∪ u2 ) = P (u1 ) + P (u2 ) = .25 + .5 = .75, by the definition of
the probability function the theorem indicates. Perhaps another significance of this
example is that the probability of events can be furnished by an assignment consistent
with Kolmogorov’s axioms-there does not need to be any experimental justification for
the assignment, as one might intuitively expect.
2. 1.3 Consider the experiment of tossing a fair coin and recording the outcome. Notice
that the sample space of this experiment consists of the two simple events Ω = {H, T }.
(a) Determine the sample space corresponding to this experiment.
(b) Determine P (H).
(a) Solution (a) The sample space is the set of predicates of the atomic sentences
describing this experiment. One agrees a convenient set of predicates would be
Ω = {H, T }.
17
(b) Solution (b) Observe that there exists a probability function on Ω by the
theorem at the end of this section. However, rather than choose arbitrary pi
satisfying its hypotheses, we should choose pi and in particular, say P (H) = p1 ,
that reflect the hypothesis the coin being flipped is fair. Such a choice reflects
the soundness of our choice, which experimenters should expect in a model of an
experiment. As such, we can reflect the fairness hypothesis by the equation
P (H) = P (T ). Then, as Kolmogorov’s axioms are satisfied, we have both
P (Ω) = 1 and Ω = T ∪ H, so that 1 = P (H ∪ T ) = P (H) + P (T ). Therefore
1
1 = 2P (H) or P (H) = .
2
3. 1.3 Consider the experiment of tossing a fair coin three times and recording the number
of heads.
(a) Describe the sample space associated to this experiment.
(b) Assign probabilities to the simple events identified in (a) that are consistent
with Kolmogorov’s axioms and experimentally sound.
(c) Compute the probabilities of the events A = {1, 2, 3} and B = {0, 3}.
(a) Solution (a) Proceed by the definition of the sample space by listing the simple
events associated with this real-world experiment. Since we are recording the
number of heads which might appear on the faces of three tossed coins, one
agrees a convenient set of predicates would be the number of heads. Thus, with
obvious notation, we have Ω = {0, 1, 2, 3} is the sample space.
(b) Solution (b) As there are four simple events, we may choose any pi satisfying
the condition that their sum is 1, for i = 1, 2, 3, 4. However, we must select
experimentally sound values for these pi . Since the experiment is described
recording the results of fair coins, then for P (i − 1) = pi , we have
3
3
1
1
p1 = , p2 = , p3 = and p4 = .
8
8
8
8
(c) Solution (c) As simple events are mutually disjoint, we have by then theorem
P (A = 1 ∪ 2 ∪ 3) = P (1) + P (2) + P (3) = p2 + p3 + p4 and, similarly,
P (B = 0 ∪ 3) = P (0) + P (3) = p1 + p4 .
4. 1.3 Consider the experiment in which three horses, labeled 1,2 and 3, race against each
other once and the winner is recorded.
(a) Assume 1 is twice as likely to win as 2 and 2 is twice as likely to win as 3.
Compute the probabilities of each horse winning the race.
(b) Compute the probability that either horse 1 or horse 2 wins the race.
(a) Solution (a) Proceed by the definition of the sample space by listing the simple
events associated with this real-world experiment. Since we are recording which
18
of the three horses wins the race, we can write Ω = {1, 2, 3} as the sample space
for the experiment, with obvious notation. Let pi be the probability horse i
wins. Then by hypothesis p1 = 2p2 and p2 = 3p3 . As
P (Ω) = P (1) + P (2) + P (3) = p1 + p2 + p3 = 6p3 + 3p3 + p3 = 10p3 = 1, so
3
6
1
p3 = , p2 = , and p1 = .
10
10
10
(b) Solution (b) We must compute the probability of the event 1 ∪ 2. As the
elements of this union are disjoint, we have, by Kolmogorov’s axioms that
9
P (1 ∪ 2) = P (1) + P (2) = .
10
5. 1.3 Consider an n component system whose i-th component is labeled xi for 1 ≤ i ≤ n.
Write x1 x2 · · · xn for a single execution of the system, where components function or
do not function independently of each other, and we denote these states as a 1 or 0
value for the xi , respectively. Suppose the system fails if any three components do
not function, otherwise the system runs. Consider the experiment of observing a 4
component system execute and recording whether it runs or fails.
(a) Compute experimentally sound probabilities of the simple events in this
experiment.
(a) Solution (a) Let us write Ω = {R, F } for the simple events “runs" and “fails",
respectively. The question asks us then to compute experimentally sound values
for P (R) and P (F ). Let us observe that, by hypothesis, the system fails if any
three components do not function. As the components are independent of each
other, the order in which they do not function is irrelevant. Therefore, by
counting techniques we shall discuss in chapter 2, one sees immediately there are
4 executions of the system that fail without all components not functioning plus
the execution of the system in which every component does not function. As
such, an experimentally sound assignment of a probability to the event F is
5
since there are 16 total executions of the system. Moreover,
P (F ) =
16
9
P (R) = 1 − P (F ) =
by Kolmogorov’s second axiom.
16
6. Consider the experiment consisting of two men, say m1 and m2 and three women, say
w1 , w2 , and w3 who are competing in a chess tournament and recording the winner.
Suppose that those of the same sex have equal probability of winning but each woman
is twice as likely to win as a man.
(a) Compute the probability of the event that a woman wins the tournament.
(b) Compute the probability of the event m1 ∪ w2 wins the tournament.
(a) Solution (a) Set P (mi ) = p for i = 1, 2 and P (wj ) = q for j = 1, 2, 3. Then by
Kolmogorov’s second axiom we have 2p + 3q = 1 and by hypothesis 8p = 1 since
19
1
1
and q = . Therefore, the probability of the event a woman
8
4
3
wins the tournmanet is P (w1 ∪ w2 ∪ w3 ) = by Kolmogorov’s third axiom.
4
3
(b) Solution (b) Similarly, P (m1 ∪ w2 ) = .
8
q = 2p. Thus p =
7. Let a die be weighted so that the probability of a number appearing when the die
is tossed is proportional to the given number. Let A be the event an even number
appears, B a prime number appears, and C an odd number appears.
(a) Describe the sample space corresponding to this experiment and the probability
function.
(b) Compute P (A), P (B), and P (C).
(c) Compute P (A ∪ B), P (B ∩ C), and P (A ∩ B c ).
(a) Solution (a) The sample space consists of the numbers 1 through 6 indicating
the result of the die after it is thrown. As for the probability function, let
P (1) = p, then P (i) = ip for i = 1, 2, . . . , 6. By Kolmogorov’s second axiom,
P6
6·7
1
= 1 and p = .
i=1 ip = 1 so that p
2
21
10
9
12
(b) Solution (b) By the first solution, P (A) = , P (B) = , and P (C) = .
21
21
21
20
8
10
(c) Solution (c) Similarly, P (A ∪ B) = , P (B ∩ C) = , and P (A ∩ B c ) = .
21
21
21
1.4
The Relative Frequency of an Event and Equiprobable Probability Spaces
In this section we indicate how the canonical definition of the probability function given
in the theorem of 1.3 above can formalize the common-place definition of the likelihood of
an event in an experiment. This intuitive conception of the probability of an event is that it
is the number of times it is observed in an experiment where it might occur among the total
number of times the experiment is performed. More abstractly, if we call this event A then
|A|
. This interpretation of the probability P (A) is often referred to as
its likelihood is the
|Ω|
the relative frequency of the event A. Let us now formalize this perspective by encoding the
assumptions that underlie it in our canonical formula for P .
The relative frequency interpretation of probability actually entails the following hypotheses: first, suppose that the performance of a real-world experiment may be repeated a finite
number of times, say n. Second, suppose these performances may be repeated in exactly the
same manner. Third, suppose that the likelihoods of each outcome in the experiment are
the same. Fourth, and finally, that each performance of the experiment is independent of
the prior performance, in the sense that subsequent performances will satisfy the first three
hypotheses. Then given these hypotheses are satisfied by a sequence of performances of an
20
experiment then the likelihood of one of the outcomes in this experiment is the number of
times this outcome occurs among the repeated performances divided by the total number of
performances.
In terms of a discrete probability space, one begins to translate the relative frequency
interpretation of probability by regarding each performance of the experiment as a simple
event thereof. In this way, using the definition of P in the theorem at the end of the previous
1
section, one computes P (u) = for a performance of the experiment, say u, by Kolmogorov’s
n
second axiom and experimental hypotheses one, two, and four, listed above. Thus, by the
canonical formula for P in section 1.3 we have that
X 1
P (A) =
n
ui | ui ∈A
where A is an arbitrary event in the experiment. Indeed, this recovers the relative frequency
interpretation and expresses hypothesis two, above, for we can simplify this formula by
recognizing that the index of the summation index is given by 1 = 1, . . . , |A|, so we are
adding a constant over the number of times the outcomes occurs. Hence, we have
X1
|A|
=
P (A) =
n
n
|A|
As n = |Ω|, we have recovered the relative frequency interpretation of the probability of A,
|A|
that is, P (A) =
.
|Ω|
We remark that this expression of the probability of an event in an experiment as a ratio
of cardinalities is one motivation for the foray into combinatorics in chapter 2. Indeed, we
have simplified the problem of determining the likelihood of an event to determining the
values |A| and |Ω|. Of course the hypotheses that entail the relative frequency interpretation
of probability do not always hold experimentally, so this observation does not, in general,
solve all questions of likelihood. Nonetheless, we will often require the sophisticated counting
techniques of combinatorics in order to compute probabilities as some exercises below will
indicate.
In summary, the above discussion of the relative frequency interpretation of the probability of an event or outcome in a certain type of experiment restrained by the aforementioned
hypotheses is nothing more than a special case of the theorem of 1.3. However, given its
ubiquity in culture and science, it merits the following definition, obtained by formalizing
the effect its hypotheses induce upon the probability function of the discrete probability
spaces corresponding to such experiments. Namely that the probabilities of simple events is
constant.
Definition 19. Let (Ω, β, P ) be a discrete probability space such that P (u) is constant for
all simple events u ∈ Ω. Then we say it is an Equiprobable Probability Space.
Thus, the formalization as mathematical objects of experiments such the likelihoods of
their events is given by the relative frequency interpretation of probability are equiprobable
spaces. In problems and experiments, one often recognizes the equiprobable hypothesis in a
question according to phrases such as "selected at random" or "a fair coin" or "a fair die"
etc.
21
1.4 WORKED EXAMPLES
1. 1.4 In the following implied experiments, determine by the relative frequency interpretation the probability of each of the following events:
(a) A = A king appears in the drawing of a single card from an ordinary deck of 52
cards
(b) A = At least one tail appears in the toss of three fair coins
(c) A = A white marble appears in drawing a single marble from an urn containing
4 white marbles, 3 red marbles, and 5 blue marbles
Solution (a) A king appears in 4 ways when drawing a single card from an ordinary
4
deck of 52 cards. Therefore, P (A) = .
52
Solution (b) A least one tail appears in the toss of three fair coins, represented by
triples Ω = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }, in exactly 7 simple
7
events. Therefore P (A) = .
8
Solution (c) A white marble appears in drawing a single marble from an urn
containing 4 white marbles, 3 red marbles, and 5 blue marbles in 4 ways. Therefore,
4
P (A) = .
12
2. 1.4 Consider the experiment of drawing two cards at random without replacement from
an ordinary deck of 52 cards and recording the which two cards were drawn from the
deck. Compute the probabilities of the following events:
(a) A = both cards drawn are spades
(b) B = one is a spade and one is a heart
Below in the solutions we may proceed by computing the relative frequency of each of
these events because the drawing of the cards are made randomly.
|A|
. First, let us identify the
|Ω|
simple events in Ω so it is clear that our counting problem is more sophisticated than
the previous exercises. Indeed, the simple events in Ω consist of pairs of cards, chosen
without replacement from the deck, and in no order. Later we shall call such an
arrangement of cards a 2-combination of the deck. As such, |Ω|, is the number of
these, which is 1326. We shall obtain the formula for counting these arrangements in
chapter 2. Similarly, there are 78 ways to draw two spades from the suit of spades in
78
a 52 card deck. Therefore, P (A) =
.
1326
Solution (a) To compute P (A), we must compute
22
|B|
. We observe that |Ω| = 1326
|Ω|
as in the solution (a) and for the same reason. As for |B|, there are 169 ways to draw
a spade and a heart. We shall prove this below in chapter 2 after we learn the
169
.
multiplication rule. Therefore, P (B) =
1326
Solution (b) To compute P (B), we must compute
3. 1.4 Twelve individuals gathered in a room are standing together in pairs. Consider the
following experiments and compute the probabilities of events thereof:
(a) Consider the experiment of choosing two individuals at random from the room
and the event A = the two individuals chosen were paired. Compute P (A).
(b) Consider the experiment of choosing four individuals at random from the room
and the event B = both of the couples contained in the four individuals chosen
were paired
(c) Consider the experiment of choosing four individuals at random from the room
and the event C = exactly one couple contained in the four individuals chosen
were paired
In the following solutions we may proceed by computing the relative frequency of each
of these events because the choices are made randomly.
|A|
. First, we must identify what
|Ω|
the simple events in Ω are. As they consist of pairs of individuals chosen from the
group of 12 individuals, this means they are pairs chosen without replacement and in
no order from this set of people. Therefore, we see once more that such an
arrangement of people is a 2-combination of the group of 12. As such, there are
|Ω| = 66 ways to pair 12 individuals, for this is the number of 2-combinations of the
12. Now as for simple events contained in the set of pairs A, there are only |A| = 6
6
for there are 6 pairs of twelve people. Therefore, P (A) = .
66
Solution (a) To compute P (A), we must compute
|B|
. First, we observe there are
|Ω|
495 ways to choose 4 individuals from 12. We shall prove this later in chapter 2 by
recognizing 495 as the number of 4-combinations of a set of 12 elements. This is
because the simple events in Ω are arrangements of 4 of the 12 individuals, chosen
without replacement, and in no order. Now there are 15 ways to choose 2 pairs from
6 pairs as the pairs chosen from the set of 6 pairs are chosen without replacement and
in no order. Again, we shall show why this is true below, however, for the time being,
we take these results as motivation to learn combinatorics to compute the probability
15
of events in a discrete probability space. Therefore, P (B) =
.
495
Solution (b) To compute P (B), we must compute
23
|C|
. First, |Ω| = 495 for the same
|Ω|
reason as in the previous solution. To compute |C|, we recognize C can be described
as the complement of the event, "either both couples were originally in pairs or
15
by the previous
neither were". The first half of this description has probability
495
240
exercise. The second half of this description has probability
. The reason is
495
because there are 240 ways to choose two couples where no member of the couple was
originally standing together in a pair. This follows from the following argument.
There are 15 ways to choose 4 people from 6 couples, and since we insist they were
not originally standing together in a pair, we must choose one ofthe 2 from each of
15
240
240
the 4 pairs standing together originally. Therefore, P (C) = 1 −
+
=
495 495
495
as the events "both couples were originally in pairs" and "neither were" are disjoint
events. We shall examine this final point more carefully in the following section on
the calculus of discrete probability spaces.
Solution (c) To compute P (C), we must compute
4. Consider the experiment of selecting three light bulbs at random from a box containing
15 light bulbs, 5 of which are defective, and recording how many of the three light bulbs
are defective.
(a) Compute the probability of the event none of the three light bulbs selected are
defective.
(b) Compute the probability of the event exactly one of the three light bulbs
selected are defective.
(c) Compute the probability of the event at least one of the three light bulbs
selected are defective.
(a) Solution (a) The simple events in this experiment are 3-tuples of the 15 bulbs
or arrangements of three, selected without replacement, and in no order. Thus,
as we shall discover in chapter 2, there are 455 such simple events. Now, the
number of simple events favorable to the event described in part a are similar
3-tuples selected from the 10 functioning bulbs. Again, we will explain how to
compute this figure below, but for now it suffices to observe that there are 120
120
= 0.26 or 26
such simple events. Therefore, the probability of this event is
455
%.
(b) Solution (b) Similarly, we must count the number of simple events such that
exactly one bulb is defective and the other two are functioning. There are 45 of
45
these simple events, so the probability of the event in part b is
= 0.09 or 9 %
455
(c) Solution (c) Finally, as we will see in the next section, we can regard the
probability of the event "at least one" as the complement of the event, "none".
120
As such, we have 1 −
= 0.74 or 74 %
455
24
1.5
The Calculus of Discrete Probability Spaces
In this final section of chapter 1 we prove some basic results establishing the calculus
involved in computing the probabilities of arbitrary events. We conclude with the statement
and proof of the law of total probability, a result that looms large over the remainder of
the results in this text. In particular, we require it to prove the famous Baye’s theorem in
chapter 3 and it plays a fundamental role in proving that the transition probabilities between
states in a Markov chain satisfy the fundamental axiom distinguishing Markov processes from
stochastic ones.
Proposition 9. Let (Ω, β, P ) be a discrete probability space and A ∈ β an arbitrary event.
The following statements are true.
1. P (∅) = 0
2. P (A) ≤ 1
3. P (Ac ) = 1 − P (A)
The proofs of these statements are elementary. Notice that Ω = A ∪ Ac for any event
and that since P satisfies Kolmogorov’s axioms, we have
P (Ω) =P (A ∪ Ac )
1 =P (A) + P (Ac )
which gives both statements 3 and 1 immediately-the latter by taking the special case when
A = Ω. As for statement 2, we know that 1 = P (A) + P (Ac ) because 1 is true, so that
P (A) ≤ 1 since P (Ac ) ≥ 0.
Next, we consider the probability of the union of arbitrary events in the following proposition. It should be obvious why such a computation would be of interest, for events whose
probabilities are not a computed by Kolmogorov’s axioms will in general be those that will
occur most often in problems and real-world applications.
Proposition 10. Let (Ω, β, P ) and A, B ∈ β be arbitrary events. The following statements
are true.
1. P (Ac ∩ B) = P (B) − P (A ∩ B)
2. P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
3. Assume A ⊂ B, then P (A) ≤ P (B), that is, P is an inclusion preserving function.
4. Assume A ⊂ B, then P (B \ A) = P (B) − P (A)
The trick to this proof is to recognize that can one decompose an arbitrary event B
relative to A as
B = (B ∩ Ac ) ∪ (B ∩ A)
25
After applying P , this decomposition generalizes the computation of the axioms, for there one
takes A = Ω to compute what one might say is the absolute probability of B in this context.
Specifically, under that substitution, one has B = (B ∩ ∅) ∪ (B ∩ Ω) which tautologically
is B. Hence P (B) = P (B ∩ ∅) ∪ (B ∩ Ω). Thus, in this generality, one might say we are
considering the relative probability of B with respect to A.
Given the relative set theoretic description of an event B with respect to A, the proofs
of the three statements become more or less obvious. Statement 1 is true because B ∩ A and
B ∩ Ac are mutually exclusive events, so 1 follows from Kolmogorov’s third axiom.
Statement 2 is true because A ∪ B = A ∪ (B ∩ Ac ), so that, as the events on the right
hand side are mutally exclusive, we have P (A ∪ B) = P (A) + P (B ∩ Ac ), but then
P (A ∪ B) =P (A) + P (B ∩ Ac )
=P (A) + P (B) − P (A ∩ B)
by substituting statement 1.
To show statement 3 is not difficult, either, given the hypothesis that A ⊂ B. Indeed, it
then follows from statement 1, since P (B ∩ Ac ) = P (B) − P (B ∩ A) ≥ 0 by Kolmogorov’s
axiom 1. Statement 1 then simplifies to P (B) − P (A) ≥ 0 by the hypothesis, for A ∩ B = A
when A ⊂ B. The rest of the argument is obvious.
Last, statement 4 follows directly from the proof of statement 3 since both B \A = B ∩Ac
and B ∩ A = A when A ⊂ B.
A well-known corollary of the proposition is Bonferroni’s Inequality. Indeed, by statement
2, we have P (A ∪ B) ≤ 1 which gives that P (A) + P (B) − P (A ∩ B) ≤ 1 or
P (A ∩ B) ≥ P (A) + P (B) − 1
which is the inequality.
Although interesting, statement 2 is not the most general, as it leaves open the question,
"what is the probability of a finite union of arbitrary events, say A1 , A2 , . . . Ar ?" We shall
pursue the proof of the answer to this question in chapter 2 and for now merely state that its
conclusion in the context of discrete probability spaces. The following result is combinatorial
in nature if one thinks of P as a specific measurement of size of ∪ri=1 Ai in Ω. If one takes
the general notion of size, that is, of cardinality, then the combinatorial result computes the
size | ∪ri=1 Ai |. Even in the context of discrete probability spaces we use the same name of
the result that computes P (∪ri=1 Ai ) as we use for the result that computes | ∪ri=1 Ai |, that
is, the probabilistic principle of inclusion-exclusion.
Theorem 3. Let (Ω, β, P ) be a discrete probability space and A1 , A2 , . . . , Ar ∈ β be a sequence of arbitrary events, then
Pr
i=1 P (Ai ) −
P
i<j P (Ai ∩ Aj ) +
P (∪ri=1 Ai ) =
P
r−1
r
i<j<l P (Ai ∩ Aj ∩ Al ) + · · · + (−1)
i<...<l P (∩i=1 Ai )
P
One has as a direct application of the inclusion-exclusion principle Boole’s Inequality,
which simply states that, with hypotheses and notation as above, that
P
P (∪ri=1 Ai ) ≤ ri=1 P (Ai )
26
One has an equality whenever A1 , A2 , . . . , Ak is a sequence of pairwise disjoint events, according to Kolmogorov’s axiom 3. Thus, Boole’s inequality furnishes an upper bound in the
general case.
Finally, we conclude both this section and chapter with the law of total probability, a
result whose implications we shall explore both explicitly and implicitly for the remainder
of this text. Again, we hope to indicate now its relevance to Markov processes and Markov
chains below in order to justify its importance to the reader. First we trivially adapt the
definition of partitions to the context of discrete probability spaces.
Definition 20. Let (Ω, β, P ) be a discrete probability space, then we say a sequence of events
{Ai }i∈I ∈ β that are pairwise-disjoint and that cover Ω in the sense that
Ω = ∪i∈I Ai
is a partition of Ω. We refer to the events of the sequence {Ai }i∈I as the cells of the partition.
Furthermore, when |I| = r for some finite natural number r, we say A1 , A2 , . . . , Ar is a
partition of length r.
A partition of a discrete probability space helps us to compute the probability of an
arbitrary event in terms of its restriction to the cells of the partition. As this result applies
to an arbitrary event B, one refers to the following theorem as the law of total probability.
Theorem 4. Let (Ω, β, P ) be a discrete probability space together with a partition of length
r, say A1 , A2 , . . . , Ar . Then for an arbitrary event B ∈ β, we have
P (B) =
r
X
P (B ∩ Ai )
i=1
The proof relies upon the relative view of the probability of an event B with respect to
the cells of the partition. Indeed, by hypothesis, the partition of length r covers Ω, so that
we can write
B =B ∩ Ω
=B ∩ (∪ri=1 Ai )
= ∪ri=1 (B ∩ Ai )
by the distribution law of intersections across unions. The events Ai ∩B are pairwise-disjoint
by the hypothesis that the Ai are the cells of a partition. Therefore, by Kolmogorov’s third
axiom, we have
P (B) =P (∪ri=1 (B ∩ Ai ))
r
X
=
P (B ∩ Ai )
i=1
as desired.
1.5 WORKED EXAMPLES
27
1. 1.5 Consider the experiment of predicting the order made by a diner in a restaurant.
Suppose we know the probability a diner will order at least three of the meals served
in this restaurant, which are apples, beans, or carrots; the probabilities one knows
are, with obvious notation, are: P (A) = 0.7, P (B) = 0.8, P (C) = 0.75, P (A ∪ B) =
0.85, P (A ∪ C) = 0.9, P (B ∪ C) = 0.95, and P (A ∪ B ∪ C) = 0.98.
(a) What is the probability that order is for at least one of apples, beans, or carrots?
(b) What is the probability the diner will order none of apples, beans, or carrots?
(c) What is the probability the diner will only order apples and neither beans nor
carrots?
(d) What is the probability the diner will only order one of the three meals, apples,
beans, or carrots?
Solution (a) We must compute P (A ∪ B ∪ C) but this is given to us in the question,
so P (A ∪ B ∪ C) = .98
Solution (b) We must compute P ((A ∪ B ∪ C)c ). Therefore,
P ((A ∪ B ∪ C)c ) = 1 − .98 = .02
Solution (c) There are several steps in this computation, as we must excise the
subset of events consisting of apples and beans, apples and carrots, and apples,
beans, and carrots. Naively we can write OnlyA = A − A ∩ C − A ∩ B + A ∩ B ∩ C
for this event by inspecting a Venn diagram. First, we compute
P (A ∩ B) = P (A) + P (B) − P (A ∪ B) = .65 after substituting the values in our
hypothesis. Next, we compute both P (A ∩ C) = P (A) + P (C) − P (A ∪ C) = .55 and
P (B ∩ C) = P (B) + P (C) − P (B ∪ C) = .6 in a similar manner, which is to say that
we solve for the intersection of events in the principle of inclusion and exclusion
formula. We compute the latter probability in anticipation of the final computation
we shall require. Indeed, last, we compute
P (A∩B ∩C) = −P (A∪B ∪C)+P (A)+P (B)+P (C)−P (A∩B)−P (A∩C)−P (B ∩C)
again by solving for the intersection of events in the principle of inclusion and
exclusion formula. Substituting the values from the hypotheses and our work so far,
we have P (A ∩ B ∩ C) = .53 and therefore, P (onlyA) = .7 − .55 − .65 + .53 = .03.
Solution (d) Last, we must compute P (onlyA ∪ onlyB ∪ onlyC). The probabilities of
the events onlyB and onlyC are computed in a manner similar to the computation of
P (onlyA). Therefore, P (onlyA ∪ onlyB ∪ onlyC) = P (onlyA) + P (onlyB) + P (onlyC)
since these events are mutually disjoint, and so,
P (onlyA ∪ onlyB ∪ onlyC) = P OnlyA) + P (OnlyB) + P (OnlyC) = .03 + .08 + .13 = .24.
Again, referring to a Venn diagram should be enough to convince one that these
events are indeed mutually exclusive.
28
2. 1.5 Consider the experiment of observing people walk past a bench and recording
whether a passerby trips, falls, or rolls after falling as they pass over a crack in the
sidewalk. Suppose that after observing enough walkers one assigns empirically the
following probabilities the simple events in this experiment, with obvious notation,
P (T ) = .12, P (F ) = .07, P (R) = .05, P (T ∪ F ) = .13, P (F ∪ R) = .14, P (T ∪ R) = .1,
and P (T ∪ F ∪ R) = .01.
(a) What is the probability that a passerby does not trip?
(b) What is the probability a passerby both trips and falls?
(c) What is the probability a passerby both trips and falls but does not roll?
(d) What is the probability a passerby experiences at most two of the events, e.g.
trip and fall?
Solution (a) Compute P (T c ) = 1 − P (T ) = 1 − .12 = .88 by the complement rule.
Solution (b) We must compute
P (T ∩ F ) = P (T ) + P (F ) − P (T ∪ F ) = .12 + .07 − .13 = .06
Solution (c) We must compute P (T ∩ F ∩ Rc ) to solve the problem, for we want to
know the probability of the event that the passerby trips AND falls AND does not
roll. A Venn diagram should convince the reader that T ∩ F ∩ Rc = T ∩ F \ T ∩ F ∩ R
so that P (T ∩ F ∩ Rc ) = P (T ∩ F ) − P (T ∩ F ∩ R) = .06 − .01 = .05. Notice that we
leave the computation of P (T ∩ F ∩ R) to the reader, for both it and the required
probabilities of intersections of events are computed in a manner similar to the
computations made in the previous example.
Solution (d) The probability of this last event is obtained by recognizing that
probability of at most two of the events is the complement of the probability of all
three occurring. Therefore, by the complement rule, we have
P (atmost2) = 1 − P (T ∩ F ∩ R) = 1 − .01 by the previous solution.
1.6
Independent Events
As we have learned how P : β → [0, 1] extends to an arbitrary finite union of events
in a discrete probability space via the probabilistic principle of inclusion and exclusion, we
now turn our attention to its extension to arbitrary finite intersections. Understanding how
P applies to both will completely characterize it, for β with its σ-algebra structure is its
domain, and its σ-algebra structure is generated by the same. In this section, however, we
consider only a special case in order to prepare for the general one.
Let {Ai } ∈ β be a sequence of events, then we say that their intersection ∩i Ai ∈ β
experimentally is a joint event and that its probability P (∩i Ai ) is the joint probability. In
this section we consider a sequence of events whose joint probability is analogous to that
of the probability of the union of a sequence of pair-wise disjoint events, or, experimentally
speaking, are mutually exclusive. The joint probability of this sequence of events will furnish
P with an experimentally relevant joint probability.
29
Definition 21. Let A, B ∈ β be events in an experiment Ω, then we say A and B are
independent events if
P (A ∩ B) = P (A)P (B)
One may perhaps presume that independent events are equivalent to mutual exclusive.
Indeed, both conditions describe a pair of events such that P preserves their logical operation
arithmetically. However, one must not allow this analogy to confuse the two concepts, for
they are not equivalent. Indeed, suppose A and B are mutually exclusive, then P (A∩B) = 0
which would not, in general, mean A and B were independent. Conversely, supposing A and
B are independent, then, in general, P (A)P (B) ̸= 0, so that P (A ∪ B) ̸= P (A) + P (B).
Therefore, mutual exclusivity and independence are logically distinct equivalent notions.
Experimentally, one can interpret independence to mean that, given two events A and
B that occur sequentially, that the outcome of A in the experiment does not predict the
outcome of B. This experimental interpretation is plainly different than the interpretation
of mutual exclusivity, which is to say two events have no simple events in common.
Next we define the independence of a finite sequence of events in an experiment. Our
conditions are that to be independent both the joint probability factorizes into a product of
probabilities and that the pair-wise joint probabilities also factorize. We shall desire such a
definition below in chapter 5 for stochastic processes. Formally, the defintion is as follows.
Definition 22. Let A1 , A2 , . . . , Ar ∈ β be a sequence of events in an experiment Ω. We say
the sequence of events is independent if both
Q
P (∩ri=1 Ai ) = ri=1 P (Ai )
and, for each 1 ≤ i, j ≤ r such that i ̸= j, we have
P (Ai ∩ Aj ) = P (Ai )P (Aj )
Let us consider a few routine consequences of the definition in the following proposition.
Proposition 11. Let A, B ∈ β be independent events in an experiment Ω. Then the following
events are also independent.
1. A and B c
2. Ac and B
3. Ac and B c
To prove this proposition, let us begin by proving the first item. To that end, recall from
chapter 1 that we can write A = (A ∩ B) ∪ (Ac ∩ B), so that
P (A) =P (A ∩ B) + P (A ∩ B c )
P (A) − P (A ∩ B) =P (A ∩ B c )
P (A) − P (A)P (B) =
P (A)(1 − P (B)) =
P (A)P (B c ) =P (A ∩ B c )
30
which illustrates the first item of the proposition by the definition of independence. The
proof of the second item is the same mutatis mutandi. The third claim is an application of
DeMorgan’s law. Observe
P (Ac ∩ B c ) =P ((A ∪ B)c )
=1 − P (A ∪ B)
=1 − (P (A) + P (B) − P (A ∩ B))
=1 − (P (A) + P (B) − P (A)P (B))
=(1 − P (A))(1 − P (B))
=P (Ac )P (B c )
which again exhibits the complements of independent events are independent themselves
by the definition of independence. Let us now consider several examples that require us to
compute the probability of independent events before considering the general case below.
1.6 WORKED EXAMPLES
6
1. Consider the experiment of tossing an unfair coin twice, where P (H) =
is the
10
probability of heads, and recording the sequence of results.
(a) What is the probability of the event, "two Heads"
(b) What is the probability of the event, "one Head"
(c) What is the probability of the event, "no Heads"
(d) What is the probability of the event, "at least one Head"
Solution (a) The simple events of this experiment are given by pairs
Ω = {(H, H), (H, T ), (T, H), (T, T )}. Clearly the second flip is independent of the
first, experimentally speaking. Let A, B ∈ β be the events "Heads first" and "Heads
second," respectively. The solution requires us to compute the probability of the
simple event (H, H). Observe, P (H, H) = P (A ∩ B), as they are described above. So,
2
6
.
given these events are independent, we have that P ((H, H)) = P (A)P (B) =
10
Solution (b) To answer this question, let us charactertize this event, say C, in terms
of A and B, as above. As such, write C = (A ∩ B c ) ∪ (Ac ∩ B). Now the sets in this
union are disjoint, so that P (C) = P (A ∩ B c ) + P (Ac ∩ B). Now, as A and B are
independent events, so too are the intersections of events in the arguments of P by
6 4
4 6
12
the earlier proposition. Therefore, we have P (C) =
+
= .
10 10 10 10
25
Solution (c) Let us compute the probability of the event "no Heads". Again, we
shall describe this event in terms of A and B, as above. Hence, let D = Ac ∩ B c and
c
c
notice that, once more, A
and
2 B are independent by the proposition. Therefore,
4
P (D) = P (Ac )P (B c ) =
.
10
31
Solution (d) Let us compute the probability of the event "at least one Head". We
can achieve this indirectly by recognizing that this event, say E, is indirectly given in
16
.
terms of A and B by E = Dc , so that P (E) = 1 − P (E c ) = 1 − P (D) = 1 −
100
Equivalently, we could write E = A ∪ B.
2. Consider the experiment of tossing two fair four-sided dice and recording the sequence
of results. Notice that the sample space of this experiment consists of pairs Ω =
{(i, j) | 1 ≤ i, j ≤ 4}.
(a) Consider the events "the first roll is i" and "the second roll is j" denoted A(i)
and B(j), respectively. Determine whether A(i) and B(j) are independent for
admissible i, j by verifying the definition.
(b) Continuing with this experiment, let use determine whether the events
A = {(i, j) | max(i, j) = 2 } and B = {(i, j) | min(i, j) = 2 } are independent.
Solution (a) Notice that, by the multiplication rule, |Ω| = 16. Moreover, by the
fairness hypothesis, the simple events are equally as likely when we adopt the
1
canonical definition of P , so that P (i, j) = . Equivalently, Ω is an equiprobable
16
space.
We answer this question by verifying the definition of independence, as follows. First,
4
4
and P (B(j)) = . Second, observe that
it is clear that both P (A(i)) =
16
16
1
P (A(i) ∩ B(j)) = P ((i, j)) = . Therefore, A(i) and B(j) are independent, by
16
definition, since P (A(i) ∩ B(j)) = P (A(i))P (B(j)).
Solution (b) As above, we shall verify the definition, but here we may compute the
cardinality of these events by inspection. Observe, C = {(1, 2), (2, 1), (2, 2)} and
3
5
and P (D) = . Yet,
D = {(2, 2), (2, 3), (3, 2), (2, 4), (4, 2)}, so P (C) =
16
16
1
P (C ∩ D) = , thus C and D are not independent. Notice that in part (a) we could
16
have asserted the independence of A and B experimentally for they occur
sequentially. The aspects of the events underlying C and D do not, thus we must
reckon whether they are independent by using the definition.
3. Consider the experiment of tossing two fair dice and recording the sequence of results.
Notice that the sample space consists of pairs Ω = {(i, j) | 1 ≤ i, j ≤ 6 }.
(a) Let A = {(i, j) | i + j = 7, 11 }, that is, the event that the total of the results is
7 or 11. Compute P (A).
32
(b) Consider the related experiment of tossing two fair dice and recording the
results twice. Let B be the event "neither a 7 nor 11 total on either toss".
Compute P (B).
Solution (a) To compute P (A) we can proceed to compute by inspection of Ω in
conjunction with the hypothesis that Ω is equiprobable, as the dice are fair. We leave
8
the details of the reader to show there are 8 simple events in A, and so, P (A) = .
36
Solution (b) Here we recognize that B may be described as Ac1 ∩ Ac2 , where the
subscript indicates the occurrence of the toss of the pair, and the complement is used
to indicate that we want the total to be neither 7 nor 11. As such, the question asks
us to compute P (B) = P (Ac1 ∩ Ac2 ). Plainly A1 is independent of A2 , therefore Ac1 is
7
2
independent of Ac2 . By the previous part, we know that P (Ac ) = 1 − = .
9
9
49
c
c
Therefore, P (B) = P (A1 )P (A2 ) = , as desired.
81
4. Consider the experiment consisting of sequences of n ∈ N flips of an unfair coin, where
1
the probability of heads is P (H) = , and recording the sequence of results.
3
(a) Compute the probability of x heads in n flips.
(b) Compute the probability of 4 heads in 5 flips.
Solution (a) We proceed by the independence of the flips and the general addition
rule. Observe, by independence,
probability of x heads in a sequence of n flips is
x the
n−x
1
2
P (x H ′ s, n − x T ′ s) =
. Now we invoke the general addition rule to
3
3
compute the probability of the mutually exclusive various combinations of these x
heads and n − x tails. In particular, there
arenCx
such combinations,
chapter 2,
n−x
x by
n−x
x
PnCx 1
2
n
1
2
′
′
so P (∪nCx
=
i=1 {x H s, n − x T s}i ) =
i=1
3
3
x
3
3
Solution (b) We recognize the solution to this question is obtained by substituting
the values n = 5 and x = 4 in the parameters characterizing the probability of such
events in general from the previous question. We can summarize the probability of
this events in terms of the parameters n, p and x, denoted by B(n, p)(x) in general
4 1
5
1
2
10
and B(4, )(4) =
= 5 in particular.
3
4
3
3
3
33
1.7
Conditional Probability and Baye’s Theorem
In this section we consider the so-called conditional probability of an event A, that is,
the probability of an event A given another event B has occurred already in an experiment. Abstractly, one could say that conditional probability introduces a notion of relative
probability. Moreover, this consideration furnishes us with the appropriate generalization of
the joint probability analogous to the principle of inclusion and exclusion. Considering the
myriad of ways in which such conditional probabilities occur experimentally, a fortiori we
should desire a precise formulation.
In terms of section 3, this section generalizes its material by replacing Ω by B with respect
to the computation of P (A). It is in this sense the following notion of condition probability is
the relative probability of an event A. Indeed, rather tautologically, one could think of P (A)
as only meaning the probability of A given that Ω was performed, in experimental terms.
In terms of a discrete probability space, however, we can characterize this experimental
interpretation as the events in A and Ω as a subset of Ω. As stated, this interpretation is
tautological, for A∩Ω = A, however, it is still useful, for it suggests that we can generalize the
probability P (A) by replacing Ω with an arbitrary event B ∈ β. Experimentally we interpret
this replacement to mean the event A given that B was performed. The generalization of
the event A and its probability to the relative setting gives rise to the following definitions.
Definition 23. Let A, B ∈ β be events in an experiment Ω. We say the conditional event
of A with respect to B is the intersection of A and B as a subset of B, denoted A|B. We
read this as "A given B". Furthermore, the probability of A given B is
P (A|B) =
P (A ∩ B)
P (B)
and is called the conditional probability.
Observe that one should not expect in general for the conditional events A|B and B|A
to be the same events in an experiment. In particular, one should not expect that their
probabilities are the same. Suppose they are, for arbitrary A, B ∈ β, then P (A|B) = P (B|A)
implies P (A) = P (B) which is absurd for arbitrary events. Thus, the operation of forming
conditional events is non-commutative.
This relative view of the probability of an event A also avails one of a an experimental
meaning of independent events. Indeed, supposing A and B are independent, then P (A|B) =
P (A)P (B)
P (A ∩ B)
=
= P (A). Thus, we obtain the experimental interpretation of the
P (B)
P (B)
previous section that two events are independent if the two events are performed in sequencehere B first, then A second-and the performance of the first event does not predict the
performance of the second event. A well-known alternative to our definition of independence
is that A and B are independent events if P (A|B) = P (A) and vice versa. Of course, one
must be careful to specify this alternative definition only applies to B ̸= ∅ when defining
independent events with respect to conditional probability.
This last point often justifies defining conditional probability without implicitly assuming
that B is not the empty set. In this case, one defines conditional probability through the
so-called multiplication rule for probabilities, which becomes an expression for the joint
34
probability, viz. P (A|B)P (B) = P (A ∩ B). This reformulation of conditional probability
in terms of joint probability is the basis for induction used to prove the following joint
probability formula for an arbitrary finite sequence of events. Indeed, this is the analogue
for joint probability of the probabilistic principle of inclusion and exclusion.
Theorem 5. Let A1 , A2 , . . . , Ak ∈ β be a sequence of events in an experiment Ω. Then
P (∩ki=1 Ai ) = P (A1 )P (A2 |A1 )P (A3 |A1 ∩ A2 )P (A4 |A1 ∩ A2 ∩ A3 ) · · · P (Ak |A1 ∩ A2 ∩ · · · ∩ Ak−1 )
In the case when k = 3, we have that P (A1 ∩ A2 ∩ A3 ) = P (A3 |A1 ∩ A2 ) · P (A1 ∩ A2 ) =
P (A3 |A1 ∩ A2 ) · P (A2 |A1 )P (A1 ) by iterating the definition. It is a straightforward induction
argument to demonstrate the conclusion of the theorem with this observation as the basis
for induction.
Last we state and prove a most famous theorem which computes the a posteriori probability of a cell in a partition of an experiment given an arbitrary event. Experimentally we
refer to it as an "after the fact" probability because the theorem computes the likelihood that
an event of a cell in an experiment has occurred given that an arbitrary event has occurred
already. If we regard the cells of an experiment as its foundational aspects, we are therefore
inquiring about the occurrence of a cell after the fact once an arbitrary event has occurred.
This result is known as Baye’s Theorem.
Theorem 6. Let (Ω, β, P ) be a discrete probability space together with a partition of length
r, say A1 , A2 , . . . , Ar . Then for an arbitrary event B ∈ β, we have
P (Ai )(P (B|Ai ))
P (Ai |B) = Pr
j=1 P (Aj )P (B|Aj )
for any 1 ≤ i ≤ r.
The proof is fairly easy as it is an application of the definition of conditional probability
and the law of total probability. Indeed, the numerator is given by P (Ai ∩ B) by the
definition of condition probability, which simplifies as P (Ai )P (B|Ai ). The denominator is
given by P (B), again by the definitionPof conditional probability. Furthermore, the law of
total probability states that P (B) = rj=1 P (B ∩ Aj ) since the {Ai } is a partition of Ω.
Therefore, following the simplification of the single term in the numerator, we have, for each
j, that P (B ∩ Aj ) = P (Aj )P (B|Aj ). as desired.
1.7 WORKED EXAMPLES
1. Consider the experiment consisting of a pair of fair six-sided dice being thrown and
recording the results of each die. Notice that the sample space of this experiment
consists of pairs Ω = {(i, j) | 1 ≤ i, j ≤ 6 }.
(a) Compute the probability of the event such that i + j = 8 given i, j ∈ 2N, that is,
both i and j are even.
35
Solution (a) Let us express this event as the conditional event A given B has
occurred, where A ∈ β is the set of ordered pairs (i, j) such that i + j = 8 and B ∈ β
is the set of ordered pairs (i, j) such that i, j ∈ 2N. In terms of this notation, we must
P (A ∩ B)
compute P (A|B) =
P (B)
Proceeding by inspection, we have A = {(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)} and
B = {(2, 2), (2, 4), (2, 6), (4, 2), (4, 4), (4, 6), (6, 2), (6, 4), (6, 6)}. Thus,
3
9
A ∩ B = {(2, 6), (4, 4), (6, 2)}, so P (A ∩ B) = . Next, we have P (B) = .
36
36
1
Therefore, P (A|B) =
3
2. 3.2 Consider an experiment that involves an urn containing five blue balls and seven
grey balls and an experimenter draws two balls randomly in the order without replacement after drawing from the urn. The experimenter records the color of the balls that
are drawn and the order in which they are drawn.
(a) Compute the probability of the event that both balls drawn are blue.
(b) Compute the probability of the event that the second ball drawn is blue, but
that the first ball drawn is not.
(c) Compute the probability that the second ball drawn is blue.
(d) Compute the probability of the event that one of the balls drawn is blue.
Solution (a) Let us denote by A the event that the first ball drawn is blue and by B
the event that the second ball drawn is blue. In terms of this notation, we want to
compute P (A ∩ B) which in terms of its condition probability may be expressed as
P (A ∩ B) = P (B|A)P (A), we shall proceed to compute the terms on the right hand
5
by
side of the equation to solve the problem. First, observe that P (A) =
12
inspection and the relative frequency interpretation of probability. Similarly, we have
4
20
P (B|A) = . Therefore, P (A ∩ B) =
.
11
132
Solution (b) With notation as above, we can write the event as Ac ∩ B. Observe
7
5
that P (Ac ) =
and P (B|Ac ) = . Therefore,
12
11
7 5
35
c
c
c
P (A ∩ B) = P (B|A )P (A ) =
·
=
.
12 11
132
Solution (c) With notation as above, observe that we can partition B as
B = (B ∩ A) ∪ (B ∩ Ac ). Hence, by the previous two problems, we have
20
35
55
P (B) =
+
=
132 132
132
36
Solution (d) Again, with notation as above, we can write this event as
P (A ∪ B) = P (A) + P (B) − P (A ∩ B) by chapter 1. Thus,
55
20
90
5
+
−
=
P (A ∪ B) =
12 132 132
132
3. Consider the experiment of conducting a medical test that screens for the presence of
a disease known to be found in 5 people per 1000 in the population. Suppose that the
test falsely indicates the disease 3 % of the time and fails to detect the disease 1% of
the time. In epidemiological terms, one says the former case is a false positive and the
latter case is a false negative.
(a) Compute the probability of the event that a randomly chosen individual who
tests positive for the disease actually is infected by the disease.
(b) Compute the probability of the even that a randomly chosen individual who
tests negative for the disease is not infected by the disease.
We proceed by first clarifying how to characterize both false positive results and false
negative results of the test. Observe that a test result is not equivalent to the state
of being infected, or not, by a disease, as the test results are fallible. The thresholds
given for false positive and negative results are quantifications of this observation.
Accordingly, these quantifications have bearing on the probabilities of the events in
question. Specifically, they will tell us the conditional probabilities of the test results
given the event one actually is infected or not by the disease. We use this below to
answer the questions and to solve such false positive/negative questions, in general.
Solution (a) Let us write A1 for the event that one actually does have the disease
and A2 for its complement, that is, for the event that one does not. Furthermore, we
can write B for the event that one tests positive for the disease given a likelihood of
false positives of 3 % and a likelihood of false negatives of 1 %. Then with this
notation, the question asks us to compute P (A1 |B) which can be computed in terms
of A1 and A2 with respect to Baye’s theorem for one either does or does not have a
disease. Accordingly, we have
P (B|A1 )P (A1 )
P (B|A1 )P (A1 ) + P (B|A2 )P (A2 )
0.99 · 0.005
=
0.99 · 0.005 + 0.03 · 0.995
=0.1422
P (A1 |B) =
since P (B|A1 ) = .99, for this is the probability of the complement of a false negative;
P (B|A2 ) = .03, for this is the probability of a false positive; and P (A1 ) = 0.005,
P (A2 ) = 1 − P (A1 ) = 0.995, by hypothesis.
37
Solution (b) With notation as above, let us write C for the event that one tests
negative for the disease give a likelihood of false positives of 3 % and a likelihood of
false negatives of 1 %. As above, we proceed by Baye’s theorem with respect to the
partition of the population A1 , A2 to obtain
P (C|A2 )P (A2 )
P (C|A1 )P (A1 ) + P (C|A2 )P (A2 )
0.97 · 0.995
=
0.01 · 0.005 + 0.97 · 0.995
=0.999948
P (A2 |C) =
since P (C|A2 ) = .97, for this is the probability of the complement of a false positive;
P (C|A1 ) = .01, for this is the probability of a false negative; and P (A1 ) = 0.005,
P (A2 ) = 1 − P (A1 ) = 0.995, by hypothesis.
4. Consider an experiment consisting of two urns, one containing 3 red marbles and 2
blue marbles, and the other containing 2 red marbles and 8 blue marbles. A fair coin
is tossed by an experimenter and checked: if the coinc lands on heads, a marble is
chosen from the first urn, whereas if the coin lands on tails, a marble is chosen from
the second urn.
(a) Compute the probability of the event in this experiment that a red marble is
chosen.
(b) Compute the probability of the event the first urn was chosen given a red
marble had been chosen.
Solution (a) Observe that the experiment is partitioned by choosing marbles from
either the first or second urn. Let us denote the cells of this partition by U1 , U2 ,
respectively. Then, for R the event a red marble is chosen, we have by the law of total
1
probability that P (R) = P (R|U1 )P (U1 ) + P (R|U2 )P (U2 ). Both P (U1 ) = P (U2 ) =
2
by the hypothesis that an urn is chosen by the flip of a fair coin. As for the condition
probabilities, we have, by the relative frequency interpretation
ofprobability that
2
1 3
2
2
3
P (R|U1 ) = and P (R|U2 ) = . Therefore, P (R) =
+
=
5
10
2 5 10
5
Solution (b) With notation as above, we wish to compute the condition probability
P (U1 |R) so we may proceed by Baye’s theorem to do so, since the Ui partition the
experiment. Observe,
P (R|U1 )P (U1 )
P (R|U1 )P (U1 ) + P (R|U2 )P (U2 )
0.6 · 0.5
=
0.4
=0.75
P (U1 |R) =
38
5. Consider a chess tournament and suppose that the likelihood of winning a game is 30
% against half the players, 40 % against a quarter of the players, and 50 % against the
remaining quarter of players.
(a) Compute the probability of winning a game of chess in the tournament.
(b) Compute the probability of the event that you have played an opponent of the
half of players whom one has only a 30 % chance of victory against given one
has won their game.
Solution (a) Let W be the event winning a game of chess in the tournament.
Further, let A1 be the event play half the players with 30% chance of victory.
Similarly define events A2 and A3 , then clearly the Ai partition the experiment of the
chess tournament. Therefore, by the law of total probability, we have
P (W ) = P (A1 )P (W |A1 ) + P (A2 )P (W |A2 ) + P (A3 )P (W |A3 ) and by the hypotheses,
we have
P (W ) =P (A1 )P (W |A1 ) + P (A2 )P (W |A2 ) + P (A3 )P (W |A3 )
=0.5 · 0.3 + 0.25 · 0.4 + ·0.25 · 0.5
=0.375
Solution (b) With notation as above, we must compute P (A1 |W ). To do this, we
shall proceed by Baye’s theorem, since the Ai partition the experiment. Observe,
P (W |A1 )P (A1 )
P (W )
0.3 · 0.5
=
0.375
=0.4
P (A1 |W ) =
1.8
Chapter 1 Homework Exercises
1. Allen, Baker, Cabot, and Dean are to speak at a dinner. They will draw lots to determine the order in which they will speak. Please answer the following questions:
(a) List all the elements of a sample space Ω associated to the experiment of recording
the order in which these four individuals speak at the dinner.
(b) Mark with a check the simple events in part a.) contained in the event A =
{Allen speaks before Cabot} ⊂ Ω
39
(c) Mark with a cross the elements of the event,
A = {Cabot′ s speech is between those of Allen and Baker}
(d) Mark with a star the elements of the event
A = {The four persons speak in alphabetical order}
2. Consider the experiment "A fair coin is tossed five times."
a.) Determine the sample space Ω associated to this experiment. Furthermore, count
the number of simple events in the same. In set theoretic notation, this means to
compute |Ω| i.e. the cardinality of the sample space as a set.
Please use the fact that the discrete probability space (Ω, β, P ) determined by the experiment in a.) is an equiprobable sample space to compute the following probabilities
of the events named in the following questions:
b.) Heads never occurs twice in a row.
c.) Neither heads nor tails ever occurs twice in a row.
d.) Both heads and tails occur at least twice in a row.
3. Consider the experiment whose of simple events are the record of "two dice are thrown"this sample space Ω therefore consists of pairs (i, j) such that 1 ≤ i, j ≤ 6
Let
A = {The total is two}
B = {The total is seven}
C = {The number shown on the first die is odd}
D = {The number shown on the second die is odd}
E = {The total is odd}
be events in this experiment. Given that (Ω, β, P ) is an equiprobable sample space,
compute the following probabilities:
a.) P (A)
b.) P (B)
c.) P (C)
40
d.) P (D)
e.) P (E)
f.) P (A ∪ B)
g.) P (A ∩ B)
h.) P (A ∪ C)
i.) P (C ∩ D ∩ E)
j.) P (B ∪ Dc )
4. Prove, by the aid of Venn diagrams, that for any probability space (Ω, β, P ) and events
A, B, C ∈ β that
P (A ∩ B) + P ((A ∖ B) ∪ (B ∖ A)) + P (Ac ∩ B c ) = 1
Recall the following definition in set theory.
Definition 24. Let A and B be sets, then we say
A ∖ B = {x ∈ A|x ∈ A, x ∈
/ B}
is the difference of B in A.
Remark: "by aid of Venn diagrams" means drawing a set of Venn diagrams that
exhibits the underlying set theoretic relation in conjunction with a remark to that
effect that, "because P is a probability distribution function it does this by the axioms
of a probability space" is sufficient. TLDR: you can just use pictures.
5. Recall the defintion of the power set of an arbitrary set, A: We say the set of all
subsets of A, denoted 2A , is the power set of A.
Let A = {a, b, c} be a set of three elements. Consider the set function
f : 2A → Z
defined by f (B) = |B| that is, the image of a subset of A under f is the number of its
elements.
Draw a diagram that illustrates this function that includes both its domain and its
image. Use this diagram in conjunction with the definition to illustrate why f is not
an injection i.e. is not 1-1.
41
6. Recall, a rule assigning elements of a set A to a set B is said to be well-defined if the
assignment of elements in A to those of B is unique. Furthermore, such a rule is said
to be a function when it is well-defined. Consider the rule
f :Q→Z
where f
m
n
= m − n for any q =
m
∈ Q. Explain why this rule is not a function.
n
7. Suppose A, B, C partition the set Ω and that (Ω, β, P ) is a probability space. Suppose
further P (A) = 0.3 and P (B) = 0.5. Compute both the probability P (A ∪ B) and the
probability P (C).
8. Suppose A, B ∈ β are events in the probability space (Ω, β, P ) and P (A) = 0.8, P (B) =
0.7, and P (A ∩ B) = 0.6. What is the probability P (A ∪ B)?
9. Prove that, for any probability space (Ω, β, P ) and events A, B ∈ β, if
P (B)
1
1
P (A)
+
=
+
P (A ∩ B) P (A ∩ B)
P (A) P (B)
then A and B are independent.
rmk : the“proof" here is just a manipulation of the given equality to obtain the definition
of independence, just like the“proofs" in class. Remember, too, that you are allowed
to assume the centered identity, so to demonstrate/prove that the conclusion“A and
B are independent" from it, you merely have to derive the definition of independence
given the centered identity.
10. Consider the following experiment. A playing card is drawn from a deck of 52 cards
and replaced, then a second card is drawn. The associated sample space Ω consists of
pairs which record the suit and denomination of the two cards drawn. Let A be the
event, "the first card is a spade. Let B be the event, "the second card is a spade." Let
C be the event, "both cards have the same color." Please recall that playing cards are
either red, the hearts and diamonds, or black, the spades and clubs.
Determine whether
a.) A and B are independent.
b.) B and C are independent.
c.) A, B, and C are independent.
rmk : Note that to solve c.) you will need to use the general definition of independence
we gave in class, because there are three events whose independence of which you are
trying to determine.
42
11. Consider the following experiment of rolling both one six sided red die and one six
sided blue die and recording the individual outcomes. What is the probability of the
event the sum of the numbers on both the red and the blue dice is 7 and the value on
the blue die is larger than the value on the red die?
12. Suppose an urn contains 25 red balls and 15 blue balls. Consider the experiment of
choosing 2 balls from the urn, without replacement. The associated sample space consists of the pairs recording the color of the balls selected.
What is the probability of the event, "both balls are red?"
13. A test that screens for illegal drug use, i.e. a“drug test," is used in a large population
of people in which 4 % of who actually use drugs. Suppose that the false positive
rate in the drug test is 3 % and that the false negative rate in the drug test is 2 %.
Accordingly, an individual who actually uses drugs tests positive for drug use 98 % of
the time, whereas an individual who does not use drugs tests negative for drug use 97
% of the time. What is the probability that an individual randomly chosen who tests
positive for illegal drug use actually uses illegal drugs?
14. A laboratory blood test is 95 % effective in detecting a certain disease when it is
actually present. However, the test also yields a “false positive" result for 1 % of the
healthy persons tested. If .5 % of the population actually has the disease, what is the
probability a person has the disease given the test result is positive.
43
2
Elementary Combinatorics and Arrangements of Sets
In this chapter we study the introduction to counting ideas and concepts required to
compute the probability of events with respect to the relative frequency interpretation of
probability discussed in section 1.4, to find experimentally sound values for pi in section
1.3, and to demonstrate that parametric families of random variables satisfy Kolmogorov’s
axioms, to mention a few motivations for the study of combinatorics that arise in this text.
These motivations demand a more sophisticated counting technique than inherent in the
commonplace methods of counting, which usually means simply to enumerate objects.
The most well-known interpretation of the probability of an event is to count the number
of performances of an experiment favorable to it and to divide this figure by the total number
of performances of the experiment-both figures require a counting technique germane to
counting the cardinality of a set. Counting by enumerating the elements of a set is subject
of combinatorics. We therefore count by constructing bijections between sets following the
proposition in the previous chapter relating bijections to set cardinality. In particular, our
introduction to combinatorics proceeds according to both the addition and multiplication
rule, both of which rely upon the aforementioned point of view. Once these rules have been
established we shall apply them to counting arrangements of the elements of a finite set,
which are sequences of their elements defined by hypotheses determining the order of their
elements and the method of selection for inclusion in the list.
After we have introduced discrete random variables in chapter 3, we shall apply the
counting techniques of this chapter to simplifying the probability functions of parametric
families of random variables into formulas. Moreover, we shall rely upon results worked
out in this chapter to verify said formulas satisfy Kolmogorov’s axioms. These arguments
will rely upon recognizing arrangements in Bernoulli processes, which to enumerate, will of
course require combinatorics.
Let us introduce a fundamental counting principle that shall underlie both everything
discussed below in the prose as well as to obviate for the student any potential confusion
over questions of methodology. The following is referred to as the fundamental theorem of
counting below.
Theorem 7. Let S be a finite set, say |S| = n. Then if the elements of S are counted in
two distinct ways, the conclusions are the same.
This deceptively simple statement should convince the reader that if their method for
counting the elements of a set differ from that of the text, provided the method is sound,
that the reader’s approach is equal to that of the text. Soundness means more specifically in
bijective correspondence. It is in this manner that we dispel lingering questions about the
"right" way to undertake a combinatorial argument. Indeed, rather tautologically one could
say, the pedagogical perspective we adopt is that the correct answer is independent of the
method of counting.
2.1
The Addition and Multiplication Rules
In this section we state and prove the multiplication, which is the counting technique that
shall underlie all of the combinatorics in this text. It is for this reason we assert that our
44
treatment of this subject is elementary. We begin with a simple counting lemma we shall
use several times below.
Lemma 1. Let r, n ∈ N such that r ≤ n, then the number of integers from r through n is
n−r+1
The formal proof is by induction and it is not difficult. Informally, one must keep track
of the position of the integers and count the total number of positions. Observe that, in the
following list
r =r + 0
r + 1 =r + 1
...
r + i =r + i
...
n = r + (n − r)
i keeps track of the position of an integer from r through n. Therefore, r + i is in the 1 + i-th
position. In particular, there are n − r + 1 integers from r through n.
We turn to the statement and proof of the multiplication rule by stating a seemingly
unrelated result. Indeed, it is the set theoretic generalization of the law of total probability.
Proposition 12. Let S be a countable set and A1 , A2 , . . . , Ar be a partition of length r.
Then for any B ⊂ S, we have
|B| =
r
X
|B ∩ Ai |
i=1
To prove this proposition, we proceed by the perspective that informs us two sets have
the same size if and only if there exists a bijection between them. That, together with
the agreement that the cardinality of a union of disjoint sets is equal to the sum of the
cardinalities of the sets in the union compels us to consider whether a bijection
f : B → ∪ri=1 B ∩ Ai
exists, since the induced sequence of sets B ∩ A1 , B ∩ A2 , . . . , B ∩ Ar are pairwise-disjoint
Since S is countable and B ⊂ S by hypothesis, impart a countable labeling to the
elements of S and write l for this labeling of b ∈ B. Let us define f on B by f (bl ) = bil ,
where i is the unique index of the partition such that bl ∈ Ai . Then with this definition, f
is an injection, for if bil = bjk then i = j by the hypothesis the Ai are a partition of length
r. So, f restricted to Ai for each i maps bl to bil , therefore, if bil = bik that gives their fibres
are equal. Since f restricted to Ai fibres it as Ai = ∪l bl , this gives l = k, so f is injective.
Similarly, f is surjective, for suppose bl ∈ ∪ri=1 B ∩ Ai , then there exists a unique i such that
bl ∈ B ∩ Ai , but then bil = f (bl ) for bl ∈ B. As bl was chosen arbitrarily, f is surjective.
Therefore, f is a bijection,
Pr for it is both an injection and a surjection, and so, we have that
r
|B| = | ∪i=1 B ∩ Ai | = i=1 |B ∩ Ai |, since the sets of B ∩ A1 , B ∩ A2 , . . . , B ∩ Ar partition
B, so we can count and add their elements separately, as desired.
45
An important
corollary of this result is when we take the trivial case that B = S, for
Pr
then |S| = i=1 |Ai |, as Ai = S ∩ Ai . Let us add to our repetoire by declaring a partition of
length r is of size ni if |Ai | = ni , for 1 ≤ i ≤ r. Then we have the following famous result,
the so-called addition rule.
Proposition 13. Let S be a finite set, say n = |S|, together with a partition of length r of
size ni . Then
n=
r
X
ni
i=1
There is nothing to prove as the addition rule is a special case of the proposition. There
is also a difference rule, dual to the addition rule. Define A \ B = A ∩ B c , for some pair
B ⊂ A ⊂ S.
Proposition 14. With notation as above, |A \ B| = |A| − |B|
The proof is again an application of the proposition, this time taking B, A \ B as a
partition of length two of A itself. The result follows from the addition rule solving for
|A \ B|.
The important application of these results that we have in mind is to stating and proving
the multiplication rule, the principle underlying our counting arguments throughout this
chapter. To achieve this aim, we shall digress once more, this time to computing the cardinality of the Cartesian product of a sequence of sets. Here we begin to develop a theme that
will culminate in our study of Markov chains.
Proposition 15. Let A1 , A2 , . . . , Ak be a sequence of pairwise-disjoint sets such that ni =
|Ai | and S = A1 × A2 × · · · × Ak their Cartesian product. Then
|S| =
k
Y
|Ai |
i=1
where
Qk
i=1 ni = n1 · n2 · · · nk is the notation for the product of the numbers ni .
We will not prove this statement in its entirety, as a full proof would require induction.
Instead, we will prove the basis for induction on k by exhibiting the proposition for k = 2.
We begin by exhibiting a bijection f between the sets A2 and a × A2 , where a ∈ A1 and with
notation as in the proposition otherwise. So, for f : A2 → a × A2 let us define for b ∈ A2 the
function f (b) = (a, b) ∈ a × A2 . Clearly f is both injective and surjective as A1 ∩ A2 = ∅.
Therefore, as usual, |A2 | = |a × A2 |.
Second, notice that our above choice of a ∈ A1 was arbitrary. Moreover, we can decom1
pose A1 = ∪ni=1
ai , where |A1 | = n1 . As such, the sequence a1 × A2 , a2 × A2 , . . . , an1 × A2 is
a partition of A1 × A2 of length n1 . Therefore,
X
|A1 × A2 | =
|ai × A2 |
ai ∈A1
46
Now, the cardinalities are all equal by the transitivity of the relation "there exists a bijection."
To wit, A2 ∼
= ai × A2 ∼
= aj × A2 for any ai , aj ∈ A1 and where ∼
= means "there exists a
bijection". Since |A1 | = n1 and |A2 | = n2 by hypothesis,we have
|A1 × A2 | =
n1
X
n2
i=1
=n1 · n2
as desired. As mentioned at the outset, one merely requires the induction hypothesis to
finish the proof, which would proceed in exactly the same manner up to replacing A1 by
A1 × · · · Ak−1 .
We have covered enough material now to state the multiplication rule. However, we no
longer must prove it, as we have accomplished this already. Indeed, the proof of the multiplication is merely to forget the labels we added to the sets in the above proposition for
its statement. In other words, abstractly, the proposition is the multiplication rule. However, such an austere presentation is inappropriate for what we have in mind, so we phrase
matters differently in order for the proposition to comport with the counting arguments and
applications that we have in mind.
Let us define a process to be the k-fold Cartesian product S = A1 ×A2 ×· · ·×Ak . Furthermore, we assert that a performance of a process is a k-tuple thereof, say (a1 , a2 , . . . , ak ) ∈ S.
Furthermore, one says that a step in this process is one of the factors of the Cartesian process itself, thus there are k independent steps. In order to distinguish processes from the
abstract rendering of elements of Cartesian products we shall write words or concatenations
of the elements of the k-tuple, instead. Accordingly, with notation as in the proposition, the
multiplication rule states there are
k
Y
ni
i=1
ways to perform the process or that, the number of ways to perform a process that consists
of k steps is the product of the number of ways to perform each step. We summarize this
discussion in the following theorem.
Theorem 8. (Multiplication Rule) Let A1 , A2 , . . . , Ak be a sequence of pairwise-disjoint sets
up to indexing, such that ni = |Ai | and S = A1 × A2 × · · · × Ak their Cartesian product.
Define S to be a process, Ai a step in the process, and a performance of the process to be a
word a1 · a2 · · · ak obtained by concatenation of elements such that ai ∈ Ai for each 1 ≤ i ≤ k.
Then there are n1 · n2 · · · nk performances of the process S.
There is nothing to prove for, as mentioned above, the theorem is nothing more than a
re-phrasing intended for applications to a statement we have already proven that computes
the cardinality of the a Cartesian product of pairwise-disjoint sets. However, the hope is
these labels help one to organize their arguments in order to learn the basics of counting.
In particular, the phrase "up to labeling" is intended to suggest two sets in the sequence
underlying a process are disjoint if their indices are distinct. This will allow us to use
recursive arguments below to count various arrangements of the elements of a set, for we are
47
then able to distinguish elements of a performance of a process by the step in which they
are introduced or abstractly speaking, their index.
A process may be visualized informally as a rooted tree diagram. We stress the informality
of this presentation to stress its contradistinction with respect to the rigorous or formal
alternative, which requires initially the definition of a graph. We shall provide this material
later in chapter 5, yet it is easy enough to work with those concepts now, as long as one
agrees to ignore ambiguities and the imprecision of the informal treatment. Granting this
approach, let us consider the set of performances of a process S as the set of branches in a
tree diagram whose vertex labels are the elements of the corresponding k-tuple. We will call
such a rooted tree a process tree.
A process tree is a visual illustration of the elements of a Cartesian product of sets. The
branches of the tree correspond to tuples or elements of the Cartesian product. The elements
k
of the factors the product, S = i=1 Ai , say Ai , appear written horizontally at the i-th level,
where level is counted from left to right starting at i = 1 and ending at i = k. Given this
ordering, we would say an element in Ai is of height i. One writes the elements that comprise
the performance of the process as the labels of the vertices in the corresponding path. One
writes the word that is determined by the vertex labels to the right of the leaf determining
the same. Such words furnish the correspondence between elements of the Cartesian product
and branches of the process tree. One notes that the number of ramifications of each vertex
in a process tree of height i is |Ai+1 |.
Consider the following example of a process tree associated to the Cartesian product
n
B of n sets Bi = {s, f } for all i = 1, 2, . . . , n
i=1 i
×
×
s
sss · · · sf
s
f
sss · · · ss
s
f
f ss · · · sf
s
f ss · · · ss
s
s
f
s
s
f
f
···
r
f
f
f
s
s
f
s
where r is the root. This example of a process tree features prominently below once we
consider Bernoulli processes and our main examples of parametric families of discrete random
variables in Chapter 4. Indeed, we shall take Ω to be the set of its paths and β the canonical
σ-algebra to define a so-called measure space. Each of the parametric families we have in
mind will be defined with respect to this measure space.
48
In chapter 5, we shall generalize the definition of a process to that of a stochastic process
by replacing the process tree defined above by an rooted tree of finite height and determining
a measure space with respect to its set of paths. Once we have established our measure space,
defining a random vector on the same will give us a stochastic process. Of course, Markov
chains are special cases of this construction, therefore, an important motif in this subject
begins here with processes.
Returning to the multiplication rule to count the number of permutations of a set S of
size n. A permutation is our first example of an arrangement, which is a sequence of its
elements with a prescribed order and method of selecting elements for inclusion. In the case
of a permutation of S, a permutation of its elements is an ordered list of all of its elements
selected without replacement from S. As all of the elements are selected for inclusion in
the sequence, what distinguishes one such list from another is the order in which they are
selected, as duplicate selections are precluded by the prescription that replacement is not
allowed.
Proposition 16. Let S be a finite set, say |S| = n. Then the number of permutations of S
is n! = n · n − 1 · · · 2 · 1, where n! is read as n factorial.
To prove this proposition, let us proceed by the multiplication rule. To count the number
of permutations of a set S of size n, we want to exhibit a permutation as a performance of a
process, which according to the multiplication rule, is defined as a word with respect a Cartesian product of pairwise-disjoint sets up to indexing. Accordingly, impose an order on the set
S simply by labeling the finite number of its elements, say n, by natural numbers. This order
will determine the steps or indices of the process. To wit, write S = {a1 , a2 , . . . , an }. Next,
let us define a sequence of pairwise-disjoint sets up to indexing by the following recursion rule.
Define A1 = S, A2 = S(a1 ), . . . , Ai = S(a1 , a2 , . . . , ai−1 ), . . . , An = S(a1 , a2 , . . . , an−1 ) = an ,
where the notation S(a1 , a2 , . . . , ai−1 ) is defined to be S \ {a1 , a2 , . . . , ai−1 } to indicate the
preceding i − 1 elements of S where selected, without replacement. This labeling determines
the steps in our process. Thus, with notation as above, a permutation is a performance of
the process
S=
× A
n
i=1
i
By construction, |Ai | = n − i + 1, so that, by the multiplication rule, the number of permutations of the set S is
n! =
n
Y
n−i+1
i=1
as desired.
Interpreted combinatorially n! is the number of permutations of a set of size n ≥ 0. In
particular, as |∅| = 0 we have, vacuously, that 0!=1, for there is one arrangement of no
elements. In subsequent sections we shall apply the multiplication rule less formally for the
sake of clarity. Nevertheless, in what follows, the reader may reconstruct a more formal
argument in terms of the aforementioned definition of a process.
A second illustration of the multiplication rule, presented less formally, is to counting the
number of subsets of a set S of size n.
49
Proposition 17. Let S be a finite set, say |S| = n, then |2S | = 2n .
Indeed, the process of forming a subset is determined by deciding whether to include a
particular element. To simplify the demonstration, impart an order to the elements of S so
that one may visualize a subset as an n-tuple (a1 , a2 , . . . , an ) such that either ai is included
in this n-tuple or it is not. If it is not, then leave the i-th position blank. Hence, there are
n steps in this process, and two ways to perform each step. Therefore, by the multiplication
rule, we have that the number of subsets of S is 2n . Indeed, we have proven that |2S | = 2n ,
as the power set is the set of all subsets of S, as desired.
2.1 WORKED EXAMPLES
1. 2.1 Consider the experiment of randomly selecting a single integer from the set of
integers from 120 through 999.
(a) Count the number of integers from 120 through 999 divisible by 5.
(b) Compute the probability of selecting an integer from 120 through 999 divisible
by 5.
Solution (a) Observe that one may write every fifth number in this range as
divisible by 5, so 120=24 ·5, . . ., 995=199·5. Therefore, the number of integers from
120 through 999 divisible by 5 is equivalent to the number of integers from 24
through 199 divisible by 5. Therefore, by the lemma, 199-24 +1 = 176, is the number
of integers from 120 through 999 divisible by 5.
Solution (b) Observe that relative frequency interpretation of an event applies here,
as the likelihood of selecting any integer from 120 through 999 is the same. In
particular, it is the reciprocal of the number of these integers. Thus, by the relative
frequency interpretation, we have
176
880
since 999-120+1=880 is the number of integers from 120 through 999 by the lemma.
2. 2.1 Suppose a computer installation has four input/output units, say a, b, c, and d and
three central processing units, say x, y, and z.
(a) How many ways are there to pair an input/oupt unit with a central processing
unit?
(b) List three performances of this process.
50
Solution (a) To answer this question, we must determine the number of steps in this
process and the number of ways to perform each step, then, by the multiplication
rule, we take the product of these figures to find the total number of ways to pair
input/output units with central processing units. In more sophisticated terms, we
must determine the factors of the Cartesian product product corresponding to this
process. Namely, A1 = {a, b, c, d} and A2 = {x, y, z} so that the entire process is
represented by the Cartesian product S = A1 × A2 whose cardinality
|S| = |A1 | · |A2 | = 12 is the total number of ways to pair input/output units with
central processing units.
Solution (b) Proceeding by definition, which is to identify a performance of a
process with a tuple in the Cartesian product representing the same, we must simply
write down three 2-tuples in S. So, for example, (a, x), (a, y), and (a, z) would do as
examples of performances of the process. However, so would (b, x), (b, y), and (b, z).
3. 2.1 Consider the following nest loop which runs for i = 1 to i = 4 and for j = 1 to
j = 3, next j, next i.
(a) How many total iterations of this nested loop exist?
Solution (a) Perhaps this solution is best illustrated by a process tree, but
nevertheless, the outer loop corresponding to i is iterated four times and the inner
loop corresponding to j is iterated times, once each time the outer loop is iterated.
Therefore, by the multiplication rule, there are two steps: the first performed four
times, the second performed three times. Therefore, there are twelve total iterations
of the nested loop.
4. 2.1 A combination lock has integer labels around a wheel consisting of the integers
from 0 through 39, called its dial. A combination code consists of a 3-tuple of integers
selected from 0 through 39, in order, with repetition allowed.
(a) How many total combination codes exist for a combination lock?
Solution (a) We proceed by the multiplication rule to count the total number of
combination codes by first identifying that there are three steps in this process, one
for each selection of an integer from 0 through 39. The number of ways to perform
each step is equivalent to the number of integers from 0 through 39, according to the
fact that a performances amounts to selecting an integer. There are 39-0+1=40 ways
to do this. Therefore, by the multiplication rule, there are 403 total combination
codes.
51
5. 2.1 Let us define [m]⋆ = {1, 2, . . . , m} to be the discrete set of integers from 0 through m
for m ∈ N. Consider an n-tuple comprised of elements selected from this set, chosen for
inclusion with replacement in the sense that one is allowed to select the same element
repetitively.
(a) Count the number of n-tuples as described above such that no adjacent entries
are equal.
Solution (a) Let us proceed by the multiplication rule. First, we are observe there
are n steps in this process, one for each selection of elements for inclusion in the
tuple. The first step may be performed in m ways, as there is no entry adjacent to it
ab initio which must be avoided. However, each subsequent step demands we select
an element different from the adjacent entry, bearing in mind that replacement of
selection is allowed. Therefore, there is (m − 1) ways to perform the remaining n − 1
steps.
rule, we have that there are
Qn As such, by the multiplication
n−1
such tuples.
m i=2 (m − 1) = m(m − 1)
2.2
The Definition of an Arrangement and r-Orderings
In this section we introduce the first of four arrangements of a finite set S. To this end,
let us first define what an arrangement is. We remark that below by an ordered set S we
mean there exists a bijection o : S → [n]⋆ such that s < s′ if and only if o(s) < o(s′ ), where
[n]⋆ is given the standard order. We say S is unordered if this is not the case.
Definition 25. Let S be a finite set, then an arrangement is a sequence of its elements such
that
1. elements in the sequence are either ordered or unordered
2. elements in the sequence are selected from S either with replacement after selection or
without replacement after selection for inclusion in the sequence
Plainly the process of forming arrangements of S can be done in four ways since in the
first step we must choose whether the sequence is ordered or unordered and in the second step
we choose whether or not to replace elements selected for inclusion. This observation shall
prefigure the remainder of this chapter, as we characterize each of these four arrangements
and consider topics related to them. The first of these four arrangements that we shall
consider is called an r-ordering.
Definition 26. Let S be a finite set, say |S| = n, and r ∈ N. We say an r-ordering of S is
an ordered arrangement of r of its elements with replacement after selection for inclusion.
Immediately one should notice that the integers r and n are independent of each other
because the selection of elements for inclusion in an r-ordering from S is inexhaustible;
that is, one selects elements for inclusion with replacement. Next we count the number of
r-orderings of a finite set S. We proceed by the multiplication rule.
52
Theorem 9. Let S be a finite set, say |S| = n, and r ∈ N. The number of r-orderings of S
is nr .
The proof is a straightforward application of the multiplication rule once we identify the
process of forming an r-ordering of S. Indeed, a step in this process corresponds to the
selection of an element. Therefore, there are r steps in this process. Further, there is n ways
to perform each step by the hypothesis that we are selecting elements for inclusion with
replacement of the same. Therefore, the theorem follows by simplifying the r-fold product
of n with itself obtained from the multiplication rule.
2.3 WORKED EXAMPLES
1. 2.2 An easy example is to count the number of 6-orderings of the set S = {a, b, c, d, e}.
(a) Count the number of 6-orderings of the set S = {a, b, c, d, e}.
Solution (a) By the theorem, since n = 5, we have there are 56 = 15625.
2. 2.2 Let AD be the set of Roman alphabet letters and the numerals 0 through 9. An
automobile license plate is a 7-ordering of the set AD.
(a) Count the number of license plates.
Solution (a) Again, we proceed by the theorem to obtain that there are 367 license
plates, since there are |AD| = 36 elements in AD.
3. 2.2 Let f : S → T be a set function and |S| = n, |T | = m. One writes M or(S, T ) for
this set of functions.
(a) Count the number of such functions f or compute |M or(S, T )|.
Solution (a) Our goal may be accomplished by recognizing that each such function
f is an n-ordering of the m elements of T . Specifically, consider that the process of
forming a set function is by definition a well-defined rule that assigns each element of
S uniquely to an element of T . As such, a set function is that of a list n positions
long, one for each element of S meant to be assigned to an element of T . Note that
by listing the elements of S we have imposed an ordering on the elements T . Now as
f is arbitrary, we do not require that f is either an injection or a surjection, so we
can determine f by selecting from T which element to assign to it in S with
53
replacement allowed. Therefore, an arbitrary f is an n-ordering of the m elements of
T . Thus by the lecture there are mn such set functions or |M or(S, T )| = mn .
Notice that we placed no restrictions on these functions beside their definition, that
is, that they are well-defined. We shall use this argument below to show that the
number of bijections of a set with itself is n! and to thereby prove thereby that
n! ≤ nn , as the set of bijections if a subset of M or(S, S).
4. 2.2 The fingerprinting classification system consists of classifying each print of each
of the ten digits of one’s hands. These points of classification include the plain arch,
tented arch, radial loop, ulnar loop, plain whorl, accidental whorl, double loop whorl,
peacock’s whorl, composite whorl, and central packet whorl.
(a) Prove no two living people have the same finger prints.
Solution (a) Recall there are approximately 8 billion humans alive at the time of
this writing. Comparing this figure to the number of possible fingerprints identified
by this classification system, we see this figure is less than the latter. Indeed, each
point of classification is attributable to each of the ten digits. Accordingly, there are
10 steps in the process of identifying an individual’s finger prints, and 10 ways to
perform each step as there are 10 points of classification. Therefore, by the
multiplication rule, there are 1010 possible sets of fingerprints, which is more than the
total number of living humans.
2.3
r-Permutations
In this section we shall consider the following kind of arrangement, which is a generalization of the permutation of a finite set S discussed in section 2.2.
Definition 27. Let S be a finite set and n = |S|. We say an r-permutation of S is an
ordered arrangement of r of its elements without replacement after selection for inclusion.
This arrangement is the second of the four arrangements that structure this chapter.
Notice, as the second in our sequence, the alteration in hypotheses implies that r ≤ n, for if
r = n the elements available for selection in the arrangement have been exhausted.
Counting the number of permutations of a finite set S generalizes the argument in section
2.2. Indeed, recall the notation from that section and define A1 = S, A2 = S(a1 ), . . . , Ar =
S(a1 , a2 , . . . , ar−1 ), where the notation Ai = S(a1 , a2 , . . . , ai−1 ) is defined to be S\{a1 , a2 , . . . , ai−1 }
to indicate the preceding i − 1 elements of S where selected, without replacement. Then
with notation as above, an r-permutation is a word with respect to Cartesian product
× A
n
i=r
i
By construction, |Ai | = n − i + 1, so that, by the multiplication rule, the number of rpermutations of the set S is
54
n · (n − 1) · · · (n − r + 1)
since there are (n − r + 1) integers from r through n.
While this formula is the correct conclusion from the multiplication rule, it is not convenient for applications or memorization, for that matter. To that end, we simplify it further
as follows:
n · (n − 1) · · · (n − r + 1)
n!
(n − r)!
=
(n − r)!
(n − r)!
since n · (n − 1) · · · (n − r + 1) · (n − r)! = n! by definition. We have now proven the following
theorem.
Theorem 10. Let S be a finite set, |S| = n, and a non-negative integer r ≤ n. The number
of r-permutations of S is
nP r =
n!
(n − r)!
2.4 WORKED EXAMPLES
1. 2.4 Count the number of 3 permutations of the set S = {a, b, c, d, e}.
5!
(a) Solution As |S| = 5 we have, by the theorem, that there are
= 5 · 4 · 3 = 60.
2!
Moreover, suppose that one of the letters of such a 5-permutation of S must be
b, then how many such 5-permutations are there of S? As one expects, since one
of the positions in the arrangement is fixed, such arrangements are tantamount
to 4-permutations of the set S ′ = {a, c, d, e} so that, by the theorem, there are
4!
= 12.
2!
2. 2.4 Show
n!
n!
+
= n2
(n − 2)! (n − 1)!
(a) Solution We merely simplify, observe:
n!
n!
n!(n − 1) + n!
+
=
(n − 2)! (n − 1)!
(n − 1)!
n! · n
=
(n − 1)!
= n2
3. 2.4 Consider the number of different ways in which five ranked prize winners may be
chosen from a group of one hundred people. Count the number of such lists of recipients.
100!
Accordingly, there are
= 9, 034, 502, 400. Hopefully, the great magnitude of this
95!
answer convinces the reader that sometimes such large computations are better left in
their original notation.
55
(a) Solution Such a list of recipients is ordered by the ranking. Of course, prizes
are awarded uniquely, so therefore such a list is a 5-permuation of the 100
100!
= 9, 034, 502, 400.
people.Accordinly, there are
95!
4. 2.4 Let f : S → S be a bijection, where |S| = n. One writes Sn for the set of such
functions.
(a) Count the number of such bijections of S onto itself or |Sn |.
(b) Argue that n! ≤ nn
Solution (a) Our objective is to count the number of such f as r-permutations.
More specifically, consider the process of forming a set bijection. We have already
argued above that an arbitrary function may be conceived of as an ordered list with
as many entries as the cardinality of the domain. Next, since a set bijection is both
injective and surjective, we must select elements from S to assign to elements in S to
without replacement for selection and exhaustively. Thus a bijection of S onto itself
is a permutation of the elements of S or |Sn | = n!.
Solution (b) With notation as above, we can embed Sn in M or(S, S) in the
straightforward manner by identifying an element of Sn with itself as a subset of
M or(S, S). This embedding is injective but not, in general, surjective. Therefore, by
chapter 1, we have |Sn | ≤ |M or(S, S)| or n! ≤ nn , as desired.
5. 2.4 Suppose a mall has six stores and that each store assigns one employee to the
Supreme Mall Committee. Suppose further that from these six individuals, three are
chosen to individually occupy one of three ranked positions within the Supreme Mall
Committee.
(a) How many ways are there to fill these ranked positions within the Supreme Mall
Committee?
Solution (a) One recognizes that such triples are 3-permutations of the set of six
individuals, for the positions are assigned to only one individual at a time and they
6!
= 6 · 5 · 4 = 120.
are ranked. Therefore, there are 6P 3 =
3!
6. 2.4 Suppose 5 people are chosen from a group of 100 prize candidates and each are to
receive one of five ranked prizes.
(a) How many ways can the recipients be selected from the 100 candidates?
56
Solution (a) As the prizes are distinct from each other owing to their different
names, we may regard the set of individuals awarded prizes as an ordered set. As
such, an arrangement of these five individuals is an ordered sequence without
repetition of selection or is a 5-permutation of the set of 100 potential recipients.
100!
= 9, 034, 502, 400 such lists of prize recipients.
Accordingly, there are 100P 5 =
95!
2.4
r-Combinations
In this section we both introduce and count the number of so-called r-combinations of a
finite set. This particular arrangement is important, for it underlies the binomial theorem
and is also used to compute its several variable generalization. Both theorems have a role
to play in constructing parametric families of discrete random variables, but the greatest is
perhaps played by the binomial theorem. Indeed, our proof in chapter 5 that the binomial
random variable satisfies the induced Kolmogorov axioms depends upon its formula. Given
the major role of the binomial random variable in modern mathematics, the importance of
combinations cannot be overstated. Therefore let us consider the following definition.
Definition 28. Let S be a finite set and n = |S|. We say an r-combination of S is an
unordered arrangement of r of its elements without replacement after selection for inclusion.
The process of forming an r-combination of S consists indirectly of but two steps. We shall
obtain the number of r-combinations by recounting the number of r-permutations. Observe,
first we must choose r elements of S without replacement and in no order. Let nCr denote
the number of these arrangements. Next, let us order the r elements we arranged in the first
step. As we know, there are r! ways to put the r elements in order, or such an ordering is
a permutation of the r elements we have arranged. Accordingly, by the multiplication rule,
we have that
nCr · r! = nP r
but then nCr is precisely the figure we meant to count. This gives the following theorem.
Theorem 11. Let S be a finite set, n = |S|, and r a non-negative integer r ≤ n. The
number of r-combinations of S is
nCr =
n!
r!(n − r)!
Of course the theorem follows from solving for nCr in our application of the multiplication
rule to the task of counting the number of unordered arrangements of S without replacement
of selection. Another famous notation for nCr is nr and both symbols are pronounced as
"n choose r." We shall reserve the latter notation for the binomial theorem, below.
2.5 WORKED EXAMPLES
1. 2.4 Enumerate the following
57
(a) The number of ten person committees of United States senators.
(b) Consider forming a team of five from a group of twelve colleagues. Suppose two
members of this group insist on being together on any five person team for which
either is selected. How many five person teams can be formed from this group?
(c) Suppose a similar group of twelve people is partitioned according the color of
their shirts; five people are wearing blue shirts and seven people are wearing
pink shirts. How many five person teams can be formed from this group that
contain at least one blue shirt?
Solution (a) A committee would be an example of 10-combinations of the United
States senate, which consists of 100 members. Accordingly, there are
100!
= 17, 310, 309, 456, 440. Perhaps this figure sheds some light on the
100C10 =
10!90!
reason why politics rather than arbitrary selection are required to narrow down the
list of possible committee assignments!
Solution (b) We may partition the set of five person teams by those that include
the pair of colleagues and those teams which do not. The number of the former five
person teams is a 3-combination of 10 of the colleagues, for the pair is not included.
The number of the latter five person is a 5-combination of the 10 colleagues, because
the pair is not included. As such, there are
10!
10!
+
= 120 + 225 = 372 five person teams formed from the
10C3 + 10C5 =
3!7! 5!5!
twelve colleagues.
Solution (c) To solve this problem, we can proceed by the difference rule. Let us
write S for the set of all five person teams formed from this group, and A for the set
of five person teams with no members wearing a blue shirt. Then, logically speaking,
the set of teams with at least one blue shirt is characterized by the complement of A
in S, namely, by the set S \ A. Moreover, tautologically, A and its complement in S
partition S, we have, by the difference rule, that |S \ A| = |S| − |A|. Now as the
cardinality of the set of fiver person teams without prescription are the
5-combinations of the twelve person group, we have |S| = 12C5. Further, as the
number of five person teams with no blue shirts is the number of 5-combinations of
remaining seven people, so |A| = 7C5. Therefore |S \ A| = 12C5 − 7C5 = 771.
2. 2.4 Let us consider counting various hands in the game of poker. In particular, proceeding by the relative frequency interpretation of probability, we shall compute the
probabilities of the hands we present in our examples. Recall that the game of poker is a
card game that consists of a deck of 52 playing cards, partitioned into four suites, called
hearts, diamonds, clubs, and spades, whose cards are labeled 2, . . . , 10 and J, Q, K, A.
A hand is a subset of size five cards taken from the deck. In other words, a hand is
a 5-combination of the deck. Hence, the total number of hands in a game of poker is
52C5-we will require this fact in a moment.
58
(a) Given our description of the game, let us both count the number of two pair
hands, defined to be hands that consist of two pairs, where a pair of cards is
defined to be two cards whose labeling matches, and a fifth card whose label is
different from that of either of the pairs. Let us proceed by the multiplication
rule. One should notice there are several distinct ways to apply it to this
problem, but each application arrives at the same result, so by the fundamental
theorem of counting they are equivalent.
Solution (a) We will divide this process into five steps as follows. First, let us
choose the labels of both pairs. As there are thirteen labels, there are 13C2 ways to
perform this step. Second, choose the suites of the smaller pair-traditionally the order
of the labels is the one indicated above. Since suits partition the deck, and there are
two cards in a pair, we can perform this step in 4C2 ways. Third, choose the suites
for the larger pair. Again, there is 4C2 ways to perform this step. Fourth, choose the
label of the fifth card. Since it must be distinct from that of the paired labels, there
are 11C1 ways to perform this step. Last, choose the fifth card’s suite. Clearly there
is 4C1 ways to perform this step. Therefore, by the multiplication rule, there are
13C2 · 4C22 · 11C1 · 4C1 = 123, 552 two pairs hands in poker. Moreover, the
probability is obtained by the relative frequency of this event, so that probability of
123, 552
or approximately
the event "a two pair hand" is obtained by the quotient
52C5
4.7 %.
3. 2.4 Out of five mathematicians and seven physicists, a committee consisting of two
mathematicians and three physicists is to be formed.
(a) How many committees may be formed?
(b) How many committees may be formed if one physicist is selected ahead of time?
(c) How many committees may be formed if two mathematicians are selected ahead
of time?
We proceed by recognizing that a committee, no matter additional restrictions are
imposed upon its formation, is a 5-combinations of the 12 scientists. Accordingly, we
can count the number of committees in each problem by finding the correct parameters
for nCr.
Solution (a) Observe that we must proceed by the multiplication rule, for there are
two steps to the process of forming a committee. The first step consists of, say,
selecting mathematicians for membership, and the second, the selection of physicists.
7!
5!
Accordingly, there are 5C2 · 7C3 =
·
= 10 · 35 = 350.
2!3! 3!4!
59
Solution (b) Similarly, we have, by the multiplication rule,
5C2 · 6C2 = 10 · 15 = 150.
Solution (c) Similarly, we have, the by multiplication rule, 3 · 7C3 = 3 · 35 = 105.
4. 2.4 How many different salads can be made from the greens lettuce, escarole, endive,
watercress, and chicory?
Solution One can partition the set of salads containing at least one of the five greens
into cells that determined by the number of greens contained in the salad, e.g. those
with only one green, or those salads with only two greens, . . ., or those salads with
only five greens. Accordingly, we have by the addition rule that total number of
salads comprised of these five greens is 5C1 + 5C2 + · · · + 5C5 = 31.
Alternatively, one should remark that there is another way to count the number of
such salads without appealing to partitioning the set of such salads into various
combinations of the greens. Indeed, in this second version, one decomposes the
process of creating a salad into five steps, and each step is comprised of deciding
whether to include or not to include a particular green. Therefore by the
multiplication rule there are 25 − 1 = 31 such salads. Notice we must substract one
from the product obtained by the multiplication rule as it includes the "trivial salad“
comprised of no greens. We only mention this because, by the fundamental theorem
of counting, we have implied in this example that 5C1 + 5C2 + · · · + 5C5 = 25 − 1.
Proving this identity is the content of the next example.
5. 2.4 Prove the identity
n
X
nCr = 2n − 1
r=1
Solution Let S be a finite set such that |S| = n. Then we know that |2S | = 2n and,
in particular, that 2n − 1 is the number of subsets of S excluding ∅. With this
notation at hand, let us partition the set of subsets of S containing n ≥ r ≥ 1
elements and denote the cells by Cr . Again, we know |Cr | = nCr by definition and
the proof such Cr partitionP
the set of non-empty subsets of S is trivial. Therefore, by
the addition rule, we have nr=1 nCr = 2n − 1, as desired.
6. 2.4 Consider the game of Poker and the definitions of various hands within the game
in the following questions. In each question, count the total number of such hands in
the deck. Furthermore, compute the probability of each hand by the relative frequency
definition of probability.
(a) By definition, a straight hand consists of five cards in order, where the ordering
of the cards is imparted by the order of their denomination.
60
(b) By definition, a pair hand consists of two cards with matching denominations
and three arbitrary cards of distinct denominations, both from that of the pair
and of each other.
(c) By definition, a three of kind hand consists of three cards with matching
denominations and two arbitrary cards of distinct denominations, both from
that of the three of a kind and of each other.
Solution (a) By inspection, there are only ten sequences of denominations that
produce straight hands, assuming one is allowed to regard A or the ace card as the
number one in the ordering imparted by the denomination. To see this for oneself,
consider counting up from 1 (or A) by five. We regard this argument by inspection as
the first step in the process of forming a straight hand. Naturally, the second step
consists of determining the suits of the cards included in the hand. These are
5-orderings of the set of suits, so there are 45 ways to perform the second step of this
process. Therefore, by the multiplication rule, there are 10 · 45 = 10240 straight
hands in a game of Poker. Moreover, by the relative frequency interpretation of
probability, the chance of drawing one from the deck is
P (straight) =
10240
= 0.003
2598960
or, roughly speaking, a 0.3% chance.
Solution (b) Let us decompose the process of forming a pair hand into the following
steps. First step is to choose the four denominations in the hand. Second, choose the
suits of the two cards of matching denominations. Third, choose suits for the
remaining two cards. Fourth step is to choose which of the four denominations in the
hand to be duplicated. Therefore, by the multiplication rule, we have
13C4 · 4C2 · 43 · 4C1 = 1098240
Moreover, by the relative frequency interpretation of probability, the chance of
drawing one from the deck is
P (Pair) =
1098240
= 0.422
2598960
or, roughly speaking, a 42.2 %
Solution (c) Let us decompose the process of forming a three of a kind hand into the
following steps. First step is to choose the three denominations in the hand. Second,
choose the suits of the three cards of matching denominations. Choose suits for the
remaining two cards. Fourth step is to choose which of the three denominations in
the hand to be triplicated. Therefore, by the multiplication rule, we have
61
13C3 · 4C3 · 42 · 3C1 = 54912
Moreover, by the relative frequency interpretation of probability, the chance of
drawing one from the deck is
P (3ofKind) =
54912
= 0.021
2598960
or, roughly speaking, a 2.1 %
2.5
The Binomial Theorem and the Principal of Inclusion and Exclusion
In this section we state and prove the famous binomial theorem. As mentioned above,
this formula underlies the proof that the probability space induced by the binomial random
variable satisfies Kolmogorov’s axioms. This result alone is enough in the author’s opinion
to justify its inclusion. Beside its renown, one of the proofs shall exhibit the technique of
proof by induction, which is a topic of mathematics worthy of its own study.
In conjunction with this technique, our first proof shall utilize the combinatorial interpretation of the number of r-combinations of a finite set. Let us consider a set S of size
n together with a labeling of its elements, say S = {a1 , a2 , . . . , an }. Plainly an arbitrary
subset Ai = {ai,1 , ai,2 , . . . , ai,r } ⊂ S is an r-combination of the elements of S, as one selects
elements without replacement for inclusion, and there is no particular order in which they
must be arranged. Therefore the combinatorial interpretation of nCr is that it counts the
number of subsets of S of size r.
As an illustration of this perspective’s application to the binomial theorem, we shall prove
the symmetry of the binomial coefficient with respect to its lower argument. Namely, in the
context of counting problems germane to the number of subsets of various sizes, we write
n
r
instead of nCr for emphasis.
For a set S of size n, let Ci be the set of subsets of S of size i.
Lemma 2.1. With notation as above, |Cr | = |Cn−r |. In particular,
n
n
=
r
n−r
As usual, to prove two sets have the same cardinality we shall exhibit the existence of
a bijection between the same. To this end, define f : Cr → Cn−r by f (Ai ) = S \ Ai . The
map is both injective and surjective by the law of the excluded middle-namely, an element
is either itself
or itn is
not and never both-together with the definition of set complement.
Therfore, nr = n−r
, as desired.
Next, we prove Pascal’s identity, which is the result that validates the inductive step in
the proof of the binomial theorem.
62
Theorem 12. Let S be a finite set, n = |S|, and r a non-negative integer r ≤ n. Then
n
n−1
n−1
=
+
r
r
r−1
To prove this identity, we shall adopt the notation of the previous paragraphs in this
section. Let ai ∈ S and Cr \ {ai } for the set of subsets of size r that do not contain ai and
Cr (ai ) for the set of subsets of size r that do contain ai . Plainly these sets partition Cr , so
by the addition rule, we have
|Cr | = |Cr \ {ai }| + |Cr (ai )|
Observe that Cr \ {ai } consists
of r-combinations of a set of size n − 1 for ai is excldued.
n−1
Therefore, |Cr \ {ai }| is r . Similarly, Cr (ai ) consists of r − 1 combinations of a set of
size n − 1 as ai must be included. Therefore, |Cr (ai )| is n−1
. Thus, by the addition rule,
r−1
we have
n
n−1
n−1
=
+
r
r
r−1
as desired.
We are now in a position to prove the binomial theorem. We shall prove it twice: once by
induction and a second time combinatorially. The first proof requires establishing the basis
for induction, which we hope sheds light upon the combinatorial argument. The induction
step requires Pascal’s identity, which is why we began with it.
Before we proceed to prove the theorem, let us recall the technique of proof by induction.
Let π(n) be any logical statement that depends on an integer n ≥ n0 , where n0 is some
fixed integer. The π(n) is true for all such n provided both π(n0 ) and if π(n) is true, then
π(n + 1) for all n ≥ n0 . The statement π(n0 ) is often referred to as the basis for induction
and π(n) the induction hypothesis, for one is allowed to assert this statement in order to
demonstrate π(n + 1). There are several equivalent formulations of this technique in the
literature, but this one shall suffice for our purposes. Let us now exhibit this technique to
prove the following theorem.
Theorem 13. Let n ∈ N, then for any binomial (x + y), we have
n X
n n−k k
(x + y)n =
x y
k
k=0
Let π(n) be the statement of the theorem. We shall establish the basis for induction by
choosing n0 = 4, although we could choose other more or less convenient values, since it is
63
true for any n ≥ 0 as above. So, consider the expansion
(x + y)4 =(x + y)(x + y)(x + y)(x + y)
=x4 +
x3 y + x2 yx + xyx2 + yx3 +
x2 y 2 + x2 y 2 + xy 2 x + yx2 y + y 2 x2 + y 2 x2 +
y 3 x + y 2 xy + yxy 2 + xy 3 +
y4
4 4
4 3
4 2 2
4
4 4
3
=
x +
x y+
xy +
xy +
y
0
1
2
3
4
4 X
4 4−k k
=
x y
k
k=0
Notice several lines above are apparently needless, but they shall help clarify our combinatorial proof below. Nonetheless, at present, let now assume that π(n) is true and show that
it implies π(n + 1). We shall use Pascal’s identity, as follows.
(x + y)n+1 =(x + y)n (x + y)
n X
n n−k k
=
x y (x + y)
k
k=0
n n X
n n−k+1 k X n n−k k+1
=
x
y +
x y
k
k
k=0
k=0
n n X
n n−k+1 k X
n
n+1
=x
+
x
y +
xn−k+1 y k + y n+1
k
k
−
1
k=1
k=1
n
X
n
n
=xn+1 +
+
xn+1−k y k + y n+1
k
k−1
k=1
n
X n+1
=xn+1 +
x(n+1)−k y k + y n+1
k
k=1
n+1
X (n + 1)
=
x(n+1)−k y k+1
k
k=0
which is indeed π(n + 1), as desired. Notice the penultimate line required Pascal’s identity.
The combinatorial proof is suggested by the basis for induction, as indicted earlier. Indeed, (x + y)n = (x + y) · · · (x + y) n-times. Each monomial collects one variable from each
binomial term in the product, therefore, each monomial is of total degree n. In particular,
it is of the form xn−k y k for some k ∈ {0, 1, . . . , n} representing the number of times y was
chosen. We see this process for determining monomials in the basis for induction when n = 4.
Continuing then, the numberof such monomials is equivalent to the number of subsets of n
variables of size k, namely nk . Accordingly, combining like terms gives
64
n X
n n−k k
(x + y) =
x y
k
k=0
n
as desired.
As an application of the binomial theorem, we both state and prove the formula for
computing the cardinality of a finite union of sets. We recall that it is germane to probability
theory insofar as one might regard the probability P (A) of an event A ∈ β in a discrete
probability space as a measurement of its size in Ω. Accordingly, P is a special case of the
cardinality of the set A, namely |A|, as a subset of a set S. Of course, in the case of probability
theory, the additional restriction rendering P (A) as a special case is that P (Ω) = 1. This
would be analogous to insisting that |S| = 1, which in general is untrue. In its general form,
the proof goes by the name of the Principle of Inclusion-Exclusion.
Theorem 14. Let S be a finite set, gi , i = 1, . . . , r be predicates of elements of S, and Ai
subsets of S such that gi is a predicate for the same index. Then
P
P
| ∪ri=1 Ai | = i |Ai | − i<j |Ai ∩ Aj | ± . . . (−1)r | ∩ri=1 Ai |
To show this, let x ∈ ∪ri=1 Ai and suppose that x ∈ ∪kj=1 Aij , that is,x is contained in k
of the subsets Ai . Our goal is the show that the right hand side of the formula only counts
x once as well. If this is the case, then the two sides of the formula are equal, for the right
hand side then counts each of the elements of the union only once, that is, it computes its
cardinality.
So, given x, it is contained in k sets in the first term of the right handside, and therefore,
counted k times by that term. One notices that k times is equal to k1 . The second term
adds over ordered pairs i < j, so that x is counted in this term k2 times, while as the third
term adds over triples i < j < l, so x is counted in that term k3 times, and so on until it is
counted over the ordered set [k]⋆ so it is counted there kk times. Thus, x is counted on the
right hand side
k
− k2 + k3 ± . . . (−1)k kk
1
Thus, to prove the principle of inclusion and exclusion, we must show that the left hand
side equals the right hand side, which in this argument means
1 = k1 − k2 + k3 ± . . . (−1)k kk
since x is counted once on the left hand side of the formula. We will accomplish this with
the binomial theorem. Observe,
0 =0k
=(1 − 1)k
k X
k k−i
=
1 (−1)i
i
i=0
k
k
k
k k
=1 −
−
+
± . . . (−1)
1
2
3
k
as desired.
65
2.6 WORKED EXAMPLES
1. 2.5 Let us recall from elementary calculus that, for a real-valued function y = f (x), its
f (x + h) − f (x)
dy
= lim
derivative is given by the limit
h→0
dx
h
(a) Use the Binomial theorem to prove the power rule, that is, for y = xn , its is true
dxn
that
= nxn−1
dx
Solution (a) Proceed by the binomial theorem and the definition of derivative.
Observe,
(x + h)n − xn
dxn
= lim
h→0
dx
h
n X
n n−k k
x h − xn
k
= lim k=0
h→0
h
n
n−1
x + nx h + · · · nxhn−1 + hn − xn
= lim
h→0
h
nxn−1 h + · · · nxhn−1 + hn
= lim
h→0
h
n−1
= lim nx
+ · · · nxhn−2 + hn−1
h→0
n−1
=nx
as desired.
2. 2.5 Let n ∈ N
n−1
(a) Prove n2
n
X
n
=
k
k
k=0
Solution (a) Proceed by the binomial theorem. In particular, write
n n
X
X
n k
n k−1
n
n−1
(1 + x) =
x . Differentiating, we have n(1 + x)
=
k
x .
k
k
k=0
k=0
Evaluating at x = 1 gives the identity.
3. 2.5 Let M, N, k ∈ N. The following identity is known simultaneously as Vandermonde’s
Sum and the Hypergeometric Identity. We shall return to it in chapter 5 to demonstrate
Kolmogorov’s axioms hold for the so-called Hypergeometric random variable.
66
X
k M +N
M
N
(a)
=
k
i
k−i
i=0
Solution (a) We shall provide a different, algebraic proof below in Chapter 5. Here
we shall attempt a combinatorial argument. Accordingly, consider the set of subsets
of size k of a set consisting of M identical letter s’s and N identical letter f ’s,
denoted b(M, N ). Accordingly, we have
M +N
|b(M, N )| =
k
by 2.4. Alternatively, we can partition b(M, N ) into k + 1 cells by the following
argument. Define bi (M, N ) to be the set of subsets of size k consisting of i letter s’s,
for i = 0, 1, . . . , k. Clearly, bi (M, N ) ∩ bj (M, N ) = ∅ for i ̸= j as it is impossible for a
subset to have both i letter s’s and j letter s’s for i ̸= j. Further, these cells cover
b(M, N ) by its definition, so b(M, N ) = ∪ki=0 bi (M, N ) and
|b(M, N )| =
k
X
|bi (M, N )|
i=0
by the addition rule. Next, notice there are Mi ways to select i s’s to be in elements
N
of bi (M, N ) and, equivalently, k−i
ways to select the remaining k − i f ’s to be in
N
elements of bi (M, N ). Therefore, by the multiplication rule, |bi (M, N )| = Mi k−i
.
Therefore, putting all of this together, we have
M +N
k
=
k X
M
N
i=0
i
k−i
as desired.
2.6
The Multinomial Theorem and Ordered Partitions
In this section, we mean to generalize the binomial theorem by considering a new type of
arrangement, predicated upon the previous arrangements, and count the number of those.
The generalization we seek is that of the multinomial theorem and the underlying arrangement is that of the following definition.
Definition 29. Let S be a finite set, n = |S|, and {Ai }ri=1 a partition of S of length r. We
say such a partition is of size ni if each cell |Ai | = ni . We say a permutation of S such that
the first n1 elements belong to A1 , the second n2 elements belong to A2 and so forth until the
last nr elements belong to Ar is an ordered partition of S.
67
An alternative, although perhaps more opaque, definition of an ordered partition of S
is a bijection of S with itself, where a label is added to indicate how an element of S is
assigned to a cell in the given partition of length r and size ni . However, as this chapter is
intended to be an introduction to combinatorics, we prefer to avoid such an advancement in
our concepts. Let us now count the number of ordered partitions of S.
The process of forming ordered partitions of S consists of r steps, one for each cell of the
partition of S. Moreover, assigning ni elements of S without replacement to a particular cell
imposes the following counting argument. In the first step, choose a subset of size n1 . There
are nn1 ways to perform this step. In the second step, choose a subset of size n2 from the
remaining n−n1 elements, for selection is made without replacement in a permutation. There
1
are n−n
ways to perform this step. And so on and so forth until the final or r-th step in
n2
this process. In this step, choose a subset of size nr from the remaining n−n1 −n2 −· · ·−nr−1
elements. Altogether, by the multiplication, there are
n−n1 n
r−1
· n2 · · · n−n1 −n2n−···−n
n1
r
ways to perform this process of forming ordered partitions of S. Simplifying this argument,
we have, in terms of the binomial coefficients,
(n − n1 )!
(n − n1 − n2 − · · · − nr−1 )!
n!
·
···
n1 !(n − n1 )! n2 !(n − n1 − n2 )!
nr !(n − n1 − n2 − · · · − nr−1 − nr )!
which cancels pairwise, denominator against subsequent numerator, except in the r-th factor.
There, in the denominator, we notice that, by the addition rule, n−n1 −n2 −· · ·−nr−1 −nr = 0
since n = n1 + n2 + · · · + nr by the hypothesis that {Ai } is a partition of length r and size
ni . Altogether, we have now prove the the following theorem.
Theorem 15. Let S be a finite set, n = |S|, and {Ai }ri=1 a partition of length r and size ni .
Then the number of ordered partitions of S is given by
n
n!
=
n1 ! · n2 ! · · · nr !
n1 , n2 , . . . , nr
68
We refer to the figure in the theorem, obtained by the simplification of our multiplication
rule argument, as the multinomial coefficient. The reason for this name is that the number
of ordered partitions determines the coefficients in the expansion of the n-th power of r-fold
multinomial.
Theorem 16. Consider the multinomial x1 + x2 + · · · + xr , then the expansion of its n-th
power is given by the formula
X
n
n
(x1 + x2 + · · · + xr ) =
xn1 1 · xn2 2 · · · xnr r
n
,
n
,
.
.
.
,
n
1
2
r
r
(n1 ,n2 ,...,nr )∈Z |n1 +n2 +···+nr =n
An interesting but trivial computation shows that, by setting xi = 1, we have
X
n
n
r =
n1 , n2 , . . . , nr
r
(n1 ,n2 ,...,nr )∈Z |n1 +n2 +···+nr =n
2.7 WORKED EXAMPLES
1. 2.6 Compute the values of the following multinomial coefficients.
12
(a)
4, 3, 5
25
(b)
7, 6, 5, 7
60
(c)
7, 10, 9, 8, 19, 7
Solution (a)
12!
= 27720
4!3!5!
Solution (b)
25!
= 7067582121600
7!6!5!7!
Solution (c)
60!
7!10!9!8!19!7!
2. Define a word to be a permutation of a set of symbols, regardless of whether particular
symbols are repeated for inclusion in the permutation.
(a) Count the number of words formed by the set whose elements are comprised of
the letters of MISSISSIPPI
(b) Count the number of words formed by the set whose elements are comprised of
the letters of MATHEMATICS
(c) Count the number of words formed by the set whose elements are comprised of
the letters of SASKATOONSEQECHUWAN
69
Solution (a) We treat a word as an ordered partition of the set of letters in the
word MISSISSIPPI. Indeed, we partition this set by repeated letters. So, for example,
we have A1 = {M }, A2 = {I, I, I, I}, A3 = {S, S, S, S}A4 = {P, P }. As such, the
number of words is determined by the
numberof ordered partitions of the set
11
S = {M, I, S, S, I, S, S, I, P, P, I} or
1, 4, 4, 2
11
Solution (b) mutatis mutandi
2, 2, 2, 1, 1, 1, 1
20
Solution (c) mutatis mutandi
3, 3, 1, 1, 2, 2, 2, 1, 1, 1, 1, 1
3. 2.6 Below we shall define Bernoulli processes as experiments consisting of a succession
of trials in which only one of two mutually exclusive possible outcomes, say success
and failure, may occur at each stage in the process. Moreover, as we shall see, in
a fixed number n of such trials, the number of successes x and n − x failures that
may occur is given by nx . More generally, here we consider Multinomial processess
consisting of a succession of n multi-trials in which s1 , s2 , . . . , sr mutually exclusive
possible outcomes may occur at eachP
stage in the process. Similarly, the number of
s1 ’s x1 , s2 ’s x2 , . . . ,sr ’s xr such that
xi = n may occur in the multinomial process
n
process is x1 ,x2 ,...,xr
(a) Consider an experiment consisting of 10 multi-trials which consist of three
possible outcomes. Over 10 multi-trials, what is the number of multi-trials with
5 outcomes favorable to the first type, 4 favorable to the second type, and 1
favorable to the first type?
Solution (a) Proceeding
by the discussion, we recognize the number of such
10
multi-trials as 5,4,1
.
2.7
r-Unorderings
The last type of arrangement we consider is a so-called r-unordering. These are arrangements that are perhaps initially the most counter-intuitive, but nevertheless, commonly occur
in recognizable counting problems and real-world experiments.
Definition 30. Let S be a finite set and n = |S|. We say an r-unordering of S is an
unordered arrangement of its elements with replacement after selection for inclusion.
70
The method of counting such arrangements is of interest in its own right, as the method
is a common and effective one of combinatorics. It is the so-called stars and bars argument.
We shall characterize an r-unordering as a combination of stars and bars and count the
number of those instead. Indeed, let S = {a1 , a2 , . . . , an } be the elements of S. We can
represent these elements by the compartments induced by n − 1 bars. To see this, remove
the ai notation from the sequence
a1 |a2 | · · · |an
The efficacy of this translation is that replacement after selection for inclusion is easy to
represent. To do this, we insert a star in the compartment corresponding to ai ∈ S each
time it is selected for inclusion. For example, an r-unordering where a1 is selected twice,
a2 is selected three times, etc. an−1 is selected once and an is selected not at all could be
represented by the following sequence of stars and bars, viz.
⋆ ⋆ | ⋆ ⋆ ⋆ |···| ⋆ |
Here we have informally established a bijection between the set of r-unorderings of S
and sequences of n − 1 stars and r bars, in no order, with replacement not allowed. Thus
we may count the number of the latter kind of sequences to count the former number of
arrangements. Observe that a sequence of n − 1 bars and r stars corresponding to an runordering of S is a r-combination of the set of n − 1 bars and r stars. In particular, as
there are r elements of S in an r-unodering, it is an r-combination of the set of n − 1 bars
and r stars. Therefore, as r-unorderings are in bijective correspondence with sequences of
n − 1 bars and r stars, we have proven the following theorem.
Theorem 17. Let S be a finite set, |S| = n, and r ∈ N, then the number of r-unorderings
of S is
n−1+r
r
Notice the formula for the number of such arrangements reflects the fact that r is independent of n, as replacement of selection for inclusion is allowed. Indeed, since the lower
index of the binomial coefficient depends on r alone, r may be an arbitrary non-negative
integer for n fixed. Let us now complete this chapter with some examples of r-unorderings.
2.8 WORKED EXAMPLES
1. 2.7 Consider the host of a party who wishes to set out fifteen assorted cans of soft
drinks for his guests. He stops at a store that sells five different brands of soft drink.
(a) How many different arrangements of fifteen assorted cans of soft drink selected
from the five brands may the party host set out?
(b) How many different arrangements of the fifteen assorted cans of soft selected from
the five brands drink may the party host set out if 6 must be selected from the
root beer brand?
71
solution (a) The solution to this question is obtained by recognizing that an
assortment of soft drinks is an unordering of the available brands at the store.
Indeed, the host is considering 15-orderings of the 5 brands because an arrangement
is comprised of soft drinks selected with repetition
allowed in no order from the five
5−1+15
available brands. As such, there are
= 3, 876 such assortments for the host
15
to select.
solution (b) Suppose next that the same host still wishes to present an assortment
of fifteen soft drinks, but six cans must be of the root beer brand-how many such
assortments are possible, now? What changes is the number of cans we are selecting
for our assortment, since 6 cans are now chosen ab initio. So, there are 5−1+9
= 715
9
such assortments in this second case.
2. 2.7 Consider counting non-increasing sequences of integers. Let n be a non-negative
integer and consider triples of integers (i, j, k) such that 1 ≤ i ≤ j ≤ k ≤ n, that is,
non-increasing sequences. Count the number of such non-increasing sequences.
Solution To do this, we must recognize how the above non-increasing sequences are
examples of unorderings. Represent such a triple a bars and stars representation of
an r-unordering, with the n integers determined the n − 1 bars and the stars entered
in a compartment they determine as the selection of the integer i, j or k in the
compartment corresponding to the same. The hypothesis the triple is non-increasing
is tantamount to selection of these integers with replacement.
such,
the
As
n−1+3
n+2
parameters we need are n = n and r = 3, to obtain
= 3 such
3
non-inreasing sequences.
3. 2.7 Count the number of integral solutions to linear equation
x1 + x2 + x3 + x4 = 10
Solution One observes that an integral solution is an unorderings, as the variables
determine the bars and the number of ones whose sum is its integral value determine
the number of stars placed in each compartment. Hence we shall proceed to count
the number of equivalent combinations of stars and bars.
Recall each non-negative integer n is an n-fold sum of the unit 1. We may therefore
represent a integral solution to the given equation as a combination of ten stars and
three bars. Each compartment demarcated by the three bars represents a variable in
the equation. A star in each compartment represents an addition of 1 to obtain the
integer substituted for the corresponding variable. For example, ⋆ ⋆ | ⋆ ⋆ ⋆ | ⋆ ⋆ ⋆ | ⋆ ⋆
72
represents the solution x1 = 2, x2 = 3, x3 = 3, x4 = 2. As such, the number of integral
solutions to the given equation is equal to the number of 10-unorderings of 4
variables, which there are 4−1+10
= 286 in total. Furthermore, supposing we
10
insist
4−1+6
one only consider positive integral solutions, or xi ≥ 1, then there are
= 84,
6
as four ones are immediately placed in each of the compartments following the
previous argument.
4. 2.7 We define a lattice path from (0, 0) to (n, r) for n, r ∈ N to be a union of line
segments joining elements in the integer lattice contained in the first quadrant of R2
that connect (0, 0) to (n, r).
n+r
(a) Show that the number of lattice paths joining from (0, 0) to (n, r) is
r
(b) Compute the number of lattice paths from (−1, 2) to (7, 10).
Solution (a) By definition a lattice path resides in the first quadrant, thus a lattice
path is equivalent to n + 1 moves to the right, or the number of integers from 0
through n, and r moves upward. Accordingly, the set of paths are equivalent to the
number of
of the n + 1 moves to the right. Thus, there are
r-unorderings
n+1−1+r
n+r
=
lattice
paths from (0, 0) to (n, r), as desired.
r
r
Solution (b) Translating the problem to the first quadrant by mapping (−1, 2) onto
the origin, we recognize that we can solve the problem by counting the number of
lattice pathsfrom (0, 0) to (8, 8). According to the previous exercise, there are
therefore 16
= 12870 lattice paths.
8
5. 2.7 In how many ways can 14 chocolate bars be distributed among five children?
Solution Observe that a distribution of 14 chocolate bars to 5 children is a
14-unordering of the 5 children if we conceive of the children as the compartments
into which
we
place chocolate bars. Therefore, we take n = 5 and r = 14 to obtain
5−1+14
18
= 14 = 3060 ways to distribute the 14 chocolate bars to the 5 children.
14
2.8
Chapter 2 Homework Exercises
1. equivalence class of permutations of the elements in S, where two permutations are
equivalent if they produce the same ordered list.
remark: I am bothering to define a "word" as a synonym for "permutation" in our
notes to help clarify some of the verbiage, or way things are written, in Gordon’s text
73
book. A point of emphasis so far in our lectures has been to identity specific arrangements of elements of a set, so qualifications like whether they are ordered or repetition
is allowed are important pieces of information to help solve problems. I feel Gordon’s
text does not adequately convey these qualifications, therefore the definition is meant
to help read both the textbook and books alike it. The phrase "equivalence class"
is meant to explain how to identify two permutations which produce the same list.
If there are redundant elements in S, then the strict definition of permutation would
distinguish ordered lists of elements if the same element with different subscripts was
ordered differently in the list. However, by taking equivalence classes, we identify such
distinct lists qua permutations in the strict sense. In this manner, one is not computing simply |S|! to answer these questions. Instead, one needs to use our work that
computes the number of allocations of a set S to a partition, or more numerically, the
so-called multinomial coefficient formula.
a.) How many words on the set S = {F,L,U,F,F} exist?
b.) How many words on the set S = {R,O,T,O,R} exist if T is in the middle?
2. How many numbers can be made each using all the numerals in the set S = {1, 2, 2, 3, 3, 3, 0}?
3. Five persons, A,B,C,D, and E, are going to speak at a meeting.
a.) In how many orders can they take their turns speaking if B must speak (sometime)
after A?
b.) In how many orders can they take their turns speaking if B must speak immediately
after A?
74
4. At a table in a restaurant, six people ordered roast beef, three ordered turkey, two
ordered pork chops, and one ordered flounder fish. Of course, no two portions of any
of these items are absolutely identical. The 12 servings are brought from the kitchen.
In how many ways can they be distributed so that everyone gets their correct order?
5. a.) In how many ways can eight people sit at a lunch counter with eight stools?
b.) In how many ways can four couples sit at the lunch counter if each sits next to one
another?
c.) In how many ways can eight people sit at a round table?
remark: The idea of a lunch counter is that it is a single row of stools against a
counter. So the stools are arranged in a straight line. The intermediate stools have
a neighbor on both the left and the right. The stools at the ends have either no left
neighbor or no right neighbor.
6. How many non-negative integer solutions are there to the equation
x1 + x2 + x3 + x4 = 30
7. A camera shop stocks eight different types of batteries.
a.) How many ways can a total inventory of 30 batteries be distributed among the
eight different types of batteries?
b.) Assuming that one of the types of batteries is A76, how many ways can a total
inventory of 30 batteries be distributed among the eight different types if the inventory
must include at least four A76 batteries?
n
n−r n
by computing the symbol on the left hand side to
8. Prove that
=
r+1 r
r+1
be equal to the symbol on the right hand side of the equality.
n
HINT: Compute the left hand side until you get the right hand side with n−r
instead,
then use the lecture notes on the binomial theorem section to finish the problem. At
least that’s how I did it.
9. a.) Completely simplify
10
X
10 i
2
i
i=0
b.) Expand (1 − x)6 and simplify.
c.) Expand (x + x−1 )5 and simplify.
75
3
Discrete Random Variables
In this chapter we introduce discrete random variables in the generality of probability
theory, which is to say, we introduce their general features and postpone the important
classes of examples until the next chapter. A random variable is a powerful computational
tool inasmuch as it is a formalization of how one may characterize the likelihood of events in
an experiment with numerical parameters. Indeed, given such a characterization one is able
to compute the probability of events that are described parametrically with the numerical
parameters. This technique is so effective it is often suppressed in introductory courses on
this subject by teaching the topic as though it were self evident. Yet, in this chapter, we
explore this topic at the level of rigor appropriate to an undergraduate course.
Beside the convenience of computing probabilities by finding parameter values discerned
within an experiment, one could argue that the theoretic justification for the introduction
of random variables is that they are implements whereby experiments with sufficiently large
sample spaces are reduced to computationally smaller experiments, associated to the original
experiment in a canonical way. Indeed, by recognizing subsets or events of such an experiment
as characterized by one or more parameters, one may define a random variable or vector
whose image in the real numbers are these parameter values. Consequently, the entire
subset or event is translated into its image under this variable. Moreover, the probability of
the same is the probability of the image, which in turn, is the probability of the fibre. As we
learned in chapter 1, set functions fibre their domains, thus the random variable or vector
captures entirely the sufficiently larger original experiment in terms of its comparatively
smaller image. Understanding the relationship between fibering and images is the topic of
the first section, below. Once this has been established, we shall introduce both random
variables and the canonical spaces associated to them, then explore the implications of this
experimental addition.
3.1
Partitions qua Equivalence Relations and Quotient Sets
This section is intended to help explain what one means by the probability of the value
of a random variable, or plainly speaking, a real number, by explaining the relationship
between the fibering of the domain of a set function and its image. Those readers familiar
with abstract algebra might compare this result to the first isomorphism theorem. We begin
this discussion by generalizing the notion of equality of set elements, as this shall introduce
the relevant concept of equivalence class.
Definition 31. Let S and T be sets, then we say a relation R between S and T is any subset
of their Cartesian product. To wit, R ⊂ S × T is a relation. Furthermore, we say s ∈ S is
related by the relation R to t ∈ T and vice versa if (s, t) ∈ R. We denote the ordered pairs
of a relation R by sRt. In the special case when S = T we simply say that R is a relation
on S.
Upon consideration this definition may appear to be a mere triviality, for it seems to only
impart a name to what is otherwise an ordinary subset of a Cartesian product. However, the
sense in which it specifies a relation between components of an ordered pair is by specifying
a membership condition and so is uesful after we add more features to R.
76
Consider for example what is perhaps the most famous relation of all, the set of rational
m
such that n ̸=
numbers, Q. Indeed, by definition, this set is defined to be ratios of integers
n
∗
∗
0. Equivalently, (m, n) ∈ R = Z × Z , where Z = Z \ 0. One notices immediately that this
description of rational numbers as a relation within the aforementioned Cartesian product is
incomplete, for it neglects the equivalence fractions. Indeed, two fractions (m, n) and (s, t)
are equivalent if (mt − sn) = 0. This condition is therefore the additional requirement for
membership in R defined above. As this requirement satisfies several properties we wish to
abstract, it motivates the next definition.
Definition 32. We say a relation R on a set S is an equivalence relation if it satisfies the
following three axioms. Let s1 , s2 , s3 ∈ S, then R must satisfy
1. R is reflexive, viz. s1 Rs1
2. R is symmetric, viz. s1 Rs2 and s2 Rs1
3. R is transitive, viz. if s1 Rs2 and s2 Rs3 , then s1 Rs3
Furthermore, if R is an equivalence relation and s1 , s2 ∈ S such that s1 Rs2 , then we say s1
and s2 are equivalent. Given s ∈ S, we say the subset Es ⊂ S of all s′ ∈ S s such that s′ is
equivalent to s is the equivalence class of s in S.
Observe that the existence of an equivalence relation on a set S imparts a new way to
organize its elements, namely, by organizing them into a set of equivalence classes. The
structure the equivalence classes of an equivalence relation R impart to S is a familiar one,
for it is that of a partition, whose cells are the classes themselves. We prove this assertion
as a theorem.
Theorem 18. Let R be an equivalence relation on S, then the set of equivalence classes with
respect to R
{Es }s∈S
induce a partition on S. Conversely, given a partition of S of length r, say A1 , A2 , . . . , Ar ,
then the relation R on S defined by
sRs′ if and only if s, s′ ∈ Ai
for 1 ≤ i ≤ r is an equivalence relation.
First, we note, however obvious, that s ∈ Es , so that every element of S is contained
in some equivalence class. Therefore, the set of equivalence classes cover, i.e. S = ∪s∈S Es .
So to show that the set of equivalence classes is a partition of S, we must show that the
classes are pairwise-disjoint. To this end, suppose that a ∈ Es ∩ Es′ for some a ∈ S. Then,
by definition, both aRs and aRs′ , so that sRs′ , as R is transitive. Thus Es = Es′ or they are
disjoint. Accordingly, the set of equivalence classes {Es }s∈S is a partition of S.
Conversely, given a partition of length r as in the theorem, together with the definition
of R, we must verify that it is reflexive, symmetric, and transitive. However, each of these
matters is routine. Therefore, the proof of the converse is trivial. The relationship between
equivalence classes and partitions allows us to formalize the set of the former in the following
defintion.
77
Definition 33. Let R be an equivalence relation on S. Then we say the set of its equivalence
classes, SR , is the quotient set of S with respect to R.
This definition, incidentally, completes our analogy between arithmetic operations and set
theoretic operations initiated in chapter 1, as we see partitions are effectively the quotients
of sets under this analogy. Now we shall end this section with the result germane to our
motivation, that is, to rigorously define in the next section the probability of a value of a
random variable by understanding the relationship between the fibering of the domain of a
set function and its image.
Let f : S → T be a function of sets. Observe that the function f defines an equivalence
relation on its domain, S, by the theorem we have just proven since its set of fibres partition
the same. Indeed, two elements in S are equivalent if they belong to the same fibre of f
or, equivalently, if their images under f are the same. That is, the canonically induced
equivalence relation R on S is defined by
sRs′ if and only if s, s′ ∈ f −1 (t) if and only if f (s) = f (s′ )
Clearly R is both reflexive and symmetric. That R is also transitive is just as obvious,
but we spell this out anyway by writing when both s1 Rs2 and s2 Rs3 or f (s1 ) = f (s2 ) and
f (s2 ) = f (s3 ) then s1 Rs3 or f (s1 ) = f (s3 ). This summarize this equivalence relation in the
following definition.
Definition 34. Let f : S → T be a set function and R the relation on S induced canonically
by f . Then we say the equivalences classes with respect to this relation are the fibres of f .
Furthermore, we denote this equivalence relation by f and its equivalence classes by
{f = t}
where {f = t} = f −1 (t)
The manner in which a function fibres its domain is an important insight intellectually and
speaks to the ontological role that functions play in mathematics. Indeed, mathematicians
of the previous century made great strides in the subject through their appreciation of this
principle. Yet, beside whatever great strides we are alluding to may have been, there is an
elementary result, now at hand, we shall use below to compute the probability of values of
a random variable.
Theorem 19. Let f : S → T be a function of sets and Sf the corresponding quotient set.
Then there is a bijection ϕ : Sf → f (S). That is, the quotient set of S with respect to f is
in bijective correspondence with its image.
To prove this theorem, let us define a map ϕ : Sf → f (S) by ϕ({f = t}) = t. Let
us first show that ϕ is well-defined. Suppose for t, t′ ∈ f (S) that t ̸= t′ , then we have
{f = t} ∩ {f = t′ } = ∅. Hence ϕ is well-defined. To show ϕ is injective, note that t = t′
implies {f = t} = {f = t′ } because f −1 (t) = f −1 (t′ ). To show that ϕ is surjective, one notes
that f necessarily surjects onto its image. In other words, by definition, for t ∈ f (S) there
exists an s ∈ S such that f (s) = t. As the fibres of f partition S, there is a fibre containing
78
s, say {f = t}. Then, by construction, ϕ({f = t}) = t, showing ϕ is a surjection. As ϕ is
both an injection and a surjection it is therefore a bijection, as desired.
As noted above, this discussion concludes the analogy between set theoretic operations
and arithmetic ones. Whereas unions correspond to sums, intersections to products, and
complements to difference, one could say that equivalence relations correspond to division.
It is best to think of this operation as dividing a set or forming quotients of its elements
by regarding equivalence classes as generalizations of fractions. Indeed, this way of thinking
continues on in other subjects of mathematics, such as forming quotient groups in group
theory, or of course, quotient stacks in algebraic geometry. Below we shall use the last entry
in this analogy to associate to a discrete probability space (Ω, β, P ) in an canonical manner
another discrete probability space whose sample space is the quotient set ΩX with respect
to the equivalence relation induced on Ω by the set function X.
3.1 WORKED EXAMPLES
1. 3.1 Define the set function rn : Z → [n − 1] by the formula, rn (z) = r, where for z ∈ Z,
we have z = qn + r, that is, r is the remainder after division by n.
(a) Compute the fibres of rn . Equivalently, define the equivalence relation
determined by the partition of Z into the fibres of rn .
Solution (a) Perhaps it is first informative to determine the equivalence relation
before considering the corresponding partition of Z induced by the fibres of rn . So,
define R ⊂ Z × Z by (a, b) ∈ R if and only if b − a is an integer divisible by n. First,
R is reflexive, for n|a − a or (a, a) ∈ R. Second, R is symmetric, for n divides both
b − a and −(a − b) and both are integers. So, aRb implies bRa. Last, R is transitive,
for if n|b − a and n|c − b, then n divides c − a = c − b + b − a for n divides both c − b
and b − a by hypothesis. So, the fibres of rn is in bijective correspondence with the
set of equivalence classes induced by R. Let us write Z/nZ for the set of these
equivalence classes. It is clear by the definition of division that |Z/nZ| = n, and
choosing a representative for each equivalence class we can construct a bijection
between Z/nZ and [n − 1]. We may regard the latter, by the theorem of this section,
as the fibres of rn .
2. Consider two points (x0 , y0 ), (x1 , y1 ) ∈ R2 . Define (x0 , y0 )R(x1 , y1 ) by y0 −x20 = y1 −x21 .
(a) Show that R is an equivalence relation.
(b) Compute the equivalence classes of R.
Solution (a) Obvious by properties of the equality of real-numbers.
79
Solution (b) The equivalence classes are the points on the parabola, that is, the
graph of the function f (x) = x2 in R2 .
3. 4.1 Let σ : [n]⋆ → [n]⋆ be a permutation, that is, a bijection of [n]⋆ onto itself determined by an ordered list of its elements selected without replacement. Suppose
x, y ∈ [n]⋆ . Define xRy if and only if there exists k ∈ N such that y = σ k (x).
(a) Show that R is an equivalence relation.
Solution (a) Let us proceed in order. First, to show that R is reflexive, observe that
there does indeed exist such k ∈ N such that x = σ k (x) or xRx. Namely, k = 0.
Next, to show that R is symmetric, we recall that a bijection such as σ possesses an
inverse and that composition is associative. Therefore, we take k in x = σ k (y) to be
the k applications of σ −1 to y given k such that y = σ k (x). There is more to say upon
how σ is related to its inverse to completely justify this assertion, but we feel such a
digression is beyond the scope of this exercise. Last, R is transitive by composition of
functions and the definition of σ. Namely, if xRy and yRz where y = σ k (x) and
z = σ l (y) then z = σ k+l (x), as desired. An important remark is that the equivalence
classes of R are often called orbits.
4. 4.6 Consider R2 be the Cartesian plane.
(a) Let R ⊂ R2 consisting of pairs (a, b) ∈ R2 such that (c, d) ∈ R if and only if
a = c. Determine whether R is an equivalence relation. Determine the
equivalence classes.
(b) Let R ⊂ R2 consisting of pairs (a, b) ∈ R2 such that (c, d) ∈ R if and only if
a + d = b + c. Determine whether R is an equivalence relation. Determine the
equivalence classes.
Solution (a) We proceed to verify the axioms of an equivalence relation. First,
(a, b)R(a, b) since a = a, so R is reflexive. Second, if (a, b)R(c, d) which gives a = c,
but then (c, d)R(a, b) since a = c implies c = a. Hence R is symmetric. Last, if both
(a, b)R(c, d) and (c, d)R(e, f ) then a = c = e so that (a, b)R(e, f ). Therefore, R is
transitive and R is an equivalence relation. By inspection, the equivalence classes
{E(a,b) } are vertical lines in R2 .
Solution (b) We proceed to verify the axioms of an equivalence relation. First,
(a, b)R(a, b) since a + b = a + b, so R is reflexive. Second, if (a, b)R(c, d) which gives
a + d = b + c, but then (c, d)R(a, b) since a + d = b + c implies c + b = d + a. Hence R
is symmetric. Last, if both (a, b)R(c, d) and (c, d)R(e, f ) then a + d = b + c and
c + f = d + e. We want to show (a, b)R(e, f ), that is, a + f = b + e. Observe, we have
80
a + d = b + c implies a + c + f − e = b + d + e − f which gives
(a + f ) + (c − e) = (b + e) + (d − f ). This is true since (c, d)R(e, f ) or c + f = e + d.
Therefore, (a, b)R(e, f ) and R is transitive. By inspection, the equivalence classes
{E(a,b) } are squares in R2 spanned by the rectangles both (a, b) and (c, d) determine.
3.2
Discrete Random Variables
Broadly speaking, a random variable is a numerical summary of an experiment whose
sample space is sufficiently large. We express such a numerical summary in our theory as a
discrete probability space together with a real-valued function whose domain is the sample
space. One could say that the assignment of simple events parameterized by numbers to
those parameter values is the manner in which the additional data of the real-valued function
summarizes the events of the experiment. Let us recall, however, that the probabilities of
events are defined by P applied to elements of the σ-algebra. Thus, we require that our
summary respect this structure if we intend to us to compute the probabilities of events. All
together, we obtain the following definition.
Definition 35. Let (Ω, β, P ) be a discrete probability space. Then we say a real-valued
function
X:Ω→R
such that {X = x} ∈ β for all {X = x} ∈ ΩX is a discrete random variable.
A few comments on this fundamental definition are in order. First, notice that the image
of X is itself discrete. Indeed, by definition, any function surjects onto its image, and by
chapter 1, the cardinality of Ω is an upper bound for the cardinality of its image X(Ω) which
is in bijective correspondence with ΩX under ϕ of the previous section. As such, ΩX is a
discrete set.
Second, and perhaps most importantly, the condition that {X = x} := x ∈ β means that
for all x ∈ ΩX , we have that P is defined on x by the formula p(x) = P ({X = x}). Another
way to read the second condition is that ΩX ⊂ β.
Third, as we shall only consider discrete random variables in this text, we shall often
ignore the adjective "discrete" in our exposition. One always assumes the existence of a
discrete probability space for which X is defined if one is not mentioned explicitly.
The numerical summary that a random variable X furnishes an experiment is insufficient
for our purposes of simplifying the computation of the probability of events in an experiment,
in terms of such a summary, alone. Indeed, X in the definition merely assigns a real-number
to a set of simple events that can be characterized by the real-number or parameter in an
experiment. In order to elevate the addition of such a numerical summary or parameterization of simple events, X, to an implement that can also express the probability of the event
it summarizes, we shall introduce a new probability space, that of the induced probability
space.
So, consider an arbitrary discrete random variable X defined with respect to a discrete
probability space (Ω, β, P ). We want to produce from this data a new discrete probability
space (ΩX , βX , PX ), canonically associated to the same which we will call the induced probability space in order to compute the probabilities of events parameterized by X. In order
81
to do this we may take the quotient set of Ω with respect to X as the induced sample space
for this triple. As we observed above, ΩX is a countable set, by construction, therefore, by
chapter 1, we adopt as the induced σ-algebra the canonical choice, namely, βX = 2ΩX , the
power set of ΩX . Lastly, the reason for our digression in the first section of this chapter into
equivalence relations and quotient sets was to define the induced probability function PX in
such a way that it satisfies Kolmogorov’s axioms.
Given a random variable X, since the set of its fibres is a subset of the σ-algebra, β, we
define, for {X = x} ∈ β the probability function PX : βX → [0, 1] by the formula
PX (x) = P ({X = x})
We must emphasize this definition is only possible because the set of fibres of X is a subset of
β, the domain of the probability function P , by the definition of a discrete random variable.
It is in this manner that a random variable induces a probability space, provided that the
PX defined above satisfies Kolmogorov’s axioms. That is the content of the next theorem.
Theorem 20. Let (Ω, β, P ) be a discrete probability space and X : Ω → R a random variable.
Then the function PX (x) = P ({X = x}) satisfies Kolmogorov’s axioms.
To prove this theorem, we only consider the case when ΩX is finite. Suppose ΩX =
{x1 , . . . , xn }, then, with this notation, we proceed by the theorem of chapter 1.3 that gives
the canonical formula for probability functions. To do this,P
we must find pi = PX ({X = xi })
to assign to the simple events in ΩX such that pi ≥ 0 and i pi = 1. To this end, take
X
pi = PX ({X = xi }) =
pj
j|uj ∈{X=xi }
where P (uj ) = p for j = 1, . . . , m.
Observe, since P is itself a probability function and {X = xi } ∈ β, that P ({X = xi }) ≥ 0
since both P ({X = xi }) = pi , hence pi ≥ 0 for each i because pj ≥ 0. This verifies
the first
P
condition of the theorem of 1.3. Next, to verify the second condition, that i pi = 1, we
have
1=
=P (Ω)
=P ∪xi ∈X(Ω) {X = xi }
n
X
=
P ({X = xi })
i=1
=
n
X
pi
i=1
Therefore, by the theorem of 1.3 PX satisfies Kolmogorov’s axioms, as desired. One remarks
that lines two and three require that X fibres its domain, as discussed in chapter 1. Lines
four and five require that P satisfies Kolmogorov’sX
axioms, which it does, by hypothesis.
Therefore, by chapter 1, the assignment PX (A) =
pi , for any A ∈ βX , is a probability
i|xi ∈A
function on the induced probability space (ΩX , βX , PX ).
Altogether, our discussion here proves the following theorem and definition.
82
Theorem 21. Let (Ω, β, P ) be a discrete probability space and X : Ω → R a random
variable. Then the triple (ΩX , βX , PX ) defined above is a discrete probability space. We say
this probability space is the induced probability space. Furthermore, to unburden notation,
hereafter, we denote the induced probability function PX (x) = p(x), unless otherwise noted.
Introducing random variables to the study of the likelihood of events in an experiment
does more than furnish a convenient numerical summary which can be used to the compute
the probability of an an event it parameterizes. They also impart to experiments additional
insightful details, and the study of these is what follows in the subsequent sections of this
chapter. Of course the material presented below is only a small portion of the vast expanse
of additions random variables provide for to the study of experiments. We note that as this
course is focused on discrete mathematics we neglect the important topic of inferential statistics, one of the key elements of the additional information variables bring to experimental
analysis.
3.2 WORKED EXAMPLES
1. Let us consider the experiment of tossing a fair coin three times and recording the
sequence of results together with its attendant discrete probability space, (Ω, β, P ),
where P is defined by the relative frequency interpretation of probability. Let X :
Ω → R be discrete random variable defined on simple events by the number of heads
in the sequence of results.
(a) Compute ΩX
(b) Compute the fibre over 2 ∈ ΩX
(c) Evaluate p(2)
We proceed by the theorem for each question.
Solution (a) So, by the definition of X, its consists of the number of heads in a
sequence of three flips of a fair coin. Therefore, using the notation of chapter 2,
ΩX = [3].
Solution (b) To compute the fibre over x = 2, that is, u ∈ β such that X(u) = 2, we
must count the number of sequences with two heads in 3 flips. Assuch
a sequence in
3
a 2-combination of sequence of results, so we have that there are
= 3 of them.
2
Explicitly, we have {X = 2} = {HHT, HT H, T HH} ∈ β
Solution (c) Given that the coin being flipped is fair or, equivalently, we use the
relative frequency interpretation of P to determine probability of events in (Ω, β, P ),
3
we have that p(2) = P ({X = 2}) =
8
83
2. 4.2 In this example, we exhibit the virtue of summarizing experiments in terms of a
numerical parameter to compute the probability an event when the sample space is too
large to practically enumerate. To wit, consider the experiment of sampling 3 marbles
at random from an urn containing 12 marbles, 3 of which are blue, 9 of which are grey
together with its attendant discrete probability space (Ω, β, P ), where P is defined by
the relative frequency interpretation of probability. Let X : Ω → R be the number of
blue marbles in a sample of 3 marbles.
(a) Compute |Ω|
(b) Compute p(x) for each x ∈ ΩX .
Solution (a) A sample of 3 random
marbles from the urn is a 3-combination of the
12
12 marbles. Therefore, |Ω| =
. Simplifying this, we have |Ω| = 220, so we see it
3
is impractical to enumerate the simple events in this experiment for the purposes of
computing p(x).
Solution (b) By the relative frequency hypothesis, we must compute |{X = x}| for
each x ∈ ΩX = [3], by the definition of X. Let us proceed in order by the
mulitiplication rule, where our first step is to count the number of blue marbles in the
sample, and the second is to count the number of grey marbles.
First, observe that the number of samples with
x = 0 blue marbles is the number of
9
3-combinations of the 9 grey marbles, so 3 . Similarly, for x = 1 we can count the
number of 2-combinations of the 9 grey marbles for each of the 3 positions we reserve
for a blue marble, so 3 · 92 . For x = 2, we recognize that these samples are
2-combinations of the 3 blue marbles
and only 9 ways to fill in the position left open
for a grey marble. Therefore, 32 · 9. Finally, there is only 1 sample containing 3 blue
marbles. As such, we have
9
3
· 3
p(0) = 0
220
3
1
· 92
p(1) =
220
3
2
· 91
p(2) =
220
3
3
· 90
p(3) =
220
84
3. 4.2 In this example, we exhibit the virtue of summarizing experiments in terms of a
numerical parameter to compute the probability an event when the sample space is too
large to practically enumerate. To wit, consider the set of strings of 0’s and 1’s fifty
positions long together with the attendant discrete probability space, (Ω, β, P ), where
P is defined by the relative frequency interpretation of probability. Let X : Ω → R be
number of 1’s in a given string.
(a) Evaluate p(x) for every x ∈ ΩX
Solution (a) By definition, p(x) = P ({X = x}), where {X = x} ∈ β is the set of
events that map to x ∈ ΩX . We know by hypothesis that the probability of a simple
1
event is 50 , so we only must count |{X = x}| to compute p(x). To this end, we see
2
that x 1’s in a string of 0’s and 1’s isanx-combination of the 50 positions available
50
to place them. Therefore, there are
such simple events in β that map to x, or
x
50
50
|{X = x}| =
. Therefore, by definition, p(x) = x50 for any x ∈ ΩX . One notes
2
x
1
1
that this probability is equivalent to p(x) = 50
( )x ( )50−x . We shall recognize this
x
2 2
later on as an example of the binomial random variable with parameters n = 50 and
1
p= .
2
4. 4.2 Consider the experiment Ω of recording the number of children in a family. Define
the discrete random variable X : Ω → R where X(u) = x, the number of children in a
family u. Given the frequency table


x | |{X = x}|
0

| 10


1

|
30


2

| 520


3

|
250


4

| 70


5

| 17
6
| 3
consider the following questions.
(a) Determine the induced probability space. In particular, determine the formula
for its probability function.
(b) Compute p(5)
(c) Can you compute p(8)? Why or why not?
85
(d) Compute p(x ≤ 2)
(e) Compute p(x ≥ 1)
Solution (a) Given the frequency table together with the definition of X, we have
that ΩX = {0, 1, 2, . . . , 6}. As usual, βX is canonically supplied to be the power set.
|{X = x}|
|{X = x}|
Finally, by definition, P ({X = x}) =
=
|Ω|
900
Solution (b) p(5) =
17
= 0.0188
900
Solution (c) No, because x = 8 ∈
/ ΩX , so p(x) is undefined.
Solution (d) p(x ≤ 2) = P ({X = 0} ∪ {X = 1} ∪ {X = 2}) = P ({X =
10
30
520
0}) + P ({X = 1}) + P ({X = 2}) = p(0) + p(1) + p(2) =
+
+
= 0.6222
900 900 900
Solution (e) p(x ≥ 1) = 1 − p(0) = 1 −
10
= 0.99
900
5. 4.2 Consider the experiment Ω of recording the number of human beings awaiting their
lunch at a food serving establishment. Define the discrete random variable X : Ω → R
where X(u) = x, the number of human beings in a waiting-line u. Given the table of
values
of p(x)for x ∈ ΩX :

x | p(x)
 0 | 0.011


 1 | 0.035


 2 | 0.089


 3 | 0.150


 4 | 0.186


 5 | 0.172


 6 | 0.132


 7 | 0.098


 8 | 0.063


 9 | 0.035


10 | 0.019


11 | 0.004
12 | 0.006
consider the following questions.
(a) Explain how to verify that p(x) satisfies Kolmogorov’s axioms.
(b) Compute p(8) and explain its meaning.
(c) Compute p(x ≥ 10) and explain its meaning.
(d) Compute p(x ≤ 10)
86
Solution (a) We can confirm that p(x) satisfies Kolmogorov’s axioms by adding the
values in the p(x) column and determining whether they add to 1, as the values in
the x column arePthe simple events in ΩX . As for whether p(x) is non-negative and
whether p(I) = J⊂I p(J), we have only to inspect whether the p(x) column has any
negative values and to apply the definition of p.
Solution (b) p(8) = 0.063 by the table and its value means there is 6.3 % of lunch
lines are 8 people long.
Solution (c) p(x ≤ 10) = 1 − p(11) − p(12) = 1 − 0.004 − 0.006 = 0.99 and it means
that at least 99 % of lunch lines have 10 or more human beings waiting.
Solution (d)
3.3
The Histogram and The Cumulative Distribution Function
In this section we shall introduce the first additional detail a random variable imparts to
an experiment and present some examples. It is a visual representation of discrete random
variables common throughout the literature, that of its histogram. We emphasize at the
outset that the definition below is distinct from that of the graph of a probability function.
So much so, we single out the following definition.
Definition 36. Let X be a discrete random variable with respect to an experiment Ω and
(ΩX , βX , PX ) the induced probability space. Then we say the graph of PX is the distribution
of X. We denote its distribution by ΓX .
Recall by chapter 1 that the distribution of a discrete random variable X is the set of
ordered pairs (x, y) ∈ R2 such that y = p(x) for x ∈ ΩX ⊂ R. In terms of the previous section,
ΓX ⊂ R2 is a relation, where (x, y)ΓX (x′ , y ′ ) if and only if p(x) = p(x′ ). We emphasize that
by the hypotheses in our course that ΓX is a discrete subset of the plane, which is rather
unusual in light of the familiar examples of graphs of functions in say, an introductory course
in calculus. To obviate this unfamiliarity and recover a geometric interpretation of the axioms
of Kolmogorov, we introduce the following extension of the distribution of a random variable
to the plane.
So, let X be a random variable defined with respect to some discrete probability space.
As noted in the previous section, ΩX is a discrete subset of R. Let us suppose further for
the sake of simplicity that |ΩX | = n, for this supposition allows us to ignore the issue of
convergence of improper integrals. Write ΩX = {x1 , x2 , . . . , xn } and it is straightforward to
see that ΩX is in bijective correspondence with the set [n]⋆ of integers 1 ≤ i ≤ n by xi 7→ i.
We have enough notation for the following definition.
Definition 37. Let X be a random variable defined with respect to a discrete probability
space. Write ΩX = {x1 , x2 , . . . , xn } for the simple events of the induced probability space.
Define hX to be the real-valued, piece-wise continuous function on the interval [0, n) by the
formula hX (x) = p(xi ) for x ∈ [i − 1, i), where 1 ≤ i ≤ n is an integer. Then the graph of
hX in R2 is called the histogram of X.
87
One notices that the area of the planar region determined by ΓP
hX , that is, the
R rhistogram
r
of X, is equal to 1 by Kolmogorov’s
R n first axiom. Indeed, one has i=1 p(xi ) = 0 hX (x)dx,
for 1 ≤ r ≤ n and in particular 0 hX (x)dx = 1. Its very definition illustrates the second
axiom in the plane and as for the third, one can compute the likelihood of a disjoint sequence
of events merely by adding the areas of the rectangles the histogram determines. As such,
we regard the histogram as a geometric interpretation of the distribution ΓX . One can call
the above sum the cumulative distribution function of X, but we ignore this concept in this
text. Its geometric significance motivates the following ubiquitous concept that this text
largely neglects.
Definition 38. Let X : Ω → R be a discrete random variable with histogram hX . We say
for 1 ≤ r ≤ n that
P
FX (xr ) = ri=1 p(xi )
is the cumulative distribution function of X.
This definition is so common that occasionally it is even used to define discrete random
variables, albeit analytically. Indeed, one could say that a random variable is discrete if
its cumulative distribution function is a step function. Beside this interesting feature, the
histogram allows one to visualize an experiment as a system of particles ΩX together with
weights p(xi ). We shall pursue this observation more substantially below when we discuss the
expectation of a random variable as well as its variance. Indeed, histograms provide shape,
center of mass, variability, and other geometric features such an outliers, to experiments. We
emphasize now that one must substitute [n]⋆ for ΩX below in order to fulfill this intuition
entirely.
3.3 WORKED EXAMPLES
1. 4.3 We construct the histogram associated to the random variable X defined in example
2 of the previous section.
Solution (a) Recall, ΩX = {0, 1, 2, 3} in that example, so we construct a function
hX : [0, 4) → R. Indeed, by providing labels to the elements of ΩX , say
ΩX = {x1 = 0, x2 = 1, x3 = 2, x4 = 3}, then we define hX (x) = p(xi ) for x ∈ [i − 1, i),
for 1 ≤ i ≤ 4 an integer. According to that example, we have
108
27
1
84
, p(x2 ) =
, p(x3 ) =
and p(x4 ) =
, which determines the values
p(x1 ) =
220
220
220
220
for hX over the subintervals [i − 1, i). Therefore, the graph of hX is the histogram of
the random variable of example 2 in the previous section.
2. 4.3 Consider the experiment of recording the results of the math SAT. Scores range
from 200 to 800 and the record reflects scores in intervals of 100. Furthermore, suppose
88
we have a specific sample of data, consisting of 1,530,128 students and their scores:
SATscore 200 − 299 →
SATscore 300 − 399 →
SATscore 400 − 499 →
SATscore 500 − 599 →
SATscore 600 − 699 →
SATscore 700 − 800 →
97, 296 students
295, 693 students
449, 238 students
454, 497 students
197, 741 students
35, 663 students
(a) Construct a histogram describing the sample data.
Solution (a) Solving the problem only amounts to converting the data into the
aspects of the definition. Indeed, we can label the score ranges from 1 to 6 so that xi
is the label for the SATscore score range of the i-th range, for 1 ≤ i ≤ 6 an integer.
si
Now construct a function hX : [0, 6) → R by letting hX (x) =
, where si is
1, 530, 128
the number of students whose SAT math score is in the i-th range, for x ∈ [i − 1, i).
Furthermore, as we assume the tests are fairly administered, we take p(xi ) to be the
relative frequency of the number of students in the i-th score range. Accordingly, we
have p(x1 ) = 0.064, p(x2 ) = 0.193, p(x3 ) = 0.294, p(x4 ) = 0.297, p(x5 ) = 0.129, and
p(x6 ) = 0.023. As such, the set of ordered pairs determining the step function over
si
[0, 6) consisting of ΓX = {(i, p(xi )) |1 ≤ i ≤ 6, p(xi ) =
} is the histogram of
1, 530, 128
the random variable X assigning the label of an interval of SAT scores to an integer
between 1 and 6.
3.4
Transformations, Generating Functions, and the Law of the Unconscious Statistician or the Transformed Expected Value Rule
In this section we both introduce and define so-called morphisms of discrete probability
spaces, referred to as transformations in these notes. Our approach to proving the law of the
unconscious statistician is therefore through naive category theory, where the objects of our
category are discrete probability spaces replete with the conventions we adopted in chapter
1. Pursuant to the goal of proving the law of the unconscious statistician we introduce
generating functions and treat the generating function associated to a random variable X as
an invariant of its induced probability space. We use this generating function in conjunction
with the concept of the transformation of a random variable to prove the theorem. The
theorem is useful throughout the subject in general and, in particular, for the families of
discrete random variables we present in chapter 5.
The subject of classical category theory is beyond the scope of this text, so as mentioned
above, we shall adopt a naive perspective, which will mean to only pick and choose those
elements of its repertoire conceptually useful to discrete probability theory. In particular, the
89
notions of objects and morphisms between them as a means of relating the expected value
of a random variable that is a function of another to the expected value of the variable upon
which it functionally depends. Perhaps the best comparison at this level of discussion is to
linear algebra, where the objects are vector spaces and morphisms between them are linear
transformations. In our context, we have already introduced the objects of our category:
namely, discrete probability spaces. The corresponding morphisms are defined as follows.
Definition 39. Let both (Ω, β, P ) and (Ω′ , β ′ , P ′ ) be discrete probability spaces. Then we
say a transformation from (Ω, β, P ) to (Ω′ , β ′ , P ′ ) is a set function f : Ω → Ω′ such that the
formal inverse f −1 : β ′ → β obtained by extending f is a function of sets.
Beside using this definition to prove the law of the unconscious statistician we demonstrate how the concept of random variables may be understood in terms of such transformations. Indeed, consider a discrete random variable as in the definition of section 3.2., then
X is a discrete random variable if and only if X is a transformation of discrete probability spaces between (Ω, β, P ) and (ΩX , βX , PX ); that is, one must have that X −1 (x) ∈ β or
equivalently that the formal inverse X −1 : βX → β obtained by extending X : Ω → ΩX
is a function of sets. In terms of the compatibility of X with the probability functions of
the codomain and domain, we have PX (x) = P ({X = x}) = P (X −1 (x)). We shall use this
observation to clarify below the relationship between the induced probability functions of
discrete random variables given a transformation, defined below,Pof the same.
x
Next, a formal power series is an element of the set R[[z]] = { ∞
x rx z }, where rx ∈ R for
all x. What distinguishes these power series from those of say, an elementary course in Calculus, is that the notion is convergence is ignored, whence the adjective "formal." Nonetheless,
this set is endowed with the same structure it has when convergence is maintained, that is
to say, one may add and subtract or multiply and divide elements as usual. Furthermore,
one also has the operation of formal differentiation which also ignores the convergence of
P
d
on the set R[[z]] by
limits. Let f (z) = x rx z x ∈ R[[z]], then one defines an operator
dz
the formula
df (z) P
= x rx xz x−1
dz
This formula gives the formal derivative of an element of R[[z]]. We shall use this formula
below to define the both the formal expectation and the formal variance of a random variable.
It must be emphasized that, although this definitions are independent of an experimental
context, the formulas they produce are most useful and so are the ones we still use in
applications below.
We require the concept of a formal power series to instantiate the concept of a generating
function.PRecall, if {ai }i∈I is a sequence of real numbers, then we say the formal power
fa (z) = i∈I ai z i is the generating function for the sequence {ai }∞
i=i0 . We shall be interested
in generating functions for the sequence {ai = p(xi )}xi ∈ΩX , for some random variable X.
The following definition provides all of the details.
Definition 40. Let X be a random variable with respect to some discrete probability space.
Then we say the formal power series g(z) such that
90
X
g(z) =
p(x)z x
x∈ΩX
is the generating function of X.
It must be emphasized that the generating function generates X in the sense that the
sequence of its coefficients completely determine the distribution of X. Indeed, the following
computational observation justifies this remark. Observe that, for X a random variable such
that ΩX = {0, 1, 2, . . .} with generating function g(z) we have,
g(0) =
∞
X
p(x)0x = p(0)
x=0
′
g (0) =
X
p(x)x0x−1 = p(1)
x=1
..
.
=
..
.
g (n) (0) =n!p(n)
..
.
.
= ..
so g(z) generates the distribution of X by supplying its ordered pairs (x, p(x)). We shall
take advantage of this feature in conjunction with the formal derivative to obtain two more
important characteristics of the random variable X.
Definition 41. Let X be a random variable and g(z) its generating function. Then we say
dg(z)
|z=1 = g ′ (1) is the expectation of X.
dz
2
d2 g(z)
dg(z)
dg(z)
2. V (X) =
|z=1 +
|z=1 −
|z=1 = g ′′ (1)+g ′ (1)−(g ′ (1))2 is the variance
2
dz
dz
dz
of X.
p
3. σX = V (X) is the standard deviation of X.
1. E(X) =
Although we have yet to define these features of a random variable in terms of an experiment, we shall recognize their definitions later in the following computations. First, given the
generating function g(z) of a random variable X, we have, computing its formal derivative
X
g ′ (z) =
xp(x)z x−1
x∈ΩX
so that, evaluating at z = 1, we have
g ′ (1) =
X
x∈ΩX
91
xp(x)
which will proffer the experimental definition of the expectation of a random variable X
below.
A similar computation shows that E(X 2 ) = g ′′ (1) + g ′ (1), so that, V (X) = E(X 2 ) −
(E(X))2 . This second computation is mysterious for two reasons. First, it is not perfectly
clear what E(X 2 ) means. However, it is not hard to imagine that if we write Y = X 2 ,
then it simply means the first derivative of the generating function of the random variable
Y . Yet, how this is expressed in terms of the original or independent random variable X
shall motivate the following discussion on transformations of random variables. The second
aspect that shall remain mysterious is why V (X) can be simplified in terms of a linear
combination of expectations. We shall pursue this point in subsequent sections and for now
content ourselves with following formula ubiquitous in the subject.
The definition of variance suggests we must make sense of E(Y ) for a random variable
Y = f (X) that depends on a random variable X. Accordingly, a transformation of random
variables, say X and Y , both defined with respect to the same discrete probability space
(Ω, β, P ), is a transformation of their respective induced probability spaces. To wit, a set
function f : ΩX → ΩY such that the formal inverse f −1 : βY → βX is a set function. We
denote such transformations by Y = f (X). We will consider how the probability function
of the domain is compatible with that that of the codomain in this specific instance, below.
We have enough now to prove the following well-known theorem, the aforementioned law of
the unconscious statistician.
Theorem 22. Let X and Y be random variables together with a transformation, say Y =
f (X). Then
X
E(Y ) =
f (x)p(x)
x∈ΩX
To prove this theorem, let us first observe that, by the hypothesis the formal inverse of
f is a set function, it is well-defined and so f −1 (y) ∈ βX , for all y ∈ ΩY . Furthermore,
it follows that X −1 (f −1 (y)) ∈ β by the definition of random variable.
Hence, in terms
P
of the probability functions, this means pY (y) = pX (f −1 (y)) =
x∈f −1 (y)|f (x)=y pX (x) by
Kolmogorov’s axiom. This observation expresses the probability function of Y in terms of
X, given a transformation Y = f (X).
We shall use this observation to express the expected value of Y in terms of the expected
value of X, as the theorem requires. So, consider the generating function of Y and write
X
g(z) =
p(y)z y
y∈ΩY
92
Then we can deduce the formula of the theorem in the following steps:
X
g(z) =
p(y)z y
y∈ΩY
′
g (z) =
X
yp(y)z y−1
y∈ΩY
E(Y ) =
X
yp(y)
y∈ΩY
X
=
f −1 (y)p(f −1 (y))
f −1 (y)∈f −1 (ΩY )
=
X
f (x)p(x)
x∈ΩX
We can replace the summation index for E(Y ) by the hypothesis that f −1 is well-defined.
Similar remarks apply to how we replace y and pY (y), as well. In the last step we extend
the summation index to all of ΩX as the function f fibres ΩX . Etymologically, the name for
this theorem is due to impression a statistician could use it while unconscious. We shall use
it below to obtain several key facts.
3.4 WORKED EXAMPLES
1. Let X be a random variable with ΩX = {0, 1, 2, 3} and generating function g(z) =
z
(2 + 3z 2 )
5
(a) Compute the distribution of X.
Solution (a) To compute the histogram of X, we must compute the first three
derivatives of its generating function and evaluate them at z = 0. Following our
demonstration in the section, we have
2
18
2 9
0!p(0) = g(0) = 0, 1!p(1) = g ′ (0) = + z 2 |z=0 = , 2!p(2) = g (2) (0) = z|z=0 = 0,
5 5
5
5
18
18
(3)
and 3!p(3) = g (z) = |z=0 = . So the distribution of X is the graph of the
5
5
function p(x) as defined in section 4.2 together with the values of x ∈ ΩX under p(x)
that we have computed.
2. Let X be a random variable such that g(z) = eλ(z−1) for λ ∈ R.
(a) Compute E(X)
(b) Compute V (X)
93
Solution (a) Proceed by the definition, so that E(X) = g ′ (1). Computing the
derivative, we have g ′ (z) = λλ(z−1) and so g ′ (1) = λ.
Solution (b) Again, proceed by the definition, so that
V (X) = g ′′ (1) + g ′ (1) − (g ′ (1))2 . As g ′′ (z) = λ2 eλ(z−1) , we have
V (X) = λ2 + λ − λ2 = λ.
3. Let X be a random variable such that g(z) = (pz + q)n , for p ∈ [0, 1] and q = 1 − p.
(a) Compute E(X)
(b) Compute V (X)
Solution (a) Proceed by the definition, so that E(X) = g ′ (1). Computing the
derivative, we have g ′ (z) = np(pz + q)n−1 and so g ′ (1) = np.
Solution (b) Again, proceed by the definition, so that
V (X) = g ′′ (1) + g ′ (1) − (g ′ (1))2 . As g ′′ (z) = n(n − 1)p2 (pz + q)n−2 , we have
V (X) = n(n − 1)p2 + np − n2 p2 = np − np2 = npq.
4. Let X be a random variable such that g(z) =
p
for p ∈ [0, 1] and q = 1 − p.
1 − qz
(a) Compute E(X)
(b) Compute V (X)
Solution (a) Proceed by the definition, so that E(X) = g ′ (1). Computing the
pq
q
derivative, we have g ′ (z) =
and so g ′ (1) = .
2
(1 − qz)
p
Solution (b) Again, proceed by the definition, so that
2pq 2
V (X) = g ′′ (1) + g ′ (1) − (g ′ (1))2 . As g ′′ (z) =
, we have
(1 − qz)3
2q 2 q q 2
q
V (X) = 2 + − 2 = 2 .
p
p p
p
3.5
Discrete Random Vectors
In this section we shall introduce random vectors. Random vectors are used below in
several ways: first, to help demonstrate properties of expectation and variance once we define
these experimentally. Second, to examine the independence of random variables. Third,
to define stochastic processes and in particular, Markov processes in our derivation of the
formalism of Markov chains from discrete probability theory. In general, they are sequences
94
of random variables defined with respect to a fixed probability space that determine a vectorvalued function on the same. Experimentally, however, random vectors are arise in contexts
where one wishes to study the likelihood of the occurence of more than one numerical feature
of an experiment.
Let us briefly review double summation. We define the double sum as follows. Consider
some set theoretic function zij = f (i, j) defined on a discrete rectangle R = {1 ≤ i ≤ m, 1 ≤
j ≤ n}. Then the double sum over the discrete rectangle R is the sum of the mn elements
of the array {zij }1≤i≤m,1≤j≤n determined by f . To wit,
X X
zij
(i,j)∈R
is the double sum. Computationally there is a analogue of Fubini’s theorem for double sums.
It says that the double sum may be computed as an iterated sum. Assuming R is obtained
as a Cartesian product of discrete intervals, say R = I × J, then
!
X X
X X
zij =
f (i, j)
i∈I
(i,j)∈R
j∈J
!
=
X X
j∈J
f (i, j)
i∈I
where the first line on right hand side is interpreted as an iterated sum, meaning to first sum
over J to obtain a single sum dependent upon i, second to compute the sum over I, and the
second line on the right hand side vice versa.
Let us now proceed to describe random vectors in somewhat less rigor than we have
described random variables, so far, but that is nonetheless useful below. Let X and Y both
be random variables defined with respect to the discrete probability space (Ω, β, P ). Observe
that both X and Y furnish maps to R, by definition, so together (X, Y ) : Ω → ΩX ×ΩY ⊂ R2 ,
where (X, Y )(u) = (X(u), Y (u)) for u ∈ Ω, is a vector-valued function. We say such a
function is a random vector provided X −1 (x) ∩ Y −1 (y) ∈ β for all (x, y) ∈ ΩX × ΩY . One
extends this definition to a sequence of n random variables (X1 , X2 , . . . , Xn ) in the obvious
way to obtain the general concept of random vector.
Next, we obtain the probability space induced by a random vector with two components.
First, we obtain an induced sample space proceeding in analogy with the case of a single
variable. Specifically, we take ΩX × ΩY to be the induced sample space with respect to
(X, Y ). Next, as both ΩX and ΩY are countable, so too is their Cartesian product ΩX × ΩY ,
so therefore βX×Y = 2ΩX ×ΩY has a σ-algebra structure. Finally, we can define a probability
function on ΩX × ΩY by the formula,
pXY (x, y) = P ({X = x} ∩ {Y = y})
called the joint probability function. Let us see how this function satisfies Kolmogorov’s
axioms, at least informally.
Observe that, the random variable X fibres its domain, so we can write
Ω = ∪x∈ΩX {X = x}
95
Further, for any y ∈ ΩY , by the law of total probability we can express the probability of its
fibre with respect to the partition of Ω induced by X, viz.
X
P ({Y = y}) =
P ({X = x} ∩ {Y = y})
x∈ΩX
Since the probability function of Y satisfies Kolmogorov’s axioms, by hypothesis, we have
!
X
X X
1=
P ({Y = y}) =
P ({X = x} ∩ {Y = y})
y∈ΩY
y∈ΩY
x∈ΩX
which shows that, by the discrete Fubini’s theorem,
X X
pXY (x, y) = 1
ΩX ×ΩY
All together, we summarize these results in the following theorem.
Theorem 23. Let X and Y be discrete random variables with respect to the discrete probability space (Ω, β, P ) and (X, Y ) : Ω → R the corresponding discrete random variable. Then
the function pXY (x, y) = P ({X = x} ∩ {Y = y}) satisfies Kolmogorov’s axioms.
As in the case of a single random variable, we take pi,j = P ({X = xi } ∩ {Y = yj }) for the
values required by the theorem for the canonical probability function in Chapter 1. Indeed,
by the same theorem, we have that for A ∈ βX×Y
X
X
P ((X, Y ) ∈ A) =
pi,i
(i,j) | (xi ,yj )∈A
Following the case of a single random variable, we have the following theorem.
Theorem 24. Let (Ω, β, P ) be a discrete probability space and (X, Y ) : Ω → R2 a random
vector. Then the triple (ΩX × ΩY , βX×Y , pXY ) defined above is a discrete probability space.
This theorem generalizes in the obvious manner to random vectors of length n, that is,
(X1 , X2 , . . . , Xn ).
Let us next consider statistics. Given the data of a random vector, we obtain a transformation Z : Ω → R of (X, Y ) by Z(u) = f (X(u), Y (u)), where f : R2 → R is a real-valued
function restricted to the image of (X, Y ). We say f in this context is a statistic.
Definition 42. Let X and Y be discrete random variables with respect to the discrete probability space (Ω, β, P ) and (X, Y ) : Ω → R2 the corresponding random vector. Then we say
the transformation of Z = f (X, Y ) inducing the commutativity of the following diagram
(X, Y )
Ω
Z
ΩX × ΩY ⊂ R2
f (X, Y )
R
96
is a statistic.
We can use statistics to define the expected value and covariation, which is the several
variables generalization for random vectors, of random vectors, by appealing to the law of
the unconscious statistician.
Proposition 18. Let (X, Y ) be a X
random
X vector such that the statistic Z = f (X, Y ) is a
random variable. Then E(X, Y ) =
f (x, y)pXY (x, y)
(x,y)
We define the figure in the proposition to be the expected value of the random vector (X, Y ). Again, this definition extends in the obvious manner to a random vector
(X1 , X2 , . . . , Xn ) of length n.
Associated to a random vector are new experimental features, imperceptible if one considers only single random variables. The first example is the marginal probability function.
These are the restrictions of the joint probability function. These restrictions recover the
individual probability functions of X and Y from the joint probability function.
Definition 43. Let Z be a joint random variable with respect to the random vector(X, Y ).
Then we say
X
1. pX (x) =
pXY (x, y) is the marginal probability function with respect to X
y|p(x,y)>0
2. pY (y) =
X
pXY (x, y) is the marginal probability function with respect to Y .
x|p(x,y)>0
Observe that the marginal probability functions recover the probability functions of X
and Y point-wise. Indeed, with notation as above,


X
X
X

pX (x) = 1
pXY (x, y) =
x∈ΩX
x∈ΩX
y|p(x,y)>0
since X is assumed to be a random variable. We will discuss the other features random
vectors expose below when we discuss covariance and the coefficient of correlation to examine
the independence of random variables. We shall also have occasion to consider such features
when generalizing about how the variance operator applies to linear combinations of discrete
random variables, a specific kind of statistic in light of the definition in this section.
3.5 WORKED EXAMPLES
3.6
The Expected Value E(X) and Variance V (X) of a Discrete
Random Variable X
In the previous section we formally defined both the expectation and variance of a random
variable via its generating function. This section is intended to convey how these aspects
of a random variable express intuitive features of an experiment. To this end, let us begin
by defining the expectation of a random variable in terms of the experiment to which it is
associated.
97
Definition 44. Let X be a random variable and (ΩX , βX , PX ) the induced probability space.
Then we say
X
E(X) =
xp(x)
x∈ΩX
is its expected value.
Of course this definition is that of the previous section, however, now it is intended
to be understood indirectly in terms of (Ω, β, P ) and directly in terms of (ΩX , βX , PX ).
To elucidate its intuitive meaning, let us consider the special case when ΩX = {x1 , x2 }.
We conceive of this special case as a "lever" with end points x1 and x2 in R and weights
p(x1 ), p(x2 ), respectively. There exists a fulcrum point E(X) ∈ R between x1 and x2 that
balances this lever, in the sense that d1 p(x1 ) = d2 p(x2 ), where d1 is the distance from x1 to
E(X) and d2 is the distance from x2 to E(X).
To find E(X) explicitly, we simply solve for it, as follows. By hypothesis, d1 = (E(X)−x1 )
and d2 = (E(X) − x2 ), so solving for E(X) gives
(E(X) − x1 )p(x1 ) =(x2 − E(X))p(x2 )
E(X)p(x1 ) + E(X)p(x2 ) =x1 p(x1 ) + x2 p(x2 )
x1 p(x1 ) + x2 p(x2 )
E(X) =
p(x1 ) + p(x2 )
where the denominator sums to 1 by Kolmogorov’s first axiom.
The upshot of this is that the fulcrum or balancing point on our "lever" ΩX has coordinate
E(X) = x1 p(x1 ) + x2 p(x2 ). We therefore generalize this illustration to an "arbitrary lever"
ΩX , as above in our definition, and interpret the expected value as the fulcrum point. A
more sophisticated perspective is to regard E(X) as the center of gravity for the weighted
particle system ΩX , but an adequate explanation of this perspective is beyond of the scope
of this text.
Next we intend to prove a famous linearity result, namely:
Theorem 25. Let both X and Y be random variables defined with respect to the discrete
probability space (Ω, β, P ), then
E(X ± Y ) = E(X) ± E(Y )
Now, to prove this theorem, observe that we can write the definition of the expectation
of Z in terms of the marginal probability functions by defining the statistic Z = f (X, Y ) =
98
X + Y in conjunction with the law of the unconscious statistician. Observe,
X
E(Z) =
zp(z)
z∈ΩZ
=
X
X
(x + y)pXY (x, y)
(x,y)∈ΩX ×ΩY
=
X X
x∈ΩX y∈ΩY
=
X
X X
xpXY (x, y) +
ypXY (x, y)
y∈ΩY x∈ΩY
xpX (x) +
x∈ΩX
X
ypY (y)
y∈ΩY
=E(X) + E(Y )
as desired. An immediate corollary is the following theorem.
Theorem 26. Let X1 , X2 , . . . , Xn be a sequence of P
random variables defined with respect
to the discrete probability space (Ω, β, P ) and Z = ni=1 Xi be the corresponding multiple
random variable, then
P
E(Z) = ni=1 E(Xi )
We continue with a few other easy properties, with the goal to collectively demonstrate
that E is a linear operator on the vector space of random variables defined with respect to
a discrete probability space.
Proposition 19. Let c ∈ R be a constant, then E(c) = c
The proof of this theorem is immediately from the definition of expectation.
Indeed,
P
treating the constant c as a constant random variable, we have E(c) = x∈Ωc xp(x), but
Ωc = {c} so that E(c) = c, since p(c) = 1, as desired. Next, to complete demonstrating the
linearity of E as an operator, we have the following theorem.
Proposition 20. Let X be a random variable and c ∈ R a constant, then E(cX) = cE(X).
Again, we may appeal to a statistic to prove this proposition. Define Z = f (X, c) = cX,
then we have, by definition
X
E(Z) =
zp(z)
z∈ΩZ
X X
=
X X
=
cxpX (x)
c
x∈ΩX
X
=
cxpcX (x, c)
c
x∈ΩX
cxpX (x)
x∈ΩX
=c
X
xp(x)
x∈ΩX
=cE(X)
as desired. So, we have the following corollary to the above results.
99
Theorem 27. Let X1 , X2 , . . . , Xn be a sequence of random variables defined with respect to
the discrete
probability space (Ω, β, P ), c1 , c2 , . . . , cn ∈ R a sequence of random variables, and
Pn
Z = i=1 ci Xi be the corresponding multiple random variable, then
P
E(Z) = ni=1 ci E(Xi )
that is, expected value is a linear operator.
Next we introduce the second significant aspect of a random variable invoked to understand experiments quantitatively, that of its variance. Our introduction follows from our
interpretation of E(X) as the average value of X. Indeed, according to this interpretation,
we shall measure how spread out the values of X are about E(X): this measurement is the
variance of X. Specifically, the manner in which we take this measurement is computing the
average size of the deviation, weighted by the likelihoods, from the expected value. To this
end, define for x ∈ ΩX its deviation from the mean to be the difference x − E(X). Then perhaps it is tempting to compute the average of the set of deviations from the mean to quantify
variation in an experiment. However, the following computation shows this endeavor would
be in vain:
X
E(X − E(X)) =
(x − E(X))p(x)
x∈ΩX
=
X
X
xp(x) −
x∈ΩX
E(X)p(x)
x∈ΩX
=E(X) − E(X)
=0
where the penultimate line follows from both the definition of expectation and factoring the
constant E(X) from the sum and using Kolmogorov’s axiom to simplify the remaining sum
as 1. Thus, to quantify variation in an experiment, we compute instead the average of the
set of deviations from the mean squared. This comment gives us the following definition.
Definition 45. Let X be a random variable and (ΩX , βX , PX ) the induced probability space.
Then we say
V (X) = E(X − E(X))2
X
=
(x − E(X))2 p(x)
x∈ΩX
is its variance. Furthermore more, with the same notation, we define the square root of the
variance to be the standard deviation of X, denoted σX
p
Observe that the average distance of X from its mean is |E(X−E(X))| = E(X − E(X))2 .
As the square root function is increase, this observation implies that a large value of V (X)
would indicate that the average distance of the values of X from its expected value to be large
whereas a small value would indicate that this average distance is small. Notice that this
distance is defined formally as the standard deviation, above. Experimentally, in plain language, the variance V (X) comports with our intuitive notion of variation in an experiment.
We shall now enumerate the properties of V (X).
100
Proposition 21. Let X be a random variable and V (X) its variance, then
V (X) = E(X 2 ) − (E(X))2
The proof is just an exercise in simplifying the definition. Notice that V (X) = E(X −
E(X))2 = E(X 2 − 2XE(X) + E(X)2 ) = E(X 2 ) − 2E(X)2 − E(X)2 = E(X 2 ) − (E(X))2 , as
desired. It is convenient in applications and proofs to remember this formula. Next, unlike
the expected value of a random variable, the variance is not a linear operator.
Proposition 22. Let X be a random variable and c ∈ R a constant. Then
1. V (c) = 0
2. V (cX) = c2 V (X)
Proving both items follows from the previous theorem with respect to the constant random variable, say X = c. Then we have V (c) = E(c2 ) − (E(c))2 = c2 − c2 = 0. Next, define
Y = cX, then by definition, V (Y ) = E(c2 X 2 ) − (E(c))2 = c2 E(X 2 ) − c2 (E(X))2 = c2 V (X),
as desired. In the next section we will discover what hypotheses are required so that V
behaves as an affine operator on a subspace of the vector space of linear operators defined
with respect to a discrete probability space.
3.6 WORKED EXAMPLES
1. Consider a Bernouli Trial, that is, a discrete probability space whose sample space
Ω = {s, f }, consists only of two simple events, such that P (s) = p ∈ [0, 1]. Notice,
P (f ) = q = 1 − p. Define the random variable X : Ω → R by X(s) = 1 and X(f ) = 0.
(a) Compute E(X)
(b) Compute V (X)
Solution (a) By definition, E(X) = 0p(0) + 1p(1) = p
Solution (b) By the law of the conscious statistician, we have
E(X 2 ) = 02 p(0) + 12 p(1) = p, so that by the variance theorem, we have
V (X) = p − p2 = p(1 − p) = pq
2. Consider a random variable X with sample space ΩX = {1, 2, 3, 4} and probability
1
3
2
4
function p(1) = , p(2) = , p(3) =
and p(4) =
10
10
10
10
(a) Compute E(X)
(b) Compute V (X)
101
Solution (a) Proceed by the definition to compute E(X)
Solution (b) Proceed by the theorem to compute V (X)
3. 4.5 Just after birth, each newborn is rated on a scale called the Apgar scale. The
possible ratings are 0, 1, . . . , 10 with the child’s rating determined by muscle tone,
heartbeat, and reflex irritability. Let X be the random variable whose value is a
child’s Apgar scale rating together with a probability function determined by the values: p(0) = 0.002 , p(1) = 0.001 , p(2) = 0.002 , p(3) = 0.005 , p(4) = 0.02 , p(5) =
0.04 , p(6) = 0.18 , p(7) = 0.37 , p(8) = 0.25 , p(9) = 0.12 and p(10) = 0.01.
(a) Compute the expected value of a child’s Apgar rating.
SolutionP(a) Proceed by the definition of expected value, where
E(X) = x∈ΩX xp(x). In our example ΩX = {0, 1, . . . , 10} by hypothesis, so in
P
particular, we have E(X) = 10
x=0 xp(x) = 0 · 0.002 + 1 · 0.001 + 2 · 0.002 + 3 · 0.005 +
4 · 0.02 + 5 · 0.04 + 6 · 0.18 + 7 · 0.37 + 8 · 0.25 + 9 · 0.12 + 10 · 0.01 = 7.15
4. An American roulette wheel has 38 spaces. The spaces are numbered from 1 through
36, 0, and 00. Half of the spaces labeled from 1 through 36 are black, whilst the other
half are red; the spaces labeled 0 and 00 are green. As one is able to bet upon the
occurrence of either black or red on an American roulette wheel, casinos will pay evenly
on such bets. So, for example, a successful 1 dollar bet upon red occurring will pay 1
dollar to the gambler. Let X be the random variable whose value is the pay off from
a 1 dollar bet on red.
(a) Compute E(X).
SolutionP(a) As usual, we proceed by definition. We have that
E(X) = x∈ΩX xp(x), so we must determine ΩX . We do so by inspection; indeed,
one either wins or losses a 1 dollar bet on red, so that ΩX = {−1, 1}. Next, assuming
that the roulette wheel is fair, we have by the relative frequency interpretation of
18
20
and p(1) = . Then, by definition,
probability that p(−1) =
38
38
20
18
−2
E(X) = (−1) ·
+ (1) ·
=
. The interpretation is that one expects to lose 5.26
38
38
38
on a 1 dollar bet on red.
5. Compute the following expectations E and variances V of linear combinations of the
given random variables, say {Xi }, and their values below, by the section’s results.
102
√
(a) Given E(X1 ) = 4, E(X2 ) = 3 and V (X2 ) = 3, compute E(πX1 + 2X2 ) and
V (7X2 )
√
2
(b) Given E(X1 ) = 0, E(X2 ) = and E(X3 ) = 3 and V (X1 ) = 0 V (X3 ) = 6,
3
compute E(eX1 + eX2 + eX3 ) and V (eX1 ) + V (5X3 ).
Solution √
(a) By the theorem, we
√ have
√
√
E(πX1 + X 3 ) = E(πX1 ) + E( 2X3 ) = πE(X1 ) + 2E(X3 ) = 3π + 3 2.
Furthermore, V (7X2 ) = 72 · 3
Solution (b) Again, by the theorem, we have E(eX1 + eX2 + eX3 ) =
2 √
E(eX1 ) + E(eX2 ) + E(eX3 ) = eE(X1 ) + eE(X2 ) + eE(X3 ) = e(0 + + 3). Next,
3
V (eX1 ) + V (5X2 ) = e2 · 0 + 52 · 6.
3.7
Independent Random Variables and the Coefficient of Correlation
In this section we explore further properties of the expected value and the variance. In
particular, under what condition is E multiplicative and V affine. The first condition is an
important property of sequences of random variables in its own right, and deserves to be
singled out by a definition. Furthermore, it is one of the conditions we need below to simplify
stochastic processes below.
Definition 46. Let X and Y be discrete random variables, Z = f (X, Y ) a joint random
variable, and pXY (x, y) their joint probability function. Then we say X and Y are independent if their joint probability function is separable, that is
pXY (x, y) = pX (x)pY (y)
for all (x, y) ∈ ΩZ
Under the auspices of this definition, we have the following proposition.
Proposition 23. Let X and Y be independent random variables. Then the following statements are true:
1. E(X · Y ) = E(X) · E(Y ) or E is multiplicative.
2. V (X + Y ) = V (X) + V (Y ) or V is affine.
103
We proceed by definition. Observe, for Z = X · Y , we have
E(X · Y ) =
=E(Z)
X
=
zp(z)
z∈ΩZ
=
X
X
=
X X
(xy)pXY (x, y)
(x,y)∈ΩX ×ΩY
xpX (x)ypY (y)
x∈ΩX y∈ΩY
=E(X) · E(Y )
by the law of the unconscious statistician, as desired. To demonstrate the second item, we
rely upon the truth of the first. Observe,
V (X + Y ) =E(X + Y )2 − (E(X + Y ))2
=E(X 2 + 2XY + Y 2 ) − (E(X) + E(Y ))2
=E(X)2 − (E(X))2 + E(Y )2 − (E(Y ))2 + 2E(XY ) − 2E(X)E(Y )
=V (X) + V (Y ) + 2E(X)E(Y ) − 2E(X)E(Y )
=V (X) + V (Y )
and therefore, V is affine.
Next, we generalize the independence of random variables by measuring their correlation.
When discrete random variables are uncorrelated, we say that they are independent in the
above sense. Consider the following measurement of how strongly two random variables are
related to one another.
Definition 47. Let X and Y be discrete random variables. Then we define their covariance
to be
CoV (X, Y ) = E ((X − E(X))(Y − E(Y )))
An interpretation of the covariance is that X −E(X) and Y −E(Y ) measure the deviation
of the variables from their expected values so that covariance is the expected value of the
product of their deviations. As such, in the special case when X = Y , we recover the variance
V (X) = Cov(X, X).
Proposition 24. Let X and Y be discrete random variables. Then
Cov(X, Y ) = E(XY ) − E(X)E(Y )
The proof is a practice in the definition of joint distribution together with the transfor-
104
mation of the expected value rule for discrete random vectors. Observe,
X X
Cov(X, Y ) =
(x − E(X))(y − E(Y ))p(x, y)
x∈ΩX y∈ΩY
X X
=
(xy − xE(Y ) − yE(X) + E(X)E(Y ))p(x, y)
x∈ΩX y∈ΩY
=
XX
(xy)p(x, y) −
x
y
+
XX
x
XX
x
xE(Y )p(x, y) −
y
XX
x
yE(X)p(x, y)
y
E(X)E(Y )p(x, y)
y
=E(XY ) − E(Y )
X
xpX (x) − E(X)
x
X
ypY (y) + E(X)E(Y )
y
=E(XY ) − 2 (E(X)E(Y )) + E(X)E(Y )
=E(XY ) − E(X)E(Y )
as desired. This proposition allows us to define the coefficient of correlation.
Definition 48. Let X and Y be discrete random variables. Then we say
ρXY =
Cov(X, Y )
σX σY
is the coefficient of correlation.
Notice that if X and Y are independent, then by the proposition ρXY = 0. However, if
ρ = 0, then that does not necessarily mean that X and Y are independent. Accordingly,
the converse of such a claim that uncorrelated variables are the same as independent ones is
false. Other relationships between X and Y are now appreciable that covariance has been
introduced.
Proposition 25. Let X and Y be discrete random variables and a, b, c, d ∈ R . Then the
following statements are true:
1. ρ(aX+b)(cY +d) = ρXY
2. −1 ≤ ρXY ≤ 1
3. ρXY = ±1, then Y = aX + b
We begin by recognizing that σaX+b = aσX since V (aX + b) = a2 V (X), etc. Therefore,
let us proceed by the proposition that simplifies the covariance formula
E((aX + b)(cY + d)) − E(aX + b)E(cY + d)
aσX cσY
acE(XY ) − ac(E(X)E(Y ))
=
acσX σY
Cov(X, Y )
=
σX σY
=ρXY
ρ(aX+b)(cY +d) =
105
Next, let us define η(t) = V (X)t2 + 2Cov(X, Y )t + V (Y ). Notice that η(t) ≥ 0 since
by unwinding the definitions of the coefficients shows that it can be written as the expected
value of a non-negative discrete random variable. As such, it has at most one real root.
Accordingly, its discriminant ∆ = 4Cov(X, Y )2 − 4V (X)V (Y ) ≤ 0. This is equivalent to the
inequality −σX σY ≤ Cov(X, Y ) ≤ σX σY or, by definition, −1 ≤ ρXY ≤ 1 as desired.
Last, suppose that ρXY = 1, then that implies ∆ = 0. This in turn implies η(t) has one
real root t0 of multiplicity two. This is true if and only if P ({(X − E(X))t0 + (Y − E(Y ))2 =
0}) = 1 which in turn means P ({Y = aX + b = 1}) = 1 or Y = aX + b with a = −t0
and b = E(X)t0 + E(Y ), ending the proof, for the case when ρXY = −1 is obtained mutatis
mutandi.
3.7 WORKED EXAMPLES
1. Suppose X1 , X2 , . . . , Xn are independent random variables.
(a) Show that for Y =
Pn
i=1 Xi that gY (z) =
Qn
i=1 gXi (z).
SolutionP(a) By definition
Q P
gY (z) = ΩX X ···Xn p(x1 , . . . , xn )z x1 +···+xn = ni=1 ΩX p(xi )z xi since the sum is
1 2
i
taken over ΩX1 X2 ···Xn and the Xi are independent by hypothesis.
2. Consider random variables X and Y with joint probability function and marginal
probability functions

(x, y) | (1, 1) (1, 2) (1, 3) (2, 1) (2, 2) (2, 3)


p(x, y) |
2
15
4
15
3
15
1
15
1
15


4 
15
(a) Compute E(X)
(b) Compute E(Y )
(c) Compute V (X)
(d) Compute V (Y )
(e) Compute ρXY
Solution
(a) First
 we must compute the distribution of X, which is

X
1
2
P
9
6
6


Next, we compute E(X) = 2x=1 xp(x) =
+2 =1+


9
6
15
15
15
p(x)
15 15
106
Solution (b) First we must compute the distribution of Y , which is

Y
1
2
3


Next, we compute

3
5
7
p(y)
15 15 15
P
3
5
7
4
E(Y ) = 3y=1 yp(y) =
+2 +3 =2+
15
15
15
15
Solution (c) We proceed by the formula V (X) = E(X 2 ) − [E(X)]2 . So, we have by
the law of the unconscious statistician that
P
6
3
33 441
54
9
+ 22 = 2 + . Therefore, V (X) =
−
=
E(X 2 ) = 2x=1 x2 p(x) =
15
15
15
15 225
225
Solution (d) We proceed by the formula V (Y ) = E(Y 2 ) − [E(Y )]2 . So, we have by
the law of the unconscious statistician that
P
3
5
7
11
E(Y 2 ) = 3y=1 y 2 p(y) =
+ 22 + 32 = 5 + . Therefore,
15
15
15
15
86 1156
134
V (Y ) =
−
=
15
225
225
Solution (e) First, we must compute Cov(X, Y ). Proceeding by definition, we have
XX
(X − E(X))(Y − E(Y ))p(x, y)
Cov(X, Y ) =
ΩXY
=
=
3
X
2
X
y=1
x=1
!
(X − E(X))(Y − E(Y ))p(x, y)
3 X
171
y=1
396
(Y − E(Y ))p(1, y) +
(Y − E(Y ))p(2, y)
225
225
171 91
171 316
171 541
p(1, 1) +
p(1, 2) +
p(1, 3)
225 225
225 225
225 225
396 316
396 541
396 91
p(2, 1) +
p(2, 2) +
p(2, 3)
+
225 225
225 225
225 225
171 91 2
171 316 4
171 541 3
=
+
+
225 225 15 225 225 15 225 225 15
396 91 1
396 316 1
396 541 4
+
+
+
225 225 15 225 225 15 225 225 15
1407951
=
155
= 1.81
=
Therefore, ρXY =
1.81
(0.77) · (0.48)
3. Consider random variables X and Y with joint probability function and marginal
probability functions
107


p(x, y) 500 1000 5000 Y
 100

.3
.05
0



 500
.15
.2
.05


 1000

.1
.1
.05
X
(a) Compute p(x)
(b) Compute p(y)
(c) Compute E(X)
(d) Compute E(Y )
(e) Compute Cov(X, Y ) by the definition.
X
Solution (a)
p(x)
Y
Solution (b)
p(y)
100 500 1000
.35 .4
.25
500 1000 5000
.55 .35
.1
Solution (c) E(X) = 100(.35) + 500(.4) + 1000(.25) = 485
Solution (d) E(Y ) = 100(.55) + 500(.35) + 1000(.1) = 1125
Solution (e) P
Proceeding
by definition, we have
P
(X
− E(X))(Y − E(Y ))p(x, y) =
Cov(X, Y ) =
ΩXY
(100 − (485)(500 − 1125)(.3) + . . . + (1000 − 485)(5000 − 485)(.05) = 136, 875 where
we are summing over the nine ordered pairs (x, y) ∈ ΩXY .
4. Suppose X and Y are independent random variables.
(a) Prove Cov(X, Y ) = 0
(b) Prove ρXY = 0
Solution (a) Proceed by the theorem to write Cov(X, Y ) = E(XY ) − E(X)E(Y )
which in turn is 0 because, by the independence hypothesis, E(XY ) = E(X)E(Y ), as
desired.
Solution (b) By the result in (a) we have ρXY =
0
= 0.
σX σY
5. Suppose X and Y are random variables.
(a) Prove V (aX + bY ) = a2 V (X) + b2 V (Y ) + 2abCov(X, Y )
108
(b) Suppose further that X and Y are independent random variables, prove that
V (X + Y ) = V (X) + V (Y ).
Solution (a) Proceed by the definition and write
V (aX + bY ) = E(aX + bY − E(aX + bY ))2 . Simplifying, we have
E(aX + bY − E(aX + bY ))2 =E(a(X − E(X)) + b(Y − E(Y )))2
=E(a2 (X − E(X))2 + b2 (Y − E(Y ))2 + 2ab(X − E(X))(Y − E(Y )))
=a2 V (X) + b2 V (Y ) + 2abCov(X, Y )
by the linearity of E, as desired.
Solution (b) By (a) We have V (X) + V (Y ) + 2Cov(X, Y ) which equals
V (X) + V (Y ) since Cov(X, Y ) = 0 by the independence hypothesis, as desired.
6. 4.6 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
7. 4.6 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
109
Solution (c) C.
8. 4.6 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
3.8
Chebyshev’s Inequality
In this section we consider how the variance of a random variable determines the likelihood
that the values of a random variable are far from its expected value. Indeed, according to
our experimental interpretation of the expected value of a random variable as the center of
gravity of a system of particles the intuition that a typical particle should be far away from
the system’s center of gravity is unlikely is made precise by Chebyshev’s inequality.
Let us consider a discrete random variable X such V (X) exists and t ∈ R≥0 . Consider
the probability of the equivalence class P ({|X − E(X)| ≥ t}). The likelihood of this event
measures how likely it is that X is at least t away
value. By hypothesis,
P from its expected
2
V (X) exists and by definition we have V (X) = x∈ΩX (X − E(X)) . Accordingly, we have
V (X) =
X
x∈ΩX ||x−E(X)|≥t
V (X) ≥
X
X
(X − E(X))2 p(x) +
(X − E(X))2 p(x)
x∈ΩX ||x−E(X)|<t
(X − E(X))2 p(x)
x∈ΩX ||x−E(X)|≥t
V (X) ≥
X
t2 p(x) = t2 P ({|X − E(X)| ≥ t})
x∈ΩX ||x−E(X)|≥t
The final line yields the following famous result, called Chebyshev’s inequality.
Theorem 28. Let X be a discrete random variable such that V (X) exists. Then
P ({|X − E(X)| ≥ t}) ≤
110
V (X)
t2
Notice that, as t → ∞, then P ({|X − E(X)| ≥ t}) = 0, by Chebyshev’s inequality.
This result confirms the intuition proffered by conceptualizing induced probability spaces as
particle systems whose particles are the values of a random variable. In other words, the
likelihood of the event a value of X is arbitrarily far from its expected value is negligible.
We can strengthen this intuition by making a significant substitution.
Corollary 3.1. Let X be a discrete random variable such that V (X) exists and c ∈ R≥0 .
Then
1
P ({|X − E(X)| ≥ cσX }) ≤ 2
c
The corollary is obtained from Chebyshev’s inequality by substituting t = cσX . Its
relevance is that it measures the likelihood of a value of X being far from its expected value
in terms of the standard deviation. In particular, it says that the likelihood of this event is
bounded above by the inverse square of the number of standard deviations away from the
expected value X is. Clearly, as c → ∞, we have P ({|X − E(X)| ≥ cσX }) → 0.
An corollary of this result follows from De Morgan’s law. Specifically, we take as a
corollary to Chebyshev’s Inequality the following result.
Proposition 26. Let X be a discrete random variable such that V (X) exists. Then
P ({|X − E(X)| ≤ t}) ≤ 1 −
V (X)
t2
Notice that {|X − E(X)| ≥ t}c = {|X − E(X)| ≤ t} Thus P ({|X − E(X)| ≤ t}) =
V (X)
P ({|X − E(X)| ≥ t}c ) = 1 − P (({|X − E(X)| ≥ t}) ≤ 1 − 2 by Chebyshev’s inequality,
t
as desired.
3.8 WORKED EXAMPLES
1
6
1
1. 4.7 Let X be a random variable with distribution ΓX = {(−1, ), (0, ), (1, )}.
8
8
8
(a) Compute E(X) and V (X).
(b) Compute an upper bound for the probability that X is two standard deviations
away from its expected value.
1
Solution (a) By definition, we have E(X) = 0 and V (X) = .
4
Solution (b) By Chebyshev’s Inequality we have this upper bound is equivalent to
1
P (|X|) ≥ 1) ≤
4
1
2. 4.7 The corollary of Chebyshev’s Inequality can be interpreted as 1 − 2 · 100 % of
c
the elements of ΩX are within c − σX ’s of E(X).
111
(a) Given the induced probability space (ΩX , βX , PX ) such that |ΩX | = 10, how
many of its elements are within of 2 standard deviations of E(X)?
(b) How many standard deviations are 99 % of its elements from E(X)?
1
· 100
Solution (a) By Chebyshev’s inequality, we have c = 2 are within 1 −
4
%=75 % standard deviations from its expected value. Therefore, 7.5 of its elements.
1
Solution (b) We must solve for c in 99 % = 1 − 2 · 100 %. Plainly c = 10.
c
3. 4.7 Consider a set consisting of the IQ scores of students as the image of a discrete
random variable X. Its elements are real numbers between 0 and 200. It is determined
from this set that E(X) = 100 and σX = 16.1.
(a) Determine the minimum percentage of students who have IQ scores within 3
standard deviations of the expected value by Chebyshev’s inequality.
(b) Determine the minimum percentage of students who have IQ scores between
67.8 and 132.2 by Chebyshev’s inequality.
1
· 100 %=88 % are
Solution (a) Proceed by the corollary with c = 3 so that 1 −
9
within 3 standard deviations of 100=E(X).
Solution (b) Observe that we must solve 100 − c1 (16.1) = 67.8 for c1 and
100 − c2 (16.1) = 132.2 for c2 . In both cases, we have c1 = c2 = 2, therefore, the
minimum percentage of students who have IQ scores between 67.8 and 132.2 is 75 %.
3.9
Chapter 3 Homework Exercises
1. The graph of a probability function of a discrete random variable X is given by the
following values:
p(0) = .54
p(1) = .16
p(2) = .06
p(3) = .04
p(4) = .20
112
i) Compute E(X).
ii) Compute V (X).
2. The graph of a probability function of a discrete random variable X is given by the
following values:
p(1) = .05
p(2) = .10
p(3) = .12
p(4) = .30
p(5) = .30
p(6) = .11
p(7) = .01
p(8) = .01
i) Compute E(X).
ii) Compute V (X).
iii) Compute p(x ≤ 5)
iv) Compute p(x > 3)
3. The graph of the probability function of a discrete random variable X is given by the
values:
p(0) = .073
p(1) = .117
p(2) = .258
p(3) = .322
p(4) = .230
i) Compute E(X).
ii) Compute V (X).
4. Suppose that X1 , X2 , and X3 are independent random variables. Suppose further that
E(X1 ) = 4
E(X2 ) = 5
E(X3 ) = 7
113
and
V (X1 ) = 4
1
V (X2 ) =
2
√
V (X3 ) = 2
i.) Compute E(4X1 + πX2 + eX3 + 13)
ii.) Compute E(X1 X3 + X2 )
√
√
iii.) Compute V ( 2X2 + 3X3 + 17)
114
4
Parametric Families of Discrete Random Variables
In this chapter we introduce the most famous parametric families of discrete random
variables. Elements of these families are determined by the selection of parameter values,
which in turn are themselves determined by the specifics of an experiment. These examples
arise in the literature according to the ubiquity of their applications and are in many respects
the extent to which the general public is familiar with probability theory altogether. We,
however, can view at least a few of them as an introduction to stochastic processes which
we will introduce in a generality appropriate to this textbook in chapter 6 in order to bridge
the gap between the probability theory of this textbook and the algebraic interpretation
of Markov processes commons in applications. The introduction to stochastic processes we
have in mind are so-called Bernoulli processes, defined below. Beside this, they serve as
powerful testaments to the wide applications of discrete probability theory in society.
4.1
Bernoulli Processes
In this section we introduce a discrete measure space that depends upon discrete values of
time. This seemingly innocent hypothesis-that of a time dependence in the experiment-shall
introduce an entirely new class of probability spaces from which we will construct Markov
chains in chapter 6. It is remarkable how they are already applicable to parametric families
of random variables. Below we shall say a measure space is a pair (Ω, β) consisting of a
countable set Ω and a σ-algebra, β. Moreover, we will still refer to Ω as the sample space of
a measure space and its elements as simple events. One defines random variables on measure
spaces exactly the same way they are defined on discrete probability spaces. In particular,
a random variable X defined on a measure space is a real-valued function X : Ω → R such
that {X = x} ∈ β for each x ∈ ΩX .
We begin by defining a Bernoulli trial to be a discrete probability space (B, β = 2B , P )
where B = {s, f } consists of two mutually exclusive simple events, referred to informally as
success and failure, respectively, and the assignment P (s) = p ∈ [0, 1]. The last hypothesis
implies that P (f ) = 1 − p = q, since P is a probability function by hypothesis. It is trivial
to show that a Bernoulli trial is a probability space. Bearing this definition in mind, we next
work to define the ambient measure space required for several of the parametric families
later in this chapter.
Definition 49. We say a Bernoulli process is a measure space (ΩB(p) , β) consisting of the
Cartesian product ΩB = ×∞
i=1 Bi of the elements of a sequence of Bernoulli trials {Bi | Bi =
{s, f } ∀i }∞
,
where
B
is
independent of Bj for any i ̸= j. Furthermore, we insist that
i
i=1
P (s) = p and P (f ) = q for fixed p. Last, we say Bi is performed at time i. We take
β = 2ΩB(p) .
This definition is sound, for the countable Cartesian product of countable sets is countable. Therefore, the sample space is a countable set and so, its power set has a σ-algebra
structure. Beside this remark, we shall adopt a somewhat informal approach to representing
simple events in a Bernoulli process following this remark. In the next chapter we shall undertake a more precise formulation of our manner of representing simple events. The reason
to postpone a more technically accurate representation is to utilize the famous parametric
115
families below as both motivations for and introduction to stochastic processes without the
additional burden of these details.
A simple event in a Bernoulli process is a tuple of successes and failures in ΩB . One may
visualize a simple event as a branch in the countably infinite binary tree determined by the
Bernoulli process in the sense of chapter 2. Specifically, we introduce binary trees which are
graphs T such that, for any vertex of a fixed height, there are exactly two edges emanating
from it. Binary trees begin at a root which is a vertex with no predecessor. The height of
a vertex will be its distance from this root measured in the least number of edges joining
it to the same. The branches in a binary tree, which are paths beginning at the root and
containing exactly one edge for each vertex at a given height, represent the simple events in
a Bernoulli process by labeling the vertex of height i with the label in the i-th component
of the simple event. Note that the height of the root is 0. Furthermore, we shall treat the
concatenation of vertex labels as the word representing the branch.
Given a Bernoulli process (ΩB(p) , β) the following binary tree to represents the above
prescriptions:
s
s
s
s
f
f
s
s
f
f
r
···
· · · time n
f
f
f
s
s
s
f
s
f
s
So, for example, up until time n, that is, when Bn is performed at time n following the
· · · the uppermost branch is represented by the word sss · · · ss.
Let us now explore how this measure space can be embedded as a discrete probability
space in our naive category of the same. So, consider an element of a Bernoulli process
up until time n, say, sf s · · · f s, consisting of x successes. Then this word can be taken to
represent the simple event s ∩ f ∩ s ∩ · · · f ∩ s in an experiment whose events are comprised
of performing n Bernoulli trials in sequence. That is, an experiment where there are only
two mutually exclusive outcomes at each step. Accordingly, we interpret the vertex labels
to mean that s occurred when B1 was performed at time 1 and f occurred when B2 was
performed at time 2 and s occurred when B3 was performed at time 3 and · · · and s occurred
when Bn was performed at time n. We extend this interpretation to arbitrary simple events
in the obvious way up until time n. Interpreting a word as a joint event therefore entails
116
that the probability of this simple event is given by the joint probability formula of chapter
3, that is
P (sf s · · · s) =P (s ∩ f ∩ s ∩ · · · f ∩ s)
=P (s)P (f | s)P (s | s ∩ f ) · · · P (s | s ∩ f ∩ s ∩ · · · ∩ f )
=P (s)P (f )P (s) · · · P (f )P (s)
=px q n−x
The second line on the right hand side is the general probability of the intersection of a sequence of events discussed in chapter 3 and the third line is the consequence of the hypothesis
that the Bernoulli trials performed at each time value in the sequence are independent: the
last line is obtained by the hypothesis that there are x successes in this particular simple
event. Although we have not provided it yet, this value for P on β determines a probability
function. We shall appeal to this observation when computing the probability of simple
events in induced probability spaces predicated on Bernoulli processes below as well as for
how to interpret elements of a Bernoulli process experimentally.
4.2
The Binomial Random Variable B(n, p)
In this section we introduce the most well-known parametric family of discrete random
variables, that of the Binomial random variable, denoted B(n, p) below. To this end, let us
consider a Bernoulli process of time n, that is, a Bernoulli process whose simple events all
have exactly height n. In the notation of the previous section we write (ΩB(n,p) , β) for this
process. The corresponding binary tree is
s
sss · · · sf
s
f
sss · · · ss
s
f
f ss · · · sf
s
f ss · · · ss
s
s
f
s
s
f
f
···
r
f
f
f
s
s
f
s
where the right most column in the above diagram enumerates the words representing simple
events in (ΩB(n,p) , β). Thus simple events are represented by words of length n consisting of
x successes and therefore n − x failures, for each x ∈ [n].
117
We define the following discrete random variable B(n, p) : ΩB(n,p) → R on a Bernoulli
process of time b by the assignment b 7→ x where x is the number of successes in the word
representing b ∈ ΩB(n,p) . It is clear that B −1 (n, p)(x) ∈ β for each x since the fibre consists
of simple events that contain x successes. We interpret this value experimentally as the
number of successes that occur in the performance of n Bernoulli trials sequentially. We
have the following theorem.
Theorem 29. Let (ΩB(n,p) , β) be a Bernoulli process of time n. Define
B(n, p) : ΩB(n,p) → R
by B(n, p)(b) = x, where x ∈ [n] is the number of successes in the word representing b. Then
(ΩB(n,p) , βB(n,p) , pB (x)) is a discrete probability space and
pB (x) = nx px q n−x
First we observe that ΩB(n,p) = [n] and βB(n,p) = 2[n] , so to prove the theorem we simply
must show that pB (x) satisfies Kolmogorov’s axioms. We provide the formula for pB (x) of
the theorem proceeding by the definition p(xi ) = P ({X = xi }) of chapter 3.2, then show the
probabilities it assigns to simple events satisfy the conditions of theorem 1.3 in order to show
pB (x) is a probability function. This will complete the proof that the induced probability
space of B(n, p) is a discrete probability space.
Let us determine
probabilities px = pB (x) for the simple events x ∈ ΩB(n,p) such that
P
px ∈ [0, 1] and
px = 1. These px are the probabilities of the fibres of B(n, p) by chapter
3. Thus, consider {B(n, p) = x}, then by the definition of chapter 3.2, we have
pB (x) =
X
P ({B(n, p) = x}) =
px q n−x
b∈{B(n,p)=x}
where px q n−x is the probability of the simple event b ∈ ΩB(n,p) consisting of x successes and
n − x failures, by the previous section. We can complete the computation by computing the
size of the summation index as the summand values are constant for a fixed x ∈ [n].
Observe that {B(n, p) = x} is the set of words b in (ΩB(n,p) , β) consisting of x successes
and n − x failures. These words are determined by selecting, without replacement and in no
order, x successes for x of the components and n − x failures in the remaining components
of the tuple represented by the word b ∈ {B(n, p) = x}. As such, these words b are
x-combinations
of the n-components of the tuple represented by b. Therefore, there are
n
=
|{B(n,
p)
= x}| words in the fibre. This gives the summation index, so canonical
x
probability formula simplifies to
n x n−x
px =
p q
x
recalling px = pB (x) by hypothesis.
118
Observe, px ≥ 0 for x ∈ [n] since each factor in its expression is positive. Furthermore,
n
X
n X
n x n−x
px =
p q
x
x=0
x=0
=(p + q)n
=1
by the Binomial theorem of chapter 2. Thus px ∈ [0, 1] and so the px for x ∈ [n] satisfy the
requirements of theorem 1.3, concluding our proof.
Definition 50. We say B(n, p) is the Binomial random variable. We say that both n and
p are its parameters, where n is the number of Bernoulli trials performed in an experiment
and p is the probability of success in a Bernoulli trial.
It is more accurate to refer to B(n, p) as a parametric family of discrete random variables,
for there is one random variable corresponding to each pair of parameter values (n, p). However, we abuse language by referring to B(n, p) as the Binomial random variable to remain
consistent with the broader literature.
Next we shall compute both the expected value and variance of B(n, p) by the formal
method in chapter 3. First, let us compute the generating function of B(n, p). We have
n X
n x n−x x
gB (z) =
p q z
x
x=0
n X
n n−x
=
q (pz)x
x
x=0
=(pz + q)n
by the Binomial theorem.
Recall E(B(n, p)) = gB′ (1), then observe that
gB′ (1) =n(p(1) + q)n−1 p
=np(1)n−1
=np
by the chain rule and the hypothesis that p + q = 1.
Recall that the variance of a discrete random variable can also be computed formally.
We have
V (B(n, p)) = gB′′ (1) + gB′ (1) − (gB′ (1))2
which is equal to gB′′ (1) + np − n2 p2 by the above argument. Let us finish the computation
by evaluating gB′′ (1). Indeed, gB′′ (1) = n(n − 1)p2 (p(1) + q)n−2 = (np)2 − np2 . Substituting
our work into the formula for V (B(n, p)) we have V (B(n, p)) = (np)2 − pp2 + np + (np)2 =
np(1 − p) = npq. Collectively, these computations prove the following proposition which is
a summary of our work.
119
Proposition 27. Let B(n, p) be the Binomial random variable, then its generating function
is given by
gB (z) = (pz + q)n
and
1. E(B(n, p)) = np
2. V (B(n, p)) = npq
4.2 WORKED EXAMPLES
1. Let B(n, p) be the Binomial random variable with parameter values n = 16 and p = 0.6.
(a) Compute both E(B(16, 0.6)) and V (B(16, 0.6)).
(b) Compute pB (4), that is, the probability of exactly 4 successes.
(c) Compute pB (7 ≤ x ≤ 10), that is, the probability of between 7 and 10 successes.
(d) Compute pB (x ≥ 1), that is, the probability of at least 1 success.
Solution (a) By the proposition, both the expected value and variance can be
computed in terms of the parameters n and p. Therefore, we have
E(B(16, 0.6)) = 16 · 0.6 = 9.6 and V (B(16, 0.6)) = 16 · 0.6 · 0.4 = 3.84.
Solution (b) By the theorem, we have pB (x) may be computed in terms of the
parameters n and p. In particular, we have, for x = 4 that
pB (4) = 16
(0.6)4 (0.4)12 ∼
= 0.003957
4
Solution (c) Observe that
pB (7 ≤ x ≤ 10) = pB (x = 7 OR x = 8 OR x = 9 OR x = 10). By chapter 4, we
recognize that these values for x correspond to cells of a partition of β, so as such, we
have,
that pB (x = 7 OR x = 8 OR x = 9 OR x = 10) =
P10 by Kolmogorov’s
P10 16axioms
x
16−x ∼
= 0.612841
x=7 pB (x) =
x=7 x (0.6) (0.4)
Solution (d) Observe that pB (x ≥ 1) = 1 − pB (0) since 0 is the complement of the
discrete interval x ≥ 1 in [16] = ΩB(16,0,6) . Therefore,
(0.6)0 (0.4)16 ∼
pB (x ≥ 1) = 1 − 16
= 0.999983
0
2. Let B(n, p) be a Binomial random variable and B ′ (n, q) be the corresponding "failure"
Binomial random variable, as the success parameter q = 1 − p, which is the probability
of failure in B.
(a) Show that PB (x) = pB ′ (n − x)
120
Solution (a) We proceed by the symmetry of the binomial coefficient and the
definition of the probability functions associated to B and B ′ . Observe,
n x n−x
pB (x) =
p q
x
n
=
q n−x pn−(n−x)
n−x
=pB ′ (n − x)
as desired. The second line follows from the symmetry of the binomial coefficient
exhibited in chapter 2. A useful remark is that one really only needs to consider
B(n, p) for p ∈ [0, 0.5], as one get understand B(n, p) for p > 0.5 as B ′ defined above.
After all, at most x s’s is equivalent to at least n − x f ’s.
3. Studies indicate that 1 in 20 children have a food allergy. Consider selecting a random
sample of 25 children and testing them for food allergies.
(a) Compute the probability that at most 3 children in the random sample have
food allergies.
(b) Compute the probability that at least 4 children in the random sample have
food allergies.
(c) What is the expected value of children with food allergies in this sample?
(d) If the sample size is increased to 50, what is the probability no children have
food allergies.
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
Solution (d) D.
4. Suppose 2 % of the people taking a test require extra time to answer its questions.
Consider a random sample of 25 test takers.
(a) What is the probability one of the test takers requires extra time?
(b) What is the probability at least 2 test takers require extra time?
(c) Suppose a person who does not require extra time to complete the test takes 3
hours to answer its questions, whereas a person who does requires 4.5 hours.
What is the average time one should expect for the random sample of 25 people
to complete the exam be?
121
Solution (a) A.
Solution (b) B.
Solution (c) C.
5. A toll bridge charges 2 USD for cars and 3 for other vehicles. Suppose during nighttime
hours, 60 % of the vehicles that pass over the bridge are cars. Suppose further that 25
vehicles cross the bridge during the midnight hour.
(a) What is the expected value of the toll bridge revenue?
Solution (a) Let r(x) = expected bridge revenue. Observe, this is a linear function
of x, the number of cars.
6. 5.2 Compare the upper bounds furnished by Chebyshev’s theorem to B(20, 0.5) for
c = 2, 3.
Solution (a) Observe that for B(20, 0.5), we have E(B(20, 0.5)) = 10 and
V (B(20, 0.5)) = 5 so that Chebyshev’s theorem gives
√
1
P ({|B(20, 0.5) − 10| ≥ 2 5) ≤ for c = 2. Similarly,
4
√
1
P ({|B(20, 0.5) − 10| ≥ 3 5) ≤ for c = 3. Apparently the likelihood of being 3
9
standard deviations away from the expected value decreases by 56 % as we increase
from 2.
4.3
The Negative Binomial Random Variable Q(r, p)
In this section we consider another parametric family of random variables predicated upon
a Bernoulli process, that of the Negative Binomial random variable, denoted Q(r, p) below.
To prove its induced probability is a discrete probability space, we require the following
theorem, known as the negative Binomial theorem. The Negative Binomial random variable
Q(r, p) is thus named for its reliance upon this result. One demonstrates this result by
computing the MacLaurin series expansion of f (x) = (1 + x)−r .
Theorem 30. Let r be a positive integer. Then
P∞ r+t−1
1
=
(−1)t xt
t=0
t
(1 + x)r
for |x| < 1
122
Let (ΩB(p) , β) be a Bernoulli process and define the following discrete random variable
Q(r, p) : ΩB → R for b ∈ ΩB(p) by the formula b 7→ t, where t is the number of failures
until there are r successes in the word representing b ∈ ΩB(p) reading from left to right. We
interpret this simple event experimentally as the number of Bernoulli trials that must be
performed sequentially in an experiment to obtain r successes among them. We have the
following theorem.
Theorem 31. Let (ΩB(p) , β) be a Bernoulli process. Define
Q(r, p) : ΩB(p) → R
by Q(r, p)(b) = t, where 0 ≤ t is the number of failures in the word representing b until
there are r successes reading it from left to right. Then (ΩQ(r,p) , βQ(r,p) , pQ (t)) is a discrete
probability space, where
r t
pQ (t) = r+t−1
pq
t
First we observe that ΩQ(r,p) = N and βQ(r,p) = 2N , so to prove the theorem we simply
must show that pQ (t) satisfies Kolmogorov’s axioms, since the power set of a countable set
has the structure of a σ-algbra. We provide the formula for pQ (t) of the theorem proceeding
by the definition of 3.2, then show the probabilities it assigns to simple events satisfy the
conditions of theorem 1.3 in order to show pQ (t) satisfies Kolmogorov’s axioms. This will
complete the proof that the induced probability space of Q(r, p) is a discrete probability
space.
P Let us determine probabilities pt = pQ (t) for the simple events t ∈ ΩQ(r,p) such that
pt = 1. This will prove pQ (t) satisfies Kolmogorov’s axioms as an application of the
canonical formula in chapter 1. These pt are the probabilities of the fibres of Q(r, p) by
chapter 3. Thus, consider {Q(r, p) = t}, the fibre of Q(r, p) over t ∈ ΩQ(r,p) . Then, by the
definition of chapter 3.2, we have
pQ (t) =
P ({Q(r, p) = t}) =
X
pr q t
b∈{Q(r,p)=t}
where pr q t is the probability of the simple event b ∈ ΩB(p) consisting of t failures until there
are r successes reading from left to right in b. As the probability of a simple event is constant,
we can complete the computation by computing the size of the summation index for fixed
r ∈ N.
Observe that |{Q(r, p) = t}| is the number of t-un-orderings of the r-successes in the
sequence corresponding to b. Indeed, treat the successes as the bars of an un-ordering, then
the t failures are treated as the dots of the same, distributed among those bars, for they
are selected with replacement allowed and in no order for the unordering of stars and dots
corresponding to b. Hence, any b with r fixed successes and t failures corresponds to a tun-ordering of the r successes. Thus, |{Q(r, p) = t}| = r+t−1
by chapter 2. Therefore, we
t
have by 4.2
r t
pQ (t) = r+t−1
pq
t
123
Observe, pt ≥ 0 for t ∈ N since each of the factors is positive. Furthermore,
X
X r + t − 1
pt =
pr q t
t
t∈N
t∈N
∞
X
r+t−1 r t
=
pq
t
t=0
∞ X
r+t−1
r
(−1)2t q t
=p
t
t=0
=pr (1 − q)−r
=pr p−r
=1
by the Negative Binomial theorem stated above. Thus the pt for t ∈ N satisfy the requirements of theorem 1.3, concluding our proof.
Definition 51. We say Q(r, p) is the Negative Binomial random variable. We say that both
r and p are its parameters, where r is the number of successes and p is the probability of
success in a Bernoulli trial.
Next we shall compute both the expected value and variance of Q(r, p) by the formal
method in chapter 4. Observe, the generating function of Q(r, p) is given by
gQ (z) =
∞ X
r+t−1
t
t=0
=
∞ X
r+t−1
t
pr q t z t
pr (qz)t
t=0
∞ X
r+t−1
r
=p
t
t=0
(qz)t
1
=pr
(1 − qz)r
r
p
=
1 − qz
′
Recall that E(Q(r, p)) = gQ
(1), then we have
′
gQ
(1) =r
=
p
1 − q(1)
r−1
p(−1)(1 − q(1))−2 (−q)
rq
p
′
To compute the variance, we have the straightforward computations gQ
(z) =
124
pr rq
(1 − qz)r+1
′′
and gQ
(z) =
pr q 2 r(r + 1)
. It follows from chapter 4 that
(1 − qz)r+2
2
′
′
′′
(1)
(1) − gQ
(1) + gQ
V (Q(t, r)) =gQ
q 2 r(r + 1) rq r2 q 2
− 2
+
p2
p
p
q 2 r2 + q 2 r + rpq − r2 q 2
=
p2
rq(q + p)
=
p2
rq
= 2
p
=
since (p + q) = 1. Collectively, these computations prove the following proosition which is a
summary of our work.
Proposition 28. Let Q(r, p) be the Negative Binomial random variable, then its generating
function is given by
r
p
gQ (z) =
1 − qz
and
1. E(Q(r, p))) =
rq
p
2. V (Q(r, p)) =
rq
p2
Next we consider a transformation of Q′ (r, p) = f (Q(r, p)), where prima facie Q′ (r, p)(b) =
t is the number of Bernoulli trials until the r-th success in the word b reading from left
to right. Notice we are abusing the notation for t. The transformation f is given by
Q′ (r, p)(t) = Q(r, p)(t)+r. The following corollaries to both the theorem and the proposition
express their results in terms of this transformation.
Corollary 4.1. Let (ΩB(p) , β) be a Bernoulli process . Define
Q′ (r, p) : ΩB(p) → R
by Q′ (r, p)(b) = t ≥ r, is the number of Bernoulli trials until the r-th success in the word b
read from left to right. Then (ΩQ′ (r,p) , βQ′ (r,p) , pQ′ (t)) is a discrete probability space, where
t−1 r t−r
pQ′ (t) = r−1
pq
One notes that ΩQ′ (r,p) = {r, r + 1, . . . | r ∈ N}. We will not verify Kolmogorov’s axioms
in this case, however, we emphasize that it satisfies the axioms because Q′ is a transformation
125
of Q. Naively, substituting t = t − r, we have
r−1+t
r − 1 + (t − r)
=
t
t−r
(t − 1!)
=
(t − r)!(t − 1 − (t − r))!
(t − 1!)
=
(t − r)!(r − 1)!
(t − 1!)
=
(r − 1)!(t − 1 − (r − 1)!
t−1
=
r−1
t−1 r t−r
so that pQ′ (r,p) (t) = r−1
p q . As Q′ is a transformation of Q, these computations are
justified because in general, a transformation entails that P ({Y = y}) = P ({f (X) = y}) =
P ({X = f −1 (y)}). In this case, f −1 (Q) = Q′ − r.
Next we have both the expected value and variance of Q′ are obtained from the expected
value and variance of Q by the law of the unconscious statistician.
Corollary 4.2. Let Q′ (r, p) be the Negative Binomial random variable, then
1. E(Q′ (r, p)) =
r
p
2. V (Q′ (r, p)) =
rq
p2
Indeed, since Q′ (r, p) = Q(r, p) + r and E(aX + b) = aE(X) + b then by the law of the
unconscious statistician, we have
rq
+r
p
rq + rp
=
p
r
=
p
E(Q′ (r, p)) =
Similarly, since in general V (aX +b) = a2 V (X) and Q′ is a translation of Q by a constant,
V (Q(r, p)) = V (Q′ (r, p)), as desired.
4.3 WORKED EXAMPLES
1. Suppose a machine produces a single part and that, after production, the part it
9
produces is labeled either non-defective or defective. Let
be the probability that
10
the part produced is defective. An inspector observes parts produced by the machine
until they see 4 defective parts produced.
126
(a) What is the probability they will observe 7 parts produced in order to observe 4
defective parts made?
(b) What is the expected number of parts produced to observe 4 defective parts?
(c) What is the variance of the expectation in part (b)?
Solution (a) If we regard the production of a defective part as a success in this
experiment, then we may apply the negative binomial random variable Q′ with
9
parameter values r = 4 and p = . Therefore, the probability of observing t = 7
10
parts produced
to
witness
r
=
4
defective parts is
4 3
6
9
1
pQ′ (7) =
= 0.078
3
10
10
Solution (b) Proceeding by the observation in part (a), we have
40
4
9
=4+
E(Q′ (4, )) =
10
9
9
Solution (c) Again, proceeding by the observation in part (a), we have
9
40
V (Q′ (4, )) =
10
81
2. Consider an exit poll after an election. You are polling to determine whether voters
cast their votes for the independent party. Suppose the probability of voting for the
2
.
independent party is
10
(a) What is the probability 15 voters must be polled before 5 voters who voted for
the independent party are observed?
(b) What is the expected value of the number of voters to poll to observe 5
independent party voters?
(c) What is the variance of the expectation in part (b)?
Solution (a) If we regard the polling of an independent voter as a success in this
experiment, then we may apply the negative binomial random variable Q′ with
2
parameter values r = 5 and p = . Therefore, the probability of polling t = 15
10
voters as they
exit
the
voting
booth
5 10 to observe r = 5 votes for the independent party
8
14
2
= 0.034
is pQ′ (15) =
10
10
4
Solution (b) Proceeding by the observation in part (a), we have
2
40
E(Q′ (5, )) =
= 25
10
9
127
Solution (c) Again, proceeding by the observation in part (a), we have
2
V (Q′ (5, )) = 20
10
3. 5.3 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
4.4
The Geometric Random Variable G(p)
In this section, we consider a special case of the Negative Binomial random variable
Q′ (r, p), that of the Geometric random variable. It is defined by the Negative Binomial
random variable with r = 1, that is to say, experimentally it computes the probability that
the first success in a sequence of Bernoulli trials occurs on the t-th trial. We have the
following definition.
Definition 52. We say G(p) is the Geometric Random variable. We p is its parameter,
where p is the probability of success in a Bernoulli trial.
Beside the definition, we state the what the attendant theorem should be without an
appeal to the specification Q′ (1, p) = G(p).
Theorem 32. Let (ΩB(p) , β) be a Bernoulli process. Define
G(p) : ΩB(p) → R
by G(p)(b) = t is the number of trials until the first success occurs in the word b read from
left to right. Then (ΩG(p) , βG(p) , pG (t)) is a discrete probability space, where for t ≥ 1
pG (t) = pq t−1
First we observe that ΩG(p) = {1, 2, 3, . . .} and βG(p) = 2ΩG(p) , and that, once more, the
power set of a countable set has the structure of a σ-algebra. As for pG (t), we notice that
by substituting r = 1 into the probability function of pQ′ (t) gives the probability function
128
PG (t) = pq t−1 . Thus, the induced probability function satisfying Kolmogorov’s axioms is
tantamount to the convergence of the Geometric series and the hypothesis |q| < 1. Observe,
∞
X
q t−1 p =p
t=1
1
1−q
=1
since p = 1 − q.
To compute the expected value and variance, we can proceed by the formal method of
chapter 3. Equivalently, we could merely set the r = 1 in the corollaries of the previous
section giving the expressions for the expected value and variance of the transformation
Q′ (r, p). Computing the generating function also depends upon the sum of the Geometric
series, as well. Observe,
gG (z) =
∞
X
q t−1 pz t
t=1
∞
X
p
(qz)t
q t=1
p
=
q(1 − zq)
=
′
(z) =
Computing the derivatives, we have gG
2pq
p
′′
(z) =
and gG
. Hence
2
(1 − zq)
(1 − zq)3
1
′
(1) = . Notice this is the same as the expected value of Q′ (1, p) in the previous section.
gG
p
As for the variance of the Geometric random variable, we have
2
′′
′
′
V (G(p)) =gG
(1) + gG
(1) − (gG
(1))
2q 1
1
= 2+ − 2
p
p p
2q + p − 1
=
p2
q
= 2
p
We summarize our work with the generating function in the following proposition.
Proposition 29. Let G(p) be the Geometric random variable, then its generating function
is given by
gG (z) =
p
(1 − zq)
and
1. E(G(p)) =
1
p
129
2. V (G(p)) =
q
p2
Before turning to examples,we make note of the following forgetfulness property of the
Geometric random variable. Consider the following conditional probability
P ({G(p) > a} | {G(p) > b})
for a, b ∈ ΩG(p) such that a > b. Then we have, by both independence and the definition of
the induced probability function of a discrete random variable, that
P ({G(p) > a} ∩ {G(p) > b})
P ({G(p) > b})
P ({G(p) > a})
=
P ({G(p) > b})
P
t−1
pq a ∞
t=1 q
= b P∞
t−1
pq
t=1 q
P ({G(p) > a} | {G(p) > b}) =
=q a−b
=P ({G(p) > a − b})
so that the final probability is interpreted to mean that the Geometric random variable
forgets the conditional relationship between the events {G(p) > a} and {G(p) > b}.
4.4 WORKED EXAMPLES
1. 5.4 Let G(p) be the Geometric random variable with parameter value p = 0.3
(a) Compute both E(G(0.3)) and V (G(0.3))
(b) Compute pG (4 ≤ t ≤ 6), that is, the probability of the first success occurring
between the 4th trial and the 6th trial.
(c) Compute pG (t ≥ 4)
Solution (a) We can compute both the expected value and variance of the
Geometric random variable in terms of its parameter, p. In this case, we have
0.7
0.7
and V (G(0.3)) =
.
E(G(0.3)) =
0.3
0.32
P6
Solution
(b)
p
(4
≤
t
≤
6)
=
p
(t
=
4
OR
t
=
5
OR
t
=
6)
=
G
G
t=4 pG (t) =
P6
t−1
(0.7)
(0.3)
=
0.225351
t=4
P
Solution (c) pG (t ≥ 4) = 1 − pG (t ≤ 3) = 1 − 3t=1 (0.7)t−1 (0.3) = 0.343.
2. Suppose a basketball player has a 52.4 % chance of scoring a basket in the game of
basketball.
130
(a) What is the probability that this player will score a basket after 3 attempts?
(b) What is the expected number of attempts before they score?
(c) What is the variance in the expectation of part (b)?
Solution (a) If we interpret the player scoring a basket as a success in this
experiment, then we may compute the probability of this event by the geometric
random variable with parameter value t = 3, so that pG (1) = (0.524)(0.476)2 = 0.118
Solution (b) Proceeding by the observation in part (a), we have
1
= 1.9
E(G(0.524)) =
0.524
Solution (c) Again, proceeding by the observation in part (a), we have
0.476
= 1.73
V (G(0.524)) =
(0.524)2
3. 5.4 Suppose the probability a light bulb will fail to function is determined experimentally to be 0.1 %.
(a) What is the probability a light bulb will fail after 30 days?
Solution (a) The probability of observing a light bulb fail after a varying number of
days is modeled by the geometricP
random variable with parameter value p = 0.001.
t−1
=
Therefore, we have pG (t > 30) = ∞
t=31 (0.001)(0.999)
30
P
(0.001)(0.999)
t−1
=
(0.001)(0.999)30 ∞
= 0.97
t=1 (0.999)
0.001
4.5
The Hypergeometric Random Variable H(M, N, k)
In this section we consider a parametric family of discrete random variables predicated
upon a simple event in a Bernoulli process of time N , say b ∈ ΩB(N ) . Given a branch b in this
process we write b(M, N ) to indicate the number of successes M in the word b representing b
simple event. We shall define a random variable H(M, N, k) with respect to a sample space
wrought from such a b. We will show that a sequence of these random variables converges
M
to the Binomial random variable B(k, p) such that p = limN →∞ . As such, the Binomial
N
random variable approximates this variable we shall consider for N large. Alternatively, we
obtain another perspective on what the Binomial random variable is, if one reflects on this
representation of the probability of success in a Bernoulli trial. Experimentally, H is used
to compute the likely of a certain number of objects in a set of two mutually exclusive types
in a sample from this set of a fixed size.
Let us consider a branch b(M, N ) ∈ ΩB(N ) in a Bernoulli process at time N . We shall
construct a discrete probability space predicated upon this branch together with a discrete
random variable, the Hypergeometric random variable, denoted H(M, N, k).
131
Theorem 33. Let Ωkb(M,N ) be the set of subsets of letters in the word b(M, N ) of size k and
(Ωkb(M,N ) , β) the attendant measure space. Define
H(M, N, k) : Ωkb(M,N ) → R
by H(M, N, k)(e) = x is the number of letters s in the subset e. Then (ΩH(M,N,k) , βH(M,N,k) , pH (x))
is a discrete probability space, where for 0 ≤ x ≤ k
M N −M
pH (x) =
x
k−x
N
k
Since ΩH(M,N,k) = [k], we have that βH(M,N,k) has the structure of a σ-algebra. So, to
prove the theorem, we must show that the induced probability function satisfies Kolmogorov’s
axioms. We shall proceed as in the previous sections.
Let
P us determine probabilities px = pH (x) for the simple events in x ∈ ΩH(M,N,k) such
that
px = 1. This will prove pH (x) satisfies Kolmogorov’s axioms as an application of the
canonical formula in chapter 1. These px are the probabilities of the fibres of H(M, N, k) by
chapter 3. Thus, consider {H(M, N, k) = x}, the fibre of H(M, N, K) over x ∈ ΩH(M,N,k) .
Then, by definition of chapter 3.2, we have
pH (x) =
X
1
e∈{H(M,N,k)=x}
N
k
P ({H(M, N, k) = x}) =
where P (e) =
1
N
k
is the probability of a simple event in Ωkb(M,N ) . We can complete
the computation by computing the size of the summation index as the summand values are
constant.
Observe that {H(M, N, k) = x} is the set of subsets e of size k consisting of x of the
letter s. To compute the cardinality
of this fibre, we proceed by the multiplication rule, in
M
two steps. First, there are x subsets of size x consisting of M letters s that we may select
−M
from the letters in b(M, N ) without replacement and in no order. Second, there are Nk−x
subsets of size k − x consisting of N − M letters f that we may select from the letters in
b(M, N ) without replacement and in no order. Therefore, by the multiplication rule, we have
N −M
|{H(M, N, k)(e) = x}| is M
. Consequently, we have
x
k−x
M N −M
pH (x) =
x
k−x
N
k
To prove that these px satisfy the conditions of the theorem of chapter 1.3, we require
the following proposition.
Pj
N
M
Lemma 4.3. N +M
= l=0 j−l
j
l
132
The proof we present depends upon the Binomial theorem. An alternative proof is given
in chapter 2 in the exercise pertaining to the Vandermonde sum. Observe,
(x + y)N +M =
! M !
N X
X M
N
(x + y)N (x + y)M =
xN −k y k
xM −l y l
k
l
k=0
l=0
N
M
XX N
M N +M −k−l k+l
=
x
y
k
l
k=0 l=0
and so we have
PN +M
j=0
N +M
j
N +M −j j PN PM N M N +M −k−l k+l
y
x
y = k=0 l=0 k l x
Comparing coefficients in this equality, we see that j = k + l and l = j − k. Further,
since k ≥ 0, we have k ≤ j. Therefore
N N +M −j j
PN +M N +M N +M −j j PN +M Pj
M
x
y
x
y
=
l=0 j−l
j=0
j=0
l
j
so it follows that
N +M
j
=
Pj
M
l=0 j−l
N
l
To finish our proof, observe that px ≥ 0 since each of the factors in the expression are
positive. Furthermore,
Pk
k
M N −M
X
x=0 x
k−x
px =
N
k
x=0
N
k
N
k
=
=1
which follows from the lemma, completing the proof of the theorem. We provided an alternative combinatorial proof to Vandermonde’s identity in chapter 2.
Definition 53. We say H(M, N, k) is the Hypergeometric random variable. We say M, N
and k are its parameters, where M is the number of successes in a sample of size k in a set
of size N .
Let us now compute
the
expected value and variance in terms of the generating function,
M N −M
P
z x . Observe,
gH (z) = kx=0 x Nk−x
k
133
′
(1) =
gH
k
X
x M
x
N −M N
k
x=0
k−x
k
−1 N −M
X
M M
x−1
k−x
=
N
k
x=0
k−1
X
kM
=
N x=0
=
M −1
x−1
(N −1)−(M −1)
k−x
N −1
k−1
kM
N
where the sum in the penultimate line simplifies to 1 by Kolmogorov’s axiom, as it is
the Hypergeometric random variable for parameter values H(M, N, k − 1). Therefore,
kM
E(H(M, N, k)) =
.
N
To compute the variance, a basic calculus computation gives
M N −M
x(x
−
1)
P
x
′′
k−x z x−2
gH
(z) = kx=0
N
k
So
k
M N −M
X
x(x
−
1)
x
′′
k−x
gH
(1) =
N
x=0
k
k
X
N −M
M (M − 1)(M − 2)! k(k − 1)(k − 2)!(N − k)!
=
x(x − 1)
k − x x(x − 1)(x − 2)!(M − x)!
N (N − 1)(N − 2)!
x=0
k
−2 N −M
X
M (M − 1)(k)(k − 1) M
x−2
k−x
=
N −2
N
(N
−
1)
k−2
x=0
k−2 M (M − 1)(k)(k − 1) X M − 2
N −M
=
−2
x
k−2−x
N (N − 1) Nk−2
x=0
−2
M (M − 1)(k)(k − 1) Nk−2
=
−2
N (N − 1) Nk−2
=
M (M − 1)(k)(k − 1)
N (N − 1)
134
′′
′
′
At last we may apply the usual formula V (H(M, N, k)) = gH
(1) + gH
(1) − (gH
(1))2 which
gives
2
kM
M (M − 1)(k)(k − 1) kM
+
−
V (H(M, N, k)) =
N (N − 1)
N
N
kM
kM (M − 1)(k − 1)
+1−
=
N
N −1
N
kM N (M − 1)(k − 1) + N (N − 1) − kM (N − 1)
=
N
N (N − 1)
2
kM N − N k − N M + kM
=
N
N (N − 1)
kM (N − M )(N − k)
=
N
N (N − 1)
We summarize these laborious computations in the following proposition.
Proposition 30. Let H(M, N, k) be the Hypergeometric random variable, then
1. E(H(M, N, k)) =
kM
N
kM
2. V (H(M, N, k)) =
N
(N − M )(N − k)
N (N − 1)
One might note that, in contradistinction to the earlier sections, we do not offer a closed
form for gH (z) in this proposition. This is because there is not a closed form for the generating
function of this variable. We refer the reader to the broader literature to explore the extent
to which one can write down a more compact formula for the generating function of the
Hypergeometric random variable.
In what follows we shall argue that lim pH (x) = pB (x), for B(k, p) such that p =
N →∞
M
lim
. Accordingly, for N large, B(k, p) ∼
= H(M, N, k) for such p. One way of interpreting
N →∞ N
this result is that p, the probability of success in a Bernoulli trial, is obtained as the limit of
a sequence of approximations to the same in the equiprobable spaces (Ωkb(M,N ) , β, P ), where
1
P (e) = N for each e ∈ Ωkb(M,N ) .
k
Proposition 31. Let pH (x) be the probability function of the Hypergeometric random variable H(M, N, k), then limN →∞ pH (x) = pB (x), the probability function of the Binomial ranM
dom variable B(k, p) such that p = lim
.
N →∞ N
M N −M
k x k−x
x
k−x
=
p q . To this end, we simplify the
We want to show that lim
N
N →∞
x
k
135
quotient in the limit as follows
M N −M
x
k−x
N
k
M ! · (N − M )! · k! · (N − M )!
x! · (M − x)! · (k − x)! · (N − k − (M − x))! · N !
k M !/(M − x)! (N − M )!/(N − M − (k − x))!
·
=
(N − k + (k − x))!/(N − k)!
x N !/(N − x)!
Y
k−x
x
k
(M − x + l) Y (N − M − (k − x) + m)
·
=
·
N − x + l m=1
(N − k + m)
x
l=1
=
l−x
M
+
(M − x + l)
N = lim M = p and
Next, observe that both lim
= lim N
l−x
N →∞ N − x + l
N →∞
N →∞ N
1+
N
N −M
M
(N − M − (k − x) + m)
= lim
= 1 − lim
= q, since x ∈ [k] and M
lim
N →∞
N →∞ N
N →∞
(N − k + m)
N M N −M
k x k−x
x
k−x
increases with N . Therefore, we have lim
=
p q , as xk is constant with
N
N →∞
x
k
respect to the limiting index, as desired. In the next section, we shall consider when pB (x)
converges as n → ∞.
4.5 WORKED EXAMPLES
1. Suppose a retailer buys good in lots and each good is deemed to be either acceptable
or unacceptable. Suppose further that the same retailer purchases a lot of 25 goods, 6
of which are deemed to be unacceptable.
(a) What is the probability of no unacceptable goods in a sample of 10 goods from
the lot of 25?
(b) What is the expected number of unacceptable goods in a sample of 10?
(c) What is the variance in the above expectation?
Solution (a) We recognize that the probability of this event may be computed by
the hypergeometric random variable, with parameters N = 25, M = 6, and K = 10
by regarding unacceptable goods as one of two mutually
exclusive classes of objects
in the experiment. Accordingly, we have pH (0) =
6
0
19
10
25
10
= 0.028
Solution (b) We have, by the observation in part (a), that
10 · 6
2
E(H(25, 6, 10)) =
=2+
25
5
Solution (c) Again, proceeding by the observation in part (a), we have that
60 · 19 · 15
14
V (H(25, 6, 10)) =
=1+
25 · 25 · 24
100
136
2. Consider an urn consisting of 37 marbles, 12 of which are blue, and 25 of which are
red. Now consider the experiment that consists of sampling 20 marbles from the urn.
(a) What is the probability there are 3 blue marbles in the sample?
(b) What is the expected value of blue marbles in the sample?
(c) What is the variance in the expectation?
Solution (a) We recognize that the probability of this event may be computed by
the hypergeometric random variable, with parameters N = 37, M = 12, and K = 20
by regarding blue marbles as one of two mutually
exclusive classes of objects in the
experiment. Accordingly, we have pH (3) =
12
3
25
17 = 0.014
37
20
Solution (b) We have, by the observation in part (a), that
18
20 · 12
=6+
E(H(12, 37, 20)) =
37
37
Solution (c) Again, proceeding by the observation in part (a) we have that
20 · 12 · 17 · 25
286
V (H(12, 37, 20)) =
=2+
37 · 37 · 36
4107
3. 5.5 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
137
4.6
The Poisson Random Variable π(λ)
In this section we consider the limit as n → ∞ of Binomial random variables B(n, p)
defined on an interval of real numbers of finite length. In order to compute this limit, we
start by considering a process related to a Bernoulli process, that of a Poisson structure on a
real interval of finite length. Our process is naive, so we use the phrase "structure" instead of
the phrase "Poisson process" that one finds rigorously in the literature. A Poisson structure
may be regarded as a continuous extension of a Bernoulli process at time n.
An important application of the continuous extension is to approximate the Binomial
random variable B(n, p) by the so-called Poisson random variable π(λ), determined by the
convergent limit of B(n, p)’s when experimentally n is small compared to p. Abstractly
this will mean that the parameter of the Poisson random variable is given by λ = np. As
a variable in its own right, the Poisson random variable π(λ) is interpreted to count the
number of occurrences of certain events in an interval of finite length that occur, completely
at random, at an average rate of occurrence λ. Thus, its induced probability function pλ (x)
shall compute the probability of the event "x certain such events occur" etc.
Let us now consider the following simplification of the general case of a Poisson process.
First, we construct the sample space to model experiments in which events that happen at
a given rate, but completely at random, occur. To this end, fix a positive real number l > 0
l
and partition the interval [0, l] into n disjoint sub-intervals of equal width ∆ = , viz.
n
[0, l] = ∪ni=1 (xi−1 , xi ] ∪ {0}
l
that is, for fixed n ∈ N, such that |((xi−1 , xi ]| = . We regard each cell in this partition as
n
the i-th episode of a process which transpires over a duration l. Assume at each episode a
Bernoulli trial Bi is performed and that P (s) = p for each performance. Moreover, assume p
is proportional to the duration of the i-th episode, that is, p = λ · ∆ for some fixed constant
of proportionality λ ∈ R, where 0 ≤ p ≤ 1 as it is the probability of success in a Bernoulli
trial. Notice this assumption implies that q = 1 − λ∆ in a Bernoulli trial Bi . We shall say
that a certain event occurs at the i-th episode if the performance of Bi results in s, and
likewise that a certain event does not occur if Bi results in f .
Under these auspices, the interval [0, l] is our model for the occurrence of certain events
occurring at n episodes, completely at random, and at a particular, constant rate λ. We
should remark that this constant rate in experiments is taken to be the average rate of
occurrence of a certain event. We say such an real-interval has a Poisson structure.
We are now able to achieve our aim of computing the naive limit of a sequence of Binomial
random variables. To this end, define an N-valued function πn : [0, l] → R on the Poisson
structure by the formula
πn (t) = x ∈ N
for t ∈ (0, l] to be the number of occurrences of a certain event in (0, t] ∩ [0, l]. Moreover,
we insist that πn (0) = 0 since initially no Bernoulli trial has been performed. Observe that
πn (t) is a discrete random variable such that Ωπn = [n], whose induced probability function
is given by
138
n x n−x
pπn (x) =
p q
x
the details of this are left to the reader but follow the argument in 4.2.
Given our goal, we would like to now compute lim pπn (x) but let us pause to consider
n→∞
the following. In the present context, we stress that such a limit arises as the limit of the
l
sequence {πn (t)} as ∆ → 0 for this is tantamount to n → ∞, since ∆ = . Yet there is an
n
obstruction to computing the limit in this generality.
Let B(n, p) be an arbitrary Binomial random variables. By Chebyshev’s inequality, we
have
npq
P ({|B(n, p) − np| ≥ t}) ≤ 2
t
or, by the law of the unconscious statistician,
P ({|
B(n, p)
pq
− p| ≥ t}) ≤ 2
n
nt
which tends to 0 as n → ∞, implying that for arbitrary p, the sequence {B(n, p)} → 0 or in
the above notation, {πn (t)} → 0, for arbitrary n. Accordingly, the probability of x successes
in n Bernoulli trials as n becomes large vanishes in general.
To obviate this obstruction to a non-trivial limit, we insist that p is small compared to
n. Abstractly, we insist that p is independent of n. This means that np = λ for λ a fixed
constant. Accordingly, by Chebyshev’s inequality, we have
P ({|B(n, p) − λ| ≥ t}) ≤
λq
t2
which is now independent of n when n is small compared to p. Thus, the sequence {B(n, p)}
does not vanish as n → ∞. We shall incorporate this hypothesis below so that the limit we
obtain is non-trivial.
Theorem 34. Let (Ωπn (t) , β, pπn (x)) be the induced probability space with respect to πn (t)
defined on the Poisson structure [0, l]. Define
π(λ) : [0, l] → R
where π(λ)(t) = x ≥ 0 is the number of occurrences of a certain event in (0, t] ∩ [0, l]. Then
lim πn (t) = π(λ) exists non-trivially if p is small compared to n and (Ωπ(λ) , β, pλ (x)) is a
n→∞
discrete probability space, for x ≥ 0 where
pλ (x) =
λx −λ
e
x!
139
We shall verify that pλ (x) = limn→∞ pπn (x) when p is small compared to n, that is, λ = np.
Without loss of generality, take l = 1. Observe,
n x n−x
lim pπn (x) = lim
p q
n→∞
n→∞ x
x n−x
n
λ
λ
= lim
1−
n→∞ x
n
n
n
x
n!
λ
λ
lim
1−
q −x
=
x! n→∞ (n − x)!nx
n
n −x
λx
n!
λ
λ
=
lim
1−
1−
x! n→∞ (n − x)!nx
n
n
x
λ
= e−λ
x!
n
λ
λx
where by elementary calculus lim 1 −
= e−λ . We conclude that pλ (x) = e−λ , thus
n→∞
n
x!
providing the formula for the induced probability function of the Poisson random variable.
To show that this probability function satisfies Kolmogorov’s axioms, we proceed to verify
that each holds definition. Notice that as ∆ → 0 we have n → ∞, so that the induced sample
space is Ωπ(λ) = N. We appeal once more to the fact that the power set of a countable set
has the structure of a σ-algebra to verify β is a σ-algebra.
Let us determine probabilities px = pλ (x) for the simple events x ∈ Ωπλ . Observe, by our
previous argument,
pλ (x) =
P ({π(λ) = x}) =
λx −λ
e
x!
Observe, px ≥ 0 for x ∈ N since ex is a positive function and the ratio we multiple it by
is always positive. Furthermore,
X
x∈N
px =
∞
X
λx
x!
x=0
=e−λ
e−λ
∞
X
λx
x=0
−λ λ
x!
=e e
=1
as desired.
Definition 54. We say π(λ) is the Poisson random variable. We say that λ is its parameter
where λ is the average number of successes in a Poisson process of length l.
140
Next we shall compute both expected value and variance of π(λ) by the formal method
in chapter 3. First, let us compute the generating function of π(λ). We have
gλ (z) =
∞
X
λx
x!
x=0
=e
=e
−λ
e−λ z x
∞
X
(zλ)x
x=0
−λ zλ
x!
e
=eλ(z−1)
which is a rather convenient closed form for the Poisson variable’s generating function.
Next let us compute both g′λ (1) and gλ′′ (1) + g ′ λ(1) − (gλ′ (1))2 in order to compute the
expected value and variance of π(λ), respectively. It is trivial that gλ′ (1) = λ and gλ′′ (1) = λ2 .
We summarize our work in the result of the following proposition.
Proposition 32. Let π(λ) be the Poisson random variable, then its generating function is
given by
gλ (z) = eλ(z−1)
and
1. E(π(λ)) = λ
2. V (π(λ)) = λ
In the above discussion, we have computed the probability of a certain number of events in
an interval of finite length l occurring at an average rate of λ. A good question we can answer
is how does this figure change if only the length of the interval changes? Experimentally, this
change in length is interpreted as a change in the length of time the experiment is performed.
As the the probability of an event in an interval of length l is parameterized by the average
rate of occurrence λ, this question is tantamount to how does λ change when the length of
the interval does? The answer is given by the following lemma.
Lemma 4.4. Let π(λ) be the Poisson random variable on a Poisson structure [0, l] with
average rate of occurrence of a certain event λ. Then π(λ) extends to the Poisson structure
[0, k = hl] by the formula π(hλ), that is, with average rate of occurrence of a certain event
in [0, k] is hλ.
Before proceeding to examples, we state a theorem that justifies approximating B(n, p)
by π(λ) when n is small compared to p. Regarding the work we in the demonstration above,
we see that the Poisson random variable approximates the Binomial random variable under
this condition. In experiments, one interprets the condition that n is small compared to p
to be acceptable to use when n ≥ 100 and np ≤ 10. Accordingly, we have the following
theorem.
141
Theorem 35. Let π(λ) be the Poisson random variable and B(n, p) the Binomial random
variable. Suppose that p is small compared to p. Then π(λ) ∼
= B(n, p) with λ = np, that is
∼
to say, pλ (x) = pB (x).
This theorem is quite useful to approximate the Binomial random variable. More generally, one could compare this theorem to the manner in which the Central Limit Theorem is
used to approximate the Binomial random variable by the Normal random variable. In our
text, when n is small compared to p in the experimental sense, above, we can replace the
(np)x −np
e .
probability pB (x) = nx px q n−x by pλ (x) =
x!
4.6 WORKED EXAMPLES
1. Suppose there is a disease, whose average incidence is 2 per million people. What is
the probability that a city of 1 million people has at least twice the average incidence?
(a) What is the probability that a city of 1 million people has at least twice the
average incidence?
Solution (a) Assume that pλ (x) =number of cases in 1 million people summarizes
the events in the experiment, with parameter λ = 2 by the hypothesis in the
question. Then the question asks us to compute pλ (x > 4) since 4 is twice the
incidence of 2 per million people. Proceeding by DeMorgan’s law, we have
pλ (x > 4) =1 − pλ (3 < x)
21
22
23
20
=1 − (e−2 + e−2 + e−2 + e−2 )
0!
1!
2!
3!
=0.143
2. Suppose that disease α occurs with incidence 1.7 per million and that disease δ occurs
with incidence of 2.9 per million.
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
142
Solution (c) C.
3. Consider a telephone operator who, on average, handles 5 calls every 3 minutes.
(a) What is the probability that there will be no calls in one minute of time for
which the operator is handling calls?
(b) What is the probability of at least 2 call in one minute of time for which the
operator is handling calls?
Solution (a) As the number of occurrences of the phone operator handling calls is
given in terms of a 3 minute interval of time, we may assume that the probability of
5
events in this experiment are modeled by the Poisson random variable, with λ = .
3
−3
0
3 e 5
As such, pλ (0) =
= 0.189
5
0!
Solution (b) Proceeding as in part (a), we have

−3
−3 
0
1
 3 e 5
3 e 5 
 = 0.496 “ “ “ “ “ ‘
pλ (x ≥ 2) = 1 − pλ (x ≤ 1) = 1 − 
+
 5
0!
5
1! 
4. 5.6 Live births occur in a hospital at an average of 1.8 per hour.
(a) What is the probability of observing 4 births in 2 hours?
(b) What is the probability of observing at least 5 births in 2 hours?
Solution (a) As the number of occurrences of live births in a hospital is given in
terms of a 1 hour interval of time, we may assume that the probability of events in
this experiment are modeled by the Poisson random variable, with λ = 1.8. To
extend this variable to a 2 hour interval of time, we apply the lemma to obtain
e3.6
λ′ = 2 · 1.8 = 3.6. Therefore, pλ′ (4) = (3.6)4
= 0.191
4!
Solution (b) Again,proceeding as above in part (a), we have pλ′ (x ≥ 5) =
3.6
3.6
3.6
3.6
3.6
0 e
1 e
2 e
3 e
4 e
1 − pλ′ (x ≤ 4) = 1 − (3.6)
+ (3.6)
+ (3.6)
+ (3.6)
+ (3.6)
0!
1!
2!
3!
4!
5. A type-setter, on average, makes 1 error in every 500 words that they type. A typical
page of text contains 300 words.
143
(a) What is the probability there will be no more than 2 errors in 5 pages?
Solution (a) prima facie, it would seem the solution is given by the application of
the Binomial random variable with parameter values n = 1500 trials and probability
1
. Observe, however,
of success p given by the probability of a typing error, or p =
500
that such a computation for most values of x is practically impossible, viz.
x 1500−x
499
1500
1
pB (x) =
500
500
x
So, instead, we recognize that p is small compared to n, that is, n · p = 3 and take
this product to be the parameter value for λ in the Poisson approximation of the
Binomial random variable. Accordingly, we compute instead pλ (x ≤ 2) to
e−3
e−3
e−3
approximate pB (x ≤ 2). As such, we have pλ (x ≤ 2) = 30
+ 31
+ 32
0!
1!
2!
6. Suppose that in a large population the proportion of people who have a certain disease
1
. Consider the experiment of testing people for the disease in a sample of 200
is
100
taken from the same large population.
(a) What is the probability at least 4 people have the disease.
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
4.7
The Multinomial Random Variable M (n; xi ; pi )
In this section we consider a joint distribution that generalizes the Binomial random
variable at the beginning of this chapter to the setting of k mutually exclusive simple events
rather than merely 2. While the previous sections considered it as either a limit of variables
itself or what the limit of it would be when it exists, this section considers the generalization
to a random vector.
First, let us consider a generalization of a Bernoulli trial itself, namely, a multi-trial and
then define the multinomial random vector as a random vector on a so-called multi process
at time n. Define a multi-trial to be a discrete probability space (M, β = 2M , P ), where
M = {s1 , s2 , . . . , sk } consists of k mutually exclusive simple events si and the assignment
144
P
P (si ) = pi ∈ [0, 1] such that i pi = 1 for 1 ≤ i ≤ k. By chapter 1, this assignment defines
a discrete probability space.
Now, by analogy with a Bernoulli process at time n, we say that process ΩM (n,pi ) =
×nj=1 Mj is a multi process at time n if Mj is a multi-trial for each 1 ≤ j ≤ n, Mj and Ml are
independent for different j, l, and P (si ) = pi for all j.
We define the discrete random variable Xi : ΩM (n,pi ) → R to be xi , the number of si in
the word representing u in this process. We state without further elaboration that we regard
the simple events in a multi-process at time n to be branches of a rooted tree such that,
for any fixed height, a vertex has exactly k edges emanating from it. One then obtains the
word u in a manner that straightforwardlyP
generalizes how it is obtained from a binary tree.
Notice that ΩXi = [n] for each i, and so, ki=1 xi = n. Proceeding by chapter 2, a word u
with x1 of s1 ’s, and . . ., xk of sk ’s is equivalent P
to an ordered partition of the set S with
partition Ai consisting of xi symbols si such that
xi = n. We shall elaborate on this point
further below to prove the following theorem.
Theorem 4.5. Let (ΩM (n,pi ) , β) be a mutli-process at time n. Define
Xi : ΩM (n,pi ) → R
by Xi (u) = xi the number of symbols si in the word u representing the simple event u, for
i = 1, 2, . . . , k. Let ⟨Xi ⟩ki=1 : ΩM (n,pi ) → Rk be the corresponding discrete random vector,
then
M (n; xi ; pi ) : ΩM (n,pi ) → R
where M (n; xi ; pi ) =
k
X
Xi is a joint random variable, defined by M (n; xi ; pi )(u) = n.
i=1
Furthermore, we have that (ΩM (n,x,p) , β, pM (x1 , . . . , xk )) is a discrete probability space, where
pM (x1 , x2 , . . . , xk ) = x1 ,x2n,···xk px1 1 px2 2 · · · pxkk
We say this joint random variable is the multionomial random variable. Let
first show
us
n
x1 x2
that the induced probability function is give by pM (x1 , x2 , . . . , xk ) = x1 ,x2 ,···xk p1 p2 · · · pxkk .
As usual, we can proceed by the multiplication rule in two steps. First, by the independence
of the multi-trials, we have that for u ∈ ΩM (n,pi ) such that there are x1 of s1 and . . . , and xk
xk
x1
k
k
of sk that the
P probability of u is p1 · · · pk . Second, the fibre of M (n; xi ; pi ) over ⟨xi ⟩i=1 ∈ R
such that i xi = n consists of ordered partitions of the set of x1 of s1 and . . . and xk of
sk . By chapter 2.6 there are x1 ,x2n,...,xk such ordered partitions of this set. Thus, by the
definition in chapter 3.2, we have that pM (x1 , . . . , xk ) = x1 ,x2n,···xk px1 1 px2 2 · · · pxkk as above.
To show that pM (x1 , . . . , xk ) satisfies Kolmogorov’s axioms we must appeal
to the multiP
−1
k
k
nomial theorem of chapter 2. Indeed, f (ΩM (n,x,p) ) = {⟨xi ⟩i=1 ∈ R |
i xi = n }, where
f (X1 , X2 , . . . , Xk ) = M (n, pi ) is the statistic in the theorem. So, by definition,
X
n
pM (ΩM (n,x,p) ) =
px1 1 px2 2 · · · pxkk
x1 , x2 , · · · xk
P
k
k
⟨xi ⟩i=1 ∈R
|
i xi =n
=(p1 + p2 + . . . pk )n
=1n = 1
145
by the multinomial theorem and the hypothesis
is left to the reader.
P
i pi = 1.
Verifying the remaining axioms
4.7 WORKED EXAMPLES
1. 5.7 Let us consider a multi process at time 10, where each multi-trial consists of three
simple events, say M = {s1 , s2 , s3 }. Assuming P (s1 ) = 0.25, P (s2 ) = 0.5, and P (s3 ) =
0.25
(a) Compute pM (5, 3, 2)
(b) A genetic allele of a pea section can be either AA Aa, or aa. A simple genetic
model specifies that P (AA) = 0.25, P (Aa) = 0.5, and P (aa) = 0.25. If the
alleles of 10 independently obtained sections are determined, compute the
probability that 5 are Aa and two are AA.
Solution (a) By the section, we must compute
10
pM (5, 3, 2) = 5,3,2
(0.25)5 (0.5)3 (0.25)2 = 2520(0.25)5 (0.5)3 (0.25)2 = 0.019
Solution (b) We recognize that the genetic allele of a pea section is a multi trial,
with simple events M = {s1 = AA, s2 = Aa, s3 = aa}. The determination of 10 such
sections is a multi process at time 10, so that, the probability of 5 Aa and 2 AA is
given by pM (2, 5, 3) since the remaining three sections must be aa as their
determination is a multi process. Therefore, the probability of this event is 1.9 % by
the previous exercise.
2. 5.7 An inspection plan calls for inspecting five chips and for either accepting each chip,
rejecting each chip, or submitting it for reinspection, with probabilities of p1 = 0.7,
p2 = 0.2, and p3 = 0.1, respectively.
(a) What is the probability that all five chips must be reinspected?
(b) What is the probability that none of the five chips must be reinspected?
(c) What is the probability that at least one of the chips must be reinspected?
Solution (a) We observe that the process of inspecting a chip is a multi-trial with
simple events M = {s1 = A, s2 = R, s3 = I} where we use A, R, I to denote accept,
reject, and re-inspect, respectively. Accordingly, the inspection routine in the
problem is a multiprocess at time 5. Therefore, the question asks us to compute
5
PM (0, 0, 5) = 0,0,5
(0.7)0 (0.2)0 (0.1)5 = (0.1)5
Solution (b) Proceeding as above, wePcompute
such that a + b = 5 or
pMa(a, b, 0)
5
5
5−b
a = 5 − b. This gives us pM (a, b, 0) = a=0 a (0.7) (0.2)
= (0.7 + 0.2)5 by the
binomial theorem. Therefore, pM (a, b, 0) = 0.59 for 0 ≤ a, b ≤ 5 such that a + b = 5.
146
Solution (c) As usual, we treat the probability of an event of the form, "at least
one" as the complement of "none". In this case, pM (x3 ≥ 1) = 1 − pM (a, b, 0) such
that a + b = 5. Thus, by the previous example, the probability that at least one chip
is inspected is 1 − 0.59 = 0.41.
3. 5.7 An American roulette wheel has 38 spaces, 18 labeled black, 18 labeled red, and 2
labeled green. Assume such a roulette wheel is fair.
(a) Determine the probability function associated to spinning the roulette wheel 10
times and recording whether the ball lands on black, red, or green.
(b) Compute the probability of 5 black wins, 3 red wins, and 2 green wins over 10
spins of the wheel.
(c) Compute the probability x1 = x2 .
Solution (a) From the perspective of the question, a spin of the roulette wheel is a
multi-trial with M = {s1 = B, s2 = R, s3 = G} with the obvious notation for its
1
9
and p3 =
by the fairness hypothesis,
simple events. As such, since p1 = p2 =
19
19
we have that
9 x1 9 x2 1 x3
10
pM (x1 , x2 , x3 ) = x1 ,x2 ,x3
19
19
19
9 5 9 3 1 2
10
Solution (b) We have pM (5, 3, 2) = 5,3,2
= 0.015
19
19
19
P
solution (c) If x1 = x2 , then pM (x1 , x2 , x3 ) = pM (x1 , x1 , 10 − 2x1 ) since i xi = 10.
4. 4.8 Let us show that the Binomial random variable may be treated as a statistic with
respect to P
the random vector ⟨Xi ⟩ni=1 where Xi is the Bernoulli random variable and
B(n, p) = ni=0 Xi
4.8
The Multivariate Hypergeometric Random Variable
In this section we introduce another example of a discrete random vector in the form
of the multivariate hypergeometric random variable. Let us consider N items that can be
partitioned into k cells A1 , A2 , . . . , Ak with n1 , n2 , . . . , nk , respectively. Then the probability
function of the random variable corresponding to the random vector ⟨Xi ⟩ki=1 representing
the number of elements selected from cells A1 , A2 , . . . , Ak in a random sample of size n, is
Qk ni p(x1 , . . . , xk ) =
where
k
X
i=1
xi = n and
k
X
ni = N
i=1
147
i=1 xi
N
n
4.9
Chapter 4 Homework Exercises
1. American Airline flights from Dallas to Chicago arrive on-time 80 % of the time. Suppose 15 flights are randomly selected, and the number of on-time flights are recorded.
(a) Define the random random variable associated with the experiment of recording
on-time flights by giving both its definition and its probability mass function.
(b) Compute the probability exactly 10 flights are on-time.
(c) Compute the probability that fewer than 10 flights are on-time.
(d) Compute the probability that at least 10 flights are on-time.
(e) Compute the probability that between 8 and 10 flights are on-time.
(f) Compute the expected value of the random variable computed in the first
question.
(g) Compute the variance of the random variable computed in the first question.
2. According to a study I just made up, the probability a randomly selected individual
will not cover their mouth when sneezing is 20 %. Suppose you sit in public and record
the sneezing habits of 10 randomly selected people.
(a) Define the random variable associated with the experiment of recording the
sneezing habits of 10 randomly selected people by giving both its definition and
its probability mass function.
(b) What is the probability that among 10 randomly selected people exactly 4 do
not cover their mouth when sneezing?
(c) What is the probability that among 10 randomly selected people fewer than 3
do not cover their mouths?
(d) What is the expected value of 10 randomly selected people not covering their
mouths when sneezing?
(e) What is the variance of 10 randomly selected people not covering their mouths
when sneezing?
(f) Suppose you now observe 15 randomly selected people rather than 10-what is
the probability then that at most 2 do not cover their mouths when sneezing?
(g) What is the probability that you observe at least 3 people not covering their
mouths when sneezing if you are observing 15 random individuals?
3. A false-positive in a polygraph test, that is, an individual who fails a lying-detector
test even though they are telling the truth, has a probability of 15 %. Consider the
experiment of administering a lie-detector or polygraph test (these are synonyms or
words that have the same meaning in common usage) to a group of randomly selected
people and recording false-positives.
148
(a) What is the probability that the first false-positive will occur when the third
person is tested?
(b) What is the probability that fewer than 4 people are tested before the first
false-positive occurs?
(c) What is the probability that more than 3 people are tested before the first
false-positive occurs?
(d) What is the expected value of false-positives?
(e) What is the variance in this experiment?
4. Consider a roulette wheel, where we are referring to the gambling wheel featured in
casinos or gambling parlors around the world. It is a wheel that has two green slots
(the 0, and 00 slots on an American table; recall, American wheels are different than
European wheels), 18 red slots, and 18 black slots.
(a) What is the probability it will take x = 1 trial or spin of the wheel before
observing r = 1 green slot?
(b) What is the probability it will take x = 30 trials or spins before observing r = 3
green slots?
(c) What is the expected number of trials or spins before observing 3 green slots?
(d) What is the expected value in this experiment?
(e) What is the variance in this experiment?
5. Suppose 40 % of students who drive to campus carry jump cables in their cars in case of
a dead battery emergency. Suppose further your car battery dies and that you do not
have jumper cables in your car. Consider the experiment of stopping other students
and asking to borrow jumper cables so that you are able to "jump" or re-energize your
car battery. Accordingly, let X =“number of students who must be stopped before find
a student with jumper cables."
(a) Give the probability function for this random variable, i.e. PX (X = x)
(b) Compute the probability PX (X = 1).
(c) Compute the probability PX (X ≤ 3)
(d) Compute the probability PX (X > 5).
(e) What is the expected value in this experiment?
(f) What is the variance in this experiment?
6. A gold digging company determines there is a 20 % chance of successfully mining gold
in a given mining site.
149
i) What is the probability that the first successful extraction of gold comes from the
third mining site?
ii) What is the probability that the third successful extraction of gold comes from the
seventh mining site?
iii) What is the mean number of mining sites that must be mined to successfully
extract gold 3 times?
iv) What is the variance of the same number of mining sites?
7. Suppose a lot of 25 machine parts is delivered and that a part is considered acceptable
only if it passes a tolerance test. Suppose further we sample 10 parts and find that
none are defective. What is the probability of this event if there are 6 defective parts,
that is, parts that do not pass a tolerance test, in a lot of 25?
8. i) Suppose an urn contains 23 balls, 13 of which are red, 12 of which are blue.
Suppose we drawn a sample of 6 balls and find there are 3 red balls in our sample.
What is the probability of this event, that is, that there are 3 red balls in the sample
of 6 balls we have withdrawn from the urn?
ii) What is the expectation of this experiment?
iii) What is the variance in this experiment?
9. According to "research" 90 % of adult cigarette smokers started smoking before
the age of 21 years old. Ten smokers 21 years old or older are randomly selected
and the number of smokers who started smoking before 21 is recorded.
a.) What random variable models the number of smokers who began smoking before
the age of 21 in the random sample of 21 adult smokers?
150
b.) Write down the probability mass function for your answer in a.)
c.) What is the probability fewer than 9 smokers started smoking before the age of
21?
d.) What is the expected value of smokers who began smoking before the age of 21 in
this experiment?
e.) What is the variance in this experiment?
10. The potholes on a major highway in the city of Chicago occur at the rate of 3.4
per mile.
a.) What random variable models the number of potholes over a randomly selected 3
mile stretch of highway?
b.) Write down the probability mass function for your answer in a.)
c.) What is the probability of fewer than 3 potholes over a 3 mile stretch of randomly
selected highway?
d.) What is the expected value of potholes over a 3 mile stretch of randomly selected
highway?
e.) What is the variance in this experiment?
151
11. A shipment of 100 machine parts is received at the port in New York City. It is
known in advance that 30 of the machine parts are defective. A sample of 50 parts
is withdrawn from the shipment.
a.) What random variable computes the probability that x of the parts in the sample
of 50 parts are defective?
b.) Write down the probability mass function for your answer in a.)
c.) What is the probability that 15 of the parts in the sample are defective?
d.) What is the expected value of defective parts in this sample?
e.) What is the variance in this experiment?
12. Automobiles are testing for emissions. It is determined that 30 % of automobiles
whose emissions levels are inspected fail the inspection. We shall conduct the
experiment of inspecting car emissions upon individual cars.
a.) What random variable computes the probability that x cars are inspected for
their emission levels until one car fails?
b.) Write down the probability mass function for your answer in a.)
c.) What is the probability that 5 cars are inspected until a car fails its emissions
test?
152
13. A book publisher determines that the probability of at least one misprint on any given
page is .005 in a book containing 600 pages.
a.) What random variable computes the probability there are x misprints in the 600
page book?
b.) Approximate the probability that at least three pages have a misprint.
14. An individual plays high school basketball. He success at shooting free throws is 70 %.
a.) What is the probability that an individual will have to shoot 7 free throws to
score 3 baskets?
b.) What is the expected value of successful free throws in 7 shots?
c.) What is the variance in b.)?
153
5
Markov Chains
This chapter both introduces and defines one of the main mathematical objects of applications used in the modern world, that of the Markov chain. In this textbook a Markov
chain is a sequence of probability vectors together with an endomorphism of the same that
satisfy a single recurrence relation or difference equation. In general objects of a category
together with an endomorphism, so-called automatons, are used to model the evolution of a
system of states. Accordingly, we should think of Markov chains as probabilistic models of
evolving systems of states.
The nature of a Markov chain qua an automaton conceals its explicit relationship to
probability theory, specifically, the formalism of discrete probability spaces and random
variables. Pursuant to this concern, we first introduce stochastic processes to establish the
relationship between the content of this text so far and the aforementioned Markov chain.
This material was foreshadowed by the informal introduction in the previous chapter of
Bernoulli processes and processes vis a’ vis the multiplication rule.
The delineation of Bernoulli processes as binary trees and sequences of random variables
in chapter 4 extends in the straightforward manner to arbitrary trees and sequences of
random variables in the present one. We call such extensions stochastic processes, below.
Stochastic processes, together with an axiomatic simplification of the probabilities of simple
events become Markov processes. We first interpret the axiom distinguishing these processes
in terms of linear algebra by way of the law of total probability discussed in chapter 1 and
then to translate all of the data contained within a Markov process into linear algebra to
arrive at the Markov chain, as described above.
After we translate Markov processes into linear algebra as Markov chains, we abandon
the explicit use of probability theory for the remainder of the chapter and in the following
one on Markov Decision Processes. Moreover, we study steady-state vectors and how they
are interpreted in applications to computer science. It is our hope that this most important
chapter convinces the reader why a modern student of computer science would be interested
in the foundations of discrete probability theory. In the next chapter we continue studying
other properties of Markov chains in the presence of more structure. One endeavors to
include more data in order to approach the sophistication required to model problems in
finance and artificial intelligence.
5.1
Naive Category Theory and the Category of Automata CA
In this section we introduce category theory in a naive manner sufficient to define automata
and to then give a few examples. We are motivated to do this by our goal to represent Markov
chains below as important examples of automata or more specifically, probabilistic automata.
Definition 55. We say C is a category if it consists of the following data:
1. A class of objects, denoted objC or written as a sequence by A1 , A2 , . . . such that
Ai ∈ objC .
2. To every pair of objects, say A, B ∈ objC the assignment of a set M or(A, B).
3. For all objects A ∈ objC a unique element 1A ∈ M or(A, A).
154
4. Given objects A, B, C ∈ objC , for every pair of elements in the sets M or(A, B) and
M or(B, C), say (f, g) ∈ M or(A, B)×M or(B, C) an element g◦f in the set M or(A, C).
These data must satisfying the following compatibility axioms:
1. For any f ∈ M or(A, B), the equality 1B ◦ f = f ◦ 1A holds.
2. The assignment of item 4. above is associative, that is, the equality (h◦g)◦f = h◦(g◦f )
for f ∈ M or(A, B), g ∈ M or(B, C), and h ∈ M or(C, D), where A, B, C, D ∈ objC .
All of the familiar ideas in mathematics can be expressed in terms of this definition and
perhaps even more. The first example of a category that we encountered in this text was that
of the category of sets, whose class of objects are sets S and whose elements in the assignment
M or(S, T ) we referred to as set functions in chapter 1. According to this reference, one says
in general that the elements of M or(A, B) are morphisms, arrows, functions, maps, and
so on. Furthermore, we shall assume the reader recognizes the element g ◦ f above as the
composition of morphisms and the axiom this operation satisfies as the ordinary composition
of functions. The second example of a category that we have encountered in this text is that
of the category of discrete probability spaces, whose class of objects are discrete probability
spaces (Ω, β, P ) and whose elements in the assignment M or((Ω, β, P ), (Ω′ , β ′ , P ′ )) are the
transformations discussed in chapter 4.
The next example we consider is that of the category of automata. This end, let us
consider a fixed category C . We can define from this a new category called the category of
automata, denoted AutC , as follows. The class of objects objAutC consist of pairs (A, f )
where A ∈ objC and f ∈ M or(A, A). Incidentally, one says such f are endomorphisms of
the object A. The morphisms M or((A, f ), (B, g) are so-called commuting diagrams, that is,
diagrams
A
f
ϕ
B
A
ϕ
g
B
such that the compositions along the pairs of arrows agree, that is, the equality ϕ ◦ f = g ◦ ϕ
holds, where ϕ ∈ M or(A, B) in C . We say an object in the category of automata is an
automaton. In terms of a real-world interpretation, one regards an automaton as an object
that models the evolution of states in situation. If the object A underlying the automaton
is a set, then the elements of this set should be regarded as states and the application of f
to these states as the manner they evolve.
An important example of the category of automata below has objects countable sequences
n
of vectors {xk }∞
k=0 , where xk ∈ R together with an endomorphism expressed as a matrix
M ∈ M atn (R) satisfying the difference equation
M xk = xk+1
155
and whose morphisms satisfy the commutative diagram condition required in the category
of automata in general. We say the objects in this particular category denoted Aut(Seq n )
of automata are dynamical systems.
The following sections are included in order to demonstrate how to obtain such an automaton from an object of the category of discrete probability spaces. Phrased more experimentally, how to interpret a real-world experiment that evolves through discrete time
values k in terms of such a dynamical system in order to study the likelihood of its events.
In the language of category introduced above, one can regard the following sections as the
establishment of a functor from a subcategory of discrete probability spaces to the category
of dynamical systems. In general, a functor F is a mapping categories C and D, that associates objects in C to objects in D in a manner that respects the composition of morphisms
in the source category. Accordingly, we can describe the following sections as a functor
M C : DP S → Aut(Seq n )
where a stochastic process satisfying the Markov axiom (ΩT , βT , P ) ∈ objDP S maps to
(S, M, x0 ) ∈ objAut(Seq n ) with {xk }∞
k=0 such that M xk = xk+1 under M C.
5.1 WORKED EXAMPLES
5.2
Elementary Introduction to Graphs and Trees
In this section we introduce graphs and rooted trees. These concepts are necessary for
several purposes relevant to Markov chains below. We can understand Markov chains as
probabilitistic models of evolving systems of states as certain types of discrete probability
spaces whose sample spaces are determined by the branches of rooted trees together with
an additional axiom. Such discrete probability spaces are interpreted experimentally as
experiments that occur in k episodes of discrete time. Elsewhere the reader may be familiar
with such a concept as a stochastic processes. The branches of rooted trees in this context
represent joint events in terms of the conditional events that determine their probability in
chapter 3 by labeling the edges with the conditional events determined by their vertex labels.
This representation in conjunction with the Markov axiom allows us to obtain a sequence of
probability vectors that we will treat as the states of an automaton and a matrix that will
serve as its endomorphism.
Beside setting up for Markov chains conceptually as experiments whose events evolve
over discrete time, we require notions from elementary graph theory to express the transition
graph related to a Markov chain and to study random walks on finite graphs as a particular
class of examples of Markov chains. These remarks motivate the following definition.
Definition 56. A graph G is a pair of sets V and E, referred to as vertices and edges,
respectively, denoted G = (V, E). The set V is an ordinary set whose elements are referred
to as vertices. Furthermore, E is a not necessarily proper subset of the Cartesian product
V × V whose elements are referred to as edges. We say G is a finite graph if |V | is finite.
As is usual in mathematics, we may consider substructures of a given one, so that we
define a subgraph of G to be a pair of subsets H = (V ′ , E ′ ) of G such that ϵ(e′ ) ∈ ϵ(E) for
each e′ ∈ E ′ . More plainly, we require that all of the endpoints of edges in H are in the
156
endpoints of the edges of G. What follows next is a selection of definitions pertinent to our
use of graphs outlined above. One should note that this subject is profuse with definitions
and pathologies so that studying it at initially is laborious...
First, let us describe the structure of the edges component E that is a part of the object
G. Identifying an edge with the elements of V in V × V comprising it is in fact a function
ϵ : E → 2V such that e 7→ {v, v ′ } under ϵ, where v or v ′ may be the empty set, but not
both. We say the image of e under ϵ are its endpoints and call ϵ the endpoint function. We
emphasize that ϵ is in general not an injection, see the definition of parallel edges below.
Definition 57. Let G = (V, E) be a finite graph and ϵ : E → 2V the endpoint function.
Then we say
1. If there exists e ∈ ϵ−1 ({v, v ′ }) that v and v ′ are adjacent.
2. If there exists e ∈ ϵ−1 ({v}) that e is a loop. Moreover, we say v is adjacent to itself.
3. The elements e ∈ E such that e are the pre-image ϵ−1 ({v, v ′ }) are parallel edges.
4. If ϵ−1 ({v}) = ∅ that v is a an isolated vertex.
5. If ϵ(e) is a directed set that e is a directed edge. Moreover, assuming v < v ′ , that v is
the tail of e and v ′ is the tip of e.
We say a graph G is directed if each of its edges are directed edges. If a vertex v ∈ V
is only the tail of a directed edge, that is, there is no e ∈ E such that ϵ(e) = {v, v ′ } with
v < v ′ , we say v is a terminal vertex.
Secondly we review those concepts we require to describe rooted trees, the graphs that
we shall use to instantiate stochastic processes as discrete probability spaces in this text. We
definte the degree of a vertex v ∈ V to be the number of edges that contain it. In general,
a vertex’s degree is greater than or equal to the number of edges that are adjacent to it.
An interesting fact about finite graphs is that the sum of the degrees of all its vertices is
equal to twice the number of its edges. We shall not make use of this fact explicitly, as the
purposes of graphs in this text is to either model time dependent experiments or to determine
Markov chains via matrix representations of transition graphs. However, we do note fact
should convince the reader that trees below are useful models of experiments that not only
occur over time, but ones in which only a finite number of subexperiments are allowed to
be performed in each time episode if we represent the simple events in subexperiments by
edges.
Pursuant to defining rooted trees we describe ways that vertices can communicate with
each other via a sequence of adjacent vertices in terms of their corresponding adjoining
edges. We want to recall these notions to define the brances of trees, below, and to discuss
the regularity of random walks on finite graphs later on. Let v, v ′ ∈ V , then a walk from
v to v ′ is a finite sequence of edges, say e1 , . . . , ek , such that v ∈ ϵ(e1 ) and v ′ ∈ ϵ(ek ) and
ei ∩ ei+1 ̸= ∅ for 1 ≤ i ≤ k. Notice this definition does not exclude parallel edges, so to
stress when they are excluded, we say a path is a walk with no parallel edges. Neither the
definition of walk nor the definition of path excludes repeated vertices, so a path without
repeated vertices is called a simple path. A graph is connected if there exists a walk in G
157
between any two vertices. We say that the length of a walk from vj to vi in G to be the
number of edges contained in a simple path joining them. Thus, there exists a walk of finite
length between all pairs of vertices in a connected graph.
A tree will exclude simple paths that back track if we order its vertices with respect to
the root, so to this end, we define a closed walk to be a walk that begins and ends at the
same vertex, a circuit to be a closed walk that is also a path, and a simple circuit to be a
circuit and a simple path.
Definition 58. Let T be a graph. We say T is a tree if T has no non-trivial circuits and is
connected. Assume that T is a tree with at least three vertices, then a vertex of degree one is
called a leaf and a vertex of degree greater than one is called a branch vertex. We write LT
for the set of leaves in T .
One can refine the structure of a tree T by distinguishing a leaf vertex as a root, as
follows. Let T be a tree, then given a vertex v ∈ VT , we say the ramifications of v are the
vertices v ′ ∈ VT such there exists a directed edge e ∈ ET where v is the tail and v ′ is the tip.
Given l ∈ LT with ramification r, we say l is a root of T . Once a root is chosen, we omit l
from the graphical representation retaining only r together with its ramifications. We say a
tree T with a choice of root r is a rooted tree.
The simple paths connecting the root r to the leaves l ∈ LT are called branches. We write
ΩT for the set of branches in a rooted tree. A subbranch is a simple path from the root r to
a branch vertex. The height of a branch vertex is the number of edges in the subbranch it
determines, denoted h(v). We remark this is a well-defined function from the set of vertices
onto the natural numbers since T has no non-trivial circuits and is connected. Furthermore,
the ramifications of a vertex v are related to the the same in terms of height by the formula
ht(v ′ ) = ht(v) + 1. Lastly, the maximal height of the leaves in T determine the height of
T . In time dependent experiments, we shall use the height of a branch vertex to represent
the time episodes at which the event whose predicate symbol is used to label it occurs in a
subexperiment.
Lemma 5.1. Any tree T such that |VT | ≥ 1 has at least one vertex of degree 1.
We shall require this lemma for the following proposition. To prove that it is true, suppose
you choose an arbitrary vertex v ∈ VT and follow v along some path contained in T . One
computes the degree of each vertex contained in such a path; if a vertex has degree 1, the
lemma is true, otherwise, one continues along said path. Since T is circuit free, by definition,
the vertices in the path never repeat and since |VT | < ∞ this degree checking routine must
halt in a finite number of steps. As such, at most, the last step in this routine will reach a
vertex of degree 1, as desired.
Proposition 33. Suppose T is a tree such that |VT | = n, then |ET | = n − 1.
We shall proceed by induction on the number of vertices. Indeed, the base case of n = 1
is trivial. So, let us assume the proposition is true for n. So consider a tree with T with
n + 1 vertices. By the lemma, T has a vertex of degree 1. Since T is connected, this vertex,
say v, is only connected to the rest of T by a single edge e. Removing v and e from T leaves
158
a subtree T ′ with n vertices and, by induction, n − 1 edges. Therefore, T has n edges, as
desired.
We conclude this section by remarking that this result justifies our assertion below that
simple events in a stochastic process-an experiment whose associated sample space is the
set of branches in a rooted tree-is a finite, and therefore discrete, probability space. Beside
satisfying the conditions to be an object in the category of discrete probability spaces, we
also observe that demonstrating that the edge set is finite justifies using rooted trees to
instantiate experiments that evolve over time that must also satisfy the so-called stochastic
axioms.
5.2 WORKED EXAMPLES
1. Prove for any positive integer n, if G is connected graph with n vertices and n − 1
edges, the G is a tree.
Solution Let n be a positive integer and suppose G is a graph as in the hypothesis.
It suffices to show that G contains no non-trivial circuits. Suppose otherwise, and let
C be such a non-trivial circuit. We may delete C from G and the result, say G′ ,
remains connected. If G′ has a non-trivial circuit, say C ′ , repeat this step. Continue
so until one obtains a connected graph that is circuit free, say Γ. Then by definition
Γ is a tree. Moreover, VG = VΓ by construction. As such, Γ has n − 1 edges, yet at
least one edge was removed to obtain Γ, so that the number of its edges is also less
than n − 1. This is plainly absurd. QED
2. We define a binary tree to be a rooted tree T such that each branch vertex has at most
two ramifications. We say it is full if the height of each branch is the same. Let k be
a positive integer and T a full binary tree with k branch vertices. Prove that T has a
total of 2k + 1 vertices and k + 1 leaf vertices.
Solution Partition VT = P1 ∪ P2 , where P1 ⊂ VT such that each vertex in P1 is a
ramification and P2 ⊂ VT such that each vertex in P2 is not a ramification. Plainly,
P2 = {r}, the root of T . So, by the addition rule, we have |VT | = |P1 | + |P2 |. Observe
that |P1 | = 2k since each branch vertex has exactly 2 ramifications. Therefore,
|VT | = 2k + 1.
Concurrently, we have |VT | = |B(V )| + |L(V )|, that is, branch vertices plus leaf
vertices. So, |VT | = k + |L(V )|. This gives 2k + 1 = k + |L(V )| or |L(V )| = k + 1, as
desired. QED
3. Let T be a binary rooted tree of height n and m = |L(V )|.
(a) Prove that m ≤ 2n
159
(b) Does there exist a binary rooted tree of height 5 with 38 leaf vertices.
Solution (a) As usual, we will proceed by induction on n. So, let n = 0, then we
must show that the number of leaves of T is less than or equal to 1. Then T consists
of only its root, so the base case is established. Now assume the statement is true for
n.
Solution (b) By the previous exercise, we know any binary tree of height 5 has at
most 32 = 25 leaf vertices. As such, there is no binary tree with the height and
number of leaf vertices of the question.
5.3
Matrix Representations of Graphs
In this section we introduce a relationship between finite graphs G and matrices A. We
call matrices obtained under this relationship matrix representations of a graph. Associating
a certain matrix to a graph G will be an important aspect of computing the probability a
Markov chain resides in a particular state at a particular time, below. In terms of the functor
described in section 1, we will obtain M from the transition graph associated to a stochastic
process satisfying the Markov axiom. We refer the reader to the literature for a treatment
of basic matrix algebra needed in this text.
Definition 59. Let A = {vij }1≤i≤n,1≤j≤n be an n×n-matrix and G = (V, E) be a finite graph
such that |V | = n, then we say A is a matrix representation of G if there exists a real-valued
function γ : ϵ(E) → R such that
Γ = {vij = γ({vj , vi })}1≤i≤n,1≤j≤n
The first example we consider is that of the adjacency matrix representation of a graph G.
To determine this example, we must compute the image of γ according to some rule obtained
from the graph G, itself. Recall from the previous section that two vertices vj and vi are
said to be adjacent if there exists e ∈ E such that ϵ(e) = {vj , vi }. For a fixed non-isolated
vertex vj , define the integer adj(vj ) = |{vi ∈ V | ∃ e ∈ E, ϵ(e) = {vj , vi }}|. That is, the
number of vertices in G adjacent to vj . If vj is an isolated vertex, we define adj(vj ) = 1.
Then adj(vj ) determines a matrix representation of G by the formula γ({vj , vi }) = adj(vj )
if there exists an edge e ∈ E such that ϵ(e) = {vj , vi } and 0 otherwise if vj is not an isolated
vertex. If vj is an isolated vertex, then the corresponding column is the elementary column
ej . Notice, in general, that adj(vj ) ̸= adj(vi ). Our second example is the normed adjacency
matrix representation associated to a finite graph G to be the adjacency representation with
adj(vj ) = 1 for all j above.
We present the following theorem together with its proof to emphasize an application
of the normed adjacency matrix representation of a graph to the question of determining
whether two vertices can communicate between each other via a walk of some finite length.
This theorem in part helps to justify the Google Page ranking Algorithm at the end of this
chapter.
160
Theorem 5.2. Let G = (V, E) be a finite graph and write V = {v1 , . . . , vn }. Let A be the
normed adjacency matrix representation. Then, for each l ∈ N, we have that the (i, j)-entry
of Al is the number of simple paths of length l from vj to vi
We proceed by induction on l. Let us establish the base case of l = 1. By the definition
of A, ai = 1 if vj is adjacent to vi , thus the number of walks of length 1. Now suppose the
statement of the theorem is true for l. So, consider the (i, j)-entry of Al+1 = A · Al , that is
ai1 b1j + · · · + ain bnj
where A = {aij } and Al = {bij }
By induction, ai1 = 1 if vj is adjacent to v1 and b1j is the number of simple paths of
length l from v1 to vj . But any edge from vj to v1 can be joined with any simple path of
length l from v1 to vi to proffer a simple path of length l + 1 from vj to vi . Thus, by the
multiplication rule, we have that ai1 b1j is the number of simple paths from vj to vi with
v1 as the second vertex in the path. We interpret each summand in the same way mutatis
mutandi. Since each walk from vj to vi of length l + 1 will have vr with 1 ≤ r ≤ n as the
second vertex in the walk, by the addition rule, the (i, j)-entry indeed computes the total
number of simple paths of length l + 1 from vj to vi , as desired.
One remarks that if an entry of Al is 0 for all l, then there are no paths of any length
between the corresponding pair of vertices in G. This inability for certain vertices to find
a path to one another, or one says to communicate, is of considerable importance in applications. Indeed, below we shall regularize certain stochastic matrices related to the normed
adjacency matrix representation in order to ensure that all vertices in a finite graph are able
to communicate, albeit probabilistically.
This last theorem is included to convince the reader that a Markov chain possesses socalled steady state vectors, which, abstractly, are elements of the eigenspace associated to
the eigenvector 1.
Theorem 36. The normed adjacency matrix is real symmetric. In particular, assuming
A ∈ M atn (R), then there exists an eigenbasis for Rn , say wi for i = 1, 2, . . . , n such that,
for the corresponding eigenvalues λi , one has
λi ≥ λi+1
and |λ1 | = 1
5.2 WORKED EXAMPLES
1. Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
161
Solution (a) A.
Solution (b) B.
Solution (c) C.
2. 6.2 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
3. 6.2 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
162
5.4
Stochastic Processes
In this section we set-up for the important link between the discrete probability theory we
have developed so far in terms of probability spaces and random variables and the formalism
of Markov chains, which may be viewed as a type of probabilistic automata. To establish
this link qua a functor, we shall introduce stochastic processes, which are generalizations of
processes in the sense of Chapter 2 and Bernoulli processes in chapter 4, for this purpose
alone. Indeed, we shall replace the rooted tree graph of height the number of factors in
the corresponding Cartesian product of sets by an arbitrary rooted tree graph T of finite
height to effect this generalization, in part. This replacement together with stochastic axioms
instantiates stochastic processes in our text as discrete probability spaces. We take Markov
processes as special cases of stochastic processes in the next section and then we interpret
those Markov processes in the context of standard linear algebra through an application of
the law of total probability. This interpretation proffers the so-called Markov chain as it is
commonly known in applications.
We begin by considering experiments that occur in k stages of discrete time as discrete
probability spaces. To do this, we must impose some restrictions on the scope of such
experiments: the restrictions we impose take the shape of the following three axioms in
experiments.
1. Axiom 1 Experiments that occur in k stages of discrete time require that k is a finite,
non-negative integer.
2. Axiom 2 The simple events of experiments that occur in k stages of discrete time
are determined by the sequence of simple events that occur in the sub-experiments
performed in each stage of the experiment contained within the simple event and the
probabilities of all of the simple events that occur in the sub-experiments are known.
3. Axiom 3 At any stage in an experiment that occurs in k stages of discrete time only a
finite number of sub-experiments are performed and only finitely many simple events
occur in each sub-experiment.
We say any experiment occurring in k stages of discrete time satisfying these three hypotheses is a stochastic process. Moreover, we refer to these hypotheses collectively as the
stochastic axioms. The second axiom is important, for we will use it to both determine predicate symbols for simple events in a stochastic process and to assign probabilities to these
events. The third axiom will allow us to verify the assignment of probabilities induced by
axiom 2 satisfy Kolmogorov’s axioms.
Thus, to instantiate a stochastic process as an object in our naive category of discrete
probability spaces, we impart the structure of a discrete probability space to a rooted tree T
with n vertices, hereafter written as sj ∈ VT , of height k by assuming the stochastic axioms
hold. Let ΩT be the set of branches b in T . According to the theorem of section 2, ΩT is finite
since it has finitely many edges and no cycles, and so, as usual, its σ-algebra is the power
set βT = 2ΩT . Thus we obtain a measure space (ΩT , βT ). In what follows we construct a
probability function with respect to this measure space that will satisfy Kolmogorov’s axioms
if we assume the stochastic axioms hold.
163
First, we model stochastic axiom 1 by regarding the heights t of the vertices of the rooted
tree as the time episodes, for t = 1, . . . , k.
Secondly, fix a branch vertex of height t − 1, say sj ∈ VT , and write si,y for the set of
y ∈ N ramifications of sj . The set of these ramifications models the y simple events of a
sub-experiment performed at time t for t = 1, . . . , k. This convention represents stochastic
axiom 3. If we write Jt for the set that indexes the vertices of height t − 1, then we denote
by Ωsj the ramifications of the vertices sj , j ∈ Jt . Observe that stochastic axiom 3 entails
that Ωsj is a discrete probability space for each j ∈ Jt .
Third we model stochastic axiom 2 as follows. We shall illustrate this labeling with only
a single branch and sketch the proof that our probability assignment satisfies Kolmogorov’s
axioms through this sketch, below. The reason for this is due to the surfeit of notation a
rigorous treatment would entail. The author feels the amount of notation distracts one from
the simple principle, rooted in chapter 1, underlying this construction.
By hypothesis, the vertex set VT is countable, so the set of leaves LT is a finite set, so
write LT = {l1 , . . . , lm } for its vertices. Furthermore, consider the ramifications of the root
r, a finite set also, and write Ωr = {r1 , . . . , rp } for its vertices. Then, with this notation, for
b ∈ ΩT , write bhg for the branch that has rh as its height 1 vertex and lg as its leaf. A key
point at this step is that, with this notation at hand, we interpret branches bhg ∈ ΩT in terms
of a stochastic process as its simple events. We do this because we interpret this simple event
as the joint event obtained from the sequence of outcomes of sub-experiments performed in
the stages that the simple event contains whose predicate symbols are the vertex labels.
Moreover, we write this sequence of outcomes from left to right in order of increasing height
as events whose intersection comprises the joint event, viz. rh ∩ s ∩ s′ . . . ∩ s′′ ∩ lg , for bhg ,
where s occurs at stage 2, s′ occurs at stage 3, and so on. We will remove the intersection
symbols ∩ and replace them with commas , below.
Let bhg be as above, then applying the formula for the probability of a joint event in
chapter 1 to a simple event with this labeling convention gives the probability function
P (b) = P (rh )P (s|rh )P (s′ |rh , s) · · · P (lg |rh , s, s′ , . . . s′′ )
for simple events bhg ∈ ΩT . Let us at last proceed to represent a stochastic process as
a the discrete probability space by showing this assignment satisfies Kolmogorov’s axioms
whenever the stochastic axioms hold. We will call this probability space the root space
associated to a stochastic process. Simple events in a stochastic process are thus represented
by the branches bhg ∈ ΩT with probability phg . The probability of an arbitrary event A ∈ βT
in a stochastic process is determined by the same formula we have used from chapter 1
throughout this text, viz.
X
P (A) =
phg
bhg ∈A
Theorem 37. Let T be a rooted tree with n vertices of height k and ΩT its set of branches.
Given any bhg ∈ ΩT , define
P (b) = phg
as above. Then the root space (ΩT , βT = 2ΩT , P ) associated to a stochastic process is a
discrete probability space if we assume that the stochastic axioms hold.
164
The proof of this theorem can be reduced to several steps. First, observe that we can
partition the root space by the ramifications of the roots into a partition of length p. So we
will write ΩT = ∪ph=1 Ah where each cell Ah contains branches who first vertex is rh ∈ Ωr . The
claim we need P
to sketch the proof of is P (rh ) = P (Ah ). Once we have established this, we
have P (ΩT ) = ph=1 P (Ah ) by the law of total probability. However, by stochastic axiom 3,
the sub-experiments performed at very stage in the experiment satisfy Kolmogorov’s axioms.
In particular,
Pp the sub-experiment performed at stage 1. Thus, if we establish the claim, then
P (ΩT ) = h=1 P (rh ) = 1. The fact that P (b) ≥ 0 follows from stochastic axiom 2. As
for Kolmogorov’s third axiom that, as usual, follows from our canonical probability function
formula, explicated above.
Now, to finish the section, we represent stochastic processes as random vectors in order
to make sense of the Markov Axiom in the section which gives the transition probabilities
crucially required to construct the functor from discrete probability spaces to Markov chains.
So, suppose (ΩT , βT , P ) is a root space. Observe there is a filtration of βT induced defining
βt to be the σ-algebra generated by the union of the branch spaces of height t. We say this
filtration is the branch filtration of the root space. Define Xt (b) = i for the subbranch b ∈ βt
such that the branch ends at branch vertex si at height t with predicate symbol i, called
the state. We emphasize here what the notation means by indicating that outcomes of the
sub-experiments performed at time t could have the same predicate symbol. We indicate
this by writing i so that, overall, the way to interpret the value of Xt is, "the outcome of
the stochastic process at time t is the one with vertex label i". Then the random vector
Z = (X1 , X2 , . . . , Xk ) is adapted to a root space with this filtration in the sense that Xt
defines a random variables with respect to the measure space (Ωt , βt ) with obvious notation.
This fact motivates the following definition.
Definition 60. Let (ΩT , βT , P ) be the root space of a stochastic process and Z as above a
random vector adapted to the branch filtration. Then we say Z is a stochastic process if for
each t = 1, . . . , k we have that the induced σ-algebra βXt is in bijective correspondence with
the σ-algebra β⟨X1 , . . . , Xt ⟩ generated by the t induced σ-algebras of the Xt .
Observe that a stochastic process Z has joint probability function
p(z) = P (X1 = i1 , X2 = i2 , . . . , Xk = ik )
In particular, we have
P (X1 = i1 , X2 = i2 , . . . , Xk = ik ) =
=P (X1 = i1 )P (X2 = i2 |X1 = i1 ) · · ·
P (Xk = ik | X1 = i1 , X2 = i2 , . . . , Xk−1 = ii−1 )
for b ∈ ΩT . Accordingly, we can solve for P (Xk = ik | X1 = i1 , X2 = i2 , . . . , Xk−1 =
ii−1 ). One should interpret the probability of this event in terms of the defining condition
of a stochastic process that relates the σ-algebras of the individual variables to the one
generated by the prior variables as to mean stochastic processes remember their history.
This experimental observation motivates the consideration of one of its special cases wherein
the experiment is only required to remember the prior stage in order to determine the
165
likelihood of its events. This remark motivates the introduction of the Markov axiom in the
next section. We shall further assume that the vector components are pair-wise independent
and given the same probability function. Under these assumptions we will obtain a Markov
Process from a stochastic processes defined as above. This is the penultimate step in the
construction of the Markov chains that we intend to study.
5.4 WORKED EXAMPLES
1. A part of the training routine for a high jumper is to attempt to clear the bar multiple
times at a fixed height. One routine consists of repeated attempts at clearing the bar
until they have two consecutive successes or two consecutive failures, or a total of four
attempts. A high jumping coach records the results of each attempt and determines
the following probabilities: the probability of success on the first attempt is 0.5, if any
attempt is successful the chance of success on the subsequent attempt is 0.7, and if
any attempt is a failure, then the probability of success on the subsequent attempt is
0.4. Assume the stochastic axioms.
(a) Illustrate the branch space of the stochastic process.
(b) Compute the probability of P (s|f, s)
(c) Compute the probability of the event the jumper clears the bar exactly twice.
(d) Compute the probability of the event the jumper fails exactly once.
Solution (a)
s
0.7
s
0.3
0.5
r
f
0.4
s
0.7
0.3
0.6
f
0.5
f
0.4
s
0.7
s
f
s
0.3
0.6
f
f
0.4
s
0.6
f
166
Let us give the probabilities with the following table assuming the stochastic axiom.

b |
ss
sf ss sf sf sf f

p(b) | 0.35 0.42
0.18
f ss
f sf s f sf f f f


0.09 0.14 0.024 0.036 0.3
Solution (b) By the stochastic axiom, P (s|f, s) = 0.4 as the subbranch sf s
describes the event, "a success, given a failure and a success".
Solution (c) Let A ∈ βT be the event "clears the bar exactly twice". As any b with
two s’s isPin A, then
P (A) = b∈A p(b) = P (ss) + P (sf sf ) + P (f ss) + P (f sf s) = 0.532
Solution (d) Proceeding as above with C ∈ βT the event "misses the bar exactly
once" weP
need to choose b with exactly one f . As such,
P (C) = b∈C P (b) = P (sf ss) + P (f ss) = 0.32
2. Consider the experiment of choosing at random one of two coins, labeled A and B,
respectively. The A coin is fair whereas the B coin has a heads on both of its sides.
Whatever coin is taken, the coin is then flipped. If a tails is obtained, a fair six sided die
is rolled. If a heads is obtained, the coin is tossed again. Assume the stochastic axioms
and recognize that, at each stage in this experiment, the individual subexperiments
are independent.
(a) Illustrate the branch space of the stochastic process.
(b) Compute the probability of the event a tail 2nd given that coin A is chosen.
(c) Compute the probability of a die roll of 1 given a tail 2nd and that coin A is
chosen.
Solution (a)
167
1
2
H
1
2
H
T
1
2
1
2
1
6
3
4
5
A
r
1
2
T
6
1
6
1
2
1
2
B
1
H
1
H
Let us give the probabilities with the following table assuming the stochastic axiom.



b |
p(b) |
AHH AHT AT 1 AT 2 AT 3 AT 4 AT 5 AT 6 BHH
1
8
1
8
1
24
1
24
1
24
1
24
1
24
1
24
1
2



Solution (b) So, we must compute the probabiity of the event T 2nd|A1st ∈ βT . To
this end, we proceed by the canonical definition of the probability function and solve
P (T 2nd ∩ A1st)
. Let us compute the numerator and
for P (T 2nd|A1st) =
P (A)
denominatorPindividually. To compute the numerator, we have
P (A1st) = b∈A1st p(b) where we sum over b ∈ ΩT such that A is the coin chosen. As
1
such, we have P (A) = by adding the values in the second row of the histogram in
2
1
part (a). As for the numerator, we have P (A1st ∩ T 2nd) = , again, by adding in the
4
1
given histogram. Thus, P (T 2nd|A1st) = .
2
1
1
Solution (c) P (1|T 2nd, A1st) = 24 =
1
6
4
3. 6.3 Determine the probability the following events:
168
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
5.5
Markov Processes and Markov Chains
In this section we render a stochastic processes with an additional axiom as a probabilistic automata in the sense of the first section. The establishment of this link introduces the
Markov chain and completes the informal construction of the functor from discrete probability spaces to dynamical systems. To this end, we first consider the stochastic processes
distinguished by an additional axiom that simplifies their structure. In applications, this simplification is often described as forgetfulness in the stochastic process. The sense in which
the axiom forgets anything is the sense in which it asserts one can know the probability a
stochastic process is presently in a state in terms of the previous state alone, thereby forgetting the prior states which the process had occupied altogether, where a state is defined to
be the fibre of a component variable of a stochastic process, viz. Xt−1 (i). Experimentally,
one can visualize states of a stochastic process as the various sub-branches ending at branch
vertex si of height t with label i.
This assertion drastically simplifies the complexity inherent in arbitrary stochastic processes and so motivates the following fundamental definition. Before stating it, let us recall
that one says a random vector Z = (X1 , X2 , . . . , Xk ) is independent and identically distributed if the component variables are pair-wise independent in the sense of chapter 4 and
p(x) = PXi (x) for all i, that is, the marginal probability functions are the same for each of
the component functions.
Definition 61. Let (ΩT , βT , Pr ) together with Z = (X1 , . . . , Xk ) be a finite stochastic process.
Then we say it is a Markov process provided
P (Xk = i | Xk−1 = j, X1 = i1 , X2 = i2 , . . . , Xk−2 = ij−1 ) = P (Xk = i | Xk−1 = j)
and Z is independent and identically distributed.
This axiom means that the probability a stochastic process is in state si at stage l given
its occupancy in the states s1 , s2 , . . . , sj at times 1, 2, . . . , l − 1 can be computed entirely in
terms of its conditional probability of being in a prior state, alone. Indeed, as emphasized
above, the axiom seems to forget the stochastic process’ occupation in its prior states or to
forget its history. Intuitively, one would not expect such an assertion to be true, in general,
169
in an experiment that occurs over time. However, there are many plausible scenarios in
which this axiom is experimentally verified.
The Markov axiom allows one to disregard the stage of a stochastic process in which a
subexperiment transpires. Accordingly we dispense with the tree graph representation of the
stochastic process to replace it by a directed graph representation. We will say that the leaf
vertices of the tree representation, namely LT = {s1 , . . . , sn } become the states of the Markov
process. Moreover, we will now write S = LT for a Markov process and refer to this set as
the state space of the Markov process. Indeed, by the defining axiom of a Markov process,
the states of a stochastic process simplify to the aforementioned state space. Moreover, the
state space shall supply the vertex set for the directed graph underlying a Markov process.
Observe next that the axiom of a Markov process defines the probabilities pij = P (Xl =
si | Xl−1 = sj ), called the transition probabilities from state sj to state si . They are of special
significance in what follows, for they determine the edges of a directed graph TM called the
transition graph associated to the Markov process. Given a Markov process, its transition
graph TM = (V, E) is the directed graph such that V = S and E consists of directed edges
e = {sj , si } if pij > 0, that is, the transition probability is strictly positive. In this way the
Markov axiom transforms the rooted tree underlying a stochastic process into this directed
graph illustrates how the Markov process defining axiom simplifies the former processes.
The matrix representation γ : ϵ(E) → R of TM defined by γ({sj , si }) = pij . The
corresponding matrix M = {pij }1≤i≤n,1≤j≤n is called the stochastic matrix of the Markov
process. This matrix representation is the first step we take toward constructing the Markov
chain, the expression of a Markov process as an automata. We shall investigate below the
behavior of Markov chains for arbitrarily large values of the time variable k, where l =
1, . . . , k was the stage variable of the stochastic process. The introduction of the stochastic
matrix M motivates our digression into some linear algebra that arises after translating
Markov processes into objects in the category of dynamical systems.
We say a vector v ∈ Rn such that its entries are both positive and add to 1 is a probability
vector. In particular, the set ColM of the transition matrix consists of probability vectors.
More generally, we say a matrix M such that ColM are all probability vectors is a probability
matrix.
proposition 5.3. Let M, N ∈ M atn (R) be probability matrices, then so is their product
M N ∈ M atn (R).
To prove the proposition, we reduce to show that if M is a probability matrix and v is
a probability vector, then their matrix-vector product is again a probability vector. We can
do this by the definition of matrix multiplication, that is, since M N = [M n1 M n2 · · · M nn ],
it suffices to show each M ni is a probability vector. Observe, M v = v1 m1 + v2 m2 + · · · vn mn
where mi ∈ ColM . We must show the n-entries of this column vector add to 1. Adding
entry-wise, we have v1 (m11 + m21 + . . . mn1 ) + v2 (m12 + m22 + . . . + mn2 ) + · · · vn (mn1 + mn2 +
. . . + mnn ) = v1 + v2 + . . . + vn = 1, as desired. We obtain the following immediate, and
useful, corollary.
Corollary 5.4. Let M ∈ M atn (R) be a probability matrix, then M k is a probability matrix
for k ∈ N.
170
Let us recall that a dynamical system is a sequence of vectors {xk }∞
k=0 together with
a matrix M that satisfies the difference equation M xk = xk+1 for all k. Given a Markov
process (ΩT , βT , Pr ), Z we have so far obtained the putative matrix M of the dynamical
system we claim corresponds to the same from the defining axiom. Next we will explain
how the sequence of vectors is obtained from the defining axiom from the requirements of a
stochastic process from the law of total probability.
Introduce the vector


P (X0 = s1 )


 P (X0 = s2 ) 
x0 = 

..


.
P (X0 = sn )
whose entries consist of the probabilities the Markov process occupies its various states at
time 0. It is for this reason that x0 is called an initial vector. Notice that by stochastic
axiom 2, these entries add to 1. Now to entirely translate a Markov process into a dynamical
system, we must the remaining vectors of the sequence. We have the following theorem.
Theorem 38. Let (ΩT , βT , Pr ) together with Z = (X1 , . . . , Xk ) be a Markov process, then
the transition probabilities pij satisfy the relation
X
P (Xl = si ) =
P (Xl−1 = sj )pij
j
The proof requires the law of total probability. Observe that the fibres of Xl−1 generate
the σ-algebra βl−1 and recall that since Z is adapted to the filtration of βT , we have βl−1 ⊂ βl .
Hence, by the law of total probability
X
P (Xl = sj ) =
P (Xl−1 = sj ∩ Xl = si )
j
=
X
=
X
P (Xl = si | Xl−1 = sj )P (Xl−1 = sj )
j
P (Xl−1 = sj )pij
j
where the last line is obtained by invoking the Markov axiom.
Continuing toward our goal of translating a Markov process into a dynamical system by
determining the remainder of the sequence of vectors that is a part of its data, we declare


a1 = P (Xk = s1 )
..


.




xk =  ai = P (Xk = si ) 


..


.
an = P (Xk = sn )
171
to be the vector whose entries are the probabilities the Markov process occupies its various
states at time k. In particular, entry ai is interpreted experimentally as the probability that
the Markov process is in state si at time k. Furthermore, we call these vectors the transition
vectors. Notice that by the theorem that xk is defined for arbitrary time values k.
To complete our objective of translating the a Markov process into a dynamical system
that preserves the probabilistic conclusions of the former, we have the following theorem to
make explicit how to interpret the Markov axiom in terms of the corresponding automata.
Theorem 39. Let (ΩT , βT , Pr ) together with Z = (X1 , . . . , Xk ) be a Markov process and
(S, M ) the induced state space and stochastic matrix. Consider the sequence of transition
vectors {xk }∞
k=0 defined above. Then the Markov axiom
P (Xl = i | Xl−1 = j, X1 = i1 , X2 = i2 , . . . , Xl−2 = il−2 ) = P (Xl = i | Xl−1 = j)
is equivalent to
M xk = xk+1
for all l.
The proof amounts to showing that the transition vectors satisfy the difference equation.
The entries of xk+1 are P (Xk+1 = si ), for i = 1, . . . , n, that is


a1 = P (Xk+1 = s1 )
..


.




xk+1 =  ai = P (Xk+1 = si ) 


..


.
an = P (Xk+1 = sn )
X
By the equality ai = P (Xk+1 = si ) =
P (Xk = sj )pij of the theorem can be interpreted as
j
the dot product of mi ∈ RowM and xk , where RowM = ColM T . This proves the theorem.
We have enough material now to define a Markov process as a dynamical system.
Definition 62. Let (ΩT , βT , Pr ) together with Z = (X1 , . . . , Xk ) be a Markov process. Then
we say the state space S together with the dynamical system {xi }∞
i=0 with endomorphism M ,
the stochastic matrix, is a Markov chain. Furthermore, we say x0 is the initial vector of the
Markov chain.
With notation as above, M xk = xk+1 is equivalent to M k+1 x0 = xk+1 for all k by the
corollary. Hereafter, we shall study the dynamical system ({xk }∞
k=0 , M ) to study Markov
processes. These objects are rich in structure and address several distinct experimental
scenarios of individual interest. To unburden notation, we shall denote Markoc chains by
(S, M, x0 ), below.
5.5 WORKED EXAMPLES
172
1. Consider the weather in a certain town. We assume for simplicity that it is always in
one of two states: bright or cloudy. Furthermore, consider the experiment of recording
which state the weather is in each day of a single week. Suppose the strange weather
in this town obeys the following rules. First, if it is bright one day, it is equally as
likely to be either bright or cloudy on the next day. Second, if it cloudy one day, then
there is a 75 % chance the next day is bright and a 25 % chance the next day is cloudy.
(a) If it is bright on Sunday, what is the probability it is bright on Thursday?
Solution (a) We agree that the weather events over a course of a week in this
certain town can be regarded as a Markov chain, with state space
S = {s1 = b, s2 = c}, for bright and cloudy, respectively. By hypothesis or weather
rules, we have that the transition probabilities are given by
p11 =0.5 p21 = 0.75
p21 =0.5 p22 = 0.25
0.5 0.75
which in terms of the stochastic matrix is M =
Finally, as the initial
0.5 0.25
1
vector is x0 =
we have, by the Markov axiom, that the probability the weather is
0
bright on Thursday is found in the first
entry of x4 . We obtain the value of this entry
0.60156
by computing M 4 x0 = x4 =
Accordingly, the probability the weather will
0.3984
be bright on Thursday is approximately 60 %.
2. Three human beings are throwing a ball to each other, say s1 , s2 , and s3 . Human being
s1 always throws the ball to human being s2 and human being s2 always throws the
ball to human being s3 . However, human being s3 is just as likely to throw the ball to
either of the other human beings.
(a) Compute the probability that the ball is in the hands of human being s3 after 7
throws if the ball is initially thrown by human being s3 .
Solution (a) We agree that the experiment of keeping track of the ball as it is
passed between the three human beings can be regarded as a Markov chain, with
state space S = {s1 , s2 , s3 }, representing human beings 1, 2, and 3, respectively. By
the hypothesis or the description of how the human beings throw the ball, we have
that the transition probabilities are given by
p11 = 0 p12 = 0 p13 = 0.5
p21 = 1 p22 = 0 p23 = 0.5
p31 = 0 p32 = 1 p33 = 0
173


0 0 0.5
which in terms of the stochastic matrix is M = 1 0 0.5 Finally, as the initial
0 1 0
 
0

vector is x0 = 0 we have, by the Markov axiom, that the probability the ball is
1
possessed by human being 3 after 7 throws is the
entry of x7 . We obtain the
 third 
0.1875
value of this entry by computing M 7 x0 = x7 = 0.4375 Accordingly, the probability
0.3750
that human being 3 is possesses the ball after 7 throws is 37.5 %.
3. Suppose a bit of data is always in one of two states, that of a 1 or that of a 0. Suppose
further it is transmitted in a signal that conveys the bit to itself every second and that
the probability its state is correctly transmitted is a real number p ∈ [0, 1]. Assume
that the signal transmission satisfies the axioms of a Markov process.
(a) Given the signal transmission corresponds to a Markov chain, determine its
stochastic matrix.
(b) Supposing p = 0.99, compute the probability the bit is in state 0 after 2 seconds
if it begins in state 0.
(c) Continuing, what is the probability the bit is in state 1 after 2 seconds?
Solution (a) Given our assumptions, we must only determine the transition
probabilities for states S = {s1 = 0, s2 = 1}. By the hypotheses, we have
p11 =p p21 = q
p21 =q p22 = p
where q = 1 − p since a correct and an incorrect
are mutually exclusive.
transmission
p q
This determines the stochastic matrix M =
q p
1
Solution (b) Assuming p = 0.99, we have, by hypothesis that x0 =
. By the
0
0.9802
Markov axiom, we have M 2 x0 = x2 =
Therefore, the probability the bit is
0.0198
in state 0 after 2 seconds is approximately 98 %.
Solution (c) Similarly, the probability the bit is in state 1 after 2 seconds is
approximately 1.9 %.
174
4. There are two white marbles in an urn labeled by the numeral 1 and 3 red marbles in
an urn labeled by the numeral 2. Moreover, urn 1 may only hold two marbles and urn
2 may only hold 3 marbles. Consider an experiment consisting of drawing one marble
at random from each urn in the following manner. Each minute one draws a marble
from each urn and the two marbles drawn are replaced into the urn from which they
were not drawn. The drawing repeats thusly every minute. Assume this experiment
satisfies the axioms of a Markov process.
(a) Determine the stochastic matrix M associated to this Markov chain.
(b) Compute the probability there are two red marbles in the urn labeled 1 after 3
minutes.
(c) Compute the probability there is 1 white marble in the urn labeled 2 after 5
minutes.
Solution (a) Let us first determine the states in the Markov process. There are
several ways to do this, but we shall consider S = {si } where si is the number of red
marbles i in the urn labeled 1. Accordingly, there are three states in this experiment.
Let us proceed to compute the stochastic matrix associated to this Markov chain.
Observe, if the experiment is in state s0 , it must transition to state s1 , for urn 2
contains three red marbles. Accordingly, one is deposited in urn1 
once the drawn
0
marbles are exchanged. Thus, column 1 of M is given by m1 = 1
0
Next, if the experiment is in state s1 , it transitions to state s0 if a red marble is
1 1
drawn from urn 1. The probability of this event is · . On the other hand, s1
2 3
transitions to state s2 if a red marble is taken from urn 2. The probability of this
1 2
event is · . Lastly, if the white marbls are exchanged, the system remains in state
2 3
1 1
1
s1 . The probability of this event is given by 1 − − = . Altogether, column 2 of
6 3
2
 
1
6
 
 
1

M is given by m2 = 
2
 
 
1
3
Finally, if the experiment is in state s2 . Suppose first a red marble is drawn from urn
2, then since a red marble must be drawn from urn 1, in this case, the experiment
1
shall remain in s2 . The probability of this event is . Next, if a white marble is
3
175
drawn from urn 2, then the experiment transitions to s1 . The probability of this
 
0
 
2
 
2

event is . Therefore, column 3 of M is m3 = 
3
3
 
 
1
3


0 0.16 0
Altogether, by the definition of stochastic matrix, we have M = 1 0.5 0.66
0 0.33 0.33
Solution (b) In terms of notation of the first solution, we must compute third entry
of x3 . Note that the experiment begins in state s0 by hypothesis.

 Thus we obtain this
0.08
value by the Markov axiom, which gives M 3 x0 = x3 = 0.6278 Accordingly, the
0.2739
probability of 2 red marbles in urn 2 after 3 minutes is approximately 27 %.
Solution (c) One white marble in the urn labeled
2 means
the experiment is in state


0.0919
s1 . As usual, we must compute M 5 x0 = x5 = 0.5841 Accordingly the probability
0.2878
of one white marble in urn 2 after 5 minutes is approximately 58 %.
5. 6.4 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by

1
1 0


Solution (a) A.1 0 31 
1 0 1
Solution (b) B.
Solution (c) C.
176
5.6
Communication Classes of Markov Chains and Simple Random
Walks
In this section, we define the communication classes of a Markov chain and look to the
most famous examples of Markov chains as important applications. Later in this chapter we
shall return to this definition to motivate the page rank algorithm.
Definition 63. Let (S, M, x0 ) be a Markov chain and si , sj ∈ S be states thereof. We say sj
communicates with si if there exists some powers k and l such that both the (i, j) entry in
M k and the (j, i) entry in M l are strictly positive.
Given this definition, we can define a relation R ⊂ S × S on the state space S of the
Markov chain by the rule
sj Rsi if and only if sj communicates with si
We have the following lemma.
Lemma 5.5. Let (S, M, x0 ) be a Markov chain. Then the relation R on S given by sj Rsi if
and only if sj communicates with si is an equivalence relation.
Let us show that R is reflexive by noting that taking both l and k to be 0 we have that
M 0 = I the identity matrix, so that R is reflexive. Next, the relation R is defined to be
symmetric by the definition of communication, so R is symmetric. Lastly, it is a routine
exercise to show that R is transitive. Suppose sh communicates with sj , so that there exist
′
k ′ and l′ satisfying the definition of R. Then one can show that the (i, h) entry of M k +k
′
and (h, i) entry of M l +l are strictly positive given that the relvant entries of M k and M l are
strictly positive, as well. The proof of this lemma invites the following definitions.
Definition 64. Let (S, M, x0 ) be a Markov chain and R the equivalence relation induced by
communicating states on the state space S. Then we say the equivalence class of sj with
respect to R, say Csj , is the communication class of sj . Furthermore, given sj , we say
1. The state sj is absorbing if Csj = {sj }
2. The state sj is reflecting if Csj = Csj+1 or Csj−1
3. The state sj is simply transitive if Csj = S. Equivalently, we say (S, M, x0 ) is a regular
Markov chain if there exists a simply transitive state.
According to the results of chapter 3, we can partition S into communication classes. The
effect this has is to decompose the attendant stochastic matrix M into block matrix form,
where a block corresponds to a communication class. We shall examine this effect below
through random walks, which are special cases of Markov chains. We may refine these cases
by dividing random walks into absorbing or reflecting walks induced by the definition.
Geometrically speaking, we define a simple random walk to be a Markov chain whose
transition graph is a directed walk between two vertices; more precisely, we have the following
definition.
177
Definition 65. Let (S, M, x0 ) be a Markov chain, such that |S| = n,. Then we say it is a
simple random walk if
1. S is a totally ordered set, that is s1 < s2 < · · · < sn−1 < sn is the total order on S
2. Given si ∈ S \ {s1 , sn } for i = 2, 3, . . . , n − 1, the transition probabilities are pi,i+1 = q
and pi+1,i = p, where 0 < p < 1 and q = 1 − p
3. We say it is an absorbing random walk if the least and greatest states with respect to
its total order are absorbing states and we say it is a reflecting walk if the same states
are reflecting states, instead.
In terms of the stochastic matrix M , we can distinguish absorbing random walks from
reflecting random walks according to the transition probabilities in the first and last columns.
To wit, if (S, M, x0 ) is an absorbing random walk, then its stochastic matrix is given by the
n × n matrix


1 p 0 ··· 0 1
0 0 p 0 0 0




..
. 0 0
0 q 0

M =
0 0 q 0 p ... 



. . . .
 .. .. .. . . 0 0
0 0 0 0 q 1
and if (S, M, x0 ) is a reflecting random walk, then its stochastic matrix is given by


0 p 0 ··· 0 1
1 0 p 0 0 0 




..
. 0 0
0 q 0

M =
0 0 q 0 p ... 



. . . .
 .. .. .. . . 0 1
0 0 0 0 q 0
Notice that random walks of either class are not regular Markov chains. Indeed, the
above stochastic matrices are in block form for distinct communication classes. We label the
transition probability as i increases above the connecting edge and below the connecting edge
as i decreases to indicate the matrix representation of the path for i = 2, 3, . . . , n−1. We can
indicate whether the least and greatest states are absorbing by loops and the appropriate
transition probability labeling. Observe that
⟲1 s1
q
0
s2
q
p
···
q
p
sn−1 0 ⟳1 sn
p
illustrates an absorbing walk. Similarly, we can indicate a reflecting random walk by the
appropriate below edge labeling. Observe.
178
s1
q
1
s2
q
p
···
q
p
sn−1
1
p
sn
illustrates a reflecting random walk.
5.6 WORKED EXAMPLES
1. This example is well-known as The Gambler’s Ruin. Consider the following game: a
gambler flips a fair coin and calls heads or tails. If correct, they win a dollar and if
incorrect, they lose a dollar. Suppose the gambler quits the game after they lose all
of their money or win a fixed amount and must pay an integral portion of that fixed
amount to buy-in or begin playing. Assume this game satisfies the axioms of a Markov
process.
(a) Describe the state space corresponding to this game.
(b) Compute the stochastic matrix M determined by the game.
(c) Assuming the gambler has 7 dollars to bet, and buys-in to play with 4 dollars,
what is the probability gambler has 6 dollars after 10 flips?
Solution (a) The states in this process are described by the monetary values
awarded when playing, so S = {0, 1, 2, . . . , n}, where n is the fixed maximum of
money that can be won.
Solution (b) As the Gambler’s Ruin is an absorbing random walk, in general, its
stochastic matrix is the (n + 1) × (n + 1) matrix


1 0.5 0 · · · 0 0
0 0 0.5 0
0 0




..
. 0 0
0 0.5 0

M =
0 0 0.5 0 0.5 ... 


. .

.
.
..
. . 0 0
 .. ..
0 0
0
0 0.5 1
where p = q = 0.5 as the win and lose of a dollar is determined by the flip of a fair
coin.
 
0
0
 
0
 
0

Solution (c) The initial vector for this random walk is given by x0 = 
1 since the
 
0
 
0
0
179
gambler buys in with 4 dollars.
 As such, we compute the 7th entry of x10 by
a0
a1 
 
a2 
 
a3 
10

computing M x0 = x10 = 
a4 
 
a5 
 
a6 
a7
so the probability of having 6 dollars after 10 flips is a6 %. Remark: This problem was
too large to compute with free online sources, so I will fill in the actual value at a later
date.
2. 6.5 An mathematical called a Q is at an integral point on the x-axis between the origin
and the point 5.
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
3. 6.5 Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
180
5.7
Random Walks on Finite Graphs
In this section we define a random walk on a graph to be a Markov chain whose state
space is the vertex set a finite graph G. In this sense, the material of this section generalizes
that of the previous section if one regards finite graphs as more general than directed walks
between vertices. Accordingly, in general, such Markov chains will be irregular, as it is
possible for finite graphs to possess both loops and isolated vertices. Such features of a finite
graph are translated into Markov chains as a non-trivial partition of the state space into
communication classes which is to say there exist no simply transitive vertices.
The key application of this material below is to the world wide web and, in particular, to
describing a page ranking algorithm for the same by modeling webpages online as vertices and
communication classes as hyperlinks between the same. We can rank how popular websites
are, in a sense, when the underlying Markov chain is regular. However, as in general there
exist obstructions to the regularity of Markov chains predicated upon finite graphs, we shall
furnish an algorithm for regularizing such chains. Before we explore this algorithm and its
meaning in the next section of this chapter, we must define the adjacency index of a vertex
v ∈V.
Definition 66. Let G = (V, E) be a finite graph and v ∈ V . Suppose v is not an isolated
vertex in G, then define A(v) = {v ′ ∈ V | e ∈ E, ϵ(e) = {v, v ′ } } then define its adjacency
index to be
adj(v) = |A(v)|
Otherwise, if v is an isolated vertex, we define adj(v) = 1.
Suppose further that G is a directed finite graph. Then one defines, for v ∈ V , the set
A(v) = {v ′ ∈ V | e ∈ E, ϵ(e) = {v, v ′ }, v < v ′ }. Then one defines the directed adjacency
index by replacing A(v) by A(v), denoting this index by dadj(v).
With this definition at hand, we can define random walks on finite graphs.
Definition 67. Let (S, M, x0 ) be a Markov chain and G = (V, E) be a finite graph. We say
the Markov chain is a random walk on G if
1. The state space S = V is given by the vertices of G
2. The stochastic matrix is determined by the reciprocal adjacency matrix representation
of G. Specifically, for M = [m1 · · · mj · · · mn ]
where
 
a1
 .. 
.
 
mj =  ai 
.
 .. 
an
181
1
if there exists e ∈ E such that ϵ(e) = {vj , vi } and 0 otherwise, for 1 ≤ i, j ≤ n.
adj(vj )
Suppose that G = (V, E) is a directed finite graph. We say the Markov chain is a directed
1
in the definition of mj .
random walk on G if one replaces ai by ai =
dadj(v)
ai =
One notes that the columns of M consist of probability vectors, that is, the sum of their
entries is 1, by the definition of adjacency index. In general, M shall be in block form, as
in general the quotient set with respect to the communication equivalence relationship is
non-trivial. The reciprocal adjacency matrix representation preserves the partition of S. In
particular, isolated vertices and cycles in G will obstruct the corresponding random walk on
G from being a regular Markov chain. As already stated, we shall explore the implications of
this observation in the following section and learn there of an algorithm to attach a regular
Markov chain to an arbitrary finite graph G.
5.7 WORKED EXAMPLES
1. 5.7 Consider the finite graph G
2
3
4
1
5
6
7
(a) Compute the stochastic matrix of the associated random walk on G.
(b) Assuming one begins to walk on G at vertex 6, what is the probability to walk
from vertex 6 to vertex 2 in 3 steps?
(c) Assuming one begins to walk on G at vertex 1, what is the probability to walk
from vertex 1 to vertex 4 in 5 steps?
182
Solution (a) To compute M , we shall compute the reciprocal adjacency matrix. We
have adj(1) = 2, adj(2) = 3, adj(3) = 4, adj(4) = 1, adj(5) = 2, adj(6) = 3 and
adj(7) = 1. Therefore, by definition, we have

0 0.33 0.25 0 0
0
0.5 0 0.25 0 0.5 0

0.5 0.33 0 1 0 0.33

0 0.25 0 0
0
M =
0
 0 0.33 0 0 0 0.33

0
0 0.25 0 0.5 0
0
0
0 0 0 0.33

0
0

0

0

0

1
0
 


0.0833
0
0
0.0417
 


0
0.4028
 


. As x3 =  0  so there is a
0
Solution (b) We compute M 3 x0 = x3 for x0 = 
 


0
0.2778
 


1
 0 
0
0.1944
4.17 % chance of being at vertex 2 in 3 steps.
 
1
0
 
0
 
5

Solution (c) Similarly, one computes M x0 = x5 for x0 = 
0
0
 
0
0
and takes the a4 entry in x5 .
2. 6.6 Consider the experiment of robot that moves from his current position in a room
to an adjacent room accesible by a doorway once every minute. The robot is equally
as likely to move to an adjacent room that is accessible by a doorway. Suppose there
are five rooms, and room 1 is adjacent to rooms 2 and 2; room 2 is adjacent to rooms
1 and 4; room 3 is adjacent to rooms 1,4, and 5; room 4 is adjacent to rooms 2, 3, and
5; and room 5 is adjacent to rooms 3 and 4.
(a) Compute the probability the robot is in room 3 after 5 minutes if it begins in
room 3.
(b) Compute the probability the robot is in room 4 after 10 minutes if it begins in
room 1.
183
Solution (a) Let us model the adjacent rooms by a finite graph G and to treat the
robot’s meanderings as a random walk on the same. Thus, G is given by
5
3
4
1
2
where an undirected edge connects two vertices if the rooms corresponding to their
labels are adjacent. As such, we 
compute the stochastic matrix
as the reciprocal

0 0.5 0.33 0
0
0.5 0
0 0.33 0 



0 0.33 0.5
adjacency matrix. Indeed, M = 0.5 0
 so we compute
 0 0.5 0.33 0 0.5
0
0 0.33 0.33 0


 
0.1507
0
0.2143
0


 
 for x0 = 1. So the probability that the robot is in room 3
0.2701
M 5 x0 = x5 = 


 
0.2143
0
0.1507
0
after 5 minutes if it began moving from room 3 initially is 27.01 %.
 
1
0
 

Solution (b) Similarly, we compute M 10 x0 = x10 and take its a4 entry, for x0 = 
0
0
0
3. 6.6 Consider the finite directed graph G
184
2
3
4
1
5
6
7
(a) Compute the stochastic matrix of the associated directed random walk on G.
(b) Assuming one begins to walk on vertex 5, what is the probability to walk to
vertex 3 after 4 steps?
(c) Assuming one begins to walk on vertex 6, what is the probability to walk to
vertex 7 after 6 steps?
Solution (a) To compute M , we shall compute the reciprocal directed adjacency
matrix. We have adj(1) = 1, adj(2) = 2, adj(3) = 3, adj(4) = 1, adj(5) = 2, adj(6) = 3
and adj(7) = 1. Therefore, by definition, we have

0 0.5 0 0 0
0
0 0 0.33 0 0.5 0

1 0
0 0 0 0.33


0
M = 0 0 0.33 1 0
0 0.5 0 0 0 0.33

0 0 0.33 0 0.5 0
0 0
0 0 0 0.33

0
0

0

0

0

0
1
185
 


0
0.1319
0
0.0833
 


0
0.0860




. As x4 = 0.1389 so there is a
0
Solution (b) We compute M 4 x0 = x4 for x0 = 
 


1
0.2199
 


0
0.0833
0
0.2546
8.6 % chance of being at vertex 3 in 4 steps.
 
0
0
 
0
 
6

Solution (c) Similarly, one computes M x0 = x6 for x0 = 
0
0
 
1
0
and takes the a7 entry in x6 .
5.8
Regular Markov Chains and the Page Rank Algorithm
Let (S, M, x0 ) together with its sequence of transition vectors {xk = M k x0 }∞
k=0 be a
Markov chain. We say lim xk = σ is a steady state vector of the Markov chain provided
k→∞
the limit exists. We emphasize that σ ∈ {xk }∞
k=0 so σ is a probability vector. Equivalently,
we say a probability eigenvector in the eigenspace of the eigenvalue 1 for the stochastic
matrix, viz. σ ∈ Null(M − I), is a steady state vector for (S, M, x0 ). One recognizes how
these definitions are equivalent according to the naive computation lim xk = lim M k x0 =
j→∞
k→∞
M lim xk−1 = M σ = σ.
k→∞
We interpret the entries of σ with respect to the Markov chain in two ways. First, the
entries of σ are interpreted as the long run probabilities of the Markov chain, which is to
say, they reflect the probabilities the Markov chain resides in its various states after an
indefinite period. To abuse notation for insight, one could write x∞ = σ to remember this
interpretation. Second, as k is arbitrarily large, one could interpret the entries of σ as the
proportion of time the Markov chains resides in its various states. It is according to this
second interpretation that one obtains the page ranking algorithm.
Theorem 40. Let (S, M, x0 ) be a Markov chain and suppose there exists a steady state vector
σ. Then
lim M k = [σ σ · · · σ]
k→∞
where [σ σ · · · σ] is the n × n matrix whose columns are σ.
The theorem shows that, given the existence of a steady state vector, one is able to
compute high powers of M . Accordingly, the existence of such σ is a desirable feature of a
186
Markov chain (S, M, x0 ). As such, one would like to know conditions under which a a steady
state vector exist for a Markov chain. We have the following definition, which is equivalent
to the definition of section 5.6 in terms of communication classes of S.
Definition 68. Let M be the stochastic matrix of a Markov chain (S, M, x0 ). Then we say
M is regular if there exists some j ≥ 0 such that M j has only positive entries.
When M is the stochastic matrix of a Markov chain and M is regular, one interprets
this feature in terms of the Markov chain to mean that all states can communicate with
each other in exactly j stages if j is the minimum value such that M j has only positive
entries. Equivalently, there exists a simply transitive state in S, so its partition with respect
to its communication classes has only one cell. The following theorem characterizes when
this occurs.
Theorem 41. Let (S, M, x0 ) be a Markov chain such that M is regular. Then there exists
a unique steady state vector σ for the Markov chain.
The proof relies upon the fact that if M is regular, it is a real-symmetric matrix, and so,
by the spectral theorem, has a basis of eigenvectors together with an eigenvector associated
to eigenvalue λ = 1. Since N ull(M − I) is cyclic, therefore, dimR N ull(M − I) = 1. Thus,
there exists a unique stead state vector σ ∈ N ull(M − I).
Our main application of this material is the page ranking algorithm. Let (S, M, x0 ) be a
arbitrary Markov chain. The page ranking algorithm shall associate it to a regular Markov
chain, say (S, MR , x0 ), in a canonical way. The justification for this algorithm is that by
executing it one is assured that there exists a steady state vector σ with respect to the
regularized Markov chain. The interpretation of the entries of its steady state vector σ as
the proportion of time the Markov chain occupies its various states can be used to rank
the states according to these occupation times. In particular, if we model the internet by a
directed graph as in the previous section, regularizing the associated Markov chain allows us
to rank its webpages. This application is both famous and lucrative.
Given a random walk on a finite directed graph G = (V, E), say (S, M, x0 ), the two
obstructions to its regularity we must consider are loops and isolated vertices. The reason
for this, we recall, is that they correspond to elementary columns in M obtained from
the reciprocal adjacency matrix representation, representing in block matrix form distinct
communication classes. It follows from matrix multiplication that no matrix with elementary
columns is regular. It is for this reason the following algorithm is focused upon amending
these obstructions to regularity for a random walk on a finite directed graph.
Proposition 34. PAGE RANKING ALGORITHM
Let (S, M, x0 ) be a random walk on a finite directed graph G = (V, E). Then the following
algorithm replaces its stochastic matrix with a necessarily regular matrix, MR .
1. If vj ∈ V is a terminal vertex, replace the corresponding column j of the matrix representation of G by the column vector whose entries are all the reciprocals of the cardinality of V . Specifically, if |V | = n, then change the column j under vj into
187
1
n
 .. 
.
 
1
 
n
 .. 
.
 
1
n
Let M ⋆ be the result of applying step 1 to all terminal vertices.
1
2. Let 0 ≤ p ≤ 1 and K be the n × n matrix whose entries are all identically , that is,
n
1
ki,j = for all 1 ≤ i, j ≤ n. Then compute
n
MR = pM ⋆ + (1 − p)K
To prove this proposition, we simply note that the algorithm produces another probability
matrix, MR . Insofar as whether it is regular, by construction, each entry is positive so a
fortiori the matrix MR is regular. We say that MR regularizes M . The algorithm motivates
the following definition.
Definition 69. Let (S, M, x0 ) be a random walk on a finite directed graph G = (V, E).
Then we say the Markov chain (S, MR , x0 ) is the regularization of (S, M, x0 ), where MR =
pM ⋆ + (1 − p)K.
5.8 WORKED EXAMPLES
1. Prove that M has a steady-state vector if and only if det(A − I) = 0
2. Prove that every probability matrix M has a steady-state vector.
Solution Suppose that M has a steady state vector, then the linear system
(M − I)x = 0 has a non-trivial solution. As such, (M − I) is a singular matrix and
det(A − I) = 0. Conversely, if the characteristic equation is vanishes, then λ = 1 is an
eigenvalue of M , so λ − 1 has multiplicity at least one or dim(N ull(M − I)) ≥ 0, as
desired.
P
Solution Observe that, as M is a probability matrix, i mTi − ei = 0, where
ei ∈ ColIn , since the columns of M are probability vectors. Accordingly, the set
{mTi − ei } is linearly dependent and as such, M T − I = [mT1 − e1 · · · mTn − en ] is a
singular matrix. Therefore, det(M T − I) = 0, but
det(M T − I) = det((M − I)T ) = det(M − I). Thus, by the previous proof, M has a
steady-state vector, as desired.
188

0



3. Given the probability matrix M = 
1



0
1
6
1
2
1
3

0


1

3


2
3
(a) Does a steady-state vector exist?
(b) If a steady steady-state vector does exist for M , please compute it.
(c) Compute limj→∞ M j
Solution (a) Since M is a probability matrix, we have by the previous exercise that
a steady state vector exists.
Solution (b) Let σ be the steady state vector of M . Then to compute it we must
find a basis of the eigenspace associated to λ = 1. As such,we
have
1
 
1
 13 
6
 
 
 
 where c ∈ R. We shall take σ =  6 
N ull(MI ) = c 
1
 13 
 
 
 
6
1
13
Solution (c) 
Proceeding bythe theorem in this section, we have
1
1
1
 13 13 13 


6

6
6
j

limj→∞ M = 
 13 13 13 




6
6
6
13 13 13
4. Determine the probability the following events:
(a) A =
(b) B =
(c) C =
We proceed by
Solution (a) A.
Solution (b) B.
Solution (c) C.
189
5.9
Chapter 5 Homework Exercises
1. Consider the undirected graph with vertex set V = {1, 2, 3, 4, 5} and edge set E as
indicated in the diagram below:
1
2
5
3
4
Compute the stochastic matrix M for the random walk (S, M, x0 ) on this undirected
graph.
2. Consider the directed graph with vertex set V = {1, 2, 3, 4} and edge set E as indicated
in the diagram below:
1
2
3
4
Compute the stochastic matrix M for the random walk on this directed graph.
 
0

3. Given a Markov Chain (S, M, x0 ) where S = {s1 , s2 , s3 }, x0 = 1, and
0






.1 .2 .3
.26 .23 .2
.21 .22 .23
M = .2 .3 .4 M 2 = .36 .33 .3 M 3 = .31 .32 .33
.7 .5 .3
.38 .44 .5
.47 .45 .44
190






.22 .22 .22
.22 .22 .22
.22 .22 .22
M 4 = .32 .32 .32 M 5 = .32 .32 .32 M 6 = .32 .32 .32
.44 .45 .45
.45 .45 .45
.45 .45 .45
compute the probability the chain is in state s3 at time 4.
4. Consider an unbiased random walk on the set S = {1, 2, 3, 4, 5} i.e the transition
1
probability between adjacent states is p = q = . What is the probability of moving
2
from state 2 to state 3 in exactly three steps if:
a.) the walk has reflecting boundaries?
b.) the walk has absorbing boundaries?
5. Given the directed graph
1
2
3
4
5
respond to the following questions:
a.) Compute the stochastic matrix M associated to the random walk on this directed
graph.
b.) Compute the modified matrix M ⋆
c.) Given your response in b.) compute the regular matrix MR for p = .85.
d.) Compute the steady-state vector σ associated to MR , that is, σ such that
(M − I)σ = 0. You may use a computer algebra application in order to do this and
write the solution.
6. Given the directed graph below
191
1
2
3
5
6
4
7
respond to the following questions:
a.) Compute the stochastic matrix M associated to the random walk on this graph.
b.) Compute the modified matrix M ⋆ .
c.) Compute the regular matrix MR for p = .45.
d.) Does a steady-state vector σ for the regular matrix exist? If so, why?
192
6
Markov Decision Processes
6.1
Introduction
In this section we introduce reinforced learning with an eye toward obtaining from its
introduction an interpretation of Markov Decision Processes as a special case thereof in
the next section. These are important generalizations of Markov Processes and Markov
chains discussed in the previous chapter, with applications in artificial intelligence that are
compelling to both modern students and researchers.
Reinforcement learning is a paradigm for machine learning, which broadly speaking is
a school of thought that asserts machines qua agents might learn by interacting with their
environments. Unlike other such machine learning paradigms, there is no supervisor, rather
only a reward signal; unlike in general, feedback from the environment to the agent may
be delayed, or is not necessarily instantaneous; there is a (discrete) time variable in the
environment that both induces a so-called history for the agent in their environment; and the
agent’s actions affect the subsequent information it receives from its environment. Together,
these various specifications outline what is meant by reinforcement learning.
To set up the reinforcement learning paradigm, let us suppose there exists an agent in
some environment where time is measured discretely by a variable t who receives a reward
from their environment give by some scalar value, say rt , at time t. We shall regard this scalar
as an indication of how the agent in performing in their environment at time t. Furthermore,
we shall insist that the agent’s goal is to maximize rt . Our insistence upon this goal is usually
phrased in terms of the reinforcement learning paradigm as the so-called reward hypothesis,
that is, all goals may be described by the maximization of their expected cumulative reward.
An experimental virtue of including such an ansatz in the context of machine learning is
that it focuses our study of sequential decision making on a selection of actions for an agent
to make in their environment that maximizes their total future reward. Since the actions
that an agent may take can have long term consequences in their environment, then, for
example, under the reward hypothesis, significant rewards might be delayed by actions but
for the purpose of achieving their maximization.
Pursuant to a rigorous formulation of the reward hypothesis in the paradigm of reinforcement learning, let us next imagine that when an agent observes their environment at time t,
symbolized by ot , this interaction incurs reward rt at the same time. We will then write at
for the action they had taken at time t. Experimentally speaking the environment receives
at and returns feedback to the agent ot+1 , which in turn, incurs reward rt+1 et cetera. Allowing time to begin discretely at time value t = 1 such an outline of agent and environment
interactions induces a history of the agent’s interactions until time t defined as the sequence
Ht = (o1 , r1 , a1 , o2 , r2 , a2 , . . . , at−1 , ot , rt )
with notation as above.
The role that history plays in the reinforcement learning paradigm is that it determines
both the state of the agent and the state of the environment after the sequence of observations, rewards, and actions that H − T represents. Indeed, we define, abusively for the
moment, the state of the agent at time t, say sat , to be a function of Ht , or st = f (Ht ) where
f is a function of several variables. The state of the environment is defined similarly. In
193
the generality of reinforcement learning, we should agree that it is experimentally plausible
to distinguish between the state of the agent and the state of the environment. The latter
state may in general be concealed from the agent, so that the agent’s state may not depend
upon observations relevant to the actions undertaken by the environment. Experimentally
we shall consider a specific agent state, called an information state or equivalently, a Markov
state, for it is defined by the property that an agent state sat is Markov if and only if it
satisfies the Markov axiom, that is, P (sat+1 |sat ) = P (sat+1 |sat , sat−1 , . . . , sa1 ). Given a Markov
state, once an agent state is known, the agent’s history may be disregarded. Observe that if
either an agent or environment state is Markov, then so is its history.
There are already afoot several possibilities for an experimenter to simplify a reinforcement learning paradigm. For example, one could assume a reinforcement learning paradigm
is fully observable or, in terms of our notation so far, that 0t = sat = set for all t. This assumption will determine a Markov Decision Process in the next section. Other possibilities exist,
of course, that one might characterize, for example, as partially observable, say when an
agent indirectly observes their environment. Such reinforcement learning paradigms are partially observable Markov Decision processes when their states are Markov. Experimentally
relevant examples of these are recurrent neural networks.
Beyond the observable capacity of an agent in a reinforcement learning paradigm, there
are other components induced by the reward hypothesis that require consideration. In this
text, the additional components that we consider are policies, value functions, and agent
models. A policy is a function from the set of agent states to the set of actions, informally
denoted by π : A → S where s 7→ s means the policy is deterministic and s 7→ P (a|s) means
the policy is stochastic. A value function of a policy π, say vπ is a prediction of future reward
for the agent. Formally, it is a real-valued function of the agent states, given by
vπ (s) = E[rt+1 + γrt+2 + γ 2 rt+3 + . . . |st ]
where γ ∈ R is a so-called discounting factor. Finally, an agent model will predict what the
environment does next in the sense of predicting the observation ot that it emits to the agent.
To symbolize a model, we shall write P for its prediction of set and R for its prediction of
reward rt+1 . We shall expound upon these components in greater detail under the auspices
of a Markov Decision process, below.
Two fundamental problems arise in reinforcement learning paradigms. First, one can
consider learning paradigms such that the environment is initially known and that the agent
interacts with in order to improve their policy. The second is when the agent model is
known instead, so that the agent deliberates, plans, and reasons about their actions before
undertaking them. In this sense, given that both problems are present in reinforcement
learning paradigms, one regards this approach to sequential decision making as a trial and
error style of the same. Moreover, the agent should work to discover a good policy from its
experience of the environment without losing too much reward along the way. Below we shall
now consider a more specific example of this paradigm so that we may be more concerete.
6.2
Finite Horizon Markov Decision Processes
Given an agent that can influence the selection of states in a Markov process as it evolves
through episodes of time by choosing actions, we consider an agent whose goal it is to choose
194
a sequence of actions which causes the Markov process to perform optimally with respect to
some pre-determined performance criterion or set of rewards. Below we provide the details
of this process under certain restrictions.
Explicitly, the agent makes decisions at points in time called episodes. Let T denote the
set of decision episodes. We shall adopt the convention that T is discrete i.e. either finite or
countably infinite. Furthermore, we shall insist that when T is discrete that the agent makes
at decision at each element of T . Lastly, if |T | = N for some finite integer, then we say that
corresponding Markov decision process is of finite horizon. We adopt the convention that no
decision is made at time episode N .
At each episode the system occupies a state. Let S be the set of states. If the agent
observes the system in state s ∈ S at some episode in the system, he may choose an action
from the set of actions allowable at state s, denoted As . Define A = ∪s∈S As to be the action
set. We will assume both S and As are discrete in what follows.
Next, actions may be either deterministic or probabilistic. Define P(As ) to be the set
of probability functions defined on βs = 2As . As we assume the As are countable, then so
is the action set A, itself. Therefore, define similarly P(A). Given the above, we say that
choosing an action a at s probabilistically means to choose P ∈ P(As ) such that action a
is chosen with probability P (a).
Third we consider both the rewards and transition probabilities that characterize an
MDP. Fix a ∈ As at episode t ∈ T . We write rt (s, a) for the reward the agent receives at
episode t given action a at state s. Intuitively we think of rt (s, a) ≥ 0 as a reward and
rt (s, a) < 0 as a cost at episode t for the agent, respectively. Further, we write Pt (s′ |s, a)
for the probability that the agent transitions to state s′ at episode t given action a at state
s. We insist that, in this notation, the reward’s value or expected value to be known to the
agent before choosing an action and that it not be affected by future actions.
When the reward depends on the state of the process occupies at the next episode, we
write rt (s, a, s′ ) to denote the reward for the agent at episode t + 1 at state s′ given action a
at state s in episode t. Thus the expected value at episode t may be computed by evaluating
P
rt (s, a) = s′ ∈S rt (s, a, s′ )Pt (ss ′|s, a)
We have the following definition.
Definition 70. We say the sextuple
(S, (A(i), A (i)), pij (a), r(i, a), Vβ )
is a discrete time Markov Decision Process, where
1. S is the state space.
2. (A(i), A (i)) are countable sets.
3. When the agent is in state i ∈ S at a particular episode and an action α ∈ A(i)
is undertaken, the agent will then transition to state j in the next time episode with
probability pij (α).
4. When the agent undertakes action α ∈ A(i) at a particular episode, it incurs an extendreal valued reward r(i, α).
195
Let (S, M, x0 ) be a Markov chain. Suppose that the agent observes the chain in state sj
at time k, then the agent may choose some action α for the chain at this state and time.
Specifically, we regard actions A as the set of set-endomorphisms of the state space of the
Markov chain, End(S) and define Asj = {α ∈ A|α(sj ) ∈ S, that is, the set of actions defined
at sj .
6.3
Exercises
196
7
Quantum Probability with an Eye Toward Quantum
Computing
197