Contents
0 Introduction
0.1 Quantifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
0.2 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
0.3 Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
0.4 Preliminary Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
2
2
4
4
1 Independence
6
2 Representation
2.1 Probability distributions . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.1 Joint probability distribution . . . . . . . . . . . . . . . . . . . .
2.1.2 Leveraging independence . . . . . . . . . . . . . . . . . . . . . . .
2.2 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1 The Naive Bayes’ Model . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . . . . .
8
8
8
9
10
10
11
3 Bayesian Networks
3.1 Factorisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1 Probability distribution . . . . . . . . . . . . . . . . . . . . . . .
3.1.2 Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Relation between the two factorisations . . . . . . . . . . . . . . . . . . .
3.2.1 I-maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Defining a Bayesian Network . . . . . . . . . . . . . . . . . . . . . . . . .
13
13
13
14
14
14
15
4 Inferring Independencies
4.1 Local Markov Independencies . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Markov Blanket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Markov Boundary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 D-separation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
17
18
18
19
1
Chapter 0
Introduction
0.1
Quantifiers
• ∀
Universal quantifier; read ”for all”.
E.g. ∀x, y ∈ R, x − y ∈ R can be read as:
“For all real numbers x and y, the expression x − y is also a real number.”
• ∃
Existential quantifier; read ”there exists”.
E.g. ∀x ∈ R, ∃x′ ∈ R : x + x′ = 0 ∈ R can be read as:
“For all real numbers x, there exists a real number x′ such that (:) their sum is the
real number 0.”
0.2
Notations
The following are the notations that will be followed unless any exceptions arise, in which
case the newly introduced notation will be defined within the text.
Sets
• A\B
Set difference; given by
A \ B := {x ∈ A : x ∈
/ B}
Graphs
• G = (X , E)
A graph object, G, with its set of nodes X , and its set of edges E.
2
• A→B
A directed edge (aka an arc) from node A to node B.
• A−B
An undirected edge between nodes A and B.
• A⇌B
An arbitrary connection between two nodes, i.e. it can be directed (in any direction)
or undirected.
• Given a subset of nodes X ⊆ X ,
◦ PaGX is the set of parent nodes in the graph G of the nodes contained in X.
◦ CGX is the set of child nodes in the graph G of the nodes contained in X.
◦ DX is the set of descendants of the nodes contained in X.
◦ NonDescendantsX is the set of all non-descendants of the nodes contained in
X, i.e.
NonDescendantsX := X \ DX
Probability
• P
Any probability distribution.
• Given a random variable A,
◦ If it is binary (true or false), then the true outcome will be denoted by a0 and
the false outcome by a1 .
◦ If it is not binary and has n possible outcomes (where n ≥ 2), then they will
be enumerated as a1 , a2 , . . . , an .
• ¬A
Complement of event A.
• P (A)
The prior probability of event A occurring.
• P (A, B)
The joint probability of two events A and B occurring.
3
• P (A|B)
The posterior probability of event A occurring given the observation of event B.
• Ω
The event space (set of all possible events) of a given experiment/situation.
0.3
Formulas
The following are the formulas that will be used without further explanation within the
text.
• Chain rule
P (X1 , . . . , Xn ) =
n
Y
P (Xi |X1 , . . . , Xi−1 )
i=1
• Conditional probability in terms of joint and prior probability
P (A|B) =
0.4
P (A, B)
P (B)
Preliminary Definitions
Definition 0.1
A child of a node X is any node connected to X via a directed edge originating from
X. The set of all child nodes of a set of variables V in a graph G = (X , E) is given
by
CGV := {x ∈ X : ∃v ∈ V such that v → x}
Definition 0.2
A parent of a node X is any node connected to X via a directed edge to X. The
set of all parent nodes of a set of variables V in a graph G = (X , E) is given by
PaGV := {x ∈ X : ∃v ∈ V such that x → v}
Definition 0.3
A path in a graph G = (X , E) is a set of nodes X1 , . . . , Xn such that Xi → Xi+1 for
all i = 1, . . . , n − 1.
4
Definition 0.4
A descendant of a node X is any node connected to X via a path originating from
X. The set of all descendants of a set of variables V in a graph G = (X , E) is given
by
DGV := {x ∈ X : ∃x1 , . . . , xn ∈ X such that v = x1 → . . . xn = x}
Any node that is not a descendant of a variable is called a non-descendant of that
variable.
Definition 0.5
A trail in a graph G = (X , E) is a set of nodes X1 , . . . , Xn such that Xi ⇋ Xi+1 for
all i = 1, . . . , n − 1.
Definition 0.6
A topological ordering of a set of nodes, X ⊆ X , in a graph G = (X , E), is
a sequence X1 , . . . , Xn such that PaGXi ⊆ {X1 , . . . , Xi−1 } for all i = 1, . . . , n and
Xi ∈ X. In other words, given any node in the sequence, all of its parents will be
contained in the sequence of nodes that came before it.
5
Chapter 1
Independence
Consider a probability distribution, P, defined over a set of random variables V. Intuitively, any two random variables are independent of each other if any one of them
occurring does not influence the other. But given any real life context where we model a
given situation, true independence is rarely seen as there will almost always be some form
of influence from one or more factor. A more useful notion is that of conditional independence, where two random variables do not influence each other only when certain
observations have been made.
Example 1.1
A job interviewer evaluates a candidate for an academically-inclined position. The
evaluation results, E, are binary (pass or fail). Upon seeing the candidate’s excellent
academic background, A, the chances of employing him increases. But it was noticed
that this candidate had records, R, from a a previous job in a similar position where
he underperformed, resulting in his termination. Now,
P (E|A, R) = P (E|R)
or in other words, given the candidate’s previous job record, the chances of employing
him remain the same regardless of his academic performance.
This can be generalised to any number of random variables, where two sets of variables
do not influence each other given a set of observations, from which we get
Definition 1.1
Two sets of random variables A, B ⊆ V are conditionally independent given a set
of observations or evidence, E ⊆ V, if
P (A|B, E) = P (A|E) or equivalently P (B|A, E) = P (B|E).
This is written as
(A ⊥
⊥ B | E) or (B ⊥
⊥ A | E)
6
Let (A ⊥
⊥ B | E). Then,
P (A, B, E) = P (A|B, E)P (B|E)P (E)
=⇒ P (A, B, E) = P (A|E)P (B|E)P (E)
=⇒
[chain rule]
[conditional independence]
P (A, B, E)
= P (A|E)P (B|E)
P (E)
=⇒ P (A, B|E) = P (A|E)P (B |E)
Definition 1.2
Two sets of random variables A, B ⊆ V are marginally independent if the set of
observations E = ∅. In other words,
P (A|B, ∅) = P (A|∅) =⇒ P (A|B) = P (A).
This is written as
(A ⊥
⊥ B)
Following from the result obtained before, if (A ⊥
⊥ B), then E = ∅, or
P (A, B|∅) = P (A|∅)P (B|∅) =⇒ P (A, B) = P (A)P (B).
Remark
Marginal independence does not imply conditional independence. It simply means
that the given variables do not influence each other when no observations are made,
but they may influence each other upon observing any other variable.
From here onwards, the set of independencies of a distribution P will be defined by the
set
I(P) := {(X ⊥
⊥ Y | Z) : X, Y, Z ⊆ V}
or in other words, the set of all independencies of the form (X ⊥
⊥ Y | Z), where X, Y, Z
are subsets of the set of random variables V.
7
Chapter 2
Representation
2.1
Probability distributions
2.1.1
Joint probability distribution
Any probability distribution P can be represented in terms of the probabilities of its
atomic events in the form of a joint probability distribution.
Definition 2.1
An atomic event is a complete, indivisible specification of the values the variables
under a distribution can take.
Consider tossing two coins and recording the faces they land on. The resulting event
space is
Ω = {HH, HT, T H, T T }.
Here, an example of an atomic event can be HH ∈ Ω. In general, an atomic event is a
single element in an event space.
Example 2.1
Consider the case where a coin is tossed and a fair six-sided die is rolled.
Let A be the random variable which denotes obtaining tails in the coin (a0 = Tails,
a1 = Heads), and B be the random variable which denotes obtaining a factor of 9
from the die (b0 = 1 or 3, b1 = 2, 4, 5, or 6). Then the joint probability distribution
will be
a0
b0
b1
a1
1/6 1/6
1/3 1/3
which is, in essence, the distribution of P (A, B). Here, the sum of all the entries
must be 1, or more formally
X
P (a, b) = 1.
a,b
8
The issue one runs into when utilising joint probability distributions is size. As the
number of variables, or features, increase, modelling every possible event results in an exponentially growing distribution. As an example, given the random variables X1 , . . . , Xn ,
with each variable having only two possible outcomes, the number of entries in the distribution equal 2n . In general, the product of the number of outcomes for each variable
determines the number of entries in the table.
2.1.2
Leveraging independence
A solution arises for the aforementioned problem when one can make use of independence.
Example 2.2
Consider the random variables A, B, and C, with 3, 2, and 4 possible outcomes,
respectively. Also, let (A ⊥
⊥ B | C) and (B ⊥
⊥ C). Constructing a joint probability
distribution for P (A, B, C) results in 3 × 2 × 4 = 24 entries. But the distribution can
be factorised by using the chain rule as such:
P (A, B, C) = P (A|B, C)P (B|C)P (C) = P (A|C)P (B)P (C).
Now, each of these factors can be used to represent the distribution as 3 separate
distributions: one conditional probability distributions (CPDs) for P (A|B) and
two distributions for P (B) and P (C).
C
a1
a2
a3
c1
c2
c3
c4
P (a1 |c1 )
P (a1 |c2 )
P (a1 |c3 )
P (a1 |c4 )
P (a2 |c1 )
P (a2 |c2 )
P (a2 |c3 )
P (a2 |c4 )
P (a3 |c1 )
P (a3 |c2 )
P (a3 |c3 )
P (a3 |c4 )
c1
c2
P (c1 ) P (c2 )
b0
b1
P (b0 )
P (b1 )
c3
c4
P (c3 )
P (c4 )
This reduces the number of entries down to 18.
Remark
The entries in a CPD will not sum to 1. Consider binary random variables A and
B. Then P (A|B) can be represented as
9
a0
B
b0
b1
a1
P (a0 |b0 ) P (a1 |b0 )
P (a0 |b1 ) P (a1 |b1 )
In general, negating a conditional probability does not affect the condition, i.e.
1 − P (A|B) = P (¬A|B) ̸= P (A|¬B).
This shows that the distributions for P (A|B) and P (A|¬B) can be viewed as separate distributions since they do not influence each other. Generalising further, each
probability obtained by varying the evidence results in a separate distribution, and
when constructing a CPD, these separate distributions precisely correspond to each
row of the table. As such, the entries for each row must sum to 1. More formally,
X
P (a|b) = 1, ∀b
a
2.2
Graphs
As shown in the preceding sections, a highly significant portion of representing any given
distribution is to be able to infer its set of independencies. A graph or a graphical network
is a very convenient method of representing the dependencies among variables, where in
general, the nodes represent random variables and directed edges represent the direction
of influence among variables.
2.2.1
The Naive Bayes’ Model
As seen in Example 2.2, the greater the number of independences that can be interpreted,
the smaller (and hence more computationally feasible) the representation becomes. Extending this notion to its extremes, the size of said representation can be minimised if it is
assumed that given a set of observations E, every other variable is conditionally independent of each other. More concisely, given the set of random variables X = {X1 , . . . , Xn }
and a set of observations variables E = {E1 , . . . , Em }, then, for i ̸= j,
(Xi ⊥
⊥ Xj | Ek ), ∀Xi , Xj ∈ X and Ek ∈ E.
This is the naive Bayes’ assumption, and is called such since conditional independence
cannot be assumed to hold among every variable in a real-life situation given the observation of a fixed set of factors. In fact, the larger the number of features (variables) that
is being modelled, the more prone to erroneous inference the model gets. This is due to
the increasing number of independence assumptions that are being made which may not
10
be reflected in reality. The following graph structure can be used to represent this model:
E
X1
···
X2
Xn
Figure 2.1: Naive Bayesian Network
This network shows that each of X1 , . . . , Xn is influenced by the observations made of
the variables in the set E. By using a previous result on conditional independence, the
distribution can be factorised as such:
P (X1 , . . . , Xn , E) = P (E)P (X1 , . . . , Xn |E) = P (E)
n
Y
P (Xi |E)
i=1
An observation can be made regarding the factors here and the relation it holds to the
graph structure shown. For this particular graph, each factor is of the form
P (X|PaGX )
with the exception of P (E), but that is only because its parent set is empty resulting
in P (E|∅) = P (E). It will be later seen that this form holds true in general for any
Bayesian network, providing a convenient method of factorising any given distribution
using the graph object.
Remark
It is worth noting at this point that whenever the notation P (A) or P (A|B) is used,
this refers to the distributions rather than the numerical probability, in which case the
notation would be P (ai ) or P (ai |bj ). This distinction is made to avoid any confusion
on when the aforementioned factorisation holds true. When using a graph structure,
it is, by definition, assumed to be normalised, or that the relevant factors sum to 1 as
probabilities should. It will be seen later that when probabilities are calculated that
are not directly shown within a parent-child connection in the graph, the factorisation
leads to a proportionality as opposed to an equality.
2.2.2
Bayesian Networks
A more useful (albeit less simple) version of the naive Bayesian network is the Bayesian
network that does not rely on the extensive use of the conditional independence assump11
tion, but rather properly models the direction of influence between each node. Consider
the following network:
A
B
C
D
In here, the direction of the edges show the direction of influence. In other words, A and
B influence C, and C influences D. The probability distribution containing the variables
A, B, C, and D can be factorised as
P (A, B, C, D) = P (A)P (B)P (C|A, B)P (D|C).
A Bayesian network structure is always a directed acyclic graph (DAG). The graphs
need to be acyclic since a valid topological ordering (Definition 0.5) is required for the
factorisation to be possible, since each variable is conditioned on the variables that they
depend on. If a cycle like A → B → C → A, were to exist, then this implies that
A depends on C while C also depends on A. This is a problem since, to define the
distribution for A you would need to know C, and vice versa, resulting in a contradictory
loop.
Inferences can be made about the independences that are encoded by this graph structure
by examining the connected nodes. This concept will be formalised in the following
sections.
12
Chapter 3
Bayesian Networks
This chapter will start off by thinking of a probability distribution P and a graphical
network G = (X , E) as separate objects. The goal is to build up to the necessary condition that needs to be satisfied from which one can conclude that P is properly being
represented by G.
3.1
Factorisation
3.1.1
Probability distribution
Let V be the set of random variables that are defined under P. From the chain rule, one
knows that
n
Y
P (V1 , . . . , Vn ) =
P (Vi |V1 , . . . , Vi−1 ).
i=1
where each Vi ∈ V.
Remark
Viewing this with a smaller set of variables, consider V1 , V2 , V3 . Then,
P (V1 , V2 , V3 ) = P (V3 |V1 , V2 )P (V1 , V2 ) = P (V3 |V1 , V2 )P (V2 |V1 )P (V1 )
which is done by successively applying the chain rule first to P (V1 , V2 , V3 ) then
P (V1 , V2 ).
Note that the joint probability distribution P (V1 , V2 , V3 ) does not change if the order
of the variables is changed, i.e. P (V2 , V1 , V3 ) also defines the same distribution and
so does any other permutation of the variables. Hence, the chain rule need not
necessarily be applied in the order stated above where the last variable in the row
is conditioned on the remaining variables, but can be done in any order deemed
necessary.
Given any number of independencies, the above decomposition can be simplified, resulting
in a factorised distribution, where each factor gets its own CPD.
13
3.1.2
Graph
As seen previously, a Bayesian network graph can be used to obtain a factorisation of
the random variables (nodes) being modelled. Consider the graph G = (X , E). Then, the
factorisation is of the form
P (X1 , . . . , Xn ) =
n
Y
P (Xi |PaGX ),
i=1
where each Xi ∈ X . The distributions P (Xi |PaGX ) are each CPDs and are also referred
to as local probabilistic models since the independencies of a subset of nodes within
the graph can be inferred from them (see Chapter 4).
3.2
Relation between the two factorisations
To get a more intuitive understanding of when P can be represented by G, the significance of the independencies contained within the graph and the distribution must be
appreciated.
A graph, by design, is used to show directions of influence between variables (nodes) only,
and these in turn define the independencies. It is these independencies that dictate how
the factorisation will be for any represented distribution. Once these independencies are
properly defined, inferences can be made from the graph by means of further analysis
through the use of assigned parameters. Therefore, it is necessary that the graph being
used only shows independencies that are defined in the distribution P, and nothing more,
as that will distort the situation being modelled by P.
3.2.1
I-maps
Definition 3.1
Let K be any graph and let I be any set of independencies. If I(K) ⊆ I, where I(K) is
the set of independencies represented in K, then K is called an I-map (independency
map) for I.
Note that the above definition asserts the condition for an I-map between the independencies of a graph and another arbitrary set of independencies. So, considering the graph
G and the distribution P, if I ′ (G) ⊆ I(P), then G is an I-map for I(P). Here, I ′ (G) is
any subset of all the independencies represented in G.
14
Definition 3.2
Let I(G) and I(P) be the set of all independencies as defined in a graph G and a
distribution P. Then G is an I-map for P if I(G) ⊆ I(P).
Definition 3.2 differs from Definition 3.1 in that this new definition relates a graph and a
distribution, as opposed to a graph and a set of independencies. From Definition 3.2, it
can also be concluded that G is an I-map for I(P). So it follows that G being an I-map
for P implies that G is an I-map for I(P).
The set of all independencies in G, I(G), is called the global Markov independencies
and will be elaborated on in Chapter 4.
Significance
The first condition that needs to be satisfied for G to represent P is that G must be an
I-map for P. This means that the graph models some of, if not all, the independencies of
the distribution. Some independencies in the distribution may be left out of the graph.
In simpler words, the graph does not lie about the distribution, but rather uses some, if
not all, of the truth as portrayed by the distribution.
Theorem 3.1
G is an I-map for P if and only if P can be factorised as
P (X1 , . . . , Xn ) =
n
Y
P (Xi |PaGX ),
i=1
where X1 , . . . , Xn ∈ X ⊆ V.
This creates a link between the notion of I-maps and the factorisation of Bayesian networks.
3.3
Defining a Bayesian Network
Definition 3.3
Let G = (X , E) be a Bayesian network structure, and P be a distribution defined
over the same set of variables as G. Then P is said to factorise over G or factorise
according to G if P can be expressed as
P (X1 , . . . , Xn ) =
n
Y
P (Xi |PaGX ).
i=1
This provides the final preliminary definition required to concisely define a Bayesian
network for a distribution P.
15
Definition 3.4
A Bayesian network B = (G, P) is a pair consisting of a DAG, G, and a distribution,
P, which factorises over G. Here, P is specified as a set of CPDs in terms of the
nodes of G and is often annotated as PB .
Thus we have a graph G, which represents P while being
• faithful; properly represents the independencies that are true in the distribution as
well,
• compact; just enough independencies of P as required for it to be correct is being
represented in G.
16
Chapter 4
Inferring Independencies
It was stated earlier that the graph structure of a Bayesian network can be examined to
infer which variables are independent of each other and under which conditions. This
chapter enumerates the sets of independencies obtained from the graph using different
inference methods.
4.1
Local Markov Independencies
Consider the graph shown in Chapter 2, Section 2.2.2. The variables A, B, C, and D can
be factorised as
P (A, B, C, D) = P (A)P (B)P (C|A, B)P (D|C)
while using the chain rule, the distribution can be factorised as
P (A, B, C, D) = P (D|A, B, C)P (C|A, B)P (B|A)P (A).
Since this a Bayesian network, these two factorisations are equivalent and so
P (A)P (B)P (C|A, B)P (D|C) = P (D|A, B, C)P (C|A, B)P (B|A)P (A)
⇐⇒ P (D|C)P (B) = P (D|A, B, C)P (B|A)
Both sides of the equation have distributions of B and D but conditioned on different
variables. Equating them, one gets
P (D|A, B, C) = P (D|C) and P (B|A) = P (B)
which precisely imply
(D ⊥
⊥ A, B | C) and (B ⊥
⊥ A)
respectively. Further examination of the graph shows that, for each of these independencies, a variable is independent of its non-descendants when conditioned on its parents.
This is generally true for any Bayesian network.
17
Definition 4.1
The set of local Markov independencies (or simply local independencies) is
the set of all independencies of the form
(X ⊥
⊥ NonDescendantsX | PaGX )
and is denoted by Il (G).
In other words, any set of variables X is independent of its non-descendants given its
parents in the graph. Variables satisfying this independence property are said to satisfy
the local Markov property.
4.2
Markov Blanket
Definition 4.2
Let X be the set of random variables in a Bayesian network structure G. The Markov
blanket of a variable X ∈ X is any subset of random variables S ⊆ X that satisfies
following condition:
(X ⊥
⊥ X \ S | S)
In other words, a variable X is conditionally independent of every variable outside its
Markov blanket given its Markov blanket.
4.3
Markov Boundary
Definition 4.3
A Markov boundary of a random variable X is a minimal Markov blanket of X.
A minimal Markov blanket refers to a Markov blanket, which does not remain a Markov
blanket if any element is removed from the set. In other words, it is the smallest possible
Markov blanket of a random variable.
Theorem 4.1
Given a Bayesian network structure G = (X , E), the Markov boundary, MX , of a
variable (or a set of variables) X is given by
MX := PaGX ∪ CGX ∪ PaGCG
X
In simpler terms, it means that the Markov boundary of a variable X consists of its
parents, its children, and the other parents of its children. So, the variable X, given its
18
Markov boundary, is conditionally independent of every variable in the graph outside its
Markov boundary.
4.4
D-separation
D-separation is used to determine whether two variables in a Bayesian network are independent or not. In simple terms, two nodes are said to be d-separated if no influence
can “flow” between them, rendering them independent of each other. Note that while it
may be intuitive to think that influence can only flow between two nodes along a path
connecting the two, this is only one of 3 cases in which this is true. As such, we will look
at the trails between two nodes whose d-separation are to be examined as opposed to the
paths.
Consider that we are trying to determine whether two variables X and Y influence each
other given Z. Then the trails between X and Y that contain the node Z need to be
examined. The possible cases are hence
• X→Z→Y
In general, this means that Z influences Y and X influences Z. We also know that
(X ⊥
⊥ Y | Z) since Z is a parent of Y and X is a non-descendant of Y , implying
that given Z, X does not influence Y . Hence, X and Y are “cut off” from each
other if Z is given. The same logic would apply if Y → Z → X.
• X←Z→Y
This means that Z influences both X and Y . Here, the three variables once again
satisfy the local Markov property, i.e. (X ⊥
⊥ Y | Z) and by the same reasoning as
before, X and Y do not influence each other if Z is given.
• X→Z←Y
This means that Z is influenced by both X and Y . Here, X and Y are marginally
independent, i.e. (X ⊥
⊥ Y ), so there is no direct influence between them. Now,
consider the factorisation of this network
P (X, Y, Z) = P (X)P (Y )P (Z|X, Y ).
Consider the case where Z has a high value while X has a low value. Using the
factor P (Z|X, Y ), since X has a low value, to explain the high value of Z, Y must
have a high value. This is called explaining away (yes, this is a technical term).
What we effectively have is that, given a fixed observation of Z, the value of X
influences the value of Y , and vice versa. So, X and Y are cut off from each other
when not given Z. This structure is called a v-structure or collider.
19
Additionally, consider that Z has one or more descendants, resulting in a graph
structure that, as an example, looks like this
X
Z
Y
A
B
Now, Z itself is dependent on A and B since they not its non-descendants. So,
given information on A and/or B, information about Z can be obtained since they
directly influence Z. Hence, X and Y can influence each other when Z or any of
its descendants are given.
This concept can once again be extended to sets of variables rather than singular nodes.
Definition 4.4
Let the set of observed conditions be Z ⊆ X . An active trail between two nodes
X, Y ∈ X is a trail X = X1 ⇋ · · · ⇋ Xn = Y such that
• If there is a v-structure Xi−1 → Xi ← Xi+1 , then Xi and its descendants are
in Z.
• No other node along the trail is in Z.
Definition 4.5
Let G = (X , E) be a Bayesian network structure. Given a set of conditions Z, the sets
of random variables X and Y are d-separated if there are no active trails between
any nodes contained in X and any nodes contained in Y . This is denoted as
d-sepG (X; Y |Z)
Collectively, and in simpler terms, this simply means that, to check if two sets of variables
X and Y are d-separated given Z, one must check whether any trail between them is
“cut off” in the sense talked about at the beginning of this section. A trail that is cut off
allows no influence to flow between nodes in X and in Y , and a trail that is not cut off
is an active trail.
20
Now, if no influence can flow between X and Y given Z, then that means X and Y are
independent of each other given Z. So, d-separation provides another method of finding
a set of independencies within G. In fact, this precisely gives all possible independencies
within G. More formally,
Definition 4.6
The set of all independencies in a Bayesian network structure G, is given by
I(G) := {(X ⊥
⊥ Y | Z) : d-sepG (X; Y |Z)}
and it is called the set of global Markov independencies.
21