Bayesian Networks
Quiz: Probabilistic Reasoning
1. What is P(F), the probability that some creature can
fly?
2. Creature b is a bumble bee. What’s P(F|B), the
probability that b can fly given that it’s a bumble bee?
3. b has unfortunately met a malicious child, who has
torn off b’s wings. What is P(F|B,N), the probability
that b can fly given that it has no wings?
4. b somehow makes its way onto a jumbo jet, where it
survives by drinking juice spilled by passengers. What
is P(F|B, N, L=j), the probability that b can fly given
that it has no wings and its location is a jet?
Example
Has
diabetes?
(D or D)
Unobservable
node
V = a set of random variables
E = directed edges between
them (cycles not allowed)
Test was
positive?
(+ or -)
Observable
node
Diab? Test? P(T|D)
Diab? P(D)
D
0.01
D
0.99
BN = (V, E, P)
D
+
0.9
D
-
0.1
D
+
0.2
D
-
0.8
P = for every node in the
network, a conditional
probability distribution for
that random variable, given
its parents in the graph
Simple probabilistic reasoning
You already know how to figure out:
• P(D)  stored in the Bayes Net
• P(+|D)  stored in the Bayes Net
• P(D,+)  multiply P(D)P(+|D)
• P(+)  apply marginalization to P(D, +)
• P(D|+)  apply Bayes’ Rule
Purpose behind Bayes Networks
Battery
age
Alternator
broken
Battery
dead
Battery
meter
Lights
Fan belt
broken
Not
charging
Battery
flat
Oil
light
No
oil
Gas
gauge
No
gas
Bayes Nets help figure out more difficult
cases:
What’s
P(Battery Dead |
Car won’t start, Battery is 5 years old)?
or
P(Alternator broken |
Car won’t start, oil light is on, lights are dim)?
Fuel line
blocked
Car won’t
start
dipstick
Starter
broken
Types of Bayes Net Queries
Bayes Nets let you solve “queries”, or probabilistic questions.
There are different types of queries for a Bayes Net with
random variables X1, …, XN:
1. Joint queries: What is P(car starts, oil light on)?
2. Conditional queries: What is P(alternator broken, battery
light dim | oil light off, lights dim)?
3. Maximum a posteriori (MAP):
what values (true or false) for “Will Car Start?” makes
this probability the biggest:
P(Will Car Start? | battery is 5 years old, lights dim)
The Bayes Net Equation
A BN specifies the joint distribution over all
random variables in the graph, using this eqn:
𝑃 𝑋1 , … , 𝑋𝑁 =
𝑃 𝑋𝑖 |𝑝𝑎𝑟𝑒𝑛𝑡𝑠(𝑋𝑖 )
𝑋𝑖
Example
Has
diabetes?
(D or D)
P(Diab, Test) =
P(Diab|parents(Diab))
*P(Test|parents(Test))
Test was
positive?
(+ or -)
=
P(Diab)
*P(Test|Diab)
Quiz: Two-test Diabetes
Diab? P(D)
D
0.01
D
0.99
Has
diabetes?
(D or D)
Test 1 was
positive?
(+ or -)
Diab? Test1?
1. What is
P(Test1=+|D)?
P(T1|D)
2. What is
P(Test1=+|D,Test2=+)?
Test 2 was
positive?
(+ or -)
Diab? Test2?
3. What is
P(D|Test1=+,Test2=+)?
P(T2|D)
D
+
0.9
D
+
0.9
D
-
0.1
D
-
0.1
D
+
0.2
D
+
0.2
D
-
0.8
D
-
0.8
4. What is
P(D|Test1=+,Test2=-)?
Conditional Independence in a BN
Has
diabetes?
(D or D)
Test 1 was
positive?
(+ or -)
Test 2 was
positive?
(+ or -)
In this BN,
T1  T2 | D
This means, e.g.:
P(T1=+|D, T2=+)
is the same as
P(T1=+|D)
Quiz: Two-test Diabetes
Has
diabetes?
(D or D)
Test 1 was
positive?
(+ or -)
Test 2 was
positive?
(+ or -)
What is P(T1=+|T2=+)?
Absolute vs. Conditional Independence
Has
diabetes?
(D or D)
Test 1 was
positive?
(+ or -)
Test 2 was
positive?
(+ or -)
Remember:
T1  T2 | D
Does this mean that
T1  T2 ?
In other words,
P(T1) =? P(T1 | T2)
S?
P(D)
S
0.7
S
0.3
Confounding Cause
Sunny?
(S or S)
1. What is P(R | S)?
Raise?
(R or R)
Happy?
(H or H)
R?
P(R)
R
0.01
R
0.99
Happy? Sunny? Raise?
P(H|S,R)?
H
S
R
1.0
H
S
R
0.7
H
S
R
0.9
H
S
R
0.1
2. What is P(R | H, S)?
3. What is P(R | H,S)?
4. What is P(R | H)?
Absolute vs. Conditional Independence
Sunny?
(S or S)
Raise?
(R or R)
Remember:
RS
Does this mean that
RS|H?
Happy?
(H or H)
In other words,
P(R | H) =? P(R | H, S)
D-Separation
D-separation is the technical method for
determining conditional independence in a BN.
Active Triplets
Inactive Triplets
…
D-Separation
Node A is d-separated (short for directional-separated)
from node B if
all paths from A to B contain at least one inactive triplet.
A  B | K1, …, Km

nodes A and B are d-separated when nodes K1, …, Km are
known
D-Separation Quiz 1
C  A?
A
C  A | B?
B
D
C  D?
C  D | A?
C
E
E  C | D?
D-Separation Quiz 2
A  E?
A
B
A  E | B?
A  E | C?
C
A  B?
D
E
A  B | C?
D-Separation Quiz
A
C
F
F  A?
F  A | D?
B
E
F  A | G?
D
H
F  A | H?
G
Counting BN Parameters
A
B
C
D
E
A complete joint
distribution over 5
binary variables would
require 31 = 25-1
parameters.
This BN requires
10 = 1+1+4+2+2
parameters.
Quiz
A
B
E
A full joint over 6
binary variables
requires 26-1 = 63
parameters.
C
D
F
How many parameters
does this network
require?
Quiz
A
B
C
A full joint distribution
over 7 binary variables
requires 27-1 = 127
parameters.
D
E
F
G
How many parameters
does this network
require?
Quiz
Battery
age
Alternator
broken
Battery
dead
Battery
meter
Lights
A full joint distribution over
Fan belt 16 binary variables requires
16
broken 2 -1 = 65,535 parameters.
How many parameters does
this network require?
Not
charging
Battery
flat
Oil
light
No
oil
Gas
gauge
No
gas
Fuel line
blocked
Car won’t
start
dipstick
Starter
broken