Variable elimination and junction tree
Exact inference
Exponential in tree-width
Mean field, loopy belief propagation
Approximate inference for marginals/conditionals
Fast, but can get inaccurate estimates
Sample-based inference
Approximate inference for marginals/conditionals
With “enough” samples, will converge to the right answer
2

What is the approximating structure?
? ?
𝑃 𝑄 𝑄
How to measure the goodness of the approximation of
𝑄 𝑋
1
, … , 𝑋 𝑛
to the original 𝑃 𝑋
1
, … , 𝑋 𝑛
?
Reverse KL-divergence 𝐾𝐿(𝑄| 𝑃 mean field
𝑄
?
How to compute the new parameters?
Optimization Q ∗ = argmin
Q
𝐾𝐿(𝑄| 𝑃
𝑁𝑒𝑤 𝑃𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠
3

Approximate
𝑃 𝑋 ∝ Ψ 𝐷 𝑗
with
𝑄 𝑋 = 𝑄 𝑖
𝑋 𝑖 𝑠. 𝑡. ∀𝑋 𝑖
, 𝑄 𝑖
𝑋 𝑖
> 0, ∫ 𝑄 𝑖
𝑋 𝑖 𝑑𝑋 𝑖
= 1
𝐷(𝑄| 𝑃 = −∫ 𝑄 𝑋 ln 𝑃 𝑋 𝑑𝑋 + ∫ 𝑄 𝑋 ln 𝑄 𝑋 𝑑𝑋 + 𝑐𝑜𝑛𝑠𝑡.
= −∫ 𝑄
= − ∫ 𝑖
𝑋 𝑖 𝑖∈𝐷 𝑗
( ln Ψ 𝐷 𝑗
𝑄 𝑖
𝑋 𝑖
)𝑑𝑋 + ln Ψ 𝐷 𝑗 𝑑𝑋
𝐷 𝑗
∫ 𝑄 𝑖
𝑋
+ ∫ 𝑄 𝑖 𝑖
( ln 𝑄 𝑖
𝑋 𝑖 ln 𝑄 𝑖
𝑋 𝑖
𝑋 𝑖
)𝑑𝑋 𝑑𝑋 𝑖
𝑄
1
𝑋
1
Ψ(𝑋
1
, 𝑋
5
)
E.g., ∫ 𝑄
∫ 𝑄
1
𝑋
1 𝑖
𝑄
5 𝑖
𝑋 𝑖
𝑋
5 ln Ψ 𝑋
1 ln Ψ 𝑋
1
, 𝑋
5
, 𝑋
5 𝑑𝑋 = 𝑑𝑋
1 𝑑𝑋
5
𝑋
6
𝑋
5
𝑋
2
𝑋
1
𝑋
4
𝑋
7
Ψ(𝑋
1
, 𝑋
2
)
𝑋
3
Ψ(𝑋
1
, 𝑋
3
)
Ψ 𝑋
1
, 𝑋
4
4

Let 𝐿 be the Lagrangian function ( 𝑛𝑜 𝑄 𝑖
𝐿 =
− ∫ 𝑖∈𝐷 𝑗
𝑄 𝑖
∫ 𝑄 𝑖
𝑋 𝑖 ln 𝑄 𝑖
𝑋 𝑖
𝑋 𝑖 ln Ψ 𝐷 𝑗 𝑑𝑋 𝑖 𝑑𝑋
𝐷 𝑗
+
𝑋
− 𝛽 𝑖
(1 − ∫ 𝑄 𝑖 𝑖
𝑋 𝑖
> 0 ?)
)𝑑𝑋 𝑖
For KL divergence, the nonnegativity constraint comes for free
If 𝑄 is a maximum for the original constrained problem, then there exists 𝜆 𝑖
, 𝛽 𝑖
such that ( 𝑄, 𝜆 𝑖
, 𝛽 𝑖
) is a stationary point for the Lagrange function (derivative equal 0).
5

𝐿 =
− ∫ 𝑖∈𝐷 𝑗
𝑄 𝑖
𝛽 𝑖
(1 − ∫ 𝑄 𝑖
𝑋 𝑖
𝑋 𝑖 ln Ψ 𝐷
)𝑑𝑋 𝑖 𝑗 𝑑𝑋
𝐷 𝑗
+ ∫ 𝑄 𝑖
𝜕𝑄 𝑖 𝛽 𝑖
𝜕𝐿
(𝑋 𝑖
)
= 0
= − 𝑗:𝑖∈𝐷 𝑗
( 𝑘∈𝐷 𝑗
\i
𝑄 𝑘
𝑋 𝑘
𝑋 𝑖 ln 𝑄 𝑖
𝑋 𝑖 ln Ψ(𝐷 𝑗
)) + ln 𝑄 𝑖 𝑑𝑋 𝑖
−
𝑋 𝑖
−
𝑄 𝑖
1
𝑍 𝑖
𝑋 𝑖
= exp 𝑗:𝑖∈𝐷 𝑗 exp 𝑗:𝑖∈𝐷 𝑗
( 𝑘∈𝐷 𝑗
\i
( 𝑘∈𝐷 𝑗
\i
𝑄 𝑘
𝑋 𝑘
𝑄 𝑘
𝑋 𝑘 ln Ψ(𝐷 𝑗 ln Ψ(𝐷
)) 𝑗
)) + 𝛽 𝑖
𝑄 𝑖
𝑋 𝑖
=
1
𝑍 𝑖 exp
𝐷 𝑗
:𝑋 𝑖
∈𝐷 𝑗
𝐸
𝑄 ln Ψ 𝐷 𝑗
=
6

Initialize 𝑄 𝑋
1
, … , 𝑋 𝑛
= 𝑄 𝑋 𝑖
(eg., randomly or smartly)
Set all variables to unprocessed
Pick an unprocessed variable 𝑋 𝑖
Update 𝑄 𝑖
:
𝑄 𝑖
𝑋 𝑖
=
1
𝑍 𝑖 exp 𝐸
𝐷 𝑗
:𝑋 𝑖
∈𝐷 𝑗
𝑄
Set variable 𝑋 𝑖 as processed
If 𝑄 𝑖
changed
Set neighbors of 𝑋 𝑖
to unprocessed ln Ψ 𝐷 𝑗
Guaranteed to converge
7

𝑄
1
𝑋
1
𝑋
6
𝑋
5
𝑄(𝑋
5
) ln Ψ 𝑋
1
, 𝑋
5 𝑥5
𝑄
2
𝑋
2
𝑋
6
𝑄(𝑋
6
) ln Ψ 𝑋
1
, 𝑋
6 𝑥6
𝑋
5
𝑋
2
𝑋
1
𝑋
4
𝑋
2
𝑋
1
𝑋
4
𝑋
7
𝑋
3 𝑋
7
𝑋
3
𝑄 𝑋 𝑥
2
2 ln Ψ 𝑋
1
, 𝑋
2
𝑄 𝑋 𝑥
4
4 ln Ψ 𝑋
1
, 𝑋
4
𝑄 𝑋
1 𝑥
1 ln Ψ 𝑋
1
, 𝑋
2 𝑄 𝑋
7 𝑥
7 ln Ψ 𝑋
2
, 𝑋
7
𝑄
1
𝑋
1
𝑄
2
𝑋
2
𝑄 𝑋
3 𝑥
3 ln Ψ 𝑋
1
, 𝑋
3
=
=
1
𝑍
1
1
𝑍
1 exp 𝐸
𝑄(𝑋
2 exp 𝐸
𝑄(𝑋
1
)
) ln Ψ 𝑋
1
, 𝑋
2 ln Ψ 𝑋
1
, 𝑋
2
+ 𝐸
𝑄(𝑋
3
) ln Ψ 𝑋
1
, 𝑋
3
+ 𝐸
𝑄(𝑋
6
) ln Ψ 𝑋
2
, 𝑋
6
+ 𝐸
𝑄(𝑋
4
) ln Ψ 𝑋
1
, 𝑋
4
+ 𝐸
𝑄(𝑋
5
) ln Ψ 𝑋
1
, 𝑋
5
+ 𝐸
𝑄(𝑋
7
) ln Ψ 𝑋
2
, 𝑋
7
8

Mean field equation
𝑄 𝑖
𝑋 𝑖
=
1
𝑍 𝑖 exp
𝐷 𝑗
:𝑋 𝑖
∈𝐷 𝑗
𝐸
𝑄
Pairwise Markov random field ln Ψ 𝐷 𝑗
𝑃 𝑋
1
, … , 𝑋 𝑛
= exp (𝜃 𝑖𝑗
∝ exp 𝜃
𝑋 𝑖
𝑋 𝑗
) 𝑖𝑗
𝑋 𝑖
𝑋 𝑗
+ 𝜃 𝑖
𝑋 𝑖
exp (𝜃 𝑖
𝑋 𝑖
) ln Ψ 𝑋 𝑖
Ψ 𝑋 𝑖
, 𝑋 𝑗
, 𝑋 𝑗
= 𝜃 𝑖𝑗
𝑋 𝑖
𝑋 𝑗
Ψ 𝑋 𝑖 lnΨ 𝑋 𝑖
= 𝜃 𝑖
𝑋 𝑖
The mean field equation has simple form:
𝑄 𝑖
=
𝑋 𝑖
=
1
𝑍 𝑖 exp 𝑗∈𝑁 𝑖
1
𝑍 𝑖 exp 𝑗∈𝑁 𝑖 𝜃 𝑖𝑗
𝑋 𝑖
𝜃
< 𝑋 𝑗
> 𝑖𝑗
𝑄 𝑗
𝑋 𝑖
𝑋 𝑗
+ 𝜃
𝑄 𝑗 𝑖
𝑋
𝑋 𝑖 𝑗
+ 𝜃 𝑖
𝑋 𝑖
𝑄 𝑖
𝑋 𝑖
< 𝑋 𝑠
>
𝑄 𝑠
< 𝑋 𝑙
>
𝑄 𝑙
< 𝑋 𝑗
>
𝑄 𝑗
< 𝑋 𝑘
>
𝑄 𝑘
9

𝐶
2
𝐶
3
𝐶
1
𝐶
4
𝑄 𝐶
1 exp(
∝ 𝑖∈𝐶
1 𝜃 𝑖
𝑋 𝑖
+ 𝑖𝑗 ∈𝐸,𝑖∈𝐶
1
,𝑗∈𝐶
1 𝜃 𝑖𝑗
𝑋 𝑖
𝑋 𝑗
Node potential within 𝐶
1
Edge potential within 𝐶
1
𝑀𝐵(𝐶
1
) 𝑖∈𝐶
1
,𝑗∈𝐶 𝑘
,𝑗∈𝑀𝐵 𝐶
1 𝜃 𝑖𝑗
𝑋 𝑖
< 𝑋 𝑗
>
𝑄 𝐶 𝑘
)
Mean of variables in Markov blanket
Edge potential across 𝐶
1
and 𝐶 𝑘
10

Previous inference tasks focus on obtaining the entire posterior distribution 𝑃 𝑋 𝑖 𝑒
Often we want to take expectations
Mean 𝜇
𝑋 𝑖
|𝑒
= 𝐸 𝑋
Variance 𝜎 2
𝑋 𝑖
|𝑒 𝑖 𝑒 = ∫ 𝑋
= 𝐸 (𝑋 𝑖 𝑖
𝑃 𝑋
−𝜇
𝑋 𝑖
|𝑒
) 2 𝑖 𝑒 𝑑𝑋 𝑖 𝑒 = ∫ (𝑋 𝑖
−𝜇
𝑋 𝑖
|𝑒
) 2 𝑃 𝑋 𝑖 𝑒 𝑑𝑋
More general 𝐸 𝑓 = ∫ 𝑓 𝑋 𝑃 𝑋|𝑒 𝑑𝑋 , can be difficult to do it 𝑖 analytically
Key idea : approximate expectation by sample average
𝑁
1
𝐸 𝑓 ≈
𝑁
𝑓 𝑥 𝑖 𝑖=1 where 𝑥
1
, … , 𝑥
𝑁
∼ 𝑃 𝑋|𝑒 independently and identically
11

Samples: points from the domain of a distribution 𝑃 𝑋
The higher the 𝑃 𝑥 , the more likely we see 𝑥 in the sample
𝑃(𝑋) 𝑥
1 𝑥
5 𝑥
2 𝑥
6 𝑥
4 𝑥
3
𝑋
Mean 𝜇 =
1
𝑁
𝑥 𝑖
Variance 𝜎 2 =
1
𝑁
(𝑥 𝑖
−𝜇 ) 2
12

For 𝑛 independent Gaussian variable with 0 mean and unit variance , use 𝑥 ∼ 𝑟𝑎𝑛𝑑𝑛 𝑛, 1 in Matlab
For multivariate Gaussian 𝒩(𝜇, Σ) 𝑦 ∼ 𝑟𝑎𝑛𝑑𝑛 𝑛, 1
Transform the sample: 𝑥 = 𝜇 + Σ 1/2 𝑦
𝑋 takes finite number of values, e.g., 1, 2, 3 , 𝑃 𝑋 = 1 =
0.7, 𝑃 𝑋 = 2 = 0.2, 𝑃 𝑋 = 3 = 0.1
Draw a sample from uniform distribution U[0,1], 𝑢 ∼ 𝑟𝑎𝑛𝑑
If 𝑢 falls into interval [0, 0.7) , output sample 𝑥 = 1
If 𝑢 falls into interval [0.7, 0.9) , output sample 𝑥 = 2
If 𝑢 falls into interval [0.9, 1] , output sample 𝑥 = 3
0
Value 1 Value 2 Value 3
1
0.7 0.2 0.1
13

BN describe a generative process for observations
First, sort the nodes in topological order
Then, generate sample using this order according to the CPTs
1
𝐴𝑙𝑙𝑒𝑟𝑔𝑦
𝑆𝑖𝑛𝑢𝑠
Generate a set of sample for (A, F, S, N, H):
Sample 𝑎 𝑖
Sample 𝑓 𝑖
Sample 𝑠 𝑖
Sample 𝑛 𝑖
Sample ℎ 𝑖
∼ 𝑃 𝐴
∼ 𝑃 𝐹
∼ 𝑃 𝑆 𝑎
∼ 𝑃 𝑁
∼ 𝑃 𝐻 𝑖 𝑠 𝑠
, 𝑓 𝑖 𝑖 𝑖
𝑁𝑜𝑠𝑒
4
3
𝐹𝑙𝑢
2
𝐻𝑒𝑎𝑑𝑎𝑐ℎ𝑒
5
14

How to draw sample from a given distribution?
Not all distributions can be trivially sampled
How to make better use of samples (not all samples are equally useful)?
How do know we’ve sampled enough?
15

Direct Sampling
Simple
Works only for easy distributions
Rejection Sampling
Create samples like direct sampling
Only count samples consistent with given evidence
Importance Sampling
Create samples like direct sampling
Assign weights to samples
Gibbs Sampling
Often used for high-dimensional problem
Use variables and its Markov blanket for sampling
16

Want to sample from distribution 𝑃 𝑋 =
1
𝑍 𝑓(𝑋)
𝑃(𝑋) is difficult to sample, but f(X) is easy to evaluate
Key idea: sample from a simpler distribution Q(X), and then select samples
Rejection sampling (choose
𝑀 s.t. 𝑓 𝑋 ≤ 𝑀 𝑄 𝑋
) 𝑖 = 1
Repeat until 𝑖 = 𝑁 𝑥 ∼ 𝑄 𝑋 , 𝑢 ∼ 𝑈 0,1 if 𝑢 ≤ 𝑓 𝑋
𝑀 𝑄 𝑋
, accept 𝑥 and 𝑖 = 𝑖 + 1 ; otherwise, reject 𝑥
Reject sample with probability 1 − 𝑓 𝑋
𝑀 𝑄 𝑋
17

Sample 𝑥 ∼ 𝑄(𝑋)
and reject with probability
1 − 𝑓 𝑋
𝑀 𝑄 𝑋
𝑀 𝑄(𝑥
1
)
𝑀 𝑄(𝑋) 𝑓(𝑥
1
)
Between red and blue curves is rejection region 𝑓(𝑋) 𝑢
1
∼ 𝑈[0,1]
𝑋 𝑥
1
∼ 𝑄(𝑋)
The likelihood of seeing a particular 𝑥
𝑃 𝑥 =
∫ 𝑓 𝑥
𝑀𝑄 𝑥 𝑓 𝑋
𝑀𝑄 𝑋
𝑄(𝑥)
𝑄 𝑋 𝑑𝑋
=
1
𝑍 𝑓(𝑥)
18

Can have very small acceptance rate
𝑀 𝑄(𝑥
1
)
𝑀 𝑄(𝑋)
Big rejection region 𝑓(𝑋) 𝑢
1
∼ 𝑈[0,1] 𝑥
1
∼ 𝑄(𝑋)
Adaptive rejection sampling: use envelop function for Q
𝑋
19
𝑋

Support sampling from 𝑃 𝑋 is hard
We can sample from a simpler proposal distribution 𝑄 𝑋
If
𝑄 dominates P (i.e.,
𝑄 𝑋 > 0
whenever
𝑃 𝑋 > 0
), we can sample from 𝑄 and reweight
𝐸
𝑃 𝑓 𝑋 = ∫ 𝑓 𝑋 𝑃 𝑋 𝑑𝑋) = ∫ 𝑓(𝑋)
𝑃 𝑋
𝑄 𝑋
Sample 𝑥 𝑖
∼ 𝑄 𝑋
Reweight sample 𝑥 𝑖
with 𝑤 𝑖
=
𝑃 𝑥 𝑖
𝑄 𝑥 𝑖 approximate expectation with
1
𝑁
𝑓 𝑥 𝑖
𝑄(𝑋)𝑑𝑋 𝑤 𝑖
20

Instead of reject sample, reweight sample instead
𝑄(𝑥
2
)
𝑄(𝑥
1
)
𝑄(𝑋)
𝑃(𝑥
2
) 𝑃(𝑥
1
)
𝑃(𝑋) 𝑤
2 𝑥
2
∼ 𝑃 𝑥
2
∼ 𝑄(𝑋)
/𝑄 𝑥
2 𝑥
1 𝑤
1
∼ 𝑄(𝑋)
∼ 𝑃 𝑥
1
/𝑄 𝑥
1
The efficiency of importance sampling depends on how close the proposal Q is to the target P
𝑋
21

Suppose we only evaluate 𝑃 𝑋 =
1
𝑍 𝑓(𝑋) is difficult to compute), and don’t know Z
(e.g., grid or MRF, Z
We can get around the normalization Z by normalize the importance weight 𝑟 𝑋 = 𝑓 𝑋
𝑄 𝑋
⇒ 𝐸
𝑄 𝑟 𝑋 = ∫ 𝑓 𝑋
𝑄 𝑋
𝑄 𝑋 𝑑𝑋 = ∫ 𝑓 𝑋 𝑑𝑋 = 𝑍
𝐸
𝑃 𝑔 𝑋 = ∫ 𝑔 𝑋 𝑃 𝑋 𝑑𝑋 =
1
𝑍
∫ 𝑔 𝑋
∫ 𝑔 𝑋 𝑟 𝑋 𝑄 𝑋 𝑑𝑋
∫ 𝑟 𝑋 𝑄 𝑋 𝑑𝑋
≈
𝑔 𝑥 𝑖 𝑟 𝑥
𝑟 𝑥 𝑖 𝑖 𝑓 𝑋
𝑄 𝑋
𝑄 𝑋 𝑑𝑋 = 𝑑𝑒𝑓𝑖𝑛𝑒 𝑤 𝑖
= 𝑟 𝑥 𝑖
𝑟 𝑥 𝑖
𝑎𝑠 𝑡ℎ𝑒 𝑛𝑒𝑤 𝑖𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒 𝑤𝑒𝑖𝑔ℎ𝑡
22

Difference between normalized and unnormalized
Unnormalized importance sampling is unbiased
Normalized importance sampling is biased, eg. for N=1
𝐸
𝑄 𝑓 𝑥
1 𝑤
1 𝑤
1 = 𝐸
𝑄 𝑓 𝑥
1
≠ 𝐸
𝑃 𝑓 𝑥
1
However, the normalized importance sampler usually has lower variance
But most importantly, unnormalized importance sampler work for unnormalized distributions
For unnormalized sampler, one can also do resampling based on 𝑤 𝑖
(multinomial distribution with probability 𝑤 𝑖
for 𝑥 𝑖
)
23

Both rejection sampling and importance sampling do not scale well to high dimensions
Markov Chain Monte Carlo (MCMC) is an alternative
Key idea : Construct a Markov chain whose stationary distribution is the target distribution 𝑃 𝑋
Sampling process: random walk in the Markov chain
Gibbs sampling is a very special and simple MCMC method.
24

Wan to sample from 𝑃 𝑋 , start with a random initial vector X
𝑋 𝑡 : 𝑋 at time step 𝑡
𝑋 𝑡
transition to 𝑋 𝑡+1
with probability
𝑄(𝑋 𝑡+1
| 𝑋 𝑡 , … , 𝑋 1 ) = 𝑇 (𝑋 𝑡+1
| 𝑋 𝑡 )
The stationary distribution of 𝑇 (𝑋 𝑡+1
| 𝑋 𝑡 ) is our 𝑃 𝑋
Run for an intial 𝑀 samples (burn-in time) until the chain converges/mixes/reaches the stationary ditribution
Then collect 𝑁 (correlated) sample as 𝑥 𝑖
Key issues : Designing the transition kernel, and diagnose convergence
25

A very special transition kernel, works nicely with Markov blanket in GMs.
The procedure
We have variables set 𝑋 = 𝑋
1
, … , 𝑋
𝐾
variables in a GM.
At each step, one variable 𝑋 𝑖
is selected (at random or some fixed sequence), denote the remaining variables as 𝑋
−𝑖
, and its current value as 𝑥 𝑡
−𝑖
Compute the conditional distribution 𝑃(𝑋 𝑖
| 𝑥 𝑡
−𝑖
)
A value 𝑥 𝑖 𝑡
is sampled from this distribution
This sample 𝑥 𝑖 𝑡
replaces the previous sampled value of 𝑋 𝑖
in 𝑋
26

Gibbs sampling
𝑋 = 𝑥 0
For t = 1 to N 𝑥 𝑥
1
2
… 𝑥 𝑡
𝐾
= 𝑃(𝑋
1
| 𝑥
= 𝑃(𝑋
2
| 𝑥
= 𝑃(𝑋
2
| 𝑥 𝑡
1
, … , 𝑥
, … , 𝑥
, … , 𝑥 𝑡
𝐾−1
)
)
)
Variants:
Randomly pick variable to sample sample block by block
𝐹𝑜𝑟 𝑔𝑟𝑎𝑝ℎ𝑖𝑐𝑎𝑙 𝑚𝑜𝑑𝑒𝑙𝑠
Only need to condition on the
Variables in the Markov blanket
𝑋
2
𝑋
3
𝑋
4
𝑋
1
𝑋
5
27

Pairwise Markov random field
𝑃 𝑋
1
, … , 𝑋 𝑘
∝ exp 𝜃 𝑖𝑗
𝑋 𝑖
𝑋 𝑗
+ 𝜃 𝑖
𝑋 𝑖
In the Gibbs sampling step, we need conditional 𝑃(𝑋
1
| 𝑋
2
, … 𝑋 𝑘
)
𝑃(𝑋
1
| 𝑋
2 exp 𝜃
12
, … 𝑋 𝑘
𝑋
1
𝑋
2
) ∝
+ 𝜃
13
𝑋
1
𝑋
3
+ 𝜃
14
𝑋
1
𝑋
4
+ 𝜃
15
𝑋
1
𝑋
5
+ 𝜃
1
𝑋
1
𝑋
6
𝑋
5
𝑋
2
𝑋
1
𝑋
4
𝑋
7
𝑋
3
28

Good chain
Sampled Value
Iteration number
29

Bad chain
Sampled Value
Iteration number
30

Direct Sampling
Simple
Works only for easy distributions
Rejection Sampling
Create samples like direct sampling
Only count samples consistent with given evidence
Importance Sampling
Create samples like direct sampling
Assign weights to samples
Gibbs Sampling
Often used for high-dimensional problem
Use variables and its Markov blanket for sampling
31