Javier Turek and Eyal Regev
Polytrees
…
U1
Ui
…
Un
e+
X
Y
Z
e-

Like a causes
tree, butsharing
with multiple
parents.
Several
a common
effect.
Link matrix Tx|u

Tx|u contains the conditional probabilities:
 P(X=x | U1=u1,…,Un=un)
1.
The table is huge (contains 2n entries).
2.
Who can know such information anyway?
 You cannot expect to find the P(X | U1,…,Un) table.
 However, you may know how every Ui influences X
separately.
The OR-gate
Caught
cheating
Haven’t done
homework
OR
Failed an
exam
More likely!
Inhibitors
Haven’t
done
homework
Paid
the TA
Caught
cheating
Re-doing
the course
AND
AND
OR
Failed an
exam
Inhibitors
I1
U1
I2
U2
AND
AND
AND


Inhibitors are
independent
Associate
probability to
an inhibitor.
In
Un
OR
X
Noisy OR-Gate
I1
U1
I2
U2
In
Un
AND
AND
AND
OR
X
P   x |  ui ,  u k : k  i   qi
Noisy OR-Gate
I1
U1
I2
U2
AND
In
Un
Tu
AND
AND
OR
Tu   i : U i  T ru e 
X
 qi
 i  Tu
Px |u  
1   q i
i  Tu

if  x
if + x
Message Passing – updating X
B elief

x  P x | e ,e




 

P e | x P x |e

   D x Ax
U
 u 
C X u 
e+
X
UY x
  y
UZ x
Y
Z
e-
D  x   U Y  x U Z  x 
Ax 
T
u
x |u
C X u 
Message Passing – updating X
B elief

x  P x | e ,e




 

P e | x P x |e

   D x Ax
U
 u 
W
C X u 
CX w
e+
X
UY x
  y
UZ x
Y
Z
e-
D  x   U Y  x U Z  x 
Ax 
T
u u, w
CC  u u  C X  w 
x |u , w X X
Message Passing – updating U,Y,Z
U X u  
 D  xT
CY  x    A  x U Z  x 
x |u
x
U
U X u 
e+
X
CZ  x
CY  x 
Y
Z
e-
D  x   U Y  x U Z  x 
Ax 
T
u
x |u
C X u 
Message Passing – updating W,U,Y,Z
U
UXX u u 
D
 xD  xT x|u T,w Cx |uX  w 
x
x
w
CY  x    A  x U Z  x 
U
W
U X u 
U X w
e+
X
CZ  x
CY  x 
Y
Z
e-
D  x   U Y  x U Z  x 
Ax 
T
u u, w
C C  u u  C X  w 
x |u , w X X
Message Passing – many parents
U X  ui  
 D x 
u k :k  i
x
T x |u  C X  u k

CY  x    A  x U Z  x 
k i
…
Ui
U1
…
Un
U X  ui 
e+
U X  un 
U X  u1 
X
Link
matrix
CZ  x
CY  x 
Y
Z
e-
D  x   U Y  x U Z  x 
Ax 
 T  C u 
x |u
u
X
k
k
Belief Update – Noisy OR-Gate
 D   x    qi  C X  u k 

u iT u
k

B el  x   


 D   x    1   qi   C X  u k 

u 
i T u
 k
  q C  u  C  u 
i
u

iT u
i
X
k
k T u
  q C   u  C   u 
i
u

X
X
i
i T u
X
k
k T u
  q C   u  1  C   u  
i
u
iTu
X
i
X
k Tu
k
if  x
if + x
Belief Update – Noisy OR-Gate

  q C   u  1  C   u  
i
u

X
i
iTu
X
k
k Tu
 q C u   1  C u 
i
X
i
X
i
i

 1  1  q  C   u  
i
X
i
i
  D   x   1  1  q i  C X   u i  

i
B el  x   


  D   x  1    1   1  q i  C X   u i   

i


if  x
if + x
Update messages
U X  ui 


 



 D   x  q i  i  D   x   1  q i  i  
if + u i
 D   x   i  D   x   1   i  
if  u i
Where  i 
 1  1  q  C   u  
k
X
k
k i

The message to the child is the same:
CX
 yi     U Y  x A  x 
k i
k
Example
Windows
D1
Vista
No
D2
electricity
Virus
D3
2KDbug
4
Does not
M1
start
Wrong
M2
date
Stolen
M3
Password
Lost
M4data
Example
p i \ (1  p i )
0.01\0.99
0.1\0.9
0.2\0.8
0.2\0.8
D2
D3
D4
D1
q ij
0.2
0.8
0.5
0.2
0.8
0.1
M1
0.1
0.9
M2
M3
W here q ij  P   m j |  d i ,  d k for k  i 
0.7
M4
Conjunction query

Conjunction query q: find the belief that
several events happen simultaneously.
q 
X i  xi
i I Q

Applying the chain rule:
P  q   P  x m , x m 1 , x m  2 , , x 2 , x 1  
 P  x1   P  x 2 | x1 
P  x m 1 | x m  2 ,
 P  x m | x m 1 , x m  2 ,
 Product of m belief updates
, x 2 , x1 
, x1 
Answering a query

In our example:
q    d 1     m1     m 2     m 3     m 4 

Computing P(q):
P  q   P   d 1   P   m1 |  d 1   P   m 2 |  m1 ,  d 1 
 P   m 3 |  m 2 ,  m1 ,  d 1 
 P   m 4 |  m 3 ,  m 2 ,  m1 ,  d 1 
Example
Update the belief on M1
P   m1 |  d 1   1  1  1  q11  1  1  1  q 2 1  p 2   0.272
 Update likelihoods and priors:
U  d    1  q q  ,1  q    0.92, 0.2 

M1
2
11
21
11


C M 4  d 2    U M 1   d 2  p 2 , U M 1   d 2  1  p 2    0.338, 0.662 
D1
D2
D3
D4
M3
M4
CM4 d2 
p1  1
p2
M1
U M1  d 2 
M2
Example
Update the belief on M2
P   m |  m ,  d   P   m |  d   1  1  q  1  1  1  q  p   0.81

2

1
1
2
1
12
42
4
Update likelihoods and priors:
U M 2  d 4    q12 q 42 , q12    0.45, 0.9 


C M 4  d 4    U M 2   d 4  p 4 , U M 2   d 4   1  p 4    0.111, 0.889 
D1
D2
D3
D4
p4
U M2 d4 
1  1p
M1
M2
CM4 d4 
M3
M4
Example

Update the belief on M3
P   m 3 |  m 2 ,  m1 ,  d 1   P   m 3 |  d 1 
 1  1  1  q13  1  1  1  q 33  p 3   0.836

Update likelihoods and priors:
U M 3  d 3    1  q13 q 33 ,1  q13    0.98, 0.8 


C M 4  d 3    U M 3   d 3  p 3 , U M 3   d 3   1  p 3    0.23, 0.77 
D1
D2
p1  1
D3
D4
CM4 d3 
p3
U M3 d3 
M1
M2
M3
M4
Example

Update the belief on M4
P   m 4 |  m 3 ,  m 2 ,  m1 ,  d 1  
 1  1  q  C   d    0 . 7 81
i4
i  2 ,3 ,4
D1
M4
D2
i
D3
CM4 d2 
M1
M2
D4
CM4 d3 
M3
CM4 d4 
M4
Example

And the final solution is…
Example

And the final solution is…
P  q   P   d 1   P   m1 |  d 1   P   m 2
|  m1 ,  d 1 
 P   m 3 |  m 2 ,  m1 ,  d 1 
 P   m 4 |  m 3 ,  m 2 ,  m1 ,  d 1  
 0.01  0.272  0.81  0.836  0.781  1.439  10
3
Thank You!