Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models Reconstruction
Graph Theory Course Project
Firoozeh Sepehr
April 27th 2016
Firoozeh Sepehr — Graphical Models Reconstruction
1/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Outline
1 Overview
2 History and Background
3 Graphical Models
4 Reconstruction
5 Open Issues
Firoozeh Sepehr — Graphical Models Reconstruction
2/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Outline
1 Overview
2 History and Background
3 Graphical Models
4 Reconstruction
5 Open Issues
Firoozeh Sepehr — Graphical Models Reconstruction
3/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Overview
What are graphical models?
Graphical Models 1;2
Combination of Probability Theory and Graph Theory
Tackling problems of uncertainty and complexity
Utilizing modularity for complex systems
Graphical representation of dependencies embedded in probabilistic
models
b
a
c
g
e
d
f
Bayesian/Belief Networks
Firoozeh Sepehr — Graphical Models Reconstruction
b
a
c
g
e
d
f
Markov Networks
4/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Overview
Markov vs Bayesian Networks
Markov Networks
Undirected graphical models
Correlations between variables
Mostly used in physics and vision communities
Bayesian/Belief Networks
Directed graphical models
Directed Acyclic Graphs (DAGs)
Causal relationships between variables
Mostly used in AI and machine learning communities
Use Bayes’ rule for inference
Firoozeh Sepehr — Graphical Models Reconstruction
5/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Overview
Applications
So many different applications
Pattern recognition
Diagnosis of diseases
Desicion-theoretic systems 4
Statistical physics
Signal and image processing
Inferring cellular networks in biological systems 3
Firoozeh Sepehr — Graphical Models Reconstruction
6/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Outline
1 Overview
2 History and Background
3 Graphical Models
4 Reconstruction
5 Open Issues
Firoozeh Sepehr — Graphical Models Reconstruction
7/50
Overview History and Background Graphical Models Reconstruction Open Issues References
History and Background
Probability theory
Foundations of probability theory 2
Go back to 16th century when Gerolamo Cardano began a formal
analysis of games of chance, followed by additional key developments by
Pierre de Fermat and Blaise Pascal in 17th century. The initial
development involved only discrete probability spaces and the analysis
methods were purely combinatorial.
Gerolamo Cardano
Italian, 1501-1576
Science, maths,
philosophy, and literature 9
Firoozeh Sepehr — Graphical Models Reconstruction
Pierre de Fermat
French, 1601-1665
Mathematics and law 10
Blaise Pascal
French, 1623-1662
Theology, mathematics,
philosophy and physics 11
8/50
Overview History and Background Graphical Models Reconstruction Open Issues References
History and Background
Probability theory
Foundations of probability theory - cont’d
The foundations of modern probability theory were laid by Andrey
Kolmogorov in the 1930s.
Andrey Kolmogorov
Russian, 1903-1987
Mathematics
Known for Topology, Intuitionistic logic,
Turbulence studies, Classical mechanics,
Mathematical analysis, Kolmogorov complexity 12
Firoozeh Sepehr — Graphical Models Reconstruction
9/50
Overview History and Background Graphical Models Reconstruction Open Issues References
History and Background
Bayes rule
Bayes theorem 2
Shown in the 18th century by Reverend Thomas Bayes. This theorem
allows us to use a model that tells us the conditional probability of event
a given event b in order to compute the contrapositive: the conditional
probability of event b given event a. This type of reasoning is central to
the use of graphical models - Bayesian network.
Thomas Bayes
English, 1701-1761
Statistician, philosopher
and Presbyterian minister 13
Firoozeh Sepehr — Graphical Models Reconstruction
10/50
Overview History and Background Graphical Models Reconstruction Open Issues References
History and Background
Origins of graphical models
Origins of graphical models 2
Representing interactions between variables in a multidimensional
distribution using a graph structure originates in several communities
Statistical physics: Gibbs - used an undirected graph to represent
the distribution over a system of interacting particles
Genetics: path analysis of Sewal Wright - proposed the use of a
directed graph to study inheritance in natural species
Statistics: Bartlett - analyzing interactions between variables in the
study of contingency tables, also known as log-linear models
Computer science: Artificial Intelligence (AI) to perform difficult
tasks such as oil-well location or medical diagnosis, at an expert level
Firoozeh Sepehr — Graphical Models Reconstruction
11/50
Overview History and Background Graphical Models Reconstruction Open Issues References
History and Background
Origins of graphical models
Expert systems 2
Need for methods that allow the interation of multiple pieces of
evidence and provide support for making decisions under uncertainty
Huge success in predicting the diseases using evidences like
sysmptoms and test results in the 1970s
Fell into disfavor in AI community
1
2
AI should be based on similar methods to human intelligence
Use of strong independence assumptions mae in the existing expert
systems was not a flexible, scalable mechanism
Firoozeh Sepehr — Graphical Models Reconstruction
12/50
Overview History and Background Graphical Models Reconstruction Open Issues References
History and Background
Origins of graphical models
Expert systems - cont’d
Widespread acceptance of probabilistic methods began in the late
1980s
1
Series of seminal theoretical developments
Bayesian network framework by Judea Pearl and his colleaagues in
1988
Foundations for efficient reasoning using probabilistic graphical
models by S. L. Lauritzen and D.J. Spiegelhalter in 1988
2
Construction of large-scale, highly successful expert systems based
on this framework that avoided the unrealistically strong assumptions
made by early probabilistic expert systems
Pathfinder expert system (which assists community pathologists with
the diagnosis of lymph-node pathology) constructed by Heckerman
and colleagues in 1992 14
Firoozeh Sepehr — Graphical Models Reconstruction
13/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Outline
1 Overview
2 History and Background
3 Graphical Models
4 Reconstruction
5 Open Issues
Firoozeh Sepehr — Graphical Models Reconstruction
14/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Definitions
Directed and undirected graphs
G = (N, E ) is an undirected graph
G = (N, E~ ) a directed graph
Degree, indegree and outdegree
For a vertex y ∈ N
degree is deg (y )
indegree is deg − (y )
outdegree is deg + (y )
Root and leaf
If deg − (y ) = 0, y is a root and if deg + (y ) = 0, y is a leaf
Firoozeh Sepehr — Graphical Models Reconstruction
15/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Definitions
Chains and paths
A chain starting from yi and ending in yj is an ordered sequence of
distinct nodes (yπ1 , yπ2 , ..., yπl−1 , yπl ) where yi = yπ1 , yj = yπl and
(yk , yk+1 ) ∈ E~
A path starting from yi and ending in yj is an ordered sequence of
distinct nodes (yπ1 , yπ2 , ..., yπl−1 , yπl ) where yi = yπ1 , yj = yπl and
either (yk , yk+1 ) ∈ E~ or (yk+1 , yk ) ∈ E~
Note
Chains are a special case of paths!
Firoozeh Sepehr — Graphical Models Reconstruction
16/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Definitions
Parents, Children, Ancestors, Descendants
Consider a directed graph G = (N, E~ ) and yi ∈ N. Given a set X ⊆ N:
yi is a parent of yj if there is a directed edge from yi to yj
pa(X ) := {yi ∈ N|∃yj ∈ X : yi is a parent of yj }
yj is a child of yi if there is a directed edge from yi to yj
ch(X ) := {yj ∈ N|∃yi ∈ X : yj is a child of yi }
yi is an ancestor of yj if there is a chain from yi to yj
an(X ) := {yi ∈ N|∃yj ∈ X : yi is an ancestor of yj }
yj is a descendant of yi if there is a chain from yi to yj
de(X ) := {yj ∈ N|∃yi ∈ X : yj is a descendant of yi }
Neighbors
ngb(yi ), are the union of parents and children set.
Firoozeh Sepehr — Graphical Models Reconstruction
17/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Definitions
Visualize ...
Roots, Leaves
Paths, Chains
Parents, Children, Ancestors, Descendants, Neighbors
b
a
c
g
e
d
f
Firoozeh Sepehr — Graphical Models Reconstruction
18/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Definitions
Forks, inverted forks and chain links 6
Consider a path (yπ1 , yπ2 , ..., yπl−1 , yπl ) in a directed graph G = (N, E~ ).
Vertex yπi is
a fork if (yπ , yπ ) and (yπ , yπ ) are in E~
i
i−1
i
i+1
an inverted fork (or collider) if (yπi−1 , yπi ) and (yπi+1 , yπi ) are in E~
a chain link in all other cases
b
a
c
g
e
d
f
Firoozeh Sepehr — Graphical Models Reconstruction
19/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
What is factorization?
Factorization
Joint probability distribution
Using the chain rule and assuming an arbitrary order d on variables 2
p(x1 , x2 , ..., xn ) = Πni=1 p(xi |x1 , x2 , ..., xi−1 )
(1)
Using graphical models - leads to a compact representation 8
Undirected GM
p(x1 , x2 , ..., xn ) =
1
Π(i,j)∈E φk (xi , xj )
Z
Undirected Tree GM (using junction tree theory)
p(xi , xj )
p(x1 , x2 , ..., xn ) = Πni=1 p(xi )Π(i,j)∈E
p(xi )p(xj )
(2)
(3)
Directed GM
p(x1 , x2 , ..., xn ) = Πni=1 p(xi |pa(xi ))
Firoozeh Sepehr — Graphical Models Reconstruction
(4)
20/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
What is factorization?
Example 1
Consider we have N binary random variables, for representation of joint
probability distribution
chain rule requires O(2N ) parameters
GM requires O(2|pa| ) which could reduce the number of parameters
exponentially depending on which conditional assumptions we make
- helps in inference and learning
Firoozeh Sepehr — Graphical Models Reconstruction
21/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
What is factorization?
Example 2
Joint probability distribution 1
Using the chain rule
p(x1 , x2 , x3 , x4 , x5 , x6 ) = p(x1 )p(x2 |x1 )p(x3 |x1 , x2 )
p(x4 |x1 , x2 , x3 )p(x5 |x1 , x2 , x3 , x4 )
(5)
p(x6 |x1 , x2 , x3 , x4 , x5 )
Using graphical models
p(x1 , x2 , x3 , x4 , x5 , x6 ) = p(x1 )p(x2 |x1 )p(x3 |x2 , x5 )
p(x4 |x1 )p(x5 |x4 )p(x6 |x5 )
x2
x1
(6)
x3
x6
x4
Firoozeh Sepehr — Graphical Models Reconstruction
x5
22/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Fun application of joint distribution factorization
In rooted trees
Joint probability distribution is the same! Use the Bayes’ rule ...
b
a
b
c
a
Undirected
b
c
a is root
a
b
c
b is root
a
c
c is root
p(a, b, c) = p(a)p(b|a)p(c|b)
p(b) p(a, b)
p(c|b)
p(b) p(a)
= p(b)p(a|b)p(c|b)
= p(a)
(7)
= p(c)p(b|c)p(a|b)
Firoozeh Sepehr — Graphical Models Reconstruction
23/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Undirected Graphical Models
Undirected graphical models
Family of multivariate probability distributions that factorize
according to a graph G = (N, E )
Set of vertices, N, represents random variables
Set of edges, E , encodes the set of conditional independencies
between variables
Definition
Random vector X is said to be Markov on G if for every i, the random
variable xi is conditionally independent of all other variables given its
neighbours.
p(xi |x\i ) = p(xi |ngb(xi ))
(8)
where p is the joint probability distribution.
Firoozeh Sepehr — Graphical Models Reconstruction
24/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Undirected Graphical Models
Tree-structured graphical models
Family of multivariate probability distributions that are Markov on a
tree T = (N, E )
Firoozeh Sepehr — Graphical Models Reconstruction
25/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Definition
d-separation 6
A subset of variables S is said to separate xi from xj if all paths
between xi and xj are separated by S
A path P is separated by a subset S of variables if at least one pair
of successive edges along P is blocked by S
block 6
Two edges meeting head-to-tail or tail-to-tail at node x (x is a chain
or a fork) are blocked by S if x is in S
Two edges meeting head-to-head at node x (x is an inverted fork)
are blocked by S if neither x nor any of its descendants is in S.
Firoozeh Sepehr — Graphical Models Reconstruction
26/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Definition
d-separation Example 6
x1
d–sep(x2 , x3 |{x1 })?
x2
d–sep(x2 , x3 |{x1 , x4 })?
x4
x3
x5
d–sep(x2 , x3 |{x1 , x6 })?
x6
Firoozeh Sepehr — Graphical Models Reconstruction
27/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Interesting application
Lumiere project 5
The Lumiere Project centers on harnessing probability and utility to
provide assistance to computer software users. Lumiere prototypes served
as the basis for components of the Office Assistant in the Microsoft
Office ’97 suite of productivity applications.
Infers a user’s needs by considering a user’s background, actions, and queries
Challenges are
Model construction about time-varying goals of computer users
Needs a large database - over 25,000 hours of usability studies were
invested in Office ’97
Firoozeh Sepehr — Graphical Models Reconstruction
28/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Outline
1 Overview
2 History and Background
3 Graphical Models
4 Reconstruction
5 Open Issues
Firoozeh Sepehr — Graphical Models Reconstruction
29/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
What is reconstruction?
Reconstruction
The problem is that samples are available only from a subset of
variables
The goal is to learn the minimal latent tree - trees without any
redundant hidden nodes
Latent and minimal latent trees
A latent tree is a tree with node set N = V ∪ H, where V is the set
of observed nodes and H is the set of latent (hidden) nodes.
Set of minimal latent trees, T≥3 , is the set of latent trees that each
hidden node has at least three neighbors (hidden or observed)
Note
All leaves are observed, although not all observed nodes need to be leaves.
Firoozeh Sepehr — Graphical Models Reconstruction
30/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Graphical Models
Interesting application
Vista system 4
A decision-theoretic system that has been used at NASA Mission Control
Center in Houston for several years.
Uses Bayesian networks to interpret
live telemetry and provides advice
on the likelihood of alternative
failures of the space shuttle’s
propulsion systems.
Considers time criticality and
recommends actions of the highest
expected utility
Employs decision-theoretic methods
for controlling the display of
information to dynamically identify
the most important information to
highlight
Firoozeh Sepehr — Graphical Models Reconstruction
31/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Additive metric
Define a measurement 8
Information distances
Defined for pairwisse distributions
For guassian graphical models, correlation coefficient of two random
variables xi and xj
ρij = p
cov (xi , xj )
var (xi )var (xj )
(9)
Information distance
dij = − log |ρij |
(10)
Inverse relation between information distance and correlation
Extendable to discrete random variables
Firoozeh Sepehr — Graphical Models Reconstruction
32/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Additive metric
Proposition 8
The information distances dij are additive tree metrics. In other words, if
the joint probabiliry distribution p(x) is a tree-structured graphical model
Markov on the tree Tp = (N, Ep ), then the information distances are
additive on Tp .
X
dij
(11)
∀k, l ∈ N : dkl =
(i,j)∈Pathkl
Proof
Homework!
Firoozeh Sepehr — Graphical Models Reconstruction
33/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Sibling grouping
Lemma 8
For distances dij for all i, j ∈ V on a tree T ∈ T≥3 , the following two
properties on Φijk = dik − djk hold.
1
Φijk = dij for all k ∈ V\i,j iff i is a leaf and j is its parent
1
Φijk = −dij for all k ∈ V\i,j iff j is a leaf and i is its parent
2
−dij < Φijk = Φijk 0 < dij for all k, k ∈ V\i,j iff both i and j are
leaves and they have the same parent (they belong to the same
sibling group)
0
Proof of 2
Homework!
Firoozeh Sepehr — Graphical Models Reconstruction
34/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Sibling grouping
Proof of 1
⇐: Using the additive property of information distances, if i is a leaf
and j is its parent, dik = dij + djk , therefore, Φijk = dij for all k =
6 i, j.
⇒: By contradiction, i and j are not connected with an edge. Then
there exists a node u 6= i, j on the path connecting i and j. If
u ∈ V , then let k = u, otherwise, let k be an observed node in the
subtree away from i and j which exists since T ∈ T≥3 . Therefore,
dij = diu + duj > diu − duj = dik − dkj = Φijk which is a contradiction.
i
u
j
k
Firoozeh Sepehr — Graphical Models Reconstruction
35/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Sibling grouping
Proof of 1 - cont’d
⇒:
By contradition, if i is not a leaf, then there exists a node u 6= i, j
such that (i, u) ∈ E . Let k = u if u ∈ V , otherwise, let k be an
observed node in the subtree away from i and j. Therefore,
Φijk = dik − djk = −dij < dij which is again a contradiction,
therefore, i is a leaf.
j
i
u
k
Firoozeh Sepehr — Graphical Models Reconstruction
36/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Sibling grouping
Using previous Lemma to determine node relationships 8
For every pair of i, j ∈ V consider the following:
1
If Φijk = dij for all k ∈ V\i,j , then i is a leaf node and j is a parent of
i. Similarly, if Φijk = −dij for all k ∈ V\i,j , then j is a leaf and i is a
parent of j.
2
If Φijk is constant for all k ∈ V\i,j but not equal to either dij or −dij ,
then i and j are leaves and they are siblings.
If Φijk is not equal for all k ∈ V\i,j , then there are three cases:
3
(a) Nodes i and j are not siblings nor have a parent-child relationship.
(b) Nodes i and j are siblings but at least one of them is not a leaf.
(c) Nodes i and j have a parent-child relationship but the child is not a
leaf.
Firoozeh Sepehr — Graphical Models Reconstruction
37/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Sibling grouping
Visualize ...
Case 1
Case 2
d1
d1
d3
d2
d7
d4
d5
d7
d4
d6
d5
d6
i
i
j
d3
d2
d8
j
d8
Φijk = −d8 = −dij
Φijk 6= dij
Φijk = d6 − d7
dij = d6 + d7
for all k ∈ V \ i, j
Firoozeh Sepehr — Graphical Models Reconstruction
38/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Sibling grouping
Visualize ...
Case 3a
Case 3b
d1
k
0
i
d6
k
d3
d2
d7
d5
d1
d3
d2
d4
Case 3c
d1
d4
j
d8
Φijk 6= Φijk 0
Φijk = d4 + d2 + d3 − d7
Φijk 0 = d4 − d2 − d3 − d7
Firoozeh Sepehr — Graphical Models Reconstruction
k
k
0
d7
d5
i
k
0
i
d4
d6
Φijk 6= Φijk 0
Φijk = d4 + d5
Φijk = d5
0
for all k, k ∈ V \ i, j
d7
d5
j
d8
d3
d2
d6
j
k
d8
Φijk 6= Φijk 0
Φijk 0 = d4 − d5
Φijk 0 = −d5
39/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Recursive Grouping (RG) Algorithm
Recursive Grouping (RG) Algorithm
1
Initialize Y = V
2
Compute Φijk = dik − djk for all i, j, k ∈ Y
3
4
Using sibling grouping, define {Πl }Ll=1 to be partitions of Y such
that for every subset Πl (with |Πl | ≥ 2), any two nodes are either
siblings which are leaves or they have a parent-child relationship in
which the child is a leaf
Add singles sets to Ynew
5
For each Πl with |Πl | ≥ 2, if Πl contains a parent node, add it to
Ynew , otherwise, create a new hidden node and connect it to all the
nodes in Πl and add the node to Ynew
6
Update Yold to be Y and Y to be Ynew
7
Compute the distances of new hidden nodes
8
If |Y | ≥ 3, go to step 2, otherwise, if |Y | = 2, connect two
remaining nodes in Y and stop. If |Y | = 1, stop.
Firoozeh Sepehr — Graphical Models Reconstruction
40/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Recursive Grouping (RG) Algorithm
Visualize ...
h3
h2
1
2
4
3
h1
5
1
2
First iteration
h3
h2
2
4
h1
5
6
6
Original latent tree
1
5
4
3
h1
6
Second iteration
Firoozeh Sepehr — Graphical Models Reconstruction
h3
h2
3
1
4
2
h1
5
3
6
Third iteration
41/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Recursive Grouping (RG) Algorithm
Proof of step 7
7
Compute the distances of new hidden nodes
Let i, j ∈ ch(h) and k ∈ Yold i, j. We know that
dih − djh = dik − djk = Φijk and dih + djh = dij . Therefore, we can recover
the distances between a previously active node i ∈ Yold and its new
hidden parent h ∈ Y using
dih =
1
(dij + Φijk )
2
(12)
For any other active node l ∈ Y , we can compute dhl using a child node
i ∈ ch(h) using
dil − dih ,
if l ∈ Yold
dhl =
(13)
dik − dih − dlk , otherwise, where k ∈ ch(l)
Firoozeh Sepehr — Graphical Models Reconstruction
42/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Reconstruction
Recap
Steps to learn a latent tree
1
Define an additive metric
2
Perform sibling grouping test to determine nodes relationships
3
Perform RG algorithm
Firoozeh Sepehr — Graphical Models Reconstruction
43/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Outline
1 Overview
2 History and Background
3 Graphical Models
4 Reconstruction
5 Open Issues
Firoozeh Sepehr — Graphical Models Reconstruction
44/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Open Issues
What next?
Improvement!
Probabilistic models are used as a key component in some
challenging applications and they remain to be applied in some other
fields
Learning other types of GMs
Polytrees
General graphs
Applying the theorems on random processes
Define interrelations
Firoozeh Sepehr — Graphical Models Reconstruction
45/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Homework
Question 1
Prove that information distances are additive tree metrics.
Question 2
Prove that for distances dij for all i, j ∈ V on a tree T ∈ T≥3 , the
following the following property on Φijk = dik − djk holds
2
0
−dij < Φijk = Φijk 0 < dij for all k, k ∈ V\i,j iff both i and j are
leaves and they have the same parent (they belong to the same
sibling group)
Firoozeh Sepehr — Graphical Models Reconstruction
46/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Homework
Question 3
Draw the digraph associated with the following matrix and answer
the followings.
d–sep(x1 , x2 |{x6 , x7 })?
d–sep(x4 , x5 |{x1 , x2 , x3 , x6 })?
d–sep(x1 , x7 |{x3 , x4 , x5 })?

0
0

0

M=
0
0

0
0
Firoozeh Sepehr — Graphical Models Reconstruction
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
0
0
1
0
1
0
0
0
0
0
0
1
1
0
0
0

0
1

0

0

1

1
0
(14)
47/50
Overview History and Background Graphical Models Reconstruction Open Issues References
Questions?
Firoozeh Sepehr — Graphical Models Reconstruction
48/50
Overview History and Background Graphical Models Reconstruction Open Issues References
References I
[1] Probabilistic Reasoning in Intelligent Systems: Networks of Plausible
Inference J. Pearl, 1988
[2] Probabilistic Graphical Models, Principles and Techniques D. Koller,
N. Friedman, MIT Press, 2009
[3] Inferring Cellular Networks Using Probabilistic Graphical Models N.
Friedman, Vol 303, Issue 5659, pp. 799-805, 2004
[4] Vista Goes Online: Decision-Analytic Systems for Real-Time
Decision-Making in Mission Control M. Barry, E. Horvitz, C.
Ruokangas, S. Srinivas, N94-35063, 1994
[5] The Lumiere Project: Bayesian User Modeling for Inferring the
Goals and Needs of Software Users E. Horvitz, J. Breese, D.
Heckerman, D. Hovel, K. Rommelse, 1998
[6] Fusion, Propagation, and Structuring in Belief Networks J. Pearl,
Artificial Intelligence 29, 1986
Firoozeh Sepehr — Graphical Models Reconstruction
49/50
Overview History and Background Graphical Models Reconstruction Open Issues References
References II
[7] The Recovery of Causal Polytrees from Statistical Data G. Rebane,
J. Pearl, Proceedings of the Third Conference on Uncertainty in
Artificial Intelligence, 1987
[8] Learning Latent Tree Graphical Models M. J. Choi, V. Y. F. Tan, A.
S. Willsky, Journal of Machine Learning Research, Volume 12, 2011
[9] Gerolamo Cardano https://en.wikipedia.org/wiki/Gerolamo Cardano
[10] Pierre de Fermat https://en.wikipedia.org/wiki/Pierre de Fermat
[11] Blaise Pascal https://en.wikipedia.org/wiki/Blaise Pascal
[12] Andrey Kolmogorov
https://en.wikipedia.org/wiki/Andrey Kolmogorov
[13] Thomas Bayes https://en.wikipedia.org/wiki/Thomas Bayes
[14] An Evaluation of the Diagnostic Accuracy of Pathfinder D. E.
Heckerman, 1991
Firoozeh Sepehr — Graphical Models Reconstruction
50/50