Presenter： Kuang-Jui Hsu
Date ：2011/5/23(Tues.)
Outline
• Introduction
• Conditional Independence Properties
• Factorization Properties
• Illustration: Image De-noising
Relation to Directed Graphs
Introduction
• Based on a undirected graph
• The MRF model has a simple form and is easy to use
• Based on conditional independence properties
Conditional Independence Properties
• In an undirected graph, there are three sets of nodes,
denoted A, B, C, and A is conditionally independent
of B given C
• Shorthand notation:
<=> p(A|B, C) = p(A|C)
Conditional independence property
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Testing Methods in a Graph
Simple Form
• A node will be conditionally independent of all other
nodes conditioned only on neighbouring nodes
Factorization Properties
• In a directed graph
• Generalized form:
In an Undirected Graph
• Consider two nodes and
that are not
connected.
• Must be conditionally independent
• So, the conditional independence property can be
expressed as
The set x of all variables with
and
Factorization Property
removed
Clique
• This leads us to consider a graphical concept:
Clique:
Clique
Maximal Clique:
Potential Function
• Define the factors in the potential function by using
the clique
 Generally, consider the maximal cliques, because
other cliques must be the subsets of maximal cliques
Potential Function
 Potential function over the maximal cliques of the graph
Clique
The set of variables in that clique
 The joint distribution:
to zero orconstant
positive
Partition function: Equal
a normalization
Partition Function
• The normalization constant is the major limitations
• A model with M discrete nodes each having K states,
then the evaluation involves summing over
• Needed for parameter learning
states
Exponential growth
• Because it will be a function of any parameters that
govern the potential functions
Connection between Conditional
Independence And Factorization
• Define
:
For any node
, the following conditional property holds
The neighborhood of
All nodes expect
• Define
:
A distribution can be expressed as
The Hammerley-Clifford theorem states that the sets
and
identical.
Potential Function Expression
• Restrict the potential function to be positive
• It is convenient to express them as exponentials
Energy function
Boltzmann distribution
• The total energy is obtained by adding the energies of
each of the maximal energy
Illustration: Image De-noising
• Noisy image
• Described by an array of binary pixel values
, where the index i = 1, . . ., D runs over
all pixels.
Illustration: Image De-noising
• Noise-free image
• Described by an array of binary pixel values
, and randomly flipping the sign of
pixels with some small probability
Create the MRF Model
• A strong correlation between and
• A strong correlation between the neighbouring pixles
• MRF model:
• The graph has two types of cliques,
each of which contain two variables.
• The clique form
, uses the
form of the energy function
• The clique form
, uses the
form of the energy function
• The parameters
and
and are positive,
are neighbour
The Energy Function
• The complete energy function:
2.
1. negative
postitve
• The joint distribution
Solve by ICM
• For the purpose of image restoration, find an image x
having a high probability
• Use a simple iterative technique called iterated
condition mode ( ICM)
• Simply an application of coordinate-wise gradient ascent
The steps of ICM
Evaluate the total energy for -1 and 1
choose the lower energy, and update
Stop until convergence
Result
Use ICM
Use graph-cut
Relation to Directed Graphs
• Solve the problem of taking a model that is specified
using a directed graph and trying to convert it to
undirected graph
Directed graph
Undirected graph
Relation to Directed Gaphs
This is easily done by identifying
Relation to Directed Graphs
 Consider how to generalize this construction
 This can be achieved if the clique potentials of the
undirected graph are given by the conditional
distributions of the directed graph.
 Ensure that the set of variables that appears in each of
conditional distributions is a member of at least one
clique of the undirected graph
Generalize This Construction
 For nodes having one parent
Convert the Directed Graph to the
Undirected Graph
 For nodes having more than one parent
Moral graph
Involving
thehas
fourbecome
variables,
so they
must belong to a
The process
known
as moralization
single clique if this conditional distribution is to be
absorbed in a clique potential
Convert the Directed Graph to the
Undirected Graph
 Discard some conditional independence properties
 In fact,we can simply using a fully connected
undirected graph
 However, this would discard all conditional properties
 The moralization adds the fewest extra links and so
retain the maximum number of independence
properties
Special Graph
 There are two type of graph that can express different
conditional independence properties
 Type 1: dependence map(D-map)
 Type 2: Independence map(I-map)
Dependence Map(D-Map)
 Every conditional independence statement satisfied by
the distribution is reflected in the graph
 A completely disconnected graph
Independence Map(I-Map)
 Every conditional independence statement implied by
a graph is satisfied by a specific distribution
 A full connected graph
 A perfect map: both I-map and D-map
Perfect Map
Directed
graph
Undirected
graph
The set of all distributions P over a given set of
variables