Mean-Field Theory and Its Applications In
Computer Vision1
1
Introduction
• Problem formulation
• Mean-field based inference method
• Strategy for incorporating different costs
2
Labelling problem
Assign a label to each image pixel
Object segmentation
Stereo
Object detection
3
Problem Formulation
Find a Labelling that maximize the conditional probability
4
Inference
Message Passing
Move-Making
• T. Minka. Expectation Propagation for
Approximate Bayesian Inference, UAI,
2001
• Murphy. Loopy Belief Propagation: An
Empirical Study, UAI, 1999
• Jordan et.al. An Introduction to Variational
Methods for Graphical Models, ML-1999
• J. Yedidia et al. Generalized Belief
Propagation, NIPS, 2001
• Besag. On the Statistical Analysis of Dirty
Pictures, JRSS, 1986
• Boykov et al. Fast Approximate Energy
Minimization via Graph Cuts, PAMI 2001
• Komodakis et al. Fast Approximate
Optimal Solutions for Single and Dynamic
MRFs, CVPR, 2007
• Lempitsky et al. Fusion Moves for Markov
Random Field Optimization, PAMI, 2010
Convex Relaxations
Other Algorithms
• Chekuri et al. Approximation Algorithms for
Metric Labelling, SODA, 2001
• M. Goemans et al. Improved Approximate
Algorithms for Maximum-Cut, JACM, 1995
• M. Muramatsu et al. A New SOCP
Relaxation for Max-Cut, JORJ, 2003
• RaviKumar et al. QP Relaxation for Metric
Labelling, ICML 2006
• K. Alahari et.al. Dynamic Hybrid
Algorithms for MAP Inference, PAMI
2010
• P. Kohli et al. On Partial Optimality in
Multilabel MRFs, ICML, 2008
• C. Rother et al. Optimizing Binary MRFs
via Extended Roof Duality, CVPR, 2007
5
Inference
Message Passing
• T. Minka. Expectation Propagation for
Approximate Bayesian Inference, UAI, 2001
• Murphy. Loopy Belief Propagation: An
Empirical Study, UAI, 1999
• Jordan et.al. An Introduction to Variational
Methods for Graphical Models, ML-99
• J. Yedidia et al. Generalized Belief
Propagation, NIPS, 2001
• Variational message passing algorithm
• We focus on mean-field based inference
6
Mean-field methods
• Mean-fields methods (Jordan et.al., 1999)
• Intractable inference with distribution
• Approximate distribution
from tractable family
P
7
Variational Inference
• Minimize the KL-divergence between Q and P
8
Variational Inference
• Minimize the KL-divergence between Q and P
9
Variational Inference
• Minimize the KL-divergence between Q and P
10
Variational Inference
• Minimize the KL-divergence between Q and P
11
Markov Random Field (MRF)
• Graph:
• A simple MRF
Product of potentials defined over cliques
12
Markov Random Field (MRF)
• Graph:
• In general
Un-normalized part
13
Energy minimization
• Potential and energy
14
Variational Inference
Entropy
of Q
Expectation of cost
under Q distribution
15
Naïve Mean Field
• Family
: assume all variables are independent
16
Variational Inference
• Shannon’s entropy decomposes
17
Variational Inference
• Stationary point solution
• Marginal update in mean-field
• Normalizing constant:
18
Variational Inference
• Marginal for variable i taking label l
19
Variational Inference
• Marginal for variable i taking label l
• An assignment of all variables in clique c
20
Variational Inference
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
21
Variational Inference
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
• Marginal distribution of all variables in c apart from x_i
22
Variational Inference
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
• Marginal distribution of all variables in c apart from x_i
• Summation evaluates the expected value of cost over
distribution Q given that x_i takes label l
23
Simple Illustration
Naïve
mean-field
approximation
24
Mean-field algorithm
• Iterative algorithm
• Iterate till convergence
• Update marginals of each variable in each
iteration
25
Q distribution
26
Max posterior marginal (MPM)
• MPM with approximate distribution:
• MAP solution / most likely solution
• Empirically achieves very high accuracy:
27
Structured Mean Field
• Naïve mean field can lead to poor solution
• Structured (higher order) mean-field
28
How to make a mean-field algorithm
• Pick a model
• Unary, pairwise, higher order cliques
• Define a cost
• Potts, linear truncated, robust PN
• Calculate the marginal
• Calculate the expectation of cost defined
29
How to make a mean-field algorithm
• Use this plug-in strategy in many different models
• Grid pairwise CRF
• Dense pairwise CRF
• Higher order model
• Co-occurrence model
• Latent variable model
• Product label space
30