Random Swap EM
algorithm for GMM and
Image Segmentation
Qinpei Zhao, Ville Hautamäki, Ismo
Kärkkäinen, Pasi Fränti
Speech & Image Processing Unit
Department of Computer Science, University of Joensuu
Box 111, Fin-80101 Joensuu
FINLAND
zhao@cs.joensuu.fi
Outline
Background & Status
 RS-EM
 Application

Background: Mixture Model
Background: EM algorithm
EM algorithm -> {α, Θ}

E-step (Expectation):
Q(, (i 1) )  E[log p( X , Y | ) | X , (i 1) ]

M-step (Maximization):
(i )  arg max Q(, (i 1) )


Iterate E,M step until convergence
α- mixing coefficient
Θ- model parameters, eg. {μ,∑}
Local Maxima
Let’s describe it as mountain climbing……
2160m
3099m
600km
Initialization Effect
Initialization and Result(1)
Initialization and Result(2)
Sub-optimal Example
The situation of local maxima trap
Status





Standard EM for Mixture Models(1977)
Deterministic Annealing EM (DAEM) (1998)
Split-Merge EM (SMEM) (2000)
Greedy EM (2002)
RS-EM coming…
Outline
Background & Status
 RS-EM (Random Swap)
 Application

RSEM: Motivations





Random manner
Prevent from staying near the unstable or
hyperbolic fixed points of EM.
Prevent from its stable fixed points corresponding
to insignificant local maxima of the likelihood
function
Avoid the slow convergence of EM algorithm
Less sensitive to its initialization
Formulas

SMEM

Greedy EM

RSEM
Random Swap EM
Comparisons(1)
Comparisons(2)
Q1
Q2
S1
S4
Outline
Background & Status
 RS-EM
 Application

Application




Image Segmentation
Color Quantization
Image Retrieval
……
Conclusion

Introduce Randomization into algorithm
Performs better
Without heavy time complexity
Wider applications

Thanks!☺


