Outline
• S. C. Zhu, X. Liu, and Y. Wu, “Exploring
Texture Ensembles by Efficient Markov
Chain Monte Carlo”, IEEE Transactions On
Pattern Analysis And Machine Intelligence,
Vol. 22, No. 6, pp. 554-569, 2000
Limitations of Linear Representations
• Linear representations do not depend on the
spatial relationships among pixels
– For example, if we shuffle the pixels and
corresponding representations, then the
classification results will remain the same
• But in images spatial relationships are
important
May 29, 2016
Computer Vision
2
Image Features
May 29, 2016
Computer Vision
3
Spectral Representation of Images
• Spectral histogram
– Given a bank of filters F(a), a = 1, …, K, a
spectral histogram is defined as the marginal
distribution of filter responses
I(a ) (v)  F (a ) * I(v)
H
(a )
I
1
(a )
( z) 
δ
(
z

I
(v))

|I| v
H I  ( H I(1) , H I( 2) ,, H I( K ) )
May 29, 2016
Computer Vision
4
Spectral Representation of Images - continued
• An example of spectral histogram
May 29, 2016
Computer Vision
5
Image Modeling - continued
• Given observed feature statistics {H(a)obs},
we associate an energy with any image I as
K
(a )
Ε (I)   |H I(a ) ( z )  H obs
( z ) |p
a 1 z
• Then the corresponding Gibbs distribution is
q(I) 
1
E (I)
exp( 
)
ZT
T
– The q(I) can be sampled using a Gibbs sampler
or other Markov chain Monte-Carlo algorithms
May 29, 2016
Computer Vision
6
Image Modeling - continued
Image Synthesis Algorithm
• Compute {Hobs} from an observed texture image
• Initialize Isyn as any image, and T as T0
• Repeat
Randomly pick a pixel v in Isyn
Calculate the conditional probability q(Isyn(v)| Isyn(-v))
Choose new Isyn(v) under q(Isyn(v)| Isyn(-v))
Reduce T gradually
• Until E(I) < e
May 29, 2016
Computer Vision
7
A Texture Synthesis Example
Observed image
May 29, 2016
Initial synthesized image
Computer Vision
8
A Texture Synthesis Example
Temperature
Image patch
Energy
Conditional probability
• May
Energy
and conditionalComputer
probability
of the marked
29, 2016
Vision
pixel
9
A Texture Synthesis Example - continued
• A white noise image was
transformed to a perceptually
similar texture by matching
the spectral histogram
May 29, 2016
Computer Vision
10
Average spectral histogram error
A Texture Synthesis Example - continued
• Synthesized images from different initial conditions
May 29, 2016
Computer Vision
11
Texture Synthesis Examples - continued
Observed image
Synthesized image
• A random texture image
May 29, 2016
Computer Vision
12
Texture Synthesis Examples - continued
Observed image
Synthesized image
• An image with periodic structures
May 29, 2016
Computer Vision
13
Texture Synthesis Examples - continued
Mud image
Synthesized image
• A mud image with some animal foot prints
May 29, 2016
Computer Vision
14
Texture Synthesis Examples - continued
Observed image
Synthesized image
• A random texture image with elements
May 29, 2016
Computer Vision
15
Texture Synthesis Examples - continued
Observed image
Synthesized image
• An image consisting of two regions
– Note that wrap-around boundary conditions were
used
May 29, 2016
Computer Vision
16
Texture Synthesis Examples - continued
Synthesized image
Original cheetah skin patch
• A cheetah skin image
May 29, 2016
Computer Vision
17
Texture Synthesis Examples - continued
Observed image
Synthesized image
• An image consisting of circles
May 29, 2016
Computer Vision
18
Texture Synthesis Examples - continued
Observed image
Synthesized image
• An image consisting of crosses
May 29, 2016
Computer Vision
19
Texture Synthesis Examples - continued
Observed image
Synthesized image
• A pattern with long-range structures
May 29, 2016
Computer Vision
20
Object Synthesis Examples
• As in texture synthesis, we start from a random image
• In addition, similar object images are used as boundary conditions in that the
corresponding pixel values are not updated during sampling process
May 29, 2016
Computer Vision
21
Object Synthesis Examples - continued
May 29, 2016
Computer Vision
22
Object Synthesis Examples - continued
May 29, 2016
Computer Vision
23
Linear Transformations of Images
• Linear transformations include
–
–
–
–
–
Principal component analysis
Independent component analysis
Fisher discriminant analysis
Optimal component analysis
They have been widely used to reduce dimension of images
for appearance-based recognition applications
• Each image is viewed as a long vector and projected
into a set of bases that have certain properties
May 29, 2016
Computer Vision
24
Principal Component Analysis
• Defined with respect to a training set such
that the average reconstruction error is
minimized
May 29, 2016
Computer Vision
25
Principal Component Analysis - continued
May 29, 2016
Computer Vision
26
Eigen Values of 400 Eigen Vectors
May 29, 2016
Computer Vision
27
Principal Component Analysis - continued
Original Image
May 29, 2016
Reconstructed
using 50 PCs
Computer Vision
Reconstructed
using 200 PCs
28
Principal Component Analysis - continued
• Is PCA representation a good representation
of images for recognition in that images that
have similar principal representations are
similar?
– Image generation through sampling
– Roughly speaking, we try to generate images that
have the given coefficients along PCs
May 29, 2016
Computer Vision
29
Principal Component Analysis - continued
May 29, 2016
Computer Vision
30
Principal Component Analysis - continued
May 29, 2016
Computer Vision
31
Difference Between Reconstruction and Sampling
Reconstruction is not sufficient to show the adequacy of a representation and
sampling from the set of images with same representation is more informational
May 29, 2016
Computer Vision
32
Object Recognition Experiments
• We compare linear methods in the methods
including
–
–
–
–
Principal component analysis (PCA)
Independent component analysis (ICA)
Fisher discriminant analysis (FDA)
Random component analysis (RCA)
• For fun and to show the actual gain of using different bases is
relatively small
• Corresponding linear methods in the spectral
histogram space including
– SPCA, SICA, SFDA, and SRCA
May 29, 2016
Computer Vision
33
COIL Dataset
May 29, 2016
Computer Vision
34
3D Recognition Results
May 29, 2016
Computer Vision
35
Experimental Results - continued
• To further demonstrate the effectiveness of
our method for different types of images, we
create a dataset of combining the texture
dataset, face dataset, and COIL dataset,
resulting in a dataset of 180 categories with
10160 images in total
May 29, 2016
Computer Vision
36
Linear Subspaces of Spectral Representation
May 29, 2016
Computer Vision
37
Experimental Results - continued
• Combined dataset – continued
– Not only the recognition rate is very good, but
also it is very reliable and robust, as the average
entropy of the p0(i|I) is 0.60 bit (The
corresponding uniform distribution’s entropy is
7.49 bits)
May 29, 2016
Computer Vision
38
Experimental Results - continued
• Combined dataset – continued
– Not only the recognition rate is very good, but also it is
very reliable and robust, as the average entropy of the
p0(i|I) is 0.60 bit (The corresponding uniform
distribution’s entropy is 7.49 bits)
Entropy=6.78bits
Entropy=0.60 bit
May 29, 2016
Computer Vision
39