Outline
• Texture modeling - continued
– Julesz ensemble
FRAME Model – review
• FRAME model
– Filtering, random field, and maximum entropy
– A well-defined mathematical model for textures
by combining filtering and random field models
– Maximum entropy is used when constructing the
probability distribution on the image space
– Minimum entropy is used when selecting filters
from a large bank of filters
– Together this is called min-max entropy principle
5/29/2016
Visual Perception Modeling
2
FRAME Model – review
• Maximum Entropy Distribution
– Given the expectations of some functions, the
maximum entropy solution for p(x) is
– where
1
 N

p( x; ) 
exp   n n ( x)
Z ()
 n 1

 N

  (1 , 2 ,, N ) and Z( )   exp -  n n ( x)dx
 n 1

5/29/2016
Visual Perception Modeling
3
FRAME Model – review
• Maximum Entropy – continued
–   (1 , 2 ,, N ) are determined by the
constraints
– Gradient ascend to maximize log p ( x;  )
dn d log p( x; )

 E p ( x; ) [ n ( x)]   n
dt
dn
n  1, 2,  , N
5/29/2016
Visual Perception Modeling
4
Julesz Ensemble
• The original texture modeling question
– What features and statistics are characteristics of
a texture pattern, so that texture pairs that share
the same features and statistics cannot be told
apart by pre-attentive human visual perception?
--- Julesz, 1962
5/29/2016
Visual Perception Modeling
5
Summary of Existing Texture Features
5/29/2016
Visual Perception Modeling
6
Existing Feature Statistics
5/29/2016
Visual Perception Modeling
7
Most General Feature Statistics
5/29/2016
Visual Perception Modeling
8
Julesz Ensemble – cont.
• Definition
– Given a set of normalized statistics on lattice 
h {h( ) :,   1,2, K }
a Julesz ensemble W(h) is the limit of the
following set as   Z2 and H  {h} under
some boundary conditions
W ( H )  {I : h( I )  H}
5/29/2016
Visual Perception Modeling
9
Julesz Ensemble – cont.
• Feature selection
– A feature can be selected from a large set of
features through information gain, or the
decrease in entropy
5/29/2016
Visual Perception Modeling
10
Julesz Ensemble – cont.
5/29/2016
Visual Perception Modeling
11
Julesz Ensemble – cont.
• Sampling the Julesz ensemble
– In the Julesz ensemble, a texture type is defined
as all the images sharing the observed statistics
and features
• It is an inverse problem in order to generate texture
images or verify the statistics
– The problem is again the dimensionality
• If the image size is 256x256 and each pixel can have 8
values, there are 865536 different images
– Markov chain Monte-Carlo algorithms
5/29/2016
Visual Perception Modeling
12
Julesz Ensemble – cont.
• Given observed feature statistics {H()obs}, we
associate an energy with any image I as
K
( )
Ε (I)   |H I( ) ( z )  H obs
( z) | p
 1 z
• Then the corresponding Gibbs distribution is
q (I) 
1
E (I)
exp( 
)
ZT
T
– The q(I) can be sampled using a Gibbs sampler or other
Markov chain Monte-Carlo algorithms
5/29/2016
Visual Perception Modeling
13
Image Synthesis Algorithm
• Compute {Hobs} from an observed texture image
• Initialize Isyn as any image, and T as T0
• Repeat
Randomly pick a pixel v in Isyn
Calculate the conditional probability q(Isyn(v)|
Isyn(-v))
Choose new Isyn(v) under q(Isyn(v)| Isyn(-v))
Reduce T gradually
• Until E(I) < e
5/29/2016
Visual Perception Modeling
14
A Texture Synthesis Example
Observed image
5/29/2016
Initial synthesized image
Visual Perception Modeling
15
A Texture Synthesis Example
Temperature
5/29/2016
Image patch
Energy
Visual Perception Modeling
Conditional probability
16
A Texture Synthesis Example - continued
Average spectral histogram error
5/29/2016
Visual Perception Modeling
17
Texture Synthesis Examples - continued
Observed image
5/29/2016
Synthesized image
Visual Perception Modeling
18
Texture Synthesis Examples - continued
Observed image
5/29/2016
Synthesized image
Visual Perception Modeling
19
Texture Synthesis Examples - continued
Mud image
Synthesized image
5/29/2016
Visual Perception Modeling
20
Texture Synthesis Examples - continued
Observed image
Synthesized image
5/29/2016
Visual Perception Modeling
21
Texture Synthesis Examples - continued
Observed image
5/29/2016
Synthesized image
Visual Perception Modeling
22
Texture Synthesis Examples - continued
Synthesized image
Original cheetah skin patch
5/29/2016
Visual Perception Modeling
23
Texture Synthesis Examples - continued
Observed image
5/29/2016
Synthesized image
Visual Perception Modeling
24
Texture Synthesis Examples - continued
Observed image
5/29/2016
Synthesized image
Visual Perception Modeling
25
Texture Synthesis Examples - continued
Observed image
Synthesized image
5/29/2016
Visual Perception Modeling
26
An Synthesis Example for Fun
5/29/2016
Visual Perception Modeling
27
Comparison with Texture Synthesis Method - continued
• An example from Heeger and Bergen’s
algorithm
Cross image
5/29/2016
Heeger and Bergen’s
Visual Perception Modeling
Our result
28
Julesz Ensemble – cont.
• Remarks
– The results shown here are based on histograms
of filter responses
– However, the Julesz ensemble applies to any
features/statistics of your choice
– You can also define Julesz ensemble for images
other than textures
5/29/2016
Visual Perception Modeling
29
Julesz Ensemble – cont.
• Applications
– This essentially provides a framework to
systematically verify the sufficiency of chosen
features/statistics
• Normally, features/statistics are evaluated empirically.
In other words, features are evaluated on a limited
number of images
5/29/2016
Visual Perception Modeling
30