Person Re-identification by Matching
Compositional Template with Cluster
Sampling
Yuanlu Xu
Advisor: Prof. Liang Lin
merayxu@gmail.com
Problem
Person Re-identification
Identifying The Same Person Under Different Cameras
Basic Assumption:
1. Face is unreliable due to view, low resolution and noises.
2. People's clothes should remain consistent.
Difficulty
Large Intra-class Variations
Pose/View Variation
Illumination Change
Occlusion
Problem
Multiple Setting
Query Person
Scene
S vs. S
M vs. S
Search
Representation
Multiple-Instance Compositional
Template (MICT)
1. Body into 6 parts, limbs further into 2
symmetric parts.
2. Leaf nodes contain multiple instances.
3. Contextual relations between parts:
 kinematics
 symmetry.
Problem Formulation
Matching-based Formulation
Given the template, the problem is
formulated as
 Selecting an instance for each part.
 Finding the matched part in target.
…
Problem Formulation
13
12
11
13
11
21
22
12
24
23
21
22
24
23
33
31
31
32
32
42
34
41
41
42
43
(a) Query Person
(32,34)
44
(b) Test Scene
(41,42)
(13,13)
(41,41)
(11,12)
(32,33)
(12,11)
(31,33)
(24,24)
(32,32)
(24,23)
(31,32)
(24,22)
(31,31)
(24,21)
Candidacy Graph:
 Vertices – possible matching
pairs
Problem Formulation
13
12
11
13
11
21
22
12
24
23
21
22
24
23
33
31
31
32
32
42
34
41
41
42
43
(a) Query Person
(32,34)
44
(b) Test Scene
(41,42)
(13,13)
(41,41)
(11,12)
(32,33)
(12,11)
(31,33)
(24,24)
(32,32)
(24,23)
(31,32)
(24,22)
(31,31)
(24,21)
Solving the problem:
 Labeling vertices in the graph
(selecting matching pairs)
 NP hard – incorporating graph
edges
Problem Formulation
13
12
11
13
11
21
22
12
24
23
21
22
24
23
33
31
31
32
32
42
34
41
41
42
43
(a) Query Person
(32,34)
44
(b) Test Scene
(41,42)
(13,13)
(41,41)
(11,12)
(32,33)
(12,11)
(31,33)
(24,24)
(32,32)
(24,23)
(31,32)
(24,22)
(31,31)
(24,21)
Compatible Edges:
 Encouraging matching pairs to
activate together in matching
 Defined by contextual
constraints
Problem Formulation
13
12
11
13
11
21
22
12
24
23
21
22
24
23
33
31
31
32
32
42
34
41
41
42
43
(a) Query Person
(32,34)
44
(b) Test Scene
(41,42)
(13,13)
(41,41)
(11,12)
(32,33)
(12,11)
(31,33)
(24,24)
(32,32)
(24,23)
(31,32)
(24,22)
(31,31)
(24,21)
Competitive Edges:
 Depressing conflicting matching
pairs being selected at the same
time
 Defined by matching constraints
Inference
State A
(32,34)
Clusters
(41,42)
(13,13)
(41,41)
(11,12)
(32,33)
Cluster1
(12,11)
(31,33)
(24,24)
(32,32)
(24,23)
(31,32)
(24,22)
(31,31)
(24,21)
Cluster2
State B
(32,34)
Re-identification
Re-identification
Clusters
(41,42)
(13,13)
(41,41)
(11,12)
(32,33)
Cluster1
(12,11)
(31,33)
(24,24)
(32,32)
(24,23)
(31,32)
(24,22)
(31,31)
(24,21)
Cluster2
Using Cluster Sampling [1] for inference:
1. Sampling edges in candidacy graph to generate clusters.
2. Randomly selecting/deselecting the clusters.
3. Decide whether to accept the new state.
[1] J. Porway et al., “C4: Exploring multiple solutions in
graphical models by cluster sampling”, TPAMI 2011.
Dataset
VIPeR Dataset:
1. Classic ReID dataset
2. Well-segmented people, limited
pose/view
3. Heavy illumination changes, lack
occlusion
D. Gray et al., "Viewpoint Invariant Pedestrian Recognition
with an Ensemble of Localized Features”, ECCV 2008.
Dataset
Query Instance
Video Shot
Target Individual
EPFL Dataset:
1. Cross-camera tracking dataset
2. Few people, shot scene provided,
various pose/view
3. Little illumination changes, limited
occlusions
F. Fleuret et al., "Multiple Object Tracking using KShortest Paths Optimization”, TPAMI 2011.
Dataset
Query Instance
CAMPUS-Human Dataset:
1. Camera and annotate by us
2. Many people, shot scene provided,
various pose/view
3. Limited illumination changes, heavy
occlusions
Video Shot
Target People
Result
Setting 1:
Re-identify people in
segmented images,
i.e. targets already
localized.
Result
Setting 2:
Re-identify people from scene shots
without provided segmentations.
Result
Component Analysis
Evaluating feature and constraints effectiveness
Conclusion
1. A solution for a new surveillance problem.
2. A person-based model, a graph-matching-based formulation, a more
complete database for evaluation.
3. Exploring robust and flexible person models [1], efficient search method [2]
in future.
[1] J. B. Rothrock et al., “Integrating Grammar and Segmentation for
Human Pose Estimation”, CVPR 2013.
[2] J. Uijlings et al., “Selective Search for Object Recognition”, IJCV 2013.
Published Papers
1. Yuanlu Xu, Liang Lin, Wei-Shi Zheng, Xiaobai Liu. “Human Re-identification
by Matching Compositional Template with Cluster Sampling”. ICCV 2013.
2. Liang Lin, Yuanlu Xu, Xiaodan Liang, Jian-Huang Lai. “Complex Background
Subtraction by Pursuing Dynamic Spatio-temporal Manifolds”. IEEE TIP 2014,
under revision.
3. Yuanlu Xu, Bingpeng Ma, Rui Huang, Liang Lin. “Person Search in a Scene by
Jointly Modeling People Commonness and Person Uniqueness”. ACMMM
2014, submitted.
QUESTIONS?
Cluster Sampling
Generating a composite cluster
1. Given a candidacy graph and the current matching state 𝑀, we first separate graph
edges into two sets: set of inconsistent edges
and set of consistent edges in the other two cases.
2. Next we introduce a boolean variable
to indicate an edge is being turned
on or turned off. We turn off inconsistent edges deterministically and turn on every
consistent edge with its edge probability 𝑝𝑒 .
Cluster Sampling
Generating a composite cluster
3. Afterwards, we regard candidates connected by ”on” positive edges as a cluster 𝐶𝑙
and collect clusters connected by ”on” negative edges to generate a composite cluster
𝑉𝑐𝑐 .
Composite Cluster Sampling
Using Metropolis-Hastings method to achieve a reversible transition between two
states 𝑀 and 𝑀′, the acceptance rate of the transition is defined as
state transition probability ratio
posterior ratio
Composite Cluster Sampling
The state transition probability ratio is computed by
edges being turned off
around 𝑉𝑐𝑐 ,
Composite Cluster Sampling
Inference Algorithm