Multi-Attribute Spaces: Calibration for Attribute
Fusion and Similarity Search
Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult,
CVPR 2012
University of Oxford
5th December 2012
Attributes based image description
4-Legged
White
Male
Orange
Symmetric
Asian
Striped
Ionic columns
Beard
Furry
Classical
Smiling
Slide Courtesy: Neeraj Kumar
Attribute Classifiers
Attribute and Simile Classifiers for Face Verification
N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar
ICCV 2009
FaceTracer: A Search Engine for Large Collections of Images with Faces
N. Kumar, P. N. Belhumeur, and S. K. Nayar
ICCV 2009
Attributes Fusion
FaceTracer: “smiling asian men with glasses”
Slide Courtesy: Neeraj Kumar
Score Normalization: Problem
• Necessary to prevent high confidence for one attribute
from dominating the results.
• Ideal normalization technique should,
1) Normalize scores to a uniform range say, [0,1]
2) Assign perceptual quality to the scores.
• Positive and negative distributions of different classifiers do
not necessarily follow same distribution.
• Fitting a Gaussian or any other distribution to scores
satisfies condition 1 but doesn’t satisfy condition 2.
Negative Scores Distributions
Positive Scores Distributions
Score Normalization: Solution
• Model distance between positive scores and the negative
scores .
• If we knew distribution of negative scores, we could do a
hypothesis test for each positive score using that distribution.
• Unfortunately, we don’t know anything about overall negative
distribution.
But, we know something about tail of the negative score
distribution.
Extreme Value Theory
• Central Limit Theorem:
• The “mean” of a sufficiently large iid random
variables will be distributed according to Normal
distribution
• Extreme Value Theory:
• The maximum of a sufficiently large iid random
variable will be distributed according to Gumbell,
Frechet or Weibull distribution.
• If the values are bounded from above and below, the
the values are distributed according to “Weibull”
distribution.
Weibull Distribution
• Weibull Distribution
PDF
CDF
k and λ are shape and location parameters respectively.
PDF
CDF
Extreme Value Theory: Application
Overall Negative Score Distribution
Tail
Maximum values of random variables
• Tail of negative scores can be seen as a collection of maxima of some random
variables.
• Hence it follows Weibull distribution according to Extreme Value Theory.
W-score normalization: Procedure
For any classifier,
• Fix the decision boundary on the scores
(Ideally this should be at score = 0 )
• Select maximum N (tail size) samples from
negative side of the boundary.
• Fit a Weibull Distribution to these tail scores.
• Renormalize scores using Cumulative Density Function
(CDF) of this Weibull distribution.
Results: Dataset
• “Labeled Faces In The Wild” dataset.
• About 13,000 images of 5000 celebrities.
• 75 different attribute classification scores available from
“Attribute and Simile Classifiers for Face Verification”. Kumar et al. ICCV 09.
Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained
Environments.
Results
Multi Attribute Fusion:
• Joint score can be computed as multiplication of individual
attribute probabilities.
• Attributes may not be independent.
• Low probability due to:
• bad classifier
• absence of images belonging to an attribute.
• Instead of product, authors propose use l1 norm of
probabilities as a fusion score.
Results
Similarity Search:
• Given an image and a set of attributes, find nearest images.
• Perceived difference between images in different ranges
might be similar.
• Distances between query attribute and its nearest neighbor
needs to be normalized.
• Normalize query attribute scores on query image.
• Get nearest neighbor distances.
• Fit Weibull distribution to distances.
Summary
• Provides way of normalizing scores intuitively.
• Provides way for combining attributes.
• Relies on finding the right threshold and tail size. Requires
fair bit of tuning.
Questions?