DESCRIPTORS
(DESCRIPTION OF INTEREST REGIONS WITH LOCAL BINARY PATTERNS)
Yu-Lin Cheng
(03/07/2011)
OUTLINE

Scale Invariant Feature Transform (SIFT) Descriptor

Local Binary Pattern (LBP) Descriptor


Center-Symmetric LBP (CS-LBP) Descriptor
Histogram of Oriented Gradients (HOG) Descriptor
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
)
SIFT Algorithm:
descriptor
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
)
Scale-space Extrema Detection:

Stable feature points ----- (scale invariant)


Principle:
 A local maximum over scales by using combination of normalized
derivatives can be treated as a characteristic point of local structure
Use LoG to find maximum
bad
scale
Good !
scale
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Scale-space Extrema Detection:
 Use
DoG instead of LoG ---- (computational efficiency)
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Scale-space Extrema Detection:
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Scale-space Extrema Detection:

Local extrema detection:

Compare to 26 neighbors

Keep the same keypoint in all scale
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Scale-space Extrema Detection:

Reject points with low contrast
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
)
Accurate keypoints localization:
 Quadratic function to interpolate the location of maximum

Eliminate edge response:
r: threshold,
H: Hessian matrix
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
)
Orientation Assignment:

Assign a consistent orientation to achieve orientation invariant

Method:
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
)
Orientation Assignment:

Calculate gradient magnitude and direction of neighboring pixels
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Orientation Assignment:

Calculate weighted orientation histogram
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Orientation Assignment:

Calculate weighted orientation histogram
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Orientation Assignment:

Calculate weighted orientation histogram
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
Keypoints Descriptor:

Empirical result:
Cell size: 4×4 pixels
 Block size: 4×4 cells
 Dimension: 4×4 (cells) × 8 (bins) = 128

Weighted magnitude
)
SIFT(S

CALE INVARIANT FEATURE TRANSFORM
)
Keypoints Descriptor:

Avoid all boundary effect


Use trilinear interpolation
Normalization: (illumination invariant)
Normalize to unit length
 Threshlod the maximum value to 0.2
 Match the magnitudes for large gradients is no longer important
 Renormalize to unit length

LBP(L
OCAL BINARY PATTERN)

A powerful mean of texture description

LBP operator:

Standard LBP:

Illustration:
LBP(L
OCAL BINARY PATTERN)

Example:

Parameters:
P : Number of neighboring pixels
 R : Radius

LTP(L

OCAL TRINARY PATTERN)
LTP operator:


t : threshold
Illustration:
CS-LBP(C

CS-LBP operator:

Illustration:
ENTER-SYMMETRIC LOCAL BINARY PATTERN)
CS-LBP DESCRIPTOR

Flow diagram:
CS-LBP DESCRIPTOR

Interest Region Detection:

Detectors:
1. Hessian-Affine (blob-like structure)
 2. Harris-Affine (corner-like structure)
 3. Hessian-Laplace (scale-invariant version)
 4. Harris-Laplace (scale-invariant version)

41×41
CS-LBP DESCRIPTOR

Feature Extraction:

CS-LBP operator:

Parameters:
 R: radius


N: number of neighboring pixels


N = 6, 8
T: threshold


R = 1, 2
T = 0.2
Descriptor Construction:
Location grids
 3×3 cells/4×4 cells
 Avoid boundary effects:
 Using ‘bilinear interpolation’

41×41
CS-LBP DESCRIPTOR

Descriptor Normalization: (illumination invariant)
Normalize to unit length
 Thresholding
 Renormalize to unit length

24 × 4 × 4 = 256
COMPARISON(SIFT

V.S
. CS-LBP)
Assumption:

Computations cannot be reused from detection algorithm

Comparison:

Conclusion:

Computational efficiency and better performance than SIFT
HOG(H
ISTOGRAM OF
ORIENTED GRADIENTS)
HOG(H

ISTOGRAM OF
Gradient Computation:
ORIENTED GRADIENTS)
HOG(H

ISTOGRAM OF
Gradient Computation:
ORIENTED GRADIENTS)
HOG(H

ISTOGRAM OF
ORIENTED GRADIENTS)
Spatial/Orientation Binning:

Weighted votes


Avoid aliasing


Function of magnitude
Interpolation
Parameters:
Number of orientation bins
 Cell size
 Block size

Cell
Block
HOG(H

ISTOGRAM OF
ORIENTED GRADIENTS)
Spatial/Orientation Binning:

Parameters:
Number of orientation bins: 9 bins/18bins
 Cell size: 8×8 pixels
 Block size: 2×2 cells

HOG(H

ISTOGRAM OF
ORIENTED GRADIENTS)
Normalization:

Group cells to larger blocks and normalize each block separately
(illumination invariant)

Normalization Schemes:
HOG(H

ISTOGRAM OF
Normalization:

Normalization Schemes:
ORIENTED GRADIENTS)
COMPARISON(SIFT

Comparison:
V.S
. HOG)
HOG VARIATION

‘Object Detection with Discriminatively Trained Part Based Models’

Pixel-Level Feature Maps:
Use [-1, 0, 1] to calculate gradient
 Contrast sensitive(B1), Contrast insensitive(B2)

,(p = 9)

Quantize into orientation bins
r: gradient magnitude
HOG VARIATION

Spatial Aggregation:
Rectangular cell: 8×8 pixels
 Cell-based feature map:



Avoid aliasing:


Reduce the size of feature map
Bilinear interpolation
Normalization:
HOG VARIATION

Truncation:
maximum 0.2


No renormalization
Dimension:

9 bins × 4 different normalization = 36 (contrast insensitive)
HOG VARIATION

PCA analysis:

Top 11 eigenvectors captures most of information of HOG
HOG VARIATION

PCA analysis:

Top eigenvectors lie (approximately) in a linear subspace

13-dimensional features:
Project 36-dimensional HOG feature into uk, vk
 Projection into uk : sum over 4 normalization over fixed orientation
 Projection into vk : sum over 9 orientation over fixed normalization

HOG VARIATION

For Contrast Insensitive(B2):


For Contrast Sensitive(B1):


9 bins × 4 different normalization = 36 (contrast insensitive)
18 bins × 4 different normalization = 72 (contrast insensitive)
Reduce to (18 + 9) + 4 = 31 dimension
REFERENCE

“Description of Interest Regions With Local Binary Patterns”, Pattern
Regonization ’09 Marko Heikkilä







http://www.tele.ucl.ac.be/~devlees/ref_ELEC2885/projects/RoIdescriptionLBPpr-accepted.pdf
“Effective Pedestrian Detection Using Center-symmetric Local
Binary/Trinary Patterns”, Youngbin Zheng
“Scale-space Theory” Tony Lindeberg
“Histogram of Oriented Gradients for Human Detection”, CVPR ‘05
Navneet Dalal
“Finding People in Images and Videos”, Navneet Dalal
“Feature matching” Yung-Yu Chuang
“Scale & Affine Invariant Interest Point Detectors”, IJCV ’04 Krystian
Mikolajczyk
REFERENCE


“Object Detection with Discriminatively Trained Part Based Models”
“Distinctive Image Features from Scale-Invariant Keypoints”, IJCV ’04
David G. Lowe

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.157.3843&rep=rep1&
type=pdf