Distinctive Image Features
from Scale-Invariant Keypoints
Note
이 ppt 에 있는 자료들은 Lowe 의 논문들이나 google에서 찾은 feature 관련
paper, ppt 파일들을 참조해서 개인용으로 만든 것입니다.
레퍼런스들을 명시해 놓는 것이 당연하지만 많은 자료들을 짜집기해서 레퍼런스
들을 명시하지 못했습니다.
이 점을 감안해서 혹시 그림이나 사진 등을 참고할 생각이시라면 인터넷을 통해
해당하는 paper, ppt, web-page 등을 찾으셔서 레퍼런스로 활용해주시기 바랍니
다.
( 레퍼런스 자료들의 제작자 분들께 매우 죄송하다는 말을 전합니다 ㅠㅠ )
Contents
 Introduction
 SIFT Overview
• Scale-space extrema detection
• Keypoint localization
• Orientation assignment
• Keypoint descriptor
 Test and Result
 Conclusion
Introduction
 Ideal Interest Points / Regions
<Image Features>
Introduction
 Harris Corner Detector
<Feature Detection>
• Rotation invariance
• Partial invariance to affine change
Introduction

Harris Corner Detector
edge
corner
• Non-invariance to scale change
Introduction
 SIFT
The Scale Invariant Feature Transform
Choosing features that are invariant to image scaling and rotation
Partially invariant to changes in illumination and 3D camera viewpoint
Well localized in both spatial and frequency domain
- Resistant to noise, clutter, and occlusion
• Features are highly distinctive, matched with high probability against large number
of features
• See……
•
•
•
•
 Application
•
•
•
•
•
•
Object recognition
Automatic mosaic
Tracking
Robot localization
3D scene modeling
Panoramas
Introduction
<Object Recognition>
Introduction
<Automatic Mosaic>
SIFT overview
Detector
1.
2.
Scale-space extrema detection
Keypoint localization and filtering
Descriptor
3.
4.
Orientation assignment
Keypoint descriptor
SIFT overview
Detector
1.
2.
Scale-space extrema detection
Keypoint localization and filtering
Descriptor
3.
4.
Orientation assignment
Keypoint descriptor
1. Scale-space extrema detection
 A “good” function for scale detection has one stable sharp peak
Good !
f
bad
bad
region size
 L or DOG(Difference of Gaussians) kernel is a matching filter
1. Scale-space extrema detection
 DOG(Difference of Gaussians)
• Construct scale-space
• Take differences
Downsample
Convolve with
Gaussian
1. Scale-space extrema detection
 For example
• Construct scale-space
1. Scale-space extrema detection
 For example
• Take differences
1. Scale-space extrema detection
 Scale-space extrema
• Compare a pixel with its 26 neighbors in 3*3 regions at the current and adjacent
scales
1. Scale-space extrema detection
 For example
• Scale-space extrema
SIFT overview
Detector
1.
2.
Scale-space extrema detection
Keypoint localization and filtering
Descriptor
3.
4.
Orientation assignment
Keypoint descriptor
2. Keypoint localization and filtering
 Reject points with bad contrast
• DOG smaller than 0.03 (image values in [0, 1])
2. Keypoint localization and filtering
 Reject points with strong edge response in one direction only
• To check if ratio of principal curvature is below some threshold, r
SIFT overview
Detector
1.
2.
Scale-space extrema detection
Keypoint localization and filtering
Descriptor
3.
4.
Orientation assignment
Keypoint descriptor
3. Orientation assignment
 Descriptor computed relative to keypoint’s orientation achieves
rotation invariance
 Let, for a keypointm L is the image with the closest scale
• Compute the orientation histogram
- within a region around the keypoint (16 Ⅹ 16)
• Compute gradient magnitude and orientation using finite differences
 L( x  1, y)  L( x  1, y) 
GradientVector  

 L( x, y  1)  L( x, y  1) 
3. Orientation assignment
3. Orientation assignment
3. Orientation assignment
3. Orientation assignment
3. Orientation assignment
SIFT overview
Detector
1.
2.
Scale-space extrema detection
Keypoint localization and filtering
Descriptor
3.
4.
Orientation assignment
Keypoint descriptor
Keypoint descriptor
 The computation of the keypoint descriptor
• A set of keypoints are obtained from each reference image
• Each such keypoint has a graphical descriptor – which is a 128 component vector
(4Ⅹ4Ⅹ8) ← keypoint descriptor’s complexity
Keypoint descriptor
 Storing
Keypoint descriptor
 Matching
• Test image gives a new set of (keypoint, vector) pair
• Find the nearest (top 2) descriptors in database
 Acceptance of a match
• Storage using k-d trees
- Use the Best-Bin-First(BBF) algorithm
• Ratio of distance to first nearest descriptor to that of second < threshold (0.8)
Test and Result
<Recognition under occlusion>
<3D object recognition>
Test and Result
<Recognition under differing illumination>
Test and Result
<Location recognition>
Test and Result
 SIFT didn’t work
• Large illumination change
Test and Result
 SIFT didn’t work
• Non-rigid deformation
Conclusion
 SIFT
• A novel method for detecting interest points
• Invariant to
- image scaling
- translation
- rotation
• Robust matching across substantial range of
- distortion
- change in 3D view point
- addition of noise
- change in illumination
 SIFT extensions
•
•
•
•
PCA-SIFT
SURF
Approx SIFT
GPU implementation