PERSON IDENTIFICATION IN A GROUP
PHOTOGRAPH USING SURF FEATURES
Jyoti Dalal
Mahendra Singh Meena
Prof Priti Singh
M.Tech ECE
Amity University, Haryana, India
Assistant Professor ECE
Amity University, Haryana, India
Professor
Amity University, Haryana, India
itsme.jyotidalal@gmail.com
msmeena@ggn.amity.edu
psingh@ggn.amity.edu
ABSTRACT
This paper describes an efficient face identification technique
in a group photograph. In this paper SURF i.e., Speeded Up
Robust Features technique is used in order to identify a person
in a group photograph. First of all the SURF features are
detected. 100 feature points from test image and 300 from the
group photograph were selected. The feature points were then
extracted and finally were matched. If a number of features
matches then the system identifies the person in the group
photograph and a bounding box was made around the
identified person. The test image taken was the cropped face
of a person from the group photograph. Our work was under
illumination variation. Histogram equalization was done in
order to remove the illumination variation.
Keywords
SURF Features, Histogram Equalization, SIFT Features
1. INTRODUCTION
SURF is comprised of a feature detector based on a Gaussian
second derivative mask, and a feature descriptor that relies on
local Haar wavelet responses. The core concept of feature
comparison is to find nearest neighbors for a numerical
vector. The SURF detector focuses its attention on blob‐like
structures in the image. These structures can be found at
corners of objects, but also at locations where the reflection of
light on specular surfaces is maximal (i.e. light speckles). To
describe each feature, SURF summarizes the pixel
information within a local neighborhood. The first step is
determining an orientation for each feature, by convolving
pixels in its neighborhood with the horizontal and the vertical
Haar wavelet filters. By using intensity changes to
characterize orientation, the descriptor is able to describe
features in the same manner regardless of the specific
orientation of objects or of the camera. This rotational
invariance property allows SURF features to accurately
identify objects within images taken from different
perspectives.
2. LITERATURE SURVEY
Geng Du et al. 2009 [1] deals with using SURF features in
face recognition and gives the detailed comparisons with
SIFT features. Their experimental results show that the SURF
features perform only slightly better in recognition rate than
SIFT (Scale Invariant Feature Transform).
Philippe Dreuw et al. 2009 [2] investigated the usage of
SURF descriptors in comparison to SIFT descriptors for face
recognition. He showed that using their proposed grid-based
local feature extraction instead of an interest point detection
based extraction, SURF descriptors as well as SIFT
descriptors can be used for face recognition. The proposed
SURF descriptors were compared to SIFT descriptors.
Experimental results on the AR-Face and CMU-PIE database
using manually aligned faces, unaligned faces, and partially
occluded faces showed that the proposed approach was robust
and can outperform current generic approaches.
Bo-Feng Liu et al. 2012[3] presents a robust face recognition
scheme. Speeded-Up Robust Features algorithm is used for
extracting the feature vectors with scale invariance and pose
invariance from face images. Then PCA was introduced for
projecting the SURF feature vectors to the new feature space
as PCA-SURF local descriptors. Finally, the K-means
algorithm was applied to cluster feature points, and the local
similarity and global similarity were then combined to
classify the face images. Experimental results showed that the
performance of the proposed scheme was better than other
methods, and PCA-SURF feature was more robust than
original SURF and SIFT local descriptors against the
accessory, expression, and pose variations. It was an effective
face recognition method that used PCA-SURF features to
classify face images. PCA-based SURF local descriptors were
more robust than original SURF and SIFT local descriptors to
the accessory, expression, and pose variations.
Donghoon Kim et al. 2013[4] used SURF local descriptors
(Speeded Up Robust Features) to generate feature vectors and
use SVMs (Support Vector Machines) as classifiers. Their
system consisted of a two-layer hierarchy of SVMs
classifiers. On the first layer, a single classifier checked
whether feature vectors were from face images or not. On the
second layer, component labeling was operated using each
component classifier of eye, mouth, and nose. The approach
had the advantage about operating time because windows
scanning procedure was not needed. Finally, the system
performed the procedure to apply geometrical constraints to
labeled descriptors. They showed experimentally the
efficiency of their approach. They showed that the method has
high detection rate. The method was also able to localize face
components and can be applied to both the recognition
whether there were faces or not in video sequence and other
object detection tasks in computer vision.
Where dx and dy are the Haar Wavelet responses in horizontal
and vertical directions respectively.
3. METHODOLOGY
First we took some group photos. Then we applied Viola
Jones method for face detection. It detected and cropped the
faces and saved them in a folder. Now these cropped images
were used as the test images. There were total 30 test images.
Then we took an image as the test image and a group photo.
We used histogram equalization to increase the contrast level
of image. Next step was detection of SURF features. We
detected SURF features from both the group and test image.
Then we selected 100 strongest feature points from the test
image and 300 from the group photo. After selection we
extracted the features points from both the images. The final
step was features matching. As the number of features of both
the images matches a bounding box was created to show the
identified person.
3.1. IMAGE ACQUISITION: After taking a group photo,
we apply Viola Jones method to find faces in it. Crop the face
images and save them. Convert them into grayscale images
and resize them. For our experiment the images were resized
to 92*112 pixels. All images should be of same size.
3.2. SURF FEATURES: SURF is a scale and in-plane
rotation invariant feature. It contains interest point detector
and descriptor. The detector detects the interest points in the
image, and the descriptor describes the features of the interest
points and constructs the feature vectors of the interest points.
3.3. Interest Point Localization: Given a point 𝑋 = (𝑥, 𝑦) in
an image I, the Hessian matrix 𝐻(𝑥; 𝜎) in x at scale σ is
defined as follows [4]:
𝐻(𝑥, 𝜎) = [
𝐿𝑋𝑋 (𝑥, 𝜎)
𝐿𝑌𝑋 (𝑥, 𝜎)
𝐿𝑋𝑌 (𝑥, 𝜎)
] − − − (1)
𝐿𝑌𝑌 (𝑥, 𝜎)
Where 𝐿𝑋𝑋 (𝑥, 𝜎) , 𝐿𝑋𝑌 (𝑥, 𝜎) and 𝐿𝑌𝑌 (𝑥, 𝜎) are the
convolutions of the Gaussian second order partial derivatives
with the image I in point x respectively. The location and
scale of interest points are selected by relying on the
determinant of the Hessian matrix. Interest points are
localized in scale and image space by applying non-maximum
suppression in a 3×3×3 neighborhood [4].
3.4. Interest Point Descriptor [4]: Around the detected
interest points, SURF constructs a circular region to assign a
unique direction and thus gain invariance to image rotations.
The direction is calculated using Haar wavelet response in
both x and y directions. The integral images can be used to
quickly calculate Haar wavelets. When the dominant
orientation is predicted and included in the interest point
information, SURF descriptors are constructed by extracting
square regions around the interest points. The windows are
split up in 4× 4 sub-regions. The underlying intensity pattern
(first derivatives) of each sub-region is described by a vector
𝑉 = [∑ 𝑑𝑥 , ∑ 𝑑𝑦 , ∑ |𝑑|𝑥 , ∑ |𝑑|𝑦 ] − −(2)
Figure 1 Gaussian second order partial derivatives and
corresponding box filter
3.5. SURF Features Extraction: SURF features were
extracted from images through SURF detectors and
descriptors. Interest points were extracted from each face
image after histogram equalization. Then because of applying
histogram equalization we obtained about 30-100 interest
points in the test and group image. The SURF feature vectors
of the group of interest points were then calculated to describe
the image and the feature vectors were normalized to 1. These
features were person-specific, as the number and the positions
of points selected by SURF detector as well as the features
around these points computed by SURF descriptor were
different in each person’s image.
3.6. SURF Features Matching: Extract the detector points
and the descriptor of both the test and group images and
compare them.
3.7. OUTPUT: When the number of the matched points of
two images was smaller than the predefined threshold then it
displayed “matched_points1 and matched_points2 do not
have enough point”. In other case it will identify the person.
Take a group photo
Detect faces from the image, crop and
save to use as test images
Detect SURF features from both the test
image and group photo
Figure 3: Detected features points in both the test and
group image
Select 100 strongest points from test
image and 300 from group photo
Extract the detected features and then
match them
Figure 4: Identified image
5. RESULT
Matched?
Yes
Identify the person
No
Give “matched_points1
and matched_points2 do
not have enough point”
Figure 2: Methodology Flow Chart
4. EXPERIMENTAL ANALYSIS
First of all Viola Jones algorithm was applied to detect the
faces from the group photos and then cropped them and saved
them to use them as test images. Then SURF feature method
was applied to detect, extract and match the features of both
the test and group image.
We had 30 test images and 6 group photos. The size of group
images was 3264*1836 pixels. All the test images were of
size 92*112 pixels. Both the test images and group images
were converted to grayscale. Figure 3 shows detected SURF
features from both the test and group image. Figure 4 shows
the result i.e., the identified image in the group photo.
Out of 30 images 25 were identified in the group photograph
and 5 showed “matched_points1 and matched_points2 do not
have enough point”. Hence the identification rate was
83.33%.
6. CONCLUSION
This paper presents efficient approach person identification in
a group photograph which can be used to identify a specific
person in a group photo containing a large number of persons.
This approach shows best result when there is no pose
variation i.e., the pose of the test image should be same as that
in the group photograph and no occlusion present. It worked
only on illumination problem that was solved by histogram
equalization. In future we will be working on pose variation
and occlusion.
7. REFERENCES
[1]. Geng Du, Fei Su, Anni Cai, Face recognition using SURF
features, Proc. of SPIE Vol. 7496, 2009
[2]. Philippe Dreuw,Pascal Steingrube,Harald Hanselmann,
Hermann Ney, SURF-Face:Face Recognition Under
Viewpoint Consistency Constraints, DREUW ET AL.:
SURF-FACE RECOGNITION,2009
[3]. Shinfeng D. Lin, Bo-Feng Liu and Jia-Hong Lin,
Combining Speeded-Up Robust Features with Principal
Component Analysis in Face Recognition System,
[4].
[5].
[6].
[7].
[8].
[9].
[10].
[11].
[12].
[13].
[14].
[15].
[16].
[17].
[18].
[19].
[20].
International Journal of Innovative Computing,
Information and Control, Volume 8, Number
12,December 2012.
Donghoon Kim, Rozenn Dahyot, Face Components
Detection using SURF Descriptors and SVMs, 2013
David G.Lowe, "Distinctive image features from scale
invariant keypoints",International Journal of Computer
Vision January, pp. 91-110, 2004
Bay,H.,Tuytelaars,T.,and Van Gool L,SURF:Speeded Up
Robust Features, In Proceedings of the Ninth European
Conference on Computer Vision, May 2006.
T. Lindeberg. Feature detection with automatic scale
selection.IJCV,30(2):79- 116, 1998.
Y.Ke and R. Sukthankar, PCA-SIFT:A more distinctive
representation
for
local
image
descriptors.In
CVPR(2),Pages 506:513, 2004.
F.Jurie and C.Schmid,Scale invariant shape features for
recognition of object categories,In CVPR,Volume
II,Pages 90:96, 2004.
M.Brown and D.Lowe,Invariant features from interest
point groups,In BMVC, 2002.
P. J. Phillips, H. Moon, S. A. Rizvi, P. J. Rauss, "The
FERET evaluation methodology for face-recognition
algorithms," IEEE transactions on pattern analysis and
machine intelligence, 22(10), (2000).
D. Lowe, "Distince image features from scale-invariant
keypoints," Int. Journal of Computer Vision, 60(2), 91110, 2004
H. Bay, A. Ess, T. Tuytelaars, L. Van Gool, "Speeded-up
robust features(SURF)," Comput. Vis. Image Underst.,
110(3),346-359, 2008
H. K. Ekenel and R. Stiefelhagen, Analysis of local
appearance-based face recognition: Effects of feature
selection and feature normalization, In VPR Biometrics
Workshop, pp.34-34, 2006.
S. Yan, H. Wang, X. Tang and T. S. Huang, Exploring
feature descriptors for face recognition, International
Conference on Acoustics, Speech, and Signal Processing,
vol.1, pp.629-632, 2007.
D. Lowe, Distinctive image features from scale-invariant
keypoints, Int. J. Computer Vision, vol.60, no.2, pp.91110, 2004.
A. Majumdar and R. K. Ward, Discriminative SIFT
features for face recognition, Canadian Conference on
Electrical and Computer Engineering, pp.27-30, 2009.
S. D. Lin, J.-H. Lin and C.-C. Chiang, Usin gradient
features from scale-invariant keypoints on face
recognition, International Journal of Innovative
Computing, Information and Control, vol.7, no.4,
pp.1639-1649, 2011.
M. Aly, Face recognition using SIFT features, CNS/Bi/EE
Report 186, pp.3277-3280, 2006.
Jyoti Dalal, Mahendra Singh Meena, Priti Singh, “A facial
Recognition Technique Using Skin Segmentation
Technique”, International Journal of Computer
Applications, 2015.
.