J Sign Process Syst (2014) 74:405–416
DOI 10.1007/s11265-012-0728-9
Uniform Local Derivative Patterns and Their Application
in Face Recognition
Huorong Ren & Jianwei Sun & Yanhong Hao &
Xinxin Yan & Yang Liu
Received: 1 March 2012 / Revised: 11 September 2012 / Accepted: 18 December 2012 / Published online: 3 January 2013
# Springer Science+Business Media New York 2012
Abstract In recent years, local feature descriptors have received more and more attention due to their effectiveness in the
field of face recognition. Local Derivative Patterns (LDPs) for
local feature descriptions attract researchers’ great interest.
However, an LDP produces 2p different patterns for p neighbors through the transition of LDPs for an image, which lead to
high dimension features for image analysis. In this paper, LDPs
are expanded to Uniform Local Derivative Patterns (ULDPs)
that have the same binary encoding way as LDPs but different
transition patterns by introducing uniform patterns. A uniform
pattern is the one that contains at most two bitwise transitions
from 0 to 1 or vice versa when the binary bit is circular. Then,
the number of the transition patterns is reduced from 2p to p(p
−1)+3 for p neighbors, e.g., 256 to 59 for p=8. For face
recognition, the histogram features are combined together in
four directions, and both non-preprocessed and preprocessed
images are used to evaluate the performance of the proposed
ULDPs method. Extensive experimental results on three publicly available face databases show that the proposed ULDPs
approach has better recognition performance than that obtained
by using the LDPs method.
Keywords Face recognition . Local derivative patterns
(LDPs) . Uniform pattern . Uniform local derivative patterns
(ULDPs).
1 Introduction
In the past decades, a great deal of attention from the
academic and industrial communities has been focused on
H. Ren (*) : J. Sun : Y. Hao : X. Yan : Y. Liu
School of Electro-Mechanical Engineering,
Xidian University, No.2, South TaiBai Road,
Xi’an 710071, China
e-mail: hrren001@163.com
the face recognition due to its wide applications to information security, network security, and personal identification.
There are many well-known methods for face recognition,
including Fisherface [1], Eigenface [2, 3], Linear Discriminant Analysis (LDA) [4], and kernel methods [5], which
have been extensively studied and proved successful. However, these methods require a great large number of trained
images, and actually the images available for training are
few or even just one. In addition, they only extract holistic
features of the face images since their local features are
neglected, while the local features are essential information
for face recognition.
Recently, non-statistical local feature descriptors called
Local Binary Patterns (LBPs) have become more and more
popular in the field of face recognition. Initially, LBPs are
designed by Ojala et al. [6] for texture feature classification
due to its invariance with monotonic gray level changes.
Ahonen et al. [7, 8] first apply LBPs to the area of face
recognition. After that, LBP-based face recognition methods
gain much more attention due to their robustness in illumination and expression. Naturally, the methods based on
LBPs are developed with several variations for improved
performance in face recognition.
Since the threshold of LBPs is defined exactly as the
value of the central pixel, LBPs are sensitive to noise in
uniform regions. To address this problem, Tan and Triggs
[9] extend LBPs to Local Ternary Patterns (LTPs) with a 3value code, in which the threshold is set to be slightly larger
than zero. Ahonen et al. [10] propose Soft Histogram for
Local Binary Patterns (SLBPs) that contribute one pixel
typically to more than one bin meanwhile the sum of contributions of the pixel to all bins is always 1. An SLBP also
suffers from the same problem that it is difficult to set
a suitable threshold, as the case in LTPs. Furthermore,
Heikkila et al. [11] propose Center-Symmetric Local Binary
Patterns (CSLBPs) by comparing center-symmetric pairs of
pixels around a central pixel instead of comparing each pixel
406
with the central pixel. Liu et al. [12] present a new face
recognition method, three-level face features, by using
CSLBPs three times.
More ways to improve LBPs are to combine LBPs with
other approaches [13, 14]. For example, the Multi-resolution
Histograms of Local Variation Patterns (MHLVPs) proposed
by Zhang et al. [15] represent face images as the concatenation of the local spatial histogram of local variation patterns computed from the multi-resolution Gabor features.
Zhang et al. [16] also report a face recognition method,
based on Local Gabor Binary Pattern Histogram Sequence
(LGBPHS), which extracts Gabor and LBPs features of face
images. The methods mentioned above suffer from a common problem: they capture non-directional first-order features only.
To find more detailed information, Zhang et al. [17]
expand LBPs to Local Derivative Patterns (LDPs) and
high-order LDPs that encode directional pattern features
at four local derivative variations. Thus, they contain
more spatial information than traditional LBPs. Their
work also designs the Gabor Local Derivative Patterns
(GLDPs) that are proved successful in illumination variation face recognition by combining Gabor Wavelet with
LDPs. All the methods mentioned above usually cause a
high dimensionality that to a large extent limits their
applications.
This paper expands LDPs to Uniform Local Derivative
Patterns (ULDPs) for face recognition under different appearance variations. In this framework, transition patterns of
LDPs are divided into two categories: uniform and nonuniform ones. Therefore, the proposed method produces p
(p−1)+3 patterns for p neighbors only, much less than 2p
patterns in normal LDPs such that the size of the features
dimensionality and the computational costs are greatly reduced. Different from the existing statistical methods, a
ULDP directly extracts local features from the input images
or feature images, as done in an LDP, which is why a ULDP
has more robustness to unpredictable distribution images
than the statistical methods. Similar to other non-statistical
methods, a ULDP requires one image for per person for
training only. Moreover, preprocessing methods are applied
to combine with ULDPs for face recognition in this paper,
which effectively enhances the performance of ULDPs in
illumination variations.
The remainder of the paper is organized as follows.
Section 2 briefly reviews LDPs method and discusses
ULDPs in detail. Section 3 proposes the application of the
proposed ULDPs method in face recognition. Furthermore,
preprocessing methods with γ correction and the Different
of Gaussian (DoG) filter are described in this section.
Section 4 reports experimental results to evaluate the
performance of the methods on face recognition. Finally,
Section 5 concludes this paper.
J Sign Process Syst (2014) 74:405–416
2 Uniform Local Derivative Patterns
In this section, we first briefly review the basic idea of Local
Derivative Patterns, and then discuss in detail the proposed
method, i.e., Uniform Local Derivative Patterns, in which the
number of different patterns is reduced from 2p to p(p−1)+3
for p neighbors.
2.1 Local Derivative Patterns
LDPs are proposed for face recognition through a high order
local pattern descriptor. It is a general framework to encode
directive pattern features from local derivative variation.
The most notable characteristic of this method is that an
LDP captures local features in four directions: 0°, 45°, 90°,
and 135°, and concatenates the transition result as a 32-bit
binary string. The first-order derivative for a pixel in 0°, 45°,
0
90°, or 135° direction, denoted as Ia ðZÞ, is calculated in a 3×
3 area around it, where α=0°, 45°, 90°, and 135°. Let Z0 be
the central point of a local region I (Z), and Zi’s are the
neighbors of Z0, where i=1, 2, ..., 8, as shown in Fig. 1.
Hence the first-order derivatives at Z = Z0 are as follows
[17]:
0
I0 ðZ0 Þ ¼ I ðZ0 Þ
I ðZ4 Þ
ð1Þ
0
I ðZ3 Þ
ð2Þ
0
I ðZ2 Þ
ð3Þ
I45 ðZ0 Þ ¼ I ðZ0 Þ
I90 ðZ0 Þ ¼ I ðZ0 Þ
0
I135 ðZ0 Þ ¼ I ðZ0 Þ
I ðZ1 Þ
ð4Þ
Z9
Z10
Z11
Z12
Z13
Z24
Z1
Z2
Z3
Z14
Z23
Z8
Z0
Z4
Z15
Z22
Z7
Z6
Z5
Z16
Z21
Z20
Z19
Z18
Z17
Figure 1 Example of LDPs representation.
J Sign Process Syst (2014) 74:405–416
407
The second-order directional LDPs derived from the firstorder derivative, in α direction at Z = Z0, can be defined as:
n 0
0
0
o
0
0
0
LDPa2 ðZ0 Þ ¼ f Ia ðZ0 Þ; Ia ðZ1 Þ ; f Ia ðZ0 Þ; Ia ðZ2 Þ ; ; f Ia ðZ0 Þ; Ia ðZ8 Þ
where f( , ) is a binary coding function, determining the
types of local pattern transitions. It encodes the cooccurrence of two derivative directions at different neighboring pixels, as shown in [17]:
0
0
f Ia ðZi Þ; Ia Zj
¼
2.2 The Theory of ULDPs
An LDP directly encodes the binary result of the secondorder derivatives in four directions. For eight neighbors, 256
(28) different local derivative patterns are produced in each
direction.
It is most remarkable that some local derivative patterns contain the vast majority of the image features,
sometimes over 90% in LDPs, and it is likely to LBPs
[18]. These patterns, called uniform ones, have the same
specialty in common that contains little spatial transitions. We design uniform local derivative patterns
(ULDPs) by introducing a uniform pattern that reduces
the number of the transition patterns of LDPs. ULDPs
have the same binary encoding way as LDPs, while the
final ULDPs transition patterns have a new encoding
way that is different from LDPs.
To formally define ULDPs, the transition patterns of
LDPs are first divided into two categories: uniform patterns
and non-uniform patterns, according to the number of spatial transitions (bitwise 0/1 change) of their patterns. An
LDP pattern is said to be uniform if its binary pattern
contains at most two bitwise transitions from 0 to 1 or vice
versa when the bit is circular, while all the other binary
patterns are attributed to non-uniform patterns. For example,
000000002 (0 transitions) and 011000002 (2 transitions) are
uniform whereas 010100002 (4 transitions) and 100000102
(4 transitions) are non-uniform. For eight neighbors, there
are nine types of patterns which are uniformed as shown in
Fig. 3, where patterns 111111112 and 000000002 have zero
time transitions, while the others have exactly 2 times transitions when the bit is circular. Each of the nine types of the
patterns except for 111111112 and 000000002, has eight
kinds of rotation forms when the bit is circular. Therefore,
there are totally 7×8+2=58 different uniform patterns for
0
0
0; if Ia ðZi Þ Ia Zj > 0
0
0
1; if Ia ðZi Þ Ia Zj 0
a ¼ 0 ; 45 ; 90 ; and 135 ; i ¼ 1; 2; ; 8:
ð6Þ
The second-order directional LDPs can also be defined
as:
LDPa2 ðZ0 Þ ¼
8
0
X
0
f Ia ðZ0 Þ; Ia ðZn Þ 2n
ð7Þ
1
n¼1
Finally the second-order LDPs, denoted by LDP2(Z), is
defined as the concatenation of the four second-order directional LDPs, complied with Eq. (8):
n
o
0
LDP2 ðZÞ ¼ LDPa ðZÞ a ¼ 0 ; 45 ; 90 ; and 135
ð8Þ
A computational example of LDPs is illustrated in Fig. 2.
To compute an LDP in α=0° direction at the central pixels,
we calculate the first-order derivatives by Eq. (1). Then, we
obtain the second-order directional LDPs in 0° direction by
Eq. (5) or Eq. (7), i.e., LDP02 ¼ 11000000, an 8-bit string,
which is denoted by transition pattern 3 (the left-most bit is
the low bit). Similarly, the calculated second-order LDPs in
45°, 90°, and 135° are “00001100” (transition pattern 48),
“00011110” (transition pattern 120), and “00001100” (transition pattern 48), respectively. Finally, all the second-LDPs
binary results are concatenated to gain the LDP 2 (Z),
“110000000000110000011111000001100”, a 32-bit string.
Figure 2 Example to obtain
the second-order LDPs in α=0°
direction.
ð5Þ
(a)
(b)
(c)
81
75
72
72
89
65
64
58
54
55
6
4
-1
1
53
52
53
54
56
-1
-1
-2
0
51
52
59
67
75
-7
-8
-8
0
58
57
59
69
81
Original sub-region
First-order derivative
1
0
0
0
0
Second-order LDPs
408
J Sign Process Syst (2014) 74:405–416
Figure 3 Nine uniform binary patterns that can occur in the circularly symmetric neighbor set, where black and white circles correspond to bit
values of 0 and 1 in the 8-bit output of the operator, respectively.
eight neighbors. Then ULDPs in α direction at Z = Z0 for
8 neighbors can be calculated by the following equations:
8"
#
8
>
p 1
P
0
0
>
>
f Ia ðZ0 Þ; Ia Zp 2
>
>
>
< p¼1"
#
ULDP ¼
8
p 1
P
0
0
>
>
N
f Ia ðZ0 Þ; Ia Zp 2
; if g ðLDPÞ 2
>
>
>
p¼1
>
:
58
; otherwise
ð9Þ
0
0
0
0
f Ia ðZ0 Þ; Ia ðZ1 Þ
gðLDPÞ ¼ f Ia ðZ0 Þ; Ia ðZ8 Þ
7
P
0
0
0
0
f Ia ðZ0 Þ; Ia Zpþ1
f Ia ðZ0 Þ; Ia Zp
þ
p¼1
ð10Þ
where N(x) is a function to count the total number of the
non-uniform patterns, which is less than x, and g(LDP) is
used to calculate the number of spatial transitions in which
bitwise 0/1 changes of LDPs patterns occur. Equation (10)
implies an LDP pattern (an 8-bit string) first moves one bit,
and then subtracts the original binary value of LDPs pattern
from the resulting pattern after removal. Finally, the number
of spatial transitions is obtained by calculating the sum of
absolute values of the difference.
It is show in Eq. (9) that a pattern is uniform only if its
number of spatial transitions is not more than 2, denoted by g
(LDP) ≤2. By the previous descriptions, it is clear that there
are exactly p(p-1)+2 uniform patterns numbered from 0 to p
(p-1)+1 for p neighbors (i.e., from 0 to 57 for 8 neighbors) and
we define all the non-uniform patterns as pattern p(p-1)+2 that
is exactly 58 for p=8. An example of ULDPs computation for
eight neighbors is illustrated in Fig. 2. Noted that the 8-bit
binary result in Fig. 2, i.e., “11000000”, contains two times
bitwise transitions. Consequently, it is a uniform pattern and
its value of ULDPs pattern is 3 by Eq. (9). The three results,
“00001100”, “000111110”, and “00001100”, are uniform
patterns, corresponding to the values of ULDPs patterns: 17,
25, and 17, respectively, which are different from LDPs patterns calculated previously.
A ULDP produces p(p-1)+3 different patterns that consist of p(p-1)+2 uniform patterns and one non-uniform
pattern. Compared with 2p different patterns produced by
LDPs, the quantity is reduced greatly. As well known, the
final image features used for image analysis are the histogram features accumulated by feature descriptors of an input
image. The reason that why ULDPs histogram features
provide better discrimination compared against the LDPs
histogram features comes down to the difference in their
statistical properties. The relative proportion of the nonuniform patterns of all patterns accumulated into a histogram is so small that their probabilities cannot be
estimated reliably. In addition, the uniform patterns reduce the influence of the high frequency noise that can
be defined as non-uniform patterns. This means that it
can extract representative features of the face images
with the feature vectors of the uniform and nonuniform patterns. As shown in Fig. 4, the ULDPs produce a 59-dimensional feature vector less than the 256dimensional feature vector produced by LDPs. Furthermore, we can also see that there are a large number of
zeros between patterns 100 and 125 in LDPs histogram,
and no such sparse problem exists in ULDPs histogram.
3 ULDPs for Face Recognition
This section discusses the application of ULDPs to face
recognition. The overall framework of the proposed
method used for histogram feature extraction is illustrated
in Fig. 5. In this method, the ULDPs-based face recognition is modeled by the following procedures: (1) normalizing an input face image according to the location of
the eyes and preprocessing it by using γ correction and
the Different of Gaussian (DoG) filter; (2) labeling the
image by the ULDPs descriptor in four directions; (3)
dividing the transition results of each image into several
rectangular sub-regions and calculating histogram features
in each region; and (4) establishing the face model with
the concatenation of all the histograms sequence and
recognizing the face image by the histogram intersection
and the nearest neighbor classifier. In what follows, we
discuss these procedures in detail.
3.1 Image Normalization and Preprocessing
For face recognition, images from a database are in a generally normalized size by cropping images according to the
location of eyes to eliminate the impact of the background.
Moreover, preprocessing methods are often used to decrease
noise factors, such as γ correction and the Different of
Gaussian (DoG) filter, which are believed to be the preferable methods for this problem.
J Sign Process Syst (2014) 74:405–416
409
250
250
225
225
200
200
175
175
150
150
125
125
100
100
75
75
50
50
25
25
0
0
0
25
50
75
100
125
150
175
200
225
250
0
5
10
15
20
Histogram of LDPs
25
30
35
40
45
50
55 58
Histogram of ULDPs
Figure 4 Differences of the histogram between LDPs (middle) and ULDPs (right).
γ correction is a nonlinear gray-level transformation that
replaces gray-level I with Iγ (for γ >0) or log(I) (for γ=0),
where γ is a user-defined parameter ranging from 0 to 1,
reflecting the performance of nonlinear transformation. γ
correction increases the contrast of images by enhancing the
local dynamic ranges in the shadowed regions of the image
while compressing them in bright regions. It is clear that the
only factor in γ correction is the value of γ that is widely
used in the range [0, 0.5].
To correct the uneven illumination, an image is first
changed into the frequency domain by two-dimensional
Fourier transform. Then it is filtered by the Different of
Gaussian (DoG) filter to remove redundant information,
while preserving the useful low-frequency information.
The DoG filter is a common filter in the field of computer
vision and image analysis, which consists of two Gaussian
functions with different standard deviations [19]:
GðsÞ ¼ A1 e
s2
2σ2
1
A2 e
s2
2σ2
2
A1 A2 > 0; σ1 > σ2
ð11Þ
It is obvious that the performance of the DoG filter is
principally decided by two parameters: σ1 and σ2. By a series
of experiments, we find that when γ=0.2, A1 =A2 =1, σ1 =2 and
σ2 =1, the proposed ULDPs can achieve the best performance.
Thus, these values are determined as the default setting.
3.2 Histogram of ULDPs
The purposes to develop the ULDPs-based method in the
previous sections are used for face recognition. The images
collected under various conditions such as illumination,
accessory, and expression are really challenges to face recognition. We model the distribution of ULDPs by spatial
histograms [8, 17, 20] because of their robustness to the
various conditions. A ULDP captures histogram features in
four invariant directions: 0°, 45°, 90°, and 135°. To avoid
the loss of too much Micro-local information after ULDPs
encoding, an image is divided into several sub-regions. In α
direction, ULDPsα are spatially divided into rectangular
sub-regions represented by S1, S2, ..., and SN, and then the
spatial histograms of ULDPs, denoted as HULDP [17], can
be extracted as follow:
HULDPði; aÞ ¼ fHULDPa ðSi Þji ¼ 1; ; N ;
Original image
Histogram
Feature
Preprocessed
image
ULDP at four directions
Figure 5 The framework of ULDPs-based histograme feature extraction.
ð12Þ
ULDP
Normalized
image
a ¼ 0 ; 45 ; 90 ; and 135 g
410
J Sign Process Syst (2014) 74:405–416
where HULDPa ðSi Þ is the ULDPs histogram features extracted
from the local sub-region Si.
For histogram matching, the histogram intersection is
low in computational complexity and simple in operations
among existing similarity measurements. Thus, it is used for
the matching measurement of the two histograms. Histogram intersection is defined as follows:
DHI ðH; S Þ ¼
min
SHI
B
P
i¼1
Hi ;
B
P
ð13Þ
Si
i¼1
Normalized image
B
X
minðHi ; Si Þ
ð14Þ
i¼1
where DHI ðH; S Þ is the histogram intersection statistic result
with H = (H1,H2, ...,HB)T and S = (S1,S2, ...,SB)T, and B is the
total number of the histogram features. The similarity of the
two histograms calculated by Eqs. (13) and (14) is used for
the classification with the nearest neighbor classifier.
4 Experimental Results and Analysis
The preceding sections deal with theoretical aspects of the
proposed framework. To empirically test the main premises
of our work and the effectiveness of the described methodology, we conducted a series of experiments to illustrate the
performance of the proposed ULDPs for face recognition
that covers various conditions including illumination, expression, pose, and accessory. An extensive set of publicly
available face databases, FERET [21], CMU PIE [22], Extended Yale B [23, 24], and CAS-PEAL-R1 [25], were used
in the experiments. In what follows, the first subsection
conducts experiments on a subset of the widely used the
FERET database to test the robustness of ULDPs. Then, we
conducted the comparative experiments to test the performance of ULDPs on the CAS-PEAL-R1 database to test
ULDPs in expression and accessory. Furthermore, the CMU
PIE database with a variety of poses and illuminations was
utilized to evaluate the performance of the proposed method. The experiment result on the Extended Yale B database
with severe illumination variations was also reported in this
work.
Before proceeding to the experiments, we take a quick
look at the statistical data of the appearing uniform patterns
in ULDPs. In ULDPs, there are 58 uniform patterns among
the 256 possible patterns. For the images in the FERET
database, the 58 uniform patterns contribute 64.99 %,
73.01 %, 69.72 %, and 73.22 % in the four directions, 0°,
45°, 90°, and 135°, respectively. In the CAS-PEAL-R1
database, they are 81.67 %, 86.53 %, 83.7 %, and
Original image
Figure 6 Example of a normalized FERET image.
84.65 %, respectively. In the CMU PIE database, they are
73.41 %,79.64 %,69.98 %, and 79.57 %, respectively. In the
Extended Yale B database, the proportions of uniform patterns are relatively low: 62.45 %, 68.75 %, 60.87 %, and
68.86 % in the four directions, respectively. It indicates that
the uniform patterns occupy the majority of the information
in an image, which represents the essential lower frequency
information. More to the point, pattern 000000002 representing the flat region is the most frequent pattern in the all
images.
4.1 Experimental Comparisons on the FERET Database
The comparative experiments between LDPs and ULDPs
were first conducted on the FERET face database that is
widely used to evaluate face recognition algorithms. The
FERET database consists of a total of 14,051 gray-scale
images representing 1,199 individuals. The images contain
variations in lighting, facial expressions, pose angele, etc. In
this work, only frontal faces are considered. These facial
images can be divided into five sets as follows:
Recognition rate (%)
SHI ðH; S Þ ¼
100
90
80
70
60
50
40
30
20
10
0
4×4
fb
fc
dup I
dup II
Average
5×5
6×6
7×7
8×8
9×9
Sub-region number
Figure 7 Comparative face recognition rates of ULDPs on the FERET
database with different sub-regions.
J Sign Process Syst (2014) 74:405–416
(a)
411
95
Recognition rate (%)
85
fb
fc
dup I
dup II
Average
75
65
55
45
35
25
15
LBPs
ULBPs
LDPs
ULDPs
Results on the non-preprocessed images.
(b)
100
Recognition rate (%)
90
80
70
60
fb
fc
dup I
dup II
Average
50
40
30
20
PLBPs
PULBPs
PLDPs
PULDPs
Results on the preprocessed images.
Figure 8 Comparative identification accuracies on the FERET database. a Results on the non-preprocessed images. b Results on the
preprocessed images.
&
&
&
&
&
fa set, used as a gallery set, contains frontal images of
1,196 people.
fb set (1,195 images). The subjects were asked for an
alternative facial expression than in the fa photograph.
fc set (194 images). The photos were taken under different lighting conditions.
dup I set (722 images). The photos were taken later in
time between 1 min to 1031 days.
dup II set (234 images). This is a subset of the dup I set
containing those images that were taken at least after
18 months after the corresponding gallery image.
Figure 9 Samples of the
frontal image of one subject
from the CAS-PEAL-R1
database.
To ensure the reliabilty of the test, the publicly available
CSU face identification evaluation system [26] was utilized
to normalize the images. All the images were normalized
and cropped to 150×130 pixels by the coordinates of the
eyes. Figure 6 shows an example of a normalized FERET
image. In this work, fa containing 1,196 frontal images of
1,196 subjects was used as a gellery set, while fb, fc, dup I,
and dup II were used as probe sets.
The first experiment was designed to explore the influence of the sub-region number on recognition accuracy of
the normalized images. In this experiment, images were
divided into a variable number of regions, from 4×4 to 9×
9, and the obtained histograms from each region were concatenated to obtain the descriptor. Figure 7 shows that
recognition accuracy on all four subsets of the FERET
database is improved when the sub-region number increases
from 4×4 to 8×8, while fb and dup II subsets decrease when
the number reaches to 9×9. Moreover, the average curve of
recognition accuracy becomes flatter when the sub-region
number increases from 8×8 to 9×9. Therefore, we chose 8×
8 as the default size in the experiment on the FERET
database.
To observe the performance of ULDPs under different
conditions, the next experiments were conducted on the
individual probe sets. Experimental results in Fig. 8(a) demonstrate that ULDPs work better than the other methods:
LBPs, ULBPs, and LDPs in average. It is clear that ULDPs
achieve much higher recognition accuracy compared against
LDPs.
The experiments were also designed to evaluate the effectiveness of ULDPs on the preprocessed images with the
DoG filter and γ correction. The recognition rates obtained
with different descriptors on the preprocessed images are
shown in Fig. 8(b). It should be noted that PULDPs (ULDPs
with the preprocessing method) perform better than PLBPs
(LBPs with the preprocessing method), PULBPs (ULBPs
with the preprocessing method) and PLDPs (LDPs with the
preprocessing method) in average. Specially, on the fc subset, when gallery and probe images were taken under different illumination, PULDPs give the recognition accuracy
as high as 94.85 %. The results on the fc subset show that
412
J Sign Process Syst (2014) 74:405–416
(a)
75
95
Recognition rate (%)
Recognition rate (%)
90
85
80
75
Expression
Accessory
Average
65
45
non-preprocessed method
ULBPs
LDPs
with preprocessed method
25
LBPs
ULDPs
Result on the non-preprocessed images.
ULBPs
LDPs
ULDPs
Figure 12 Comparative face recognition rates of ULDPs, LDPs,
ULBPs, and LBPs on the CMU PIE databse.
over LBPs in dup I and dup II subsets, in which LBPs give
the best result with the original images. It means that
PULDPs are more robust to aging than that of ULDPs.
(b)
90
Recognition rate (%)
55
35
70
60
LBPs
65
86
4.2 CAS-PEAL-R1 Database - Expression and Accessory
Variant
82
78
Expression
Accessory
Average
74
70
PLBPs
PULBPs
PLDPs
PULDPs
Result on the preprocessed images.
Figure 10 Comparative the face recognition rates of ULDPs, LDPs,
ULBPs, and LBPs on the CAS-PEAL-R1 database. a Result on the
non-preprocessed images. b Result on the preprocessed images.
the preprocessing method is powerful for ULDPs in illumination variation. It is worth mentioning that PULDPs
achieve 10.66 % and 13.51 % performance improvements
Pose 09
Pose 27
Pose 29
The CAS-PEAL-R1 face database contains 99,594 images
among 1,040 individuals (595 males and 445 females) with
different sources of variation, especially Pose, Expression,
Accessory, and Lighting (PEAL). We choose the first 300
subjects (one image per person) from the gallery set of CASPEAL-R1 as our gallery set. Our probe set contains 1,939
frontal face pictures, including expression (878 pictures)
and accessory (1061 pictures), which are chosen from the
probe set of CAS-PEAL-R1. Some sample images are
shown in Fig. 9. In the following experiments, all the
employed images were normalized to 150×130 pixels by
the CSU face identification evaluation system to eliminate
the influence of the background.
To observe the performance of ULDPs under different
conditions in expression and accessory, the experiments
were conducted on the CAS-PEAL-R1 database that
includes variations in facial expressions and accessories.
The experimental results as shown in Fig. 10(a) demonstrate
that ULPDs obtain better recognition accuracy on expression subset than LDPs but work worse than LDPs on the
accessory subset. Since the patterns distribution in histogram introduced by the accessories is different from the face
natural variations, ULDPs are of plain robustness to accessories and have slightly better recognition accuracy to
expression.
Pose 05
Pose 07
Figure 11 Samples of the frontal image of one subject from the CMU
PIE database.
Table 1 Experiment using the CMU PIE database with different
poses.
Method
Pose 5
Pose 7
Pose 9
Pose 27
Pose 29
ULDPs
PULDPs
0.52
0.74
0.60
0.79
0.54
0.78
0.68
0.88
0.55
0.68
J Sign Process Syst (2014) 74:405–416
413
Figure 13 Samples of the frontal image of one subject from the Extended Yale B database.
We also conducted the experiments to evaluate the effectiveness of the PULDPs on the CAS-PEAL-R1 database. It
can be maintained from Fig. 10(b) that PULDPs perform
better than PLBPs, PULBPs and PLDPs in both expression
and accessory subsets. However, it should be noted from
Fig. 10 that PULDPs have some lower accuracy than that of
ULDPs. A reasonable explanation is that the preprocessing
method smoothes out the minute variations of expression on
the expression subset.
4.3 CMU PIE Database - Pose and Illumination Variant
We further explored the impact of the preprocessing on the
performance of ULDPs under various poses and illuminations using the CMU PIE database. The database contains
totally 41,368 images of 68 individuals, including a variety
of poses, illuminations, and expressions. The images with
the variations in both pose and illumination are used in the
experiment. These images include seven different poses,
labeled by Poses 05, 07, 09, 11, 27, 29, and 37, respectively,
and each pose has 21 different illumination conditions labeled by Flashes 2 to 22. Here we choose five poses for our
experiments, Pose 05, 07, 09, 27, and 29, since the five sets
have slight pose variations only. In this experiment, all the
images were cropped to 64×64 pixels according to the
location of eyes, aiming to obtain the whole facial image
and eliminate the complex background. Figure 11 shows an
example of the normalized images of a person. The frontal
images (Pose 27) with the frontal illumination condition
(Flash 08) of a person are chosen to build the gallery set.
The remaining images are used as the probe set. In this
experiment, we use 12×12 sized sub-region with 128 histogram bins for both LBPs and LDPs.
Figure 12 depicts the recognition rates of LBPs, ULBPs,
LDPs, and ULDPs on both non-preprocessed and preprocessed images by using the probe set. It can be observed
from Fig. 12 that the ULDPs have a better recognition rate
than the others on both non-preprocessed and preprocessed
images.
Compared with LBPs(39.79 %), ULBPs(40.07 %), and
LDPs (55.45 %), ULDPs(59.37 %) obtain the highest recognition rate on non-preprocessed images. Moreover, the
PULDPs (77.37 %) are also higher than PLDPs (74.1 %)
and PLBPs (40.07 %). The results show that ULDPs can
obtain more discriminative features than LBPs and LDPs.
100
R ecognition rate (% )
95
90
Table 2 Computational time (CPU seconds) of LDPs and ULDPs on
different databases.
85
non-preprocessed method
80
with preprocessed method
75
FERET
CAS-PEAL-R1
CMU PIE
Extended
Yale B
10,301
3,034
10,424
3,073
1,783
626
1,751
629
1,470
562
1,422
577
1,520
569
1,537
593
70
65
60
LBPs
ULBPs
LDPs
Figure 14 Experiment results on the Extended Yale B databse.
ULDPs
LDPs
ULDPs
PLDPs
PULDPs
414
We aslo conducted an experiment to explore the robustness of the proposed ULDPs method to poses. Tab. 1 shows
the results of this experiment.
The experiment results in Table 1 show that the PULDPs
have a better robustness to poses than ULDPs at all the five
poses. Compared with other poses, ULDPs(68 %) and
PULDPs(88 %) obtain the best recognition accuracy at Pose
27. As stated previously, the images at Pose 27 are the fontal
ones. Then, a conclusion can be reached from the results
that both ULDPs and PULDPs have poor robutness to pose
variations. Although the performance of ULDPs decreases
with the pose variations of the images in this database, it still
maintains superiority in recognition accuracy over both
LBPs and LDPs.
4.4 Extended Yale B Database - Illumination Variant
The Extended Yale B database was used for the comparative studies among ULDPs, LDPs, and LBPs under
different illumination variations. The database contains
totally 2,432 frontal images of 38 individuals (each
subject is imaged under 64 different illumination conditions). All the images have already been manually
aligned and cropped to 168 × 192 pixels for public
downloading from the database website. In our framework, these images were normalized to 84 × 96. Figure 13 shows the samples of the frontal images. The
frontal face images with lighting direction of 0o azimuth (“A+000”) and 0o elevation (“E + 00”) are used
to build the gallery set. All the remaining images belong to the probe set. Here, 12×13 pixel windows were
selected since they are good trade-off between recognition performance and feature vector length.
For the Extended Yale B database, the recognition rates
on the both non-preprocessed and preprocessed probe sets
are illustrated in Fig. 14. It indicates that ULDPs obtain a
3.13 % performance improvement over LDPs. Meanwhile
the recognition accuracy of PULDPs reaches its maximum
of 99.04 %, and obtains 31.5 % performance improvement
than that of ULDPs. It means that PULDPs are more robust
against illumination variation. We can also learn from
Fig. 14 that both PULPDs and PLDPs get much better
recognition performance than LBPs and ULBPs.
In addition, the computational time of the experiments was
also considered, as shown in Table 2. Compared with LDPs,
the time for ULDPs decreases 70.54 %, 64.89 %, 61.77 % and
62.57 %, on FERET, CAS-PEAL-R1, CMU PIE and
Extended Yale B databases, respectively. All the experiments
were carried out on a computer running Windows XP with an
AMD Athlon II CPU processor (X4 640 @ 3.0 GHz), a 2-GB
RAM. For all methods, we used the original MATLAB
implementation by using MATLAB 7.10.0 (R2010a).
J Sign Process Syst (2014) 74:405–416
5 Conclusions
Local Derivative Patterns (LDPs) are extended to Uniform
Local Derivative Patterns (ULDPs) in this paper. Experimental results, on a set of publicly available face databases,
demonstrate that the proposed ULDPs have better robustness with the respect to facial expression, illumination and
aging than LDPs. Compared with LDPs, the computational
time for ULDPs decreases over 60 % in all four databases
used in this paper, although ULDPs get worse in some
subset with accessories. We could say that the ULDPs
method is an excellent choice if the computational time is
a major concern with good recognition rates.
In addition, PULDPs demonstrate the validity and efficiency of a combination of ULDPS and the preprocessing
method for variable illumination in face recognition. The
recognition accuracy of PULDPs reaches its maximum of
99.04 % on the Extended Yale B database.
However, it should be noted that the performance of
ULDPs is still relatively low when face images contain
various accessory. Future work includes improving their
robustness with accessory variations. Another important
topic is looking for an effective feature selection method
to reduce the length of the feature vector.
Acknowledgments This work is partially supported by “the Fundamental Research Funds for the Central Universities, K50510040013”.
References
1. Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997).
Eigenfaces vs. Fisherfaces: recognition using class specific linear
projection. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 19(7), 711–720.
2. Sirovich, L., & Kirby, M. (1987). Low dimensional procedure for
the characterization of human face. Journal of the Optical Society
of America, 4(3), 519–524.
3. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition.
Journal of Cognitive Neuroscience, 3(1), 71–86.
4. Etemad, K., & Chellappa, R. (1997). Discriminant analysis for
recognition of human face images. Journal of the Optical Society
of America, 14(8), 1724–1733.
5. Yang, J., Frangi, A. F., Yang, J. Y., Zhang, D., & Jin, Z. (1997).
KPCA plus LDA: a complete kernel fisher discriminant framework
for feature extraction and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2), 230–244.
6. Ojala, T., Pietikainen, M., & Harwood, D. (1996). A comparative
study of texture measures with classification based on featured
distribution. Pattern Recognition, 29(1), 51–59.
7. Ahonen, T., Hadid, A., and Pietikainen, M. (2004). Face recognition with local binary patterns, in Proc. 8th European Conference
on Computer Vision (pp. 469–481). Prague, Czech Republic.
8. Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face description with
local binary patterns: application to face recognition. IEEE Transactions
on Pattern Analysis and Machine Intelligence, 28(12), 2037–2041.
J Sign Process Syst (2014) 74:405–416
9. Tan, X., & Triggs, B. (2010). Enhanced local texture feature sets
for face recognition under difficult lighting conditions. IEEE
Transactions on Image Processing, 19(6), 1635–1650.
10. Ahonen T., and Pietikainen M. (2007). Soft histograms for local
binary patterns, in Proc. Fin. Signal Process. Symp. (pp. 1–4).
Oulu, Finland.
11. Heikkila, M., Pietikainen, M., & Schmid, C. (2009). Description of
interest regions with local binary patterns. Pattern Recognition, 42
(3), 425–436.
12. Liu, C., Jianyun, L., & Li, L. (2011). Three-level face features for
face recognition based on center-symmetric local binary pattern.
Proceedings IEEE International Conference on Computer Science
and Automation Engineering, 4, 394–398.
13. Pang, Y., Yuan, Y., & Li, X. (2008). IEEE Transactions on Circuits
and Systems for Video Technol, 18(7), 989–993.
14. Zhang, B., Shan, S., Chen, X., & Gao, W. (2007). Histogram of
gabor phase patterns (HGPP): a novel objects representation approach for face recognition. IEEE Transactions on Image Processing, 16(1), 57–68.
15. Zhang, W., Shan, S., Zhang, H., Gao, W., and Chen, X. (2005).
Multi-resolution histograms of local variation patterns (MHLVP)
for robust face recognition, in Proc. Audio-Video-Based Biometric
Person Authentication (pp. 937–944). New York, USA.
16. Zhang, W., Shan, S., Zhang, H., Gao, W., and Chen, X. (2005).
Local gabor binary pattern histogram sequence (LGBPHS): A
novel non-statistical model for face representation and recognition,
in Proc. IEEE Int. Conf. Computer Vision vo.1, (pp. 786–791).
Beijing, China.
17. Zhang, B., Gao, Y., Zhao, S., & Liu, J. (2010). Local derivative
pattern versus local binary pattern: face recognition with high
order local pattern descriptor. IEEE Transactions on Image Processing, 19(2), 533–544.
18. Ojala, T., Pietikainen, M., & Maenpaa, T. (2002). Multiresolution
gray-scale and rotation variant texture classification with local
binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.
19. Castleman, K. R. (1996). Digital image processing [M] (pp. 212–
213). Engle wood Cliffs: Prentice Hall.
20. Zhang, H., Gao, W., Chen, X., & Zhao, D. (2005). Learning
informative features for spatial histogram-based object detection.
Proceedings IEEE International Joint Conference Neural Networks, 3, 1806–1811.
21. Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (2000). The
FERET evaluation methodology for face-recognition algorithms.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
22(10), 1090–1104.
22. Sim, T., Baker, S., & Bsat, M. (2003). The CMU pose, illumination, and expression database. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1615–1618.
23. Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001).
From few to many: illumination cone models for face recognition
under variable lighting and pose. IEEE Transactions on Pattern
Analysis and Machine Intelligence, 23(6), 643–660.
24. Lee, K. C., Ho, J., & Kriegman, D. J. (2005). Acquiring linear
subspaces for face recognition under variable lighting. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 27
(5), 684–698.
25. Gao, W., Cao, B., Shan, S., Chen, X., Zhou, D., Zhang, X., &
Zhao, D. (2008). The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Transactions on Systems,
Man, Cybernetics - Part A: Systems and Humans, 38(1), 149–161.
26. Beveridge, J. R., Bolme, D., Draper, B. A., & Teixeira, M. (2005).
The CSU face identification evaluation system: its purpose, features, and structure. Machine Vision and Applications, 16(2), 128–
138.
415
Huorong Ren received the B.Sc., M.Sc., and Ph.D. degrees in electronic engineering from Xidian University, China, in 1994, 1999, and
2004, respectively. Currently, he is an Associate Professor with the
School of Electro-Mechanical Engineering, Xidian University, China.
His research interests include face recognition, pattern recognition,
computer vision, and bio-security.
Jianwei Sun received the B.S. degree from Xidian University, Xian,
China, in 2010. He is currently a postgraduate student at School of
Electro-Mechanical Engineering, Xidian University, China. His
research interests include computer vision, machine learning and
pattern recognition.
Yanhong Hao received the B.S. degree and M.S. degree from Xidian
University, Xian, China, in 2001 and 2005, respectively. She is currently a PhD student with the Measuring and Testing Technologies and
Instruments, Xidian University, China. Her research interests include
signal processing, and pattern recognition.
416
Xinxin Yan received the B.S. degree from Xi’an University of Technology, China, in 2010. Currently, he is a postgraduate student with the
School of Electro-Mechanical Engineering, Xidian University, China.
His research interests include face recognition and pattern recognition.
J Sign Process Syst (2014) 74:405–416
Yang Liu is a postgraduate student at School of Electro-Mechanical
Engineering, Xidian University, China. He received his B. S. degree at
the Xidian University, Xian, China, in 2010. His research interests
include computer vision, machine learning and pattern recognition.