A two-stage fingerprint classification system Raffaele Cappelli Dario Maio Davide Maltoni Loris Nanni DEIS, CSITE-CNR University of Bologna viale Risorgimento 2, Bologna - Italy rcappelli@deis.unibo.it dmaio@deis.unibo.it dmaltoni@deis.unibo.it lnanni@deis.unibo.it police agencies are variants of the so-called Henry’s classification scheme. Figure 1 shows the five classes (Arch, Tented arch, Left loop, Right loop and Whorl) that are commonly used by today’s fingerprint classification techniques. Fingerprints are not uniformly distributed among these five classes: the proportions are 3.7%, 2.9%, 33.8%, 31.7% and 27.9% for Arch, Tented arch, Left loop, Right loop and Whorl, respectively. ABSTRACT In this paper we describe a fingerprint classification system based on a two-stage sequential architecture: an MKL-based classifier is first used to select the two-most-likely classes and then a second classifier (specifically trained to discriminate between the two classes) is then adopted for the final decision. The experimentation performed on NIST Special Database 4, which is one of the most important benchmarks in this area, shows that the new approach yields an error rate lower than previously published in the literature. In particular, the error rate is 4.8% and 3.7% for the five-class problem and four-class problem, respectively. Categories and Subject Descriptors I.5.4 [Pattern recognition]: Applications – Computer vision I.5.2 [Pattern recognition]: Design Methodology – Classifier design and evaluation Arch Tented Arch Left Loop Right Loop Whorl Whorl General Terms Algorithms, Experimentation, Security. Keywords Fingerprint classification, classifier combination, directional image, feature selection, subspace classifiers. 1. INTRODUCTION The identification of a person with an AFIS (Automated Fingerprint Identification System) requires the comparison of his/her fingerprint with all the fingerprints in a database, which in many applications may be very large (several millions fingerprints). Adopting a classification approach is a common strategy to reduce the number of comparisons during fingerprint retrieval and, consequently, to reduce the response time of the identification process. The first scientific studies on fingerprint classification were made by Francis Galton [14], and Edward Henry [16]. All the classification schemes currently used by Figure 1. The five fingerprint classes (two examples are reported for the whorl class, which is the most complex one). Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WBMA’03, November 8, 2003, Berkeley, California, USA Copyright 2003 ACM 1-58113-779-6/03/00011…$5.00. Automated fingerprint classification into these five classes is a difficult pattern recognition problem, due to the small interclass variability and the large intraclass variability [20] and to the noise that often affects fingerprint images. For its relevance and its intrinsic complexity, this classification problem has attracted the interest of many researches during the last thirty years and several 95 approaches have been proposed, based on a variety of classification techniques: from syntactic [27] [28] [31] and structural [6] [24] [25] methods to rule-based approaches [1] [12] [13] [17] [22] [23], from neural networks [2] [4] [15] [18] [21] [29] [30] [37] to classifier-fusion strategies [5] [7] [8] [19] [20] [26] [32] [36] [38]. In this work we propose a fingerprint classification approach based on a sequential (two-stages) architecture, where an MKL classifier [7] [10] is used to find the two-most-likely classes and an SPD (Subspace-based Pattern Discrimination) classifier specifically trained to distinguish between the two classes is selected for the final decision. SPD is a classification approach we are developing within a more general framework; SPD performance is very promising, especially for complex two-class problems. The system described has been tested on NIST Special Database 4 [34], which is one of the most used benchmarks for fingerprint classification. The rest of the paper is organized as follows: section 2 describes the feature extraction, section 3 the general classification architecture and section 4 the SPD classification approach. Finally, section 5 reports the results of the experiments and section 6 draws some conclusions. core point Figure 2. The main feature extraction steps of the proposed system. The proposed system uses the fingerprint directional image as initial feature. A directional image is a discrete matrix whose elements represent the local average directions of the fingerprint ridge lines (figure 2); it summarizes the information contained in a fingerprint pattern and can be reliably computed from noisy fingerprints. Furthermore, the local directions within a damaged area can be restored by means of a regularization process. For these reasons, most of the existing classification methods make use of fingerprint directional images. 3. CLASSIFIER ARCHITECTURE In our previous studies ([7] [8] [9]) we found that a classifier based on the multi-space generalization of the KL transform (MKL [10]) was able to achieve very good results on this problem. The basic idea of the MKL-based classifier is to represent each class with one or more low-dimensional subspaces and classify patterns according to the distance-from-space metric (the reader should refer to [7] and [10] for a throughout treatment). As in [7], the following steps are performed to obtain the feature vector of a given fingerprint (figure 2): • • Directional image Enhanced dir. image 2. FEATURE EXTRACTION • Enhanced fingerprint Input fingerprint image enhancement: the grey-level image enhanced through a filtering in the frequency domain: this step is performed as in [5]; First stage directional image computation and alignment: the Stock and Swonger method [33] is used to calculate the orientations over a 28x30 grid; then the directional image is aligned with respect to the core point [5], which is determined by using the Poincarè method [23] and by iteratively smoothing the directional image until a valid number of cores is found (as in [22]); in case the core point is not found with the above method, the R92 algorithm [35] is used for detecting a valid reference point for the fingerprint; Second stage 10 SPD classifiers two-mostlikely MKL-based classes classifier directional image enhancement: the technique initially proposed in [6] is adopted to increase the importance of the discriminant elements. Classifier selection Feature vector Selected SPD classifier Hypotized class Figure 3. The general architecture of the proposed classification system: the feature vector is first submitted to the MKL-based classifier, which returns the two most-likely classes (first stage). Then an SPD classifier trained to discriminate between these two classes makes the final decision (second stage). The enhanced directional image is treated as a single vector of dimension 1680, by simply concatenating its rows (as in [5], each element of the 28x30 directional matrix is encoded as a pair [qcos 2θ, qsin 2θ], where θ is the orientation and q an estimate of the direction reliability). A further analysis on the results obtained with the MKL-based classifier on NIST DB4 [34] showed that if the second-choice of the classifier is taken into account, the error drops to 0.9%; in 96 other words, the probability that the correct class is one of the two-most-likely classes is 99.1%. This suggested that a two-stage architecture analogous to that proposed in [20] could allow very interesting results to be achieved: an MKL-based classifier is here employed in the first stage, to determine the two-most-likely classes. This strategy allows to decompose a difficult five-class problem into ten two-class problems: for instance, if for a give fingerprint the two-most-likely classes according to the MKLbased classifier are Arch and Left loop, then an SPD classifier specifically trained to distinguish Arch and Left loop fingerprints is adopted in the second stage. both classes wA and wB and the similarity score is computed using the corresponding QDFs. The final classification rule consists in selecting the class with the highest average similarity score (calculated over all its clusters). The rationale behind SPD classification is that it is often very difficult to find good discriminant features for all the patterns in a class (especially when the intra-class variance is very high, as in fingerprint classification). Dividing a class into clusters makes it possible to find specific features that, although not well suited to discriminate all the patterns, are optimal to distinguish a subset of patterns in a class from the other class. An example in the fingerprint domain may be the following: a complex class such as Whorl contains fingerprints which are very different one from the other. Consider for instance the fingerprints in figure 3: the features that are optimal for discriminating Whorl fingerprint A from Left loop fingerprint B are not necessarily the best suited to discriminate fingerprint C from B. Figure 3 summarizes the general architecture of the proposed system. 4. SPD CLASSIFIER This section describes the SPD training and classification procedures. Training Each SPD classifier is trained, through the following steps, using a training set X={x1,x2,…,xn} of feature vectors xi∈ℜd and belonging to two classes wA and wB. 1. Feature orthogonalization and reduction: vectors in X are transformed in order to minimize correlation and decrease their dimensionality without loosing too much information. Any dimensionality reduction technique that produces orthogonal features may be used for this task; in this work we simply calculated a KL transform on the whole training set and reduced the vectors from d=1680 to r=80 features. A) Whorl B) Left Loop C) Whorl Figure 3. Two Whorl fingerprints (A and C) and a Left loop fingerprint (B). Clustering: classes wA and wB are independently partitioned into clusters to group together similar patterns. Let A and B be the subsets of X containing vectors of class wA and wB, respectively (obviously A∪B=X), and let ℘A={A1,A2,…,ANa} and ℘B ={B1,B2,…,NNb} be the partitioning of A and B, in Na and Nb clusters, respectively. We found that a fuzzy CMeans [3] algorithm is a good choice for this step, but in principle any reasonable clustering approach may be adopted. 5. EXPERIMENTAL RESULTS 3. Feature selection: a different subspace is defined for each cluster Ai or Bi, by determining an optimal subset of features. To this purpose, in this work we used a simple featureranking technique which selects the features that maximize the distance between the centroid of the cluster Ai (or Bi) and the centroid of the whole set of patterns belonging to the other class B (or A). The classification error of the proposed system is 4.8%; the confusion matrix is reported in table 1. Please note that the rows of table 1 do not sum up to 400; in fact, some ambiguous fingerprints are assigned to two classes in the database; in such cases, when any one of the two labels matches the output of our classifier, the result is assumed to be correct. 4. Learning of cluster-specific metrics: the similarity between an unknown pattern x and a given cluster Ai (or Bi), is computed thought a quadratic discriminant function (QDF [11]), whose parameters are tuned to maximize discrimination between the cluster itself and all the patterns belonging to the other class B (or A); all the patterns involved in the learning are first projected into the subspace corresponding to the cluster Ai (or Bi). 2. The experimentation was performed on NIST DB4 [34], which consists of 4000 fingerprint images from 2000 fingers, equally distributed in the five classes. Two different fingerprint instances (named ”F” and “S”) are present for each finger: the images are numbered from F0001 to F2000 and from S0001 to S2000. We used the 2000 images from the first 1000 fingers (F0001-F1000 and S0001-S1000) as a training set and we tested the system on the remaining 2000 images (F1001-F2000 and S1001-S2000). Table 1. Confusion matrix on NIST DB4. True Hypotized class class A L R W T 7 10 1 15 387 A Classification of an unknown pattern Given an unknown pattern x, its dimensionality is first reduced from d to r, then it is projected into the subspace of each cluster of 97 L 3 373 2 4 4 R 4 1 381 9 3 W 0 4 7 391 0 T 6 3 7 5 373 It is worth noting that many errors are due to the classification of some Tented arch fingerprints as Arch: in fact, these two classes have a substantial overlap and it is very difficult to separate them, even for human experts. For this reason, in some previous works, Arch and Tented arch are merged into a single class (four-class problem): in this case the error rate of the system decreases to 3.7%. four-class problem. The SPD classifier we adopted for the second stage proved to be a promising classification approach, although more experiments in other real-world pattern recognition problems are needed to better assess its effectiveness. 7. REFERENCES [1] Ballan M., Sakarya F.A. and Evans B.L., “A Fingerprint Since NIST DB4 contains an equal number of fingerprints for each class, some authors prefer to weight the results according to the natural class distribution (see section 1): in this case the error rate of the system is 5.0% and 3.4% for the five-class problem and the four class-problem, respectively. Classification Technique Using Directional Images”, Asilomar Conference on Signals, Systems, and Computers, 1997. [2] Bernard S., Boujemaa N., Vitale D. and Bricot C., “Fingerprint Classification Using Kohonen Topologic Map”, Proceedings International Conference on Image Processing ICIP'2001, Thessaloniki, Greece, 2001. Table 2 reports the results of published classification approach that were tested on NIST DB4 with the same training and test sets. The error rate of both the five-class and four-class problems are reported when available; each cell of the table reports two values: the error rate for the non-weighted case (on the left) and for the weighted case (on the right); “n.a.” indicates that the corresponding value was not published and it could not be derived from the confusion matrix. [3] J.C.Bezdek, Pattern Recogniton with Fuzzy Objective Function Algorithms, New York, Plenum, 1981. [4] Bowen J.D., “The Home Office Automatic Fingerprint Pattern Classification Project”, in proceedings IEE Coll. on neural network for image processing applications, 1992. [5] Candela G.T. et. al. “PCASYS - A Pattern-Level Classification Automation System for Fingerprints”, NIST tech. report NISTIR 5647, August 1995. Table 2. Error rates on NIST DB4 for the five-class problem and the four class-problem, in the non-weighted / weighted case. All the results are reported at 0% rejection, with the exception of the methods marked with (*), which discarded 1.8% fingerprints during feature extraction. Method Candela et al. 1995 [5] Jain et al. 1999 [20] (*) Cappelli et al. 1999 [9] Marcialis et al. 2001 [26] (*) Yao et al. 2001 [38] (*) Senior 2001 [32] Proposed system Five classes n.a. / n.a. 10.0% / 7.0% 7.9% / 6.5% 12.1% / 9.6% 10.7% / 9.0% n.a. / n.a. 4.8% / 5.0% [6] Cappelli R., Lumini A., Maio D. and Maltoni D., “Fingerprint Classification by Directional Image Partitioning”, IEEE Transactions on Pattern Analysis Machine Intelligence, vol.21, no.5, pp.402-421, May 1999. Four classes 11.4% / 6.1% 5.2% / n.a. 5.5% / n.a. n.a. / n.a. 6.9% / n.a. n.a. / 5.1% 3.7% / 3.4% [7] Cappelli R., Maio D. and Maltoni D., “A Multi-Classifier Approach to Fingerprint Classification”, Pattern Analysis and Applications Special Issue on Fusion of Multiple Classifiers, vol.5, no.2, pp.136-144, May 2002. [8] Cappelli R., Maio D. and Maltoni D., “Combining Fingerprint Classifiers”, in proceedings First International Workshop on Multiple Classifier Systems (MCS2000), Cagliari, pp.351-361, June 2000. The results of our two-stage classifier are extremely promising. Compared with the method proposed in [20] (which has analogous architecture: a k-NN classifier for the first stage and ten neural networks for the second stage), our system shows a 52% improvement for the five-class problem and a 29% improvement for the four-class problem; compared with the MKL classifier alone ([9]), the improvement is 39% and 33%, respectively. The SPD classifier proved to be well-suited to solve the ten two-class problems. Consider that, in [20], the output of the first stage (twomost-likely classes) was 98% correct and the neural networks in the second stage introduced an additional 8% error. In the proposed system, the first-stage accuracy is 99.1% and the additional error is just 3.9%. [9] Cappelli R., Maio D. and Maltoni D., “Fingerprint Classification based on Multi-space KL”, in proceedings Workshop on Automatic Identification Advances Technologies (AutoID’99), Summit (NJ), pp.117-120, October 1999. [10] Cappelli R., Maio D. and Maltoni D., “Multi-space KL for Pattern Representation and Classification”, IEEE Transactions on Pattern Analysis Machine Intelligence, vol.23, no.9, pp.977-996, September 2001. [11] Duda R and Hart P., Pattern Classification and Scene Analysis, Wiley, New York, 1973. [12] Cho B.H., Kim J.S., Bae J.H., Bae I.G. and Yoo K.Y., 6. CONCLUSIONS “Core-based Fingerprint Image Classification”, Int. Conf. Pattern Recognition (15th), Vol 2, pp. 863-866, 2000. We have proposed a fingerprint classification system based on a sequential (two-stages) architecture, where an MKL classifier finds the two-most-likely classes and a specifically trained SPD classifier is selected to make the final decision. The feature vector used by all the classifiers is derived from the fingerprint directional image. The system was tested on NIST DB4 and achieved an accuracy definitely better than that of all the fingerprint classification approaches we are aware of, yielding an error rate of 4.8% for the five-class problem and 3.7% for the [13] Chong M.M.S. et. al., “Geometric Framework for Fingerprint Image Classification”, Pattern Recognition, v. 30, no. 9, pp. 1475-1488, 1997. [14] Galton F., Finger Prints, Mcmillan, London, 1892. [15] Halici U. and Ongun G., “Fingerprint Classification Through SelfOrganizing Feature Maps Modified to Treat 98 Uncertainties”, Proceedings IEEE, Vol. 84, No. 10, pp. 1497-1512, 1996. [28] Moayer B. and Fu K.S., “An application of stochastic languages to fingerprint pattern recognition”, Pattern Recognition, vol. 8, pp. 173-179, 1976. [16] Henry E.R., Classification and Uses of Finger Prints, Routledge, London, 1900. [29] Moscinska K. and Tyma G., “Neural Network based Fingerprint Classification”, in proceedings 3rd Int. Conf. on Neural Network, pp. 229-232, 1993. [17] Hong L. and Jain A.K., “Classification of Fingerprint Images”, Scandinavian Conf. (11th) on Image Analysis, June 7-11, Kangerlussuaq, Greenland, 1999. [30] Neto H.V. and Borges D.L., “Fingerprint classification with neural networks”, IVth Brazilian Symposium on Neural Networks, pp. 66-72, 1997. [18] Hughes P.A. and Green A.D.P., “The use of Neural Network for Fingerprint Classification”, in proceedings 2nd Int. Conf. on Neural Network, pp. 79-81, 1991. [31] Rao K. and Balck K., “Type classification of fingerprints: A syntactic approach”, IEEE Trans. on Pattern Analysis Machine Intelligence, v. 2, no. 3, pp.223-231, 1980. [19] Jain A.K., Prabhakar S. and Hong L., “A Multichannel Approach to Fingerprint Classification”, ICCVGIP'98, pp. 153-158, Dec 21-23, 1998, Delhi, India. [32] Senior A., “A Combination Fingerprint Classifier”, IEEE Transactions on Pattern Analysis Machine Intelligence, vol. 23, no. 10, pp. 1165-1174, 2001. [20] Jain A.K., Prabhakar S. and Hong L., “A Multichannel Approach to Fingerprint Classification”, IEEE Transactions on Pattern Analysis Machine Intelligence, Vol.21, No.4, pp. 348-359, April 1999. [33] Stock R.M. and Swonger C.W., “Development and evaluation of a reader of fingerprint minutiae”, Cornell Aeronautical Laboratory, Technical Report CAL no. XM2478-X-1:13-17, 1969. [21] Kamijo M., “Classifying Fingerprint Images using Neural Network: Deriving the Classification State”, in proceedings 3rd Int. Conf. on Neural Network, pp. 1932-1937, 1993. [34] Watson C.I. and Wilson C.L., “Nist Special Database 4, Fingerprint Database”, U.S. National Institute of Standards and Technology, 1992. [22] Karu K. and Jain A.K., “Fingerprint Classification”, Pattern Recognition, v. 29, no. 3, pp. 389-404, 1996. [35] Wegstein J.H., “An automated fingerprint identification [23] Kawagoe M. and Tojo A., “Fingerprint Pattern system”. U.S. National Institute of Standards and Technology, NBS Special Publication 500-89, February 1982. Classification”, Pattern Recognition, v. 17, no. 3, pp. 295303, 1984. [24] Lumini A., Maio D. and Maltoni D., “Inexact Graph [36] Wei D., Yuan Q. and Jie T., “Fingerprint Classification Matching for Fingerprint Classification”, Machine GRAPHICS & VISION Special Issue on Graph Trasformations in Pattern Generation and CAD, vol.8, no.2, pp.231-248, September 1999. System with Feedback Mechanism Based on Genetic Algorithm”, Int. Conf. Pattern Recognition (14th), 1998. [37] Wilson C.L., Candela G.T. and Watson C.I., “Neural Network Fingerprint Classification”, Journal of Artificial Neural Networks, Vol. 1, No. 2, pp. 203-228, 1994. [25] Maio D. and Maltoni D., “A Structural Approach to Fingerprint Classification”, Int. Conf. (23th) on Pattern Recognition, Vienna (Austria), Aug. 1996. [38] Yao Y., Frasconi P. and Pontil M., “Fingerprint Classification with Combination of Support Vector Machines”, Int. Conf. (3rd) on Audio- and Video-Based Biometric Person Authentication, pp. 253-258, Sweden, June 6-8, 2001. [26] Marcialis G.L., Roli F. and Frasconi P., “Fingerprint Classification by Combination of Flat and Structural Approaches”, Int. Conf. (3rd) on Audio-and Video-Based Biometric Person Authentication, pp. 241-246, Sweden, June 6-8, 2001. [27] Moayer B. and Fu K.S., “A Syntactic Approach to Fingerprint Pattern Recognition”, Pattern Recognition, v. 7, pp. 1-23, 1975. 99