Research Manual Fingerprint Recognition Supervisor: Nigel Whyte Student: Dayu Chen Student ID: C00131022 Submit date: 2010-11-26 Content Abstract ..................................................................................................................................... 1 1. Introduction ....................................................................................................................... 1 2. Existing Fingerprint recognition system ............................................................................ 2 3. 2.1. Forensic Latent Criminal Justice AFIS-Lite System ..................................................... 2 2.2. CUBS Matlab Toolbox for Fingerprint Verification..................................................... 2 2.3. Fingerprint recognition demo software ..................................................................... 4 Fingerprint acquisition....................................................................................................... 4 3.1. Getting fingerprint with digital camera...................................................................... 5 3.2. Getting fingerprint with fingerprint scanner.............................................................. 7 3.2.1. Portable Dual Fingerprint Live Scanner .............................................................. 7 3.2.2. Secugen Hamster plus - USB Fingerprint Scanner .............................................. 8 3.3. 4. 5. Brief description of fingerprint image type ................................................................ 8 Pre-processing algorithm .................................................................................................. 9 4.1. Normalization ............................................................................................................. 9 4.2. Orientation estimation ............................................................................................. 10 4.3. Segmentation ........................................................................................................... 12 4.3.1. Normal method of image segmentation (using Histogram-based methods) .. 12 4.3.2. Edge detection .................................................................................................. 12 4.4. Ridge extraction ....................................................................................................... 15 4.5. Thinning .................................................................................................................... 16 Recognition (Feature extraction and matching) ............................................................. 17 5.1. Fingerprint minutiae representation ....................................................................... 17 5.2. Minutiae extraction .................................................................................................. 19 5.2.1. Using a 3X3 template mask base on the binary thinned fingerprint image ..... 19 5.2.2. Using 3X3 mask base-on neural network ......................................................... 20 5.3. Minutiae Matching ................................................................................................... 22 5.3.1. Approach based on alignment.......................................................................... 22 5.3.2. Approach based on local minutiae ................................................................... 23 5.4. 6. 7. Additional problem .................................................................................................. 24 5.4.1. False minutia removal ...................................................................................... 24 5.4.2. Classifier............................................................................................................ 25 Implementation ............................................................................................................... 25 6.1. Programming language ............................................................................................ 25 6.2. Database................................................................................................................... 26 Conclusion ....................................................................................................................... 26 7.1. My fingerprint recognition system........................................................................... 26 7.2. Scanner ..................................................................................................................... 27 7.3. Pre-processing .......................................................................................................... 27 7.4. Recognition .............................................................................................................. 28 7.5. Implementation........................................................................................................ 29 Reference ................................................................................................................................ 30 Abstract Fingerprint is an approach for human identification, because of its uniqueness. Fingerprint recognition is using for compare two fingerprints, in order to make sure that these fingerprints came from the same finger. This report is written for researching some topics about fingerprint recognition system. It includes researching exist application, fingerprint scanner, fingerprint image pre-processing algorithm, recognition method and some implementation techniques (e.g. programming language, database and tool). 1. Introduction Fingerprint is one of the most well-know and publicized biometrics to identify a human, because it is unique. Different people have different fingerprints. Fingerprint has been used as an identification approach for a long time. Fingerprint recognition means provide an automated method of verifying a match between two human fingerprints. Today, Fingerprint recognition is widely used in human life. For example, in security identify a human for accessing a building or accessing a system. Some personal laptop provides a fingerprint recognition function to allow a user login. History of fingerprint recognition [10] [25] Fingerprint has been used for human identification for a long time. Human used fingerprint as their signatures. By 246 BCE, Chinese officials impressed their fingerprints into the important documents. [25] The modern fingerprint recognition techniques were invited in the late 16th century. Henry Fauld, a man who first scientifically suggested the individuality and uniqueness of fingerprints. At the same time, Herschel asserted that he had practiced fingerprint identification for about 20 years. After this, foundation theory of modern fingerprint identification had been established in this century. In the late 19th century, Sir Francis Galton conducted an extensive study of fingerprints. He provided the features for fingerprint classification in 1888. This is the earliest identification of fingerprint feature. The discovery of uniqueness of fingerprints led to fingerprints recognition method invitation. In 1899, Edward Henry provided an advance method in fingerprint identification. This is an important method in fingerprint recognition history. We called it “Henry system”. In the early 20th century, fingerprint recognition was formally accepted as a personal identification method by many countries. For example: Garda station, law enforcement agencies. Fingerprints became a standard procedure in forensics. Today, fingerprint recognition is widely used. It not only be used in law enforcement, but also be used in personal security (e.g. some personal laptop includes fingerprints recognition system for access verification. Some buildings also have fingerprint recognition for accessing. Some school using fingerprints recognition system in exam 1 for student verification). 2. Existing Fingerprint recognition system Fingerprint recognition system is a computer application that executes fingerprint recognition processing. This system’s main technique includes image processing and Pattern recognition. There are two main functions in fingerprint recognition: the first one is that: user can store fingerprint information to a database; the fingerprint is processed before storing. Another one is that: user can compare two fingerprints (one is stored in database, and one is input by fingerprint device. All fingerprints have been processed before.) System will show the result to user. There is some similar applications about fingerprint recognition system: 2.1. Forensic Latent Criminal Justice AFIS-Lite System[1] Forensic Latent Criminal Justice AFIS-Lite System is a multi-tasking fingerprint identification system for the low volume user. In this software, fingerprint image is captured by a high resolution flat bed scanner. (Figure 1) Figure 1: Forensic Latent Criminal Justice AFIS-Lite System Functions of Forensic Latent Criminal Justice AFIS-Lite: Capture fingerprint image from a scanner. It is a high quality image. Central Server control Functions: Fingerprint is stored with database, and user can manage the system data in database (e.g. searching, deleting and updating). Image Matcher: Fingerprint recognition and searching the database. Workstation Functions: Image processing and encoding. Analyst review (identify/Non-Identify determination) 2.2. CUBS Matlab Toolbox for Fingerprint Verification[2] CUBS Matlab Toolbox for Fingerprint Verification is a fingerprint recognition application that contains matlab interface. This application provides two functions only (Feature Extraction and Fingerprint recognition); each function has an independent interface. 2 Functions of CUBS Matlab ToolBox: Feature Extraction: In this function, system provides a user interface for fingerprint feature extraction. System will do the pre-processing and highlight the features in the user interface. (Figure 2) It also allows user to manually identify new minutiae or remove spurious ones. Figure 2 CUBS Matlab Toolbox for Fingerprint Verification: ToolBox Fingerprint recognition: In this function, system also will provide a user interface for fingerprint recognition. System does the fingerprint recognition, and shows the result to user. Different type of features will be highlighted with different colors (Figure 3) Figure 3 CUBS Matlab Toolbox for Fingerprint Verification: ToolBox 3 2.3. Fingerprint recognition demo software [3] Fingerprint recognition demo software is fingerprint recognition application. Its function includes the whole processing about fingerprint recognition (Figure 4). Figure 4 Fingerprint recognition demo software Functions of fingerprint recognition demo software: Capture fingerprint: System can capture fingerprint from optical scanner. It also can get the fingerprint from the external file. Pre-process the fingerprint image: system can do the pre-processing and feature extraction. System will show the feature to user in the user interface. Saving fingerprint: System can save the fingerprint. Recognition: Fingerprint verification and identification. 3. Fingerprint acquisition How to get a fingerprint image is the first question in this system. This is the basic step of fingerprint recognition. System will not executing without a fingerprint image. There are some different ways to get the fingerprint: optical, capacitive, ultrasound and thermal. Optical sensors are the most common sensor today. This section will focus on two approaches about fingerprint acquisition: One is using digital camera, another one is using fingerprint scanner. 4 3.1. Getting fingerprint with digital camera In this approach, I researched an essay; it provides a function to capture fingerprint images with a mobile camera. I think it can make a low project cost, and getting the fingerprint image more easily than getting from scanner, but it also has a problem. It cannot do preprocessing with the traditional preprocessing algorithm. “Some characteristics of fingerprint images captured with mobile cameras are quite different from those obtained by conventional touch based sensors. For example, mobile camera images are colored while conventional touch-based sensors produce only black and white images. Also, the backgrounds, or non-finger regions, in mobile camera images are very erratic depending on how the image captures place and time ” [4]. I think it will make more complex for the project. I also did some tries in this approach (Figure 5 and Figure 6). Fingerprint is not very clear in this approach, so I think it is very difficult to do the pre-processing with these images. Figure 5 getting fingerprint with digital camera 5 Figure 6 getting fingerprint with digital camera Extension [5] There is an interesting way to get the image with a digital camera. I can get a black-white image (Figure 7). It has six steps: 1, rub the side of a pencil on a piece of paper. 2 rub finger on the patch where you rubbed with the pencil. 3 get a friend to get a piece of tape. 4 put finger on the tape. 5 take the tape off. 6 put the tape on a paper, and then take a photo with a camera. I don’t like this way. It is too childish; however it can get a nice image. Figure 7 Extension way: Getting fingerprint with digital camera 6 3.2. Getting fingerprint with fingerprint scanner Getting fingerprint with fingerprint scanner is a popular approach. It can get a standard image for pre-processing, but it leads to a high project cost. I have researched some different fingerprint scanner. In my opinion, fingerprint scanner should include these goals, so it is suitable for my project: Getting a fingerprint image. Image is un-preprocessing: I found some fingerprint scanners had embedded a recognition system, I think these kinds of scanners are not suitable for my project. Can link to computer: I am developing computer software, so it is necessary to link to computer. Not too big Cheap There are two fingerprint scanners which I have researched: 3.2.1. Portable Dual Fingerprint Live Scanner (Figure 8) [6] This scanner is small and it can link to computer. There are some features which are provided in its website: Auto capture by sensing finger placement Slippage detection for rolls Halo effect and ghost image elimination Image quality check Automatic segmentation of two finger slaps Sequence check in rolls and slap acquisition International standard image compression, format and interfaces Figure 8 Portable Dual Fingerprint Live Scanner Figure 9 is an image example of this scanner. 7 Figure 9: sample image Description of Portable Dual Fingerprint Live Scanner I think this scanner is expensive, however it is small, can link the computer with USB. It also has some strong point. For example it supports more than one fingerprint be scanned at the same time. 3.2.2. Secugen Hamster plus - USB Fingerprint Scanner (Figure 10) [7] This scanner incorporates the latest in optical sensor capability, using patented SEIR technology, to provide a robust and highly reliable biometric platform that can be used in applications supporting biometric authentication, verification and scanning. There are two features which I preference: “Free Demo application to capture and save fingerprint as an image file” and “supports: Fingerprint recognition, fingerprint verification, fingerprint authentication, fingerprint scanning & fingerprint matching applications”. Figure 10 Secugen Hamster plus - USB Fingerprint Scanner Description of Secugen Hamster plus - USB Fingerprint Scanner This scanner is small and USB connection. It looks like more suitable for my project. It is not expensive, but it has a weakness point. It seems only can capture one fingerprint at the same time. 3.3. Brief description of fingerprint image type After research, I found that most type image can be pre-processing. Typically, JPEG and BMP are more popular. Some application use JPEG and some use BMP. There is a special type; it also can be used in fingerprint recognition. It is SVG [8]. With this reason, I think I also can get the fingerprint image source from the internet. I think I also need to think about the pixels of an image, because it has relationship 8 with image data. Almost fingerprint scanner has their output standard. We don’t need to consider about the image compression. Almost fingerprint scanner will do it automatically. 4. Pre-processing algorithm Pre-processing is the second step in Fingerprint recognition system. It is the most important step in this system. The fingerprint image is not suitable for recognition after we get the image from fingerprint device. There are some reasons will lead to this problem: When person is scanning finger, pressure of the finger will lead to an unclear image. If the finger is not clean, it also will make some noisy points in the fingerprint image. Per-processing is a process that making the image easy for recognition. (E.g. cleaning noisy point and making a clear image) Description of fingerprint image representation: Typically, a fingerprint image includes two parts: Foreground and background. Foreground means the part which includes clear ridge. In background, there is no ridge or the ridge is unclear. We cannot avoid the background. Each fingerprint image will have these two parts. Background makes a big influence to fingerprint recognition. The main task of pre-processing is cleaning noisy point of the image and separate background and foreground. The traditional operation of fingerprint preprocessing includes: Normalization, Orientation estimation, segementation, ridge extraction and thinning. 4.1. Normalization[9] Fingerprint image is captured by fingerprint scanner. This kind of image only has two colors (black and white), but gray level in this image is not the same. (Because pressure of finger or some reason from fingerprint scanner.) It is necessary to normalize the gray level of the image. With this step, we can make a normalize image for the following operation. The normalized image is defined as the follows mathematics formula (Figure 11): Figure 11 Normalization mathematics formula In this formula, I(I,j) means the gray level of point (i, j). M0 and VAR0 are the desired mean and variance values. “Normalization is a pixel-wise operation. It does not 9 change the clarity of the ridge and valley structures. The main purpose of normalization is to reduce the variations in gray-level values along ridges and valleys, which facilitates the subsequent processing steps.” [9] Figure 12 shows an example of image normalization. Figure 12: Sample of normalization. (a) Input image. (b) Normalized image 4.2. Orientation estimation[10] An orientation field represents the directionality of ridges in the fingerprint image. It is a very important role in fingerprint image analysis. This step is a basic step for minutiae extracted. It also prepare for image segmentation. “Fingerprint image is typically divided into a number of non-overlapping blocks (e.g. 32x32 pixels) and an orientation representative of the ridges in the block is assigned to the block based on an analysis of grayscale gradients in the block. The block orientation could be determined from the pixel gradient orientations based on, say, averaging, voting, or optimization.” [10] The following steps show the processing of orientation estimation: Divide the input fingerprint image into blocks of size WxW. Compute the gradients Gx and Gy at each pixel in each block. Estimate the local orientation at each pixel (i, j) using the equations in figure 13: Figure 13 Formula of local orientation estimation at each pixel 10 Where W is the size of the local window; Gx and Gy are the gradient magnitudes in x and y directions, respectively. Compute the consistency level of the orientation field in the local neighborhood of a block(i, j) with the following formula in figure 14: Figure 14 Formula for compute the consistency level of the orientation filed If the consistency level is above a certain threshold Tc, then the local orientations around this region are re-estimated at a lower resolution level until C(i, j) is below a certain level. This is the algorithm of orientation estimation. [10] Figure 15 shows the orientation field: Figure 15 orientation field after orientation estimation 11 4.3. Segmentation Image segmentation is a basic way for fingerprint image enhancement. We cannot extract features from a fingerprint image without image enhancement, because without image segmentation, some important features will not present clearly, some unimportant features will present, some features maybe present twice. All these will lead to a difficult feature extraction. Segment is a way for keeping the useful image information and removes the un-useful image information. Image segmentation is typically used to locate objects and boundaries in images. 4.3.1. Normal method of image segmentation (using Histogram-based methods) There are some methods for image segmentation: Clustering methods, compression-based methods, Histogram-based methods etc. In these methods, Histogram-based method is the most efficient method. “Histogram-based methods are very efficient when compared to other image segmentation methods because they typically require only one pass through the pixels. In this technique, a histogram is computed from all of the pixels in the image, and the peaks and valleys in the histogram are used to locate the clusters in the image. Color or intensity can be used as the measure.”[18] In this method, image has been divided into several blocks. Using a gray level wavelet histogram to presents each block gray level, so that, we can determine how many blocks are useful (How many blocks are in the accepting gray level). With this method, we can keep the useful information part in the image, but it has a disadvantage. Its disadvantage is that it may be difficult to identify significant peaks and valleys in the image. In this technique of image classification distance metric and integrated region matching are familiar. 4.3.2. Edge detection Edge detection is another way for image segmentation. It is more complex, but more accurate. “Edge detection is a fundamental tool in image processing and computer vision, particularly in the areas of feature detection and feature extraction, which aim at identifying points in a digital image at which the image brightness changes sharply or more formally has discontinuities”[11] In fingerprint, edge is the boundary between two regions with relatively distinct gray level properties. Through edge detection, we can reduce amount of data and throw away information which is not used for fingerprint analysis. It also can keep the information which is important for image. The idea underlying most edge-detection techniques is on the computation of a local derivative operator such as “Roberts”, “Prewitt”, “Canny” or “Sobel” operators. 12 “In practice, the set of pixels obtained from the edge detection algorithm seldom characterizes a boundary completely because of noise, breaks in the boundary and other effects that introduce spurious intensity discontinuities. Thus, edge detection algorithms typically are followed by linking and other boundary detection procedures designed to assemble edge pixels into meaningful boundaries.”[12] “Edge detection is a fundamental tool in image processing and computer vision, particularly in the areas of feature detection and feature extraction, which aim at identifying points in a digital image at which the image brightness changes sharply or more formally has discontinuities”[11] In fingerprint, edge is the boundary between two regions with relatively distinct gray level properties. Through edge detection, we can reduce amount of data and throw away information which is not used for fingerprint analysis. It also can keep the information which is important for image. The idea underlying most edge-detection techniques is on the computation of a local derivative operator such as “Canny” operators. “In practice, the set of pixels obtained from the edge detection algorithm seldom characterizes a boundary completely because of noise, breaks in the boundary and other effects that introduce spurious intensity discontinuities. Thus, edge detection algorithms typically are followed by linking and other boundary detection procedures designed to assemble edge pixels into meaningful boundaries.”[12] Figure 16 an example of a 5X5 Gaussian filter Finding the intensity gradient of the image: An edge in an image may point in a variety of directions, so the Canny algorithm uses four filters to detect horizontal, vertical and diagonal edges in the blurred image. The edge detection operator (Roberts, Prewitt, Sobel for example) returns a value for the first derivative in the horizontal direction (Gy) and the vertical direction (Gx). From this the edge gradient and direction can be determined: 13 Figure 17 Formula of Canny detector; determine the edge gradient and direction The edge direction angle is rounded to one of four angles representing vertical, horizontal and the two diagonals.”[13] 1. Gabor filter: Gabor filter is a linear filter used for edge detection. It named after Dennies Gabor. It has been found to be particularly appropriate for texture representation and discrimination. Some people used it in fingerprint preprocessing [10]. Its basic theory is that: “Its impulse response is defined by a harmonic function multiplied by a Gaussian function. Because of the multiplication-convolution property (Convolution theorem), the Fourier transform of a Gabor filter's impulse response is the convolution of the Fourier transform of the harmonic function and the Fourier transform of the Gaussian function. The filter has a real and an imaginary component representing orthogonal directions the two components may be formed into a complex number or used individually. Real (Figure 18) Figure 18 Gabor filter basic formula Imaginary (Figure 19) Figure 19 imaginary formula Where (Figure 20) Figure 20 compute the x coordinate And (Figure21) Figure21 compute the y coordinate In this equation, λ represents the wavelength of the sinusoidal factor, θ represents the orientation of the normal to the parallel stripes of a Gabor function, ψ is the phase offset, σ is the sigma of the Gaussian envelope and γ is the spatial aspect ratio, and specifies the ellipticity of the support of the Gabor function.”[14] 14 Conclusion of edge detection algorithm: These two algorithms can be used in edge detection both. Canny edge detector is traditional way and Gabor filter seems more suitable for do the edge detection in the image which includes character only, but some essays provide using Gabor filter in fingerprint edge detection, so I decide to use Gabor filter in my program. 4.4. Ridge extraction After edge detection, we can get an image that: reduce amount of un-useful data. We can extract the ridge of fingerprint. The approaches to ridge extraction use either simple or adaptive threshold. The important property of the ridges in a fingerprint image is that gray level values on ridges. There is an algorithm was provided base on gray level threshold. This algorithm is image binarization [15]. With this way, we can separate the foreground (Fingerprint part) and the background. The theory of this algorithm is that: calculate a gray level threshold, compare this threshold to each pixel of the fingerprint image. Figure 22 is the formula of this algorithm. Figure 22 formulas for binarization Where iVal is point i, Gmean[I] is this point’s gray level, Gmean[iVar] is the gray level threshold. With this way, the foreground’s gray level will be set in 1, and background’s gray level will be set in 0. The ridge can be extracted. After this, fingerprint image become a binary image. Figure 23 is an example of result image. Figure 23 result of binarization 15 4.5. Thinning After ridge extraction, the image still cannot use for minutiae extraction, because the ridge is too wide, it is not suitable for extract feature points. We have to make the ridge thin. Thinning is to representing the structural shape of a plane region is to reduce it to a graph. This reduction may be accomplished by obtaining the skeleton of the region via thinning. There is a thinning algorithm called Zhang-Suen algorithm. [16] In this algorithm, using a 3X3 window to move down throughout the image and calculations are carried out on each pixel. Figure 24 is the 3X3 windows. Figure 24 3X3 windows in thinning algorithm In figure 24, the classification given to the pixels that surround the center pixel. The algorithm runs two sub-iterations continuously until the image reaches a stable state. Pseudo Code for Zhang-Suen Thinning: Let A (P) be the number of 01 patters in the order set P2 to P9 Let B (P) be the number of non-zero neighbors of P Do until image is stable (i.e. no changes made) Sub-iteration 1: If 2<= B (P) <=6 or A (P) = 1 or P2*P4*P6 = 1 or P4*P6*P8 = 1 Delete P Sub-iteration 2: If 2<= B (P) <=6 or A (P) = 1 or P2*P4*P8 = 1 or P2*P6*P8 = 1 Delete P Figure 25is the result of thinning. It this thinning image, each line made of single point only. Figure 25 result of thinning 16 5. Recognition (Feature extraction and matching) After fingerprint image pre-processing, a fingerprint image is prepared for feature extraction. Feature extraction and matching is the last step in fingerprint recognition. It is the basic function of fingerprint recognition system. Before we extract the fingerprint minutiae, we should know some information about fingerprint minutiae. 5.1. Fingerprint minutiae representation Fingerprint minutiae present the uniqueness of a fingerprint. Fingerprint feature includes global feature and local minutiae. Global feature is the type of fingerprint patterns. There are three many patterns types: arch, loop and whorl. “An arch is a pattern where the ridges enter from one side of the finger, rise in the center forming an arc, and then exit the other side of the finger. The loop is a pattern where the ridges enter from one side of a finger, form a curve, and tend to exit from the same side they enter. In the whorl pattern, ridges form circularly around a central point on the finger. Scientists have found that family members often share the same general fingerprint patterns, leading to the belief that these patterns are inherited.” [17] Some recognition algorithms are pattern-based. In these algorithms, it compares fingerprints by pattern recognition. Local minutiae means local discontinuities in the fingerprint pattern (e.g. ridge ending). Most fingerprint systems for fingerprint comparison are based on minutiae matching. A total of 150 different minutiae types have been identified. They include termination (ending), bifurcation, lake and dot (island). “The major Minutia features of fingerprint ridges are: ridge ending, bifurcation, and short ridge (or dot). The ridge ending is the point at which a ridge terminates. Bifurcations are points at which a single ridge splits into two ridges. Short ridges (or dots) are ridges which are significantly shorter than the average ridge length on the fingerprint. Minutiae and patterns are very important in the analysis of fingerprints since no two fingers have been shown to be identical.”[17] In practice, most fingerprint recognition systems only consider two types of minutiae: ridge ending and ridge bifurcation. A ridge ending is defined as the point where a ridge ends abruptly. A ridge bifurcation is defined as the point where a ridge forks or diverges in to branch ridges (Figure 26) [9]. 17 Figure 26 ridge ending and ridge bifurcations Typically, local minutiae can be identified by its local position (X, Y coordinate) and its orientation (angel). The local position and orientation present like Figure 27. Figure 27 local position and orientation of minutiae After we know the basic information in minutiae, we also have to know the basic information in minutiae matching: How many minutiae should be matched? How to know matching is completed? These two questions is very important, they are very difficult. There is not a standard answer for these questions in my research. After research, I found some opinions about these questions: 1. “A good quality fingerprint typically contains about 40-100 minutiae.”[9] This opinion provides to matching all minutiae, and then gets a result to show how many percent of minutiae is the same. 2. “Average of 120 to 150 points” will be matched on a complete fingerprint [19]. This opinion has not answer my second question. I think its answer of my second question must be similar with the first opinion, because it provides me match all minutiae in fingerprint recognition. 3. “Fingerprint image contains about 150 minutiae’s. When we compare two fingerprint images, we compare latent and non latent fingerprint and we try to 18 find 12 minutiae’s placed on the same position on latent and non latent fingerprint images.”[20] This opinion shows me a clear answer. 4. “Template is extracted and checked if any 43 minutiae points are same if so he is authenticated.” [21] In this opinion, 43 are more than 12. 5. “When multiple sets of prints are being compared, a certain number of characteristic points must be a match. Worldwide there is no set number of how many points must match before it is accepted as a match, but it does differ in some countries. For example, in the Netherlands 12 points are required, whereas in Africa only 7 points are required. It also differs within some countries too. Paris has a requirement of 17 points, whereas the rest of France only requires 12.”[26] In this example, European countries are 12 minutiae points required, but there is different in some area (e.g. in Paris, requirement is 17, but another area of France, requirement is 12). In my opinion, in European standard, system needs to compare 12 minutiae points in common between two fingerprints. I prefer to follow this standard. 5.2. Minutiae extraction After pre-process, a fingerprint image is prepared for minutiae extraction. A fingerprint is characterized by a pattern of interleaved ridges (dark lines) and valleys (bright lines). Generally, ridges and valleys run in parallel and sometimes they terminate or they bifurcate. At a global level, the fingerprint may present regions with patterns of high curvature; these regions are also called singularity. At the local level, other important feature called minutia can be found in the fingerprint patterns. Minutia mean small details and this refers to the behavior of the ridges discontinuities such as termination, bifurcation and trifurcation or other features such as pores (small holes inside the ridges), lake (two closed bifurcations), dot (short ridges), etc. I prefer to match the minutiae for comparing, so that, I will only describe the minutiae extraction in local level. There are two algorithms for minutiae extraction. 5.2.1. Using a 3X3 template mask base on the binary thinned fingerprint image As we know, in fingerprint image pre-process, we have two operations: one is binarization (for ridge detection), another one is thinning. After this, we can get a binary thinned fingerprint image. We use a 3X3 template mask (Figure 28) to extract the minutiae. The main process of this algorithm is that: In figure 28, (x, y) is a point which is waiting for determine. N1 to N8 is 8 points which nearby the point. Searching this mask from top to down, left to right; and then calculating the translation of “0” and “1”. Rules: If the translation count is 6 Point(x, y) is a bifurcation minutia; 19 If the translation count is 2 Point(x, y) is an ending minutia; Else Point(x, y) is not minutia; Figure 28 3X3 template masks After this process, the information of minutiae will be extracted. The information contains coordination of minutiae and direction (angle) of minutiae (Direction we get it from pre-process operation: orientation estimation). This algorithm will more accurately after image thinning. Some kind of false minutiae will be omitted with this algorithm. 5.2.2. Using 3X3 mask base-on neural network [22] This algorithm is base on neural network algorithm. The minutiae are detected by using 3X3 masks. All this masks used for identifying the ridge ending and bifurcations point (Figure 29). Before minutiae extraction, we have to train the network will all the masks. Once the neural network is trained, next step is to input the prototype fingerprint image to extract the minutiae, using these masks to scan the fingerprint image. 20 Figure 29 Patterns are for neural network training. “An artificial neural network (ANN), usually called "neural network" (NN), is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes information using a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network during the learning phase. Modern neural networks are non-linear statistical data modeling tools. They are usually used to model complex relationships between inputs and outputs or to find patterns in data.”[23] In this case, 24 masks are the neurons for learning. This algorithm is the traditional algorithm in minutiae extraction. 21 These two algorithms can extract minutiae. The result will be showed as Figure 30. Figure 30 left image is minutiae extraction, right image is contains direction 5.3. Minutiae Matching Given two set of minutia of two fingerprint images, minutia matching is that determines whether these two minutia sets are from the same finger or not. In matching, we will not only compare these two set of minutia only. In a fingerprint image, there may have some problems: fingerprints may be shift, rotate and stretched. There are two approaches to solve these problems: 5.3.1. Approach based on alignment There approach will solve the alignment problem at first. It includes two parts: Alignment stage and match part. In alignment part: “Given two fingerprint images to be matched, choose any one minutia from each image; calculate the similarity of the two ridges associated with the two referenced minutia points. If the similarity is larger than a threshold, transform each set of minutia to a new coordination system whose origin is at the referenced point and whose x-axis is coincident with the direction of the referenced point.”[22] In match part: “After the set of transformed minutia points is derived, the elastic match algorithm is used to count the matched minutia pairs by assuming two minutia having nearly the same position and direction are identical.”[22] 22 5.3.2. Approach based on local minutiae This approach will try to make sure the local position at first, and then to match two fingerprints’ minutiae. Typically, this method will determine a reference minutia; calculate the distance from the reference minutia to another minutia (we can call it m), so that we can make sure the local position of minutia m. There is an evolution method: For each extracted minutia, a circular neighborhood S of radius R (e.g. R=50 pixels) for this minutia is defined. Each minutia in this circle is in the centre position, this minutia is called central minutia. In this circle, there may have n (n>=0) minutiae, there minutiae are called local minutiae. (Figure 31) Figure 31 example of circular neighborhood Rule of this method: [27] “To facilitate the matching between the source and test prints, two classes of vectors, vector 1 and vector 2, are defined for each minutia. Vector 1 is a two-component vector containing the minutia type (endpoint or bifurcation) and the minutia direction of the central minutia. Vector 2 is a four-component vector, which contains the minutia type and direction of each local minutia, and the distance and relative angle between the central minutia and each local minutia. Each minutia has exactly one vector 1 and n vector 2 for n local minutiae. Calculate the similarity of each minutia.” 23 5.4. Additional problem 5.4.1. False minutia removal The pre-processing stage does not totally heal the fingerprint image. All the earlier stages themselves occasionally introduce some artifacts which later lead to spurious minutia. Figure 32 is some types of false minutia. Figure 32 False Minutia structures “Procedures to remove false minutia are: a) If the distance between one bifurcation and one termination is less than D and the two minutiae are in the same ridge (m1 case). Remove both of them. D is the average inter-ridge width representing the average distance between two parallel neighboring ridges. b) If the distance between two bifurcations is less than D and they are in the same ridge, remove the two bifurcations (m2, m3, m8, m10, m11 cases). c) If two terminations are within a distance D and their directions are coincident with a small angle variation. And they suffice the condition that no any other termination is located between the two terminations. Then the two terminations are regarded as false minutia derived from a broken ridge and are removed (case m4, m5, m6). d) If two terminations are located in a short ridge with length less than D, remove the two terminations (m7). e) If a branch point has at least two neighboring branch points, which are each no further away than maximum distance threshold value and these branch points are closely connected on common line segment than remove the branch points (m12).”[22] 24 5.4.2. Classifier Fingerprint classification means classified into categories base on information in the global patterns of ridges. It is helpful for information storage. In matching, some fingerprint image maybe rotated. Core and delta can help to rotate the image. The basic step for classification is singularity detection. Singularity also can be used for alignment purpose used during the matching. A simple way to compute the singularities is to use the Poincare method. [24] This method is quite similar with minutiae extraction. “For each point i,j of the orientation map are computed the Poincare index P(i,j) as the sum of the difference between the orientation of the point i,j and their neighbors.”[24] The pattern like Figure 33. Figure 33 mask for core extraction Depending of the value of P (i, j): If sum of P (i, j) equals to 360 or 180 P is core If sum of P (i, j) equals to -180 P is delta 6. Implementation This part is a brief description of some available programming techniques. 6.1. Programming language There are two available programming language I preference: C++ and Java. In C++, It is an object-oriented programming language. C++ is running in windows operating system only, but it is running without virtual machine, so it is more efficiently. C++ application is easier to control the fingerprint scanner running within windows operating system. In Java, It is an object-oriented programming language too. It is similar with C++, but Java has to run in virtual machine, so it is slower than C++. 25 6.2. Database In fingerprint recognition system, data have to storage. I use database to store data. Fingerprint is not only stored image but also store some minutiae information. For example, coordinate, direction, type (e.g. ridge ending or bifurcation). There are three kind of databases I preference: mySQL, Microsoft Access, Java DB. mySQL: A very useful database. It is through the Highly Optimized. It suits to almost project, but user have to install it before use it. Microsoft Access: It is a desktop relational database, suitable for small-scale development and a small amount of data applications Java DB: A database running in Java virtual machine. Only Java application can control it. 7. Conclusion This part is a conclusion for each part which I describe above. 7.1. My fingerprint recognition system After research some similar application. I have a conclusion of my fingerprint recognition system. My fingerprint recognition system probably should include these functions base on research the similar applications: 1. Capture fingerprint: fingerprint image is captured from fingerprint scanner. It also can load fingerprint from an external image 2. Pre-processing: pre-process the fingerprint image, find out the feature and then show them to user. 3. Fingerprint database management: User can save the fingerprint information in database and manage it (adding, deleting and updating). 4. Recognition: Compare two fingerprints (one is from database, another one is from scanner). General processing of Fingerprint recognition: Firstly, getting a fingerprint image from a device: This is the first step in fingerprint recognition. Pre-processing the fingerprint image: pre-processing the fingerprint image base on the pre-processing algorithm. Through this processing, fingerprint image can be easy for feature extraction Recognition: this is the main task of this system. Searching database and comparing two fingerprints base on recognition algorithm. The detail system processing is provided in Figure 34: 26 Figure 34 my fingerprint recognition system processing 7.2. Scanner Fingerprint capturing is the basic step of this project. In fingerprint capturing, I choose fingerprint scanner as my project device, because normal digital camera cannot capture a clear fingerprint image. In these two scanners I researched above, I choose Secugen Hamster plus - USB Fingerprint Scanner as my project scanner. There are some reasons: 1. It captures BPM type image. 2. It captures fingerprint image only, did not embed any application. 3. USB connection: can link the computer through USB. 4. Supports: Fingerprint recognition, fingerprint verification, fingerprint authentication, fingerprint scanning & fingerprint matching applications. 5. Can execute in windows operating system. 7.3. Pre-processing In the image pre-processing, after researched, I decide to follow the typically pre-processing way. (Figure 35)[10] Figure 35 common flowchart of fingerprint pre-processing 27 I also make some change about this pre-processing. I add two image enhancement operation in it (Normalization and segmentation), in order to extract minutia more accurate. My pre-processing show in Figure 36: Figure 36 my fingerprint image pre-processing In my pre-processing, I choose edge detection as a way of segmentation, because some common segmentation methods have some limitations (e.g. Histogram-based methods). In edge detection, I prefer to use Gabor filter to implement edge detection. 7.4. Recognition Recognition is the basic function of fingerprint recognition system. After research, I think classification is also important. When I implement this application, I should consider classification. In minutiae extraction, I think the first algorithm (using a 3X3 template mask base on the binary thinned fingerprint image) is better than neural network algorithm, because it spends less time for implementation and it can omit some false minutiae. In neural work algorithm, although it is a traditional algorithm for minutiae extraction, but it will make the code too complex (I should implement more than 24 neural units for training), so I choose to use the first algorithm using a 3X3 template mask base on the binary thinned fingerprint image) to do the minutiae extraction. In matching, this is the last step of this system, I prefer to use the local minutia position based algorithm (The second algorithm in matching). To me, this algorithm is easy to understand and it seems simpler than the alignment, so I choose to use local minutiae position based algorithm to do the minutiae matching. During the matching, I will follow this standard: If we find out 12 minutiae are the same, we will stop matching and we can say these two fingerprints came from the same finger. After research, I think the recognition of my system probably like that: extract minutiae at first. Next extract the core and delta and then remove the fail minutiae. Match minutiae at last. The main process is provided in Figure 37. Figure 37 my fingerprint recognition processing 28 7.5. Implementation I decide to develop a desktop application. I choose Java as my programming language, although C++ seems more suitable in this project. I decide to use Java, because I have more experience in using Java than C++. In database, I prefer to use Microsoft Access in coding and testing, and using Java DB in software demonstration. Microsoft Access has GUI, easy to design database and debugging. In demonstration, Java DB does not need to install, it is easy to demonstrate. This is the conclusion of research, some idea may be changed in requirement analysis, design and coding, in order to make my project more accurate 29 Reference [1]East Shore Technologies. Forensic Latent Criminal Justice AFIS-Lite System, available: http://www.east-shore.com/CJLite.html [accessed 22 October 2010] [2]Center for Unified Biometrics and Sensors. (2009). CUBS Matlab Toolbox for Fingerprint Verification, available: http://www.cubs.buffalo.edu/code.shtml [accessed 22 October 2010] [3] OPTEL Ulreasonic Technology. (2009). Fingerprint recognition demo software, available: http://www.optel.com.pl/software/english/bpi.htm [accessed 22 October 2010] [4] Sanghoon Lee, Chulhan Lee, Jaihie Kim. Image Preprocessing of Fingerprint Images Captured with a Mobile Camera [online] available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.128.137 [accessed 20 October 2010] [5] wikiHow. How to Take Fingerprints, available: http://www.wikihow.com/Take-Fingerprints [accessed 20 October 2010] [6] Suprema. (2008). Portable Dual Fingerprint Live Scanner, available: http://www.supremainc.com/eng/product/ls_50.php?mark=53 [accessed 18 October 2010] [7] Bayometric inc. Secugen Hamster plus - USB Fingerprint Scanner, available: http://www.bayometric.com/products/Secugen-Hamster-plus-usb-fingerprint-scanner.htm [accessed 18 October 2010] [8] Wikipedia. (2010). File:Fingerprint picture.svg, available: http://en.wikipedia.org/wiki/File:Fingerprint_picture.svg [accessed 20 October 2010] [9] Lin Hong, Yifei Wan and Anil Jain (1998). Fingerprint Image Enhancement: Algorithm and Performance Evaluation [online] available: http://ai.pku.edu.cn/aiwebsite/research.files/collected%20papers%20-%20fingerprint/Fingerprin t%20image%20enhancement%20-%20algorithm%20and%20performance%20evaluation.pdf [accessed 1 November 2010] [10] Anil Jain and Sharath Pankanti. Fingerprint Classification and Matching [online] available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.3331&rep=rep1&type=pdf [accessed 1 November 2010] [11] Wikipedia. (2010). Edge detection, available: http://en.wikipedia.org/wiki/Edge_detection [accessed 8 November 2010] [12] Fingerprint Identification [online] available: webfea.fea.aub.edu.lb/dsaf/labs/projectv1.1.pdf [accessed 2 November 2010] 30 [13] Wikipedia. (2010). Canny edge detector, available: http://en.wikipedia.org/wiki/Canny_edge_detector [accessed 8 November 2010] [14] Wikipedia. (2010). Gabor filter, availavle: http://en.wikipedia.org/wiki/Gabor_filter [accessed 8 November 2010] [15]Wenzhou Liu, Xiangping Meng, Linna Li and Quande Yuan (2008), A kind of Effective Fingerprint Recognition Algorithm and Application In Examinee I dentity recognition [online] available: ftp://ftp.computer.org/press/outgoing/proceedings/csse08/data/3336d035.pdf [accessed 2 November 2010] [16] Andrew Ackerman. Fingerprint Recognition [online] available: http://www.cs.ucla.edu/honors/UPLOADS/andrew/thesis.pdf [accessed 4 November 2010] [17] Wikipedia. (2010). Fingerprint recognition, available: http://en.wikipedia.org/wiki/Fingerprint_recognition [accessed 22 October 2010] [18] Wikipedia. (2010). Segmentation, available: http://en.wikipedia.org/wiki/Segmentation_(image_processing) [accessed 8 November 2010] [19]Answers.com. (2010). How many minutiae points will match on a complete fingerprint? Available: http://wiki.answers.com/Q/How_many_minutiae_points_will_match_on_a_complete_fingerprin t [accessed 11 November 2010] [20]Miroslav Baca, Kornelije Rabuzin, Zvonko Merkas. Fingerprints preprocessiong using walsh functions [online] available: hrcak.srce.hr/file/32812 [accessed 11 November 2010] [21]IT Acumens.com. Secured authentication using fingerprint biometry, available: www.itacumens.com/project/files/securedppt.ppt [accessed 11 November 2010] [22]Manvjeet Kaur, Mukhwinder Singh, Akshay Girdhar, Parvinder S. Sanhu (2008).Fingerprint verification system using minutiae extraction technique [online] available: http://www.waset.org/journals/waset/v46/v46-85.pdf [accessed 13 November 2010] [23] Wikipedia. (2010). Artificial neural network, available: http://en.wikipedia.org/wiki/Artificial_neural_network [accessed 13 November 2010] [24]Griaule biometrics. (2008). Book-Understanding Biometrics, available: http://www.griaulebiometrics.com/page/en-us/book/understanding-biometrics/types/feature-ex traction/singularity [accessed 13 November 2010] [25]Wikipedia. (2010). Fingerprint, available: http://en.wikipedia.org/wiki/Fingerprint [accessed 5 November 2010] 31 [26] Forensic-medecine.info. Fingerprints, available: http://www.forensic-medecine.info/fingerprints.html [accessed 18 November 2010] [27] Jim Z. C. Lai, Shih-Chia Kuo (2003). An Improved Fingerprint Recognition System based on Partial Thinning [online] available: www.csie.mcu.edu.tw/~yklee/CVGIP03/CD/Paper/PR/PR-02.pdf [accessed 18 November 2010] 32