Image Manipulation for Face Recognition
Mingtao SUN
Australia National University
Comp 6701 eScience Project
June 2005
1
Table of Contents
1 Introduction ....................................................................................................................................... 1
1.1
Overview ......................................................................................................................... 4
1.1.1 Background .............................................................................................................. 4
1.1.2 Purpose ..................................................................................................................... 4
1.1.3 Scope ........................................................................................................................ 4
1.1.4 Project Objectives .................................................................................................... 4
1.2
Stakeholders .................................................................................................................... 5
1.2.1 Client ........................................................................................................................ 5
1.2.2 Supervisor ................................................................................................................ 5
1.2.3 Developer ................................................................................................................. 5
1.3 Deliverables ........................................................................................................................... 5
1.4 Schedule ................................................................................................................................. 6
1.5 Document Organization ......................................................................................................... 7
2 Requirements Analysis .................................................................................................................. 8
2.1
Detailed User Requirements ........................................................................................... 8
2.1.1 Functional Requirements ......................................................................................... 8
2.1.2 Non-functional Requirements .................................................................................. 8
2.2 Tools ..................................................................................................................................... 10
2.2.1 Hardware ................................................................................................................... 10
2.2.2 Software .................................................................................................................... 10
2.2.3 Programming language ............................................................................................. 10
2.2.4 Operating System ...................................................................................................... 10
3 Modelling ........................................................................................................................................ 11
3.1 Domain Chart ....................................................................................................................... 11
3.2 Domain Description ............................................................................................................. 12
3.2.1 ImageManipulation Domain ..................................................................................... 12
3.2.2 GUI Domain .............................................................................................................. 14
3.2.3 ImageResource Domain ............................................................................................ 15
3.2.4 Database Domain ...................................................................................................... 15
4 Implementation ............................................................................................................................... 16
4.1 Automating image capture with the webcam ....................................................................... 16
4.2 Background Removal ........................................................................................................... 19
4.3 Image Scaling ....................................................................................................................... 27
4.4 Feature Detection ................................................................................................................. 30
4.5 Image Processing in Java ..................................................................................................... 31
4.5.1 Loading/Saving an image from/to a file ................................................................... 31
4.5.2 Scaling an image ....................................................................................................... 31
4.5.3 Processing a pixel...................................................................................................... 32
4.6 JPEG or PNG? ..................................................................................................................... 33
5 Testing ............................................................................................................................................. 34
5.1 Testing Plan .......................................................................................................................... 34
2
5.2 Testing Specifications .......................................................................................................... 36
6 Project Evaluation ........................................................................................................................... 38
6.1 Current Outcome .................................................................................................................. 38
6.2 Pitfalls .................................................................................................................................. 40
6.3 Further Suggestions.............................................................................................................. 41
7 Conclusion ...................................................................................................................................... 42
8 References ....................................................................................................................................... 43
Appendix A: Schedule ....................................................................................................................... 44
Appendix B: Work Breakdown Structure .......................................................................................... 45
Appendix C: User Manual ................................................................................................................. 47
C.1 Installation ........................................................................................................................... 47
C.2 Menu Commands ................................................................................................................ 47
C.2.1 File ............................................................................................................................ 47
C.2.2 Image ........................................................................................................................ 48
C.2.3 Configure .................................................................................................................. 48
C.3 FAQ ..................................................................................................................................... 48
C.3.1 How can I start collecting Images? .......................................................................... 48
C.3.2 How can I remove the background of images? ........................................................ 49
C.3.3 How can I do background removal settings? ........................................................... 49
C.3.4. Do I have to do background removal settings every time? ..................................... 50
C.3.5 How can I pre-process the images? .......................................................................... 51
C.3.6 How can I detect feature of the images? .................................................................. 51
C.3.7 Is it possible to select the whole directory as the input? .......................................... 51
C.3.8 How can I delete the image files? ............................................................................ 51
C.3.9 Which directory stores the image files? ................................................................... 51
3
1 Introduction
1.1 Overview
1.1.1 Background
Face recognition has received significant attention during the past several years. Even though
current face recognition systems have reached a certain level of maturity, their success is limited by
certain conditions. For example, recognition of face images acquired in an outdoor environment
with changes in illumination remains a large unsolved problem. (Zhao, Chellappa 2003)
Having seen the problems of face recognition, it is also necessary to realize the importance of the
preparation work. The face recognition research usually needs lots of face images; what is the way
to acquire those photos? Digital camera seems to be a choice, but exporting photos to the computer
would become a tedious task. So question one is how to quickly get a set of photos of a person.
Question two is how to remove the background of hundreds of photos easily since only the face in
the photo counts. Manually doing these tasks is obviously out of the question, so here comes the
need for this project. The ultimate goal is to allow researchers to get the kinds of images they want
as easily as possible.
This project does not involve any core algorithm of face recognition. However, it provides a solid
foundation for further research in this field. This project consists of two parts: one is acquiring
photos from a web camera and the other one is pre-processing the images, including removing the
background, scaling the image to a standard size, making the number of pixels in a image consistent,
and so on. This project also covers a little about feature detection.
1.1.2 Purpose
The purpose of doing this project is to learn about image processing techniques and to improve
research techniques.
1.1.3 Scope
The scope of this project covers acquiring and pre-processing images for further processing in face
recognition field and does not cover implementing any face recognition algorithm.
1.1.4 Project Objectives
Developing a program which
 automates face image capture with a web camera
4

preprocesses the captured images (removes background, scales size, and so on)
1.2 Stakeholders
1.2.1 Client
Professor Tom Gedeon, tom@cs.anu.edu.au, (612) 612 51052
1.2.2 Supervisor
Mr Pascal Vuylsteker, pvk@vuylsteker.net, (612) 612 58192
1.2.3 Developer
Mr Mingtao Sun, jackysun820@hotmail.com, (61) 402556204
1.3 Deliverables
Deliverable
Source Code
Presentation
Final Report
Due Date
th
20 June
22nd June
24th June
5
1.4 Schedule
This project started on 24th Feb 2005 and finished on 24th June 2005.
Task
1. Analysing Client’s Requirements.
1.1 Contacting the client and getting the initial
requirements
1.2 Analysing the initial requirements
1.3 Discussing detailed requirements and acceptance
criteria with the client
1.4 Selecting tools
2. Modelling
2.1 Developing domain diagram
2.2 Developing class diagram for each domain
3. Implementation
3.1 Automating image capture
3.2 Removing background
3.3 Scaling size of the image
3.4 Detecting feature of the face
4. Testing and Debugging
5. Documentation
5.1 Commenting source code
5.2 Writing final report
Expected Duration Actual Duration
(days)
(days)
1
1
3
1
3
1
1
1
2
4
2
4
20
14
5
12
5
18
20
8
2
8
4
15
4
15
More details can be seen in Appendix A.
6
1.5 Document Organization
Section 1, Introduction, gives an overview of the project, as well as the stakeholders, deliverables,
and schedule.
From Section 2 to Section 5 describe the main phases of this project.
Section 2, Requirement Analysis, describes the detailed user requirements and the tools needed to
accomplish this project.
Section 3, Modelling, also known as Designing, uses UML model to design the structure of this
software.
Section 4, Implementation, describes the ideas, methods and algorithms which are used to achieve
the functionalities of the software.
Section 5, Testing, describes the testing plan and the testing result.
Section 6, Project Evaluation, evaluates the software and makes suggestion on further development.
Section 7, Conclusion, draws a conclusion from doing this project.
7
2 Requirements Analysis
2.1 Detailed User Requirements
2.1.1 Functional Requirements
No.
FR1
FR2
FR3
FR4
FR5
Name
Acquiring face
images
Description
The photos should be taken with a web camera. 100 photos
should be taken for each user. The software should make the
user look in different directions. The majority of photos
should be taken for a neutral expression face while some for
smiling and annoyed expression face. There should be some
flags which the user would be instructed to look at when
taking a photo. The user should be sitting in front of blue
cloth (background). The user’s body is covered with blue
cloth, with face and neck uncovered. The photo is only taken
of the participant’s up half body (above chest).
Pre-processing
Given a face image (one of the output images of FR1), the
image
software should remove its blue background, clip and scale
the image to a size of 80*80 pixels with the face centred. The
face should contain exactly 3350 pixels and be converted to
grey scale.
Detecting features Given a pre-processed face image (the output image of FR2),
of the face
the software should apply a center-surround-filter to the
image; the output image would be the face with its features
(eyes, nose, mouth, and so on) outstanding.
Managing users’ When the user inputs a duplicated ID number, the software
face images
should give a warning. The software should allow an
administrator to delete the user’s ID number and all his/her
faces images.
Finding useful
features for
recognition
Priority
Mandatory
Mandatory
Mandatory
Mandatory
Optional
2.1.2 Non-functional Requirements
No.
NFR1
Description
Priority
It should be extremely convenient to take a photo. For example, once the user Mandatory
clicks on ‘Taking a photo’ button, the snapshot would be done and the image
file would be saved immediately. It should just need a ‘one-click’ effort.
8
NFR2
NFR3
Instructions for acquiring face images should make users clear enough as to Mandatory
where they should look at when taking photos
The software should be able to perform batch processing. That is, the user can Mandatory
select multiple files or even a whole directory for processing.
9
2.2 Tools
2.2.1 Hardware
Macintosh G5 with three screens
iSight web camera
2.2.2 Software
Microsoft Word is used for documentation.
Microsoft Project 2000 is used for generating the Work Breakdown Structure.
Jcreator is used for editing, compiling and running Java source code.
2.2.3 Programming language
Java is chosen to implement this project.
2.2.4 Operating System
MAC OS 10.3
10
3 Modelling
3.1 Domain Chart
ImageResource
Image is acquired from
ImageResource
ImageManipulation
User operates
software
through GUI
GUI
All data is
maintained by
database
Database
11
3.2 Domain Description
3.2.1 ImageManipulation Domain
Mission Statement: This domain is responsible for acquiring images, pre-processing images,
managing user ID numbers and changing default settings.
Class Diagram:
1:1
ImageManipulation
1:1
1:1
1:1
1:1
1:1
Settings
Managing
1:1
1:1
Preprocessing
Acquiring
1:1
1:1
1:1
1:*
ImageShop
Grid
1:1
1:1
Face
ImageManipulation Class: This class acts as an interface of this domain, all the inter-domain data
communication must be through this class. This class does not involve any calculation.
Settings Class: This class stores all the settings information that can be changed during the runtime
of the software. This information includes the standard size of a pre-processed image, the number of
pixels of an image, the settings for background removal, the default output directory and so on.
Managing Class: This class manages users’ ID numbers, including deleting ID numbers and
relevant images.
12
Preprocessing Class: This class pre-processes the images, including removes the background, scales
the size of the image, and converts the image to grey scale. After pre-processing, the output images
would have the same size and same number of pixels.
ImageShop Class: This class serves as a utility class for Preprocessing class. It provides a set of
useful methods for pre-processing images.
Face Class: This class servers as a utility class for ImageShop class. It detects the boundary of the
face in a image and counts the number of pixels in the image.
Acquiring Class: This class acquires images from ImageResource Domain and saves the acquired
image to database. This class also stores displaying parameters for AcquiringDialog Class in GUI
domain.
Grid Class: This class is used by both Acquiring Class and AcquiringDialog Class. This class
represents a square on the screen, where a user would possibly look at when taking photos.
13
3.2.2 GUI Domain
Mission Statement: This domain provides a graphic user interface for users to use the functions that
the software supplies.
Class Diagram:
1:1
MainFrame
1:1
1:1
1:1
1:1
1:1
Settings
Dialog
Managing
Dialog
1:1
Preprocessing
Dialog
AcquiringDi
alog
1:1
1:*
Grid
MainFrame Class: This class displays the main window when the software starts to run. It contains
a menu bar and a window playing video streams that the web camera captures.
SettingsDialog Class: This class displays a dialog window which allows users to change the default
settings.
ManagingDialog Class: This class displays a dialog window which allows administrator to delete
the users’ ID numbers and their relevant images, to export files to another directory, and to
pre-process the images.
PreprocessingDialog Class: This class displays a dialog window which allows users to select
multiple files or directories of files for pre-processing.
AcquiringDialog Class: This class displays a dialog window which allows users to take photos with
a web camera. This class also gives sufficient instructions to assist users to complete the acquiring
face image process.
14
3.2.3 ImageResource Domain
Mission Statement: This domain is responsible for providing raw images for pre-processing.
WebCamConfiguration Class: This class configures the web camera correctly and provides video
streams as well as snapshot captured by web camera to ImageManipulation Domain.
3.2.4 Database Domain
Mission Statement: This domain is responsible for all the file I/O operations required by
ImageManipulation Domain.
Database Class: This class provides the file I/O methods needed by ImageManipulation Domain.
15
4 Implementation
4.1 Automating image capture with the webcam
Before proceeding to the description of how this is implemented, there is a question needing to be
addressed. Why can people not just use some existing software which has the functionality to do a
snapshot? Recall the steps to do a snapshot: click on a menu in the menu bar, click on a menu item
(for example, snapshot), type a name and save the photo. The whole process is obviously too long,
especially for taking 100 photos at a time. A quicker way must be found, and it should take user’s as
little effort as possible.
After inspecting almost all the software suitable for iSight (the webcam for Mac), it was concluded
that none of them would be appropriate for this project. Therefore, the answer is developing a new
program to achieve this function.
The difficulty is how to get a Java program to connect to a webcam. I had been working on this for
long but in vain. Finally, Thanks to Mr Pascal Vuylsteker, my supervisor, who found the useful link,
http://www.oreillynet.com/pub/wlg/2933, the problem was solved. With this part done, the initial
requirement is met, that is, the user can possibly just click one button to do a snapshot and the
photos can be saved automatically.
The next problem is how to instruct the user to look in different directions while taking photos. This
is more a usability issue than a technical one. With the three-screen computer, it is not hard to find a
good solution.
16
The above is a snapshot during the runtime of the program. The little face icon in the pane is the
position which the user should look at. Once the user is ready, he/she clicks on the “shoot” button.
The face icon will move to the next spot which the user should look at. The order is random in order
to make sure that the user not only moves his/her eyes but also moves his/her head. So a line is
drawn to point out where the next position is. It is necessary because the user tends to get lost easily
when sitting in front of a three-screen computer.
17
The little face icon does not only serve as a position guide, but also tells the user what kind of
expression to show. For example, the above face icon is a smiling one, and the user should be
smiling when taking the current photo.
18
4.2 Background Removal
It is mentioned in Functional Requirement 1 that the user’s body is covered with blue cloth and
sitting in front of a blue background. This eases the background removal task somewhat, but still
leaves many things to do. This section firstly describes what the exact goal of this task is, then gives
the algorithm for performing this task, and finally illustrates an example to make the reader better
understood.
Goal:
The ideal result, of course, is removing all the unwanted blue background and leaving the whole
face integrated. Nevertheless, there is always a gap between the ideal world and the reality. So here
is the compromise: the output image could have some blue pixels left but the bottom line is the face
should be integrated. Damage to the neck and the ears could be tolerable to some extent, as they
have insignificant contribution to face recognition. For example, have a close look at the right
picture (output), and it is seen the left ear is missing a bit. However, it is still an acceptable output.
Algorithm:
Firstly removes the obviously blue pixels (sets those blue pixels to be white)
Pseudo code:
for(i=0, i<image.width,i++)
for(j=0; j< image.height; j++)
if(isBluePixel(getRGB(i,j))=true)
setRGB(i,j, white);
isBluePixel(int R, int G, int B)
return true if RGB falls into BLUE_AREA;
else return false;
19
BLUE_AREA: a set of RGB ranges in which the color is blue. (See the example for more details.)
Secondly, for those remaining blue pixels, if the number of consecutive vertical or horizontal
non-white pixels is less than a constant, it is assumed those pixels do not belong to the face.
Pseudo code:
for(i=0, i< image.height,i++)
if (scanline contains less than MIN_FACE_WIDTH non-white pixels)
these non-white pixels are set to be white
for(i=0, i< image.width,i++)
if (vertical line contains less than MIN_FACE_HEIGHT non-white pixels)
these non-white pixels are set to be white
Again, a series of
MIN_FACE_HEIGHT.
experiments
are
needed
to
decide
MIN_FACE_WIDTH
and
Given good BLUE_AREA, MIN_FACE_WIDTH, and MIN_FACE_HEIGHT values, it suffices
enough to remove the background. To improve the performance, the algorithm can be further
modified. Since a face only takes up a small portion of the whole photo, it is wasteful to process all
the horizontal and vertical lines. It is much better to start the processing from the middle of the
photo, and once the program is convinced the current horizontal or vertical line does not cross the
face, it can simply ignore the rest
Line 2
Line 1
Here is an example, suppose the program starts to process from line 1 and toward line 2. When it
proceeds to line 2, it found out that line 2 does not cross the face, and then it stops processing and
simply sets all the pixels between line 2 and the left side of the image to be white.
Pseudo Code (modified from the second part of the above pseudo code):
for(i= image.height/2, i< image.height,i++)
cross_face=false
if (scanline contains less than MIN_FACE_WIDTH non-white pixels)
20
these non-white pixels are set to be white
else
cross_face = true
if (cross_face = false)
break
for(, i<photo.height, i++)
set the whole scanline to be white pixels
This pseudo code is only for processing the lower half of the image. However, the idea is the same
for the rest.
Smooth the face
After the previous steps, the above face would be a possible outcome. When the face is zoomed in,
it turns out the outline of the face is not smooth. So the next step is to smooth the outline of the face.
Here is the idea:
21
The above zigzag line represents part of the face outline. The idea is finding all the “concave
pixels” (highlighted by the circles), drawing a line between two adjacent “concave pixels”, and
cutting off the outstanding polygons. Here is the output:
Though it is not perfect yet, it is a lot better than the previous one.
The real difficulty of removing background, as mentioned earlier, is that a series of experiments
need to be carried out before determining the three important values: BLUE_RANGE,
MIN_FACE_WIDTH and MIN_FACE_HEIGHT.
Once these three values are determined, they can be reused for all the photos taken in the same
environment. However, if the environment changes, for instance, the light in the room becomes a
little dimmer, these three values may need to be reset. Hence, it is highly necessary to develop a
program which is able to set these values easily. The next part will walk through an example to see
how these values are set and how the background is removed step by step.
22
Example:
The left is the original image and the right is the control panel. Use the mouse to drag a rectangle on
the blue background but keep it small. Later it can be seen how a big rectangle will ruin the face.
At the left-upper corner, the “31 103 51 150 138 255” represents the current BLUE_AREA.
Mathematically, 31 is the lower limit for RED, 103 is the upper limit for RED, 51 and 150 for
GREEN, and 138 and 255 for BLUE. “Horizontal” represents MIN_FACE_WIDTH and “Vertical”
represents MIN_FACE_HEIGHT. The latter two values are currently 0 and they will be used once
BLUE_AREA is determined. Having seen there are still large patches of blue pixels remaining, the
previous step will be repeated. So again, drag a small rectangle on the blue area.
23
Now the BLUE_AREA is “31 103 51 150 138 255” plus “102 128 158 186 250 255”. The
“dragging rectangle” step continues until there is no big patch of blue pixels remaining.
Now that all patches of blue pixels are too small to drag a rectangle on, it is time to use the
“Horizontal” and “Vertical” value. Set both of them to 10 and check the result.
24
It is good but it could be improved by adjusting the “Horizontal” value a bit larger.
Then click on the “Save” button to save these settings to a file. The background removal program
reads the settings from the file and then is able to process all the photos taken in the same
environment.
25
This is an example of how a huge rectangle ruined the face.
26
4.3 Image Scaling
Once the background is removed, it is possible to process the images with some face recognition
algorithms immediately. However, considering the number of images (100) for one user, it is better
to scale the image to a small size and thus saving the hard disk space. A more important reason is
that the smaller the number of pixels, the less time it takes to process the image. One might raise a
question that would the image lose significant information if the size is reduced? Yes, it would but it
should not affect further processing. As long as the image is not too small, one can always identify
who the person is in the image. That is enough for face recognition because a face recognition
system is meant to simulate the human’s recognition ability. It makes no sense if a face recognition
system is only able to recognize “very clear” images but fail to do so with “not so clear” images that
a human would recognize.
According to the client’s requirement, the image is to be scaled to the size of 80*80 pixels with the
face centred. The face should contain 3350 pixels and all the rest of the pixels are white. Some edge
detection algorithms cannot be applied to color images, so it is also necessary to convert the image
to grey scale.
There are four steps in this stage:
1. Scaling the face to the number of pixels which is most approximate to and greater than 3350.
2. Making the number of pixels of the face exactly equal to 3350.
3. Making the size of the image 80*80 pixels.
4. Converting the image to grey scale.
The following paragraphs discuss these four steps in more detail.
Step 1: Scaling the face to the number of pixels which is most approximate to and greater than
3350.
To determine the scaling proportion, it firstly needs to count the current number of pixels in the face.
This is not tough to achieve, as simply counting the number of non-white pixels of the image will
do. Suppose P represents the proportion, CN represents the current number of non-white pixels, and
TN represents the target number of non-white pixels (3350), then
P=sqrt(CN/TN)
Suppose W represents the width to which to scale the image, H represents the height to which to
scale the image, WI represents the width of the original image, and HI represents the height of the
original image, then
W=floor (WI/P+1), H=floor (HI/P+1)
The use of floor is because W and H must be integers and the use of +1 is to make sure the number
of pixels of the output image will be slightly greater than 3350.
27
Then scale the image to the width of W and the height of H. Java has provided some methods to
realize the scaling given the target width and targe height, and they will be covered in section 4.5.
Step 2: Making the number of pixels of the face exactly equal to 3350.
The idea is removing the least significant pixels. For easy implementation, it has been decided that
the extra pixels be removed from the neck part. Technically, the program processes the image from
bottom to top. It sets the current non-white pixel to be white if the total number of non-white pixels
is greater than 3350.
Pseudo code:
remainingPixel=totalPixel-3350
for (x= image.height -1, x>=0, x--)
for (y=0, y<image.width, y++)
if(image.getRGB(x,y) is a non-white pixel)
remainingPixel= remainingPixel-1
image.setRGB(x,y, white)
if (remainingPixel==0) return
One may ask that would it eventually ruin the face since the processing is from bottom to top? The
answer is it is very unlikely to happen. Remember the input image for this step is actually the output
image from step 1, and step 1 has already made the number of non-white pixels close to 3350 and
there are only a few extra pixels to be removed. Therefore, it is safe to apply the above pseudo code.
Step 3: Making the size of the image 80*80 pixels.
The instinct for handling this is clipping the image into a square and scaling it to 80 by 80.
Unfortunately, it will not work because the 3350-pixel condition cannot be guaranteed. The correct
idea is cutting off the rectangle boundary of the face and adding extra white pixels to make the size
80 by 80.
70
70
80
Adding extra
white pixels
80
Cutting off the
rectangle
boundary
28
The above is just an example, and its purpose is to familiarize readers with the idea. The actual face
boundary cannot be exactly a square. Under normal circumstances, human’s face should fit 80 by
80 pixels. However, there is a special case needed to take into consideration. If a person’s face is so
long that the height of the boundary exceeds 80, the exceeding part will be ignored. But there is
nothing to worry about, since this is an extreme case and happens with an extremely low probability.
Recognizing such an unusually shaped face should not be a problem for any face recognition
system, and the loss of some small part of the image would not be a problem.
The following is pseudo code to detect the rectangle boundary of the face:
for (x=0, x<image.height, x++)
for (y=0, y<image.width, y++)
if (image.getRGB(x,y) is not white && xMin==0)
xMin=x
xMax=x
yMin=y
yMax=y
else if (image.getRGB(x,y)! is not white && xMin!=0)
xMax=x
if(y<yMin) yMin=y
if(y>yMax) yMax=y
Step 4: Converting the image to grey scale.
Suppose GREY is the grey value of the pixel, and R,G,B are RED, GREEN, BLUE value of the
pixel respectively.
GREY = R*0.3 + G*0.59 + B*0.11
Apply this formula to all the non-white pixels of the image.
29
4.4 Feature Detection
Java has provided sufficient image processing methods, making this section extremely easy.
Firstly create a kernel array. Since the filter is center-surround filter, the corresponding array is
float[] element = {
-1.0f, -1.0f, -1.0f,
-1.0f, 8.0f, -1.0f,
-1.0f, -1.0f, -1.0f
};
Then create Kernel object, create a ConvolveOp object based on the Kernel object, and finally filter
the image with the ConvolveOp object:
Kernel kernel = new Kernel(3,3,elments);
ConvolveOp op = new ConvolveOP(kernel);
op.filter(image, filteredImage);
30
4.5 Image Processing in Java
This section gives a detailed description of the technical issues which have been omitted in the
previous sections. They are separated because they are language dependant issues. The
programming language being discussed here is Java (J2SE 1.4.2) and the topics include:



Loading/Saving an image from/to a file
Scaling an image
Processing a pixel
The code here will not be explained in detail as Java API has already done so.
4.5.1 Loading/Saving an image from/to a file
Loading an image from a file:
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;
BufferedImage image = ImageIO.read(new File(“photo.jpg”));
It is noted that ImageIO only accepts files with an extension of “jpg”, “jpeg”, “png”, or “gif”.
Therefore if the input file contains an extension other than those, an external program may be used
to convert the file to one of those valid extensions.
Saving an image to a file:
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;
ImageIO.write(bufferedImage, “jpeg”, new File(“photo.jpeg”));
It is noted that ImageIO can only write to files with an extension of “jpg”, “jpeg”, or “png”.
4.5.2 Scaling an image
import java.awt.Image;
import java.awt.image.BufferedImage;
BufferedImage outputImage = new BufferedImage(width, height,
BufferedImage.TYPE_3BYTE_BGR);
Image image = inputImage.getScaledInstance(width, height, Image.SCALE_FAST);
outputImage.createGraphics().drawImage( image, 0, 0, this);
31
4.5.3 Processing a pixel
Class BufferedImage contains a method that returns an integer pixel at coordinate of (x,y)
int getRGB(int x, int y)
The returned value is an integer in base 10 but it actually is a conversion of an integer in base 2 with
a length of 32. It is easily converted back using:
String Integer.toBinaryString(int i)
Here is a possible output:
11111111 01010101 11110000 00110011
The most left 8 bits represent the alpha value of a pixel and are of no use in this project. They are
chopped off using:
String str = s.substring(8);
Then the output is:
01010101 11110000 00110011
The leftmost 8 bits of the above string represent the RED value of a pixel, the middle 8 bits for
GREEN, and the rightmost 8 bits for BLUE. There come the so-called RGB values of a pixel.
The following method is to extract the RGB values respectively:
private int[] RGBStringToInt3(String rgb){
int[] result= new int[3];
result[0] = Integer.parseInt(rgb.substring(0,8),2);
result[1] = Integer.parseInt(rgb.substring(8,16),2);
result[2] = Integer.parseInt(rgb.substring(16),2);
return result;
}
Only after reaching this stage can a real manipulation on a pixel happen. Having known how to
extract the RGB values from the raw value, updating a value of a pixel is simply a reversed process
and it is needless to describe it. The method of setting a RGB value at coordinate of (x,y) is:
void setRGB(int x, int y, int rgb)
32
4.6 JPEG or PNG?
When saving an image to file, is it better to save it as a “jpeg” format or “png” format? (There is no
need to discuss other formats, because they are the only two output formats accepted by Java as
mentioned in section 4.5.1.) This issue should attract serious attention in that not only is it a
question of size and quality, but it also could be fatal should the wrong format be chosen. It is time
to reveal the answer:
Definitely PNG
Here is an example to compare the effects of the two formats:
jpeg format
png format
Use mspaint, the drawing program of Windows, to create a picture with a black rectangle centred
and save it to jpeg format and png format respectively. Take a close look at the jpeg format image,
especially the highlighted circle part. Make no mistake about it. This is not dust on the paper but
some extra non-white pixels not supposed to be there.
These extra non-white pixels will cause at least two problems:
1. The 3350-pixel requirement (FR2) would never be met.
2. The rectangle boundary of the face would be larger than it should be, and it would cause the
face not to fit in the size of 80 by 80 pixels.
There certainly will be other potential risks when those jpeg format images are further processed.
Therefore, the above is sufficient to show that the png format should be used for image processing,
needless to mention that the png format even takes up less hard disk space.
33
5 Testing
5.1 Testing Plan
The purpose of testing is to ensure that the intended system fulfils and satisfies the specified user
requirements and that it achieves client’s expectations.
Items to be tested: All the mandatory requirements and their associated features are the items to be
tested.
Items omitted from testing: Any requirements that were not marked as mandatory in the detailed
user requirements will not be tested. These include all desirable and optional requirements.
Basic features of the system to be tested:







Is the usability of the introductory screens as expected?
Does a given input produce the appropriate or expected output? Is the output within the
permitted range?
Do the various buttons function correctly?
Are graphical outputs and dialog boxes displayed correctly and at the appropriate times?
Do radio buttons and checkboxes permit the selection of all possible answers?
Can the administrator save, add or delete different features without any constraints?
Does the help function provide the necessary information?
Techniques:




Testing each identified module separately and independently of the others is one of the best
techniques that can be employed in the testing phase. This enables the developer to trace back
to the user requirements. Moreover, the testing phase includes Black Box testing as well.
Formal inspection of the source code and walkthrough are also techniques that will be used
before entering into the testing phase.
There will be several formal review points before and during the system test. This is a vital
process in achieving a high quality product.
All testing features are based on the product life cycle chosen at the beginning of the project.
Test Activities:

Interface Testing: Interface testing will focus on the testing of the graphical data
representation on the web site. This includes functions such as displaying error messages (in a
dialogue box) for incorrect input data. All messages (in the dialogue boxes) must be short,
clear and descriptive for the user. All buttons and menu items on the actual interface should be
tested to ensure that they operate to the client’s satisfaction.

Performance Testing: From an end user’s perspective, response time is the basic measure of
34
performance used to judge the quality of the system. Administrators, on the other hand, are
concerned not only with response time but also with the system’s resource utilisation.

Regression Testing: Regression testing is used when there are subsequent releases of the
software, so as to ensure that the current release does not adversely affect the previous one.

Reliability Testing: The reliability test is run over an extended period of time to make sure that
the system does not exhibit any defects after substantial uptime and that it continues to perform
within the desired response time.
35
5.2 Testing Specifications
Test ID: T1
Test Case Description: The system should allow the user to take 100 photos quickly.
Requirement ID: FR1, NFR1, NFR2
Test Steps:
1. The user’s body is covered with blue cloth, with face and neck uncovered.
2. The user is sitting in front of a blue background and facing straight to the web camera.
3. The user clicks on “File” menu.
4. The user clicks on “New User” menu item.
5. The user types an ID number.
6. Click on “OK” button.
7. The user reads the manual and follows the instruction for taking photos.
Pass/Fail Criteria:
Fail if the user finds it too difficult or too complicated to follow the instructions.
Fail if any of 100 photos is not saved.
Test Result: Passed.
Test ID: T2
Test Case Description: The system should be able to pre-process multiple images.
Requirement ID: FR2, NFR3
Test Steps:
1. Click on “Image” menu.
2. Click on “Pre-process Files” menu item.
3. Select source files.
8. Select target directory.
9. Click on “OK” button.
Pass/Fail Criteria:
Fail if the output files are not in the target directory.
Fail if any of output images meets one of the following conditions.
 The face is severely damaged.
 Blue background is not totally removed.
 The output image is not grey scale
 The output image’s size is not 80 by 80 pixels.
Test Result: Passed.
Test ID: T3
Test Case Description: The system should be able to detect features of multiple faces.
Requirement ID: FR3, NFR3
Test Steps:
1. Click on “Image” menu.
2. Click on “Feature Detect (Files)” menu item.
3. Select source files.
4. Select target directory.
36
5. Click on “OK” button.
Pass/Fail Criteria:
Fail if the output files are not in the target directory.
Fail if features of the face do not outstand for any of the output images.
Test Result: Passed.
Test ID: T4
Test Case Description: The system should allow the administrator to delete the users’ face images.
Requirement ID: FR4
Test Steps:
1. Click on “File” menu.
2. Click on “Manage” menu item.
3. Select some user IDs.
4. Click on “Delete” button.
Pass/Fail Criteria:
Pass if the directories with those names are deleted.
Test Result: Passed.
Test ID: T5
Test Case Description: The system should give a warning if the user tries to input an existing user
ID.
Requirement ID: FR4
Test Steps:
1. Click on “File” menu.
2. Click on “New User” menu item.
3. Type an existing user ID.
4. Click on “OK” button.
Pass/Fail Criteria:
Pass if a warning message is displayed.
Test Result: Passed.
37
6 Project Evaluation
6.1 Current Outcome
Here is a checklist of what has been done against the detailed user requirements:
No.
FR1
FR2
FR3
FR4
FR5
NFR1
NFR2
NFR3
NFR4
Description
The photos should be taken with a web camera. 100 photos should
be taken for each user. The software should make the user look at
different directions. The majority of photos should be taken for a
neutral expression face while some for smiling and annoyed
expression face. There should be some flags which the user would
be instructed to look at when taking a photo. The user should be
sitting in front of blue cloth (background). The user’s body is
covered with blue cloth, with face and neck uncovered. The photo
is only taken of the participant’s up half body (above chest).
Given a face image (one of the output images of FR1), the
software should remove its blue background, clip and scale the
image to a size of 80*80 pixels with the face centred. The face
should contain exactly 3350 pixels and be converted to grey scale.
Given a pre-processed face image (the output image of FR2), the
software should apply a center-surround-filter to the image; the
output image would be the face with its features (eyes, nose,
mouth, and so on) outstanding.
When the user inputs a duplicated ID number, the software should
give a warning. The software should allow an administrator to
delete the user’s ID number and all his/her faces images.
Finding useful features for recognition
It should be extremely convenient to take a photo. For example,
once the user clicks on ‘Taking a photo’ button, the snapshot
would be done and the image file would be saved immediately. It
should just need a ‘one-click’ effort.
Instructions for acquiring face images should make users clear
enough as to where they should look at when taking photos
The software should be able to perform batch processing. That is,
the user can select multiple files or even a whole directory for
processing.
Acquiring face images and pre-processing images should be able
to take place simultaneously. That is, when the current user is
taking photos, the software can pre-process the photos of the
previous user simultaneously.
Priority
Mandatory
Completed?
Yes
Mandatory
Yes
Mandatory
Yes
Mandatory
Yes
Optional
Mandatory
No
Yes
Mandatory
Yes
Mandatory
Yes
Optional
No
38
FR5 is not completed due to the following three reasons:
1. FR1 has taken much more time to complete than expected because the client wanted the user
interface very easy to operate on and hence added some more extra requirements.
2. The iSight web camera was not available until two weeks later than scheduled.
3. FR5 is in itself a huge, complex and unclarified requirement and it is difficult to predict how
much time it would cost to complete it.
39
6.2 Pitfalls
There are two drawbacks in the current system:
1. Some settings need to be configured before the program removes the background of a series of
images and the success for each image cannot be 100% guaranteed even if the photos have been
taken in the same environment. This is because of the algorithm used to remove the background.
2. It takes too long (approximately 2 seconds) to pre-process an image. Since the code has not
been optimised, there should be some space to improve.
40
6.3 Further Suggestions
Based on the pitfalls mentioned in the previous section, here is a list of suggestions for further
development:
1. The program should remove the background without configuring any settings in advance. It is
indeed a tough task and it remains unknown whether this task could be accomplished. But it is
absolutely worth a try.
2. The code should be optimised to improve the efficiency.
3. Security should be introduced to the system so that all image files are kept in a secret location
and general users and administrators have different levels of authority.
4. The system should be working on all platforms. (So far the image acquisition function only
works on Mac platform)
41
7 Conclusion
Some useful conclusions can be drawn from this project:
Java is really a powerful programming language for image acquiring and image processing.
PNG is the format that should be used for image processing.
Finally I would like to thank Professor Tom Gedeon and Mr Pascal Vuylsteker for their kind help
and valuable suggestions.
42
8 References
C.Adamson (2003) “What’s up with the Mac OS X Java and QuickTime?”
http://www.oreillynet.com/pub/wlg/2933 accessed Apr 7, 2005
W. Zhao, R.Chellappa, P.J.Phillips, and A.Rosenfeld (2000) “Face Recognition: A Literature
Survey”
43
Appendix A: Schedule
44
Appendix B: Work Breakdown Structure
Image Manipulation
Project
Analysis
Modelling
Developing domain
diagram
Implementation
Developing
diagram
Automating
Capture
Contacting
client
and getting initial
requirement
Analysing initial
requirement
class
Image
Testing
Documentation
Commenting source
code
Detecting features
Discussing detailed
requirement
with
client
Removing background
Writing final report
Scaling size
Selecting tools
45
Appendix C: User Manual
C.1 Installation
Prerequisites
J2SE 1.4.2 http://java.sun.com/j2se/1.4.2/download.html
Any web/digital/video camera with a firewall connection to a Macintosh machine
On Macintosh operating system
1.
2.
3.
4.
Unzip the installation file
Type ‘make clean’
Type ‘make’
Type ‘make run’
On other operating systems
Image acquisition function is not available to other operating systems. However, other image
process functions are still available. The installation steps are same as those for Macintosh.
C.2 Menu Commands
The program has the following menus:
 File
 Image
 Configure
C.2.1 File
Menu Command
New User
Manage
Exit
Description
Brings up an input dialog box. After the user inputs an ID, it will ask the
user to choose an order to take photos. Then an instruction manual will
display to help the user get through the image acquisition process. The
user clicks the ‘start’ button to start the image acquisition process.
Brings up a manage dialog box. The administrator can use this dialog box
to delete images, to export pre-processed images to target directory and to
pre-process images.
Terminates the application process
47
C.2.2 Image
Menu Command
Remove Background
Preprocess
Detect feature
Description
Removes the background of the selected images. The submenu ‘Files’
allows selecting multiple files and the submenu ‘Directories’ allows to
select multiple directories.
Pre-process the selected images. The submenu ‘Files’ allows selecting
multiple files and the submenu ‘Directories’ allows to select multiple
directories.
Detect features of the selected images. The submenu ‘Files’ allows
selecting multiple files and the submenu ‘Directories’ allows to select
multiple directories.
C.2.3 Configure
Menu Command
Default Settings
Description
Brings up a default settings dialog for changing options and settings
related to image pre-processing.
Background Removal Brings up a file choosing dialog box that allows selecting multiple image
Settings
files. These image files serve as sample files for configuring the values for
background removal.
C.3 FAQ
1.
2.
3.
4.
5.
6.
7.
8.
9.
How can I start collecting Images?
How can I remove the background of images?
How can I do background removal settings?
Do I have to do background removal settings every time?
How can I pre-process the images?
How can I detect feature of the images?
Is it possible to select the whole directory as the input?
How can I delete the image files?
Which directory stores the image files?
C.3.1 How can I start collecting Images?
1.
2.
3.
4.
File->New User
Enter a name
Choose the type of order
Read the instruction manual
48
5. Click “start” button
6. Follow the instructions
C.3.2 How can I remove the background of images?
1. If you have not done the settings for background removal, you need to do that in advance.
(Check C.1.3 How can I do background removal settings?)
2. Image->Remove Background->Files
3. Select the image files (can be single or multiple)
4. Select the target directory
5. Click “OK” button
C.3.3 How can I do background removal settings?
1. Configure->Background Removal Settings
2. Select some sample image files
3. Use mouse to drag a rectangle on the blue background
49
4. Repeat step 3 until get a good result
5. Change Horizontal and Vertical value to get better result
6. Use “Undo” or “Remove” button to correct mistakes
7. Check if the settings fit other sample files
8. Click “save” button.
C.3.4. Do I have to do background removal settings every time?
Not necessary as long as the photos are taken in the same environment. However, for safety purpose,
it is always suggested to check some sample files using the previous settings.
1. Configure->Background Removal Settings
2. Select some sample image files
3. Click ‘Load’ button
50
C.3.5 How can I pre-process the images?
1.
2.
3.
4.
Image->Preprocess->Files
Select the image files (can be single or multiple)
Select the target directory
Click “OK” button
C.3.6 How can I detect feature of the images?
1.
2.
3.
4.
Image->Detect Feature->Files
Select the image files (can be single or multiple)
Select the target directory
Click “OK” button
C.3.7 Is it possible to select the whole directory as the input?
Yes. Select “Directories” instead of “Files” on the submenu. For example:
Image->Detect Feature->Directories
C.3.8 How can I delete the image files?
1. File->Manage
2. Tick the check box
3. Click “delete” button
C.3.9 Which directory stores the image files?
Under HOME_DIRECTORY/album
51
52