1
CHAPTER 1
INTRODUCTION
1.1
Introduction
This chapter describes the introduction of the research effort presented throughout
this project. It illustrates the overview of the research that encourages the establishment
of defect prediction model for software testing phase. The discussion continues with
background of the research, problem statements, research objectives and the importance
of the research. The scope of work and project outline is then explained in the last
sections of this chapter.
1.2
Introduction to Defect Prediction Model for Software Testing
As an organization that aims to become a premier applied research centre in
frontier technologies, MIMOS has always committed to develop, produce and release
high quality software to the market. One of the key aspects to ensure it can be achieved is
by having effective and efficient software development process throughout entire SDLC.
Thus, prediction or estimation of defects for particular software during testing phase is
very crucial to enhance the testing process as part of process improvement in SDLC.
2
Being the last gate before acknowledging that the particular software is ready to
go to the market requires strong and accurate data and metrics. The initiative on having
defect prediction model for testing phase helps in determining defects that are likely to
occur during test execution and contributes in providing relevant software quality
metrics. Defect prediction model for testing contributes to zero-known post release
defects of a software product. This is determined by defect containment in testing phase.
Predicting total number of defects at the start of testing allows for wider test coverage to
be put in place. As more defects contained within testing phase, it helps in improving
quality of software product being delivered to end user. By using testing metrics for
predicting total defects, it demonstrates the stability of development effort of releasing a
software product.
1.3
Background of Company
MIMOS or Malaysian Institute for Microelectronics System was established on 1st
January 1985 as a unit of Prime Minister’s department following the initiative by group
of academicians led by Tengku Dr. Mohd Azzman Shariffadeen. The initial objective is
to conduct microelectronics research to support the industries as well as to develop
indigenous products. After going through corporatization exercise as a company under
Ministry of Finance (MOF), MIMOS has been focusing on three (3) core functions:
Research and Development (R&D), National IT Policy Development and Business
Development. Since then, MIMOS has embarking on various initiatives and projects
including manufacture affordable personal computer (PC), commission industrial-class
water fabrications plant, launch first Malaysia’s first Internet Service Provider (ISP)
called JARING, initiate Computer Forensic Services and also launch AgriBazaar. On 1
July 2006, Dato’ AbdulWahab Abdullah was appointed as new President and Chief
Executive Officer of MIMOS replacing Tengku Dr. Mohd Azzman Sharifadden. The
appointment of Dato’ Abdul Wahab has turned MIMOS from the R&D organization in
ICT and microelectronics to world class R&D Centre of Excellence.
3
With the tagline “Innovation for Life”, MIMOS is now the premier applied
research centre in frontier technologies aimed at growing globally competitive indigenous
industries. Through smart partnerships with local and international universities, research
institutes, industries and Malaysia Government, MIMOS focuses on frontier technologies
by pursuing exploratory and industry-driven applied research. To date, research and
technology areas in MIMOS are refined into eight (8) technology clusters: Advanced
Informatics, Communication Technology, Cyberspace Security, Encryption Systems,
Grid Computing, Knowledge Technology, Micro Energy, and Micro Systems
(MEMS/NEMS) cluster.
1.4
Background of the Problem
As defect becomes the main intention of software testing, test engineers of Test
Centre of Excellence (Test COE) are expected to find and discover any errors, bugs and
faults in the software through various kinds of testing techniques or strategies. Estimation
of defects to be found upfront at the start of testing phase is very important to strategize
on executing the test for the software. As this research problem concentrates on
formulating a defect prediction model for testing, several issues contribute to this
research problem.
1.4.1
Issue on Better Resource Planning for Test Execution Across Projects
Currently, number of test engineers allocated to a particular testing project is
based on the size of the project including how detail the requirements are and complexity
of the software being developed. At the same time, one test engineer can work in more
than one testing project. Thus, there is a need to estimate the number of defect to be
found in testing phase. Appropriate number of test engineers can be planned and
4
allocated across multiple projects. Estimated number of defects is required to support the
resource planning activities within the testing department to ensure the resources are
optimized and productivity of every test engineer is high.
1.4.2
Issue on Wider Test Coverage to Find Defect
Defects are found while test execution is still in progress. Test engineer applies
various testing techniques to find and discover as many defects as possible based on the
baseline requirements. There is no specific pre-determined factor that a test engineer can
use as a basis to find defects. As a result, test engineer will continue to find defects until
end of the execution schedule. To ensure wider and better test coverage, prediction of
defects could improve the way test engineers find defects as they are now having a target
of defects to be found. This can be in a form of adding more type of testing to use or
adding more and relevant scenarios on how users will utilize the software which results in
better root cause analysis of defects found and improve engineer’s understanding of the
software under test.
1.4.3
Issue on Improving Test Execution Time to Meet Project Deadline
As test engineer need to discover as many defect as possible during text
execution, there might be slip in schedule and cause the delay in the deliverables of the
software work product from the actual planned release date. This due to necessity of
ensuring all testing requirements for the software are fulfilled and covered. Putting defect
prediction in place will reduce and overcome the schedule slippage problem contributed
by testing activities. By having a target on estimated number of defects to be discovered,
every test engineer would be able to plan test execution accordingly to ensure project
deadline is achieved.
5
1.4.4
Issue on Reliability of Software to be Delivered
Defects found during testing phase are given back to development team for bugs
fixing. The fixed software is then retested in several iterations to validate the defects have
been resolved. More defects found within testing phase means more defects are contained
from escaping to field or market that will be using the software. However, test engineers
cannot give the exact figure of defect that be contained within testing phase. This is
where the existence of defect prediction is really needed to provide a direction on how
many defects engineers should discover and contain within the phase. Having this
estimated figure contributes to the zero-known post release defects of the software. In the
long run, the metrics associated with the defect prediction will portray the stability of the
development effort in completing and releasing software product.
1.5
Statement of the Problem
This research is intended to tackle the issues with regard to system testing process
as explained in Section 1.4. The main question to address that is “How to predict the total
number of defects to be found at the start of system testing phase using a model?”.
Followings are the list of sub-questions to support main research questions:
i.
What are the key contributors to test defect prediction model?
ii.
What are the factors that contribute to defect found in system testing
phase?
iii.
How to measure the relationship between the factors of defect with the
total number of defects in system testing phase?
iv.
What is the type of defect category that needs to be considered to calculate
the total defects of the software?
v.
How can the prediction model helps in improving the testing process and
improve the software quality?
vi.
What type of data should be gathered and how to get them?
6
1.6
Objectives of the Study
The research is aimed to achieve following set of objectives to address the issues
and problems mentioned before. The objectives are:
1) To establish a defect prediction model for software testing phase
2) To demonstrate the approach in building a defect prediction model using
Design for Six Sigma (DfSS) Methodology
3) To identify the significant factors that contribute to a reliable defect prediction
model
4) To determine the importance of defect prediction model for improving testing
process
1.7
Importance of Study
As the organization is moving towards achieving CMMI Level 5 status company,
it needs to improve and refine the Verification and Validation (V&V) Process Area. For
this reason, Test Centre of Excellence (Test COE) plays the important role to ensure the
goal is achieved by improving the internal testing process. As the testing approach being
applied is based on V-Model, it is essential to refine all processes with regard to V&V
across all phases in the life cycle, starting from requirement until actual system testing
phase. By having a model to estimate and predict total number of defects in the system
testing phase, it does help the testing team to contribute in achieving the said target.
Putting the defect prediction in the process serves as the preventive mechanism in
reducing the occurrence of defects (Mohapatra and Mohanty, 2001). Furthermore, it
could be beneficial tool to reduce testing time. Reduction in testing time is accomplished
by implementing effective test strategy in minimizing escaped defects while utilizing
resources efficiently. At the same time, the development team can use the model to guide
them in implementing higher quality codes. Obviously by looking from the overall
perspective of software development life cycle, defect prediction model for test does
7
improve the verification and validation process, specifically in ensuring zero-known post
release defects of a software product.
1.8
Scope of Work
In this research, the scope is focused on exploring and establishing a model of
defect prediction specifically for system testing phase. Study on the defects means the
identification of faults, errors and bugs in the software during system testing phase of
software development life cycle. It can be from functional defects, security defects,
usability defects to performance defects. In order to do the prediction for these defects,
analysis is done to determine the factors or contributors to the introduction of defects in
the testing phase. This involves identifying all possible significant factors such as faults
in requirement phase, design phase, code and unit test phase, size of the software, fault
density and historical defects. Moreover, the scope of work also emphasizes on
measuring the capability of test engineers in discovering defects.
The work on modeling is done to establish the relationship between the identified
predictors against the defects found in testing phase. Individual analysis of each possible
predictor need to be performed to determine which factors that has strong connection
with defects. The output is going to be the proposed model that will be normalized to suit
all kind of projects.
1.9
Project Schedule
The research which is also serves as the professional training started from 20
October 2008 until 17 April 2009. However, the actual end date for this project is on 30
May 2009 since it needs to follow the agreed schedule as per Six Sigma Green Belt
project methodology, in which it starts with Define phase, Measure phase, Analyze
8
phase, Design phase and Verify phase. For the purpose of this project, the results and
discussion will be presented up until end of Verify phase schedule. The schedule is
presented as below:
Table 1.1: Project schedule
1.10
Phase
Start Date
End Date
Define
Measure
Analyze
Design
Verify
20/10/2008
01/12/2008
01/02/2009
01/04/2009
01/05/2009
30/11/2008
31/01/2009
31/03/2009
30/04/2009
30/05/2009
Project Outline
This research encompasses the discussion on the several topics and subjects
related to establishing a defect prediction model for testing phase. Thus, the research is
organized into the followings for further understanding of the subject matter.
Chapter 2:
This chapter discusses on the literature review of the defect
prediction model. Discussion involves overview of the several techniques in
predicting software defects across Software Development Life Cycle, some issues
with regard to defect prediction, strategies in predicting defects for software
testing phase as well as application of the defect prediction model in improving
software process.
Chapter 3:
It discusses the research methodology applied in analyzing the
research problem and formulating the proposed solution with regard to Six Sigma
Green Belt DMADV track.
9
Chapter 4:
This chapter outlines the discussion on the outcome of the research
activities that covers the characteristics of data being gathered, analysis of
relationship on possible factors that contributes to the prediction defect as well as
establishing the proposed model that be verified and validated to ensure it is fit to
be incorporated into the software process.
Chapter 5:
This chapter summarizes the research studies that include the
achievement that have been obtained and how the proposed outcome of defect
prediction model for test could contribute to the user. Then, it concludes with the
limitation of the proposed solution together with recommendations for the future
research work.