SOFTWARE TESTING PROCESS FOR SHARED BANKING SERVICES
(SBS) SYSTEM
NORAKMAR BINTI ARBAIN @ SULAIMAN
UNIVERSITI TEKNOLOGI MALAYSIA
SOFTWARE TESTING PROCESS FOR SHARED BANKING SERVICES (SBS)
SYSTEM
NORAKMAR BINTI ARBAIN @ SULAIMAN
A project report submitted in partial fulfillment of the
requirements for the award of the degree of
MSc. (Computer Science – Real Time Software Engineering)
Centre for Advanced Software Engineering
Faculty of Computer Science and Information System
Universiti Teknologi Malaysia
APRIL 2009
iii
DEDICATION
To my beloved mother and father
iv
ACKNOWLEDGEMENT
In the name of The Almighty, I thank you for giving me the strength and
courage in order to face all the obstacles that lie in my journey of completing this
report.
I am very thankful to my Academic Mentor, Mr. Mohd Ridzuan Ahmad for
all his advice and support. Next, my thanks also go to my Industrial Mentor, Mdm.
Haidayu Anuar, for the great hospitality and giving me chance to explore and
experiences the real working environment in HeiTech Padu. Thank you for believing
in me for completing the given tasks.
To my beloved parents, Hj. Arbain and Hjh. Siti Hasnah for their encourage
and love, and also to my fiancé, Mr. Ariffuddin for his patience, assist and support
for whatever barrier that I have faced in managing life in order to complete this
industrial project successfully.
Thank you also to my fellow classmates in RTSE Batch 19, especially to
Nurdatillah and Hannan, for sharing in their knowledge and memories in HeiTech.
Last but not least, thanks again for colleagues at HeiTech and all parties in CASE
that assist and support me in order to accomplish this industrial project either directly
or indirectly. I thank you.
v
ABSTRACT
Software testing is one of the main activities in software development life
cycle. This process consists of a few activities, which includes developing test plan
and strategy, test design, test execution and evaluation of test result. However, a
customized testing process is required for a specific domain in order to ensure the
correctness and completeness of the test result. This project proposed a customized
testing process for Shared Banking Services (SBS) system, which developed by
R@PIC, HeiTech. Study was conducted in order to identify the specific modules in
SBS and its characteristics which differentiate it from other software applications.
Besides that, three testing methodologies, which are HeiTech Testing Process, RUP
Test Discipline and Systematic Test and Evaluation Process (STEP), were compared
based on criteria such as main activities, roles and responsibilities, artifacts and level
of testing. The comparison result was then map to the characteristic of SBS to
produce the proposed software testing process for SBS.
vi
ABSTRAK
Pengujian perisian ialah satu daripada aktiviti utama dalam kitaran hidup
pembangunan perisian. Process pengujian ini mempunyai beberapa aktiviti seperti
membina perancangan dan strategi pengujian, merekabentuk pengujian, perlaksanaan
pengujian dan penilaian hasil pengujian.
Namun, satu proses pengujian yang
lengkap dan bertepatan diperlukan untuk domain tertentu, bagi memastikan
ketepatan dan memenuhi hasil sesuatu pengujian. Projek ini mencadangkan satu
proses pengujian yang sesuai untuk Sistem Perbankkan Perkongsian Perkhidmatan
(SBS) yang dibangunkan oleh R@PIC, di HeiTech. Oleh itu, satu kajian telah
dijalankan bagi menentukan modul khas untuk sistem SBS, dan juga ciri-cirinya
yang berbeza daripada aplikasi perisian yang lain. Disamping itu, tiga metadologi,
iaitu Proses Pengujian HeiTech, Disiplin Ujian RUP dan Proses Penilaian dan
Pengujian Bersistematik (STEP) telah dibuat perbandingan berdasarkan beberapa
kriteria seperti aktiviti utama, peranan dan tanggungjawab, artifak dan paras
pengujian. Hasil daripada pembandingan tersebut, ciri-ciri khusus bagi sistem SBS
dapat ditentukan dan kemudian satu proses pengujian khusus untuk sistem ini dapat
dihasilkan.
vii
TABLE OF CONTENTS
CHAPTER
TITLE
PAGE
DECLARATION
ii
DEDICATION
iii
ACKNOWLEDGEMENT
iv
ABSTRACT
v
ABSTRAK
vi
TABLE OF CONTENTS
vii
LIST OF TABLES
xi
LIST OF FIGURES
xiii
LIST OF ABBREVIATIONS
xv
LIST OF APPENDICES
1
xvii
INTRODUCTION
1
1.1
Organization Background
1
1.1.1 HeiTech Padu Berhad Overview
1
1.1.2 Business and Services at HeiTech Padu Berhad
1
1.1.3 Research and Product Innovation Center
(R@PIC) Division
1.2
2
3
Project Background
4
1.2.1 Industrial Project Overview
4
1.2.2 Shared Banking Services (SBS) System Overview
5
SCOPES AND OBJECTIVES
7
2.1
Vision Statement
7
2.2
Project Objectives
7
2.3
Project Scopes
8
2.4
Project Plan (Gantt chart)
8
viii
2.5
3
Project Deliverables
LITERATURE STUDY
3.1
8
9
Software Testing Process
9
3.1.1 Definition and Principles of Software Testing
9
3.1.2 Relationship between Testing with Verification
and Validation
3.1.3 Software Testing Concepts
13
3.1.4 Software Testing Methods
22
3.1.5 Testing Techniques
23
3.1.6 Types of Testing
28
3.1.7 Level of Testing
30
3.1.8 The Challenges and Current Research
36
Software Testing Model
38
3.2.1 V- Model
38
3.2.2 W- Model
41
3.2.3 Butterfly Model
44
3.2.4 Summary
47
TESTING METHODOLOGIES
50
3.2
4
11
4.1
Introduction
50
4.2
HeiTech Software Testing Process
51
4.2.1 Testing Concepts
52
4.2.2 Main Activities
53
4.2.3 Testing Levels
58
4.2.4 Testing Types
60
4.2.5 Roles and Responsibilities
66
4.2.6 Test Documentations / Artifacts
67
4.2.7 Advantages and Disadvantages
68
RUP Test Discipline
69
4.3.1 Test Concepts
73
4.3.2 Main Activities
75
4.3.3 Roles and Responsibilities
83
4.3.4 Test Documentations / Artifacts
84
4.3
ix
4.4
4.5
5
91
Systematic Test and Evaluation Process (STEP)
93
4.4.1 Testing Concepts
94
4.4.2 Main Activities
95
4.4.3 STEP Architecture
97
4.4.4 Timing of STEP Activities
98
4.4.5 Elements of STEP
100
4.4.6 Test Documentations / Artifacts
102
4.4.7 Roles and Responsibilities
102
4.4.8 Advantages and Disadvantages
103
Summary
105
SHARED BANKING SERVICES (SBS) SYSTEM
107
5.1
Introduction
107
5.2
Core Banking Systems
107
5.3
Shared Banking Services (SBS) at R@PIC
110
5.3.1 Transaction System
112
5.3.2 Utilities
114
5.3.3 System Configuration
117
Testing Characteristics for SBS System
118
5.4.1 Functionality Testing
121
5.4.2 Performance Testing
122
5.4.3 Automation Test
124
5.4.4 Security Test
125
5.4.5 Advantages
126
SBS Testing Implementation at R@PIC
127
5.5.1 Plan Test
128
5.5.2 Design Test
129
5.5.3 Implement Test
129
5.5.4 Evaluate Test
130
5.4
5.5
6
4.3.5 Advantages and Disadvantages
CUSTOMIZED SOFTWARE TESTING PROCESS
6.1
131
Issues with Software Testing Implementation at
R@PIC
131
x
6.2
Developing A Customized Software Testing
Process (CSTP)
132
6.2.1 Main Phases for Testing Correspond to
Software Development Phases
132
6.2.2 Main Activities of Software Testing
Methodologies
6.2.3 Main SBS Testing Characteristics
133
134
6.2.4 Proposed Customized Software Testing
Process (CSTP) for SBS
7
CONCLUSION AND RECOMMENDATION
135
148
7.1
Recap
148
7.2
Project Outcome
149
7.3
Future Study
151
7.4
Summary
151
REFERENCES
Appendices A - C
154
159 - 163
xi
LIST OF TABLES
TABLES NO
TITLE
2.1
List of Project Deliverables and the Targeted Recipients
3.1
Definition of Software Testing
10
3.2
Differences between Verification and Validation
12
3.3
Objectives of Software Testing
16
3.4
Black Box and White Box Testing
25
3.5
Types of Code Coverage
26
3.6
Advantages and Disadvantages of Black-box Testing
27
3.7
Software Testing Techniques
29
3.8
Advantages and Disadvantages of V-Model
40
3.9
W-Model Test Activities
43
3.10
Butterfly Test Model Description
45
3.11
Summary of Software Testing Model
48
4.1
Test Tactics for Different Software Development
Methodology
4.2
PAGE
8
51
HeiTech Application Development Life Cycles
Objectives
52
4.3
Main Activities of HeiTech Test Process
54
4.4
Detail Descriptions of Testing Levels
58
4.5
Types of Testing and Its Descriptions
61
4.6
Testing Types Consideration Based on Levels of
Testing
65
4.7
Roles and Responsibilities
66
4.8
HeiTech Test Documents
67
4.9
Advantages and Disadvantages of HeiTech Test Process
68
4.10
Objectives of RUP Iteration Phases
70
xii
4.11
Nine Disciplines of Rational Unified Process
73
4.12
Roles and Test Artifacts in RUP Test Discipline
85
4.13
Purpose of Test Artifacts
86
4.14
Advantages and Disadvantages of RUP Test Discipline
92
4.15
Major Testing Activities in STEP
97
4.16
Template for Test Documents from IEEE Std. 829-1998
102
4.17
Roles and Responsibilities in STEP
103
4.18
Advantages and Disadvantages of STEP
104
4.19
Test Methodologies Comparison based on Software
Development Phases
105
5.1
Transaction System Categories
112
5.2
Utilities Functions
114
5.3
SBS System Configuration Functions
118
5.4
Challenges and Solutions for Functional Test
121
5.5
Challenges and Solutions for Automation Test
124
6.1
Details of Requirements Reviews Process
139
xiii
LIST OF FIGURES
FIGURE NO
1.1
TITLE
PAGE
Research and Product Innovation Center (R@PIC)
Structure
3
1.2
Shared Banking Services (SBS) Components
5
3.1
Static Verification and Dynamic Validation
15
3.2
Five Schools of Software Testing
19
3.3
Differences of White-box and Black-box Testing
24
3.4
Level of Testing
31
3.5
Unit Testing
32
3.6
Top-Down Integration
33
3.7
Bottom-Up Integration
34
3.8
V-Model Test
39
3.9
W-Model Test
42
3.10
Morton's Butterfly Model
44
3.11
Butterfly Model Test
45
4.1
HeiTech Application Development Life Cycles
52
4.2
Testing Process Flow Diagram
53
4.3
HeiTech Test Process
53
4.4
Illustration of the Rational Unified Process (RUP)
70
4.5
RUP Test Discipline Workflow
75
4.6
Define Evaluation Mission Workflow
76
4.7
Verify Test Approach Workflow
77
4.8
Validate Build Stability Workflow
78
4.9
Test and Evaluate Workflow
79
4.10
Achieve Acceptable Mission Workflow
81
4.11
Improve Test Assets Workflow
82
xiv
4.12
Roles and Test Activities Workflows
84
4.13
Roles and Test Artifacts
84
4.14
Different Views of Testing
95
4.15
Major Phases in STEP
96
4.16
Project Testing Architecture in STEP
98
4.17
Activity Timing at Various Levels of Test
98
4.18
Details Activity Timing at Various Levels of Test
100
4.19
Elements of STEP
100
4.20
Parallel, Mutually Supportive Development
101
5.1
SBS Testing Characteristics
120
5.2
SBS Testing Implementation at R@PIC
127
6.1
Main Phases for Testing Correspond to Software
Development Phases
133
6.2
Main Activities of Software Testing Methodologies
134
6.3
Main SBS Testing Characteristics
135
6.4
Software Testing Methodologies Comparison
136
6.5
The Customized Software Testing Process (CSTP)
137
6.6
Requirements Reviews Process
139
xv
LIST OF ABBREVIATIONS
ADVISE
-
Application Development Information System
ANSI
-
American National Standards Institute
BRS
-
Business Requirement Specification
CA
-
Current Account
CASE
-
Centre for Advanced Software Engineering
CCA
-
Corporate Current Account
CSTP
-
Customized Software Testing Process
DSS
-
Device Service Server
e-GSS
-
Electronic Government Solution Suite
FTT
-
Foreign Telegraphic Transfer
FWTT
-
Foreign Worker Telegraphic Transfer
HeiTech
-
HeiTech Padu Berhad
HESS
-
HeiTech Enterprise Solution Suite
HTP
-
HeiTech Test Process
ICT
-
Information and Communication Technology
IEEE
-
Institute of Electrical and Electronic Engineers
PMB
-
Pos Malaysia Berhad
PSV
-
Personal Saving Account
QA
-
Quality Assurance
R@PIC
-
Research and Product Innovation Center
RFID
-
Radio Frequency Identification
RMC
-
IBM Rational Method Composer
RTSE
-
Real Time Software Engineering
RUP
-
Rational Unified Process
SA
-
Saving Account
SBS
-
Shared Banking Services
SDD
-
System Design Description
xvi
SDLC
-
Software Development Life Cycle
SEPG
-
Engineering Process Group
SOA
-
Service Oriented Architecture
SOAP
-
Simple Object Access Protocol
SOW
-
Scope of Work
SRS
-
System Requirement Specification
STEP
-
Systematic Test and Evaluation Process
UAT
-
User Acceptance Testing
UFS
-
User Functional Specifications
UML
-
Unified Modeling Language
xvii
LIST OF APPENDICES
APPENDIX
TITLE
PAGE
A
Project Plan – Gantt Chart
159
B
Progress Report
162
C
Proposed Software Testing Process For SBS
163
CHAPTER 1
INTRODUCTION
1.1
Organization Background
1.1.1
HeiTech Padu Berhad Overview
HeiTech Padu Berhad [1] is a public listed company, which also as a
Malaysia’s leading ICT Solutions and Services provider. HeiTech draws its strength
from many years of experiences by working with the customers from both public and
private sectors to transformed theirs business from manual processes to automated
systems and provide effective information system solutions. This enables relevant
business decisions to be made accurately and timely.
1.1.2
Business and Services at HeiTech Padu Berhad
i.
HeiTech Padu Berhad Core Business
HeiTech has been transforming businesses and organizations by
providing comprehensive integrated Information and Communications
Technology (ICT) services in Malaysia, which offer customers value-formoney ICT products and services in several areas such as Managed Data
Center Services, Managed Network and Communications Services,
Systems Integration Services and Solution and Consultancy Offerings.
2
ii.
HeiTech Padu Berhad - Electronic Government Solution Suite (e- GSS)
HeiTech’s Electronic Government Solution Suite (e-GSS) is a solution
that links people, process and technology in a seamlessly integrated
manner to deliver value and convenience to the citizens at large.
e-GSS is readily integrated the following solutions in it offering, which
are:-
iii.
Biometric fingerprint solution
Photo captures solution
Card personalization solution
Smart card personalization solution
Barcode solution
RFID solution
Passport printing solution (centralized and decentralized)
Card Printing solution (centralized and decentralized)
Document scanning solution
Digital signature
HeiTech Enterprise Solution Suite (HESS)
HeiTech Enterprise Solution Suite (HESS) is a set of products that ease
the implementation of an enterprise system, which enable applications
residing on legacy systems to be offered via multi-delivery channels such
as web browser, self-service kiosk and mobile devices. This suite is able
to support multi-protocols and is available both for open source and
window-based platforms. HESS consists of 4 products which are:
E-Connect
RFID Middleware
Device Service Server
Hybrid Client
3
iv.
HeiTech Padu Berhad Emerging Business
HeiTech has also ventured into non-traditional areas of expertise such as:
1.1.3
Content Development & Distribution
Data Management & Processing
Electronic Commerce
Research and Product Innovation Center (R@PIC) Division
Research and Product Innovation Center (R@PIC) Division or previously
known as Applied Research and Development (AR&D) Division was formed in
October 2001.
Figure 1.1 below shows the organizational structure of R@PIC
division.
Figure 1.1 : Research and Product Innovation Center (R@PIC) Structure
4
Objectives and scope of this division are to research, develop and enhance
HeiTech proprietary software products, to develop application components that are
application independent, to conduct research on new and emerging technologies that
could be beneficial to software development in HeiTech and also to promote
knowledge sharing culture in HeiTech as a whole.
1.2
Project Background
1.2.1
Industrial Project Overview
Software testing process is important in software development activities.
Therefore, this industrial project is focused on software testing process that currently
practices in HeiTech. This research also covers on several testing process that being
establish and currently practice in many organizations, which are RUP Test
Discipline [2,3] and Systematic Test and Evaluation Process (STEP) [4,5]. Hence,
these testing methodologies are being chosen in order to refer for several criteria that
can implemented later for the proposed software testing process on banking
application, such as Shared Banking Services (SBS) system.
Shared Banking Services (SBS) system [6,7] is currently in development
stage and the contribution requires for this industrial project is in testing phase of this
system. This SBS system is being developed by Product Development Team in
Research and Product Innovation Center (R@PIC) Division at HeiTech. Hence, this
project also defined SBS modules and its characteristics that require specific
software testing process.
Consequently, from this research project, a customized software testing
process will be recommended for R@PIC Division based on a comparison study
from that three testing methodologies.
The proposed framework of customized
software testing process is based on SBS system’s characteristics. Therefore, this
5
software testing process can be practices in future as it suits for testing client-server
application or banking application system.
1.2.2
Shared Banking Services (SBS) System Overview
Shared Banking Services (SBS) [6] is a counter-based transaction system
developed on top of a software framework (Hybrid Client) for developing a frontend, transaction based system. SBS system offers selected banking services that can
be carried out at Pos Malaysia (PMB) branches. SBS consists of two main systems
namely transaction systems and support or utility functions. Figure 1.2 shows the
overall components view of Shared Banking Services (SBS) application system.
Figure 1.2 : Shared Banking Services (SBS) Components
6
Technically, Shared Banking Services (SBS) make use of components
provided by Hybrid Client and Device Service Server (DSS) in its execution. Hybrid
Client components provide common services of a transaction system and Device
Service Server (DSS) offers services for device sharing and device integration.
This SBS system will be installed at PMB branches to enable offerings of
Bank A services at the selected locations. SBS system adopts the smart client
architecture which makes use of local PC processing power to process Bank A’s
transactions. Smart Client architecture combines the benefits of both Thin Client and
Rich Client and it offers rich user interface, ease of integrating to local devices and
simple deployment model using ClickOnce technology.
ClickOnce technology will enable the deployment of SBS modules and
updates to be centralized. It has a web server that will act as the deployment server
to distribute SBS to the selected PMB branches. The communication between PMB
branches to the servers at PMB Headquarters (PMB HQ) will be done via web
services. In addition, SBS will make use of Simple Object Access Protocol (SOAP)
to communicate with the backend (host) system.
CHAPTER 2
SCOPES AND OBJECTIVES
2.1
Vision Statement
This project is to propose a Customized Software Testing Process (CSTP)
that suitable for Shared Banking Services (SBS) system based on a comparison
between three software testing methodologies, which are HeiTech Test Process, RUP
Test Discipline and Systematic Test and Evaluation Process (STEP).
2.2
Project Objectives
The objectives of this project are:
i.
To study and understand HeiTech’s current software testing process.
ii.
To study and understand RUP Test Discipline and Systematic Test
and Evaluation Process (STEP).
iii.
To study and understand the modules and characteristics of Shared
Banking Services (SBS) system.
iv.
To conduct comparison study between (i) and (ii) and to map the
result back to SBS system
v.
To develop a customized software testing process for SBS system
based on the result produce from the comparison study in (iv)
8
2.3
Project Scopes
i.
The proposed software testing process was develop and to be
implemented on the SBS system developed by R@PIC, HeiTech
ii.
The proposed software testing process only covers the plan and
activities to be conducted.
It will not cover the content of any
documents to be produced.
iii.
This project is only an initial study. The proposed software testing
process was not implemented on any real work.
2.4
Project Plan (Gantt chart)
The project plan and timetable can be referred in the Attachment A.
2.5
Project Deliverables
Table 2.1 below shows the list of deliverables produced throughout the project
period and the targeted recipient of the deliverables.
Table 2.1 : List of Project Deliverables and the Targeted Recipients
No.
1.
2.
3.
Items
Descriptions
Recipient
Attachment
Final Report
Industrial Final
Academic Mentor
-
Project Report
Industrial Mentor
Progress
Industrial Progress
Academic Mentor
Reports
Report
Industrial Mentor
Software
The Proposed
Industrial Mentor
Testing Process
Software Process for
Academic Mentor
(STP) Diagram
SBS System
B
C
CHAPTER 3
LITERATURE STUDY
3.1
Software Testing Process
In this chapter, the discussion will highlight several other literatures, which
are related to the objectives of this report. Every literature will focus on the detail of
the definition of pertinent terminology used in this research project and also will
critically discuss and evaluating past and current research of all known similar and
relevant on-going research on this specific topic.
This literature study will emphasis on the software testing process as software
testing is one of the important phases in software or product development life cycle.
In order to conduct this research project, some reviews of relevant information from
the past and current research that related to software testing process and activities
had been made.
3.1.1
Definition and Principles of Software Testing
Software testing in general is an active process of seeking errors. From the
history of software testing, the first definition is “Testing is the process of executing
a program with the intent of finding errors” [8]. Therefore, software can be refers as
“Computer programs, procedures, and possibly associated documentation and data
10
pertaining to the operation of a computer system” [9]. While, testing can be refers as
“The process of analyzing a software item to detect the differences between existing
and required conditions (that is, bugs) and to evaluate the features of the software
item.” [10].
In addition, testing can be described as to find out how well something can
works. In terms of human beings, testing can tells what level of knowledge or skill
has been acquired. Where else, in computer hardware and software development,
testing is used as a key checkpoint in the overall process to determine whether the
objectives are being met.
Hence, software testing is “any activity aimed at
evaluating an attribute or capability of a program or system and determining that it
meets its required results” [4]. Another expanded definition of software testing can
be referred as “Testing is a concurrent lifecycle process of engineering, using and
maintaining testware in order to measure and improve the quality of the software
being tested” [5].
Therefore, software testing is used to help and identify the correctness,
completeness, security, reliability and quality of developed software, where software
testing can be defined as “questioning a product in order to evaluate it”. In addition,
testing also is a technical investigation of the product under test conducted to provide
stakeholders with quality-related information [11].
From years to years, the
definition of software testing evolved simultaneously with the notion of the
practitioner who involved in software engineering fields.
Table 3.1 shows the
summary of several established definition of software testing.
Table 3.1 : Definition of Software Testing
Year
Author
Definition
1979
Glenford J.
Testing is the process of executing a program or
Myers
system with the intent of finding errors.
11
Year
Author
1983
Bill Hetzel
Definition
Testing is the process of establishing confidence
that a program or system does what it is supposed
to.
1988
Bill Hetzel
Testing is any activity aimed at evaluating an
attribute or capability of a program or system and
determining that it meets its required results.
2002
3.1.2
Rick D. Craig
Testing is a concurrent lifecycle process of
and
engineering, using, and maintaining testware in
Stefan P.
order to measure and improve the quality of the
Jaskiel
software being tested.
Relationship between Testing with Verification and Validation
Software testing is the process of validation and verification of the software
product with intention that the product can meets the business and technical
requirements that guided its design and development, and it will works as expected
[12].
From IEEE Standard, the definition of verification is “The process of
evaluating software at the end of the software development process to ensure
compliance with software requirements” and validation is “The process of
determining whether or not the products of a given phase of the software
development cycle fulfill the requirements established during the previous phase”
[13].
In addition, Schmidt, Michael E. C. [14] further clarifies of these terms as the
verification process provides supporting evidence that the software and its associated
products:
i.
Comply with requirements (for example, for correctness, completeness,
consistency, accuracy) for all life cycle activities during each life cycle
process (acquisition, supply, development, operation, and maintenance);
12
ii.
Satisfy standards, practices, and conventions during life cycle processes;
and
iii.
Establish a basis for assessing the completion of each life cycle activity
and for initiating other life cycle activities.
Meanwhile, the validation process provides supporting evidence that the
software satisfies system requirements allocated to software, and solves the right
problem. Many of traditional verification and validation involves relatively simple
tests that focus on some part of a specification such as requirements specification,
design specification and others. Thus, testing coverage is measured by checking that
all parts of all of the controlling specifications have tests associated with them, and
not necessarily how powerful or insight provoking those tests might be [15].
Additionally, the verification and validation in software testing are often used
interchangeable, but actually these terms have differences definition. Thus, these
differences are important to software testing activities [16]. Table 3.2 below show
the differences between verification and validation in software testing practice.
Table 3.2 : Differences between Verification and Validation
Differences
Target
Verification
Validation
Are we building the product Are we building the right
right?
product?
When to
In-process activities performed Performed after the software is
perform?
concurrently
development.
with
software developed to determine if the
software correctly implements
the requirements.
13
Differences
IEEE
Verification
The
process
of
Validation
evaluating The process of determining
Standard
software at the end of the whether or not the products of a
Definition
software development process to given phase of the software
ensure compliance with software development cycle fulfill the
requirements.
requirements established during
the previous phase.
Strategies
•
Requirements Review
•
Unit Testing
•
Design Review
•
Integration Testing
•
Code Walkthrough
•
System Testing
•
Code Inspections
•
Performance Testing
•
Alpha Testing
•
User Acceptance Testing
(UAT)
3.1.3
•
Installation Testing
•
Beta Testing
Software Testing Concepts
3.1.3.1 Concepts
Generally, software testing concept is more than just debugging the program.
The purpose of testing can be quality assurance, verification and validation, or
reliability estimation on specific product or system. Testing can be used as a generic
metric as well where the correctness testing and reliability testing are two major
areas of testing activities. Therefore, software testing is a trade-off between budget,
time and quality. The importance of testing can be understood by the fact that
“around 35% of the elapsed time and over 50% of the total cost are expending in
testing programs” [17-18].
14
Consequently, software testing has three main purposes which are
verification, validation, and defect finding [12].
i.
The verification process confirms that the software meets its technical
specifications. A “specification” is a description of a function in terms of
a measurable output value given a specific input value under specific
preconditions. For example, a simple specification may be along the line
of “a SQL query retrieving data for a single account against the multimonth account-summary table must return these eight fields <list>
ordered by month within 3 seconds of submission.”
ii.
The validation process confirms that the software meets the business
requirements. A simple example of a business requirement is “After
choosing a branch office name, information about the branch’s customer
account managers will appear in a new window. The window will present
manager identification and summary information about each manager’s
customer base: <list of data elements>.” Other requirements provide
details on how the data will be summarized, formatted and displayed.
iii.
A defect is a variance between the expected and actual result.
The
defect’s ultimate source may be traced to a fault introduced in the
specification, design, or development (coding) phases.
Moreover, a
defect also can be defined as “any unintended characteristic that impairs
the utility or worth of an item, or any kind of shortcoming, imperfection,
or deficiency” [19].
In order to satisfy the objectives of the verification and validation process,
both static and dynamic techniques of system checking and analysis should be used.
Figure 3.1 shows the differences of static verification and dynamic validation
techniques.
15
Figure 3.1 : Static Verification and Dynamic Validation
A form of verification that does not require execution of the software, such as
model checking, is called static analysis. These static techniques are concerned with
analysis and checking of system representations such as requirements document,
design diagrams and others. Static techniques are also known as non-execution
based process which includes activities such as walkthroughs, inspections, formal
verification.
Conversely, as testing requires the execution of the software, it is often called
dynamic analysis. These dynamic techniques are applied when a prototype or an
executable program is available which also known as testing activities.
3.1.3.2 Objectives of Software Testing
After reviewing some definitions and concepts of software testing that being
defined from others perspective, the main purpose of practicing software testing in
software development is also needed to be put inside this literature review.
Different objectives of software testing require different testing strategies and
these will yield different test planning, different test scripts and different test results
16
[20]. In practical industry, testing activities is carried out for many different reasons
[21].
Table 3.3 below shows some objectives of software testing in real
environment.
Table 3.3: Objectives of Software Testing
Objectives
Descriptions
Find defects
This is the classic objective of testing. A test is run in
order to trigger failures that expose defects. Generally,
tester looks for defects in all interesting parts of the
product.
Maximize bug count
The distinction between this objective and “find
defects” is that total number of bugs is more important
than test coverage. Tester might focus narrowly, on
only a few high risk features, if this is the way to find
the most bugs in the time available.
Block premature
Tester stops premature shipment by finding critical bugs
product releases
or serious errors that no one would ship the product until
they are fixed. For every release decision meeting, the
tester’s goal is to have new show stopper bugs.
Help managers make
Managers are typically concerned with risk in the field.
ship / no-ship
They want to know about coverage (maybe not the
decisions
simplistic code coverage statistics, but some indicators
of how much of the product has been addressed and how
much is left), and how important the known problems
are. Problems that appear significant on paper but will
not lead to customer dissatisfaction are probably not
relevant to the ship decision.
Minimize technical
Working in conjunction with a technical support or help
support costs
desk group, the test team identifies the issues that lead
to calls for support. These are often peripherally related
17
Objectives
Descriptions
to the product under test.
For example, getting the
product to work with a specific printer or to import data
successfully from a third party database might prevent
more calls than a low-frequency and data-corrupting
crash after deployment.
Assess conformance to Any claim made in the specification is checked.
specification
Program
characteristics
not
addressed
in
the
specification are not (as part of this objective) checked.
Conform to
If a regulation specifies a certain type of coverage (such
regulations
as, at least one test for every claim made about the
product), the test group creates the appropriate tests. If
the regulation specifies a style for the specifications or
other documentation, the test group probably checks the
style. In general, the test group is focusing on anything
covered by regulation and (in the context of this
objective) nothing that is not covered by regulation.
Minimize safety-
Any error that could lead to an accident or injury is of
related lawsuit risk
primary interest. Errors that lead to loss of time or data
or corrupt data, but that do not carry a risk of injury or
damage to physical things are out of scope.
Find safe scenarios for Sometimes, all that tester is looking for is one way to do
use of the product
a task that will consistently work, with one set of
(find ways to get it to
instructions that someone else can follow that will
work, in spite of the
reliably deliver the benefit they are supposed to lead to.
bugs)
In this case, the tester is not looking for bugs. He is
trying out, empirically refining and documenting, a way
to do a task.
Assess quality
This is a tricky objective because quality is multidimensional.
The nature of quality depends on the
18
Objectives
Descriptions
nature of the product. For example, a computer game
that is rock solid but not entertaining is a lousy game.
To assess this quality, measure and report back on the
level of specify quality is required.
Tester probably
needs a clear definition of the most important quality
criteria for this product, and then also need a theory that
relates test results to the definition.
Verify correctness of
It is impossible to do this by testing. Testers can prove
the product
that the product is not correct or they can demonstrate
that no errors founds in a given period of time using a
given testing strategy.
However, they cannot test
exhaustively, and the product might fail under
conditions that they did not test. The best way to do is
by assessment test-based estimation of the probability of
errors.
3.1.3.3 Software Testing Schools of Thought
Recently, there are five schools of thought for software testing that being
established by mostly experienced people in software testing fields. Software-testing
experts often disagree with one another, but up until 2003 it was difficult to have a
conversation with a fellow tester for lack of a common frame of reference [22].
Therefore, for this reason, Pettichord B. [23] and some colleagues has
formulated a concept of schools, which known as the Five Schools of Software
Testing as shown in Figure 3.2.
19
Figure 3.2: Five Schools of Software Testing
The Schools of Software Testing began as four and has since expanded to
five schools, which are Analytic, Standard, Quality, Context-Driven, and Agile. The
schools are meant to categorize beliefs and how they influence for tester tactical
approach to testing, but they are not an absolute [22].
In addition, these schools are defined by the intellectual affinity, social
interaction and common goals that the people inside of the schools thought are
pursue to achieve.
Besides that, these schools also are made up from several
hierarchies of values, exemplar techniques, standards of criticism, based on the
institutions and some common related vocabulary. However, a common doctrine or
specific techniques will not being as consideration inside these five schools of
software testing.
i.
Analytic School
The Analytic school focuses on the use of analytics, specifications, and
measurable methods. Therefore, the proponents of the Analytic school seek
into objective measurements. They want to know what percent done they are,
what are the degrees of confidence, and what is the code coverage.
20
This code coverage is a exemplary of the Analytic school, where the code
coverage matrix that shows how many lines of code have been executed or
exercised by performing the tests is the basis of testing.
In addition, this school is less concerned with creative or exploratory
thinking, and more focused on measurable methods. This approach to software
testing may be a good fit for scientific or regulated environments of testing,
where an expectation that everything will be covered by dissecting and
analyzing things, and following a logical flow.
ii.
Standard School
The Standard school was previously known as Factory. So, the tasks
are routine along a conveyor belt, like a real factory, where this school
approaches testing as a routine or very repetitive task. Since this is the case,
these individuals like to find the lowest possible labor resource to conduct
testing.
The exemplar of this school is Requirements Traceability Matrix.
Therefore, the basic principle of the Requirements Traceability Matrix is that
every requirement and its test or test cases are listed, and after every test is
conducted and been through task, then the software is done and ready to ship.
However, this notion seems have a drawbacks when sometimes testing
does not always work out as planned, because there are cases where the
requirements are inaccurate or incomplete, or the tests identified for a
particular requirement cannot shown all of the possible ways that requirement
would pass or fail. At the end, the software that passes all the tests without
really hit the quality standard that being listed in the testing plan.
iii.
Quality School
21
The Quality school is focused more on standards and process than
testing. The people in this school may feel that they are keepers of the product.
The role of the gatekeeper is used as a key exemplar of the Quality school.
Therefore, a test manager or the QA lead serves as either the sole or primary
decision maker in whether or not a product is ready for release.
iv.
Context-Driven School
The Context-Driven school is based on seven core principles, two of
which particularly resonate. The principles number three and four address
people to work together as the most important part of the project, and the
nature of projects to unfold over time in ways that are often unpredictable.
Therefore, the exemplar that many associate with the Context-Driven School is
exploratory testing where exploratory testing is the concurrent test design, test
execution, and learning.
v.
Agile School
The principles for the Agile school are very clear, where the concepts
of always having a deliverable of a workable piece of software, where they
continuous the delivery, while having working on the software. Therefore, the
exemplar for Agile school is automated unit tests, as used in test-driven
development or test-first development.
For example, in extreme programming, which is the paradigm for
Agile, the developers do test-driven, test-first development, and they test their
own code. Hence, they build a large suite of automated unit tests that they run
before every build, with the idea that every build is a potential release
candidate.
22
This condition does not mean the software is done, but that what it is
should work. As a result, they build that out to where they believe it is ready,
and then they ship it out to the client. Finally, the client then does what the rest
of the schools label as an extended user acceptance testing.
3.1.4 Software Testing Methods
3.1.4.1 Manual Software Testing
In performing software testing, two basic methods that can be referred are
manual testing and automated testing.
Manual software testing is a process of
manually testing the software, which is having the possible forms for example, user
interfaces navigation, information submission, or attempt to hack the software or
database, carried out by an individual or individuals [24].
Consequently, some test conditions where manual software testing is more
appropriate to be performed are:
•
User interface or usability testing
•
Exploratory or ad hoc testing (where testers do not follow a ‘script’, but
rather testers ‘explore’ the application and use their instincts to find bugs)
•
Testing areas of the application which experience a lot of change.
•
User acceptance testing
However, the time commitment involved with manual software testing is one
of its most significant drawbacks. Yet, manual software testing is a labor-intensive
and slows [25]. The time needed to fully test the system will typically range from
weeks to months and this variability of results depending on who is performing the
tests, which also can increase the delay. For these reasons, many organizations will
look into automation testing as a means of accelerating the software testing process
while minimizing the variability of results.
23
3.1.4.2 Automated Software Testing
Automated software testing is a process of creating test scripts, which can be
run then automatically, repetitively, and through a number of iterations [25]. This
automated software testing can helps to minimize the variability of results, speed up
the testing process, increase test coverage and also provide greater confidence in the
quality of the software being tested.
Conversely, there are some test conditions that automated software testing is
not appropriate, which includes:
•
End user usability testing is not typically a good candidate for automated
testing.
•
Tests which will not be run more than a couple of times are typically not a
good candidate for automated tasting, since the payoff of in test
automation comes after many test executions.
•
Tests for areas of the application which experience a lot of change are
also not a good candidate for automation since this can lead to substantial
maintenance of test automation scripts. Such areas of the application may
be more effectively tested manually.
Furthermore, good test automation architecture, such as a keyword-driven
testing framework, will reduce the overall cost of ownership of the organization for
test automation by minimizing maintenance expense and increasing the number of
automated tests [24].
3.1.5 Testing Techniques
In general, software testing can be based on a strategy like White-box Testing
or Black-box Testing. These strategies also sometimes are being defined as testing
techniques.
24
Black-box testing is carried out against the functional specifications in order
to check for any abnormal system behavior. In contrast, white-box testing deals with
the program’s internal logic and code structure [25]. Figure 3.3 below shows the
differences between Black-box and White-box testing.
Figure 3.3: Differences of White-box and Black-box Testing
By comparison, white-box testing does not focus on the problem area, and
therefore may not discover that some sub-problem is left unsolved by the program,
whereas black-box testing should cover. Conversely, black-box testing does not
focus on the program text, and therefore may not discover that some parts of the
program are completely useless or have an illogical structure, whereas white-box
testing can cover.
Furthermore, these two testing techniques, white-box testing and black-box
testing are important for functionality testing. Therefore, white-box testing and
black-box testing are complementary approaches to test case generation [26]. Table
3.4 shows the types of testing and test examples for black box and white box testing
techniques.
25
Table 3.4: Black Box and White Box Testing
Techniques
Types of Testing
Example
Artifacts
Requirements
Black box
•
Functional Testing
•
Ad-hoc Testing
testing
•
Stress Testing
•
Boundary Value
•
Load Testing
•
Exploratory Testing
•
Category Partition
•
Usability Testing
•
Classification
•
Smoke Testing
•
Recovery Testing
•
Volume Testing
•
User Acceptance
Analysis
Trees
•
Graphs
•
Equivalence
Partitioning
Testing
•
Cause-effect
Alpha and Beta
•
Partition Testing
Testing
•
Predicate Testing
•
Random Testing
•
Adequacy
White box
•
Unit Testing
testing
•
Static and Dynamic
Code
Assessment
Analysis
•
Coverage Testing
•
Mutation Testing
•
Data-flow Testing
•
Statement Coverage
•
Domain Testing
•
Branch Coverage
•
Mutation Testing
•
Security Testing
•
Path Testing
•
Structural Testing
3.1.5.1 White-box Testing
In more detail, white-box testing is a strategy in which testing is based on the
internal paths, structure, and implementation of the software under test. Therefore,
white box testing generally requires detailed programming skills [27].
26
Moreover, white-box testing also involves looking at the structure of the code
[28]. Tester needs to know the internal structure of a product in order to conduct
testing to ensure that the internal operations performed according to the specification
and all internal components have been adequately exercised.
Hence, this technique involves the coverage of the specification in the code.
Table 3.5 below shows six types of code coverage in white-box testing techniques.
Table 3.5: Types of Code Coverage
Coverage Types
Descriptions
Segment
Each segment of code between control structures is executed
coverage
at least once.
Branch Coverage
or Node Testing
Each branch in the code is taken in each possible direction at
least once.
Compound
When there are multiple conditions, tester must test not only
Condition
each direction but also each possible combinations of
Coverage
conditions, which is usually done by using a ‘Truth Table’
Basis Path
Each independent path through the code is taken in a pre-
Testing
determined order.
Data Flow
To track the specific variables through each possible
Testing (DFT)
calculation, thus defining the set of intermediate paths
through the code.
This approach tends to reflect
dependencies but it is mainly through sequences of data
manipulation. It is also tends to uncover bugs like variables
used but not initialize, or declared but not used.
Path Testing
Path testing is where all possible paths through the code are
defined and covered. This testing is extremely laborious and
time consuming.
Loop Testing
These strategies relate to testing single loops, concatenated
loops, and nested loops.
Loops are fairly simple to test
unless the dependencies exist among the loop or between a
loop and the code it contains.
27
Furthermore, white-box testing is more effective rather than black-box
testing, in terms of covering numerous sorts of defects in the program. White-box
testing is also sometimes called as Structural or Glass box testing. Therefore, here
some of related defects that can be discovered:
•
Logic errors and incorrect assumptions, which are inversely
proportional to the probability that a program path will be executed.
This error tend to creep into the design and implement functions,
where the conditions or controls are out of the program
•
The logical flow of the program is sometimes counterintuitive,
meaning that unconscious assumptions about flow of control and data
may lead to design errors that are uncovered, except only when path
testing starts.
•
Typographical errors, which are random, some of which will be
uncovered by syntax checking mechanisms but others will go
undetected until the specific testing begins.
3.1.5.2 Black-box Testing
Black-box testing technique is a strategy in which testing is based solely on
the requirements and specifications. This technique requires no knowledge of the
internal paths, structure, or implementation of the software under test [27].
In
addition, black-box testing also is a test design method [28]. Therefore, this testing
treats the system as a "black-box", so it does not explicitly use any knowledge of the
internal structure of that box.
Table 3.6 below shows some advantages and
disadvantages of Black-box Testing.
Table 3.6: Advantages and Disadvantages of Black-box Testing
Advantages
Disadvantages
•
Tester can be non-technical.
•
This testing is most likely to find
that
those bugs as the user would find.
programmer.
•
Chances of having repetition of tests
are
already
done
by
28
Advantages
•
Testing
helps
vagueness
to
and
Disadvantages
identify
the •
contradiction
in
functional specifications.
•
The test inputs needs to be from
large sample space.
•
It is difficult to identify all possible
Test cases can be designed as soon
inputs in limited testing time. So
as the functional specifications are
writing test cases is slow and
complete.
difficult.
•
Chances of having unidentified
paths during this testing.
3.1.6 Types of Testing
Software testing consists of several subcategories, each of which is done for
different purposes, and often using different techniques. In some cases, there may
even have to be other types of testing such as regulatory-compliance testing,
depending on the type of software and intended industry [24].
Therefore, this software testing can involve or performed for some or all of
the following factors, which are:
•
Business requirements
•
Functional design requirements
•
Technical design requirements
•
Regulatory requirements and programmer code
•
Systems administration standards and restrictions
•
Corporate standards
•
Professional or trade association best practices
•
Hardware configuration
•
Cultural issues and language differences
29
In addition, software testing has a variety of techniques and methods as listed
in Table 3.7.
Table 3.7 : Software Testing Techniques
No.
1.
Techniques
Descriptions
Compatibility
Concentrates
on
testing
whether
the
given
Testing
application goes well with third party tools,
software or hardware platform.
2.
Recovery testing
Focuses on the software to fall in a variety of ways
and verifies that recovery is properly performed.
3.
Usability testing
Attempts to find any human-factor problems or
testing the software to prove or ensure that it is
user-friendly,
as
distinct
from
testing
the
functionality of the software.
4.
Security testing
Attempts to verify that protection mechanisms built
into a system will, in fact, protect it from improper
penetration. For example, password cracking,
unauthorized entry into the software, network
security are all taken into consideration.
5.
Stress testing
Executes a system in a manner that demands
resources in abnormal quantity, frequency, or
volume. It is also conducted for the number of
concurrent users beyond the specified limit. The
objective is to identify the maximum number of
users the system can handle before breaking down
or degrading drastically.
6.
Performance testing
To measure the Response Time, Throughput, and
Utilization of the Web site while simulating
attempts by virtual users to simultaneously access
the site. One of the main objectives of performance
testing is to maintain a Web site with low response
time, high throughput, and low utilization.
7.
Load testing
Load testing is defined as the testing to determine
30
No.
Techniques
Descriptions
whether the system is capable of handling
anticipated number of users or not.
8.
Regression testing
To test or check the effect of changes made in the
code, which is an important software maintenance
activity that attempts to ensure that the addition of
new functionality or the removal of program faults
does not negatively impact the correctness of
product.
3.1.7 Level of Testing
Level of testing can be categorized based on software development phases.
The strategic model that can describe the details of testing level is a V-Model.
Therefore, testing level can be defined start from component or unit testing. This
level of testing is the first test approach or known as test driven development, which
can include functionality, non-functionality and structural testing.
Next level of testing is integration testing, which might be a component
integration testing (between components) or system integration testing (between
systems). Furthermore, the next level of testing is system testing. This testing will
include the functional and non-requirements test activities. Finally, basic approach
of testing level will require high-order tests, which also known as user acceptance
testing.
Therefore, this testing entail the involvement of users or customers for
functional and also non-requirements tests. Additionally, some of testing level also
includes for the test system readiness activities which is focus on system installation
and deployment in the real environment. Figure 3.4 shows the level of software
testing.
31
Figure 3.4 : Level of Testing
3.1.7.1 Unit Testing
Unit testing is a series of stand-alone tests that are conducted for finding
bugs. Each test examines an individual component that is new or has been modified.
In addition, a unit test is also called a module test because it tests the individual units
of code that comprise the application. Therefore, each test validates a single module
based on the technical design documents that was built to perform a certain task with
the expectation that it will behave in a specific way or produce specific results [12].
Furthermore, unit tests primarily focus on functionality and reliability, and
the entry and exit criteria can be the same for each module or specific to a particular
module. Besides that, unit testing is done in a test environment prior to system
integration. Thus, if a defect is discovered during a unit test, the severity of the
defect will dictate whether or not it will be fixed before the module is approved.
Figure 3.5 shows the structure and criteria for Unit Testing.
32
Figure 3.5 : Unit Testing
Generally, a software application is made up of a number of Units, where
output of one Unit goes as an Input of another Unit. Therefore, a Driver is a piece of
software that drives or invokes the Unit that currently being tested. This driver
creates necessary Inputs required for the Unit and then invokes the Unit. Thus, this
Unit may reference another Unit in its logic.
A Stub takes place of such subordinate unit during the Unit Testing. Hence,
this Stub is a piece of software that works similar to a unit which is referenced by the
Unit being tested, but it is much simpler that the actual unit. Besides that, a Stub also
works as a Stand-in for the subordinate unit and provides the minimum required
behavior for that unit.
3.1.7.2 Integration Testing
Integration testing is a systematic technique for constructing the program
structure while at the same time conducting tests to uncover associated errors.
Therefore, objective of this testing is to take unit tested components and build a
program structure that has been stated in system design.
33
An integration testing also differs from system testing as new defects might
be discovered, because not all previously executed tests have to be rerun after the
bugs fixing is made. In addition, integration testing examines all the components and
modules that are new, changed, affected by a change, or needed to form a complete
system [12].
Consequently, integration testing requires involvement of other systems and
interfaces with other applications, including those owned by an outside vendor,
external partners, or the customer. Usually, the two practical methods of Integration
testing are Top-down Integration and Bottom-up Integration approach.
Firstly, Top-down integration testing is an incremental approach to
construction of program structure. Modules are integrated by moving downward
through the control hierarchy, beginning with the main control module.
These
modules subordinate to the main control module are incorporated into the structure in
either a depth-first or breadth-first manner as shown in Figure 3.6 below.
Figure 3.6 : Top-Down Integration
34
The Top-down Integration process is performed in a series of five steps, as
following:
•
The main control module is used as a test driver and stubs are substituted
for all components directly subordinate to the main control module.
•
Depending on the integration approach, selected subordinate stubs are
replaced one at a time with actual components.
•
Tests are conducted as each component is integrated.
•
On completion of each set of tests, stub is replaced with the real
component.
•
Regression testing may be conducted to ensure that new errors have not
been introduced.
Secondly, Bottom-up integration testing begins the structure and testing with
atomic modules, which are the components at the lowest levels in the program
structure as shown in Figure 3.7 below. Therefore, as components are integrated
from the bottom up, the need for stubs is eliminated and this process requires for
components subordinate to a given level is always available.
Figure 3.7 : Bottom-Up Integration
35
The Bottom-up integration testing strategy may be implemented with the
following steps, which are:
•
Low level components are combined into clusters that perform a specific
software sub function.
•
A driver is written to coordinate test case input and output.
•
The cluster is tested.
•
Drivers are removed and clusters are combined moving upward in the
program structure.
3.1.7.3 System Testing
System Testing tests all components and modules that are new, changed,
affected by a change, or needed to form the complete application [12]. Therefore,
this testing required the involvement of other systems.
However, it should be
minimized as much as possible in order to reduce the risk of externally-induced
problems.
Besides that, suppose testing the interaction with other parts of the complete
system is happened in Integration Testing. As a result, system testing is more on
validating and verifying the functional design specification and seeing how all the
modules work together.
Consequently, the first system test is often a smoke test, which is known as
an informal quick-and-dirty run through of the application’s major functions without
bothering with details of the system [12].
Hence, this term is adopt from the
hardware testing practice of turning on a new piece of equipment for the first time
and considering it a success if it does not start smoking or burst into flame.
36
3.1.7.4 User Acceptance Testing (UAT)
User Acceptance Testing is also called Beta testing, application testing, and
end-user testing. Therefore, it can be defined that the testing moves from the hands
of the IT department into the business users’ environment. In general rule of thumb,
no matter how bulletproof an application seems when it goes into user acceptance
testing, a user somewhere can still find a sequence of commands that will produce an
error. This is seems that this testing is important before can start run the real life of
product or system.
3.1.8 The Challenges and Current Research
Software is expected to work, meeting customer’s changing demands, first
time and every time, consistently and predictably. Thus, earlier software systems
were used for back-office and non-critical operations of organizations. However,
now more and more critical applications are implemented globally.
Software is not unlike other physical processes where inputs are received and
outputs are produced.
Software is differs in the manner in which it fails.
Approximately 20 percent of all small computers are abandoned after a short trial
period. The number of software packages acquired but not used is even higher [29].
Relatively, there are many of the excuses that people use to justify bad testing [30]
are such as like these:
•
Excuse 1: Tester can't do good testing without a specification.
•
Excuse 2: Tester can't do good testing without reviewing the code.
•
Excuse 3: Tester can't do good testing if the programmers keep adding
functionality while program under test.
•
Excuse 4: Tester can't do good testing if he gets along too well with the
programmers.
37
Moreover, there are many other issues that can complicate the testing of the
simple function [31]. These are all typical requirements for software systems that a
great many testers face every day during their testing careers, and which act to make
software systems highly complex and make testing an immense challenge. For
example:
•
What if the function needs to interoperate with other functions within the
same application?
•
What if the data for the calculation are obtained across a complex
Client/Server system an/or the result is returned across the Client/Server
system?
•
What if the calculation is driven via a complex Graphical User Interface
with the user being able to type the addition values into fields and push
the buttons to perform the calculation in any arbitrary order?
•
What if this function has to be delivered on a number of different
operating systems, each with slightly different features, and what if
individual users are able to customize important operating system
features?
•
What if this function has to be delivered on a number of different
hardware platforms, each of which could have different configurations?
•
What if the application this function belongs in has to interoperate with
other applications, and what if the user could be running an arbitrary
number of other applications simultaneously (such as e-mail or diary
software)?
Due to this issue, the problem with software, like other complex product, is
that it is difficult to build without defects [29]. Therefore, some of the challenges of
software testing that being defined from the past research [32] are:
•
Ambiguous and incorrect requirements.
•
Tight time schedules.
•
How to cover possible user interactions?
•
How to select a representative set of user test cases?
•
How to test without specifications?
•
How to know when testing is enough?
38
In addition, one software testing’s practitioner [33] also identify about four
fundamental challenges to competent testing, which are:
•
Complete testing is impossible - There is no simple answer for this.
Therefore testers live and breathe tradeoffs
•
Testers misallocate resources because they fall for the company’s process
myths - Testers have to rely on their wits, not on someone else’s
compliance with an (alleged but unrealistic) process
•
Test groups operate under multiple missions, often conflicting, rarely
articulated - Tester pick his own tests to conform to the testing mission
•
Test groups often lack skilled programmers, and a vision of appropriate
projects that would keep programming testers challenged
Although crucial to software quality and widely deployed by programmers
and testers, software testing still remains an art, due to limited understanding of the
principles of software.
This increased expectation for error-free functioning of
software has increased the demand for quality output from software vendors [34].
3.2
Software Testing Model
3.2.1 V- Model
Many of the process models currently used can be more generally connected
by the V-Model where the 'V' describes the graphical arrangement of the individual
phases [35]. This model is also a synonym for Verification and Validation approach
in software development model.
Figure 3.8 show the connection between
development and test activities in V-Model, which can be in time sequence and with
abstraction levels that make it clearer.
39
Figure 3.8 : V-Model Test
The V-Model was originally developed from the waterfall software process
model [36].
The four main phases are requirements, specification, design and
implementation, which have a corresponding to verification and validation testing
phase.
The implementation of codes and detailed design are tested by unit testing,
architecture design is tested by integration testing, formal system specifications are
tested by system testing and finally user acceptance testing verifies the business
requirements.
The V-Model is not only to identify the relationship between the development
and testing phases, but also to ensure that appropriate quality assurance and testing
takes place throughout the project lifecycle. Some advantages and disadvantages
[37, 38, 39] of V-Model are describes in Table 3.8.
40
Table 3.8: Advantages and Disadvantages of V-Model
No.
1.
Advantages
Disadvantages
The V-Model clearly suggests that The V-Model does not explain much
testing should be considered early in on static testing such as reviews,
the life of a project.
inspections, static code analysis and
Static testing techniques such as others.
inspections and walkthroughs can be The V-Model also treats testing as a
performed at early stage of software “back-door” activity on the right
development.
2.
hand side of the model.
The V-Model provides a consistent The V-Model has coarse division,
basis and standard for part of a test which a constructive work (including
approach, or test strategy document, the implementation) on the left-hand
which defines how testing will be side of the “V” and the more
performed throughout the lifecycle destructive tasks on the right-hand
of the project.
3.
side that shows imbalance activities.
The V-Model introduces the idea of A specific planned in removal of
specifying test requirements and defects and regression testing is not
expected
outcomes
prior
to given in this model.
performing the actual tests.
4.
The V-Model provides a focus for The timing of the phases where
defining the testing that must take testing
place
within
each
stage.
after
all
The implementation is done, is not an
definition of testing is assisted by the adequate
idea of entry and exit criteria.
occurs
approach
for
iterative
software processes.
It is also an impression may develop
that, after the implementation phase,
a ready product can be delivered.
5.
The V-Model provides a basis for One-to-one relationship between the
defining who is responsible for documents on the left hand side and
performing the testing at each stage the test activities on the right is
of testing levels.
rarely a perfect approach.
Testing is performed by:
For example:
•
Users - Acceptance testing
•
Functional specifications do not
41
No.
Advantages
Disadvantages
•
System testers - System testing
usually
•
Team leaders - Integration testing
information for a system test.
•
Programmers - Unit testing
•
provide
enough
System tests must often take
account of some aspects of the
business requirements as well as
physical design issues.
•
System testing usually draws on
several sources of requirements
information to be thoroughly
planned.
3.2.2 W- Model
A W-Model that being introduced by Paul Herzlich on 1993 [37, 40] is an
extension of the V-Model. This W-Model removes the vague and ambiguous lines
linking the left and right legs of the V-Model and replaces them with parallel testing
activities, shadowing each of the development activities [40, 39]. Furthermore, WModel of testing also focus specifically on the product risks of concern at the point
where testing can be most effective [37].
W-Model provides a better reflection of the interdependence between the
development team and the test team over the whole course of the software
development process [41]. This model includes testing tasks in the early stages that
are not reflected in the original model, such as the review of requirements, review of
system specifications, review of architecture and detailed design, and code reviews.
Therefore, these testing tasks shows in the inside layer of W-Model. Conversely, the
layer outside, such as Business Requirements block, represents other activities in the
software life cycle prior to the production launch. Figure 3.9 show the illustration of
W-Model.
42
Figure 3.9 : W-Model Test
This model of testing attempts to address shortcomings in the V-Model and
aim more on the development products themselves by represents a standard
development lifecycle with every development stage mirrored by a test activity [37].
Basically, every development activity that produces a work product is shadowed by a
test activity.
The purpose of the test activity particularly is to determine whether the
objectives of a development activity have been met and the deliverable meets its
requirements. It shows that on the left hand side, the deliverables of a development
activity such as write requirements is accompanied by a test activity like test the
requirements.
43
In this model, the verification of non-software products and tasks related with
preparation for the verification and validation of software are concentrated in the
early stages. This verification activity commonly focuses on product’s documents.
Hence, these tests are based on peer reviews, where the documents are subjected to a
review process that being established in the project quality assurance plan.
This review process defines who is responsible for the review, how
weaknesses should be corrected, and how the document is approved for use in
subsequent stages of the development cycle. Table 3.9 below lists two types of
testing activity in the W-Model.
Table 3.9 : W-Model Test Activities
Tests
On
Non-Software
Tests On Software Products
Products
•
Requirements review
•
Preparation of acceptance tests
•
Use case specification review
•
Preparation of system tests
•
Architecture design review
•
Preparation of sub-system integration
•
Detailed design review
tests
•
Preparation of unit tests
•
Code reviews
•
Performance of tests
•
Debugging and introduction of changes
For the later stages of the processes, the work focuses entirely on verification
and validation of the software produced. It is actually different from the original VModel in the later stages, where testing tasks as such as Integration test or System
test are broken down and separated from the work of debugging and error
corrections, which is performed by the developers.
44
3.2.3 Butterfly Model
Another variation on the V-Model is the little known of Butterfly Model by
Stephen Morton on 2001 [48], which shares many features of the W Model [40].
Hence, the Morton’s Butterfly Model shown in Figure 3.10 below.
Figure 3.10 : Morton's Butterfly Model
The butterfly metaphor is based on the idea that clusters of testing are
required throughout the development to tease out information about the
requirements, design and build. Therefore, these micro-iterations can be structured
into the familiar testing stages, and the early static testing stages envisaged by the VModel.
Furthermore, in this model, these micro-iterations explicitly shape the
development during the progression down the development leg. Therefore, each
micro-iteration can be represented by a butterfly, which graphically shows the left
wing for test analysis, the right wing for specification and design, and the body is test
execution, which defined as the muscle that links and drives the test [42].
45
The clear graphical of Butterfly Model for test development activities shown
in Figure 3.11 below.
Figure 3.11 : Butterfly Model Test
Consequently, as a Butterfly, this model is composed of three pieces, which
are two wings and a body. Thus, for each part of it, represents a piece of software
testing activities or process as being described in Table 3.10 below.
Table 3.10 : Butterfly Test Model Description
Butterfly
Testing
Part
Activities
Left Wing
Test Analysis
Descriptions
•
The left wing of the butterfly represents test
analysis, which includes the investigation,
quantization or re-expression of a facet of the
software to be tested.
•
Analysis is both the by product and
foundation of successful test design. In its
earliest
form,
analysis
represents
the
thorough pre-examination of design and test
artifacts to ensure the existence of adequate
testability,
including
checking
for
ambiguities, inconsistencies, and omissions.
•
Test analysis also concerned with validating
46
Butterfly
Testing
Part
Activities
Descriptions
the outputs of each software development
stage or micro-iteration, as well as verifying
compliance of those outputs to the products
of previous stages.
•
Test analysis mechanisms vary according to
the design artifact being examined.
•
Test analysis also serves a valid and valuable
purpose within the context of software
development.
Right Wing
Test Design
•
The right wing of the butterfly represents the
act of designing and implementing the test
cases needed to verify the design artifact as
replicated in the implementation.
•
The focus of test design is to implement
procedures, techniques, and data sets that
achieve the test’s objectives.
•
The outputs of the test analysis phase are the
foundation for test design. Each requirement
or design construct has had at least one
technique (a measurement, demonstration, or
analysis) identified during test analysis that
will validate or verify that requirement. The
tester must now put on his or her
development hat and implement the intended
technique.
•
Software test design, as a discipline, is an
exercise in the prevention, detection, and
elimination of bugs in software.
•
During the test design phase of test
development,
those
tentatively
selected
techniques and approaches must be evaluated
47
Butterfly
Testing
Part
Activities
Descriptions
more fully, until it is proven that the test’s
objectives are met by the selected technique.
Body
Test
•
Execution
Test execution is a separate piece of the
overall approach. Even it is the smallest
piece, as the slender insect’s body, but it also
provides the muscle that makes the wings
work.
•
Test execution includes the formal running
of the designed tests. Informal test execution
is a normal part of test design, and in fact is
also a normal part of software design and
development.
•
This formal test execution marks the moment
in the software development process where
the developer and the tester join forces.
•
In a way, formal execution is the moment
when the developer gets to take credit for the
tester’s work, which by demonstrating that
the software works as advertised.
3.2.4 Summary
Essentially, testing in the software development process is a critical activity.
A lot of evidence suggests that the earlier a fault is found, the cheaper it is to fix.
Testers often complain that they join a project too late to do much good. Although
they can report the faults they find, the critical decisions about usability and
reliability have already been made [43].
48
In real software development, most of the development models have some
deficiencies with regard to the test activities. Testing always starts after the coding
phase at which point the implementation appears to be complete. Testing is one of
the important means of software quality assurance.
Each software development paradigm requires an appropriate test model [49].
Table 3.11 below listed the summary of testing activities of three Test Models
corresponding to the basis software development phases.
Table 3. 11: Summary of Software Testing Model
Software
V-Model
W-Model
Butterfly
Development
Test
Test
Model test
Phases
on Test
Analysis,
Business
Provides Acceptance Reviews
Requirement
Test Plan for User Requirements and Test Design and
Acceptance Testing.
Promotes
Acceptance
Test Test
Plan
User for
for
Execution
Acceptance
Testing.
static Acceptance
techniques Testing.
testing
such as inspections
and walkthrough.
Promotes
Debugging
and
Changes at every
phases.
Formal
Provides System Test Reviews
Specification
Plan
for
Testing.
System System
on Test
Test Design and
Specification and Test
System Test Plan for
for
Analysis,
Execution
System
System Testing.
Testing.
Architecture
Provides
Sub- Reviews
Design
systems Test Plan for Design
on Test
Analysis,
Test Design and
49
Software
V-Model
W-Model
Butterfly
Development
Test
Test
Model test
Phases
Integration Testing.
Architecture
and Test
Sub-system
Test for
Plan
Execution
Integration
for Testing.
Integration
Testing.
Detailed Design
Performed
Testing.
Unit Design review for Test
Unit Testing.
Analysis,
Test Design and
Test
Execution
for Unit Testing.
Code
and Performed
Unit Code Review.
Implementation Testing.
Test
Analysis,
Test Design and
Test
for
Execution
code
and
software
implementation.
The Test Models that previously discussed introduced an aims to overcome
such problems and to improve the status of testing in the software development life
cycle. From this review, all of these three Test Models suggest that testing activities
or process must start at the early stage of software development life cycle, which is
during business requirements and analysis phase.
CHAPTER 4
TESTING METHODOLOGIES
4.1
Introduction
This chapter reviews three testing methodologies or frameworks which are
HeiTech Test Process [44], RUP Test Discipline [3] and Systematic Test and
Evaluation Process (STEP) [4]. In this review, several aspects of these testing
methodologies such as concepts, main activities, advantages and disadvantages are
clearly defined.
Hence, a good testing framework is important as one of software testing main
goal is to detect defects as soon as possible during the software development, looking
for the improvement of the software quality [45].
Basically, a different testing approaches or tactics are needed to be used for
different types of projects, which refer to the environment or methodology in which
the software will be developed. Table 4.1 shows different test tactics for different
type of software development methodology.
51
Table 4.1 : Test Tactics for Different Software Development Methodology
Type
Traditional
Characteristics
•
System
Development
Iterative
Test Tactics
Uses a system development •
Test
methodology
task/step / phase
•
User knows the requirements
•
Development
•
development / •
•
at
end
of
each
Verify that specifications
match needs
determines
structure
•
Test function and structure
Requirements unknown
•
Verify that CASE tools are
used properly
Structure pre-defined
Prototyping /
•
Test functionality
•
Test structure
•
Works best with release
CASE
System
•
Modify structure
Maintenance
methods
•
Requires regression testing
•
Verify that functionality
Purchased /
•
Structure unknown
Contracted
•
May contain defects
matched need
Software
•
Functionality defined in user •
Test functionality
documentation
Test fit into environment
•
Documentation
•
may
vary
from software
4.2
HeiTech Software Testing Process
Software Development Methodology in HeiTech Padu is known as ADVISE
(Application Development Information System) [44]. This methodology is based on
RUP approach. Thus, Figure 4.1 shows HeiTech Application Development Life
Cycles and the objectives for these five phases of application development life cycles
are describes in Table 4.2.
52
Figure 4.1 : HeiTech Application Development Life Cycles
Table 4.2 : HeiTech Application Development Life Cycles Objectives
Phases
Descriptions
Inception
The objective is to achieve concurrence among all stakeholders
on the lifecycle objectives for the project.
Elaboration
The objective is to baseline the architecture of the system to
provide a stable basis for the bulk of the design and
implementation effort in the construction phase.
Construction
The objective is on clarifying the remaining requirements and
completing the development of the system based upon the
baseline architecture.
The objective is to ensure that software is available for its end
Transition
users.
Maintenance
4.2.1
The objective is to support the software after deployment.
Testing Concepts
Testing process and procedures for HeiTech Defined Process [44] include the
testing process flow, ability to test, level of testing, type of testing, tools and test
documentation. Figure 4.2 show the high level of testing process flow for HeiTech
Test Process, which consists of three main flows:
i.
Prepare for Testing
•
Preparing the content of Test Plan document shall be performed in
this process
ii.
Perform Peer Review on documents
53
•
Filing up the Master Test Plan/Test Plan Review Checklist and
Test Report Review
•
iii.
Checklist shall be performed in this process
Perform Testing
•
Perform the testing as planned in the project’s Test Plan and the
result will be recorded in the project’s Test Report document
Figure 4.2 : Testing Process Flow Diagram
4.2.2
Main Activities
The four main activities in HeiTech Test Process [44] are Plan Test, Design
Test, Implement Test and Evaluate Test.
Figure 4.3 below shows four main
activities in HeiTech Test Process.
Figure 4.3 : HeiTech Test Process
54
HeiTech Test Process starts when the specific project begins its development
phases, which normally known as system or software requirement and analysis.
During this project initiation, several documents that being produced are the
prerequisites of this discipline, such as SRS, project plan and schedule and system
baseline for software and documents. Therefore, at the end of the testing process, the
evaluation on test result will affects the project closeout activity.
In each of this stage or activities, different tasks are required to be performed
and several test documents will be produced. The tasks and activities of software
testing should be completed during the Test discipline phase. However, the tasks and
activities actually performed depend on the nature of the project. Therefore, the
activities of software testing should include the following objectives:
i.
To ensure users and operations personnel know how to use and run
the system
ii.
To verify before conversion that the new system contains all required
function that work correctly
iii.
To prove that the developed system satisfies the requirements defined
in the System Requirement Specification (SRS)
iv.
To perform an integrated system test function as specified by the
design parameters
Based on ADVISE methodology, the tasks and activities for test management
shall be performed with reference to the HeiTech Testing Guideline [44]. This
testing guideline contains the proposed testing workflow, testing level and guideline
for the different testing types. For that reason, the details of each task in HeiTech
Test Process activities are describes in Table 4.3 below.
Table 4.3 : Main Activities of HeiTech Test Process
Main
Tasks
Activities
Plan Test
Artifacts/
Test Documents
•
Before start testing, a Test Plan needs to be •
Master/ Test
developed because planning the test is very
Plan
55
Main
Tasks
Artifacts/
Activities
Test Documents
•
important to ensure the success of product •
Test Plan
testing.
Review
Test Plan need to be prepared at early in
Checklist
the life of the project in order to make clear
understanding
between
and
good
developer
agreement
and
relevant
stakeholder.
•
Test Plan contains information about the
purpose of testing within project, which
also identifies the target test items, test
approach, deliverables of testing, test
workflow, environmental needs for testing
and resources needed.
•
Test Plan needs to be reviewed using
Master Test Plan or Test Plan Review
Checklist
by
the
nominated
Quality
Assurance Team.
Design
•
Test
In design test stage, first task is to identify •
and describe Test Cases.
•
Test Cases are a set of inputs, execution
conditions and expected results developed
for particular objective of software testing.
•
Then, testing procedures and criteria are
defined to ensure that the product will
fulfill its intended use when placed in its
intended environment.
•
A test procedure is a set of detailed
instructions for the set-up, execution, and
evaluation of results for a given test case.
•
Upon completion, all the information will
Test Cases
56
Main
Tasks
Artifacts/
Activities
Test Documents
be added in Test Plan.
•
In developing test cases, several steps as
follows are required:
a) Identify types of testing required
b) Types of testing for each level of
testing are identified in Master Test
Plan.
c) List criteria for testing
d) Develop the criteria that need to be
tested from the documentation for
that level.
e) Detailed
objectives
that
can
measure if the system is successful.
f) Expand Criteria into Test Cases
g) Each of the criteria identified to be
expanded into test cases that will
fully verify that each criterion is
met.
h) Uniquely numbered for each test
case and listed the expected result.
i) Rationalize test cases
j) Logical order, where testing should
be done in sequence order to
minimize the time taken.
Implement •
Test
•
The next step is to implement the test.
Testing activities are performed and the
resulting data are collected according to
the established methods, procedures and
criteria.
•
The levels of testing are:
57
Main
Tasks
Artifacts/
Activities
Test Documents
a) Unit Testing
b) Integration Testing
c) System Testing
d) Acceptance Testing
Evaluate
•
Test
The purpose of evaluating test is to •
Defect Log
generate and
Test Report
summary
•
deliver the
overall
test result •
assessment
of
the •
software or system tested.
Review
This is accomplished by reviewing and
Checklist
evaluating the test results in the Defect
Log, identifying and logging change
requests, and calculating the key measures
of test.
•
The test result shall be documented in Test
Report.
•
This Test Report will summarize the
execution of the tests and describe the
result,
including
the
status
of
all
documented defects.
•
Test result summary presents the test
results and defect data shall be recorded in
Defect Log. These data will be further
analyzed to understand the defect density
•
Test Report
Test report review will be performed by
using Test Report Review Checklist. This
review is to evaluate test report and test
result.
58
4.2.3
Testing Levels
Levels of testing for HeiTech Test Process consist of Unit Testing,
Integration Testing, System Testing and Acceptance Testing. Table 4.4 below lists
the detail descriptions of these levels and their testing procedures and documents
base.
Table 4.4 : Detail Descriptions of Testing Levels
Testing
Descriptions
Procedures
Documents
Level
Unit
Base
•
Testing
•
Testing
of
the •
Team
smallest divisible unit
shall perform Unit
of the system.
test.
Usually
at
the
function or program
•
SRS
Each unit test result
will be recorded in
level.
•
Developer
standard
The
more
comprehensive
documentation.
the
unit test plan and unit •
Completion of unit
test cases, the more
testing occurs when
economical
all unit test cases
and
quicker the correction
have
been
of errors.
successfully
conducted and their
results
have
been
reviewed.
Integration •
Testing
•
Also
known
as •
System Designer and
module testing.
the
Modules or a logical
shall
grouping of units that
Integration Test Plan
perform
a
function
of
subthe
•
Analyst
develop
Team
the
Each integration test
result
will
be
SRS
SDD
59
Testing
Descriptions
Procedures
Documents
Level
Base
system
are
recorded in standard
tested
documentation.
together.
•
Useful as an interim
test to catch errors
•
before the systemtesting phase.
•
existing
of
integration
occurs
testing
when
all
integration test cases
Performed on groups
of
Completion
units,
making better use of
team resources.
have
been
successfully
conducted and their
results
have
been
reviewed and signed.
System
•
Testing
Testing of the system •
The
as a whole and the
Support and System
system test needs to
Designer Team are
be
responsible
very
comprehensive
•
•
and
Technical
SDD
to
develop the System
exacting.
Test Plan.
Final level of testing •
As each system test is
performed before the
performed, the results
system delivered to
are recorded by the
client.
tester in the System
System testing will
Test cases.
concentrate on areas
that are more critical
to the success of the
system, especially in
terms
of
functionality.
Acceptance •
Testing
To be develop and •
The acceptance test
BRS
performed
procedure
SRS
by
the
requires
60
Testing
Descriptions
Procedures
Documents
Level
Base
•
users.
two
Based on Business
deliverables,
Requirements
are Acceptance Test
of
which
or
Plan which includes
System Requirements
the Acceptance Test
Specification.
Cases
and
To ensure that the
Acceptance
Test
system
the
Report which include
user’s requirements,
the Acceptance Test
not how well it meets
cases.
Specification
•
types
meets
the specification.
•
When all the testing
has been accepted,
the Project Sponsor
or
Representative
User
and
Project Manager will
signed off on the
Acceptance
Test
Report.
4.2.4
Testing Types
Besides that, HeiTech testing guideline also contains the principles for the
different testing types and its consideration based on levels of testing. Therefore,
Table 4.5 describes testing types that being considered to be tested at each levels of
testing.
61
Table 4.5 : Types of Testing and Its Descriptions
No
Types of
.
Testing
1.
Descriptions
Functionality
•
Most essential form of testing
•
Perform at all levels of testing
•
Tests the adherence of each level of testing to the
specifications from which it comes.
•
Unit functionality testing is develop from SDD and
SRS
•
System functionality testing is developed from SDD.
•
Tests
the
correctness
of
analysis,
design
and
specification process.
2.
Error Handling •
•
Perform at unit testing level.
Ensure that any error could occur during the processing
of unit is covered.
•
Normally for the unit “interfaces” with anything
outside itself (i.e. reading and writing to file)
•
Every possible failure must be accounted, whether
through an explicit procedure or in a “catch-all” error
routine.
3.
Spelling
& •
Grammar
•
Spelling and grammar are in any user interfaces.
To ensure any Help functions are correct and the
abbreviations are meaningful and consistent.
4.
Consistent
•
User Interface
Relates to both layouts of reports and screens, and the
methods of processing.
•
Performs at unit, module, system and acceptance
testing levels.
•
Testing layouts consistency for reports and screens,
which include:
o Consistent placement and format of standard
fields
o Consistent user of abbreviations and acronyms
62
No
Types of
.
Testing
Descriptions
o Consistent use of color, borders, fonts
o Consistent cursor control, screen navigation
•
Testing the consistency methods of processing, which
include:
o Exiting, updating, menu navigation
o Warning message, information message and
Help
5.
All Codes lines •
A program may have many paths that could be used
tested
during executions
•
Test for each condition in program (i.e. IF statements,
LOOPS, CASE statements and others)
6.
On-line help
•
Perform at unit testing level.
•
May not be applicable to all system.
•
Not only work consistently throughout the system, but
also must be relevant to the topic.
7.
Ease of Use
•
Perform at unit testing level.
•
Relates to the ability of the user to perform their job in
a logical manner, and to be able to use the system.
•
Should be thought of and investigated in the design
phase.
•
Test using the work methods defined in BRS.
•
Best performed during the system and acceptance
testing levels.
8.
Boundary
•
Exist where data is to be within a specified range.
Conditions
•
Ensure that the program will correctly process any data
for these conditions, regardless of its values.
•
Test cases need to be designed to test the following
values:
o Zero
o One less than minimum
63
No
Types of
.
Testing
Descriptions
o Minimum
o Midpoint
o Maximum
o One more than maximum
•
9.
Data
Best done at unit testing level
and • The quality of a system’s data and database is integral
Database
to the quality of a system.
Integrity
• Corrupted data could lead to processing errors and
generally hard to fix.
• Relates to two ideas:
o Ensure that only one user can update data at any
one time, which best done at integration and
system testing levels.
o Ensure that if a record is added, modified and
deleted on one file, all references to that record
on other files are suitably updated, which best
done at unit testing level.
10.
Access
and • Ensure that the system’s data is protected from
Security
unauthorized
and
unwarranted
viewing
and
modification.
• Should be installed physically and electronically.
• Program level security can be performed at unit and
module testing levels.
• System wide and all physical security are best
performed at the system testing level.
• All access and security testing are performed at the
acceptance testing level.
11.
Interfaces
– • Unit interfaces
external/unit/m
odule
o Connection between units, and will normally
take the form of parameters
o Test cases need to be designed to ensure that all
64
No
Types of
.
Testing
Descriptions
possible data that can be passed by a unit can be
correctly processed by a receiving unit.
o Best done at integration testing level.
• External interfaces
o External connections with the system, including
user interfaces
o Test cases need to be designed to ensure that a
performing module can correctly handle any
data passed to it and that full range of values for
that data is provided.
o Best done at unit testing level.
• Module interfaces
o Test cases need to be designed to ensure that a
performing module can correctly handle any
data passed to it and that full range of values for
that data is provided.
o Best done at the system testing level.
12.
Performance
• Ensure the system performs to specified levels.
• Performance parameters should include both processing
speed and processing volume.
• Processing speed:
o Measuring the response time for a number of
users doing a typical set of transactions.
o Testing should emulate normal processing and
peak testing is defined somewhere else.
• Processing volume:
o Processing the number of transactions to be
entered during a set time frame.
o Testing for background and batch jobs rather
than user response.
• Best done at the system and acceptance testing levels.
65
No
Types of
.
Testing
13.
Processing
Descriptions
• Ensure the system will correctly perform under peak
Limits
loads (include: no of users, amount of data)
• Also known as stress testing.
• Best done at the system and acceptance testing levels.
Hence, Table 4.6 shows the specific testing category for each of levels testing
that previously discussed.
Table 4.6 : Testing Types Consideration Based on Levels of Testing
Levels of Testing
Unit
Integration
System
Acceptance
Testing
Testing
Testing
Testing
x
x
x
x
Error Handling
x
-
-
-
Spelling & Grammar
x
-
-
-
Consistent User
x
x
x
x
All Codes lines tested
x
-
-
-
On-line help
x
x
-
x
Ease of Use
-
-
-
x
Boundary Conditions
x
-
-
-
Data Integrity
x
x
x
-
Access and Security
x
-
x
x
Interfaces
Levels of–Testing
external/unit/module
x
x
x
-
-
-
x
x
Test Category
Functionality
Interface
Performance
66
Test Category
Processing Limit
Documentations
4.2.5
Unit
Integration
System
Acceptance
Testing
Testing
Testing
Testing
-
-
x
x
-
-
-
x
Roles and Responsibilities
In HeiTech test guideline, the principal roles and responsibilities in testing
process also defined clearly, which explain in Table 4.7.
Table 4.7 : Roles and Responsibilities
Roles
Project Manager
Responsibilities
•
Ensuring that testing activities and processes
are carried out as described in project and
testing plan.
System Analyst
•
Ensuring the testing activities follow the
QA Manager
defined qualification methods stated in the
AD Leader
project’s System Requirements Specification.
•
Producing the project’s Test Plan and Test
Report document.
•
Perform testing.
Users/
•
Verify Test Plan & Test Cases
Subject Matter Experts
•
Perform Testing
•
Fill up the Acceptance Test Result form.
•
Auditing the testing deliverables in the manner
Quality Assurance
Committee Members:
Tech Lead/AD
Lead/Architect
QA Lead
Functional Lead
defined in HeiTech’s Process and Product
Quality Assurance Plan.
•
Verifying and validating the requirements
through testing activities.
67
Roles
Responsibilities
CM Lead
SPEG Representative
Implementation
•
Produce Test Cases.
Analyst/Business
•
Perform testing.
•
Produce Test cases.
•
Perform testing.
•
Fill up the Unit Test Results form.
•
Verifying the requirements through unit testing
Analyst/System Analyst
Application Developer
activity.
4.2.6
Test Documentations / Artifacts
In addition, once the system is implemented and the development team has
left, the main source of reference for users is the supplied documentation. This
documentation is also used as a basis for system maintenance and enhancements.
Therefore, the system documentation needs to be comprehensive, relevant and
addressed at the appropriate level.
Acceptance testing should be performed in
conjunction with the documentation to ensure that it meets the users' needs.
Basically there are five documents related to testing process and activities in a
project as shown in Table 4.8 below.
Table 4.8 : HeiTech Test Documents
Test Documents
Descriptions
Testing Plan
The Testing Plan document is made generic and applicable
Guideline
for all software/application development projects in
HeiTech. This document is to be referred by each person
of all application development-related projects specifically
in the testing phase and generally in all other phases
throughout the project lifecycle.
68
Test Documents
Descriptions
Test Plan
The Test Plan document must be prepared for planning the
testing purposes.
Planning includes test schedule, modules to be tested, and
specific test description.
The tester shall fill up the Test cases during the testing
Test Cases
activities. It records the list of test cases, test date and
many more
The Test Report document must be prepared for recording
Test Report
of results from the testing activities performed. Items to be
reported include recommended improvement, attachment
of test results form or log, and impact of test environment
4.2.7
Advantages and Disadvantages
Based on the study that conducted in HeiTech, the summary of advantages
and disadvantages for HeiTech Test Process is listed in Table 4.9 below.
Table 4.9: Advantages and Disadvantages of HeiTech Test Process
No.
1.
Advantages
Established
HeiTech
Disadvantages
Defined HeiTech practice this defined process
Process for Testing Process.
for testing activities only on several
large project.
Whereas, for small
project, testing process is rarely follow
the defined process.
2.
HeiTech Test Process clearly Main activities of testing in HeiTech are
defined the four main activities of too general and time estimation for
software testing.
testing process on several projects only
for a few weeks or days.
3.
Practice
standard
documentations
that
test HeiTech
test
documents
also
not
follow comprehensive because no automated
69
No.
Advantages
Disadvantages
HeiTech Defined Guideline.
test case or scripts that provided as
guideline.
4.
HeiTech Test Process clearly Mostly, all types of testing are clearly
defined levels of testing and types defined and planned in Test Plan.
of testing in Test Plan.
However, due to some limitation of
resources, a few testing types are not
performed during the testing activities.
5.
All
tasks
and
activities
of Basically,
software testing are required to development
most
projects
of
start
HeiTech
testing
complete during Test Discipline process or activities when the products
phase
based
on
ADVISE nearly finish the implementation stage.
methodology.
4.3
RUP Test Discipline
The Rational Unified Process is a Software Engineering Process, which
provides a disciplined approach to assigning tasks and responsibilities within a
development organization [46]. In addition, the Rational Unified Process is also a
modern software development process that uses elements of all the process bases
mentioned in the methodology section, as well as a series of best practices to obtain a
quality product.
Moreover, The Rational Unified Process is a software process product,
originally developed by Rational Software, and now available from IBM.
The
product includes a hyperlinked knowledge base with sample artifacts and detailed
descriptions for many different types of activities. Besides that, RUP is included in
the IBM Rational Method Composer (RMC) product [2] which allows customization
of the process. The evolution of the RUP process can be seen over the course of four
phases, which shown in Figure 4.4.
70
Figure 4.4 : Illustration of the Rational Unified Process (RUP)
The Rational Unified Process can be described in two dimensions, or along
two axes, which are:
i.
The horizontal axis represents time and shows the dynamic aspect of the
process as it is enacted, and it is expressed in terms of cycles, phases,
iterations, and milestones.
ii.
The vertical axis represents the static aspect of the process, which how it
is described in terms of activities, artifacts, workers and workflows.
In addition, RUP also captures many of the best practices in modern software
development in a form that is suitable for a wide range of projects and organizations.
Hence, this process is can be used as the basis for all projects of software or
application development. Table 4.10 below listed the objectives for the following
series of RUP iterations.
Table 4.10: Objectives of RUP Iteration Phases
Phases
Objectives
Conception
The objective in this phase is to establish the business
requirements to be covered by the system, identifying all
entities that will interact with the system, such as people, other
systems, hardware, and others, and also assessing the capability
71
Phases
Objectives
of the project.
Preparation
The objective in this phase is to understand the problem from
the standpoint of the development team.
This involves
preparing the framework architecture for the system and
designing the technical solution, as well as drawing up the
project plan and identifying key risks. The output obtained
from this phase consists of the defined architecture, the model
for systems requirements based on use cases specified in UML
(Unified Modeling Language), the development plan, the
quality standards for the project and the tools to be employed in
the course of the work.
Construction
This phase concentrates on the design of components and the
addition of custom software functionalities through iteration as
construction and testing go further, while allowing room for the
inclusion of changes. Deliveries can be planned at the end each
iteration, when the end user is asked to provide feedback and
changes are proposed. After an analysis of the impact of the
proposed changes, a decision is taken as to the best moment to
include them in the system. The end result is a fully operational
system with user documentation.
Transition
The last phase of RUP is concerned with shifting the software
from the development to production environments in which the
end user will run the system. Hence, depending on the type of
project, it may be necessary to use intermediate environments
such as pre-production, user acceptance, and others for rigorous
validation before moving on to the production phase.
Hence, each phase ends with various iterations involving as series of
activities. The RUP model classifies these into nine disciplines, the importance of
which changes as the project progresses towards its end. Table 4.11 describes the
nine disciplines or workflows in RUP.
72
Table 4.11 : Nine Disciplines of Rational Unified Process
Disciplines
Descriptions
Business
This set of activities seeks understanding of business needs.
modeling
General and high level requirements documentation, business
rules, glossaries and others are used to help define the software
product to be produced.
Requirements
These activities translate the needs established in the business
model to automated systems requirements of a more technical
nature.
The aim is for the development team members to
achieve an in-depth understanding of the business model.
Analysis and
Based on requirements, these activities establish the most
design
appropriate system architecture and the detailed design
necessary for implementation.
Implementation These activities consist of coding the software based on the
design and the system requirements.
Test
Verification of all of the elements produced (documents,
designs and code) to ensure compliance with the requirements
and quality standards defined for the project.
Deployment
These activities involve the installation of the system in the
environments in which it will finally operate.
Project
These activities are aimed to manage the development in terms
management
of plans, resources, monitoring and risk control and
management.
Configuration
and Change
Management of changes and other matters arising in the
construction process.
management
Environment
These activities seek to provide the project with the hardware
and software resources required to facilitate start-up and
maintenance
in
the
different
development
and
test
environments, as well as the production launch of the system.
73
4.3.1
Test Concepts
One of disciplines in Rational Unified Process model is Test. The RUP
proposes an iterative approach, which means that tester needs to perform test
throughout the project. Hence, this helps to find defects as early as possible, which
radically reduces the cost of fixing the defect. Moreover, tests are also carried out
along three quality dimensions, which are reliability, functionality, application and
system performance.
Therefore, for each of these quality dimensions, the process describes how to
go through the test lifecycle of planning, design, implementation, execution and
evaluation. For this reason, the purposes of testing based on RUP approach are:
•
To verify the interaction between objects.
•
To verify the proper integration of all components of the software.
•
To verify that all requirements have been correctly implemented.
•
To identify and ensure defects are addressed prior to the deployment of
the software.
In addition, RUP Test discipline acts as a service provider to the other
disciplines in many respects. Testing focuses primarily on evaluating or assessing
the product quality, which is realized through these core practices:
•
Find and document defects in software quality.
•
Advise on the perceived software quality.
•
Validate and prove the assumptions made in design and requirement
specifications through concrete demonstration.
•
Validate that the software product works as designed.
•
Validate that the requirements are implemented appropriately.
An interesting difference exists between Test and the other disciplines in
RUP is essentially Test is tasked with finding and exposing weaknesses in the
software product. Thus, tester needs to identify a different general philosophy than
what is used in the Requirements, Analysis and Design, and Implementation
74
disciplines, which is a subtle difference is that those three disciplines focus on
completeness, whereas Test focuses on incompleteness [2]. Furthermore, the Test
discipline is related to other disciplines as follows:
i.
The Requirements discipline captures requirements for the software
product, which is one of the primary inputs for identifying what tests to
perform.
ii.
The Analysis and Design discipline determines the appropriate design for
the software product, which is another important input for identifying
what tests to perform.
iii.
The Implementation discipline produces builds of the software product
that are validated by the Test discipline and within an iteration, multiple
builds will be tested, which typically one per test cycle.
iv.
The Deployment discipline delivers the completed software product to the
end user. Thus, while the software is validated by the Test discipline
before this occurs, beta testing and acceptance testing are often conducted
as part of Deployment.
v.
The Environment discipline develops and maintains supporting artifacts
that are used during Test, such as the Test Guidelines and Test
Environment.
vi.
The Project Management discipline plans the project and the necessary
work in each of iteration. Hence, the tasks are described in an Iteration
Plan, this artifact is an important input used when you define the correct
evaluation mission for the test effort.
vii.
The Configuration and Change Management discipline controls change
within the project team. The test effort verifies that each change has been
completed appropriately.
75
4.3.2
Main Activities
Diagram in Figure 4.5 represents the default workflow for the Test discipline
that may require variations based on the specific needs of each iteration and project.
Figure 4.5 : RUP Test Discipline Workflow
Software is refined through iterations in the RUP software development
lifecycle.
The testing lifecycle benefits from following an equivalent iterative
approach in this process environment.
Thus, in each of iteration, the software
development team produces one or more builds, with each build being a potential
candidate for testing.
Moreover, the focus and objectives of the development team differ from
iteration to iteration. Therefore, the test team members must structure their test effort
accordingly.
76
The detail breakdowns for the Test discipline workflows are explain as
follows:
i.
Workflow Detail: Define Evaluation Mission
Figure 4.6 : Define Evaluation Mission Workflow
The purpose of this workflow detail is to identify the appropriate focus of
the test effort for the iteration, and to gain agreement with stakeholders on
the corresponding goals that will direct the test effort as refer to Figure
4.6. Therefore, for each of iteration, this work is focused mainly on:
•
Identifying the objectives for, and deliverables of, the testing
effort
•
Identifying a good resource utilization strategy
•
Defining the appropriate scope and boundary for the test effort
•
Outlining the approach that will be used
•
Defining how progress will be monitored and assessed
77
ii.
Workflow Detail: Verify Test Approach
Figure 4.7 : Verify Test Approach Workflow
The purpose of this workflow detail is to demonstrate that the various
techniques outlined in the Test Approach will facilitate the planned test
effort as shown in Figure 4.7. The intent is to verify by demonstration
that the approach will work, produces accurate results and is appropriate
for the available resources.
The objective is to gain an understanding of the constraints and
limitations of each technique as it will be applied in the given project
context, and to either to find an appropriate implementation solution for
each technique or find alternative techniques that can be used. This helps
to mitigate the risk of discovering too late in the project lifecycle that the
test approach is unworkable.
78
For each of iteration, this work is focused mainly on:
•
Early verification that the intended test strategy will work and
produces results of value
•
Establishing the basic infrastructure to enable and support the test
strategy
•
Obtaining commitment from the development team to develop the
software to meet testability requirements necessary to achieve the
test strategy, and to provide continued support for those testability
requirements.
•
Identifying the scope, boundaries, limitations and constraints of
each technique
iii.
Workflow Detail: Validate Build Stability
Figure 4.8 : Validate Build Stability Workflow
79
The purpose of this workflow detail is to validate that the build is stable
enough for detailed test and evaluation effort to begin as shown in Figure
4.8. This work is also referred to as a smoke test, build verification test,
build regression test, sanity check or acceptance into testing. In addition,
this work helps to prevent the test resources being wasted on a pointless
and ineffective of testing effort.
For each Build to be tested, this work is focused on:
•
Making as assessment of the stability and testability of the Build
•
Gaining an initial understanding or confirming the expectation of
the development work delivered in the Build
•
Making a decision to accept the Build as suitable for use as guided
by the evaluation mission in further testing, or to conduct further
testing against the previous Build.
iv.
Workflow Detail: Test and Evaluate
Figure 4.9 : Test and Evaluate Workflow
80
The purpose of this workflow detail is to achieve appropriate breadth and
depth of the test effort to enable a sufficient evaluation of the items being
targeted by the tests, where sufficient evaluation is governed by the
current test motivators and evaluation mission as shown in Figure 4.9.
Typically performed once per test cycle, this work involves performing
the core tactical work of the test and evaluation effort, which namely as
implementation, execution and evaluation of specific tests and the
corresponding reporting of incidents that are encountered.
For each test cycle, this work is focused mainly on:
•
Providing ongoing evaluation and assessment of the Target Test
Items
•
Recording the appropriate information necessary to diagnose and
resolve any identified Issues
•
Achieving suitable breadth and depth in the test and evaluation
work
•
Providing feedback on the most likely areas of potential quality
risk
v.
Workflow Detail: Achieve Acceptable Mission
The purpose of this workflow detail which shows in Figure 4.10 is to
deliver a useful evaluation result to the stakeholders of the test effort,
where useful evaluation result is assessed in terms of the Evaluation
Mission. In most cases that will mean focusing testing team efforts on
helping the project team achieve the Iteration Plan objectives that apply to
the current test cycle.
81
Figure 4.10 : Achieve Acceptable Mission Workflow
For each test cycle, this work is focused mainly on:
•
Actively prioritizing the minimal set of necessary tests that must be
conducted to achieve the Evaluation Mission
•
Advocating the resolution of important issues that have a
significant negative impact on the Evaluation Mission
•
Advocating appropriate quality
•
Identifying regressions in quality introduced between test cycles
•
Where appropriate, revising the Evaluation Mission in light of the
evaluation findings so as to provide useful evaluation information
to the project team
vi.
Workflow Detail: Improve Test Assets
The purpose of this workflow detail is to maintain and improve the test
assets as shown in Figure 4.11. This workflow is important especially if
the intention is to reuse the assets developed in the current test cycle in
subsequent test cycles.
82
Figure 4.11 : Improve Test Assets Workflow
For each test cycle, this work is focused mainly on:
•
Adding the minimal set of additional tests to validate the stability
of subsequent Builds
•
Removing test assets that no longer serve a useful purpose or have
become uneconomic to maintain
•
Conducting general maintenance of and making improvements to
the maintainability of test automation assets
•
Assembling Test Scripts into additional appropriate Test Suites
•
Exploring opportunities for reuse and productivity improvements
•
Maintaining Test Environment Configurations and Test Data sets
•
Documenting lessons learned for both good and bad practices
discovered during the test cycle
83
Additionally, a test cycle is a period of independent test activity that includes,
among other things, the execution and evaluation of tests. Thus, each of iteration can
contain multiple test cycles; with the majority of iterations contain at least one.
Besides that, each test cycle starts with the assessment of a software build's stability
before it's accepted by the test team for more thorough and detailed testing.
Furthermore, the RUP recommends that each build be regarded as potentially
requiring a new cycle of testing, but there is no strong coupling between build and
test cycle. Typically, Inception iterations of new projects did not produce builds, but
iterations in all other phases do. Although each build is a potential candidate for a
cycle of testing, there are various reasons why no need to test for every software
build.
For example, in situations where a build is created daily, a useful test cycle
may take longer than the available time between builds. Thus, in this case it may be
appropriate to test every nth of software build.
4.3.3
Roles and Responsibilities
The RUP defines not just one role specifically for the Test discipline, but
four, as shown in Figure 4.12, which the activities conducted as part of the test and
evaluation work grouped by responsible roles. Thus, the four roles focusing on test
related activities in the RUP test discipline are:
i.
Test Manager
ii.
Test Analyst
iii.
Test Designer
iv.
Tester
84
Figure 4.12 : Roles and Test Activities Workflows
4.3.4
Test Documentations / Artifacts
Figure 4.13 : Roles and Test Artifacts
85
Figure 4.13 show the artifacts developed as products of the test and
evaluation activities are clearly grouped by responsible role. Meanwhile, Table 4.12
summarized the roles and responsibilities for activities and artifacts have clearly
illustrated above.
Table 4.12: Roles and Test Artifacts in RUP Test Discipline
Roles
Test Artifacts
Test Manager
Test Analyst
Test Designer
Tester
•
Test Evaluation Summary
•
Test Plan or Master Test Plan
•
Test Ideas, and Test-Idea List
•
Test Cases
•
Workload Analysis Model
•
Test Data
•
Test Result
•
Test Strategy
•
Test Automation Architecture
•
Test Interface Specification
•
Test Environment Configuration
•
Test Suites
•
Test Script
•
Test Log
In performing testing activities, the decision of what artifacts to be use and
how to effectively make use of them is important in order to perceived successful of
software testing. Table 4.13 describes the purpose and some recommendation to
make use of those artifacts that consider using in particular contexts.
86
Table 4.13 : Purpose of Test Artifacts
Artifact
Purpose
Tailoring
(Optional, Recommended)
Test
Summarizes
the
Test •
Recommended
for
most
Evaluation
Results for use primarily by
projects.
Summary
the management team and •
Where the project culture is
other stakeholders external
relatively information, it may
to the test team.
be appropriate simply to record
test results and not create
formal evaluation summaries.
•
In other cases, Test Evaluation
Summaries can be included as a
section within other Assessment
artifacts, such as the Iteration
Assessment or Review Record.
Test Results
This artifact is the analyzed •
Recommended.
result determined from the •
Most test teams retain some
raw data in one or more Test
form of reasonably detailed
Logs.
record of the results of testing.
•
Manual
testing
results
are
usually recorded directly here,
and combined with the distilled
Test Logs from automated tests.
•
In some cases, test teams will
go directly from the Test Logs
to
producing
the
Test
Evaluation Summary.
Master Test
Defines high-level testing •
Optional.
Plan
goals, objectives, approach, •
Useful for most projects.
resources,
A Master Test Plan is defines
schedule
and •
deliverables that govern a
the high level strategy for the
phase or the entire the
test effort over large parts of the
lifecycle.
software development lifecycle.
87
Artifact
Purpose
Tailoring
(Optional, Recommended)
•
Optionally, the Test Plan can be
included as a section within the
Software Development Plan.
•
Consider whether to maintain a
"Master" Test Plan in addition
to the "Iteration" Test Plans.
•
The Master Test Plan covers
mainly logistic and process
enactment
information
that
typically relates to the entire
project lifecycle, therefore it is
unlikely to change between
iterations.
Iteration
Defines finer grained testing •
Recommended
Test Plan
goals,
projects.
objectives,
motivations,
resources,
approach, •
schedule
deliverables
that
A
separate
for
Test
Plan
most
per
and
iteration is recommended to
govern
define the specific, fine-grained
iteration.
test strategy.
•
Optionally, the Test Plan can be
included as a section within the
Iteration Plan.
Test Ideas
List
This is an enumerated list of •
Recommended
ideas,
projects.
often
partially
for
most
formed, to be considered as •
In some cases these lists will be
useful tests to conduct.
informally
defined
and
discarded once Test Scripts or
Test Cases have been defined
from them.
Test Script,
Test Data
The Test Scripts and Test •
Recommended
Data are the realization or
projects.
for
most
88
Artifact
Purpose
Tailoring
(Optional, Recommended)
implementation of the test, •
Where projects differ is how
where
formally
the
embodies
Test
the
Script
procedural
these
artifacts
are
these
are
treated.
aspects, and the Test Data •
In
the defining characteristics.
informal and transitory, and the
some
cases,
test team is judged based on
other criteria.
•
In other cases, especially with
automated tests, the Test Scripts
and associated Test Data are
regarded as major deliverables
of the test effort.
Test Suite
Used to group individual •
Recommended
related tests (Test Scripts)
projects.
together
Also required to define any Test
in
meaningful •
subsets.
for
most
Script execution sequences that
are required for tests to work
correctly.
Test Case
Defines a specific set of test •
Recommend on most projects,
inputs, execution conditions,
were the conditions required to
and expected results.
conduct a specific test are
complex or extensive and where
Documenting
test
cases
they are a contractually required
allows them to be reviewed
for
completeness
and
correctness, and considered
before
effort
deliverable.
•
recommended to maintain the
implementation
is
planned
In most other cases, it is
Test-Ideas
&
List
Implemented
expended.
and
Test
the
Scripts
instead of detailed textual Test
Cases.
This is most useful where
•
Some
projects
will
simply
89
Artifact
Purpose
Tailoring
(Optional, Recommended)
the
input,
conditions
results
execution
outline Test Cases at a high
expected
level and defer details to the
and
are
particularly
complex.
Test Scripts.
•
Another style commonly used is
to document the Test Case
information
as
comments
within the Test Scripts.
A specialized type of Test •
Recommended
Analysis
Case. Used to define a
systems, especially those where
Model
representative workload to
system performance under load
allow
must be evaluated, or where
Workload
quality
risks
for
most
associated with the system
there
operating under load to be
quality risks associated with
assessed.
system operation under load.
•
Not
are
other
usually
significant
required
for
systems that will be deployed
on a standalone target system.
Test Classes
The
Design
Model
in the Design Implementation
and •
Model •
Where Required.
Stubs are a common category of
Model Test
include Test Classes and
Test
Components
Components if the project
Component.
in the
has to develop significant
Implementat additional
ion Model
Classes
and
Test
specialized
behavior to accommodate
and support testing.
Test Log
The raw data output during •
Optional.
test
Many projects that perform
execution,
produced
tests.
by
typically •
automated
automated testing will have
some form of Test Log, where
projects differ is whether the
Test Logs are retained or
90
Artifact
Purpose
Tailoring
(Optional, Recommended)
discarded after Test Results
have been determined.
•
Retain Test Logs if need to
satisfy
certain
requirements,
audit
if
want
to
perform analysis on how the
raw test output data changes
over time, or if uncertain at the
outset of all the analysis that
may be required to give.
Test
Provides
Automation
overview
an
architectural •
of
the
test •
Architecture automation system, using a
number
of
Optional.
Recommended
on
projects
where the test architecture is
different
relatively complex, when a
architectural views to depict
large number of staff will be
different
collaborating
aspects
of
the
system.
on
building
automated tests, or when the
test
automation
system
is
expected to be maintained over
a long period of time.
•
In some cases this might simply
be a white-board diagram that is
recorded centrally for interested
parties to consult.
Test
Defines a required set of •
Optional.
Interface
behaviors by a classifier •
On many projects, there is
Specification (specifically,
a
Class,
either sufficient accessibility for
Subsystem or Component)
test in the visible operations on
for the purposes of testing
classes, user interfaces and
(testability). Common types
others.
include test access, stubbed •
Some common reasons to create
91
Artifact
Purpose
Tailoring
(Optional, Recommended)
behavior,
and
diagnostic
logging and test oracles.
Test
Interface
Specifications
include UI extensions to allow
GUI test tools to interact with
the tool and diagnostic message
logging routines, especially for
batch processes.
4.3.5
Advantages and Disadvantages
The RUP testing activities are concerning continuously providing the
management and the development teams with an objective assessment of the quality
of the product. The definition of the level of quality is an engineering trade-off, an
optimization of highly dependent on the business context in which the software
product will be deployed. The expected level of quality and the requirements may
evolve during the lifecycle, and testing also should evolve with this process.
Testing is not a complete, separate workflow, parallel to or appended to
software development, but fully integrated in the iterative cycle of the RUP. Hence,
taking advantage of iterations, testing becomes an almost continuous process that
continually feeds back into the requirements, the design, and the management of the
project. It is a collaborative activity and also more effectively done by automated
tools.
In addition, testing can start early, since iterative development produces
testable code early, and can therefore provide a crucial feedback, both on the product
and the process. Thus, the role of the tester or the quality engineer in the RUP is not
primarily to find defects, but to provide other team members such as developers and
managers with an objective assessment of the quality of the product.
92
Table 4.14 below listed several advantages and disadvantages of RUP Test
Discipline [2, 3, 46, 50].
No.
1.
Table 4.14: Advantages and Disadvantages of RUP Test Discipline
Advantages
Disadvantages
RUP provides an iterative development Even RUP is an iterative process;
process.
Thus, testing does not start it still not considered particularly
with just test plans.
The
tests adopting agile approach.
themselves are developed early and
conducted early, and a useful subset of
them is accumulated in regression suites
and iteration after iteration.
2.
RUP
provides
low
up-front RUP consists of a large collection
documentation, where the Detailed Test of documents, role descriptions,
planning
is
defined
iteration
by activities
and
others.
This
iteration, based on a governing master stresses the need for tailoring to a
Test plan, to meet the needs of the team specific organization, which in
and
match
the
objectives
iteration.
of
the most projects equals downsizing
of the methodology. RUP is
considered
and
criticized
for
being a heavy-weight approach.
3.
RUP promotes a holistic approach in In technical perspective, RUP is a
order to identify appropriate tests is not large
collection of processes,
strictly and solely based on deriving artifacts and roles. These sources
tests from requirements. Tests in the must be scaled down for most
RUP are derived from the requirements projects except for the very
and from other sources.
4.
largest ones.
RUP also promotes automation testing. RUP is a commercial product.
As testing starts early in, and continues Hence, an effort must be spent on
throughout, the RUP lifecycle, it is tailoring
RUP,
where
93
impractical to work on many aspects of organizations need to revise their
testing without appropriate tool support. cost and budget.
Automated tools can help generate test
data conditions, run tests, and analyze
results.
5.
The RUP assists verifying product RUP fails to provide any clear
quality
in
the
implementation,
planning,
design, implementation
guidelines,
in
and order to access quality at early
execution,
evaluation stages. Quality assessment is development stage.
built into the process, in all activities,
involving
all
participants,
using
objective measurements and criteria,
and not treated as an afterthought, or a
separate
activity
performed
by
a
separate group.
Quality should be
reviewed
respect
with
requirements
based
on
to
the
reliability,
functionality, application performance
and system performance.
6.
The ability to manage change making RUP leaves the tailoring of its’
certain that each change is acceptable, process to the user entirely.
and being able to track changes is
essential in an environment in which
change is inevitable. The RUP process
describes how to control, track and
monitor changes to enable successful
iterative development.
4.4
Systematic Test and Evaluation Process (STEP)
94
4.4.1
Testing Concepts
The Systematic Test and Evaluation Process (STEP) were first introduced in
1985 as part of the course material for the Systematic Software Testing seminar
series [27]. STEP provides a model process and a step-by-step sequence of activities
and tasks for performing software testing at any level from unit testing through
acceptance testing and is a proprietary methodology developed by Software Quality
Engineering [4]. The methodology grew from and is built upon the foundation
provided by the American National Standards Institute (ANSI) testing standard.
STEP was unique in that it broadened the definition of test to include more
than a procedure performed after the system was fully assembled. STEP expanded
that original definition to include test planning, test case design, test implementation,
and test execution [53]. Therefore, large organizations that have a test function have
found that this approach provides significant benefits in quality and control,
meanwhile smaller organizations that did not have a separate test function have also
seen remarkable improvements in their testing process.
Furthermore, STEP is built upon the foundation of the IEEE Std. 829-1983
Standard for Software Test Documentation and subsequently updated based on the
latest version (IEEE Std. 828-1998) of this standard and the IEEE Std. 1008-1987
Standard for Software Unit Testing [27]. Hence, while retaining compatibility with
these standards, this methodology has grown in scope and now stands as one of the
leading models for effective software testing throughout the industry.
STEP covers the broad activity of software evaluation. Evaluation is defined
as that sub-discipline of software engineering concerned with determining whether
software products do what they are supposed to do. Thus, the major techniques
employed in evaluation are analysis, review and test.
Hence, STEP’s scope and objectives are focuses on testing as the most
complex of the three, but stresses overall coordination and planning of all aspects of
evaluation as a key to success. Moreover, it stresses the prevention potential of
testing, with defect detection and demonstration of capability as secondary goals.
95
Figure 4.14 : Different Views of Testing
In addition, there is an evolution approach in views of testing as shown in
Figure 4.14 above. Thus, early views saw testing as a phase that occurred after
software development, or something that programmers need to get the bugs out of
their programs. However, for more modern view sees testing as a process to be
performed in parallel with the software development or maintenance effort, which
incorporating the several activities such as:
4.4.2
•
planning to determine risks and selecting strategies
•
analysis to set up test objectives and requirements
•
design to specify tests to be developed
•
implementation to construct or acquire the test procedures and cases
•
execution to run and re-run the tests
•
maintenance to maintain and updating the tests as the software changes
Main Activities
This STEP methodology breaks down the testing process into three major
phases, which are Planning, Acquisition, and Measurement as shown in Figure 4.15.
96
Figure 4.15 : Major Phases in STEP
In planning phase, information about the software to be tested and the project
is used to develop testing objectives and an overall testing approach. The output of
the phase is a test plan that serves to guide the remainder of the testing activity and
coordinate the test levels.
Next, in Acquisition phase, more information about the software
requirements and design, along with previous testing product documentation and
data, is used to specify and develop a test configuration for each level of testing
performed. The output of this phase is the test set and its documentation.
Finally, in Measurement phase, the test set is executed. The input to this
phase is the software to be tested, and the output is test reports documenting the
execution and measurement activity along with records of any failures or incidents
observed.
97
Table 4.15 below shows the major testing activities in these three testing
phases of STEP. Then, for each of the eight activities is further subdivided into
detailed steps or tasks to be performed to complete the activity.
Table 4.15 : Major Testing Activities in STEP
Phases
Planning
Testing Activities
1. Plan the general approach
2. Determine testing objectives
3. Refine the general plan
Acquisition
4. Design the tests
5. Implement the tests
Measurement
6. Execute the tests
7. Check termination
8. Evaluate results
4.4.3
STEP Architecture
STEP assumes that the total testing job is divided into levels during planning.
A level represents a particular testing environment, such as unit testing usually refers
to the level associated with program testing in a programmer's personal development
library. Figure 4.16 illustrate the project testing architecture in STEP.
For simple projects, such as minor enhancements, the activity timing may
consist of just one or two levels of testing, such as unit testing and acceptance
testing. However, for complex projects, such as a new product development, may
have more levels that consist of unit test, function test, subsystem test, system test,
acceptance test, alpha and beta test.
98
Figure 4.16: Project Testing Architecture in STEP
4.4.4
Timing of STEP Activities
STEP specifies when the testing activities and tasks are to be performed, as
well as what the tasks should be and their sequence, as shown in Figure 4.17 below.
The timing emphasis is based on getting most of the test design work completed
before the detailed design of the software.
Figure 4.17: Activity Timing at Various Levels of Test
99
The trigger for beginning the test design work is an external, functional, or
black box specification of the software component to be tested. For higher test
levels, such as acceptance or system testing, the external specification is equivalent
to the system requirements document. Hence, as soon as that document is available,
test activity can begin on the design of the requirements-based tests.
The test design process continues as the software is being designed and an
additional tests based on the detailed design of the software are identified and added
to the requirements-based tests.
As the software design process proceeds, the
detailed design documents are produced for the various software components and
modules comprising the system.
These documents serve as functional specifications for the component or
module, and thus may be used to trigger the development of requirements-based tests
at the component or module level. As the software project moves to the coding
stage, a third increment of tests is designed based on the code and implementation
details.
The test inventory and design activities at the various levels are overlap. The
goal at each level is to complete the bulk of the test design work as soon as possible.
This helps to ensure that the requirements are testable and well thought out and that
defects are discovered early in the process. This strategy supports an effective
software review and inspection approach.
Figure 4.18 shows the details of activity timing within a given test level. The
test activity starts with developing plans and identifying specific test objectives.
Then, the task continues with test design and the implementation of test plans and
designs. Next, the activity proceeds with test execution and evaluation. Overlap of
activities is possible.
100
Figure 4.18: Details Activity Timing at Various Levels of Test
4.4.5
Elements of STEP
The STEP methodology consists of specified tasks, which is individual
actions, work products, which is documentation and implemented tests and roles to
define responsibilities associated with groups of tasks, as shown in Figure 4.19.
These elements had being packaged into a system with proven effectiveness for
consistently achieving quality software.
Figure 4.19 : Elements of STEP
101
In addition, another aspect of the STEP process model is the set of work
products produced in each phase and activity. STEP uses the word "testware" to
refer to the major testing products such as test plans and test specification documents
and the implemented test procedures, test cases, and test data files.
The word "testware" is intentionally analogous to software and, as shown in
Figure 4.20 below, is intended to reflect a parallel development process. Hence, as
the software is designed, specified, and built, the testware is also designed, specified,
and built.
Figure 4.20 : Parallel, Mutually Supportive Development
Furthermore, these two broad classes of work products support each other.
Testware development, by relying on software work products, supports the
prevention and detection of software faults. In other hand, software development, by
reviewing testware work products, supports the prevention and detection of testware
faults.
102
4.4.6
Test Documentations / Artifacts
STEP uses IEEE standard document templates as a recommended guideline
for document structure and content. Table 4.16 below lists the test documents for
STEP methodology.
Table 4.16 : Template for Test Documents from IEEE Std. 829-1998
Test Documents
Descriptions
Test Plan
Used for the master test plan and level-specific test
plans.
Test Design Specification
Used at each test level to specify the test set
architecture and coverage traces.
Test Case Specification
Used as needed to describe test cases or automated
scripts.
Test Procedure
Specification
Test Log
Used to specify the steps for executing a set of test
cases.
Used as needed to record the execution of test
procedures.
Test Incident Report
Used to describe anomalies that occur during
testing or in production. These anomalies may be in
the requirements, design, code, documentation, or
the test cases themselves. Incidents may later be
classified as defects or enhancements.
Test Summary Report
Used to report completion of testing at a level or a
major test objective within a level.
4.4.7
Roles and Responsibilities
Additionally, STEP also defined roles and responsibilities for various testing
activities.
Thus, the four major roles defined in STEP are Manager, Analyst,
Technician, and Reviewer, which listed in Table 4.17.
103
Table 4.17 : Roles and Responsibilities in STEP
Role
Manager
Description of Responsibilities
Communicate, plan, and coordinate.
Test manager is responsible for providing overall test
direction
and
coordination,
and
communicating
key
information to all interested parties.
Analyst
Plan, inventory, design, and evaluate.
Test analyst is responsible for detailed planning, inventorying
of test objectives and coverage areas, test designs and
specifications, and test review and evaluation.
Technician
Implement, execute, and check.
Test technician is responsible for implementation of test
procedures and test sets according to the designs provided by
the analyst, for test execution and checking of results for
termination criteria, and for test logging and problem
reporting.
Reviewer
Examine and evaluate.
Test reviewer provides review and oversight over all steps and
work products in the process.
4.4.8
Advantages and Disadvantages
STEP is an example of one of the testing methodologies now in place in a
number of organizations. Testing methodologies have matured and achieved proven
success. It is no longer the case that testing or development team have to practice the
art of testing and approach the activity from an ad hoc, disorganized, and
uncontrolled perspective. Thus, testing can be systematic, and appropriate models
such as STEP are available.
disadvantages of STEP.
Table 4.18 summarized the advantages and
104
Table 4.18: Advantages and Disadvantages of STEP
No.
1.
Advantages
Disadvantages
The overall goal of the testing activity
Organization
requires
extra
for STEP, which is prevention oriented, efforts on testing activity because
with a primary focus on finding activity timing in STEP permits
requirements
and
defects overlaps from one level to another
design
through early development of test level of tests.
designs. Thus, major testing activities
are begun early, in example planning
timing and activity timing begins during
requirements.
2.
STEP
using
documents
the
to
IEEE
standard If there is any amendment or new
provides
full changes
of
the
IEEE
documentation, in order to define the documentation standard, STEP
visibility of testing activities.
test documentation also need to
adopt the changes.
3.
This STEP methodology is not a tool Testing can be less effective if the
dependent and does not assume any requirements information of the
particular test organization or staffing, product is ambiguous and not
such as independent test groups. It does clearly defined.
assume a development effort, where the
requirements
information
product
the
and
for
technical
the
design
information are comprehensible and
available for use as inputs to testing.
4.
STEP takes the requirements presented NA.
(requirements-based
inventory)
and
constructs test cases to validate each
requirement. The creation of a group of
test cases with known coverage in
STEP, are mapping to the inventories of
requirements, design, and code of
software products.
105
No.
Advantages
Disadvantages
STEP sets three major objectives, which NA.
5.
are
preventing
defects,
and
defects,
detecting
demonstrating
the
capability of the software. In order to
meet these goals, the test activity is
broken up into levels of testing, which
allow for focused attention toward
specific system characteristics as they
are delivered.
6.
One of the strengths of STEP is its NA.
adaptability to change. This allows
much of the work that is constructed
initially to be
reused during the
remaining life of the system.
4.5
Summary
A summary of the comparison study on these three different testing
methodologies is listed in Table 4.19 below. This comparison study is to define the
test activities during the software development life cycle or phases.
Table 4.19 : Test Methodologies Comparison based on Software Development
Phases
Software
Systematic Test and
Development
Evaluation Process
Process
Phases
(STEP)
(HTP)
Business
•
Develop Plan
Requirement
•
Analyze
Objectives
Formal
Specification
•
Design Test
RUP Test Discipline
•
HeiTech Test
None
Define Evaluation
Mission
•
Verify Test
Approach
•
Plan Test
106
Software
Systematic Test and
Development
Evaluation Process
Process
Phases
(STEP)
(HTP)
Architecture
•
Design
•
Plan and Designs
Detailed Design •
Code and
Implement Test
RUP Test Discipline
•
Implementation •
Validate Build
•
Stability
Execute Test
•
Test and Evaluate
Check Adequacy
•
Achieve
Evaluate
Acceptable
Software and
Mission
Testing
HeiTech Test
•
Design
Test
•
Implement
Test
•
Evaluate
Test
Improve Test
Assets
As a result, based on the summary, all test activities or tasks are performed
started during business requirements analysis. However, from the study conducted in
HeiTech, the initial testing activity such as develop test plan is started after
conformation or sign off the product requirements with the stakeholder. Besides
that, in HeiTech, the Test Analyst and Test Designer roles are only effective after the
requirements documents such as Scope of Work and User Functional Specifications
has been approved by the stakeholder and the development team.
As a summary, a best approach of testing process or methodology can be
comprises to have a good test planning and strategy at early stage of software
development phases, where the defects and risks can be identified earlier. Hence, the
initial testing activities from these testing methodologies can be summarized as
represent an intensive testing for effective testing process. In addition, a good testing
methodology also require best testing types and techniques based on specific
functions, and also accommodate enough testing roles and responsibilities and
advocate early involvement in order to performed successful testing activities.
CHAPTER 5
SHARED BANKING SERVICES (SBS) SYSTEM
5.1
Introduction
This chapter will discuss on the overall of Shared Banking Services (SBS)
system and its characteristics. Generally, Shared Banking Services (SBS) system is
one of banking application system that offers core banking services at another
organization’s branches, which have business deals with the particular banks. The
objectives of this shared banking services application is to accommodate an easiness
of any transaction or activities from core banking services system. Therefore, the
next section will describe the details of core banking services and its relation with
shared banking services (SBS) system.
5.2
Core Banking Systems
Core banking is another way of saying the core functions of a bank. These
functions represent the essential or vital business activities of banking.
The
definition of core banking may have been muddied by the emergence of packaged
computer solutions which combine core banking functions with other elements of a
bank’s operations but at the most basic level core banking manages financial
transactions and their impact on the accounts of its customers [47].
108
A core banking systems often provided other routine maintenance activities
[51]. Therefore, the essential activities of core banking are opening and closing
accounts, calculating interest for both due to the customer and due from the customer
and processing customers’ standing orders.
Besides that, providing account statements and interfacing to outside systems
for making and receiving payments are also considered to be part of the core
business of banking and therefore the legitimate functions of a core banking system.
Thus, some of core banking components in details should include:
•
Interest calculations
•
Processing of cash deposits and withdrawals
•
Processing of incoming and outgoing of remittances and cheques
•
Customer management
•
Customer account management
•
Definition of the bank’s products (product management) including such
things as minimum balances, interest rates and number of withdrawals
•
Interest rate definition
•
Customer’s standing instructions
•
Maintaining records of all financial transactions
In addition, a packaged retail core banking system might include functions
that are closely related to core banking activities that may include functions such as:
•
Processing applications for loans received from branches
•
Branch management, often including facilities for maintaining teller cash
drawers and others
•
Cheque book ordering
•
Internet banking
Generally, banks can have differences in the type of business that they
conduct. They may for example be in retail banking, which is the type of banking
that is seen in the high street, or be in wholesale banking, which is the bank-to-bank
109
market often referred to as the inter-bank market or securities trading, which is the
buying and selling of stock, shares or government debt instruments.
As a result, their core businesses can be considerably different and their core
banking system will therefore reflect the type of business of the bank which such as:
•
Retail banking
•
Commercial banking
•
Wholesale banking
•
Treasury banking
•
Private banking
Consequently, a super breed of core banking systems has emerged which
offer functionality in addition to core banking system which called universal banking
systems. These systems can accommodate combinations of banking services such as
retail, wholesale, private banking and securities trading. Therefore, an advantage of
using a universal system is that data can be transferred easily between the different
modules so a bank can identify customer trends or selling opportunities for example
if a customer has high levels of cash balances and also performs securities trading
then they may be a sales opportunity for private banking services.
However, these systems still has drawbacks such as it is unlikely that a single
vendor is able to offer modules that are ‘best of breed’ in each function. Hence, the
alternative approach is to use a core banking system and augment its functionality by
using ‘best of breed’ packages for the specific functionality needed, for example such
as foreign exchange trading or portfolio management.
More recently, the arrival of Service Oriented Architecture (SOA) has caused
package vendors to redesign their core banking systems. This SOA enables broad
packages to be de-constructed into their constituent parts with each part or function
then offered as a service.
110
For example, a core banking activity of opening a new account could be split
into multiple services, such as one for establishing a customer name and address in
order to check if one already exists for that customer, one for performing a credit
check, one for opening the actual account type needed, one for ordering a ‘new
customer’ pack, one for ordering a plastic card for the account and others. As a
result, this is allows a return to the pure core banking system component of
transaction and account management.
5.3
Shared Banking Services (SBS) at R@PIC
Shared Banking Services (SBS) [6] is a counter-based transaction system
developed on top of a software framework (Hybrid Client) for developing a frontend, transaction based system. This system is the case study system that being
develops in R@PIC. Therefore, this system offers selected banking services that can
be carried out at Pos Malaysia (PMB) branches. This SBS system comprises three
major modules, which are:
•
Transaction system
•
Utilities Functions
•
System Configuration
This SBS system will be installed at PMB branches to enable offerings of
Bank A’s banking services at the selected locations. The end users of this system are
categorized into four, which are:
•
Managers
•
Administrator
•
Supervisor
•
Teller
This system adopted smart client architecture, which this solution makes use
of local PC (Personal Computer) processing power to process Bank A’s transactions.
Therefore, Smart Client architecture combines the benefits of both Thin Client and
111
Rich Client. It offers rich user interface, ease of integrating to local devices and
simple deployment model using ClickOnce technology.
ClickOnce technology will enable centralized the deployment of SBS
modules and updates. A Web Server will act as the deployment server to distribute
SBS to the selected PMB branches. Besides that, the communication between PMB
branches to the servers at PMB Headquarters (PMB HQ) will be done via web
services and to communicate with the backend (host) system, SBS will make use of
Simple Object Access Protocol (SOAP) approach.
In addition, for this services system, a generic of client/server branch delivery
system usually consists of a front-end application and a back-end or “host” system.
Hence, the front-end application is installed at every branch of the organization and
the common roles of this application is to capture transaction data, perform field
level validations, update local database, print and communicate to the back-end
system. These common roles are generic to all organizations that implement a
branch-delivery system. The variations are only at business rules and transaction
flow, which is unique to each business requirements.
Furthermore, in SBS architecture also includes this Device Service Server
(DSS) that was developed to make devices integration and device sharing easier.
Thus, this software provides a set of uniform APIs so that access to various devices
can be done in a unified manner. One of the prominent features of this server
software is its ability to enable integration with various devices for example printers,
RFID readers, thumb print scanners, and MyKad readers.
Moreover, this server software was designed and developed in a very
modular manner so that new servers for newly supported devices can be added easily
and ready to be used by applications. Besides that, another interesting feature of
DSS is it enables web-based application to control local devices in a simple way via
112
Web Service and the sharing of devices activities can be done across different
platforms such as Windows and Linux.
5.3.1
Transaction System
This module consists of several categories, which are Open Account, Cash
Activities, Remittance, Inquiry, Reversal, Passbook Maintenance, Cash Settlement
Total and Others. Table 5.1 explains the transactions listed for each category in
details.
Table 5.1 : Transaction System Categories
Category
Open Account
Cash Deposit and
Payment
Withdrawal
Descriptions
To open new account for different types of account:
•
Individual Savings Account for Passbook
•
Individual Savings Account for Passbookless
•
Joint Savings Account for Passbookless
Daily transactions are:
•
Saving Account (SA) Cash Deposit and Reversal
•
Corporate Current Account (CCA) Cash Deposit
•
Personal Saving Account (PSV) Cash Deposit
•
Loan Repayment
•
Hire Purchase (HP) Payment
•
Credit Card Payment
Daily withdrawal activities for account:
•
Remittance
Saving Account (SA) Withdrawal and Reversal
The list of transactions:
•
Foreign Telegraphic Transfer (FTT)
•
Foreign Worker Telegraphic Transfer (FWTT)
•
Inquire Membership
•
Inquire IC No
113
Category
Passbook Maintenance
Reversal
Descriptions
•
Agent NOSTRO - Philippines
•
Other Countries
•
GIRO Funds Transfer
The list of transactions:
•
SA Passbook Update
•
Change Posting Line
•
Change Passbook Status To New
•
Inquire Passbook Tagging
The list of transactions:
•
SA Cash Deposit-Reversal
•
CA Cash Deposit – Reversal
•
PSV Cash Deposit – Reversal
•
Loan Cash Repayment – Reversal
•
HP Payment via HP A/C No or Vehicle
Registration No – Reversal
•
Credit Card Cash Payment – Reversal
•
SA Cash Withdrawal – Reversal
•
FTT Reversal
•
Foreign Worker TT – Reversal
•
GIRO Funds Transfer – Reversal
Reversal can be done by in two ways:
Inquiry
•
Reversal - By Journal No.
•
Reversal - By Electronic Journal
The list of transactions:
•
Name/IC No Inquiry
•
SA Last Transaction Inquiry
•
SA Activity Inquiry
•
CA Last Transaction Inquiry
•
CA Activity Inquiry
•
Credit Card Inquiry
•
FTT Transaction Detail Inquiry
114
Category
Others
Cash Settlement Totals
5.3.2
Descriptions
•
FWTT Transaction Reference Inquiry
•
FWTT Transaction Inquiry
•
GIRO Transaction Detail Inquiry
•
HP Inquiry
•
Loan Activity Inquiry
The list of transactions:
•
Rounding Mechanism
•
GL Cash Credit and Debit
The list of transactions:
•
Teller Total
•
Branch Teller Total
Utilities
This section describes about SBS utilities that being provided for end user of
this system. Therefore, upon completion of installation, some of these utilities need
to be performed so that the users of the counter system, which are Teller, Supervisor
and Manager, can continue with daily system transaction. Table 5.2 shows the
summary of utilities functions provided in SBS System.
Table 5.2 : Utilities Functions
Utilities Functions
Descriptions
Mandatory Utility
Start of Day
To performed branch start up once a day because branch
cannot start any operations before Start of Day is
executed.
End of Day
Used by teller to complete tasks after the last transaction
of the day is done.
Transmission
Transmission is done to consolidate the branches
Quality Control (QC) file at the Hybrid Server. After
115
Utilities Functions
Descriptions
confirmation of Branch Total is done the system will
automatically transmit the branch QC file to the Hybrid
Server. Upon receiving all QC files from all the PMB
branches the system will generate a consolidated QC file
and send the file to Bank A HQ using File Transfer
Protocol (FTP).
Teller total
Teller total function provides teller total at Bank A host.
Therefore, teller could make comparison with teller total
at Bank A counter system.
Branch total
Branch total function provides branch total at Bank A
host. Therefore, the post master or the supervisor could
make comparison with total at Bank A counter system.
Final branch total
Final branch total function provides the same function
as branch total. The difference is that after performing
final branch total, teller will not be able to perform any
financial transaction.
User Management
Sign-on/off
Forced User Sign Off
Allow user to sign in or sign out from the system.
Auto sign out of user if any Hybrid application has been
idle for more than five minutes.
User ID issuance
Verify user ID validity for branch code and user ID not
exist.
Unlock ID
Password maintenance
Function to unlock user ID.
Verification
on
password
management,
such
as
Password Expired, Change Password, or Password does
not match.
Access level
Provide access level restrict on certain functions for
assignment
each of users.
Stock Control Module
Provide Passbook Control Register that can be
maintained by user.
Transaction Related
116
Utilities Functions
Descriptions
Repeat key (F9)
Teller can perform a repeat function by going to the
selected field and press Repeat button or F9 key, thus
relieving the teller from keying in the same data.
Remote override
Remote override will be triggered by any transaction
that needs approval from supervisor.
For example,
withdrawal for RM5000 and above needs an approval
from a supervisor.
Electronic Journal and
All transactions will be updated in an Electronic Journal
Viewer
for tracking and recovery exercises. Teller will be able
to do search on the journal and view activities and
transactions posted for the day. This is really helpful to
trace any abnormality if problems such as black-out or
system down occur.
Calculator function
Enables a teller to do calculations and transferred
relevant data to the respective fields.
Successful and
unsuccessful
transaction update
This is to enable the system to perform reversal of those
successful as well as unsuccessful transactions.
For
those unsuccessful transactions that is updated at the
host but the local pc get a message of ‘host timeout’
and not updated at the local pc, the system still allow the
transaction to be reversed.
Reversal
The system enables financial transaction such as
withdrawal, deposit, and remittance to be reversed after
successfully updated to host by referring to the
transaction’s journal number.
Foreign Exchange Rate
System shall download rate provided by Bank A for
remittance purpose. System shall provide utility to view
the downloaded rate.
Passbook Control
Register
This utility allows supervisor/manager to control
passbook stock.
Reactivate Online
Manager shall perform Reactivate Online Posting to
Posting
restart the system if a transaction needs to be performed
117
Utilities Functions
Descriptions
after branch has close and perform balancing.
Reference Table and
This utility is for the user, either Bank A administrators
Devices files
or PMB administrators to maintain reference tables and
Maintenance
devices configuration files.
Reports
Daily Cash Balance
Report
This report provides a summary of individual teller
cash-in, cash-out, and cash-in-hand summary.
Teller Transaction
Report
This
report
provides
Deposit/Payment,
a
summary
Remittance,
and
of
teller
Withdrawal
summary.
Teller Activity Report
Teller report produces detail transactions done by the
teller.
The report can be used to make comparison
against teller total at Bank A host.
Activity Report
This report lists detail financial and non financial of
each teller.
Reversal Transaction
This report lists detail reversal transaction of each teller.
Report
Total Summary Report
This report provides a summary of each teller cash-in,
cash-out, and cash-in-hand summary for the branch.
5.3.3
System Configuration
This module provides functions for SBS System Configuration, such as
Branch Configuration, Manager Configuration, User Profile, Table Maintenance,
View Archive, Administrator Configuration and others.
Hence, every function
provided user accessibility to add new, modify or update and delete the record.
Table 5.3 below shows the summary for SBS system configuration functions.
Table 5.3 : SBS System Configuration Functions
Category
Descriptions
118
Category
Descriptions
Branch Configuration
Administrator can perform this configuration from Bank
A HQ in order to add new branch, update or delete
existing branch, and deactivate or reactivate branch.
Manager
Administrator can perform this configuration in order to
Configuration
add new manager, update or delete manager profile. For
every creation or modification will be update in
Electronic Journal and notification email will be sent to
particular manager.
User Profile /
This function is for Administrator to manage his/her
Administrator
profile and password, such as change or reset password.
Configuration
For reset password, only HQ Administrator can reset
password of Branch Administrator.
Table Maintenance
Allows Administrator to maintain the table information
based on current technical specification. Administrator is
allowed to Add, Update or Delete table based on current
requirements.
View Archive
This function is provided for HQ Administrator to view
Electronic Journal that is kept for two years, for certain
reason. Searching options are provided, such as date,
time, journal number, successful status, account number
and also transaction amount.
5.4
Testing Characteristics for SBS System
Banking IT systems are critical to the competitiveness of banks, owing to
their sheer size, the number of transactions they facilitate, and the wide range of their
product and service offerings. These banks also begin to expand rapidly and
customer demands for centralized operations become louder, the need to integrate the
IT systems of new branches and subsidiary banks into the main IT stream is
becoming critical [52].
119
Shared Banking Services (SBS) is about knowing customers' needs and
providing them with the right banking transaction at the right time through the right
channels like 24 hours a day, 7 days a week. Hence, the best-of-breed solutions for
credit processing, document management, core banking transactions, and knowledge
management are required to be clearly identified by the development team and
stakeholder of the system.
The banks also like to integrate a whole range of financial products covering
all the foreign branches of the bank. The system is required to have external and
internal interfaces to functional sub systems such as Foreign Exchange, Money
Markets, Funds Transfer, Profitability Analysis and Credit. Hence, the customer was
looking for better adaptability, stability and reliability of this Shared Banking
Services (SBS) system.
Banks and financial institutions are sensitive to the knowledge that a single
error can result in loss of repeat business, revenues and reputation [53]. They realize
the importance of testing and re-testing systems, processes, protocols and products.
The focus acquires are more importance, when it comes to banking application
domain process.
Software testing is a very exacting job, making it imperative for this service
to be managed by dedicated experts with the necessary domain knowledge. Banks
and financial institutions today face an environment marked by rapid mergers,
acquisitions, rising customer expectations, increasing regulatory requirements and
fierce competition. In this dynamic scenario, creating, leveraging and retaining value
from customer relationships and maintaining profitably pose crucial challenges are:
•
Applications are complex in nature
•
Carry Legacy Systems
•
Involves large volumes of data
•
Creation of business scenarios for complete test coverage
•
Transactions needs to be secure
•
Ensuring data integrity
120
Establishing right software testing processes is critical for any organization
that wants to improve quality, reduce time-to-market, and manage costs [54].
Mostly, software vendors often focus on software development activities, which
leave them with very little time to develop testing competencies, with lack proven
processes and methodologies especially in testing for banking application domain.
Therefore, to test the banking application whether for functionality or any
other testing, one should have enough banking knowledge to know how the system
will behave with respect to end users. Next subsection will discuss on four defined
characteristics of SBS system as shown in Figure 5.1 that required comprehensive
testing, which are:
•
Functionality test
•
Performance test
•
Security test
•
Automation test
Figure 5.1: SBS Testing Characteristics
121
5.4.1
Functionality Testing
Functional testing service aims to find how well the system executes the
functions it is supposed to execute-including user commands, data manipulation,
searches and business processes, user screens, and integrations.
Good testing
procedures need to covers the obvious surface type of functions, as well as the backend operations. The individual components and processes required to be tested
before testing the entire system, where the industry standard defect tracking tools to
track defects in the system and deliver detailed test metrics at the end of each test
cycle can be utilized.
Besides that, based on the product functionality testing cycle, regression
testing can be conducted, which ensures the proper behavior of an application after
fixes or modifications are made to the system or its environment. Table 5.4 lists
several challenges and solutions that require the testing team to conduct the
functional testing for banking application or SBS system specifically.
Table 5.4: Challenges and Solutions for Functional Test
Challenges
Client
requirements
increasingly
are
vertical
Solutions
becoming An
end-user focused approach
focused; functional
testing
resulting
in
to
an
demanding strong domain expertise to optimized view of the functionality
ensure quality design and optimum testing cycle. A similar approach also to
coverage with minimized risk.
Module, Integration, System and UAT
testing.
Conduct end-to-end software functional
testing
Module,
services
and
Integration
include
and
also
System
Testing.
Greater degree of standardization / Performed User Acceptance Testing
productization with shorter planned (UAT)
release cycles.
services
for
end-user
organizations and system integrators,
122
including on-going regression testing of
planned and maintenance releases.
Use approach to test design is based on
"customer experience cycle" and not on
product or application flow.
Solution complexities involve high Provides data migration testing to
degree of integration across multiple ensure accurate and complete migration
products.
of data from legacy systems.
Proprietary
domain
frameworks
developed for the different sub-verticals
encapsulating deep understanding of the
domain as well as powerful design
techniques like 'Orthogonal Arrays' resulting in improved testing coverage.
5.4.2
Performance Testing
Performance testing is focus on ensuring the compliance of a system or
component with specified performance requirements facilitating conformance to high
service level goals and faster time to market. An effective extendable, repeatable,
iterative performance testing practice is in place with well defined guidelines that are
integrated into the SDLC to measure and monitor the application performance.
Performance testing spanning load testing, volume and stress testing requires
a comprehensive strategy to address the challenges like:
•
Constantly changing software applications with shorter release cycles
•
Ever increasing user base and access channels
•
Enhanced hardware capabilities resulting in more complex deployment
architectures
123
•
Demands a healthy mix of long-term assessment and planning coupled
with the ability to address problems and bottlenecks in the short term
Predicting performance capability of banking solutions is essential to succeed
in this competitive market. Performance testing services help to manage risk with
developing or deploying business critical systems, by clearly determining the scope,
complexity and size of the load and the test architecture is used for test execution.
Several performance testing services that can be performed for banking application
like SBS system are:
•
Load Testing
•
Stress/Scalability/Capacity Testing
•
Spike and Synchronization Testing
•
Failover and Resilience Testing
•
Performance Benchmarking
•
Performance Diagnostics Testing
•
Performance Tuning
•
Soak and Reliability Testing
•
Network Bandwidth Verification
•
Application Performance Monitoring
Besides that, several criteria or metrics of the system performance are
required to record, analyze and report are as followings:
•
Connect Time
•
HTTP Request Time
•
Transaction Response Time
•
Throughput
•
Web Page Break Down
•
Database Resources
•
Web Server Resources
124
5.4.3
Automation Test
Manual Testing requires operator input, analysis or evaluation throughout the
entire test cycle with emphasis on evaluating correctness and ascertaining test status.
Basically, manual testing framework gives the tester the flexibility to perform adhoc/ random tests that help detect real user bugs.
Meanwhile, Automation Testing reduces the test cycle time since it involves
very minimum operator input, analysis or evaluation leading to increased test
development time, enhanced test coverage quality, fast defect detection and
correction. Test cases for automation are prioritized based on business risk and
application complexity and accentuate, in order to improve the efficiency of the
testing process.
Table 5.5 lists several challenges and solutions regarding to
automation testing for banking application.
Table 5.5: Challenges and Solutions for Automation Test
Challenges
Solutions
Dynamic business needs requiring Provide Software Test Automation strategy,
constant enhancements / upgrades automated testing tool selection and QA test
to software solutions.
automation framework definition.
Provide Testing professionals proficient in
Mercury, Rational, Segue and other leading
industry standard test automation tools.
Enhance experience in different test and
defect
management
tools;
including
integration of the same to functional
automation tools.
Greater product driven solution Provide automation pack creation and
focus with multiple releases in a execution; with an integrated automation
year.
approach that covers test planning and
management, test scripting and defect
management.
125
Enhance experience in product environment
involving multi product suites with regional
variations and multiple planned releases in a
year.
Ever shortening software testing QA Testing Automation pack maintenance
cycle times coupled with the need with strong configuration management and
for enhanced regression coverage.
version control processes.
Process consulting practice integration to
offer end-to-end automation solutions that
also address process improvement initiatives
necessary to leverage QA test automation
benefits.
5.4.4
Security Test
Banking application customers expect quick access and tight security, and
have no patience for delays or downtime of the system. A solution that could handle
the testing challenges in order to deliver a secure and satisfying user experience on
the system is important. To ensure that the security infrastructure is solid, the testing
solution needs to evaluate overall performance, supply a high degree user and
network realism, and deliver the ability to test with both normal and malicious
traffic. In addition to providing comprehensive testing, it should also be easy to set
up and use.
Therefore, the software security testing service should involves tests targeted
to break the security built into the product. Scenarios where the application fails or
behaves abnormally, and can withstand the attack are identified and verified. Some
various attack scenarios, which include buffer overflows, memory leaks, reverse
engineering, license evasion and registry tampering (operating system specific) are
required to be defined. This security tests are conducted in a state-of-the-art security
126
test room, equipped with the requisite tools and malware and comprehensive testing
approach is adopted with a clearly defined methodology.
The security testing cycle for software begins with the scope definition and
an understanding of customer expectations, followed by multiple phases like
analysis, test planning, execution and reporting. A detailed analysis is carried out on
the application to identify relevant scenarios.
Tests are performed simulating
customer end scenarios, including testing the ability for similar applications to
operate securely. For SBS system, security tests are more required to focus on sign
on authentication and transaction flows with encryption approach.
Getting a security test done for a banking application help improves the
software quality and aids customers by reducing development and maintenance costs
as the system is shipped with minimal security issues. Besides that, the security
flaws created during the system’s development can be discovered earlier.
5.4.5
Advantages
A focus on all technologies related to the characteristics of core banking
applications or SBS system and their integration and interfaces with other banking
applications is important. The SBS application required managing the technology
related to checking accounts, saving and certificate of deposits, and funds transfer.
Therefore, a good practical approach of software testing on this banking
product lead to several advantages from business perspective, such as:
•
To protect from unauthorized and fraudulent users
•
To support multilingual, multicurrency and global operations
•
To ensure efficient data warehousing to support management in decision
making
127
•
To handle high volume of transactions from various channels such as
ATM, Internet and telecommunication devices
5.5
•
To leverage risk management, business continuity and disaster recovery
•
To meet stringent government regulations and norms
•
To determine the capabilities of the application when using over load
SBS Testing Implementation at R@PIC
Figure 5.2 below illustrate the SBS testing process that being implemented at
R@PIC that comprise of four main activities, which are Plan Test, Design Test,
Implement Test and Evaluate Test. This figure also shows the input and output
documents that required during this testing process.
Figure 5.2: SBS Testing Implementation at R@PIC
128
At system requirements phase of SBS system, a Scope of Work document is
proposed to the client. This document comprises all objectives, scope and activities
that need to be performed for this project. This document in general also defined the
test planning and strategy that later can be carry out through the project development.
Furthermore, all system requirements for SBS system have been documented
in User Functional Specification (UFS) that consists of three modules, which are
Transaction System, Utilities and, Installation and System Configuration. These
documents are more likely as user manual, where test case design only can depends
on the system functionality test approach.
5.5.1
Plan Test
The first stage in SBS testing implementation is test strategy and planning. In
this stage, a Test Plan is required to be produced, in order to make clear tactic on
how the testing is to be performed. Generally, test planning and strategy is focused
on what, how and when is the testing activities to be performed. Therefore, this test
planning will defined several criteria such as test objective and scope, test approach,
test duration and venue, roles and responsibilities, levels of testing and types of
testing.
Hence, the test planning for SBS system has followed an established HeiTech
Defined Test Plan Guideline. A Master Test Plan has been prepared for this project
and submitted to the stakeholders. However, due to certain reason, another Test Plan
is prepared, which follow the stakeholder’s Test Plan template. All these test plans
are clearly defined the testing strategy and activities to be carried out during testing
implementation on this system.
However, the preparation of this Test Plan is conducted after the development
team and stakeholder already endorse all the requirements documents. Besides that,
the test analyst and test designer roles are still not effective during this requirement
129
stage, because only project manager and software developers that involve in this
requirement discussion with the stakeholder.
5.5.2 Design Test
In SBS system, the test design is classified into three criteria which is
software, hardware and system integration between both. Therefore, in terms of
software, tests will cover for all three modules or major parts of SBS system, which
are Transaction System, Utilities Functions and System Configuration.
Then, for hardware and system integration, tests was design to focused on
Device Service Server (DSS), which connects to Printer and Host Server and its
performance towards Hybrid Client application. However, the test design for SBS
system testing is only focus on functional testing approach of three modules of SBS
system.
Then, the real test design is drawn into specific documents, which are test
cases. The test design is the most critical one, which decides the test case
preparation. Hence, the test designs portable to assess the quality of testing process.
Test cases are design based functional requirements that stated in UFS documents.
Therefore, the test cases that being prepared covers all validation of each functions
and flows of the specific modules.
5.5.3 Implement Test
SBS system is currently in test execution stage, which all testing levels are
carried out step by step. For SBS system, levels of testing that being defined in Test
Plan consists of Unit Testing, Integration Testing, System Testing and Acceptance
Testing. Hence, the developers are responsible for the Unit Testing activities. Next,
at Integration and System Testing, the developers, tester and project manager is taken
130
the responsible to perform the testing. Then, for Acceptance Testing, the developers
and tester are conducting testing at the simulation environment, which similar to the
production environment.
5.5.4 Evaluate Test
In this SBS system, development team performed integration and system
testing modules by modules, which is if bugs found, they fixed the infected modules
and continue testing on another part of modules.
Therefore, all bugs found is
reported into Defect Logs, which follows HeiTech Defined Document.
At the end of each levels of testing, a Test Report will be prepared in order to
assess the effectiveness of testing implementation for this project. However, due to
the time constraint for this industrial training, involvement in Test Report preparation
is only performed by Tester Leader. As a result, Test Report for SBS system will not
be included in this report.
As SBS testing process is an iterative activities, which need to be carried out
until whole parts of product is be released into the productions area. Therefore, the
next stage will come in is software maintenance which comprise a lot of
modifications and changes. Hence, a regression testing is the next focused for this
testing process, which can be continues by R@PIC team.
CHAPTER 6
CUSTOMIZED SOFTWARE TESTING PROCESS
6.1
Issues with Software Testing Implementation at R@PIC
Software testing is a process of analyzing the effectiveness of software.
Testing has two objectives in order to determine if the software performs as
expected, which are:
i.
To identify the differences between existing and expected conditions,
which this differences are sometimes called ‘bugs’, but it now connotes a
defect in the performance of either hardware or software.
ii.
To enable the expected performance of software to be determined before
it is used for business purposes and this type of testing is to determine
whether the right mix of software, hardware and personnel can satisfy the
business needs.
However, testing implementation for SBS at HeiTech less effective in several
aspects such as late preparation of test planning and strategy, that leads late
involvement of testing team into the project. Besides that, testing approach or types
that performed for SBS system only cover on functionality of the system. Based on
the study conducted during testing implementation on SBS system, several important
testing types or characteristics are not covered in the testing, which performance
testing, security testing and also no tool are provided in order to conduct any
automation testing.
132
In addition, during software development of this banking application, there
are no reviews conducted with the testing team or stakeholder, in order to get fully
information of this application. This practice may lead several others drawback for
post-implementation if several issues, such as incorrect requirements, faults system
design and other happened without early detection of these defects.
6.2
Developing Customized Software Testing Process (CSTP)
As software testing process is important in software development activities,
and SBS system is also important for core banking application, an intensive software
testing process is required to be developed and provided into this organization.
Therefore, the next subsection will discuss the summary of every comparison study
that being conducted previously and several testing characteristic for SBS system.
Finally, a proposed Customized Software Testing Process (CSTP) is clearly defined
based on these comparison studies.
6.2.1
Main Phases for Testing Correspond to SDLC
The Test Models that previously studied, which are V-Model, W-Model and
Butterfly Model are suggested that testing activities or process must start at the early
stage of software development life cycle, which is during business requirements and
analysis phase. Figure 6.1 shows the summary of main phases for testing activities
that correspond to software development phases.
133
Figure 6.1: Main Phases for Testing Correspond to Software Development Phases
6.2.2
Main Activities of Software Testing Methodologies
This comparison study is conducted on three different testing methodologies,
which are Systematic Test and Evaluation Process (STEP), RUP Test Discipline and
HeiTech Test Process and its activities summary shows in Figure 6.2. From this
study, several main activities for software testing process that can be performed
during the software development phases are being defined.
134
Figure 6.2: Main Activities of Software Testing Methodologies
6.2.3
Main SBS Testing Characteristics
Shared Banking Services (SBS) is about providing customer with the right and
secure banking transaction at the right time through the right channels. Hence, the
best-of-breed solutions such as cash activities, credit processing, document
management, others core banking transactions and knowledge management are
required to be clearly identified by the development team and stakeholder of the
system.
The testing focus acquires are more importance, when it comes to banking
application domain. Banks and financial institutions are sensitive to the knowledge
135
that a single error can result in loss of repeat business, revenues and reputation. They
realize the importance of testing and re-testing systems, processes, protocols and
products. Therefore, four of define characteristics of SBS system are shown in
Figure 6.3 below.
Figure 6.3: Main SBS Testing Characteristics
6.2.4
Proposed Customized Software Testing Process (CSTP) for SBS
In the whole development process, testing consumes highest amount of time.
However, most of the developers in organization oversee this issues and therefore
testing phase is generally abandoned. As a consequence, erroneous software is
released.
Hence, one of good approach to improved product quality is provide a good
testing methodology that can be implemented at early phase of software
development.
Figure 6.4 shows testing methodologies comparison with the
Customized Software Testing Process (CSTP) that is proposed in this project.
136
Figure 6.4: Software Testing Methodologies Comparison
Figure 6.5 illustrate the details of the Customized Software Testing Process
(CSTP) for banking application, such as Shared Banking Services (SBS) system,
with regard to the software development phases.
137
Figure 6.5: The Customized Software Testing Process (CSTP)
6.2.4.1 Requirements Reviews
138
Normally in many organizations nowadays, developers itself take part in the
requirements stage. Thus, a tester should also be involved in this stage, especially
for product-based organizations, since a tester can thinks from the user side whereas
a developer cannot. All the requirements should be documented properly for further
use and this document is called normally known as Software Requirements
Specifications or Business Requirements Specifications.
For banking application testing, early involvement of testing activities such as
requirement reviews can help to minimize risk. There are several common issues
that can be identified in a Requirements Review, such as:
o
Insufficient detail in the current requirements to manage the project to
a successful conclusion
o
Vague requirements which make it difficult to create a realistic
project estimate
o
Critical requirements that are not captured in the current requirements
o
Incomplete requirements making the creation of test cases and designs
difficult
Requirements Reviews [55] is when a group of people read and analyze the
requirements, look for problems, meet and discuss the problems and agree on actions
to address these problems. Figure 6.6 show the process of Requirements Reviews
and Table 6.1 describe the details of each task.
139
Figure 6.6: Requirements Reviews Process
Table 6.1: Details of Requirements Reviews Process
Tasks
Plan review
Descriptions
The review team is selected and a time and place for the
review meeting is chosen.
Distribute documents
The requirements document is distributed to the review
team members.
Prepare for review
Individual reviewers read the requirements to find
conflicts, omissions, inconsistencies, deviations from
standards and other problems.
Hold review meeting
Individual comments and problems are discussed and a set
of actions to address the problems is agreed.
Follow-up actions
The chair of the review checks that the agreed actions
have been carried out.
Revise document
The requirements document is revised to reflect the agreed
actions. At this stage, it may be accepted or it may be rereviewed.
140
6.2.4.2 Test Plan
A successful work always contains a good plan, in terms of achieving specific
goals. Hence, almost people believe that without a good plan, no work is a success.
As a consequence, software testing process also requires a good plan. Test plan
document is the most important document that brings in a process oriented approach.
Moreover, a test plan document should be prepared earlier during the requirements
stage of the project, in order to start identifying the best test strategy and approach
for further testing process.
Generally, the test plan document consists of the following information that
can help later for performing software testing activities:
•
Total number of features to be tested
•
Testing approaches to be followed
•
The testing methodologies
•
Number of man-hours required
•
Resources required for the whole testing process
•
The testing tools that are to be used
•
The test cases
In addition, during this testing planning, the test strategies which are blackbox testing and white-box testing need to be defined. Besides that, several test
approaches such functional and non-functional testing also required to be performed
for this SBS banking application system.
6.2.4.3 Test Design
Besides good test planning, a good test design is also one important aspect for
successful software testing. Test design is done based on the requirements of the
project. Test has to be designed based on whether manual or automated testing is
done. Therefore, for automation testing, the different paths for testing are to be
141
identified first and an end to end checklist has to be prepared covering all the features
of the project.
In addition, the test design is represented pictographically of software testing
activities.
This test design involves various stages that can be summarized as
follows:
•
The different modules of the software are identified first
•
Next, the paths connecting all the modules are identified
As banking application is a risky system, which provides daily transaction
and usage by many users, performance testing is one of the important types of testing
that need to be considered during this test design stage. Therefore, application and
network performance are the best testing strategy in order to improve the
performance of SBS application.
Moreover, to conduct successful performance
testing, various tools that available in market today can be utilized for this testing
activity.
Apart from performance testing, conducting security testing for banking
application system like SBS system can help to improve the reliability of the system.
Most of banking application either counter-based system or online system requires
login id and password management, in order to protect the data integrity. In SBS
system, user authentication and transaction activities are the critical parts that need to
be more secure and reliable to the customers.
6.2.4.4 Design Reviews
In a good practice of system development process, the need of software
design review is important, in order to perceived good software quality. Nowadays,
mostly software design is done in systematical manner or using the UML language.
142
Therefore, all parties involved in the project can review and understand the system
design. Besides that, testing teams and quality assurance committee also can do the
reviews over the design and can suggest for any ideas or the modifications that are
needed.
Design Review Process is a mechanism for ensuring design standards,
alignment, and diligence throughout the course of the product design process.
Standards are to ensure that designs meet appropriate standards for consistency,
accessibility, usability, download time, and others. Alignment are to ensure that
designs meet business goals, designs can be well integrated into the brands on which
they will be deployed and to minimize late-stage changes to product requirements
and concepts.
Besides that, diligence are to realize maximum value from early-stage design
methodologies, increase accountability and clarity of action plans by keeping
minutes of each design review and involve people outside the design team at
appropriate junctures. Hence, design reviews have several objectives such as:
•
Provide independent and critical assessment
•
Identify issues and resolve them off-line
•
Assure that interfaces are well understood
•
Promote communication among participants
•
Formalize and document accomplishments
•
Promote confidence in accomplishments
•
Signify that a particular accomplishment is completed
•
Formalize the expected date of the accomplishment
Here are steps in the Design Review Process:
i.
Conceptual Design Preview
This step is to ensure that the initial design direction maps to the
business goals and user needs, and to review the design for alignment
143
with broader initiatives and possible integration with other product
designs.
ii.
Design Standards Checkpoint
This step is to confirm that the design meets required design
standards.
iii.
User Interface Design Review
This step is to review specific interaction behaviors and to provide
guidance to designers on problematic issues.
iv.
Creative Direction Review
This step is to ensure that the visual design maps to the creative
direction of the project.
6.2.4.5 Test Cases Preparation
Designing good test cases is a complex art [56]. Generally, test cases should
be prepared based on the following scenarios:
•
Positive scenarios
•
Negative scenarios
•
Boundary conditions and real world scenarios
A test case is a set of conditions or variables under which a tester will
determine whether a software system meets the specifications. A test oracle is the
mechanism to determine whether a software program or system has passed or failed
in certain test conditions. An oracle also could be identified from the requirements
or use cases. Therefore, it may take many test cases to determine that a software
program or system is functioning correctly.
144
Test cases are often referred as test scripts, particularly when written. Then,
these written test cases are usually collected into test suites. A test case may include
the followings:
•
The purpose of the test.
•
Special hardware requirements, such as a modem.
•
Special software requirements, such as a tool.
•
Specific setup or configuration requirements.
•
A description of how to perform the test.
•
The expected results or success criteria for the test.
Organizations may take a variety of approaches to documenting test cases;
these range from developing detailed, recipe-like steps to writing general
descriptions. In detailed test cases, the steps describe exactly how to perform the
test. In descriptive test cases, the tester decides at the time of the test how to perform
the test and what data to use.
Hence, most organizations prefer detailed test cases because to determine
pass or fail criteria is usually easier with this type of case. In addition, detailed test
cases are reproducible and are easier to automate than descriptive test cases. This is
particularly important if testers plan to compare the results of tests over time, such as
when they are optimizing configurations.
In addition, test cases should be written by a team member who understands
the function or technology being tested, and each test case should be submitted for
peer review. Hence, test cases preparation for SBS system can be based on various
types of testing such as functional, usability, security and performance testing,
which are the most important criteria for banking application testing.
145
6.2.4.6 Code Reviews
Another good practice is performing code reviews. This code reviews are
more similar to unit testing. Therefore, the testing team or tester should be ready to
do unit testing for the code, once the code is ready for release. Tester must be ready
with his own unit test cases.
In current practice, a developer always do unit testing themselves. However,
a tester must also performed code reviews or unit testing, in order to confirm no error
or defects before the software can be released. Sometimes, the developers may
oversee some of the minus mistakes in the code, which a tester may find out.
Code reviews have two purposes, which are the first purpose is to make sure
that the code that is being produced has sufficient quality to be released. In other
words, it's it is to check whether the code should be promoted to the next step in the
process, because code reviews are very effective at finding errors of all types,
including those caused by poor structure, those that does not match business process,
and also those have simple omissions. Therefore, code reviews is good for code
quality checking.
Next, the second purpose of code review is as a teaching tool to help
developers learn when and how to apply techniques to improve code quality,
consistency, and maintainability.
The developers have the opportunity to learn
different and potentially better ways of coding after went through thoughtfully
evaluating code on a recurring basis.
However, code reviews often start off on the wrong foot because they are
perceived as an unnecessary step that has been forced upon the developers or, in
some cases, evidence that management does not trust the developers. Conversely,
neither of these perspectives is accurate.
effective way to minimize defects.
Code reviews are a proven to be an
146
Code reviews is about a time of great rejoicing, when a team of people who
write code get together and constructively review each other’s work in order to share,
learn, and teach better programming habits and techniques. Thus, spending time on
code that is just going to change or does not work is a massive waste of time.
The entrance requirements for a code reviews are:
•
Code should compile without errors or warnings according to coding
standard.
•
Code should be already be integrated with its parent branch.
•
Code should be fully unit tested.
•
Code should already pass the system tests where possible.
A good way of reducing the conflict in a code review is to create a checklist.
The list becomes a contract between the developers on the team. If everyone agrees
the items on the list then people have less ground for complaint when someone
points out things that do not meet the standards in the checklist. Secondly it helps
reviewers stay on track and keeps reviews short and sweet. This stops endless time
being devoted to reviews when reviewers keep looking until they find something.
Consequently, for SBS system, code reviews is one of effective process for
identifying defects as early as possible before the application release to the
production area. Most developers are assigned for different segments or modules of
the application, therefore, conducting this code reviews can improve the
understandability of the developers on the specific functions of this banking
application.
6.2.4.7 Test Execution and Bugs Reporting
After tester team has designed the test cases, test execution can start begin as
early as possible regarding to the available modules. When conducting tests, the
147
tester must perform each test as described in the test case, evaluate the test results,
escalate problems that arise until they are resolved, and document the test results.
In addition, the tester also needs to follow the written test cases carefully in
order to define the accurateness of assess results.
Besides that, the team members
need to know exactly which steps the tester took and the exact sequence in which
those steps were performed for reproducing a test to compare results over time.
Next, when a test is complete, the tester must evaluate the results against the
criteria in the test case to determine if the test passed or failed. Hence, not all tests
fail because there is a problem with the system implementation, but, the tests can fail
because there is a problem with the test itself, either the problem come from the test
simulation setup, or with the proposed design. Therefore, if one of these problems
causes a test to fail, the tester should take the appropriate measures to remedy it this
situation.
Normally, test execution is conducted based on levels of testing, which start
with Unit Testing, Integration Testing, System Testing and User Acceptance Testing.
For regression testing, it will be conducted if the application arrives at maintenance
phase, which is not covered in this report.
The test reports should be documented properly and the bugs have to be
reported to the developer after the testing is completed. All bugs or defects found
need to be documented in Defect Logs or any similar bugs reporting. Then, a test
report is prepared for further reviews between the developers and stakeholders.
CHAPTER 7
CONCLUSION AND RECOMMENDATION
7.1
Recap
Software testing process plays an important role in standard software
development phases.
This process consists of a few basic activities, such as
developing test plan and strategy, test design, test execution and evaluation of test
result. However, for a specific domain, like banking application, that might have
different characteristics from other software applications, a customized testing
process is required in order to ensure the correctness and completeness of the test
result.
Therefore, this project is proposing a Customized Software Testing Process
(CSTP) for the Shared Banking Services (SBS) system. This banking application
was developed by R@PIC, HeiTech.
Firstly, the study was conducted on several
testing model, which are V-Model, W-Model and Butterfly Model, in order to
identify some issues on initial testing activities that correspond to software
development phases. From this study, all of the testing model suggest that testing
tasks should be started earlier, which during business requirements gathering and
analysis. Therefore, a basis software development phases is clearly identified.
149
Besides that, the study continues for three testing methodologies, which are
HeiTech Testing Process, RUP Test Discipline and Systematic Test and Evaluation
Process (STEP), were compared based on criteria such as main activities, roles and
responsibilities, artifacts and level of testing. From this study, a summary on main
activities that correspond to software development phases that being rectify earlier
are visibly acknowledged.
Next, to propose a customized software testing process, a study on several
aspects of SBS system is needed. Thus, the study is conducted to identify its specific
modules and characteristics that different from other software applications. From the
result, four main testing characteristics that required for implementing testing on
SBS system are identified, which functionality test, performance test, security test
and automation test.
Finally, from all comparison result and summary, a proposed customized
software testing process for SBS is successfully produced based on all criteria that
being studied.
7.2
Project Outcome
Software testing is an important technique for the improvement and
measurement of a software system’s quality. Any approach to testing software faces
essential and accidental difficulties. Indeed, the construction of the needed test
programs is a major intellectual effort [57]. Hence, software testing is always focus
on the important part of system which is on core functionality that includes the parts
that are critical or popular, before looking at the other features, such as interfaces.
Thus, concentration on the application’s capabilities in common usage situations
before going on to unlikely situations is a proven strategy to produce good software
quality.
150
In addition, software testing normally involves the stages of test case
specification, test case generation, test execution, test adequacy evaluation, and
regression testing. Therefore, each of these stages in the software testing process
plays an important role in the production of programs that meet their intended
specification. Hence, while software testing is not a silver bullet that can guarantee
the production of high quality applications, theoretical and empirical investigations
have shown that the rigorous, consistent, and intelligent application of testing
techniques can improve software quality.
As a result, the implementation of software testing process for SBS system
has given a lot of knowledge and experiences about shared banking services
application and also improve skills on software testing activities. Moreover, this
industrial training exposed a good experience on working environment in IT-based
Company like HeiTech.
Furthermore, a bunch of knowledge on software testing process,
methodology, organization testing framework, activities and techniques also has been
gained from these five months industrial training. Therefore, a better approach of
software testing process for shared banking services application has been proposed in
this project. This approach is more likely suitable for testing banking application.
For this reason, a better approach for software testing process of SBS system
encompasses several stages such as testing start at requirements stage,
comprehensive test plan and test design, conduct design reviews and code reviews,
effective test cases preparation, performed test execution for each levels of testing,
prepared test reports and bugs reporting, and finally reworking on patches, which
requires regression testing as software then can be released to production area.
151
7.3
Future Study
Software testing process identifies what the test activities to carry out and
when, which what is the best time to accomplish those test activities. Hence, even
though testing may differs between organizations, there is still a testing process that
need to be performed has the basic phases such as test planning, test design, test
execution and test evaluation.
Furthermore, software testing has its own process or life cycle that intersects
with every stage of the software development life cycle.
Thus, the basic
requirements in software testing process are to control or deal with software testing
such as manual testing, automated testing and performance testing.
Consequently, several recommendations over these project that can be
include for further study are performed more details research on banking application
testing either for testing framework, techniques or strategies. Besides that, this
research project can be further carried out based on various software testing tools
available and then recommend the most suitable of performance testing tools for
banking application. In addition, there is also recommendation for future research on
security testing for banking application as this can help to improve the reliability of
the application.
7.4
Summary
As a summary, from this project, many knowledge and experience in
Software Engineering discipline especially in Software Testing is gained.
In
addition, from this industrial attachment, new experiences also gained in real
working environment. Hence, some expectations from this industrial practice are to
learn how to adapt with real working environment which need to follow their own
guideline and standard in software development approach, and also continuously
152
improving knowledge and skills in order to be successful in this software engineering
fields.
Additionally, as software testing is nowadays plays as an important phase or
steps in every software development project, a good practice and best approach of
software testing process can helps to improve the product quality. Moreover, there
are a lot of testing methodology, framework and approach with several of testing
techniques and types, that can be use and streamline based on the specific project
domain. Hence, in banking application domain, there are also several independent
testing organizations that already establish a good testing methodology, which helps
banks and any financial organizations to improve their services.
Consequently, the characteristics of a good software testing process can be
summarized as follows:
i.
Testing must be timely
Any system without a tight feedback loop is a fatally flawed
system. Test early, test often. Testing is that feedback loop for
the software development process. Therefore, it must begin in the
initial phase of the project, which is when objectives are being
defined and the scope of the requirements are first drafted. In this
way, testing is not perceived as a bottleneck operation.
ii.
Testing must be effective
The approach to test-case design must have rigor to it. Testing
should not rely solely on individual skills and experiences.
Instead, it should be based on a repeatable test process that
produces the same test cases for a given situation, regardless of
153
the tester involved. The test-case design approach must provide
high functional coverage of the requirements.
iii.
Testing must be efficient
Testing activities must be heavily automated to allow them to be
executed quickly. The test-case design approach should produce
the minimum number of test cases to reduce the amount of time
needed to execute tests, and to reduce the amount of time needed
to manage the tests.
iv.
Testing must be manageable
The test process must provide sufficient metrics to quantitatively
identify the status of testing at any time. The results of the test
effort must be predictable, for example, the outcome each time a
test is successfully executed must be the same.
154
REFERENCES
1.
HeiTech Padu Berhad. Retrieved on November 3, 2008, from
http://www.heitech.com.my/.
2.
Test Discipline in RUP. Retrieved on December 8, 2008, from
http://www.ibm.com/us/en/
3.
IBM Rational Method Composer 7.0.1. Method Composer Application. 2008
4.
Hetzel, William C. The Complete Guide to Software Testing. 2nd edition.
Wellesley, MA. : QED Information Sciences, Inc. 1988.
5.
Rick D. Craig and Stefan P. Jaskiel. Systematic Software Testing. USA:
STQE Publishing. 2002
6.
R@PIC. Shared Banking Services (SBS) Scope of Work, SOW Version 3.0.
2008.
7.
R@PIC. Shared Banking Services (SBS) User Functional Specification, UFS
Sign-off Version. 2008.
8.
Myers G. J. Revised and Updated by Tom Badgett, Todd M. Thomas with
Corey Sandler The Art of Software Testing. 2nd edition. New York, USA: John
Wiley and Sons. 2004
9.
Institute of Electrical and Electronic Engineers. IEEE Standard Glossary of
Software Engineering Terminology. New York. IEEE Std 610.12 – 1990.
1990
10.
Institute of Electrical and Electronic Engineers. IEEE Guide for Software
Verification and Validation Plans. New York. IEEE Std 1059 – 1993. 1993
155
11.
Bach, J. Exploratory Testing Explained. 2003. Retrieved on February 16,
2009, from http://www.satisfice.com
12.
John E. Bentley. Paper 141-30 Software Testing Fundamentals—Concepts,
Roles, and Terminology. Wachovia Bank, Charlotte NC.
13.
Institute of Electrical and Electronic Engineers. IEEE Standard for Software
Verification and Validation Plans. New York. IEEE Std 1012 – 1986. 1986
14.
Schmidt, Michael E.C. Implementing the IEEE Software Engineering
Standards. USA: Sams Publishing. 2000
15.
P.K. Davis, Generalizing Concepts of Methods of Verification, Validation and
Accreditation (VV&A) for Military Simulations. RAND Report. 1992
16.
Ron Patton. Software Testing. Indiana, USA : Sams Publishing. 2nd edition.
2006
17.
Beizer, B. Software Testing Techniques. 2nd edition. USA: Van Nostrand
Reinhold. 1990
18.
M. Harrold. Testing: A Roadmap. International Conference on Software
Engineering, (ICSE 00). ACM Press, New York. 2000
19.
SEI. Software Quality Measurement: A Framework for Counting Problems
Retrieved on November 3, 2008, from
www.sei.cmu.edu/pub/documents/92.reports/pdf/tr22.92.pdf
20.
Kaner C. Software Testing as a Social Science. IFIP Working Group10.4,
Florida Institute of Technology, Siena Italy. 2004
21.
William E. Perry. Effective Methods for Software Testing. 2nd edition. John
Wiley & Sons: USA. 2000
22.
Scott Barber & Karen N. Johnson. A Close Look at the Five Schools of
Software Testing. 2009
23.
Bret Pettichord. Schools of Software Testing. 2007
156
24.
Rob Pirozzi. Introduction to Software Testing. LogiGear Corporation.
Retrieved on November 19, 2008, from
http://www.logigear.com/newsletter/introduction_to_software_testing.asp
25.
Goutam Kumar Saha. Understanding Software Testing Concepts
ACM Ubiquity Vol. 9, Issue6, 2008 February 12, 2008 - February 18, 2008
26.
Peter Sestoft. Systematic Software Testin. IT University of Copenhagen,
Denmark. Version 2. 2008
27.
Lee Copeland. A Practitioner's Guide to Software Test Design, USA: STQE
Publishing. 2004
28.
Software Testing Guide Book. Software Testing Research Lab, Retrieved on
January 14, 2009, from http://www.softrel.org/Modeling.html
29.
Perry, William E. How to Test Software Packages : A step- by- step Guide to
Assuring They Do What You Want. New York, USA: John Wiley & Sons.
1986
30.
Kaner C. The Ongoing Revolution in Software Testing. Software Test &
Performance Conference. December 8. 2004
31.
John Watkins. Testing IT: An Off-the-Shelf Software Testing Process.
Cambridge, United Kingdom: Cambridge University Press. 2001
32.
Wasif Afzal. Metrics in Software Test Planning and Test Design Processes.
Msc. Thesis. Sweden. 2007
33.
Kaner C. Fundamental of Software Testing. Florida Tech, Colloquium
Presentation, Butler University. 2003
34.
Srinivasan D., Gopalaswamy R. Software Testing: Principles and Practices.
New Delhi, India: Pearson Education. 2006
35.
Test Development Model. Software Testing Research Lab, Retrieved on
January 21, 2009 from http://www.softrel.org/Modeling.html
157
36.
PMOC. V-Model Testing – Process Model Configuration Using SVG. 2003.
Retrieved on March 3, 2009 from http://www.soberit.hut.fi/T-76.115/0203/palautukset/groups/pmoc/de/vmodel.pdf
37.
Paul Gerrard. Managing Projects with Intelligence. Gerrard Consulting
Limited. 2004
38.
Software Development Models. Retrieved on January 22, 2009, from
http://sqa.fyicenter.com/FAQ/Software-DevelopmentModels/Software_Development_Models_W_Model.html
39.
Spillner A. The W-MODEL – Strengthening the Bond Between Development
and Test. University of Applied Sciences Bremen. Germany. 2002
40.
James Christie. V Model Article. Retrieved on January 26, 2009, from
http://www.clarotesting.com/page11.htm
41.
Test methodology. Retrieved on January 26, 2009, from
http://www.kynetia.com/quality/methodology.html
42.
Test Development Model. Software Testing Research Lab, Retrieved on
January 26, 2009, from http://www.softrel.org/Modeling.html
43.
Student Accountant. Developing Systems. The V model relevant to
Professional Scheme Paper 2. 2006. Retrieved on April 15, 2009, from
http://www.accaglobal.com/pubs/students/publications/student_accountant/
44.
HeiTech. Application Development Information System (ADVISE). 2008
45.
Fantinato, M. Applying Extended Finite State Machines in Software Testing.
2003
46.
Rational Software Team. RUP: Best Practices for Software Development
Team. Rational Software White Paper, 1998. Retrieved on December 6, 2008,
from http://www.rational.com
47.
Bank Negara Malaysia. Basic Banking Services. Malaysia: Info Guide –
158
Banking Info. 2005. Retrieved on March 10, 2009, from
http://www.bankinginfo.com.my
48.
Morton Stephen D. The Butterfly Model for Test Development Applied
Dynamics International. 2001
49.
Li Jin-Hua, Li Qiong, Li Jing. The W-Model for Testing Software Product
Lines. International Symposioum on Computer Science and Computional
Technology. 2008
50.
Per Runeson, Greberg P. Extreme Programming and Rational Unified Process
– Contrasts or Synonyms? Lund University, Sweden. 2004.
51.
Banking Application. Retrieved on March 18, 2009, from
http://findarticles.com/p/articles/mi_m4153/is_n1_v48/ai_10380967
52.
Infotech Banking Services. Retrieved on March 18, 2009, from http://www.3iinfotech.com/content/services/software_testing.aspx
53.
Third Eye. Software Testing. Retrieved on February 25, 2009, from
http://www.stcthirdeye.com/banking.htm
54.
Thinksoft Global. Software Testing. Retrieved on February 25, 2009, from
http://www.thinksoftglobal.com/business_areas/core_banking/
55.
Rodina. Requirements Validation. Retrieved on April 20, 2009, from
http://fsktm.um.edu.my/~rodina/WKES3202_LEC4_Rodina.ppt
56.
Kaner. C. What Is a Good Test Case? Florida Institute of Technology. STAR
East. 2003
159
Appendix A
(Project Plan & Gantt Chart)
160
Project Plan – IA Gantt Chart
161
Project Plan – IA Gantt Chart
162
Appendix B
(Industrial Attachment Progress Report)
163
Appendix C
(Proposed Customized Software Testing Process)
164
Proposed Customized Software Testing Process