SOFTWARE TESTING CHALLENGES IN PRACTICE IN
THE CONTEXT OF WEB BASED APPLICATION
by
Nisha Lamichhane
Master of Technology in Information Technology
Kathmandu University, 2014
A Thesis
Submitted to the Graduate Faculty
Of the
Kathmandu University
In Partial fulfillment of the requirements
For the degree of
Master of Technology in Information Technology
August 2014
i
DECLARATION OF ORIGINALITY
Being a student, I understand that I have an ethical and moral obligation to ensure that the
dissertation that I have submitted to the Kathmandu University is my own, original and
free of plagiarism. I exercised caution throughout the writing process, making sure all the
sources are acknowledged properly. I revised the thesis thoroughly with the references
that I had taken during the time of writing the thesis. Hence, I am satisfied that the work
which I am submitting to the Kathmandu University, Department of Computer
Engineering has all the sources properly acknowledged. I declare that the submitted work
is my own and is original.
__________________________
Nisha Lamichhane
Candidate
University Registration Number: 011141-09
i
PERMISSION
Title: Software testing challenges in practice in the context of web based application
Department: Computer Science and Engineering
Degree: Master of Technology in Information Technology
In presenting this thesis in partial fulfillment of the requirement for a graduate degree
from Kathmandu University, I agree that the library of this University shall make it freely
available for inspection. I further agree that permission for extensive copying for
scholarly purposes may be granted by the supervisors who supervised my thesis work or
in his absence, by the head of the department or the dean of School of Engineering. It is
understood that any copying or publication or other use of this thesis or part thereof for
financial gain shall not be allowed without my written permission. It is also understood
that due recognition shall be given to me and to the Kathmandu University in any
scholarly use which may be made of any material in my thesis.
_________________________
Nisha Lamichhane
Date: August 03, 2014
ii
ACKNOWLEDGEMENT
I am heartily thankful to my supervisor, Mr. Pankaj Raj Dawadi, Department of
Computer Science & Engineering, who always encouraged and inspired me and provided
more than enough guidance and support from the initial to the final level enabled as to
develop an understanding of the subject. Likewise, I would like to express my sincere
gratitude to Mr. Suresh Kumar Regmi, External Evaluator and Dr. Manish Pokharel
(Associate Professor), Head of Department, Department of Computer Science &
Engineering for their valuable guidance for the finalization of the thesis.
I would also like to thank my survey respondents for making the survey possible. I
express my gratitude to my husband Er. Apil Koirala for his continuous encouragement
and support provided during the study.
I am also very much thankful to my father Mr. Hem Raj Lamichhane for his technical
support while setting the objectives and methodology parts of the study and regular
guidance for finalizing the thesis. I always remember the support extended by my mother
Mrs. Pushpa Lamichhane during my study. It would have not been possible for me to
complete the study if my little baby had not taken care by her.
Finally, I offer my earnest regards and blessings to all who directly or indirectly
supported me in any respect during the completion of the thesis.
Nisha Lamichhane
August 2014
iii
ABSTRACT
The study entitled “Software Testing Challenges in Practice in the Context of Web
Based Application” has focused importance of the software testing for quality product
and the present status of its application in practice. The main objectives of the study are
to identify the challenges of software quality assurance in web based application and
provide solution and recommendation to mitigate those challenges.
With a view to collect the necessary information the qualitative research approach was
used. The interview was carried out with the responsible persons of the IT offices to get
primary information based on the semi structured questionnaire. The secondary data was
gathered through the related articles, white paper, PDF files, and previous
studies/researches, variety of websites. On line response from the respondents were
collected through Google docs as well.
The software development company should have quality assurance department and
necessary personnel (Quality Analysts) to ensure the quality product by reducing errors.
Required training for making ‘Test Plan’ and its usage should be provided to quality
analyst that testing work much more efficiently and effectively. Quality analyst should be
involved in daily meeting with development team. Likewise, quality analyst should have
database and programming knowledge. In order to identify the similarity/gaps and better
performance, comparative study can be carried out between manual testing and
automated software testing tools as future work.
iv
Table of Contents
DECLARATION OF ORIGINALITY ............................................................................ i
PERMISSION ................................................................................................................... ii
ACKNOWLEDGEMENT ............................................................................................... iii
ABSTRACT ...................................................................................................................... iv
LIST OF FIGURES ......................................................................................................... ix
LIST OF TABLES ............................................................................................................ x
ABBREVIATION ........................................................................................................... xii
CHAPTER –I: INTRODUCTION .................................................................................. 1
1.1
General background ............................................................................................. 1
1.1.1
Software ........................................................................................................ 1
1.1.2
Software quality ............................................................................................ 1
1.1.3
Software errors .............................................................................................. 2
1.1.4
Software testing ............................................................................................ 3
1.1.5
Software testing types ................................................................................... 3
1.1.6
Software quality assurance and quality control ............................................ 5
1.1.7
Software testing technique/process ............................................................... 6
1.1.8
Software testing tools .................................................................................... 6
1.2
Statement of the problem ..................................................................................... 6
1.3
Objectives of the study ......................................................................................... 8
1.4
Significance of the study ...................................................................................... 8
1.5
Limitations of the study........................................................................................ 9
v
1.6
Organization of the study ..................................................................................... 9
CHAPTER–II: LITERATURE REVIEW ................................................................... 10
2.1
Background ........................................................................................................ 10
2.2
Software Quality Assurance ............................................................................... 12
2.3
Test Plan ............................................................................................................. 14
2.4
Automated Software Testing Tools.................................................................... 15
2.5
Development Methodologies ............................................................................. 16
2.6
Challenges of software testing ........................................................................... 18
2.7
Challenges of Quality Analyst ........................................................................... 21
2.8
Validation and Verification ................................................................................ 22
2.9
Team efforts for quality product (Case scenario of EBpearls)........................... 23
2.10
Quality Assurance Procedure in web based application .................................... 24
CHAPTER-III: RESEARCH METHODOLOGY ...................................................... 26
3.1
Introduction ........................................................................................................ 26
3.2
Methods of data collection ................................................................................. 26
3.3
Source of primary data ....................................................................................... 26
3.4
Selection of the respondents............................................................................... 27
3.5
Methods of data analysis .................................................................................... 27
CHAPTER-IV: FINDINGS OF THE STUDY ............................................................. 28
4.1
Institutional set up and tools used ...................................................................... 28
4.1.1
Establishment of quality assurance department .......................................... 28
4.1.2
Testing process used ................................................................................... 28
vi
4.1.3
Use of Test Plan .......................................................................................... 29
4.1.4
Software testing tools .................................................................................. 30
4.1.5
Software quality assurance in agile methodologies .................................... 32
4.2
Perception of respondents .................................................................................. 32
4.2.1
Chances of error/bug with using “Test Plan” ............................................. 32
4.2.2
Chances of error/bug without using “Test Plan” ........................................ 34
4.3
Data Analysis with correlation ........................................................................... 35
4.4
Best way of software quality assurance ............................................................. 37
4.5
Problems faced by software quality analyst ....................................................... 40
4.6
Existing practice in Selected Organization ........................................................ 41
4.6.1
IT Himalaya ................................................................................................ 41
4.6.2
Auxfin Nepal ............................................................................................... 42
4.6.3
F1 Soft International ................................................................................... 42
4.6.4
Braindigit .................................................................................................... 44
4.7
Test Plan document ............................................................................................ 45
4.8
“Test Plan” (Magento E-commerce Application) .............................................. 46
4.9
“Test Plan” (Word press CMS Application) ...................................................... 57
CHAPTER-V: CONCLUSIONS AND RECOMMENDATIONS.............................. 59
5.1
Conclusions ........................................................................................................ 59
5.2
Recommendations/Future works ........................................................................ 60
References ........................................................................................................................ 61
Websites ........................................................................................................................... 65
vii
Annexes: ........................................................................................................................... 66
Annex 1: Questionnaires ............................................................................................... 66
Annex 2: List of respondents and organizations ........................................................... 71
viii
LIST OF FIGURES
Figure 1: A collaborative approach of Quality Analyst with development team ............. 23
Figure 2: The systematic procedure for quality assurance in web based application ....... 25
Figure 3: Users of Test Plan.............................................................................................. 29
Figure 4: Users of Test Plan.............................................................................................. 30
Figure 5: Automated software testing tools users ............................................................. 31
Figure 6: Causes of not using automated software testing tools ....................................... 31
Figure 7: Automated software testing tools users ............................................................. 32
ix
LIST OF TABLES
Table 1: Chances of error/bug with using “Test Plan” ..................................................... 33
Table 2: Chances of error/bug without using “Test Plan” ................................................ 35
Table 3: Correlation between dependent and independent variables................................ 36
Table 4: Quality assurance and involvement of quality analyst in SCRUM .................... 36
Table 5: Homepage of Carisma ....................................................................................... 46
Table 6: Home Page> Header-on all pages of website ..................................................... 47
Table 7: Homepage>On all pages of website> call to action .......................................... 48
Table 8: Homepage>Slide................................................................................................. 48
Table 9: Homepage> Products highlight .......................................................................... 48
Table 10: Newsletter Subscription .................................................................................... 49
Table 11: Footer ................................................................................................................ 49
Table 12: Listing Page (important note: show all results in one page: infinite scroll) ..... 49
Table 13: Creators Page (important note: show all results in one page: infinite scroll) ... 50
Table 14: Product detail Page ........................................................................................... 50
Table 15: Product detail Page popup ................................................................................ 51
Table 16: Content page ..................................................................................................... 52
Table 17: Contact page ..................................................................................................... 52
Table 18: My Account page .............................................................................................. 52
Table 19: Cart page ........................................................................................................... 53
Table 20: Order Process .................................................................................................... 54
Table 21: Dashboard ......................................................................................................... 54
x
Table 22: Payment method ............................................................................................... 55
Table 23: Transactional emails ......................................................................................... 55
Table 24: CMS Pages........................................................................................................ 56
Table 25: Types of product ............................................................................................... 56
Table 26: Homepage of Emile Garcin .............................................................................. 57
Table 27: For Sales ........................................................................................................... 58
Table 28: Order Details ..................................................................................................... 58
xi
ABBREVIATION
CD
= Compact Disk
CMS
= Content Management System
DB
= Data Base
HP QTP
= HP Quick Test Professional
IE
= Internal Explorer
IEEE
= Institute of Electrical and Electronics Engineers
IT
= Information technology
PSD
= Photo Shop Design
QA
= Quality Assurance
QAn
=Quality Analyst
SDLC
= Software Development Life Cycle
SQA
=Software Quality Assurance
SQL
= Structured Query Language
STAF
= Software Testing Automation Framework
TDD
=Test Driven Development
XP
=Extreme Programming
RUP
=Rational Unified Process
SVN
=Subversion
PDF
=Portable Document Format
HTTP
=Hypertext Transfer Protocol
IIS
=Internet Information Service
URL
=Uniform Resource Locator
xii
CHAPTER –I: INTRODUCTION
1.1 General background
Software
Software is a computer instruction or data that enable the user to interact with a
1.1.1
computer. Anything that can be stored electronically in order to instruct hardware to
perform the task for which it is designed is termed as Software. Software is complex and
invisible in the sense that it is stored in hard disk, Compact Disk (CD), etc. whereas
industrial products are visible. Due to invisibility and complexity, software testing should
be performed and quality software will be delivered to client in this competitive world
(http://en.wikipedia.org/wiki/Software).
1.1.2 Software quality
The software quality means maintaining the correctness of software, removing the bugs,
making it efficiency and complete. Software quality can be assessed with these points
(http://en.wikipedia.org/wiki/Software_quality).
Functionality: There is functionality for any type of application. For instance, an
ecommerce site should have the features of product listing page, cart page, order process
flow and delivery. For any content management system (CMS) site should have the
functions for adding, updating, deleting the items from the backend if the system is
dynamic.
Reliability: If the system is functionally acceptable then system should be able to
function or operate continuously.
1
Efficiency: Efficiency means how to respond the system as per requested. It consists of
resources like memory, network, hardware, etc. If user request for some functionality or
features, system should respond fast for that features. If it responds late then it is to be
considered as not efficiency.
Usability: Usability means user should be able to run the software easily with the
existing functionality. The information like ‘HELP’ section or ‘USER GUIDE’ should be
available for learning to use the system properly by new user.
Maintainability: Maintainability mainly refers to the code complexity of the software. If
any fault occurs in the system, it needs to be tested and fixed it. So it can be referred to as
maintainability.
Portability: Software always needs to be run or adapt in new environment. If it is
developed for several computing platform or different operating systems, then it refers to
as portability of the system.
Quality software should be provided to clients in this competitive market. If low quality
software provide to client then organization will not survive long last. So for quality
software for qualitative market, all the errors should be removed before releasing the
software.
1.1.3 Software errors
The software error means unexpected thing that can occur due to source code error,
procedure error, documentation error, software data error, etc. Human being can make
mistakes so; the error is made by human such as System Analyst, Programmer, Designer,
Quality Analyst, Project Managers etc (http://en.wikipedia.org/wiki/Software_bug).
2
The main cause of software errors are:

Errors occur while gathering requirement definition or specification.

Misunderstanding between client and developer while communication may be due
to geographical and cultural diversity

Build feature in software which is orally said not in written form due to which it
may cause to forget the original feature and twist the software as per requirements.

Design errors

Logical errors

Incomplete Test Plan and Test Case made by Quality Analyst
1.1.4 Software testing
The software testing is the process of verifying requirements for the design and
development as per provided by clients that must satisfy the clients ultimately.
1.1.5 Software testing types
There are many software testing types but some of them are explained as follows
(http://www.softwaretestingsoftware.com/all-types-of-software-testing/).
Black box testing: It is a method of testing based on requirement and functionality. So it
is also known as functionality testing. It fulfills the question of ‘what the software does?’
There is no required programming language to understand for black box testing.
White box testing: Programming knowledge is required for white box testing. It is also
known as structural testing, glass box testing. Programmer needs to test internal structure
or coding level of the software. Basically it is tested by programmer itself or by QAn who
have the knowledge of software programming.
3
Incremental integration testing: It is performed by programmer or Testers. When the
new functionality or features added in the software, testing should be done or performed.
This is called bottom up approach.
System testing: Testing overall system as per the requirements. Whole system testing
should be carried out.
Regression testing: Need to test the overall of the software after changes/modification in
one module or functionality because changes in one module can affect the other parts of
the software so it needs to be verified.
Acceptance testing: It is conducted to verify the software whether the software is
according to requirement specification or not. If it is also done by customer or user, that
is called user acceptance test.
Performance testing: Verify the system under heavy workload how it performs. So it is
called performance testing.
Usability testing: Test the system in terms of user friendliness, user friendly
environment. That’s why new user also should be able to easily access and run the system
without help of others.
Security testing: Test whether the developed system is secure or not. For instance,
username and password should be strong of admin panel and others where username and
password will be used.
Compatibility testing: Verify the software on particular hardware, software, operating
system, browsers because some designing parts are not compatible on all these
4
environments. So need to test on different platforms like IE, Mozilla Firefox, Google
Chrome, MAC Safari, IPAD, iTouch, Tablet and so on.
Alpha testing: Alpha test is performed before the software is released. It is conducted by
team of users in order to find bugs and fix the bugs. It is held before the software goes for
beta release.
Beta testing: Beta testing is the final version of testing. After then it is handed over to
clients/end users. Successful beta test refers to automatically user acceptance test.
1.1.6 Software quality assurance and quality control
The software quality assurance is the activities or monitoring software engineering
process. Quality analysts are responsible for monitoring the software according to
specification given by client. In software, errors or defects harm the quality so quality
analyst (QAn) performs variety of activities to minimize or remove the error throughout
software development process. Quality assurance (QA) is proactive process in the
development life cycle because it prevents the defects, errors in the process of
development. There is systematic procedure, approach or documentation to test the
software in order to maintain the quality.
On the other hand, software quality control is also related to software quality assurance in
the software development life cycle. So, quality control is the reactive process in the
Software Development Life Cycle (SDLC) because it identifies defects after product is
developed. There is technique or observation technique to maintain the quality of a
product or service. Quality control is basically concerned with inspecting product or
services that means control the products and services whereas software quality assurance
5
is basically concerned with process of product or services that means the control of
processes.
1.1.7 Software testing technique/process
There is manual and automated software testing process. Manual testing means testing
the software manually according to requirement specification provided by client. On the
other hand, automated software testing refers to testing the software through automated
testing tools such as selenium, apache JMeter, Load Runner and so on. Automated
software testing tools is used to test software performance as well as functional,
regression test in effective and efficient way with low cost, less manpower within time
period. But almost all testing tools are expensive only some testing tools are open source.
1.1.8
1.
Software testing tools
Telerik teststudio is for functional, load, performance and mobile app testing.
2.
HP Quick Test Professional is for functional and regression test for software.
3.
Selenium is used for record and playback tool. There is no need to understand
test scripting language and used to debug and set breakpoints.
4.
Load Runner is used to test performance of the system.
5.
IBM Rational functional tester is a software testing tools for regression testing. It
is normally used by QAn.
1.2
Statement of the problem
There are many software failures without testing software before release.

In most of the IT organization doesn’t have software quality assurance (SQA) due
to which developer and designer need to check and verify the contents, design as
6
per clients requirements but they don’t have enough time to test all these things.
That’s why software becomes of low quality.

In case of presenting QAn in Information Technology (IT) organization, there is
lots of task to be tested for software. QAn also have limited time to test the
particular software. Due to limited time to test all the development part, designing
part, and keep track of test, if QAn make Test Plan and Test Case and test the
software according to Test Plan and Test Case, it will be more effective because
documentation speaks itself if the particular things are forget to test.

The meeting is conducted with designer and developer by project manager.
Quality analyst has not been involved in the SCRUM resulting that some
important information may omit to inform to quality analyst and ultimately omit
to test.

On the other hand, manual testing consumes more time resulting high cost and
high manpower. Furthermore, Automated software testing tools are already
available in the market however tester do not prefer to use them due to lack of
knowledge or experience and they are not affordable to medium level software
company since cost is very high.
In this regards, this thesis has recommended solutions for the following research
questions.
1. How will ‘Test Plan’ improve software quality assurance in web based
application?
7
2. How do SCRUM will benefit if quality analyst also involved in it while software
development?
1.3
Objectives of the study
The general objective of the dissertation is to promote the software quality assurance in
web based application. However specific objectives are as follows:
1. To identify the challenges of software quality assurance in web based application, and
2. To provide solution to mitigate those challenges in web based application
1.4
Significance of the study
As the software company grows these days, it is very competitive to deliver the quality
software to clients among the companies. To meet the client’s requirement and satisfy the
client, any organization should be able to provide quality software to them. For that,
software testing should be done throughout the development of entire project.
There are two types of testing strategies. They are Big Bang Testing and Incremental
testing. In Big Bang Testing; software will be tested once it is completed. In Incremental
Testing, software modules or units will be tested before it is completed as a whole. Then
testing will be performed for integrated software while some units or modules are
completed. After that whole system will be tested once it is done. While performing
manual test there should make Test Plan and Test Cases to test the software and
effectively help to produce quality output.
8
The study will focus to reduce the challenges to be occurred while testing of the software.
Broadly the study will find out the challenges and recommend the solution of mitigating
the challenges.
1.5
Limitations of the study
The research has been carried out with the purpose of delivering good quality software to
clients. Software testing has two dimensions such as manual testing and automated
testing. However, this study focuses only the manual testing with Test Plan. Automated
testing has not been focused in depth.
Software testing in web based application has been mainly focused rather than other
application such as desktop application.
Due to time constraints and unavailability of the exact number of IT organizations,
limited number of organizations (17) was selected as sample for primary data collection
which may be small in size that can't represent the universe.
1.6
Organization of the study
This study is divided into different five chapters. The first chapter covers the introduction
of the study. Introduction chapter includes general background, statement of the problem,
objective of the study, significance of the study, limitation of the study and organization
of the study. The second chapter presents the review of the literature. Likewise the third
chapter describes the methodology which was applied during the study time. The fourth
chapter deals with major findings of the study and finally, the fifth chapter is about the
conclusion and recommendations.
9
CHAPTER–II: LITERATURE REVIEW
2.1
Background
Software testing is one of the parts of software development life cycle. Requirement
analysis, design, coding, testing, deploy and maintenance are the sequential phases of
SDLC. Software quality assurance plays vital role to deliver quality software to clients.
In this competitive market, quality assurance becomes more popular. Without testing by
QAn if there is QA department, software also will not be released as beta. Software
development functionality, performance, design all these things must be tested before the
software upload in live server.
In today’s world, outsourcing is most popular but geographical and cultural diversity
must be managed for that. There are many factors involved to make quality of software
that are clear requirements, good design, and good development. Likewise software
testing and quality assurance is the one of the areas in software development life cycle to
make the quality of the software.
In order to make quality of the software and effective testing, it needs to be
understandable of how to avoid expensive maintenance. On the regards of customer
satisfaction, faults, defects and failure are found before delivering the product. Similarly,
understand the test techniques for better test and structured and systematic testing for fast
test in order to find many bugs in less time.
In this competitive market, most software companies have the problems of developing
good quality software because people are not motivated, not enough skills, and
10
insufficient tools. Developing this kind of bad quality software makes expensive
maintenance.
Software testing is not just only for finding bugs. It also demonstrates that software is
working properly according to requirements. It is very important the measurement of
performance, effort to show improvement (Eldh, n.d.).
Functional test is like black box test. Multiplatform testing means test in different
platforms. Similarly, internal beta test is essential for us to gain good coverage.
Automated text execution tested regression test quickly, obtain higher coverage with
number of test cases. Under foundational, user scenarios, usability testing, requirements
for test planning. Test members should close collaborate with developers for improving
test cases. Code coverage measures the element of code (Chillarege, n.d.).
Software testing metrics provide testing effectiveness and efficiency. There are several
testing metrics. They are Organization, Project, Process, and Product. Under product,
there are static and dynamic metrics. Number of testers working on a project is static
metric and number of defects to be fixed as dynamic metric (Verma, 2011). Software test
is an activity of system and component test which is under specified condition. Two test
approaches are black box testing and white box testing (Nelson et al., 2013).
The key role of software measurement is increasing the effectiveness of testing process
(Farooq et al., 2011). Three defect detection techniques are black box, white box and
abstraction. Defect is detected by sharp eyes (Kamsties et al., 1995).
11
2.2
Software Quality Assurance
Software quality assurance plays important role in software development life cycle.
Quality assurance, quality control is fall under quality management. Quality management
is important for organization. In software industry, quality is defined in many ways.
Some focus on error free functionality, some focus on customer satisfaction.
Software quality is defined, according to the Institute of Electrical and Electronics
Engineers (IEEE), as "the degree to which software meets customer or user needs or
expectations (Grater, 2005).
Quality assurance means assuring defect less software for customer satisfaction.
Similarly testing is defined as the process of operating a system or component under
specified conditions, observing or recording the results, and making an evaluation of
some aspect of the system or component. In this complex business requirement, software
companies are highly relying on software testing. If there is QA department, software
will not be released as beta until verified by quality analyst. Software functionality,
performance, security all these things must be tested before the software will be uploaded
in live server.
When new features added in the software or product, it needs to be tested by tester by
manually. It consumes more time. So there are different automation tools available to test
the system quickly. It will take low cost. Similarly, the term “Web Based Applications”
actually means Internet based software applications. Automated software testing help to
increase the software quality assurance in web based application.
12
For any type of business product, firstly there should be clear requirement. To understand
the requirement clearly business analyst must be there or s/he can be program manager.
Software requirements are the first step of software development life cycle. After
requirement is clear then it should be analyzed before starting design and development.
Then after design and development should be performed and quality analyst make the
Test Plan to test the software. After finished to develop, then testing phase will occur and
after testing, implementation of software will be performed. While testing software, if
automated testing tools is used then it will reduce the development cost and increase the
quality of the software. But in order to use automated tools there should be strong
knowledge of human resource.
In the absence of testing, many bugs and errors are present in the software after then it
will obstruct to the overall performance of the software and overall decrease the quality
of software. Quality is more important aspect to any sort of application. Testing and
software quality assurance play critical roles in the successful completion of any software
development.
The goal of software quality assurance is to improve the software quality and
productivity. The three tier architecture is used in web based applications. The three tiers
in three-tier architecture are presentation tier, logic tier and data tier. In the Presentation
tier, User writes the URL in web browser and this is sent to web server using Hypertext
Transfer Protocol (HTTP). Likewise request will be processed by server script such as
PHP and Web Server i.e Apache or Internet Information Service (IIS) in a middle tier or
13
logic tier. Similarly, SQL language is used to interact with the data. Information can be
stored and retrieved from the database i.e. data tier. After that information is passed back
to the logic tier for processing and then user. There is server side, client side and server
side interaction client side model to achieve or enhance software quality. The server side
factors are testability, flexibility, reliability, maintainability, reusability. Similarly, client
side factors are portability, usability, privacy, standard design and correctness. In server
side interaction client side, these are the factors such as security, efficiency, integrity,
ethics (El-rayyes, 2012). Software testing is the significant part of software engineering
and it cuts the maintenance and overall cost (Shivkumar, 2012). The purpose of testing is
to reduce the risk of the software and to show the software works smoothly (Black,
2007).
2.3
Test Plan
The testing techniques are control flow, data flow, random, mutual testing (Juristo et al.,
n.d.). Effecting testing by making Test Plan which includes set the test objectives
develops a test matrix, define test administration, write Test Plan (Perry, 2006). In order
to test the software system is how to get the test cases that is the major problem (Vegas,
n.d.). Bug is removed before delivering a product for reliability of software product
(Pizza et al., n.d.). The cost of random testing is less than the cost of test case generation
(Crowell, 2009).
Testing is done with specifying priority in the task to test the task effectively and
efficiently. Every features need to perform at least once by one valid input case. It is
impossible to ensure that bug is removed completely in the system (Quadri, 2010).
14
If more attention is paid to testing in software development process, results that mitigate
the failures after implementation (Tawileh, n.d.).
The success of the test is the selection of effective and efficient test cases (Juristo, 2004).
2.4
Automated Software Testing Tools
Automated testing tools help to reduce testing time, cost effective and it needs less
human resources. Software test automation tools are very difficult area even if it saves a
lot of money if it is used properly (Eldh, n.d.).
The Software Testing Automation Framework (STAF) is a multiplatform, multi-language
approach for reusing the features to be tested. The major benefit of using software testing
tools are improvement the activities of the software quality assurance (ibid).
In software quality assurance engineering, software testing and software quality
assurance is similar meaning. Automated testing helps the quality assurance team to work
faster and do motivation on testing. In case of heavy work load testers might omit to test
some part in particular software project or product due to time limit. So, Test automation
is more useful while regression testing is needed (Smart bear software, n.d.).
Testing is one of the most important phases in quality assurance especially in the
software development and design stages. When one feature is finished to develop, it
should be tested and if error is occurring then it needs to be fixed. Otherwise, more bugs
or mistakes appear in the site and system may also crash. JUnit is a tool used for unit
15
testing tool, Security testing tools used basically by QAn. LoadTracer is Performance
testing tool but need to buy. SQL DBValidator is database testing tools (Shah, 2009).
The goals of software testing are detecting defect and increase reliability (Farooq, 2012).
Software testing can be performed manually and automatically. There is error occurs
while testing manually because of tiring situation of human being.
For automated software testing less human effort is required. Unit testing is the basic
level of testing, integration testing is the integration of modules testing, system testing is
the functionality testing as well as nonfunctional requirement of testing such as
performance, reliability etc. (Rafi et. al, 2011).
2.5
Development Methodologies
Agile development methodology is iterative and incremental development, collaboration
with cross functional team. Scrum is an agile methodology; it refers to 10 to 15 minutes
meeting for project to know the status of project. If designer, developer faced problems
they can share those problems while sitting for meeting due to which couldn’t hinder to
do the further or rest of the work. The main benefit of scrum is to finish the project on
time. So, cost saving and faster development is the key features of the agile methodology.
There can be performing two types of testing. One is if certain task is completed then
starts testing. And another one is if all tasks is completed then start testing. In the same
way, agile methodologies follow early testing rather than testing in the last. In agile,
developer also should perform testing is emphasized because developers have also
responsibility to make quality software in the software development life cycle. In smaller
16
companies, dedicated QAn is not hired for testing and don’t have any plan for testing
also. Test is performed by developer himself/herself and some parts test by clients itself.
If the tester is assigned for only one project at a time, then it will be qualitative test. On
the other hand, agile development focuses with client collaboration and communication
throughout the development of the project. When any confusion occurs in the early phase
of software i.e. requirement analysis, design or development part, Project team ask the
client about confusion at the early phase of software development. Similarly, when
applying agile methods to develop the product, if developer is not in right track then it
will help to find out the problem and can be solved the right time that means agile
methods stop to go on wrong track because all the stakeholders have meeting time to time
about the project.
In software development, agile methodology and conventional development are two
aspect of development. Agile focuses on team collaboration, quality, and relationship
between developer and customer and it is people oriented rather than process oriented.
Different agile methodologies are XP (Extreme programming), SCRUM, and Crystal.
Under XP, refactoring means restructured the program of removing duplication,
simplicity, adding flexibility without changing in functionality. Metaphor is a basic
understanding of the system architecture; develop the system according to architecture.
Pair programming is two persons working in the same screen, mouse and keyboard due to
which error can be minimized. Scrum has product backlog, sprints, sprint planning
meeting, sprint backlog, daily scrum. Daily scrum is 15 to 20 minutes meeting daily to
17
understand the project flow. In agile, testing and development goes parallel. Finding bugs
and fixing bugs earlier saves time, cost, resources and maximization to quality (Malik et
al., 2009).
Quality assurance factor in agility are compatibility, cost effectiveness, correctness, ease
of use, extendibility, performance, portability, integrity, reusability, timeliness, validation
and verification. These are also called nonfunctional attribute. Agile methodologies such
as extreme programming, scrum, crystal methodologies, feature driven development,
Rational Unified Process (RUP), dynamic system development methodology, open
source software development, and agile modeling or lean development (Sirshar et al.,
2012).
Software testing technique is not just only for effectiveness and efficiency but also for
reliability. Progress in system level testing is measurement of reliability (Farooq et.al,
2011).
2.6
Challenges of software testing
The main challenge of not delivering quality software is geographically dispersed
organization, limited resources, soaring turnover rates for skilled engineers. The success
of any product is to deliver it in right time within budget. The quality of the software is
determined by the quality of test process.
The challenges associated with functional testing life cycle:
1. Test requirement gathering: The challenge is to define clear and complete test
requirements and manage the changes requirements. The best practices are
18
requirement need to be arranged by project management team and project feature
need to be arranged from development team.
2. Test Plan: The challenge in case of making test plan, functional parts may be
missing and due to large Test Plan it is difficult to review by development team.
3. Test Strategizing: The product functionality testing is complex. The challenge is
to utilize available limited time and resources to meet optimize test coverage. The
best practice is to define priority of test case in Test Plan i.e. 1,2,3,4. The benefit
of defining priority level is that high risk features will be tested in detail rather
than low risk feature.
4. Test Execution: This phase is critical phase of testing life cycle. After finishing
the Test Plan and strategy this phase is taken as to do action according to Test
Plan with the purpose of optimum utilization of time and resources. The challenge
is to mismatch between Test Plan and actual functionality.
5. Defect Management: The critical steps of testing process are effective defect
tracking, analysis and reporting. The challenge is that incomplete and ambiguous
defect reporting.
6. Test Results Reporting: The test result report shouldn't be difficult. It should be
useful for all the stakeholders associated with the product (Infosys, n.d.). Tangible
products can be easily seen whereas intangible that is software cannot be easily
viewed. Some indication is displayed before occurring error in tangible products.
But in software, there are errors because of lack of testing or inefficiency of
testing. Much important software had failed in the previous days because of the
19
absence of quality assurance or incomplete software testing. In traditional
software engineering, the use of software testing is minor and software testing
was not focused much. Similarly, quality assurance conducted after finished
development phase. Nowadays due to complexity of software, the importance of
testing is increasing.
There are two different words in software engineering. They are debugging and software
testing. Efforts to eliminate failures are called debugging whereas software testing is to
identify any defect in the system (ibid).
In order to perform more effective testing is to find more effective ways is a key
challenge in testing (Bhusan, 2011). The primary goal of component testability is to reuse
the software component. But the challenge is how to reuse component test, how to
construct testable component (Gao, n.d.). The latest research result and hot issues on
software testing are Test Driven Development (TDD), Iterative and Incremental Test,
GUI automation test, Testability of software components. Due to complexity of modern
system, there is an obstacle for testing efficiency maximization (Hongchun, n.d.). The
basic practices of software testing are functional specifications, reviews and inspections,
functional test, multiplatform testing, internal betas, automated test execution.
Developers are unable to test every modules and system thoroughly because they
considered it as a time consuming and lacking a significant pay off. The challenges of
software testing in web based application in the area of nonfunctional requirements and
functional requirements. Performance, scalability, compatibility, accessibility, usability,
20
and security are the nonfunctional requirements. In order to verify each nonfunctional
requirements specific aim should be designed (Lucca et al., 2006).
2.7
Challenges of Quality Analyst
The challenge for the software test specialists are:
In order to avoid any unwanted defects in quality of product, they need to have
communication skills to talk to development engineers. Likewise, they need to have test
skill as well (Toshiaki, 2008).
Senior testers can deliver high quality software. The fundamental challenges of software
testing are:

There is no possibility of doing complete testing

Testers group lack programming knowledge
There are numerous ways to test such as:

Test the product in user point of view

Test the product or software sequentially
Mostly programmer ignores the bugs because tester first test the software, if bug find
then report it to developer. In case of time, the time needed to test the particular product
is infinite rather than available time. Testers do not have programming knowledge in
common testing practices. If testers have strength programming skill, then testing will be
strength else testing will be weak (Kaner, 2003).
The challenge in software testing is lack of skilled tester, the team member doesn't have
enough knowledge in domain area, poor understanding requirements in software
21
development life cycle, time constraints for testing, the lack of involvement of test team
in SDLC, no good relationship with developer and no documentation.
The challenge for tester is testers do not have enough time to test the project. If the
product goes for live before closing the bug, it automatically hampers in the quality of
software. Similarly, Testers don't understand requirement clearly and tester involve late
in the project rather than starting of the project i.e from requirement specification. The
test team must be competent in three areas domain knowledge, testing skill, technical
expertise (Rao et.al, 2011).
2.8
Validation and Verification
Software testing verifies that software is working properly rather than defect detecting.
Verification: Are we building the product or system right?
Validation: Are we building the right product or system? (Sommerville, n.d.).
Product is designed and developed as per user expectations meeting with all required
functionality this makes assured by verification. Validation comes after verification and it
involves actual testing. Any changes, addition, deletion of the requirement should be
documented and tested within time period (Rao et.al, 2011).
Software testing is a broad term from unit testing to the customer validation of a system
acceptance testing. Testing is very expensive activity in software development life cycle
(Bertolino, 2007).
22
2.9
Team efforts for quality product (Case scenario of EBpearls)
The team effort is very vital aspect for quality product of web based application. An
example of team building and collaborative approach for quality product of EBpearls is
depicted in Figure 1 below. It clearly shows the organizational human resource structure
which is very vital to build team for quality product.
Figure 1: A collaborative approach of Quality Analyst with development team
In order to make quality software, quality analyst should be involved with development
team. Development team basically include Project Manager (PM), Developer, Designer,
QAn. Firstly, Account manager create storyboard. Project manager receive storyboard
and it is forwarded to designer, developer and QAn. They studied storyboard and start to
develop and design the project. There is meeting with project development team daily at
least 10 to 15 minutes that we call it as scrum. Project generally divided into small tasks
23
and sub-tasks. If task is finished up to 80% then it is ready for testing. That is also called
alpha release. Then QAn started testing and if all the things are ok as per requirements
then status will be changed to 90% by QAn else quality analyst post the bug in bug
section. Development related bug is assigned to developer and design related bug is
assigned to designer. Developer and designer are responsible for solving corresponding
bugs. They need to solve it. After solving the bug, it is assigned to quality analyst for
retesting. Quality analyst verifies those bugs. If it is working as per need then close the
bug else reopen the bugs. All the tasks and bugs should be closed before delivering the
software product.
All the tasks are also verified by the people of Procab Studio Geneva1. If all the tasks are
completed as per wish, then they will be closed and their status will be 100%. It is gone
for beta release.
2.10 Quality Assurance Procedure in web based application
The quality assurance framework in web based application is presented in Figure 2. It
includes requirement of study, Test Plan preparation, Test Case generation, Test
execution (webapp), browser, testing Ul and finally release quality product.
1
Procab Team, is a team within Ebpearls which is formed as a direct hired employees for a
Swiss IT company called Procab Studio. Procab Studio has designers and sales representatives
(account managers) in Geneva. They develop customer base, design the websites and finalize
requirements. The development team in Nepal consisting of around 20 employees takes up the
requirements and design. Based on these, their job is to build websites and web applications.
24
Requirement Study
Test Plan Preparation
Test Case Generation
Test Execution (WebApp)
Browser
Testing UI
Test Server side
component
Test client side
component
Testing Network
Bug
Bug Fixing
Bug Finding
Release Quality Product
Figure 2: The systematic procedure for quality assurance in web based application
25
CHAPTER-III: RESEARCH METHODOLOGY
3.1
Introduction
In order to collect the necessary information the qualitative research approach has been
used. This was carried out to understand meanings, look at, describe and understand
experience, ideas, beliefs and values, intangibles such as these. The interview was carried
out with the responsible persons of the IT offices to get primary information. For that, a
set of checklist/questionnaire was developed which is in annex.
3.2
Methods of data collection
Primary information was collected through interview with concerned personnel of the
various IT organizations. On line response from the respondents were also collected
using the semi-structured questionnaires through Google docs. The secondary data was
gathered through the related articles, white paper, Portable Document Format (PDF) files,
and previous studies/researches. Some of the information was taken from a variety of
websites.
3.3
Source of primary data
The face to face interview was carried out to get the information based on the set
checklist with the concerned individual of IT Himalaya, Auxfin Nepal, F1 soft
International, Braindigit, Miracle Interface, Crossover, and Professional Computer
Systems Pvt.Ltd and EB Pearls. On line responses were collected from SPI Nepal Pvt.
Ltd., Verisk, Dear Walk Services, Eminence Ways Pvt. Ltd, Javra Software, Longtail emedia Pvt. Ltd, Peace Nepal DOT Com Pvt. Ltd., Yomari inc and Avionte in excel form
in Google drive.
26
3.4
Selection of the respondents
In order to collect primary data, the individual and organizational were selected
purposively by using deliberate sampling. This sampling method involves purposive or
deliberate selection of particular units of the universe for constituting a sample which
represents the universe. In total, seventeen IT organizations were selected and 31
respondents were responded of questionnaire for data collection.
3.5
Methods of data analysis
The collected information was tabulated and presented in various Tables and Diagrams.
The collected raw data was classified into some purposeful and usable categories. The
editing procedure was applied for improving the quality of data for coding. After coding,
raw data was tabulated in the necessary forms. Analysis work after tabulation is based on
the computation of various percentages.
27
CHAPTER-IV: FINDINGS OF THE STUDY
During the course of the interview, the following findings were presented based on the
respondent’s response in the prescribed questionnaires.
4.1
4.1.1
Institutional set up and tools used
Establishment of quality assurance department
The 17 different organizations have provided their response. Out of 17 organizations 15
organizations do have quality assurance department and 2 organizations don't have
quality assurance department i.e. 88.2% have the quality assurance department and
11.7% don't have quality assurance department in selected organizations. Likewise, 4
organizations have 1 QAn, 1 organization has 2 QAn, 3 organizations have 4 QAn and 7
organizations have more than 4 QAn. In case of not having QAn they used to test the
software by Project Manager and developer themselves.
4.1.2
Testing process used
31 individual respondents have provided their response via Google drive. Out of 31
respondents, only 17 respondents have implemented black box testing for testing types
and the rest of 14 respondents have implemented both testing types i.e. black box and
white box that means gray box also.
In case of testing types, 13 respondents are following manual process while testing the
software and the 18 respondents are following both (Manual and automatic) testing
process. However, only Verisk organization is using automatic testing tools for testing
the software. There is no manual testing is done.
28
4.1.3
Use of Test Plan
25 respondents are using Test Plan while testing the software. Only 6 respondents are not
using Test Plan while testing the software. People know the benefits of making Test Plan
are to keep track of test, store information in document and serves as a guide. The
respondents also know other benefit that is Test Plan can be reused if regression testing
occurs as a benefit; however they say that if small project is ready for testing, this is not
necessary to make test plan. In the same way, sometimes there is no enough time to make
test plan.
The Figure 3 shows that the 81% respondents use test plan. Six respondents (19%) out of
31 do not make Test Plan due to lack of time to make plan, lack of knowledge or skill
about test plan. They also think that Test Plan is not necessary to test the software.
Users of Test Plan in Percentage
19%
Used
Not Used
81%
Figure 3: Users of Test Plan
29
The Figure 4 indicates that 17 percent respondents were found that there is no need of
making Test Plan while testing software. However, 50 percent respondents have not used
Test Plan because of the lack of knowledge about the Test Plan. Similarly, due to lack of
sufficient time for preparing Test Plan 33 percent respondents have not used test plan.
Percentage of Respondents
Reasons of Not Making Test Plan
50
50
40
30
20
10
0
33
17
No Need of Test Plan
Lack of Knowledge
Lack of Sufficient Time
No Need of Lack of
Lack of
Test Plan Knowledge Sufficient
Time
Reasons
Figure 4: Users of Test Plan
4.1.4
Software testing tools
The Figure 5 shows that automated software testing tools are used by 19 (61.30%)
respondents. Whereas the rest of the 12 (38.70%) respondents have not used automated
software testing tools. 11 respondents use selenium, 8 respondents use JMeter. Likewise
LoadRunner, Acunetix, NeoLoad, MetaSploit, Kali, Burpsuite, Test Link, Load UIWeb
and Ranorax are used by respondents. The benefit of using automated software testing
tools are for fast testing save time, cost and effort, automated regression test, motivation
to tester for new technology.
30
Percentage of tools Users
Automated software testing tools users
80
61.3
60
38.7
40
20
0
Tools used
Tools not used
Figure 5: Automated software testing tools users
Out of the total respondents (12), 25% opined that automated software testing tools was
not used due to budget constraint and lack of skilled human resources. Likewise, 30%
expressed their views that lack of knowledge was the hindering part of not using testing
tools. According to 20% respondents all three causes namely budget constraints, lack of
skilled human resource and lack of knowledge were the problems of not using testing
tools (Figure 6).
Percentage
Causes of not using automated software testing tools
in Percentage
30
20
10
0
25
Budget
Constraint
30
25
20
Lack of
Lack of
All of the
Skilled Knowledge
Above
Human
Resource
Causes
Figure 6: Causes of not using automated software testing tools
31
4.1.5
Software quality assurance in agile methodologies
Out of 31 respondents 10 (32%) respondents response agile methodology not following
while developing software and the rest of the 21 (68%) respondents are following agile
methodologies. Similarly, 8 (26%) quality analysts have been involved in daily scrum
whereas 23 (74%) respondents have not been involved in daily meeting like scrum
(Figure 7).
Involvememnt of Quality Analyst in
Daily Scrum
26%
Involvement
Not Involvement
74%
Figure 7: Automated software testing tools users
4.2
Perception of respondents
Perception of the respondents was collected in two areas namely (i) ways to better
development of software quality assurance and (ii) problems faced by Software Quality
Analyst while not making Test Plan.
4.2.1
Chances of error/bug with using “Test Plan”
Table 1 below shows that 97% respondents believe that chances of error/bug occur
(below 50%) for the login form validation, registration and pagination display testing
32
while using “Test Plan”. 94% respondents believe that chances of error/bug occur (below
50%) for testing social links and category display. 90% respondents believe that chances
of error/bug occur (below 50%) for testing footer links and contact form. 88%
respondents believe that chances of error/bug occur (below 50%) for testing header
slideshow i.e transition effect or something alike. 84% respondents believe that chances
of error/bug occur (below 50%) for testing rollover effect and email templates. 65%
respondents believe that chances of error/bug occur (below 50%) for testing breadcrumb.
On the other hand, 19% respondents believe (more than 50%) chances of error occurring
for breadcrumb. Similarly, 16% respondents believe (100%) chances of error occurring
for breadcrumb however using test plan.
Table 1: Chances of error/bug with using “Test Plan”
Features
Below 50%
More than 50%
100%
Login form validation
30 (97%)
1 (3%)
0
Registration
30 (97%)
1 (3%)
0
Header (Slide show)
27 (88%)
2 (6%)
2 (6%)
Footer (Links)
28 (90%)
3 (10%)
0
Social Links
29 (94%)
2 (6%)
0
Category Display
29 (94%)
1 (3%)
1 (3%)
Roll over effect
26 (84%)
3 (10%)
2 (6%)
Breadcrumb
20 (65%)
6 (19%)
5 (16%)
Contact Form
28 (90%)
3 (10%)
0
Email Templates
26 (84%)
3 (10%)
2 (6%)
33
Features
Below 50%
More than 50%
100%
Pagination Display
30 (97%)
1 (3%)
0
Source: Field Survey, March, 2014
4.2.2
Chances of error/bug without using “Test Plan”
Table 2 below shows that 26% respondents believe that chances of error/bug occur
(100%) for testing breadcrumb and email templates while not using “Test Plan”. 16%
respondents believe that chances of error/bug occur (100%) for testing header slideshow.
6% respondents believe that chances of error/bug occur (100%) for testing rollover effect.
3% respondents believe that chances of error/bug occur (100%) for testing category
display and contact form. 94% respondents believe that chances of error/bug occur (more
than 50%) for the rollover effect without using “Test Plan”. 90% respondents believe that
chances of error/bug occur (more than 50%) for the footer links, social links without
using “Test Plan”. 87% respondents believe that chances of error/bug occur (more than
50%) for the login form validation, registration, category display and pagination display
for testing while not using “Test Plan”. 84% respondents believe that chances of
error/bug occur (more than 50%) for the slide show. 78% respondents believe that
chances of error/bug occur (more than 50%) for contact form. 74% respondents believe
that chances of error/bug occur (more than 50%) for breadcrumb, email templates. On the
other hand, 19% respondents believe that chances of defect occur (below 50%) to testing
contact form while not making test plan.
34
Table 2: Chances of error/bug without using “Test Plan”
Features
Below 50%
More than 50%
100%
Login form validation
4 (13%)
27 (87%)
0
Registration validation
4 (13%)
27 (87%)
0
Header (Slide show)
0
26 (84%)
5 (16%)
Footer (Links)
3 (10%)
28 (90%)
0
Social Links
3 (10%)
28 (90%)
0
Category Display
3 (10%)
27 (87%)
1 (3%)
Roll over effect
0
29 (94%)
2 (6%)
Breadcrumb
0
23 (74%)
8 (26%)
Contact Form
6 (19%)
24 (78%)
1 (3%)
Email Templates
0
23 (74%)
8 (26%)
Pagination Display
4 (13%)
27 (87%)
0
Source: Field Survey, March, 2014
4.3
Data Analysis with correlation
In order to validate or verify the survey data between variables, correlation has been used
of SPSS (Statistical package for the social sciences). For that need to go for menu
Analyze then click on correlate then Bivariate is selected and chosen the variables.
Correlation coefficient is Pearson and test of significance is 2-tailed. The result is
displayed below in tabular form.
35
Table 3: Correlation between dependent and independent variables
Quality Assurance
Quality
Assurance
Pearson
Correlation
Make Test Plan
1
.536
Sig. (2-tailed)
Make Test Plan
.002
N
Pearson
Correlation
31
.536
Sig. (2-tailed)
.002
N
31
1
31
31
Correlation is significant at the 0.01 level
There is statistically significant correlation between Quality Assurance (dependent
variable) and Test Plan Document (independent variable). The 2 tailed tests 0.02 has been
displayed as per result that is less than p value 0.05. P value is always measured in 0.05.
Most of the respondents make Test Plan before testing the software according to survey
value. So that correlation value is significant between variables. ‘Test Plan’ document is
necessary while testing the software in order to make quality product.
Table 4: Quality assurance and involvement of quality analyst in SCRUM
Quality
Assurance
Quality Assurance
Involvement of quality
analyst in SCRUM
1
.155
Pearson
Correlation
Sig. (2-tailed)
.405
N
31
36
31
Involvement of
quality analyst in
SCRUM
Quality Assurance
Involvement of quality
analyst in SCRUM
Pearson
Correlation
.155
1
Sig. (2-tailed)
.405
N
Correlation is significant at the 0.01 level
31
31
The correlation diagram can be concluded based on the survey results that there is no
statistically significant correlation between the involvements of quality analyst in
SCRUM (independent variable) during software development in order to make quality
product (dependent variable) in software quality assurance in case of web based
application because 0.405 > 0.05.
Scrum plays very important role to make quality product because if QAn have not been
involved in daily meeting then some information is missing to tell to QAn then s/he
couldn’t test those parts. However, correlation does not exist between two variables that
mean QAn has not been involved in SCRUM in most of the organizations during
development of software. Some organization still has practiced to be involved QAn in
SCRUM due to which deliver quality of product.
4.4
Best way of software quality assurance
1. Proper job description should be maintained in each IT industry for the position of
Quality Analyst for better development of QA field.
37
2. Required training such as automated tools and Test Plan and its Usage should be
provided to QAn personnel that testing work much more efficiently and
effectively.
3. Quality analyst should be well known about database and programming language
etc.
4. Software quality analyst should have fine knowledge in manual testing and also
should follow test driven development (TDD) process.
5. There is good demand and future growth for the QAn team so we need to boost
the community forum which will enhance to upgrade the skills and be helpful as it
seems much necessity for the QA in the growth of IT sector.
6. The software quality assurance should keep in face book pages due to which
people who are interested in SQA field they get opportunity to visit the pages and
get the job opportunity. This is one of the ways of getting knowledge of Software
quality for software vendors, software companies, Professionals, Government
Bodies.
7. Software quality assurance must be done from the very starting phase of SDLC to
the end of SDLC.
8. To make the quality product we shouldn’t focus only on the manual and
automation testing but also should be able to check the security measures also
because it matters a lot when the site is launched.
38
9. Interactions among QAn members are a must necessary for any sort of update like
what to test, how to test the software such as responsive design, mobile
application.
10. In emerging context Quality Analyst should have training for the test plan/test
cases to improve the QA.
11. Quality defines everything in the field of IT nowadays.
QA meant for
maintaining that quality. Focus on quality product is the demand of today so it’s
now necessary to follow in some s/w development methodologies like agile
methodology, test driven methodology to maintain the quality of overall product.
12. All project team should realize importance of QA in software development life
cycle.
13. For better software production, each production house needs QA. QA should be
involved in software development from the initial time so that each bugs and
errors can be control before go to production.
14. In order to improve a quality on the QA process/ quality of product being tested, a
QA team should focus on following activities:
- Appropriate planning
- Following standard quality assurance process
- Recording and Tracking mistakes /patterns of mistakes
- Reviews and retrospection
- Quality Audits
- Continuous Improvement
39
15. Systematic Process should be followed in the QA task for the betterment of QA
field such as Requirement Study, Test Plan preparation, Test Case preparation,
Test Execution
16. Quality analyst plays an integral role in the software development. All the QAn
should be involved in daily meeting with designer and development team. If new
feature request by the clients then that also should be provided to QAn from the
very beginning like developer and designer.
17. All the organization must follow agile methodology for the betterment of their
organization working procedure.
4.5
Problems faced by software quality analyst
1. Creating a Test Plan is a must on all of our projects. A Test Plan is a strategic
document that works as a guide to the test team. This document helps us to outline
what all tests are required, who will be involved, timelines of the test and tools
that will be used for test. Without this document it will be very difficult to
understand what all types of testing should be done on the system. Also, without
this document it would be difficult to ensure whether the test being performed
throughout the test cycle was appropriate.
2. Usually due to less time, QA personnel are not able to prepare test plan.
When Test Plan is not prepared, the test process will not be systematic and it will
be random. There will be no record of types of testing performed in the absence of
test plan.
40
3. When Test Plan is not prepared then the testing team will not have in depth
knowledge about the site/application and hence the testing process will not cover
all the aspects of the site that needs to be tested.
4. Difficult to track test status if there are large features in the project in the absence
of test plan.
5. Test Plan states what requirements will be verified during software development
and testing. When we do not make Test Plan then, Project information will not be
stored or will not logged features of project.
6. It is unable to track the test cases and the corresponding results.
Thus, no means of communicating the test done and results to concerned
authority. Test Plan can also be reused for similar product if once done.
7. Sometimes it is difficult to know about the expected result, if test case is not
made. For regression testing, it would be difficult if Test Plan is not made.
8. If Test Plan is ignored while testing the project, works get jumbled.
9. All features may not be tested in the absence of test plan.
4.6
4.6.1
Existing practice in Selected Organization
IT Himalaya
There is only one quality analyst now. They are conducting manual test rather than
automated test. They test the software according to user’s perspective. Test for browser
compatibility. They test the software and if there is error then they point out the errors in
test sheet and fix those problems. The problem could be in functionality, business logic.
Errors will be fixed before release the products.
41
The main cause of occurring error in the software is depends upon the developers skill for
the particular project, client’s requirement not clear, and frequent requirement change.
4.6.2
Auxfin Nepal
There is no quality analyst in the organization. Software is tested by developers. The
particular software developed by developer and once he tests the software and handover
to another developer for confirmation because developer who develop the software might
omit the error/defect while testing. So, software will be handed over to next developer for
testing. They use to follow black box testing; white box testing both that means grey box
testing.
The main cause of occurring error in the software is the lack of understanding
requirement by developers. And another one is due to poor communication skill between
client and developer.
4.6.3
F1 Soft International
There are four quality analysts. Three are full time and one is part time. These numbers of
QAn is sufficient for now because sometimes there is no heavy workload and sometimes
lots of work should be performed. In case of heavy work, it is not enough QA to test the
software. In case of no heavy load they will be free. So they are managed for work. They
are performing both manual and automated software testing tools. 90% tasks are tested in
manual and only 10% is for automated testing. Selenium and JMeter is used for
automated test. Selenium is used for functionality testing likewise when repetitive task
needs to be performed then selenium is best rather than manual testing and on the other
hand JMeter is used for performance testing.
42
In case of testing methods black box testing is performed basically. There is Continuous
Integration Testing (CIT). CIT is performed all the time in project because project never
becomes end or finish. When software is developed, designed, tested and implemented
then maintenance or support should be performed at last. These phases are the software
development life cycle. Similarly, CIT means once completed the software after some
time some task needs to be added on it or some changes needs to be done or some parts
needs to be removed as per client’s requirements. Whatever the requirements arise then
that task should be performed. After development part finished then it should be tested
and delivered. That’s why software never ends it process all the time so continuous
integrated testing should be performed while necessary.
Testing should be performed based on Test Plan and Test Case. They use their own
format for Test Plan and Test Case. Normally, Test Plan and Test Case is based on the
project nature or environment.
Project is in Subversion (SVN). So if one updates the task others can also see the updated
task. SVN make up to date the project like if one delete the some part, others also can’t
see the deleted part. They have three processes to testing the software. One is test plan,
mind maps and test case. Testers should follow three phases.
Mind map is open source tool for flowchart and Bugzilla is bugtracker tool. This is the
media to communicate with developer. If Issue found out by tester then s/he posts the
issue in bugzilla and after developer fix the issue then s/he update the status and tester
43
should test the issue again. If it is as per the requirements then it is ok otherwise reopen
the bug.
In case of automated tool, selenium IDE 2.2.0 tool in Firefox which is used for basically
repetitive task. There is listener, sampler, and controller. Listner is used for result,
sampler is used for transactions, and controller is used for logic. To test the software in
tool once we need to test by manually. Whatever we set, that thing only test by automated
tool. But here the advantage is that testing time is extremely reduced.
JMeter, thread is the number of concurrent users. There should be Google account for
JMeter user. There is loadosophia.org for the result view in graphical representation.
4.6.4
Braindigit
There are software testing department. 18 software testers are available in numbers.
There is large number of software tester available in the company. They follow automatic
software testing tools such as functional testing, security testing, performance testing,
unit testing, network testing. For functional testing, selenium is used. For security testing
backtrack software is used. For performance testing, JMeter, loadUI, soapUI is used. For
unit testing, nunit is used. And for network testing, backtracking software is used.
Black box, white box, grey box all types of testing is getting performed. The test
environment is Web based system and compatibility tested in device such as mobile and
others. Likewise browser compatibility is also getting performed.
44
4.7
Test Plan document
The Test Plan is prepared with the following features. The detail of the Test Plan
documents is presented below (Table 3).
Features are used to keep “the features” which are to be tested according to requirements.
Features are also divided into sub features. For instance, software should use three
languages such as English language, French and German; if images have conditions such
as only 4 images should be displayed in slider. These parts can be kept in sub features. In
description, “social link” title is kept for features and link provided for social links can be
kept as a description.
Similarly, there are design testing and development testing. Development testing can be
considered as a functional part testing in backend. Design testing can be done or verified
through ‘PSD’ provided by client. These designing and developing parts should be tested
in different browsers such as IE different versions (7, 8, 9, 10, 11), Mozilla Firefox,
Google chrome, Mac Safari, different mobile devices. Remarks also can be used to write
description of features if needed.
45
4.8
“Test Plan” (Magento E-commerce Application)
The Test Plan (Magento E-ecommerce Application) is presented in Table 5-25 which clearly show the features, sub-features,
description design development and browsers compatibility. Under the general document of Test Plan homepage of Carisma,
Home Page> Header-on all pages of website, Homepage>On all pages of website> call to action, Homepage>Slide,
Homepage> Products highlight, Newsletter Subscription, Footer, Listing Page (important note: show all results in one page:
infinite scroll), Creators Page (important note: show all results in one page: infinite scroll), Product detail Page, Product detail
Page popup, Content page, Contact page, My Account page, Cart page, Order Process, Dashboard, Payment method,
Transactional emails, CMS Pages and Types of product.
Table 5: Homepage of Carisma
Test For
Features
1. Header
Sub Features
Description
FR,EN,DE
Header menu
2. Slideshow
5 pictures
title, short description, link on
each pictures
3. Calls to action
FR,EN,DE
4. Link to a
section/page
5. Link to the page:
I'Univers carisma
6. Link to a
section/page
7. Link to a
section/page
8. Link to a
picture, [title, link-fr,en,de]
title,short desc,link -(FR,EN,DE)
picture,[title,link-fr,en,de]
picture,[title,link-fr,en,de]
Browsers Compatibility
Design/
Development
Design /
Development
Design/
Development
Design/
Development
Design/
Development
Design/
Development
Design/Develop
ment
Design/
Development
Design/
46
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
MAC
Safari
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Remarks
Test For
Features
Sub Features
Description
Design/
Development
Development
section/page
9. Calls to action:
link to Nous
rejoindre page
10. Newsletter
inscription
FR,EN,DE
12. Footer
link
Design/
Development
facebook link, twitter link,
pinterest link, google+ link
Design/
Development
Design/
Development
Design/
Development
FR,EN,DE
11. Social Links
FR,EN,DE
Browsers Compatibility
footer menu
Source: Story Board of Carisma
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
MAC
Safari
OK
OK
NOK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Remarks
Test Plan for E-commerce Product
Table 6: Home Page> Header-on all pages of website
Test For
Features
1. Logo
2. Main navigation
3. Customer
account login
4. Recherche on go
5. Language
Sub Features
Description
Design/Develop
ment
Design/Develop
ment
Linked to the homepage
Fr, En, De (submenus and
thumbnail managed by user in
admin)
redirect to login page if he is not
logged otherwise redirect to
dashboard page
search and display result,
category template page
Language chosen by flag- Fr, En,
De
6. Cart
Card content if something in cart
7. Currency display
choice
Euro, Chf, Dollar
Source: Story Board of Carisma
Browsers Compatibility
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
MAC
Safari
OK
OK
OK
OK
OK
OK
OK
OK
Design/Develop
ment
OK
OK
OK
OK
OK
NOT
TESTED
OK
OK
Design/Develop
ment
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Design/Develop
ment
Design/Develop
ment
Design/Develop
ment
Design/Develop
ment
Test Plan for E-commerce Product
47
Remarks
Table 7: Homepage>On all pages of website> call to action
Test For
Features
Sub Features
Description
1. Satisfied
FR,EN,DE
rollover
2. Delivery
FR,EN,DE
rollover
3. Contact
FR,EN,DE
rollover
Source: Story Board of Carisma
Browsers Compatibility
Design/Develop
ment
Design/
Development
Design
/Development
Design/
Development
Remarks
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
SKIPPED
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for E-commerce Product
Table 8: Homepage>Slide
Test For
Features
1. Slide show
Sub Features
5 images/
language
2. Title
Description
add image from admin
user will be able to define title
3. Short desc
4. Button
FR,EN,DE
5. Slide navigation
5 images
user will be able to define short
description for image
user will be able for each image in
each language to define link. Label
of the button
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
Design/
Development
Design/
Development
Design
/Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Remarks
Test Plan for E-commerce Product
Table 9: Homepage> Products highlight
Test For
Features
Sub Features
1
1,3,4,5
images
2
FR,EN,DE
Description
on roll over [grey layer with
transparencey]
title, short description, link to the
content page
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
Design/
Development
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for E-commerce Product
48
Remarks
Table 10: Newsletter Subscription
Test For
Features
1. Link to content
page
2. Newsletter
inscription
Sub Features
FR,EN,DE
FR,EN,DE
3. Social Links
Description
one link per language (Cf page
7)
Message to display on
subscription (in the text field)
facebook link, twitter link,
pinterest link, google + link
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
Design/
Development
Design/
Development
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari
in MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Remarks
Link should
be kept
Test Plan for E-commerce Product
Table 11: Footer
Test For
Features
1. footer menu
Sub Features
FR,EN,DE
Description
Footer menu
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
Remarks
Test Plan for E-commerce Product
Table 12: Listing Page (important note: show all results in one page: infinite scroll)
Test For
Features
1.
collection/category
title
2. categories
navigation filters
3. Results/product
display
4. Thumbnail
rollover
Sub Features
Description
collection or name of category
displayed,depending the user get
there
size, price,
marques(brands)/designer
according the navigation 1 and
filter selected 2, results display
as thumbnails. Title on
rollover(cf4), display the price
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for E-commerce Product
49
Remarks
Table 13: Creators Page (important note: show all results in one page: infinite scroll)
Test For
Features
1. creators slider
2.
collection/category
title
3. categories
navigation filters
Sub Features
3 slides max
4. Results/product
display
5. Thumbnail
rollover
Description
user could manage pictures, text,
link for each slide for each creator
collection or name of category
displayed, depending the user get
there
size, price,
marques(brands)/designer
according the navigation 1 and
filter selected 2, results display as
thumbnails. Title on rollover(cf4),
display the price
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
Remarks
IE7
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Link should
be kept
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for E-commerce Product
Table 14: Product detail Page
Test For
Features
Sub Features
Description
3. Product title
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
4. Price
display the price as it is entered in
admin
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
while creating new product, user
must have the choice to create a
product with or without size
and/or color because some
products will not have an attribute
size and color.
5. Product attributes
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
name of the category or collection
displayed depending the way user
get there
manage from
admin
enter the
price
including tax
max 3
(quantite,
taille,
couleur),
these
attributes are
mandatory
Remarks
Safari in
MAC
1. Breadcrumb
2.
Collection/category
title
Browsers Compatibility
Design/
Development
Design/
Development
50
Test For
Features
Sub Features
Description
Browsers Compatibility
Design/
Development
Remarks
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
NOK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Chrome
IPAD
Safari in
MAC
OK
OK
OK
on adding to
cart.
6. Adding to cart
FR,EN,DE
7. 2 tabs:
en,de
facebook,
twitter,
pinterest
manage from
admin
manage from
admin
manage from
admin
8. Social share to
display
9. first picture of
product
10. Other pictures
of product
11. Related
products
on rollover, button becomes red
Tab1, Tab2 (content managed in
admin)
Design/
Development
Design/
Development
Design/
Development
max 4 pictures
Source: Story Board of Carisma
Design/
Development
Design/
Development
Design/
Development
Test Plan for E-commerce Product
Table 15: Product detail Page popup
Test For
Features
1. Popup
Sub Features
Description
display on
adding a
product in
the cart with
grey layer
behind
2. Associated
products
when there
are some
create by user (this product with
one of these 3 allows customer to
have 20% reduction)
3. Button
continue the
shopping
close popup adding the product in
the cart
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
IE7
IE8
IE9
IE10
Design/
Development
OK
OK
OK
OK
Firefox
OK
Remarks
Design/
Development
OK
OK
OK
OK
OK
OK
OK
OK
Design/
Development
To be
tested
OK
OK
OK
OK
OK
OK
OK
Test Plan for E-commerce Product
51
one of these
three
products
20%
reduction
Table 16: Content page
Test For
Features
Sub Features
Description
Browsers Compatibility
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Remarks
Chrome
IPAD
Safari in
MAC
By default magento cms page
Source: Story Board of Carisma
Test Plan for E-commerce Product
Table 17: Contact page
Test For
Features
Sub Features
Description
1. Breadcrumb
2. Coordinates
3. Contact form
FR,EN,DE
Sentence of presentation
FR,EN,DE
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
Design/
Development
Design/
Development
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Skipp
ed
OK
OK
OK
OK
OK
OK
OK
Remarks
Test Plan for E-commerce Product
Table 18: My Account page
Test For
Features
1. If customer is
ever logged
2. If customer is not
ever logged
3. Continue to
registration
4. Forgot password
Sub Features
Description
redirect to
dashboard
page
create an
account or
signin with
email ID and
password
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Development
OK
OK
OK
OK
OK
OK
OK
OK
Development
OK
OK
OK
OK
OK
OK
OK
OK
Development
Development
FR,EN,DE
FR,EN,DE
Browsers Compatibility
After clicking this link, redirect to
form of email and he will receive
a new password by email.
52
Remarks
Test For
Features
5. Login
Sub Features
FR,EN,DE
Description
The visitor will login with his
email and password and
redirected to dashboard page.
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
Development
IE7
OK
IE8
OK
IE9
OK
IE10
OK
Firefox
OK
Chrome
OK
IPAD
Safari in
MAC
OK
OK
Remarks
Test Plan for E-commerce Product
Table 19: Cart page
Test For
Features
1. Place an order
button
Sub Features
Description
Browsers Compatibility
Design/
Development
Remarks
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
2. Delete button
after click on button, redirect to
order process
after click on this image, remove
the product from cart
3. Product image
display image of the product
Development
OK
OK
OK
OK
OK
OK
OK
OK
4. product name
display product name
display unit price without VAT
and with VAT
visitor will be able to update the
quantity of the product
display here subtotal without
VAT and with VAT
Development
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
redirect to homepage
quantity of the products in the cart
will be updated
fill coupon number, if it is valid
the discount will be applied
display total price without VAT
or with VAT
Development
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
5. Price
6. Quantity
7. Subtotal
8. Continuer vos
achats button
9. Mettre a jour le
panier button
10. Coupon
11. Total prices
Source: Story Board of Carisma
Development
Development
Development
Development
Development
Development
Development
Development
Test Plan for E-commerce Product
53
Table 20: Order Process
Test For
Features
Sub Features
Description
Browsers Compatibility
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
1. to place an order
as guest
Development
2. to registered
Development
OK
OK
OK
OK
OK
OK
OK
OK
Development
OK
OK
OK
OK
OK
OK
OK
OK
3. to sign up
Source: Story Board of Carisma
Remarks
Test Plan for E-commerce Product
Table 21: Dashboard
Test For
Features
1. Dashboard Menu
2. List of previous
orders
3. Billing and
shipping addresses
Sub Features
Description
user can update his account
information and change his
password, order status, subscribe,
unsubscribe newsletter, user can
logout after clicking appropriate
link
user will see the previous orders
he placed. Can see the details of
orders and replace an order. We
will add here link 'Reclamation',
when user click on that link
redirect to contact page.
Contact information displayed
and can be edited if we click on
edit link. Can be changed
password when we click on
change password link. Billing and
shipping address display and
update them when we click on
edit address link.
Source: Story Board of Carisma
Browsers Compatibility
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
Remarks
IPAD
Safari in
MAC
Development
OK
OK
OK
OK
OK
OK
OK
OK
Development
OK
OK
OK
OK
OK
OK
OK
OK
Development
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for E-commerce Product
54
Table 22: Payment method
Test For
Features
Sub Features
Description
Browsers Compatibility
Design/Develop
ment
IE7
IE8
IE9
IE10
Firefox
Chrome
IPAD
Safari in
MAC
Remarks
credit card with
postfinance
paypal
waiting for
information
Source: Story Board of Carisma
Test Plan for E-commerce Product
Table 23: Transactional emails
Test For
Features
1. Order
confirmation email
2. Account
confirmation email
3. Shipment
confirmation email
4. Newsletter
subscription
5. Newsletter
unsubscription
Sub Features
Description
when an order will be placed, an
email will be sent to the user to
confirm the order.
when user create account, an
email will be sent automatically
to him/her to confirm email
address
when the order is sent to the
customer, an email will be sent
to inform him/her.
when somebody subscribes to
the newsletter, s/he will receive
a welcome email
when somebody unsubscribes to
the newsletter, s/he will receive
a confirmation email
Source: Story Board of Carisma
Browsers Compatibility
Design/Developm
ent
IE7
IE8
IE9
IE10
Firefox
Chrome
Remarks
IPAD
Safari in
MAC
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Development
Development
Development
Development
Development
Test Plan for E-commerce Product
55
Table 24: CMS Pages
Test For
Features
Sub Features
Description
Browsers Compatibility
Design/
Development
IE7
IE8
IE9
IE10
Firefox
Chrome
Safari in
MAC
Remarks
IPAD
Safari in
MAC
Remarks
IPAD
Manage the content of the cms pages through a wysiwyg editor, for french, english, and german version
Source: Story Board of Carisma
Test Plan for E-commerce Product
Table 25: Types of product
Test For
Features
Sub Features
Description
Browsers Compatibility
Design/Developm
ent
IE7
IE8
IE9
Different kind of configurable products:
Clothes:managing
size, weight
Objects:managing
size, weight
Source: Story Board of Carisma
Test Plan for E-commerce Product
56
IE10
Firefox
Chrome
4.9
“Test Plan” (Word press CMS Application)
The Test Plan (word press CMS Application) is presented in Table 26-28 which represent the Homepage of Emile Garcin, For
Sales and Order Details.
Table 26: Homepage of Emile Garcin
Features
Sub
Features
1. Main menu and
header menu
2. Logo Emile Garcin
3. Search bar
4. Language choice
5. Slide show
6. Office button
7. Recent discoveries
8. Geneve rive gauche
9. Geneve rive gauche
10. Lausanne/vaud
11. Montagne
12. map of
Switzwerland
13. Magazine
Description
Link to:
http://www.emilegarcin.ch
FR,
RU,EN
3-5
elements
Test for
design/development
design/development
design/development
Default language french
design/development
design/development
Slides 5 secs with fade effect
design/development
design/development
design/development
design/development
design/development
design/development
design/development
design/development
Link to:
http://flipbook.emilegarcin.com/
14. Footer
Source: Story Board of Emile Garcin
design/development
design/development
IE9
IE10
IE11
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for CMS Product
57
Firefox
MAC
Safari
Chrome
IPAD
Remarks
Table 27: For Sales
Features
Sub
Features
1. Results information
2. Price filter
3. Sort by dropdown
menu
4. Page selector
5. Estate offer
Description
Price descending
Increasing price
Max 220 chars
Source: Story Board of Emile Garcin
Test for
design/development
design/development
design/development
design/development
design/development
design/development
IE9
IE10
IE11
Firefox
OK
OK
OK
OK
OK
OK
OK
OK
MAC
Safari
OK
OK
Chrome
IPAD
Remarks
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for CMS Product
Table 28: Order Details
Features
Sub
Features
Description
1. Banner
2. Offer navigation
button
3. Information of the
offer
4. Picture gallery
5. Back to the list
6. Request for
documentation
Source: Story Board of Emile Garcin
Test for
design/development
design/development
design/development
design/development
design/development
design/development
design/development
IE9
IE10
IE11
OK
OK
OK
OK
MAC
Safari
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
OK
Test Plan for CMS Product
58
Firefox
Chrome
IPAD
Remarks
OK
OK
OK
CHAPTER-V: CONCLUSIONS AND RECOMMENDATIONS
5.1
Conclusions
Both manual and automatic testing process is used to ensure the quality of the product. If
the Test Plan is implemented to test the software, the chances of error occurring will be
low. The Test Plan can be considered as a systematic testing technique and it is also more
effective to test the software. Likewise, correlation also exists between quality and Test
Plan significantly as per the result.
Although correlation doesn’t exist between quality assurance and involvement of quality
analyst in SCURM based on survey results, all the QAn should be involved in daily
meeting which is called SCRUM with development team in order to deliver qualitative
product because if they do not participate in the meeting, QAn may miss some
information about project and lastly quality analyst may omit to test those parts.
No any features are omitted to test in every required browsers while testing the software
by making Test Plan.
To make the quality product, not only the manual and automated testing should be
focused but also check the security measures because it matters a lot when the site is
launched. Moreover training on automated software testing tools should be provided to
QAn personnel.
All project team should realize importance of quality assurance in software development
life cycle.
Quality analysts should have knowledge of database and programming language in order
to test the backend and white box testing also.
59
5.2
Recommendations/Future works
Based on the findings of the study the recommendations presented as follows.
1. It is recommended that all the software development company should have Quality
Assurance Department and necessary personnel (Quality Analysts) to ensure the
quality product by reducing errors.
2. Systematic Process such as Requirement Study, Test Plan preparation, Test Case
generation, Test Execution, Bug find and Bug fix should be followed in the QA task
for the betterment of QA field. It is recommended that this ‘Test Plan’ document is
also very fruitful to testers of web application. All the features of project need to
test for browser compatibility testing in web app.
3. All the organization should follow agile methodology for the betterment of their
organization working procedure. All the QAn should be involved in daily meeting
which is called SCRUM with development team in order to deliver qualitative
product.
4. There is some limitation in the present research that is to understand the depth study
of automated software testing tools. In order to identify the similarities/gaps and
better performance, comparative study can be carried out between manual testing
with Test Plan and automated software testing tools as future work.
60
References
Bertolino Antonia (2007). Software Testing Research: Achievements, Challenges,
Dreams, 2007 IEEE
Bhushan Bharat Konka (2011). A case study on Software Testing Methods and Tools,
Chalmers University of Technology University of Gothenburg, Department of Computer
Science and Engineering, August 2011
Black Rex (2007). Pragmatic Software Testing, Becoming an Effective and Efficient Test
Professional, 2007
Chillarege Ram (n.d.). Software testing best practices, Center for software Engineering
IBM Research
Crowell Nathaniel (2009). Software Testing Effectiveness, October 27, 2009
Eldh Sigrid (n.d.). How to Save on Quality Assurance – Challenges in Software Testing
El-Rayyes Emad Kh. and Abu-Zaid Ibrahim M. (2012). New Model to Achieve Software
Quality Assurance (SQA) in Web Application, International Journal of Science and
Technology, Volume 2, No. 7, July 2012
Farooq Sheikh Umar, Quadri S. M. K. and Ahmad Nesar (2011). Software Measurements
and Metrics: Role in Effective Software Testing, International Journal of Engineering
Science and Technology (IJEST), Vol. 3 No. 1 January 2011, ISSN : 0975-5462
Farooq Sheikh Umar and Quadri S.M.K. (2011). Evaluating Effectiveness of Software
Testing Techniques With Emphasis on Enhancing Software Reliability, Journal of
Emerging Trends in Computing and Information Sciences, Vol. 2, No. 12, December
2011
Farooq Umar (2012). Evaluating Effectiveness of Software Testing Techniques with
Emphasis on Enhancing Software Reliability, March 2012
61
Gao Jerry (n.d.). Component Testability and Component Testing Challenges, San Jose
State University One Washington Square San Jose
Grater T. Michael (2005). Benefit of Using Automated Software Testing Tools to achieve
software quality assurance, University of Oregon, Applied Information Management,
June 2005
Hongchun Zhang (n.d.). Research on New Techniques and Development Trend of
Software Testing, Hubei Polytechnic Institute
Infosys (n.d.). Functional Testing – Challenges and Best Practices, Building Tomorrow's
Enterprise
Juristo N., Moreno A.M., Vegas S. and S. Vegas (n.d.). Limitations of Empirical Testing
Technique Knowledge, Universidad Politécnica de Madrid
Juristo Natalia, Moreno Ana M. and Vegas Sira (2004). Reviewing 25 Years of Testing
Technique Experiments, Empirical Software Engineering, 9, 7–44, 2004
Kamsties Erik and M. Lott Christopher (1995). An Empirical Evaluation of Three DefectDetection Techniques, Appeared in the Proceedings of the Fifth European Software
Engineering Conference, 26–28 September 1995
Kaner Cem (2003). Fundamental Challenges in Software Testing, Florida Tech
Colloquium Presentation at Butler University, April 2003
Lucca Giuseppe A. Di and Fasolino Anna Rita (2006). Testing Web-based applications:
The state of the art and future trends, August 2006
Malik Imran Ullah and Waqar Ali Zaidi (2009). Quality Assurance Activities in Agile,
September 2009
Nelson Akinboyewa E. and Victor O. Johnson (2013). Software Testing Process Using a
Payroll System as a Case Study, International Journal of Engineering and Innovative
Technology (IJEIT) Volume 2, Issue 7, January 2013
62
Perry E. William (2006). Effective Methods for Software Testing, Third Edition, 2006
Pizza Michele and Strigini Lorenzo (n.d.). Comparing the Effectiveness of Testing
Methods in Improving Programs: the Effect of Variations in Program Quality, Centre For
Software Reliability City University, London, England
Quadri S.M.K and Farooq Sheikh Umar (2010). Software Testing – Goals, Principles,
and Limitations, International Journal of Computer Applications (0975 – 8887) Volume
6– No.9, September 2010
Rafi Dudekula Mohammad and Moses Katam Reddy Kiran (2011). A Study of State of
Practice Automated Software Testing, December 2011
Rao K. Nageswara and Sastri A. Pathanjali (2011). Overcoming Testing Challenges in
Project Life Cycle using Risk Based Validation Approach, Department of Computer
Science and Engineering, P.V.P.Siddhartha Institute of Technology Kanuru, Vijayawada
Andhra Pradesh, India, March 2011
Shah Rohen (2009). Testing Tools, rxs07u, G53QAT (11/27/2009)
Shivkumar Hasmukhrai Trivedi (2012). Software Testing Techniques, International
Journal of Advanced Research in Computer Science and Software Engineering, Volume
2, Issue 10, October 2012
Sirshar Mehreen and Dr. Arif Fahim (2012). Evaluation of Quality Assurance Factors in
Agile Methodologies, International Journal of Advanced Computer Science, Vol. 2, No.
2, Pp. 73-78, February 2012.
SmartBear Software Whitepaper (n.d.). 6 tips to get started with automated testing
Sommerville Ian (n.d.). Software Engineering, Ninth Edition
Tawileh Anas, Mcintosh Steve, Work Brent and Ivins Wendy (n.d). The Dynamics of
Software Testing
63
Toshiaki Kurokawa and Masato Shinagawa (2008). Technical Trends and Challenges of
Software Testing, October 2008
Vegas S. (n.d.). Identifying the Relevant Information for Software Testing Technique
Selection, Universidad Politécnica de Madrid Campus de Montegancedo, 28660 Boadilla
del Monte, Madrid, Spain
Verma Vikas and Malhotra Sona (2011). Journal of Global Research in Computer
Science, Applications of Software Testing Metrics In Constructing Models of The
Software Development Process, Volume 2, No. 5, May 2011
64
Websites
http://en.wikipedia.org/wiki/Software (Accessed date: 2-1-2014)
http://en.wikipedia.org/wiki/Software_bug (Accessed date: 2-1-2014)
http://asq.org/learn-about-quality/quality-assurance-qualitycontrol/overview/overview.html (Accessed date: 2-3-2014)
http://en.wikipedia.org/wiki/Software_quality (Accessed date: 2-5-2014)
http://en.wikipedia.org/wiki/Software_quality_assurance (Accessed date: 2-7-2014)
http://searchsoftwarequality.techtarget.com/tip/Manual-testing-vs-automated-testing-Adecision-point (Accessed date: 2-4-2014)
http://www.diffen.com/difference/Quality_Assurance_vs_Quality_Control
(Accessed
date: 2-2-2014)
http://www.softwaretestinghelp.com/types-of-software-testing/
(Accessed
date:
2-6-
2014)
http://www.telerik.com/automated-testing-tools/ (Accessed date: 5-2-2014)
http://www.tutorialspoint.com/software_testing/testing_types.htm (Accessed date: 2-82014)
65
Annexes:
Annex 1: Questionnaires
Name of the organization :
Name of Respondent
:
Position of Respondent
:
Date of interview
:
1. Do you have software testing or quality assurance department?
a. Yes
b. No
If yes, how many software testers or quality analyst do you have?
a. 1
b. 2
c. 3
d. 4
e. More than four
If No, who tests the quality of the software?
a. Project Manager
b. Designer
c. Developer
d. Others
e. None of the above
2. What is the testing process you follow?
66
a. Automatic
b. Manual
c. Both
3. Which testing types or methods you follow?
a. Black box testing
b. White box testing,
c. Grey box testing
d. All of the above
4. Do you make Test Plan while testing the software?
a. Yes
b. No
5. If yes, what are the benefits of using “Test Plan”?
a. Keep track of test
b. Store information in document and serves as a guide
c. Can be reused if regression testing occurs
d. All of them
6. If no, what are the reasons for not making “Test Plan”?
a. Not necessary to make Test Plan
b. Lack of knowledge or skill about Test Plan
c. Lack of time to make Test Plan
7. What are the contents/titles that you have in “Test Plan”?
a. Features to be tested
b. Features not to be tested
c. Browsers compatibility
67
d. Item pass/fail criteria
e. Other …..
8. Do you use automated software testing tools?
a. Yes
b. No
9. If yes, what are the benefits of using automated software testing tools?
a. Fast Testing
b. Save time, cost, effort
c. Automated regression test
d. Motivation to testers for new technology
e.
All of them
10. If no, what are the reasons of not using automated software testing tools?
a. Couldn’t afford (Budget constraint)
b. Lack of skilled human resource
c. Lack of knowledge about automated software tools
d. All of the above
11. Which software testing tools have you used currently?
a. Selenium
b. JMeter
c. Load Runners
d. Others
12. Does the company follow agile methodology like SCRUM while developing
software?
68
a. Yes
b. No
13. If Yes, Has QA (Quality Analyst) involved in daily scrum with development
team?
a. Yes
b. No
14. In your opinion, what should be done for better development of QA field?
…………………………..
15. In your opinion, what are the problems you face while not making test plan?
………………………………
16. Chances of error/bug while using “Test Plan”. Give your opinion (√) based on
experience
Features
Below 50%
Login form validation
Registration
Header (Slide show)
Footer (Links)
Social Links
Category Display
Roll over effect
Breadcrumb
Contact Form
Email Templates
Pagination Display
69
More than 50%
100%
17. Chances of error/bug without using “Test Plan”. Give your opinion (√) based on
experience
Features
Below 50%
Login form validation
Registration
validation
Header (Slide show)
Footer (Links)
Social Links
Category Display
Roll over effect
Breadcrumb
Contact Form
Email Templates
Pagination Display
70
More than 50%
100%
Annex 2: List of respondents and organizations
SN
Name of Respondents
Name of organizations
Position of respondents
1
Suresh Mahto
EBPearls
Quality Analyst
2
Kabita Dhimal
Ebpearls
Developer
3
Kabindra
Miracle Interface
Software Quality
Analyst
4
Pratima Lingden
Miracle Interface
QA
5
Nisha Bhagat
Miracle Interface
QA
6
Narayan Koirala
Eminence Ways Pvt.
Managing Director
Ltd.
7
Newton Suwal
Javra Software
Quality Control
Engineer
8
Sujan Amatya
Braindigit IT Solutions
QC Incharge
Pvt Ltd
9
Soni Baidya
EbPearls
Quality Analyst
10
Nita Poudel
EbPearls
Quality Analyst
11
Sujit Shrestha
SPI Private Ltd.
Senior Test Engineer
12
Sulochana Paudel
F1soft International
php developer
13
Sajani Maharjan
F1soft International
QA Engineer
14
Enkisha Chhetri
ebPearls
QA
15
Rakesh Shrestha
f1soft Int'l Pvt. Ltd.
QA Engineer
16
Rajesh Singh
Longtail e-media Pvt.
Quality Assurance
71
SN
Name of Respondents
Name of organizations
Position of respondents
Ltd
17
Dipesh Chand
F1soft International
QA Engineer
18
Rajan Maharjan
Peace Nepal DOT Com
Manager
19
Sameer Pokhrel
Verisk IT
Software engineer
20
Sabina
Javra
Sr. QC
21
PRABIN SAPKOTA
JAVRA
SOFTWARE TEST
ENGINEER
22
Ramesh Kunwar
Yomari inc.
Software Developer
23
Suraiya Pradhan
SPI
Test Engineer
24
Shrijana Tandukar
Deerwalk Services
Software Quality
Assurance Engineer
25
Prakash Malla
SPINepal pvt ltd
Test Engineer
26
Tripti pant
Avionte
QA
27
Dileep Dawadi
PCS
QA
28
Neeta Phuyal
EBpearls
Software Quality
Analyst
29
Sabin Maharjan
Auxfin Nepal
Technical Manager
30
Sujan Shrestha
IT Himalaya
Country Manager
31
Uttam Shrestha
Cross Over Nepal
Manager
72