Outsource Testing
Parul Gupta1 and Tripti Sharma2
Ideal institute of Technology, Ghaziabad, UPTU, Lucknow
1parul.bmiet@gmail.com, 2tr_sharma27@yahoo.co.in
Abstract: As organizations expand globally, demands on IT
departments increase exponentially. To meet these demands,
many companies outsource IT services offshore. The traditional
focus has been on cost savings due to the increased availability of
lower cost skill sets in global locations. Companies often
transition offshore aggressively to capture cost savings quickly in
response to declining margins. Outsourcing firms offer other
advantages and benefits that may have been previously
overlooked, including cost savings. This paper provides an
overview of the flaws of global outsourcing and how we can
overcome those flaws. The common flaw in global sourcing
programs are: excessive focus on costs is commonly undermining
global sourcing relationships and wasting important
opportunities for companies to improve quality of service,
increase productivity and shorten time to market.
Keywords: Outsourcing, global IT outsourcing, off shoring,
cost saving, quality of service.
1. INTRODUCTION
Outsourced testing is on the verge of a major boom. Quality is
expensive for organizations to invest in, and outsourcing is
making inroads in several discrete areas of testing. Companies
should look to outsource testing as a quality gain, not just a
cost savings. Higher application quality, greater flexibility in
staffing, and, yes, reduced staffing and tool carry costs are the
primary benefits provided by testing outsourcing. However,
organizations must realize that outsourcing can result in the
loss of customer private information, increased collaboration
time between development and testing, and may require
process changes to core development activities.
In this paper we recommend following Forrester's testing
outsourcing decision path to determine what testing to
outsource, focusing on certain core competencies when
choosing an outsourcer. IT's continuing push towards cost
reduction has led to the burgeoning of the outsourcing
industry.
II. BENEFITS OF OUTSOURCE SOFTWARE TESTING
Outsourced Testing is a surprisingly wide-ranging subject
area, as there are multitudes of testing tasks that can be
carried out by a third party, from test strategy development
to penetration testing.
While acquiring testing services from outside contractors
isn’t unusual, its importance has increased considerably, as
have the importance and maturity of testing itself. This is
shown by the fact that the test outsourcing market is no
longer driven solely by independent testing companies. All
the major outsourcing companies now offer testing services
as well. Through outsourced testing, the following main
benefits can be achieved:
Speed up testing:: Test execution, located at the end of the
software development life cycle can be a bottleneck,
especially if development is already late. To make the most
of the time available, a thorough test preparation and readily
available testing staff are needed. These resources can by
procured by an outsourcing provider. Also, in some cases
testing can be sped up by taking advantage of the time zone
differences, testing at night what has been developed or
debugged during the day.
Improve quality: The need to adhere to a formal testing
process and to provide a proper test basis in the form of
well-documented and stable requirements will result in
finding more bugs, therefore improving the quality of the
end product. In addition, some of the bugs will be found
earlier, when analyzing the requirements or reviewing the
test cases.
Reduce testing costs: The main driver for outsourcing is
often cost reduction, by moving operations to low-cost
countries. Also in testing, costly IT and business resources
can be freed from repetitive and time-consuming test
activities as, for example, regression testing. The improved
software quality can also lead to cost savings.
Trends in the IT industry for the past few years demonstrate
that companies are seeking to reap the benefits of outsourcing
IT application development. Countless examples exist of
corporations outsourcing their software development needs,
call centers, data centers, hardware purchases, system support,
help desks, etc. Testing is another area of IT that is rapidly
being outsourced.
Companies are outsourcing test case executions, test script
automation, and test case development tasks to offshore based
companies, independent contractors, niche QA companies, and
system integrators. Outsourcing approaches vary widely. Some
companies outsource manual testing needs, while other
companies outsource testing tasks.
Regardless of the approach, outsourcing a company's testing
needs can militate strongly in a company's favor by lowering
costs while delivering reliable testing results. The following
1
AKGEC JOURNAL OF TECHNOLOGY, vol.1, no.1
sections explore the benefits of outsourcing testing.
Minimize Costs: Hiring full-time testers involves providing
them with company benefits and training which is a costly and
time-consuming venture. Furthermore, recruiting costs for
qualified full-time testers are additional. Companies looking to
cut costs should seriously look into outsourcing as a cheaper
alternative. The cost of employing a tester from an outsourcing
firm with specific testing experience in a given industry or
with a particular test tool is far cheaper than the cost of hiring
in-house testers.
Outsourcing firms lower testing costs by offering testers and
testing solutions at a percentage of the cost of hiring full-time
testers. Furthermore, outsourcing firms have libraries and
repositories of automated tests that can be leveraged or
recycled for other testing needs, which also lower the costs of
various test automation tasks.
III. WHAT TO OUTSOURCE
Outsourcing resource- and time-consuming test activities,
not only allows for greater capacity, scalability and
flexibility, but also has the additional benefit of an
independent validation. By that I mean testing that is not
carried out by the development team itself (typically unit &
integration testing), but by a separate test team.
intense test types like load & performance, usability
and security are best outsourced to a specialist
organisation, on a case-by-case basis.

Test Activities: Defining what activities within the
test process should be outsourced (e.g., test
planning, specification, execution and reporting)
requires a strategic decision on how much control
and knowledge is given away. It can range from
test execution only, to performance of the whole
process.
Not only do you have to decide on what parts of testing
to outsource, but also which projects are most suited for
it. If it is the first release of a totally new application, it
may not be ideal to outsource its testing. However, if
the project is an enhancement version of a stable
product, with existing test cases, it would be a very
good candidate. Possible selection criteria include:
technical complexity in-house know-how maturity of
requirements and test cases availability of test data .
A typical example of such an analysis is shown in the
figure 1.
But greater and/or cheaper capacity isn’t the only reason for
outsourced testing and a strong case can be made for
outsourcing testing tasks that require specialist know-how,
for which you don’t have the skills or only need on a oneoff basis [5]. When deciding on what parts of testing to
outsource, you will have to look at it from different angles,
mainly at the dimensions of test levels, test types and test
activities:


Test Levels: Low-level testing (Unit & Integration
test) is generally carried out by the developers
themselves. If development is outsourced, these
activities are outsourced with it. System testing on
the other hand can be performed by an independent
test team and is therefore an excellent candidate for
test outsourcing. Finally, Acceptance testing
requires business know how (for user acceptance)
and a production-like test environment. It is
therefore difficult to outsource.
Test Types: Generally speaking, all types of testing,
both functional and non-functional, can be
outsourced. By its nature, regression test is a good
candidate for cost saving, because it involves
regular repetition and test automation. Know-how
Fig.1
IV. HOW TO OUTSOURCE
Outsourcing is never a simple task and many such efforts
will fail because they haven’t been managed thoroughly.
Unrealistic expectations, the selection of the wrong partner
and communication problems are common reasons for
failure. It is therefore imperative to use a well-thought out
2
OUTSOURCE TESTING
and planned procedure, that makes use of best practices and
actively addresses the risks involved.
In the first phase, graph (b), you have to look at the feasibility
of your outsourcing effort. Beside the decision on what to
outsource (as discussed previously), this includes the
assessment of the maturity of your test process. Outsourcing is
seldom a solution for internal process problems.
On the contrary, the problem is outsourced and stays
unresolved or even becomes accentuated. Identify and address
these deficiencies first. Outsourcing may still be possible but
might involve additional costs [5].
This leads us to the calculation of a rough business case.
Consider different outsourcing options, from bringing in
external testing experts to setting up an offshore development
center, to find the most cost-effective alternative.
If you decide that it is all worthwhile, you can start the
search for a suitable outsourcing partner by developing a
request for proposal and inviting offers. It is important to
look not only for testing know how, but also for product and
technology knowledge. This will result in a steeper learning
curve, allowing you to become productive sooner.
up the infrastructure. Start with a small number of projects, to
gain experience. All going well, the transition phase will lead
you to steady state operation, typically within 6 to 9 months.
Important Aspects for Successful
Outsourcing:
It is
recommended that one should consider the following points
when deciding whether to outsource a project or application
for testing.
First, consider the nature of the project or application that you
are considering outsourcing. If the project is an enhancement
version of a stable product, with existing test scripts that could
be used by the outsourcing partner, it would be ideal to
outsource.
However, if it is the first release of a totally new application, it
may not be ideal to outsource the testing. For example, if
there are major problems with the application, testing is not
going to solve them, and if your testing partner encounters a
large number of serious errors the test coverage will probably
be lessened.
Last, carefully consider what you expect to achieve by
outsourcing. In all likelihood, outsourcing will never (and
probably should never) completely do away with the necessity
to have in-house expertise, so you should consider the loss of
knowledge and hands-on experience that your test team will
lose.
V. FLAWS IN OUTSOURCING
Not many outsourcing collaborators realize that the seeds for
outsourcing failure is often sown at the time of due diligence.
Often spending very little time on due diligence in the contract
negotiation process in a bid to 'hurry up' the negotiations leads
to this monumental pitfall. Five areas where outsourcers
should apply more thought before taking the plunge:
Unrealized cost savings: Most businesses push work overseas
in the hope of cutting labor costs. An application maintenance
worker in India, for example, earns about $25 per hour,
compared with $87 per hour in the U.S., according to Gartner.
But businesses make a mistake by looking at salaries alone.
Fig.2 showing different phases of the procedure of ‘How to
outsource’.
The selection of the provider will then kick off the transition
phase. It is all about paper work and getting to know each
other. While the lawyers do the contractual work, start
defining the governance and split of work, as well as setting
to gain experience. All going well, the transition phase will
Hidden
expenses
for
things
like
infrastructure,
communications, travel and cultural training take a bite out of
the wage differential. What's more, planning and start-up costs
are high, so offshore deals lasting less than a year may not pay
off at all, and savings from longer-term deals will emerge
slowly.
Loss of productivity: Staff at an offshore service center
probably won't be as productive as internal staff back at home,
at least not initially. Gartner offers several reasons:
3
AKGEC Journal of Technology, vol.1, no.1
Staff turnover can be high in competitive off-shoring markets
such as Bangalore, India, which also means programmers there
may be new and inexperienced. And service centers overseas
struggle with ambiguities in the work they are assigned and
shifting directives. Sending jobs overseas can also lower
morale at home, creating a drag on output.
Poor commitment and communications: Senior executives
often drift out of the picture once a deal is signed. But they
need to stay engaged to keep morale high and strategy on
track. And good communications among all parties is
paramount. Projects, goals and expectations have to be defined
clearly and in detail. On the home front, managers need to
explain clearly why work has been sent overseas and what
benefits are expected.
Cultural differences: Communication styles and attitudes
toward authority vary from region to region and those
differences can cause problems. In some cultures, questioning
authority is considered disrespectful, so a team may push
ahead with a given plan even if they see a better approach.
Offshore
should get expert advice about a local culture,
provide cultural training and even arrange exchanges among
staff on both sides.
Lack of offshore expertise and readiness: Some organizations
make the leap before they are ready. Offshore need to get
everything in place internally and secure the support of key
stakeholders in the company before launching a project. They
should also figure out the risks and how to mitigate them.
Here are the reasons why due diligence is critical. Outsourcing
is akin to acquisition. IT Outsourcing (ITO), for example,
essentially involves the acquisition of the client's IT
department (or portion of it) by the vendor. The vendor,
selected by the client as the result of a competition, acquires in
some cases the physical assets, such as computers, peripherals,
networks, etc., assumes the responsibility for third party
licenses and procurement functions, and usually hires the
employees from the client.
This is exactly what happens during an acquisition. The only
difference is that the client enters into a time bound contract
with the vendor to provide back the services previously
delivered by the client's IT department. As with an acquisition,
outsourcing requires sound due diligence and the process used
should not be any different that those used in an acquisition of
company.
Due diligence is done to confirm the baseline initially set by
the client during the RFP phase and subsequent negotiations.
Pay Attention to Dependencies and Quality of Current Test
Artifacts: A successful outsourced automation requires that
project dependencies are well understood by all stakeholders.
Access of application from offshore, access to
testers/developers/ SMEs on test cases, Access to reviewers
and code acceptance people are few dependencies that one
needs to track as part of project [6]. If automation is on the
basis of existing manual test cases- make sure that these are
detailed enough and available in a form that can be sent across
to vendor offshore team.
Offshore Outsource Improvement
Many organizations have offshored their software testing
with varying degrees of success. Some have worked well.
Many others have struggled and it is not uncommon to find
organizations that have onshored their offshore resources or
are simply having difficulty realizing the value from these
outsource deals that the business case outlined.
We have provided our offshore testing improvement service
to a number or organization across different sectors to help
them improve their results and migrate back offshore their
software testing activities.
REMARKS
What is the quintessence? Outsourced testing can speed up
testing, improve quality and reduce costs. But, no big surprise,
it won’t fall into your lap – it requires a well-managed effort
before, during and after the transition. Firstly, define the
channels of communication. Appoint a person from your
company, to act as the first level of contact between your
company and the outsourced partner, who will deal directly
with an agreed specified person on their side (ideally the test
manager for the project).
These two people are the first-level team for resolving
problems as they arise, and additionally your representative
also provides and archives all information passed to the
outsourcing partner.
Secondly, agree escalation procedures, and specify to whom
the problem is to be escalated to - on each side.
Set out milestones – but beware when setting milestones, that
YOU are the ones who state whether the outsourced testing
company has reached the milestone or not – for example, if
one milestone is the production of the test plan, if you are not
happy with the test plan then you must be in a position to state
that they have NOT reached the milestone.
Otherwise you could end up in a position whereby they gave
you a test plan that you considered inadequate, yet they are
looking for payment for reaching the milestone. To cater for
4
OUTSOURCE TESTING
this, and other concerns, it is vitally important to specify
formal review points – stages at which progress can be
measured, and where issues can be highlighted.
V. REFERENCES
[1] “Outsource Testing For Quality Gains, Not Just Cost Savings” by
Uttam Narsu on July 12, 2004 in Forrester magazine.
[2] ISTQB Standard Glossary of Terms used in Software Testing
http://www.istqb.org/
[3] “Outsourced Testing–Friend or Foe?” Published: Oct 08 2008 by
martinig via methodsandtools.com.
Tripti Sharma obtained Bachelor of
Engineering in Computer Science and
Engineering stream from GLA Institute of
Technology & Management, Mathura, in
2003. Then she obtained Master of
Technology in the same stream from U.P.
Technical University, in 2009.
She has seven years of experience in the field
of academics. Has published over ten
research papers
[4] “The effects of global outsourcing strategies on participants
attitudes and organizational effectiveness” In International Journal
of Manpower 2000 , Volume: 21 by Dean Elmuti and Yunus
Kathawala
[5] “Making the decision to outsource human resources”
In Journal - Personnel Review:
2009, Volume: 38 by Jean Woodall, William Scott-Jackson.
[6] Chapter 16” Moving People or Jobs? A New Perspective on
Immigration and International Outsourcing” in Book Series:
Frontiers of Economics and Gloabalization 2008, Volume: 4 Page:
317 – 327. Chapter URL:
http://www.emeraldinsight.com/10.1016/S1574-8715(08)04016-5.
[7] “The Benefits of Outsourced Testing”,
http://members.tripod.com/bazman/index.html.
[8] “Primer on Outsourced Software Testing “by: Thinksoft ,
published on sep 02,2005.
Parul Gupta is Senior Lecturer in Ideal
Institute of Technology, Ghaziabad with six
years of teaching experience. While teaching,
she presented and published various national
and international papers. Currently, pursuing
M. Tech (IT) from Karnataka University,
Mysore.
Obtained B.E (IT) from Maharishi Dayanand
University, Rohtak. She is a certified
Software Tester from Webtek Labs Pvt Ltd,
New Delhi.
Completed certification in Web designing. Currently she is carrying out
research in Software testing for the upcoming conferences.
5