Amdocs Testing Factory
Amdocs Testing Factory | 2
Contents
Introduction..............................................................................................3
Telco & CSP Challenges in Testing...........................................................4
The Amdocs Testing Factory Model.........................................................5
Set-up and Transition............................................................................6
Governance..........................................................................................8
Engagement model............................................................................. 10
About Amdocs Testing........................................................................... 11
Amdocs Testing Factory | 3
Introduction
The success of a new system or application is directly
influenced by the diligence with which it was tested.
Communication Service Providers are challenged to
maintain a delicate balance between their external goals
of speedily delivering upon their subscriber’s demands and
internal goals of reducing costs while improving software
quality.
This white paper will present the Amdocs Testing
Factory, a new concept in Software Testing that delivers
Communications Service Providers with a ready framework
to help them begin testing quickly as well as ensuring high
quality and efficiency while mitigating costs. This paper
offers an overview of the different stages that comprise this
concept, what each includes, roles and responsibilities of
parties involved as well as how the concept is governed.
Amdocs Testing Factory | 4
Telco and Communications Service Provider
Challenges in Testing
The constantly changing world of Telecom forces
Communications Service Providers (CSPs) to quickly
adapt to change. The market and specifically the CSP’s
subscribers demand quick, new business solutions and
smart applications that improve their experience and fulfill
the needs of their lifestyle. Regulatory bodies also demand
changes and adjustments in applications at short notice. All
these factors add immense pressure upon CSPs to quickly
adapt their systems and applications which, today, require a
significant number of changes to fulfill demands.
TESTING EXPERTISE
Finding:
• Skilled & experienced
testers
• Testers that understand
telecom business
processes
EFFICIENCY
Maintaining balance
between:
• Testing adequately
• Meeting deadlines
•L
owering costs
The challenges outlined call for a more efficient way to
develop and test systems and application solutions while
ensuring higher quality. CSPs are hard pressed to reduce
time to market and ensure their systems and applications
run like clockwork, without causing severe business
disruptions.
To effectively support the service provider’s business, Amdocs
has developed the Testing Factory concept that offers integrated
processes, methodology, governance and tools. This concept
aims to reduce costs and efficiently bring the system or application
to a state of readiness in the production environment, with lowest
possible number of defects.
TELECOM FOCUSED
TESTING INNOVATION
Focus of testing vendors
spread over too many
industries
Lack of vendors focusing on
telecom tools & techniques
• Automation
• Test design
• Test case reuse
Figure 1: CSP challenges in Testing
CONSTANTLY EVOLVING
TRENDS
Reactive approach to trends
Keeping pace with the
industry
• Mobile testing
• Agile approach
Amdocs Testing Factory | 5
The Amdocs Testing Factory Model
Amdocs perceives the Testing Factory as the industrialization
of software testing. It is essentially a software product line
that configures testing methodology, processes, tools,
patterns, frameworks and guidance, designed to test
specific kinds of systems and applications.
The Amdocs Testing Factory operational model provides a
production line with well-defined processes and activities to
meet the needs of the Communications Service Provider.
The model essentially delivers all the necessities required to
begin testing quickly and effectively. It is similar to a factory
or plant where the floor is populated with all the equipment
required to begin performing its intended function effectively.
Following is an overview of the key building blocks for
successful Testing Factory engagement:
• Set-up and Transition Phase
• Governance Model
• Engaging Testing Factory Services
Figure 2: Amdocs Testing Factory Model
Amdocs Testing Factory | 6
Set-up and Transition Phase
To ensure high quality integration into the Communication
Service Provider’s system and ensure consistency with
project goals, Amdocs leverages a set-up and transition
plan that draws upon our experience in this field to
effectively gather knowledge from the CSP’s team or thirdparty vendor.
Key phases and main activities of the plan can be seen in
the figure below. This generic plan is fine-tuned to suit the
specific Communication Service Provider’s needs during the
process of transition.
Program management
Governance, Contract, SLA and Change Management
prepare
Identify
involved
parties
Detailed
transition
plan
Build
management
and PMO
teams
Initiate
preparation
activities
Establish
forums and
committees
Stafffing plan
and initiate
recruitment
contract
LOI
LOI
High-level
transition
plan
Interview and recruit
candidates on- and
off-shore
transfer
Organize
gathered
information
Set SLA
procedures
Prepare offshore
facilities
Set up
testing tools
Create KT
materials
Prepare
infrastructure
Adjust / update
work
procedures
Establish work
procedures
Prepare onsite
facilities
Initiate
knowledge
transfer
handover
Handover
responsibility
Establish an
improved
testware baseline
Knowledge transfer (OJT,
training, documentation)
Start joint execution and
shadowing
Figure 3: Set-up and Transition Phase
Reverse
shadowing
transition completion (tbd)
plan and design
Amdocs Testing Factory | 7
The Set-up and Transition phase comprises four subphases:
• Plan and design – Amdocs assigns a transition team
during this sub-phase. A transition progress report of
expectations is also defined and includes status, risks,
and AI tracking. Timelines and milestones are reviewed
and agreed upon at this stage.
• Prepare – The primary focus of this phase is the creation
of the transition plans that comprehensively cover all
aspects of the transition.
• Handover – The purpose of this stage is to smoothly
handover all testing findings and recommendations to
the permanent testing team.
During this stage all activities are planned in detail according
to their phase. This before takes between 1 and 3 months
depending on the number of systems or applications
included in the scope.
An example timeline of the Set-up and Transition phase can
be seen below.
• Set-up / Transfer – This stage includes actual execution
of all transition plans. Another key event that takes
place here is the definition of all activities leading to a
successful handover to the permanent testing team.
month 1
week 1
week 2
week 3
month 2
week 4
week 1
week 2
week 3
month 3
week 4
week 1
week 2
week 3
week 4
Plan and Design
Transition
complete
Prepare
Set-up / Transfer
Handover
Figure 4: Set-up and TransitioN Phase Time Line
Amdocs Testing Factory | 8
Testing Governance
Governance Roles and Responsibilities
The implementation of Testing Governance needs
significant management of people, process and technology
with transparent communication and information. To give
the Communications Service Provider full control of the
engagement, Amdocs Testing Factory includes a set of
well defined roles & responsibilities, a designated testing
organization structure and strict KPIs and SLAs that make
it possible to control, assess quality and manage progress
of System Testing.
As part of our testing methodology and approach, Amdocs
recognizes that the success of the Acceptance Test
phases is mutually dependent on both Amdocs and the
Communications Service Provider. The table below presents
high-level roles and responsibilities between both parties.
TASKS
Transition period
AMDOCS
• Provide personnel to execute transition and knowledge transfer of
systems involved in this program
• Lead & perform
Test strategy & plan • Lead & perform
Test design &
detailed design
Test data
• Review & approve
• Lead the design and
detailed design
• Provide business requirements
• Design test cases and test
scenarios
• Participate
• Lead & perform
• Participate
• Allocation of business experts to participate in design and design reviews.
• Provide inputs from external systems
• Test execution management
Test execution
SERVICE PROVIDER
• Perform test execution
• Allocate super users and system operators to participate in the test
execution
• Execute on satellite system onsite
Quality
management
IT support
• Coordinate and define
• Review & approve
testing process and activities
• Request test environment
based on the test scope
• Release management (planning and provide scheduled releases and
drops, emergency releases)
• Test environment set up and maintenance.
• Provide production data including reference data and any other data that
is defined in the test execution plan
Summary & LL
• Lead & perform
• Participate
SLA easurements
and reporting
• Participate
• Lead & perform
Remote access
• Participate
• Provide remote access to test environment for execution of test cases
Table 1: Example of Governance Roles and Responsibilities
The implementation of Testing Governance needs
significant management of people, Processes and
technology with transparent communication and
information.
Amdocs Testing Factory | 9
Amdocs Testing Factory Organization Structure
We have defined a sample organization chart to cover the
Testing Factory model as seen below. The team is divided
into streams, such as Functional, Non-functional, Defect
Management, Data/Environments and Automation.
Test Manager
TMO
Functional
testing Team
Performance
Team
Defect
Management
Test Data and
Environments
Automation
Team
Figure 5: Amdocs Testing Factory organization – part of Governance structure
Key Performance Indicators (KPIs) and Service Level
Agreements (SLAs)
An SLA is a negotiated agreement between Amdocs and
the Communications Service Provider at the beginning of
the project that defines the types of service included and
their levels of availability. It is also part of our implementation
approach to define KPIs to measure activity performance.
As part of our testing methodology
and approach, Amdocs recognizes
that the success of the Acceptance
Test phases is mutually dependent
on both Amdocs and the Communications Service Provider.
Tools
At Amdocs, we invest significant efforts in innovation to
ensure our tools and methodologies deliver real value
through superior software quality, fast time to market and
lower cost of ownership. Our developers and testers have
created the best tools to comprehensively address all
types of tests and can provide these tools according to the
Communication Service Provider’s request. Amdocs uses
an HP Platform including Quality Center, Load Runner and
other tools.
Amdocs Testing Factory | 10
DEVELOPMENT
TEAM 2
GOVERNANCE
Test & Quality Management
DEVELOPMENT
TEAM 1
APPROVALS
DEFECTS
FIXED DEFECTS
DAILY REPORTS
KPI REPORTS
QUALITY
Execution
Preparation
Close/Lesson
Learned
ESTIMATION
Amdocs Test Management (TMO)
REQUEST
BUSINESS
TEAM
Design
• Evolutionary Maintenance: Development of new features
or new modules in existing systems with or without
changing the database.
Plan
• Project (New Application): Development and implementation of a new system or a major systemic change.
The diagram below shows a typical macro process framework that covers the entire Testing Factory service. In green
are the many areas requesting services from the factory,
such as Business, IT Development and Infrastructure. The
Amdocs Testing Factory is shown in gray. Inside the factory,
one can see all processes and phases supported by strong
Amdocs Testing Methodology.
Test Request
Management
The most common types of requests that generate the
requirement for testing services are as follows:
A generic version of the process or flow is presented in the
figure 6 below. As Amdocs gains a greater understanding
of the Communications Service Provider in the Set-up and
Transition phase, this flow is adjusted and customized to
effectively address the provider’s specific needs.
Defect
Management
Amdocs performs an initial estimation of the service
required in accordance with the specifications received from
the business area. The estimation process could be based
either on the number of business scenarios or test cases,
and is aligned with the Communications Service Provider in
the first stage of the project.
• Corrective Maintenance: Addressing specific functionality
in an existing systems either with or without changing
the database.
Reports
Engaging Testing Factory Services
Project Management
(Working planning, monitoring,
reporting, KPI’s and risk management)
Requirement Management
(Capture and maintain scope changes
during all project phases)
Environment Management
(Requirement definition, validation,
version management)
Tools
service request
Infra Team
Figure 6: Example of Amdocs Testing Factory
Macro Process Framework
Amdocs Testing Factory | 11
About Amdocs Testing
Amdocs Testing provides comprehensive testing services
including Acceptance Test management and planning, test
design and execution to confirm full system compliancy with
our customers’ business and operational requirements.
After many year of experience working on complex test
projects, Amdocs Testing developed the Testing Factory
service offering, combining an innovative methodology,
people, processes and governance for effective and efficient
test design and execution that delivers value at every stage.
Amdocs Testing is a dedicated unit of Amdocs Consulting
Division and offers the advantages of a highly experienced
team of industry professionals leveraging a structured, fieldproven testing methodology supported by procedures,
automated tools, testing scripts and libraries.
Over 2,000 SI experts
located across 30+ locations
Up to 50% faster execution time
Manage & operate 25+ concurrent
Telecom focused SI projects
Up to 40% reduction in testing TCO
Seamless integration between
offshore, onsite, customer
and partners
Amdocs
Testing
Up to 50% less defects
in production
Experience with world’s
most complex testing projects over 25,000MM
3 patent pending applications
Figure 7: Amdocs Testing in a nutshell
Amdocs Testing Factory | 12
QUALITY
TIME TO VALUE
COST
Unparalleled Quality –
Up to 50% less defects
in production when Amdocs Testing
conducts Acceptance Tests
Up to 50%
faster execution time
for combined systems
Reduce testing
TCO up to 40%
for combined systems
Up to 60% faster
for Amdocs systems
Up to 60% lower for Amdocs
systems
(Shama & Ben Cnaan, 2013;
Kenett & Ben Cnaan, 2013)
(Capers Jones, 2011;
Ross Report, 2012)
(Capers Jones, 2011;
Ross Report, 2012)
Figure 8 : Amdocs Testing delivers value in terms of quality, time to value and total cost of ownership
Amdocs Testing Factory | 13
What the Analysts Say
Nelson Hall
RANK
VENDOR
Est’d sts revenues 2012 (sm)
Est’d sts market share 2012 (%)
1
Amdocs
300
13
2
Wipro
240
10
3
Mahindra IT
200
9
=
Accenture
200
5
Capgemini
170
7
=
CGI (PF)
140
6
=
TCS
140
6
8
IBM
120
5
9
HP
70
3
10
AtoS
30
1
Total top 10
1,640
70
Other
~660
30
Total
2,300
100
*G
lobal Software Testing Services
9 Market
in Telecom Sector 2012 - 2016
Figure 9: “Software Testing and Assessment Forecast 2013”, Nelson Hall
TechNavio
“Amdocs is the leading firm in the Global Software Testing
market... it excels in providing software testing services in
the Telecom sector.”
ACCENTURE
9-11%
HP
9-11%
IBM
10-12%
OTHERS
52-54%
AMDOCS
15-17%
Figure 10: Global Software Testing Services Market
in Telecom Sector 2012 – 2016, TechNavio
Amdocs Testing Factory | 14
Amdocs Testing Offering
Amdocs Testing offers a comprehensive testing portfolio
that enables you to speedily launch new services of superior
quality. Our offering includes:
Core Testing Services
Testing Management Services
Specialized Testing Services
Test Automation Services
Functional
Non-Functional
System Integration Tests
Acceptance Tests
End to End Tests
Performance Tests
Operational Tests
Test
Environment
Management
Test
Assessment
Test
Management
Office
Test
Data Management
Bill to Bill
Revenue
Assurance
Tests
Data Warehouse
Business
Intelligence Tests
Mobile
Application
Tests
Functional
Web &
e-Commerce
SOA & Middleware
Figure 11: Amdocs Testing Product Offering
Amdocs Testing Factory | 15
What Our Customers Say
Amdocs Team… without your help this
would not be possible. Please continue
with the efforts, which without doubt
add value and bring benefits to our
company.
….This was definitely the BEST upgrade
we’ve done in all my years and I really, really
appreciate the care and attention that you
took to ensure this significant work went in
without any disruption to our business.
Telefonica Chile QA Lead – Test Factory
Senior VP, Tier 1 North American Service Provider
We’re very happy with the level and high
quality of service Amdocs Testing has
delivered. They’ve worked very hard
to ensure deployment occurs on time
while maintaining quality. No matter the
deadline, we’re always sure, Amdocs
Testing will deliver.
Bevan Williams,
Executive: ITS Solution Delivery Center,
Telecom South Africa
Amdocs Testing Factory | 16
Contact Us
Amdocs Testing Contacts:
For more information about Amdocs Testing Factory, please
contact the Pre-sales Manager responsible for your region.
CALA
Luiz Tenorio
luiz.tenorio@amdocs.com
+55 11 99236 9249
EMEA
Hitesh Joshi
hiteshj@amdocs.com
+44 07920847707
APAC
Manoj Jasrotia
manojj@Amdocs.com
+91 9881391905
North America
Ori Cohen
oric@amdocs.com
+1 647 449 7589
GLOBAL
David Chernitzky
davidche@amdocs.com
+1 416 702 8867
About Amdocs
For more than 30 years, Amdocs has ensured service providers’ success and embraced
their biggest challenges. To win in the connected world, service providers rely on
Amdocs to simplify the customer experience, harness the data explosion, stay ahead
with new services and improve operational efficiency. The global company uniquely
combines a market-leading BSS, OSS and network control product portfolio with valuedriven professional services and managed services operations. With revenue of $3.2
billion in fiscal 2012, Amdocs and its 20,000 employees serve customers in more than 60
countries. Amdocs: Embrace Challenge, Experience Success. For more information,
visit Amdocs at www.amdocs.com.
For the most up-to-date contact information for all Amdocs offices worldwide, please visit our
website at www.amdocs.com/corporate.asp
Copyright © 2013 Amdocs. All Rights Reserved. Reproduction or distribution other than for intended purposes is prohibited, without the prior written consent of Amdocs. Amdocs owns or has rights to use trademarks or trade names in conjunction with the sale
of our products and services, including, without limitation, each of the following: Amdocs™, Bridgewater Systems™, ChangingWorlds™, Clarify™, Cramer™, CES™, Collabrent™, DST Innovis™, Ensemble™, Enabler™, Intelecable™, Intentional Customer
Experience™, JacobsRimell™, jNetX™, MX Telecom™, OpenMarket™, Qpass™, SigValue™, Streamezzo™, Stibo Graphic Software™, Xacct™, Aging in Place™, Embrace Challenge, Experience Success™, and Quality Consumption™.