AMEX Pega Testing –
Moving towards TCoE
6th Sept. 2011
©2011, Cognizant
| ©2011, Cognizant
Image
Area
Agenda
 Cognizant’s Pega Testing footprint in AMEX
 Why move to a TCoE?
 Core-Flex model of Resourcing
 Appendix
 Case studies
©2011, Cognizant
| ©2011, Cognizant
2
Cognizant’s Pega Testing
Footprint in Amex
3
| ©2011, Cognizant
Cognizant’s footprint of PEGA engagements with AMEX
Delivered Engagements
•
•
•
•
•
•
•
•
ECM EMEA Release 1 & Release 2
ECM EMEA R1 & R2 Reporting
B2B RCubed
Global Merchant Services
Online Merchant Services
iCruse
ECM Disputes – CRs
GCM Acquirer - CRs
Services provided
•
•
•
•
•
•
•
•
•
•
Functional Testing
Integration Testing
Report Testing
Test Data identification
Regression Testing
Automation Testing
Performance Testing
User Acceptance Testing
Business Acceptance Testing
E2E Test Management
| ©2011, Cognizant
Ongoing Engagements
•
•
•
•
•
•
GCM GDN
System Assurance - UAT
iCruse – BAU
OMS Support
ECM Disputes – CRs
GCM Acquirer – CRs
What we noticed?

Different SDLC models followed : Iterative model
followed in ECM, while B2B adopted SmartBPM then
Iterative.

Differences in testing approach: Risk based testing
followed in ECM

Automation testing : Adopted in ECM release 1,
Benefits accrued in Release 2.

Comprehensive status reporting followed in ECM
R1

Common Challenges:
 Requirements Management
 Change Management
 Test Data identification & preparation
TCoE Evolution: Where are we today?
QA is a separate function
providing shared services
Cost of
Quality driven
TCoE
Business
Unit
People
People
QA is a separate function,
but each project is treated
separately
Business
Unit
Cost
Tracked &
Managed
Business
Unit
Business
Unit
Automation
Tools
Performance
Process
Business
Unit
People
People
Process
No separate
function for QA
Business
Unit
Tools
Business
Unit
Project led Independent Testing
QA costs
not
tracked
People
Shared Services Model
People
Business
Unit
People
People
People
Process
Process
Process
Tools
Tools
Tools
Development / BA led Testing
QA/ Testing is part of
Development
5
Business
Unit
| ©2011, Cognizant
Independent QA for
each project separately
QA/ Testing as a
service
Why move to a TCoE Model?
 Where do we want to be?
 Key Elements of a TCoE?
 What is different for Pega Testing?
 Benefits of moving to a TCoE
6
| ©2011, Cognizant
Why move to a TCoE model?
The need for QA Centralization
Decentralized Testing Organization
App 2
App N
Development
Team
Development
Team
Development
Team
Testing Tools
Testing Tools
Testing Tools
Testers
Testers
Testers
App 1
Software Quality Organization
Testing Center of Excellence
Standard
Processes
Common Tools &
Infrastructure
Structured
Software Testing
Resources
Knowledge
Repository
• Varied methodologies, processes, tools, infrastructure
and metrics across groups
• Consistent methodologies, processes, tools, infrastructure
and metrics across groups
• Little or no sharing of resources and knowledgebase;
Resource management (on-boarding, ramp-up and
release) processes have to be managed by each project
• Centralised work management & resource planning in
Core-Flex model
• Number of software licenses that can be used is
typically lesser that what is needed, due to cost
considerations; extensive use of Microsoft Excel
7
Centralized Testing Organization
| ©2011, Cognizant
• Shared Infrastructure and tools. Maintenance and upgrade
costs shared across projects
TCoE Evolution: Where do we want to be?
QA is a separate function
providing shared services
Cost of
Quality driven
TCoE
Business
Unit
People
People
QA is a separate function,
but each project is treated
separately
Business
Unit
Cost
Tracked &
Managed
Business
Unit
Business
Unit
Automation
Tools
Performance
Process
Business
Unit
People
People
Process
No separate
function for QA
Business
Unit
Tools
Business
Unit
Project led Independent Testing
QA costs
not
tracked
People
Shared Services Model
People
Business
Unit
People
People
People
Process
Process
Process
Tools
Tools
Tools
Development / BA led Testing
QA/ Testing is part of
Development
8
Business
Unit
| ©2011, Cognizant
Independent QA for
each project separately
QA/ Testing as a
service
Centralized QA
Key Elements of a TCoE
9

Single test organization with defined governance model

Benchmarks for productivity and SLAs based on metrics gathered over time

Centralized resource management (Core-Flex team) for efficient demand management

Common processes for On-boarding, training & competency development

Standardized templates, guidelines and checklists across STLC

Framework based test automation that is used across projects

Centralized license management for testing tools

Test environment management

Centralized test data management

In-house product and technology specialists who serve as “go-to” gurus for all projects
| ©2011, Cognizant
TCoE Ecosystem (Future State)
TCoE
Business
Project Management Office
Strategic Planning & Governance
Business process definition
Budget Allocation
Provide/validate/signoff
requirements
Overseeing TCoE SLA adherence
Recommendations to Change
Control board
Process Team*
Infrastructure Services
Environment Management
Test Data Management
Release & Configuration
Management
Development
Application Development
SLA / Metrics
Management
Test Tools COE*
Tools standardization
AMEX
Cognizant
Inter group relationship of TCOE with
external groups
Test
Planning &
Estimation
| ©2011, Cognizant
Best Practice Implementation
Test
Reporting
Test Design
&
Build
Test
Execution &
Management
Automation/NFT*
Proof of concept
Standards & Guidelines
Feasibility Analysis
Technical Consultancy
Reusable Frameworks
Service Spectrum
Functional Testing
Core-Flex
resourcing
Regression Testing
Standard process
& Frameworks
Performance /Load
Testing
Common Tools,
Infrastructure &
Automation
* Refer to Appendix for details about each team
10
Metrics Institution
Continuous Process
Improvement
Testing
Centre of
Excellence
Tools administration
Tools support
Process Definition/
Maintenance
Test Process Training
Maintenance/
Enhancements
Defect Fixes
Test Program
Management
Automation
Testing
Solution
Accelerations/
Best practices
UAT Support
Governance
structure
Centralized TCoE for Pega
Pega TCoE – what is different for Pega, that makes it
necessary to have a dedicated TCoE?
11
SDLC Model

Available models include Waterfall, Iterative, Agile and Pega’s own Smart
BPM approach. By standardizing the model, it is possible to optimize
processes, tools and templates and derive benchmarks for reference
Team composition

In addition to Business Analysts, the team needs to include Pega
specialists who understand Pega PRPC product and have prior experience
in Pega Testing
Test Strategy

Pega Testing is much more than UI based functionall testing. It is important
to know how to test Rules and Workflows, how to test web-services, how to
focus on specific flow paths for test execution during different stages of
application development, how to rules that are data intensive, and how to
use tools / utilities with Pega ie AUT, TMF, PAL etc
Tools & Automation

For some needs, Pega’s own tools i.e AUT, TMF, PAL, PLA are
recommended, whereas other tools like QTP and Cognizant’s proprietary
tools like ADPART for Pega , CRAFT, TCGEN work better in other cases.
Knowledge how each of these tools work is therefore critical
Change
Management

It is common to find requirements evolving frequently in Pega projects. With
changing requirements, it is necessary to identify changes in test scenarios
immediately, and continuously maintain regression test scripts. Tools like
ADPART for Pega can be used to deal with this challenge very effectively
| ©2011, Cognizant
Benefits of moving to a TCoE model?
Standard
Processes
• Higher system
quality
• Better planning &
estimation
• Rigorous metrics
collection
• Continuous process
improvement
12
Optimised
Resourcing
• Lower labor costs
through optimised
utilisation of resources
• Dedicated team builds
expertise over time
Common Tools
& Infrastructure
• Reduced effort through
reuse of common
frameworks, templates,
and data repositories
• Maximised test
automation
• Optimized tool licensing
requirements
Central
Repository
• Institutionalise
knowledge
• Better test coverage
• Efficient knowledge
transfer

Testing is delivered as a Shared Service, i.e, Functional Testing, Automation Testing, Performance Testing
across projects, thus reducing the cost of testing to each project

Common pool of experts is leveraged by all projects

Enables enterprise wide adoption of frameworks for Automation Testing – thus reducing cost of script
maintenance due to product upgrades (ie Pega 5.5 to 6.2)

Having a common knowledge repository ensures each project team does not go through the same learning
curve separately.
| ©2011, Cognizant
The Core-flex model of
Resourcing


13
| ©2011, Cognizant
Core-flex model
Governance Structure
Core Flex Resourcing Model
1. Staffing is done based on demand projections and average productivity observed on
a quarterly basis
2. In case of ramp-up in Core team, flexi team resources at offshore would be moved to
core, and new associates inducted in flexi team
14
| ©2011, Cognizant
Capacity Planning in the Core-Flex model
Floor Limit
Ceiling Limit
(90% of forecasted
demand)
(Forecasted demand +
10% of fixed capacity)
Fixed Capacity
1,680 hours per month
Forecasted demand
based on 3 month
rolling forecast
The capacity model is built on
 Fixed Requirements – i.e. core team of fixed number of resources having identified
skills
 Flex team to support short term requirements for scaling up at short notice
(typically up to 10% of core team size, provided core team size > 25)
 Timely Demand Forecasting
 Factoring a minimum lead time for ramp-ups (Ramp-up of Core team is done by
moving resources from flex team, and replenishing the flex team within 6 to 8
weeks typically)
 Floor and Ceiling Limits
15
| ©2011, Cognizant
Core Flex Resourcing Model
Tiers
16
| ©2011, Cognizant
Capacity Equival Resource Mix (Person
hrs / Month)
Slab (Person ent FTE
hrs/ Month)
slab
Onsite/Offshore
Ratio
Flex
Team at
offshore
ON
OFF
ON
OFF
OFF
1
2496
15
480
2016
19%
81%
-
2
4168
25
640
3528
15%
85%
2
3
6680
40
800
5880
12%
88%
4
4
10032
60
960
9072
10%
90%
6
Steady State TCoE Governance Structure
17
| ©2011, Cognizant
TCoE – Cognizant/AMEX Roles & Responsibilities
* Vision, Goals &
Objectives
AMEX
 Cognizant will assume delivery ownership
and strategy/ planning/
execution/reporting for all testing
activities done as part of the TCoE
 ** Cognizant to help AMEX with
Environment Management and
Configuration Management, by leveraging
existing/proposed Cognizant presence in
those areas
 Program management, Risk management
and Communication Management will be
shared responsibilities
18
| ©2011, Cognizant
Cognizant
 * Cognizant to partner with AMEX for
formalizing strategies and help implement
Vision, Policies and procedures and
Budget allocation, as well as Release
Planning, and Business Prioritization
Shared
 AMEX will have ownership of SME support
activities and Supplier Coordination
* Policies &
Procedures
Business Analysis &
SME
* Release Planning
** Configuration
Management
* Budget
Management
Supplier
Coordination
* Business
Prioritization
Program
Management
Risk Management
Communication
Management
Manage & Deploy
Resources
Estimation
PMO Reporting
KPI Tracking
Test
Strategy/Planning
Test
Design/Execution
** Environment
Management
Tool Administration
Performance
Management
Defect Management
Thank you
19
| ©2011, Cognizant
Transition Approach:
 from current state to TCoE
 from incumbent team
20
| ©2011, Cognizant
Typical TCoE Implementation Timeline*
Pega
Transition
from
incumbent
team
Decentr
alized
state
Metric s Testing as
benchm a service
arked offered to
BUs
Define Basic Processes
Identify initial set of applications
to be brought under Pega TCoE
Environment Set-up & access creation
Knowledge Transition
Define guidelines and frameworks
Configuration of reusable assets
Execution and base-lining
Quarterly Demand forecasting
Consolidation at Enterprise Level
M0
M1
M2
M3
M4
M5
M6
M7
* Indicative sample only. Actual timeline will be formalized after more details are gathered
21
| ©2011, Cognizant
M36
Vendor Transition - Overview
Shadow
Share
Lead
Cognizant
Incumbent
Cognizant understands existing
processes and application under the
supervision of existing team
22
Entry criteria
• SoW signed
• High level KT Plan shared
• KT team identified
Scope
•
•
•
•
Exit criteria
How is it
measured?
| ©2011, Cognizant
Test Execution by Cognizant
team on trial to validate
knowledge gained
Cognizant will undertake execution
of majority of test cases in
steady-state
• Draft KT document
reviewed
• Tester logins created
• Separate instance for
Cognizant testers
• All existing test cases
executed at least once
• Test Management
processes defined
• Identified set of test
scenarios (business
critical)
• Testing of batch processes
• Full scope of application &
Interfaces that are
available in QA
environment
• Draft KT document
prepared by Cognizant
• Gaps (between application
and test scripts) identified
• Metrics for test cases
executed by Cognizant
team
• Updated gap analysis
document
• Ongoing assessment
through agreed metrics
and delivery review
• No of topics covered,
hours of KT
• Productivity
• Defect Leakage
• Any other metric
specifically agreed
Business Processes
Application & Interfaces
QA Environment
Test Mgnt processes
Appendix
 Case-studies
23
| ©2011, Cognizant
Experience in setting TCoE for leading Industry
Players
Rich experience
Banking & Financial Services
Healthcare
establishing large
scale TCoEs across
more than 30+ clients
globally
Peak Team
Size
Customer Profile
Leading UK based financial group
800+
Customer Profile
Peak Team
Size
One of the largest U.S. health plan
450+
3rd largest health plan in the U.S.
350+
350+
One of the largest Blues
180+
Switzerland based fin. services Org
250+
One of the largest clearing houses
100+
Large U.S saving bank holding co.
100+
One of the oldest fin. services firms
in the world
One of the largest banking and
insurance group in UK
350+
Communication, Media and Entertainment
Customer Profile
Life Sciences
Customer Profile
One of the largest Pharma co.
One of the oldest EU Pharma co.
Peak Team
Size
Leading Telecom Equipment
vendor
Leading Broadband Service
provider
Large legal solutions and risk
analytics company
One of the world’s largest
information co.
TCoEs
250+
50+
Insurance
A Fortune 100 insurance company
150+
The largest U.S insurance company
250+
U.S based Intl. fin. Servicers firm
100+
Leading fin. services product co.
150+
Top 3 fin services co. of U.S
200+
| ©2011, Cognizant
70+
30+
250+
150+
?
Peak Team
Size
?
24
Customer Profile
Peak Team
Size
Retail, Travel & Manufacturing
Customer Profile
U.S based Internet travel
company
World's largest office supply
retail store chain
One of the largest
Manufacturing conglomerates
Technology
Peak Team
Size
Customer Profile
Peak
Team Size
150+
Online stock brokerage fin.
service co.
120+
100+
Largest Software Products co.
100+
100+
Largest Engg Design
Software Co.
70+
PRPC Testing – Global Financial Services Company
Project Summary
• Business Objective:
Implementation of a Credit
Cards Dispute Management
Application at the Acquirer
end
Eliminate existing manual
processes like case creation,
case processing etc.
Automate processes by
implementation of the Pega
PRPC Enterprise Case
Management system
• Testing carried out in various
business centers and Markets.
• 2 cycles of Testing in 6 months
to perform confirmation and
regression testing to ensure
that the product meets
requirements
• Used Quality Center 9.0 for Test
management tool, QTP 9.2 for
Automation and Load Runner
9.5 for Performance Testing
• Technology Stack:
 PRPC V5.5 SP1
 IBM Web-sphere Portal
 V6.x.x, JDK 1.4.2,
 IBM DB2 database
25
| ©2011, Cognizant
Key Modules
PRPC
• SSO Agent Login
• Get Work & Search Case
• Retrieval Requests
• Charge backs
• Financial adjustments
Multi Market Testing
Integration with
• GC&S (Case creation)
• OpsNet & Towerscan
(Image storage)
• Unitech
(Case validation)
•
•
•
•
•
Scope
•
•
•
•
•
•
•
Team Composition
PEGA Rules based Testing
Functional Testing
Integration Testing
Regression testing
E2E Testing
UAT Support
Performance Testing
• 1 Onsite & 6 Offshore Test Analysts
• Specialized PEGA Testing team
comprising of Manual, Automation and
Performance Test Analysts
Client Benefits
Automation Coverage
• ~40% of System Test Cases were automated - More
than 50% reduction in Test Execution time
• Reusable automation framework resulting in ~30%
reduction in script creation effort
Airlines
British Airways
Lufthansa
Pay pal
Highways
•
•
•
•
•
Automation of 90% of the manual Credit Card dispute
processes using PRPC
Significant reduction in case processing time
76% of defects were identified before UAT, ensuring
stability of the application
0% production defects
18% defects were raised in the Rules Testing phase
Applause
I wanted to share with you all the very positive feedback we have received from the UAT testers. Big wins
on two fronts: Testing Training – really engaging, useful, enjoyable; ECM System: really useable, easy to
navigate, professional, barely need training its so easy. We had a bunch of seasoned users who are not
easily impressed by things - to get this kind of feedback is a resounding success! WOW!!!!!!
Manager, Strategic Project Implementation
Just wanted to say thank you to all the Team for their tremendous effort and getting this back on track
Director, World Services Technologies
Financial Adj.
3
Unitech
3
No of Test Cases
PRPC Testing – Global Financial Services Co. …contd.
1500
1000
59
OpsNet / Towerscan
Automation
500
Cycle 1
1382
0
200
400
600
800
1000
1200
Cycle 2
1500
500
0
Manual versus
Automation Coverage
PRPC functional
2000
1000
0
107
GC&S
Manual
Automation coverage
1400
1554 test cases designed and executed across
various modules
Test Execution Productivity
Optimized
Delivery
23
60
50
Performance
Testing
50
40
30
53
20
10
42
23
15
12
0
High
Medium
Low
10
Info
5
Defects by Severity
Time in secs
76% defects identified before UAT
26
| ©2011, Cognizant
57
0
11.5
6.797
1.984 2.094
High
Medium
Low
Defects by Priority
Testing Centre of Excellence - Large Financial Services Provider
Background
Project Highlights
The client identified the need for a dedicated Testing Center of Excellence
Effort Details
(TCoE) for catering to the testing needs of all its IT systems and
•
& Ongoing
applications as well as to centralize its testing processes and inculcate
best quality practices across the organization.
Cognizant Solution
Cognizant
performed
Scope of the TCoE
a
2 Years elapsed time
•
Peak team size: 147

Onsite –

Offshore – 116
31
Key LoB
strategic assessment of the
•
Customer Ops
client’s testing organization
•
Employee Benefits
and established a dedicated
Testing Center of Excellence
Applications Tested
(TCoE) which encompasses
•
Microsoft CRM
resources
the
•
Policy Admin System
client and Cognizant jointly
•
Mainframe and Web
from
addressing
both
the
needs of the client.
27 | ©2011, Cognizant
testing
Applications
Testing Centre of Excellence - Large Financial Services Provider
Client Benefits
Cost Savings through Automation
Productivity: Automation resulted in over 90% of effort
and over 80% of cost savings
Resourcing: Established core + Flex staffing model which
enabled staff ramp up to 63% in 2 weeks
Quality: High test case coverage and continuous process
improvements ensured a very high quality of deliverables
Balance Scorecard: Establishment of a Balance
Scorecard with Key Performance Indicators (KPIs)
covering Budget, timely delivery, Quality and CSAT.
Risk Based Testing (RBT) : Cognizant has also
developed a Risk based testing model for the client to
reduce the overall testing cost by optimizing the number of
test cases and reducing the testing cycle time.
28 | ©2011, Cognizant
Risk Based Testing Approach
Test Planning in Agile model for an e-trading leader
• A Branch is cut from main trunk
(code which is in prod)
Branch & Merge Strategy
• New feature is developed on
independent branch
New feature on independent branch
Unit Test -> QA approval for
merge
Independent Branch
Final Merge,
ownership transfer
to Main
Enhancement to existing
feature cut from branch
• QA signoff/ release before an
enhancement/ feature can be
merged with the Branch
• After promotion to trunk,
regression test done on trunk
• Minimum Acceptance Tests after
regression test
Uprev’s , synch up’s
btw branches
Merge
Merge with Main Trunk, transfer of ownership
to client
Release Versions
| ©2011, Cognizant
Targeted Release
29