The Paradigm Shift – Testing 3.0
Arun Kumar Singh
Agenda
1.
2.
3.
4.
5.
6.
2
Testing 1.0
Testing 2.0
The changing market space
The new needs
Testing 3.0
The 3 type of supplier and the opportunity for them
© 2010 Wipro Ltd - Confidential
Software fails
3
© 2010 Wipro Ltd - Confidential
Source – IEEE Spectrum
Testing 1.0
Definition :
Software Testing is the process of executing a program or system with the intent of finding errors.
[Myers79]
Lifecycle
Part of debugging process , dynamic change
Techniques
Processes (typically) undocumented
Tending to be driven in an ad hoc uncontrolled and reactive manner by users or events.
Infrastructure
chaotic or unstable environment for the processes
Organization
one team. no differentiation. Development team did testing
Success due to competence and heroics of the people in the organization and not on
the use of proven processes.
4
© 2010 Wipro Ltd - Confidential
Testing 1.0
Market conditions
•
•
•
•
•
Small & nascent IT
Small pockets of manual activity automation
Startup / innovation/ research environment
Small developments
Technology use/excellence drive
T
C
Outcome
products and services that work; however, they frequently exceed the budget and
schedule of their projects
5
© 2010 Wipro Ltd - Confidential
Q
Testing 2.0
Definition :
Any activity aimed at evaluating an attribute or capability of a program or system and
determining that it meets its required results. [Hetzel88]
Software testing is an investigation conducted to provide stakeholders with
information about the quality of the product or service under test.[1]
Software testing can be stated as the process of validating and verifying that a software
program/application/product:
1. meets the requirements that guided its design and development;
2. works as expected; and
3. can be implemented with the same characteristics.
1. ^ Exploratory Testing, Cem Kaner, Florida Institute of Technology, Quality Assurance Institute Worldwide Annual Software Testing Conference,
Orlando, FL, November 2006
2. Wikipedia
6
© 2010 Wipro Ltd - Confidential
Testing 2.0
Techniques
testing processes are
well characterized and understood, part of SDLC or PLC
described in standards, procedures, tools, and methods.
improved over time
Using precise measurements, management can effectively control the software
development effort. ……. identify ways to adjust and adapt the process to
particular projects without measurable losses of quality or deviations from
specifications.
Specialized tools and techniques ( statistical models ) are used
Infrastructure
planned & dedicated
Organization
well defined and separate testing organization. Exec mgmt support.
testing G&Os aligned to corporate vision and mission
7
© 2010 Wipro Ltd - Confidential
The TCoE
Company
XYZ
Business unit A
in geo A
Business unit B in
geo B
Business unit C
in geo C
Benefits
Layers
Demand Management
 Standard engagement
process
Demand consolidation & forecasting
 Reduced cycle time
 Enterprise-wide
common processes
SME services
QA Consulting/Best Practice Exchange/domain & product expertise
 Improved
maintainability
 Standardization of
process / templates
Core Services
Manual Testing
Automation performance
SBB..xxx
 Compliance across Geos
and Business Units
 Improved productivity
Infrastructure
consolidation
8
Shared test environments
© 2010 Wipro Ltd - Confidential
 Reduced Capex and Opex
Cost to Quality
Cost to fix a defect
Requirem
ents
Time
Architectu
introduce
re
d
Constructi
on
Time detected
Requirem Architectu Constructi System
ents
re
on
test
Postrelease
1×
3×
5–10×
10×
10–100×
-
1×
10×
15×
25–100×
-
-
1×
10×
10–25×
^ McConnell, Steve (2004). Code Complete (2nd ed.). Microsoft Press. pp. 29. ISBN 0-7356-1967-0.
9
© 2010 Wipro Ltd - Confidential
Software quality factors
Functionality (exterior
quality)
Engineering (interior
quality)
Adaptability (future quality)
Correctness
Efficiency
Flexibility
Reliability
Testability
Reusability
Usability
Documentation
Maintainability
Integrity
Structure
Typical Software Quality Factors [Hetzel88]
10
© 2010 Wipro Ltd - Confidential
Testing 2.0
Market Conditions
1.
Complete enterprise computerization
1.
2.
2.
3.
4.
5.
Package software / enterprise in a box
Global enterprises
Multi channel /always-on client interface
Increased rate of refresh
1.
2.
3.
6.
7.
11
Large projects
E2E business process software
Technology platform
Hardware
Business requirements
Localization with globalization
Offshore delivery models
© 2010 Wipro Ltd - Confidential
T
C
Q
Testing 2.0 - Challenges
1.
2.
3.
4.
Tracking/measuring and defining correct outcomes & benefits.
Lack of management support (little or no visibility into the effectiveness )
Resistance from suppliers.
Organization change... Limited tools to encourage the managed approach
with HR metrics
5. Defining & doing the right amount of testing
6. Moving from technology to business process testing
7. Domain expertise
8. Cost to quality measures
9. Disconnected from the end customer experience
10. Offshoring : communications, right skill sets, management-vendor
relations, global infrastructure, ever increasing expectations/innovation
12
© 2010 Wipro Ltd - Confidential
The emerging market space – PEST analysis
Political / Legal factors
Economic factors
√ Monopoly / Oligopoly legislation
√ Inflation
√ Environmental protection laws
√ Employment
√ Taxation policy - Tax levies and tax
√ Disposable income
breaks
√ Business cycles
√ Employment laws - minimum wage or
√ Energy availability and cost
anti discrimination laws.
√ Others?
√ Government policy, Legislation
√ Voluntary codes and practices
Socio-cultural factors
√ Market regulations
√ Demographics
√ Trade agreements, tariffs or restriction
√ Distribution of income
Technological factors
√ Social mobility
– New discoveries and innovations
√ Lifestyle changes
– Speed of technology transfer
√ Consumerism
– Rates of obsolescence
√ Levels of education
– Internet
√ Others?
– Information technology
Rapid change in business environment
13
© 2010 Wipro Ltd - Confidential
Analyst market outlook
Overall: Discrete or standalone global testing services spending reached
$9.4 billion in 2010,
Growth: Worldwide growth estimated at a five-year CAGR of 15.4%. Some
analyst have
Geo-wise: United States has led other regions in adoption of discrete testing
services, in terms of both traditional and newer cloud-based testing
services models…….
an increase in regional interest and adoption of these services in Canada,
Latin America, Western Europe,and parts of Asia/Pacific is seen.
Message for testing
huge growth – plan for it. Be ready with right resources
outcome driven delivery and financial models would be needed
move from technology to business process
14
© 2010 Wipro Ltd - Confidential
Source : IDC
Market space – organizational metamorphosis
Key Findings






CFO influence in IT is growing; 42% of IT organizations report directly to the CFO.
Organizations are willing to spend to gain competitive differentiation and improve efficiency.
CFOs cite "gaining business insight" and "improving business efficiency" as top technology
opportunities
CFOs would like to reduce ERP instances, and expressed a preference for "vanilla"
implementations — i.e., implementations that do not require excessive customization.
CFOs are still concerned about achieving business benefits from IT.
IT success relies more on understanding and working with the business than on technological
excellence.
Impact on Testing:
 Communicate business benefits in financial terms to senior financial executives. Move from
technology to business process testing and focus on cost-benefit analysis.
 Understand CFO’s view of technology investment decisions.
 Testing teams must be able to explain costs, benefits and share information/influence non-IT
employees
15
Source : Gartner
© 2010 Wipro Ltd - Confidential
Analyst market outlook
Trends that are key to growth in this services segment:
–
–
–
–
Test strategy consulting
Evolution and maturation of testing services portfolio
Testing complexity and optimization
Technology excellence is hygiene factor. Business process assurance should be the
focus
– Ready to go solutions
Emerging/Future Application Testing Services Trends:
–
–
–
–
–
–
–
–
16
Model-driven testing
Mobile application testing services
PaaS testing
Cloud testing — many variants
Cross-cloud testing
Crowd sourcing for testing
Test data management
TaaS
© 2010 Wipro Ltd - Confidential
Source : IDC
WIIFM - Testing Service Providers
1. Testing as custodian of customer experience.
2. Move the testing 2.0 excellence to virtual environment ( cloud, hosted test beds )
3. Mobile application service on vendor hosted environment with remote access
4. Cloud testing offerings
5. Service delivery and pricing models – offer Range and flexibility
6. test service ROI : monitor and provide
7. Product IP protection
8. Market and build brand presence in testing services:
9. Downward price pressure
10. Partner alliances
11. Public crowd sourced testing models: it will continue to gain traction in 2011 and
beyond
Source : IDC
17
© 2010 Wipro Ltd - Confidential
WIIFM - Large testing service provider
1.
2.
3.
4.
5.
6.
Provide Testing 2.0 , your core competence to more buyers.
Experiment with testing 3.0 with your large and strongly engaged clients
For large testing deals be ready with right team, at right place at right time
Show cost flexibility
Be ready to move from 100% complete testing to business-need based testing
Domain competence will be key to differentiate, as client decision makers
become non-technical
7. Lab(s) on hire ( especially with cloud coming) will become a need
8. Specialize in sub-practices in testing.
9. Develop Joint testing solution with product and tools vendors
10. Expand offerings – be E2E testing service provider
11. All shore delivery – round the clock execution.
12. To handle cost pressure – looks at class B and C cities in India eastern
European countries.
18
© 2010 Wipro Ltd - Confidential
WIIFM - Small testing service providers
1.
There is huge room for Testing 2.0. Use on of the three strategies to
penetrate market
1. Cost leadership
2. Focus ( e.g. finance, TSP etc.)
3. Differentiation
2.
3.
4.
5.
6.
7.
19
Implement the TCoE model within your organization. Structure your
testing group accordingly
Do joint solutions ( and GTM strategy ) development with package and
tools Alliance partners. This will increase market reach and penetration.
Develop client advocates
APAC / ASEAN market has huge potential. Lower margins but easy
penetration wrt larger vendors
Experiment with local workforce ( visa, Indian staff cost, onsite HR
policies)
Establish labs and demos. Develop process assets.
© 2010 Wipro Ltd - Confidential
WIIFM - Entrepreneurs – new testing service
providers
1.
2.
3.
Implement TCoE aligned org structure internally
You may start with testing 2.0 offering to large firms with old IT groups & 3.0 to
niche product/service vendors.
Decide on one of the three strategic choices to penetrate market
1. Cost leadership
2. Focus ( e.g. finance, TSP etc.)
3. Differentiation
4. Cloud (and disruptive technologies) will be a boon / entry point
5. Bunch of excellent testing practitioners will make a huge difference. Seniors are
there in the country now.
6. Build brand
7. Develop solution and do joint GTM with product/tools vendors
8. Experiment with alliance with small firms in other countries. Cater to their off
shoring dis-advantage
9. Participate in local testing boards by becoming their members
10. propose alternate financial models to compete with large vendors
11. Process assets and package/domain accelerators will be hygiene factors
20
© 2010 Wipro Ltd - Confidential
Thank You
Arun Kumar Singh
Vice President
Arun.singh@wipro.com
 IDEAL: initiating, diagnosing, establishing, acting and learning
model ( carnegie mellon university )
 A Users Guide for Software Process Improvement
 This document describes a software process improvement (SPI) program
model, IDEAL, which can be used to guide development of a long-range,
integrated plan for initiating and managing an SPI program. The purpose of
this document is to provide process improvement managers with a generic
description of a sequence of recommended steps for SPI.
22
© 2010 Wipro Ltd - Confidential