Bringing Softtek’s Software Testing
Organization from Good to WorldClass Software Testing Organization
Proposal
Agenda








Mission Statement
Vision Statement
Goals
Assessing Software Testing Capabilities – Your current
processes
Process to achieve a world-class software testing
organization
Deployment Process
Assessing Software Testing Competencies - The quality of
your testers
Q&A
Mission Statement
Mission Statement

To bring Softtek’s Software Testing Organization
from Good to World-Class Software Testing
Organization implementing the Quality Assurance
Institute’s (QAI) World-Class Software Testing
Model.
Vision Statement
Vision Statement

To be an outsourcing Software Testing
Organization leader in providing Quality Control
services for our customers.
Goals
Goals



To implement the Quality Assurance Institute’s
World-Class Software Testing Organization Model
in the organization, to improve Software Testing
Capabilities.
To implement the Quality Assurance Institute’s
developed Common Body Of Knowledge (CBOK)
for a Certified Software Tester (CSTE) in the
organization, to improve tester’s competencies.
To have Software Tester’s certified as CSTE within
1 year.
Assessing Software Testing CapabilitiesYour Current Processes
Assessing Software Testing
Capabilities



Most testing organizations, as well as testers,
believe they are doing a good job. However, there
is no basis for making that determination.
In order to make that determination, an
assessment must be made against a “model” of a
world-class testing organization.
During the past 20 years the Quality Assurance
Institute has studied many organizations and has
developed a model of a world-class testing
organization.
Assessing Software Testing
Capabilities

The Nine components needed to achieve WorldClass Software Testing are:
– Management Support
– Processes
– Training
– Planning
– Quality Control
– Tools
– Efficiency
– Test Execution, analysis and Reporting
– User Satisfaction
Assessing Software Testing
Capabilities – QAI’s Model
User
Satisfacti
on
Quality
Control
Planning
Tools/
Techni
ques
Execution
and
Reporting
Efficiency
Processes
Training
Management Support
(Testing Environment)
Assessing Software Testing
Capabilities
Cause-Effect-Diagram
Drivers of World-Class Testing
Tools
Processes
Management
Support
Desired Result
Planning
WorldClass
Testing
Test Execution
Analysis and
Reporting
Quality
Control
Efficiency
User
satisfaction
Training
Assessing Software Testing
Capabilities

User Satisfaction:
– To involve the user at key points in the
software testing process:
 Test Strategy
 Test Objectives
 Test Plan
 Acceptance Testing
 Reporting the status and results of testing
Assessing Software Testing
Capabilities

Quality Control
– It raises the question, “Who tests the testers?”
 The answer is quality control within the test
process evaluates whether the process has
been performed correctly.
– Assessment to evaluate software testing
organization’s competencies.
 The following test processes incorporate
quality control as part of the process:
Assessing Software Testing
Capabilities
–
Eleven-step software testing process
– Assess Project Management
Development Estimate and status
– Develop Test Plan
– Requirements Phase testing
– Design Phase Testing
– Program Phase Testing
– Execute Test and Record Results
Assessing Software Testing
Capabilities
–
–
–
–
–
Acceptance Test
Report Test Results
Testing Software Installation
Test Software Changes
Evaluate Test Effectiveness
Assessing Software Testing
Capabilities

Testing Specialized Systems and
Applications
– Testing the Adequacy of System
Documentation
– Testing Web-based Systems
– Testing Off-the-Shelf Software
– Testing Security
– Creating Test Documentation
Assessing Software Testing
Capabilities

Execution and Reporting
– Provides the opportunity to evaluate
executable programs as a system
 Build Test Data
 Execute Test
 Record Test Results
 Test Report
Assessing Software Testing
Capabilities

Planning
– If you fail to plan, plan to fail
– To plan a test together with the components of
a test plan
 Develop Test Plan
Assessing Software Testing
Capabilities

Efficiency
– Involves planning and performing software
testing in a manner that provides the greatest
coverage and the maximum assessment of
business risks for the resources allocated
 Establishing a Software Testing
Methodology
 Determining Software Testing Techniques
 Selecting and Installing Software Testing
Tools
 Evaluate Test Effectiveness
Assessing Software Testing
Capabilities

Tools Techniques
– Tools are necessary to provide effective and
efficient software testing
 Processes for selecting and installing
testing tools
Assessing Software Testing
Capabilities

Processes
– Without processing, testing is an art that is
subject to significant variation in execution
 Processes bring stability and consistency to
software testing
– Building a Software Testing strategy
– Eleven –Step Software Testing Process
Assessing Software Testing
Capabilities

Training
– Effective software testing begins and ends with
education
 Define the test strategy that must include
training
 11 step testing process in which software
testers must be trained
Assessing Software Testing
Capabilities

Management Support (Test Environment)
– Effective testing requires strong management
support
 Support is needed so that the test strategy
meets the needs of the business, and
appropriate time and resources are available
to accomplish that strategy
 Establish the need for management’s
support by providing a baseline of the
current software testing capabilities and
competencies
Process to achieve a world-class
software testing organization
Process to achieve a World-Class
Software Testing Organization

A Four step process to achieve a World-Class
Software Testing Organization:
1. Use QAI’s model for setting a strategic quality
goal
2. Develop a baseline by assessing your current
processes
3. Based on the results of the assessment, a
long-range strategy will be developed to
maximize the efficiency of the organization
4. Define a quick-payback approach to implement
the strategy
Process to achieve a World-Class
Software Testing Organization
1.
Use QAI’s model for setting a strategic quality
goal
User
Satisfacti
on
Quality
Control
Planning
Tools/
Techni
ques
Execution
and
Reporting
Efficiency
Processes
Training
Management Support
(Testing Environment)
Process to achieve a World-Class
Software Testing Organization
2.
Develop a baseline by assessing your current
processes
1. Build Assessment Team
2. Complete Assessment Questionnaires
3. Build Kiviatt Chart
4. Assess Results
Assessing Current ProcessWorkbench
Software testing assessment workbench
DO
CHECK
REWORK
Task 1
Build
Assessment
Team
Knowledge of
Software Testing
Activities
Task 2
Complete
Assessment
Questionnaires
Task 3
Build Kiviatt
Chart
Task 4
Assess Results
Assessment
Testing
performed
correctly?
Your Status
versus a
World-Clas
Testing
Organization
Process to achieve a World-Class
Software Testing Organization
Task 1: Build Assessment Team
 The assessment team needs to combine
people who in totality possess the knowledge
on how software testing is performed in your
organization
Task 2: Complete Assessment Questionnaires
 It is composed of eight categories (Drivers of
World- Class Testing)
Task 3: Build Kiviatt Chart
 This chart is implemented to transcribe the
results of completing Questionnaire
Process to achieve a World-Class
Software Testing Organization
Task 4: Assess Results
 Asses status of each category versus what the
category should be in a world-class testing
organization
 Interpret your software testing assessment
Kiviatt Chart
Task 1: Build Assessment
Team
Softtek Testing Services
Complete
Assessment
Complete
Assessment
Complete
Assessment
Testing
Project 1
Lead
Testing
Project 2
Lead
Testing
Project 3
Lead
Team Member 1
Team Member 2
Team Member 3
Team Member 1
Team Member 2
Team Member 3
Team Member 1
Team Member 2
Team Member 3
Complete Assessment
Testing
Project 4
Lead
Team Member 1
Team Member 2
Team Member 3
Complete Assessment
Testing
Project 5
Lead
Team Member 1
Team Member 2
Team Member 3
Task 2: Assessment
Questionnaires
Task 3: Kiviat Chart
Management Support for
Test
5
Test
Planning
4
Use of Test Processes
3
2
1
Test
Quality
Control
Test Tools
User
satisfaction
with testg
Test Traiining
Test
Efficiency
Task 3: Kiviat Chart



For each category the number of Yes responses
should be totaled
A dot should be placed on the kiviat chart on the
line representing the number of Yes responses
For example:
– If there were three Yes responses for test
planning a dot would be placed on the test
planning line at the intersection of the line
representing three Yes responses.
Task 3: Kiviat Chart
–
–
A dot should be put on the line representing all
eight categories for the number of Yes
responses.
The dots are then connected by a line resulting
in what is called a “footprint” of the status of
your software testing organization versus a
world-class testing organization.
Task 4: Assess Results

Softtek can use the results of this assessment in
any of these three ways:
1. Determine Softtek’s current software testing
status versus a world-class testing
organization. The responses in the area to
address will indicate Softtek’s strengths and
weaknesses compared to a world-class testing
organization
2. Develop a software testing goal to become a
world-class testing organization
3. Develop an improvement plan
Deployment Process
Deployment Process

Assessment questionnaires are sent to Test leads
Test Leads answer questionnaires
Total number of “Yes” are dumped into
Capabilities Assess Results Matrix
Kiviat Chart is built
Based on Results develop testing goals

Develop improvement plan




Deployment Process
Deployment Process
Management Support
for Test
5
Test
Planning
4
Test Processes
3
2
1
Test
Quality
Control
Test Tools
1
User
satisfaction
Test Training
Test
Efficiency
Deployment Process

Interpreting Kiviat Results:
– The areas that need improvement are:
 Test Training
 Test Efficiency
 Test Quality Control
Assessing The Quality Of Your Testers
Assessing the quality of your
testers

Assessing the Quality of Your Testers
– This practice will enable you to assess your
individual testing competencies against the
five skill categories in QAI’s Common Body of
Knowledge (CBOK)
 General Skills
 Test Skills/Approaches
 Test Planning
 Executing the Test Plan
 Test Analysis and Reporting
Assessing the quality of your
testers
Test Competency Cause-Effect-Diagram
Drivers of Testing Competency
Test Analysis/
Reporting Skills
Test Planning
Skills
Desired Result
Management
Support
Fully
Competent
Tester
Test Execution
Skills
Test Skills/Test
Approaches
Test Competency Workbench
Test Competency assessment workbench
DO
CHECK
REWORK
Task 1
Understand
CSTE CBOK
CSTE Common
Body of
Knowledge
Task 2
Complete
Assessment
Questionnaires
Task 3
Build Kiviatt
Chart
Task 4
Assess Chart
Assessment
Testing
performed
correctly?
Your
Competency
versus CSTE
CBOK
Assessing the quality of your
testers
Task 1: Understand CSTE CBOK
 Before you can effectively evaluate your
software test competencies, you need to
understand the Common Body Of Knowledge
(CBOK) for software testing
Task 2: Complete Assessment Questionnaires
 It is composed of 5 categories (Drivers of
Testing Competency)
Task 3: Build Kiviat Chart
 This chart is implemented to transcribe the
results of completing Questionnaire
World-Class Software Testing
Organization Model
Task 4: Assess Chart
 Assess status of each category versus what
the category should be as indicated in the
CSTE CBOK
 Interpret your software testing competencies
against your current job responsibilities
Task 1: Understand CSTE CBOK
Task 2: Complete Assessment
Task 3: Kiviat Chart
General
Skills
Test Skill/ Approaches
Test Planing
20%
40%
60%
Test
Analysis,
Reporting
and
Ivement
80%
100%
Test Execution
Task 3: Kiviat Chart




For each of the five questionnaires, total the
number of Yes responses.
Convert the number of Yes responses to a
percentage by multiplying the number of Yes
responses by 10 (e.g., 3 Yes responses x 10=
30%)
Transcribe the percentage for each of the five
questionnaires.
For the lines corresponding to the questionnaire
name, put a dot on the kiviat chart for the
percentage of Yes responses for that assessment
competency category.
Task 4: Assess Chart


Any rating less than 100 percent indicates a
potential area of improvement in that skill
category
Interpret your software testing competencies
against your current job responsibilities
Q&A

Any Questions….