Pacific Compensation Insurance Company
IT QA Tools Business Requirements
Revision No.: 1.00
SYSTEMS AFFECTED: Jira
Date: 07/12/10
Status:
Page 1 of 7
FINAL
Confidentiality Statement
The information and data contained herein are the proprietary and confidential information and data
of Pacific Compensation Corporation.
IT QA Tools Business Requirements
Document Revision History
Version
Change Description
Date
Author
1.00
Approved
7/12/2010
M. Cordero
Document Information
File Name
Location
BRD QA Tools.docx
PacificComp wiki > Project Documentation
Related Documents
Document #
Document Title
Location
SOP-24-03
System Development Life Cycle
PacificComp wiki > IT Effective Documents
Standard Operating Procedure
References
Document Title
Location
Application LifeCycle Management
S:\QA\TestAutomation\Tools
(Software Planner)
Evaluation\AutomatedQA\SoftwarePlannerDataSheet.pdf
LoadRunner
S:\QA\TestAutomation\Tools Evaluation\HP\LRDataSheet.pdf
MICROFOCUS SilkCentral
S:\QA\TestAutomation\Tools Evaluation\MICROFOCUS\Micro FocusR
SilkCentralR Test ManagerT Datasheet.pdf
QualityCenter
S:\QA\TestAutomation\Tools Evaluation\HP\QTP datasheet 09.pdf
Contributors
Contributor Name
Responsibility
Hagay Hazan
Product owner of this Quality Assurance (QA) tools solution
Monica Cordero-Maher
QA management, supervising the QA PO
LuNell Gilliland
IT management, supervising the QA function
Approved
PacificComp Confidential
Page 2 of 11
IT QA Tools Business Requirements
Approvals
Name
Title, Department
Joe Cardenas
Vice President/Chief
Signature
Date
Information Officer,
Information Technology
Monica Cordero-Maher
Quality Assurance Manager,
Information Technology
Approved
PacificComp Confidential
Page 3 of 11
IT QA Tools Business Requirements
Contents
1.0
Executive Summary ...................................................................................................... 5
2.0
Scope ............................................................................................................................. 5
3.0
Business Case ............................................................................................................... 5
3.1.
3.2.
Problem Statement ............................................................................................................. 5
Business Drivers ................................................................................................................. 7
4.0
Workflow ........................................................................................................................ 7
5.0
Requirements ................................................................................................................ 9
5.1.
5.2.
General Requirements: Application LifeCycle Management Tool .................................... 10
General Requirements: Performance Tools ..................................................................... 10
6.0
Report Requirements .................................................................................................. 11
7.0
System Documentation............................................................................................... 11
8.0
Training Requirements ............................................................................................... 11
9.0
Other Considerations .................................................................................................. 11
Approved
PacificComp Confidential
Page 4 of 11
IT QA Tools Business Requirements
1.0
Executive Summary
Pacific Compensation Insurance Company (PacificComp) and the organizations serviced by its Information
Technology (IT) department rely on technology system solutions and services to support business operations. The
IT Quality Assurance (QA) department serves the mission to ensure that IT delivers high quality solutions and
services in a timely manner.
The IT QA department was formally established in 2009. To date it has supported significant new solutions delivery
by managing testing resources, test cases and findings manually, primarily through Excel worksheets. Based on
that experience the strong need for QA testing tools have been identified.
2.0
Scope
The scope of IT QA Tools Business Requirements (BRD) covers the management and automation of QA
processes including: requirements repository, test planning, test cases library, test execution management and
reporting, testing automation, defects management system and performance testing.
Tools selected and used by developers and operations are outside the scope of this BRD. Examples are Jira
(project tracking tool for software development teams from Atlassian) used for stories and tasks creation,
Subversion (SVN, an open source version control application) used as the code repository and ServiceDesk (help
desk application from ManageEngine) used as the change management workflow system.
3.0
Business Case
Industry practice support the adoption of QA tools to shorten the build-test time, better use testing personnel and
improve overall quality of the solutions delivered to business users.
3.1. Problem Statement
Problem (Solution
SDLC artifacts including requirements, solution design and test plans need to consistently
Documentation)
document robust, clear and accurate information on how the solution is expected to
operate. Documentation of requirements has been weak and solution design documentation
has been missing. Subsequently, test plans have been delivered late and lacking specificity.
A Requirements Traceability Matrix (RTM) that follows requirements through design, test
script and executed test findings is lacking in the System Development Life Cycle Standard
Operating Procedure (SOP-24-03) workflow deliverables.
Stakeholders
All SOP-24-03 roles: Application Architect, Business Owner (BO), Chief Information Officer
(CIO), Developer, Development Manager, Infrastructure Architect, Infrastructure Manager,
IT Manager, Product Owner (PO), QA Manager, System Administrator
Approved
PacificComp Confidential
Page 5 of 11
IT QA Tools Business Requirements
Impacts
Weak or missing SDLC documentation increases risk that the delivered solution will not
meet customer needs, creates inefficiencies in development and test, reduces the ability of
QA to verify functionality and increases confusion among delivery team members. Without
an RTM confidence in testing coverage is low.
Success Criteria
QA verifies delivery team understanding of each requirement. QA documents trace from
requirements to design through executed test script findings. QA delivers the test plan
artifact complete and when needed following SOP-24-03.
Problem (Defect
The current tool and procedures used for defect tracking and reporting, Jira and Excel
Tracking and
worksheets, are not designed for those purposes and lack significant functionality.
Reporting)
Stakeholders
CIO, Developer, Development Manager, Infrastructure Manager, IT Manager, Product PO,
QA Manager, Testers.
Impacts
The QA process lacks reporting capabilities to communicate with the project team a clear
understanding of defect descriptions, severity, status and disposition throughout the project.
Success Criteria
Keeping the defect tracking database current and updated, QA is able to generate an
accurate report that clearly illustrates the state of the solution at any time during the project
and to present measurable and accurate status of the solution compared to requirements.
Problem
As illustrated during the recent ClaimCenter 6.0 upgrade project, performance and load
(Performance and
testing coverage are critical to user acceptance of many solutions. Performance scenarios
Load Coverage)
were simulated by QA and executed manually against a very limited sample of transactions
both when the system was under load and during normal usage. But acceptance criteria,
such as a baseline performance measurement of ClaimCenter 3.1 in production, were
lacking. Only hardware was monitored; application monitoring during performance and load
test execution is needed. Without performance and load coverage, QA is unable to report
findings for capacity, for scalability and stress test for reliability.
Stakeholders
CIO, Developer, Development Manager, Infrastructure Manager, IT Manager, Product PO,
QA Manager, Testers.
Impacts
ClaimCenter 6.0 deployment experienced blocking performance issues.
Success Criteria
Based on defined baseline performance, load and stress requirements, QA is able to
execute testing and deliver actionable findings. For example, referring to the ClaimCenter
6.0 upgrade project, performance and load testing provides input to determine requirements
for new hardware and infrastructure to support the scaling of the live site as well as internal
environments and servers.
Approved
PacificComp Confidential
Page 6 of 11
IT QA Tools Business Requirements
Problem (Regulatory
Production of audit evidence satisfying the following IT control has been manually intensive
compliance)
and inefficient with existing Jira and worksheet reporting capabilities: Unit, system and
integration testing are executed for all significant changes to the current or new applications
and infrastructure technology so that deployed systems operate as intended. Anticipated
future IT controls include regression testing evidence and RTM artifacts.
Stakeholders
BO, BSA, CIO, IT Manager, QA Manager, Testers
Impacts
Risk of weak audit findings.
Success Criteria
With greater accuracy, completeness and cost effectiveness, QA is able to deliver Test
Summary and related documentation that passes audit.
3.2. Business Drivers
PacificComp is transforming from the direct writer business model to doing business with agencies and brokerages.
Significant new IT systems are in development to support easy, transparent transactions with agencies and
brokerages in a highly competitive market. Those external customer relationships heighten the need for closely
managed QA activities that produce actionable test findings, defect status and test summary reports. Additionally,
IT regulatory compliance calls for testing documentation.
4.0
Workflow
From SOP-24-03, the basic PacificComp SDLC workflow phases and gates are as follows:
Inception
Elaboration
Design
Build-
User
User
Release
Transition
Close
Phase
Phase
Readiness
Test
Acceptance
Acceptance
Readiness
Phase
Phase
Gate
Phase
Test
Test Phase
Gate
Readiness
Gate
QA tools support the SDLC workflow as follows:
SDLC Phase or
Feature of QA Tool
Feature Use by QA
Gate
Elaboration
Requirements
Phase
repository
 Capture label and description of requirements by importing from
format provided (for example, Jira issue, Word specification, email).
 Engages QA with the customer needs as the solution proposal is
being developed to better support the testing process.
 Support requirements inspection process.
 Available resource for Product Owner to verify requirements clarity
Approved
PacificComp Confidential
Page 7 of 11
IT QA Tools Business Requirements
SDLC Phase or
Feature of QA Tool
Feature Use by QA
Gate
by key delivery team members.
RTM report
 Draft record of requirements to design trace.
 Promote clear understanding of draft solution design among the
delivery team.
 Support planning for efficient test case organization and complete
test coverage.
Test cases library
 Catalog existing test cases and scripts.
 Identify tests that are candidates for regression testing.
 Help identify through key word tags applicable test cases that are
available for re-use with the proposed solution.
Repository for
 Document existing system baseline measurements, previous team
historical project
defect close rates and elapsed time gathered during previous
information
system release project.
 Support improved accuracy of test plan artifact resources and
timing estimates.
Test planning
 Provides author of the project test plan artifact with applicable
information to help size a proposed project and populate content in
a customizable template.
Build-Test
Requirements
Phase
repository
RTM
 Maintain current record of project requirements changes and team
inspection or re-inspection verification during build-test iterations.
 Update and maintain current record of requirements to design to
test case trace.
Test cases library
 Support efficient and accurate development of functional and
regression test scripts.
Release management
 Identify builds and environments for each build promoted from
development to test.
 Associate executed test cases with unique builds.
 Associate test cases with releases.
Reports
 Reduce manual report creation time significantly through
customizable reports, such as to evaluate stability and accuracy of
each iteration deliverable to determine build readiness for testing
resources to be applied.
Test automation
Approved
 Execute tests with coverage, speed and accuracy that would not be
PacificComp Confidential
Page 8 of 11
IT QA Tools Business Requirements
SDLC Phase or
Feature of QA Tool
Feature Use by QA
Gate
possible with only manual testing.
 Frees resources from manual test execution.
 Especially for redundant test execution, automation eliminates
manual test error.
Defects management
 Provide complete database customized to support needed defect
tracking, categorization, prioritization and design association.
 Deliver complete and timely defect status summary reports to
project stakeholders.
 Associate information from the QA tool defects database with
developer’s use of Jira in tasking fixes.
Performance testing,
load testing, stress
 Execute tests with coverage, speed and accuracy that would not be
possible with only manual testing.
testing
 Execute end-to-end system performance tests.
Lab workspace
 Replace manual Work Allocation Report through use of virtual
space with access to tool functions such as available test cases,
resources and resource balancing.
Requirements
repository reporting
 Provide PO with list of requirements delivered in the designated
release as input for updating the SOP-24-03 requirements artifact.
RTM
 Complete requirements to design to test findings trace.
Executed test cases
 Deliver complete and timely test findings to project stakeholders.
findings report
 Provide detailed and summary input to SOP-24-03 test summary
artifact.
User
Acceptance
Test Phase
Lab workspace
 Create recommended test cases for execution during UAT.
Executed test cases
 Compile findings reported by user acceptance testers.
findings report
 Provide detailed and summary input to SOP-24-03 test summary
artifact.
Defects management
 Support process to scrub all reported defects, record disposition of
each and verify with the delivery team
 Provide summary report for inclusion in the SOP-24-03 test
summary artifact.
5.0
Requirements
This section describes the general features required for QA tools.
Approved
PacificComp Confidential
Page 9 of 11
IT QA Tools Business Requirements
5.1. General Requirements: Application LifeCycle Management Tool
Feature Description
Requirements repository: database to manage requirements label, description and associated
testing information
Test cases library: database to store test cases and associated testing information
Repository for historical project information: searchable storage for information on completed
projects
Test planning templates and content support: intelligent identification of applicable information
to help size a proposed project and populate content in a customizable template.
Requirements Traceability Matrix: record of testing activity associated with a release and
creation of RTM report. Multi-dimensional traceability is supported between requirements, tests
and defects, across releases and test cycles.
Test automation: Test automation engineers can store automation
scripts and assets
Defects management: complete database customized to support needed defect tracking,
categorization, prioritization and status reporting with trending.
Release management: identify builds and environments for each build promoted from
development to test, associate executed test cases with unique builds, associate test cases
with releases. Aggregate quality metrics across projects.
Test lab: QA testers can run manual and automated tests. QA testers can manage
multiple versions of tests and test assets while maintaining data integrity.
5.2. General Requirements: Performance Tools
Feature Description
End-to-end system performance testing: capabilities to determine or validate speed, scalability
or stability characteristics.
Load testing: capabilities to simulate applications running under normal and peak load
conditions to verify can meet performance objectives. For example, measure response times,
throughput rates, resource utilization levels.
Stress testing: capability to determine the durability of a system by pushing it to its limits, to
surface bugs that appear only under high load conditions. For example, investigate
synchronization issues, race conditions and memory leaks.
Approved
PacificComp Confidential
Page 10 of 11
IT QA Tools Business Requirements
Testing analysis and reporting: produce meaningful analysis and actionable reporting.
Monitoring and diagnostics: capability to monitor software and run diagnostics during test
execution.
6.0
Report Requirements
Required report output from the QA Tools includes, but is not limited to, the following examples:
-
Phase based testing execution (manual and automated)
-
Testing coverage
-
Requirements coverage testing
-
Failure report
-
Traceability
-
Monitoring
7.0
System Documentation
Required reference system documentation for QA Tools includes, but is not limited to, the following examples:

IT QA Tool System Administration and Maintenance Standard Operating Procedure

Vendor documents such as solution design, installation guide and configuration guide

Vendor training materials

Vendor user guide

IT QA Department Work Instruction on use of the QA tool.
8.0
Training Requirements
Adequate training of QA department and other testers will be defined as part of the solution evaluation and
selection.
9.0
Other Considerations
QA tools must be compatible with existing IT infrastructure. QA tools must be capable of exchanging data with Jira.
To establish common understanding of QA tools terms, during implementation terms will be identified and defined
for addition to the IT Glossary (GLO-01).
Approved
PacificComp Confidential
Page 11 of 11