QI-Bench: Verification and Validation Plan
Rev 1.0
QI-Bench:
Verification and Validation Plan
July 3, 2012
Rev 1.0
Required Approvals:
Author of this Revision:
Andrew J. Buckler
System Architect: Andrew J. Buckler
Print Name
Signature
Date
Document Revisions:
Revision
1.0
BBMSC
Revised By
Andrew J. Buckler
Reason for Update
First version covering Iteration 1 and
initial production instances
Date
July 3, 2012
1 of 6
QI-Bench: Verification and Validation Plan
Rev 1.0
Table of Contents
1. INTRODUCTION ...................................................................................................................................................3
1.1. PURPOSE & SCOPE ...............................................................................................................................................3
2. RELATIONSHIP TO PROJECT MANAGEMENT PLAN AND TRACEABILITY REPORT .....................3
3. VERIFICATION PLAN .........................................................................................................................................3
3.1. VERIFICATION SCOPE ..........................................................................................................................................3
3.2. APPROACH AND REPORTING ................................................................................................................................4
4. VALIDATION PLAN .............................................................................................................................................4
4.1. RISK ASSESSMENT / RISK MANAGEMENT (RA) ...................................................................................................4
4.2. INSTALLATION QUALIFICATION (IQ) ...................................................................................................................5
4.3. OPERATIONAL QUALIFICATION (OQ) ..................................................................................................................5
4.3.1. User Scenarios .............................................................................................................................................5
4.4. PERFORMANCE QUALIFICATION (PQ) ..................................................................................................................5
4.5. VALIDATION REPORTING .....................................................................................................................................6
5. ANOMALY REPORTING AND RESOLUTION ................................................................................................6
BBMSC
2 of 6
QI-Bench: Verification and Validation Plan
Rev 1.0
1. Introduction
1.1. Purpose & Scope
This document describes procedures concerning the testing of the delivered products (product
documents and software) of QI-Bench for compliance with the requirements as staged for development
iterations. These procedures comply with the General Principles of Software Validation; Final Guidance
for Industry and FDA Staff dated January 11, 2002 and (draft) Guidance for Industry 21 CFR Part 11;
Electronic Records; Electronic Signatures Validation dated August 2001.
The functionality to be verified and validated is defined in the Architecture and Design phase of each
iteration. The goal of verifying and validating is to check whether the software product conforms to the
requirements and to ensure a minimal number of errors in the software. This project document is written
for managers and developers of QI-Bench.
Verification is colloquially described as “did we build the system right,” whereas validation seeks to
assess “did we build the right system.” As such, the verification activity is focused on a granular set of
test indications associated with the System Use Case (SUC) and various Application Scope Documents
(ASD) whereas the validation plan is centered on specific projects that use QI-Bench to accomplish the
activities identified in the Enterprise Use Case (EUC) associated with the business value described in the
Basic Story Board (BSB). We define core validation of QI-Bench with respect to the “test bed” projects
and then layer specific deployments (e.g., the QIBA/RIC and others) on top of that.
2. Relationship to Project Management Plan and Traceability Report
The activities described here are identified under the QI-Bench Project Management Plan (PMP) in
compliance with the BBMSC Development Lifecycle Process. Requirements traceability is documented in
the program Traceability Report (which document originated as the System Requirements Mapping).
3. Verification Plan
Verification proceeds by exercising functionality according to the requirements derived from supported
system use cases.
3.1. Verification Scope
The following figures highlight verifiable system behavior that links to the Verification Protocol. Each is
represented by a figure on how the functionality is related to each other.
Figure 1: Create and Manage Semantic Infrastructure and Linked Data Archives
BBMSC
3 of 6
QI-Bench: Verification and Validation Plan
Rev 1.0
Figure 2: Create and Manage Physical and Digital Reference Objects
Figure 3: Core Activities for Biomarker Development
3.2. Approach and Reporting
Starting from Integration Testing which is scoped within individual applications, verification proceeds
based on the Verification Protocol which identifies test indications for each testable requirement drawn
from the use cases. The test indications often cross application boundaries. A Verification Test Report is
created that documents test results and dispositions of discovered anomalies.
4. Validation Plan
Validation is confirmation by examination and provision of objective (documented) evidence that
computer systems conform to user needs and intended uses, requirements are consistently fulfilled, and
applicable regulatory requirements (such as 21 CFR Part 11, 21 CFR Part 203, 21 CFR Part 820, etc.)
are satisfied.
Validation, at its most basic level, is the process of ensuring that applications, systems, solutions, and/or
environments satisfy intended functionality in a consistent and reliable manner. Primarily, validation is
accomplished via software testing – the process of executing a predefined sequence of steps, emulating
typical user scenarios and data, to determine if the application, system, solution, and/or environment
under evaluation performs as intended. Validation includes ensuring:



Requirements of the system are met (both regulatory and business),
System performance (speed and accessibility) is acceptable, and
Data integrity is properly maintained (not lost, improperly modified, etc.).
Validation can be segmented into a series of distinct activities, all of which require thorough
documentation capturing the plan, execution, and results of the validation effort. Specifically, QI-Bench
validation constitutes the following activities:
4.1. Risk Assessment / Risk Management (RA)
We adopt a “risk based” approach to the validation of instances of QI-Bench. A formalized risk
assessment is conducted to guide the scope of the validation effort, strategically targeting more intensive
testing in highly regulated or high risk areas while minimizing testing in lower risk areas.
BBMSC
4 of 6
QI-Bench: Verification and Validation Plan
Rev 1.0
4.2. Installation Qualification (IQ)
Installation Qualifications (IQ) constitutes the execution of a predefined series of steps that demonstrate
the system is properly installed and ready for production usage.
4.3. Operational Qualification (OQ)
Operational Qualifications (OQ) constitutes the execution of a predefined series of steps, emulating
typical user scenarios, that demonstrate the system meets all applicable functional and regulatory
requirements (appropriate screens are displayed, data can be entered and manipulated, data integrity is
maintained, etc.).
4.3.1. User Scenarios
The following scenarios defined in the use cases are potential targets of validation, based on whether the
scenario is employed in the instance and subject to prioritization based on the risk assessment.
Figure 4: Collaborative Activities to Standardize and/or Optimize the Biomarker
Figure 5: Consortium Establishes Clinical Utility / Efficacy of Putative Biomarker
Figure 6: Commercial Sponsor Prepares Device / Test for Market
4.4. Performance Qualification (PQ)
Performance Qualifications (PQ) constitutes the execution of a predefined series of steps that
demonstrate the system performs consistently within required benchmarks for availability and throughput.
The PQ frequently follows a “system test” approach where a “day in the life” of the user is emulated to
ensure all functionality is suitable.
BBMSC
5 of 6
QI-Bench: Verification and Validation Plan
Rev 1.0
4.5. Validation Reporting
One or more Validation Reports are developed summarizing the results of the validation effort comprising
RA, IQ, OQ, and PQ for the given instance. All testing results are reported, including system deficiencies
(issues with the system), regulatory implications, and recommended actions. An overall statement
indicating whether the system under evaluation can be considered validated for usage or not is also
included.
5. Anomaly reporting and resolution
Everything that is not up to standards or otherwise does not conform to the requirements it should
conform to, is an anomaly. Staff performing verification or validation tests raise potential anomalies to the
system engineer and project manager for disposition, either judging the issue to be acceptable relative to
the test indication or for entry into the project issue management system. When such issues are created,
their identification number is noted in the report.
BBMSC
6 of 6