Verification and Validation:
A Quick Introduction
Authors
Massood Towhidnejad
Mike Rowe
David Dampier
Sponsored In Part by NSF Grant EEC-0080502
And published on www.swenet.org
Learning Objectives
1.
2.
3.
4.
5.
6.
The student will be able to define both verification and
validation, and to know the difference between them.
The student will be able to identify the appropriate activities
and their classification as either verification or validation.
The student will be able to distinguish between verification
and validation, given a set of activities.
Given a set of software artifacts, the student will understand
the appropriate verification or validation activity to that
artifact.
Given a set of projects in different categories, the student
will be able to determine which category of projects would
require more V&V activity, and which would require less.
Given a set of defects and the phases of development in
which they were found, the student will be able to estimate
the relative cost of correcting errors early or late in the
lifecycle.
Why Invest in Quality?






Cost effective
Provides competitive edge
Essential for business survival
Essential for international marketing
Helps to retain customers, and
increase profits
Hallmark of world class business
Ways to Improve Quality

Prevention of Defects
• Process Improvement
• Complexity Reduction
• Risk Management
• Causal Analysis

Detection and Correction of Defects
• Verification
• Validation
• Rework
• Causal Analysis
Verification



Verification – Are we building the product
right?
Verification is any checking process
conducted on software artifacts in an
attempt to determine if they work as
specified by the designers of the system.
Includes reviews, inspections,
walkthroughs, unit testing and integration
testing.
Validation



Validation – Are we building the right
product?
Validation is the process of evaluating
software artifacts during the software
development process in an attempt to
determine if the system works as required
by the customers. Any evaluation activity
that involves the customer can be used for
validation purposes.
Includes program reviews, system testing,
customer acceptance testing.
Verification vs. Validation

Verification
• Main purpose is to detect defects in the
artifacts of the system under
development.

Validation
• Main purpose is to show that the system
under development meets user needs,
requirements, and expectations.
Verification & Validation
Techniques

Static Methods
• Techniques applied to artifacts without
execution.

Dynamic Methods
• Techniques applied to artifacts through
execution.

Mathematically Based Methods
Static: Reviews

Walkthroughs
• Code
• Document
 ConOps, SRS
 STEP, SAD, SDD

validation
verification
Inspections
• Code
• Document Audits

verification
verification
verification
Program Reviews
• Customer involved
• No customer
validation
verification
Effectiveness of Static Verification



More than 60% of program defects can be
detected by program inspections.
More than 90% of program defects may be
detectable using more rigorous
mathematical program verification.
The defect detection process is not
confused by the existence of previous
defects.
Dynamic: Testing (Verification)

Unit Test (Detailed Design):
• Testing the individual software modules,
components, or units.

Integration Testing (Architectural Design):
• After unit test, the system is put together in
increments. Integration testing focuses on the
interfaces between software components (OO
thread-based, cluster-based testing)

System Testing (Requirements Spec):
• One goal of system testing is to ensure that
the system functions as specified in the
specification.
Dynamic: Testing (Validation)

System Testing (Requirements Spec):
• Another goal of system testing is to ensure
that the system functions as the client
expected in a controlled environment.

User Acceptance Test (ConOps):
• A set of formal tests run for the client, and
specified by the client. When the system
passes these tests, the software has been
accepted by the client as meeting the
requirements.
Verification and Validation in the
Development Lifecycle
Validate the System
Requirements
Analysis
Verify Design
Design
Execute
System Tests
Black-Box
Testing
Execute
Integration
Tests
Verify Implementation
Code
Execute Unit
Tests
White-Box &
Black-Box
Testing
Mathematics-Based Verification



Verification is based on mathematical
arguments which demonstrate that a
program is consistent with its
specification.
Programming language semantics
must be formally defined.
The program must be formally
specified.
Why Inspection is important?


Relative cost to fix a defect
Phase in which found Cost ratio
(hours)
– Requirement
1
– Design
3-6
– Coding
10
– Testing
15-70
– Operation
40-1000
Data derived from Capers Jones.
Costs of Finding and Fixing Late
Delays in identifying and fixing
defects gets geometrically more
expensive as the lifecycle progresses!
$100
$10
$1
Time in (Phase of) Development
Defects
Cost of Fixing a Defect
$1000
Cost of Quality


Includes all costs of quality-related
activities.
Quality costs =
+Prevention costs
+Detection and Appraisal costs
+Failure costs
•
•

Internal failure costs
External failure costs
Cost of Quality (COQ) – refers to the
cost of correcting defects once found.
Quality Cost Components

Direct Cost
– Reviews/inspections
– Unit testing
– System testing
– Acceptance testing
– Test planning and
design
– Computer time
– Resources (terminals,
staffs, etc.)

Indirect Cost
– Rework
– Recovery
– Corrective action cost
– Failures
– Analysis meeting
– Debugging
– Retesting
– Legal fees
Verification or… Validation?





Reviews
Unit Testing
Integration Testing
System Testing
Acceptance Testing
Verification or… Validation?

Reviews

Unit Testing



Integration Testing
System Testing
Acceptance Testing
Either
Verification or… Validation?

Reviews
Either

Unit Testing
Verification

Integration Testing


System Testing
Acceptance Testing
Verification or… Validation?

Reviews
Either

Unit Testing
Verification

Integration Testing
Verification

System Testing

Acceptance Testing
Verification or… Validation?

Reviews
Either

Unit Testing
Verification

Integration Testing
Verification

System Testing
Validation

Acceptance Testing
Verification or… Validation?

Reviews
Either

Unit Testing
Verification

Integration Testing
Verification

System Testing
Validation

Acceptance Testing
Validation
Balancing Trade-Offs


What is the real requirement?
What are the expected benefits of
V & V vs.
– cost of increasing V&V activity
– additional time required
– technical feasibility
– risk of not finding defects
» delivery
» operation
“When is it good enough?”
(Bach, 1997)
Summary


Verification and Validation are both
important to the quality of software.
Each is different:
• Verification determines that what we are doing
is done correctly.
• Validation determines whether we are doing
the right thing.


Sometimes, less quality is okay. Some
software systems require less V&V, as the
cost is not supportable. Some require
more no matter what it costs.
The earlier defects are discovered, the less
it will cost to correct them.
Backup Slides
Independent Verification and
Validation (IV&V)
Subs
Project
Manager
•Design
• Coding
• Testing
• System Requirements
• Contractor Management
Discrepancies
IV&V
• Evaluate Requirements • Monitor Tests
• Integration
• Evaluate Design
Tested
Products
Static and Dynamic
Verification & Validation
Static
verification
Requirements
specification
High-level
design
Formal
specification
Detailed
design
Program
Dynamic
validation
Prototype
Sommerville, p. 421
Test Case Development in the
V Model
Validate Requirements
Requirements
Analysis
Verify Design
Design
Execute
System Tests
Execute
Integration
Tests
Verify Implementation
Code
Test cases designed
and executed by
independent testers
Execute Unit
Tests
Test cases designed and
executed by developers