Systems Validation &
Verification, Quality and
Standards (CSE4431)
Dr Sita Ramakrishnan
School CSSE
Monash University
1
Systems Validation & Verification,
Quality and Standards (CSE4431)
Topics:
Building a Software Testing Environment
Establishing a Testing Policy
Building a software testing strategy
Establishing a software testing method
incorporating testing strategy & testing
tactics
Include test plans, test criteria, testing techniques
and testing tools used in the V&V of SDLC as part
of the Testing tactics
© S Ramakrishnan
2
Building a Software Testing
Environment
Establishing a Testing Policy:
Testing software systems is an organisational
issue - not just a project or departmental concern
in order to meet the organisational needs• cope with technology entrenchment -e.g. stock markets’
core business or airlines booking system
• cope with technology proliferation - multiple users on
different platforms - interoperability issues
• interconnected systems with common data - error in one
affects the other
• Develop a test policy under IT departmental direction,
but ensure that it also captures the organisational
requirements© S Ramakrishnan
3
Establishing a Testing Policy
Software testing is an infrastructure technology or
“infratechnology”. Infratechnologies are technical
tools to measure conformance, performance, and
interoperability during SD, and including scientific
& engineering data, metrics, test methods, best
practices & techniques widely used in the industry
(NIST Report quoting Tassey, 1997 - The Economics of R & D
Policy, Westport, CT: Quorum Books)
Software testing infratechnologies also provide
critical information about software quality to the
software user
© S Ramakrishnan
4
Establishing a Testing Policy
Effective/ adequate testing must be done by a test team IT testers, quality control, QA & inspectors
Inadequate testing and impact on software industry due to
the lack of (use of) robust standardized test technology
(such as reference data & metrics in spite of stds for test
doc. & V& V activities (IEEE/ANSI 1993) and Carnegie
Mellon’s CMM model) to address
•
•
•
•
interoperability & integration testing
automated generation of test code
lack of readily available test measurement procedures
lack of a good method to tell when to stop testing &
release a product, and results in:
• increased failures & defects due to poor quality software
• increased time to market due to inefficient testing
• increased software development costs
© S Ramakrishnan
5
Establishing a Testing Policy
Establishing a testing policy involves setting up
clear policy by the IT management and signing off
that they endorse the policy, and communicating
to the IT dept that they will enforce the regime
(Perry 2000).
A Testing policy involves setting the criteria to be
followed such as:
•
•
•
•
A clear and concise definition of Testing
Test method to be followed
Evaluation criteria - metric costs for undetected defects
Standards against which testing will be measured
© S Ramakrishnan
6
Establishing a Testing Policy,
Skills set of Test Engineers
A Structured Organisational wide Testing policy
needs to be cognizant of the test maturity of the
organisation (revisit this later)
Also look at Quality Assurance Institute’s Common
Body of Knowledge (CBOK) for certified software
test engineer (CSTE) later in the semester - show
your competencies against the skill categories
needed to become CSTE
© S Ramakrishnan
7
Building a software testing
strategy
The recommended test process involves testing in each phase of SDLC
Plan Acceptance Tests
Requirements
Acceptance
Testing
Plan System Tests
Specification
System Testing
Plan Integration Tests
Design
Integration
Testing
Plan Unit Tests
Implementation
Software
Development
& Test
Model -
Unit Testing
V-Model
© S Ramakrishnan
8
Building a software testing
strategy
The recommended test process involves the
following testing activities appropriate to that
phase of SDLC:
• analyse the structures produced for adequacy (of
requirements, design, coding as appropriate) and for
testability
• generate test data (functional &/or structural) based on
the structure at this phase
• determine structures are consistent with the structures
produced in the previous phase
• Refine test data as required
© S Ramakrishnan
9
Establishing a software
testing method
Testing method incorporating testing strategy &
testing tactics
© S Ramakrishnan
10
Testing Tactics
Include test plans, test criteria, testing techniques
and testing tools used in the V&V of SDLC as part
of the Testing tactics
 (more to come ……)
© S Ramakrishnan
11