Software testing standards ISO/IEC
29119 and 33063
1. ISO/IEC 29119 software testing
2. ISO/IEC 33063 assessment of testing
Ossi Taipale, Heikki Uusitalo, Risto Nevalainen
FISMA r.y, ISO/IEC SC7
SWG 1
Business Planning
Group
JTC 1 SC7 Structure
LCPHAG
SWG 5
Standards
Management Group
SWG22
Secrétariat
WG2
Systems & Software
Documentation
WG4
Tools and
Environment
WG6
Software Product
Measurement and
Evaluation
WG7
Life Cycle
Management
Life Cycle Process
Harmonization Advisory
Group
SC7
WG10
Process
Assessment
WG19
Techniques for
Specifying IT Systems
WG20
Software Engineering
Body of Knowledge
WG21
Software
Asset Management
Vocabulary
Maintenance
WG23
Systems Quality
Management
WG24
SLC Profiles and
Guidelines for VSE
WG25
IT Service
Management
WG26
Software Testing
WG27
IT Enabled Services
(BPO)
WG28
CIF Usability
WG40
IT Governance
WG42
Architecture
Software testing standards ISO/IEC
29119
Contents of the ISO/IEC 29119 testing standard
Part 1 Concepts and Definitions
Part 2 Testing Process
Part 3 Test Documentation
Part 4 Test Techniques
ISO/IEC 29119 Part 1: Concepts
and Definitions
Scope
This standard covers definitions and concepts in systems and software
testing.
Example, test case
set of test case preconditions, inputs (including actions, where applicable),
and expected results, developed to determine whether or not the covered part
of the test item has been implemented correctly.
NOTE A test case is the lowest level of test input (i.e. test cases are not made
up of test cases).
Generic process
Testing can be described as a process. A process is a set of interrelated or
interacting activities that transforms inputs into outputs. The objective of this
standard is to present and describe a generic testing process.
Life cycle models
Agile
Evolutionary
Sequential (i.e. the waterfall model)
ISO/IEC 29119 Part 1: Concepts
and Definitions
Static testing
Testing can be carried out on a product without executing the product on a
computer. This is called static testing in this standard and in many areas of
the industry, although other standards (e.g. IEEE1028, Standard for
Software Reviews and Audits) may more specifically call this reviews,
walkthroughs or inspections.
For static testing this standard acknowledges and identifies the role of the
tester in these activities even though they may be “owned” by other groups
or standards within a project. This is because the static testing activities
are considered highly important for complete lifecycle testing and test
involvement has been shown to be critical for early defect detection,
reduced overall project costs and an improved ability to meet schedule
demands.
ISO/IEC 29119 Part 1: Concepts
and Definitions
The relationship between the generic test sub-process, test levels and test
types.
ISO/IEC 29119 Part 1: Concepts
and Definitions
The Role of Testing in Verification and Validation
This standard addresses only parts of the verification and validation
activities. Other standards, e.g. ISO/IEC 12207, Software life cycle
processes, address other verification or validation activities. Testing, as
described in this standard, addresses both dynamic and static testing as
well as associated sub processes and activities. This standard does not
address, except in passing, demonstration activities or validation and
verification. To provide complete validation and verification of a product an
organization will need to use this standard in conjunction with other
standards as part of a comprehensive engineering program. See Annex A
for a diagram of verification and validation activities. (IEEE 1012, Standard
for Software Verification and Validation)
ISO/IEC 29119 Part 1: Concepts
and Definitions
Hierarchy of Verification and Validation
activities
ISO/IEC 29119 Part 2: Test Process
The multi-layer test process
Three levels, the approach is risk-based
ISO/IEC 29119 Part 2: Test Process
The multi-layer model showing all test processes
ISO/IEC 29119 Part 2: Test Process
Example test management process relationships
ISO/IEC 29119 Part 2: Test Process
ISO/IEC 29119 Part 2: Test Process
Dynamic test processes
ISO/IEC 29119 Part 2: Test Process
ISO/IEC 29119 Part 3: Test
Documentation
Test Documentation, includes templates and examples of test
documentation that are produced during the test process.
The test documentation in this International Standard describes
the output of the processes specified in part two of this standard.
ISO/IEC 29119 Part 4: Test
Techniques
The test
techniques in this
International
Standard are used
to derive the test
cases executed as
part of the dynamic
testing process
specified in part
two of this
standard.
ISO/IEC 29119 Part 4/Part 2: Test
Design and Implementation Process
ISO/IEC 29119 Part 4: Test
Techniques
Test Coverage Measurement
The coverage measures defined in this part of the standard are based
on test design technique coverage.
Coverage levels can range from 0% to 100%.
Quality Characteristics
Software testing can be carried out to collect evidence that required
quality criteria have been satisfied by a test item. Required quality
characteristics should be specified in the test basis. Definitions of quality
characteristics could be derived from ISO/IEC 25010 System and
Software Product Quality Requirements and
Evaluation (SQuaRE) – System and Software Quality Models.
ISO/IEC 29119 Part 4: Test
Techniques
Mapping Quality Characteristics (ISO/IEC 25010) to Test Design Techniques
and Types of Testing
ISO/IEC 29119 Part 5: NWIP,
Keyword-Driven Testing
Keyword: Word or term used as a reference to a specific action performed
during the execution of a test case.
ISO/IEC 33063 Testing assessment
ISO/IEC 33063 Testing assessment
ISO/IEC 33063 Testing assessment
Assessment indicators