ISO/IEC 29119 Parts 1 and 4
ISO/IEC 29119 Part 1, Concepts and
definitions
Ossi Taipale,
Lappeenranta University of Technology
Part 1: Concepts and definitions
 This standard defines a risk-based approach to testing. Risk-based
testing is a recommended approach to strategizing and managing testing
that allows testing to be prioritized and focused.
 The ISO/IEC/IEEE 29119 software testing series of standards contain
three standards where conformance may be claimed:
 test processes; ISO/IEC/IEEE 29119-2
 test documentation; ISO/IEC/IEEE 29119-3
 test techniques; ISO/IEC/IEEE 29119-4.
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
2
Software testing is necessary because:
 Information on the quality characteristics of the test item(s) is required
by decision makers;
 The test item(s) being tested does not always do what it is expected to
do;
 The test item(s) being tested needs to be verified (Verification ensures
that software correctly implements a specific function and it answers the
question: are we building the product right?). Basic verification methods
include inspections, walkthroughs, and technical reviews;
 The test item(s) being tested needs to be validated (Validation ensures
that software that has been built is traceable to customer requirements. It
answers the question: are we building the right product?). Validation
activities can be divided into unit testing, integration testing, usability
testing, function testing, system testing, and acceptance testing.
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
3
Introduction to software testing
 Product quality has many aspects, including conformance to
specifications, absence of defects, and fitness in meeting the
requirements of the product's users. ISO/IEC 25010, System and
software quality models defines eight quality characteristics that can be
measured or assessed in testing.
 Testing is a process. A process is a set of interrelated or interacting
activities that transforms inputs into outputs. The objective of this
standard is to present and describe generic testing processes
(ISO/IEC/IEEE 29119-2 Test processes).
 Testing processes and sub-processes can be applied to any phase
or level of testing (e.g. system testing) or type of testing (e.g.
performance testing).
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
4
Introduction to software testing
 Testing can be carried out on a product without executing the product on
a computer. This is called static testing in this standard and in many
areas of the industry, although other standards (e.g. IEEE 1028) may
more specifically call this reviews, walkthroughs or inspections. For
static testing this standard acknowledges and identifies the role of the
tester in these activities even though they may be performed by other
groups within a project or defined by other non-testing standards. This is
because the static testing activities are considered highly important for
complete life cycle testing and test involvement has been shown to be
critical for early defect detection, reduced overall project costs and an
improved ability to meet schedule demands.
 Static testing may also include the use of static analysis tools which find
defects in the code or documents without the code executing (e.g. a
compiler, a cyclomatic complexity analyzer, or a security analyzer for
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
code).
5
Introduction to software testing
 Dynamic testing consists of more than "just" running executable test
items; it also includes both preparation activities and follow-up
activities. The Dynamic Test Processes described in ISO/IEC/IEEE
29119-2 cover each of the activities to be performed in dynamic testing.
 This standard addresses testing; it does not address the other
activities of validation and verification (e.g. V&V analysis, formal
methods).
 Other standards, e.g. ISO/IEC 12207 and IEEE 1012, address also
other verification or validation activities.
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
6
Annex A: The role of testing in verification and validation
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
7
Generic test process
 The relationship between the generic test process, generic test subprocesses, test levels/phases and test types is described in more
detail in Figure 2. The generic test sub-process can be applied in the
following ways:
 As a test level or phase (e.g. component test phase, acceptance test
level);
 As a test type (e.g. performance testing, usability testing). The diagram 2
also makes explicit the relationship between test types and quality
characteristics (as defined in ISO/IEC 25010) ;
 A test sub-process associated with a test level may contain more than
one test type sub-process (e.g. functional and performance testing as part
of system testing); and
 The project test process may be comprised of a sequence of test subprocesses (e.g. component testing, integration testing, system testing and
acceptance testing test sub-processes).
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
8
Figure 2 — The relationship between the generic test subprocess, test levels and test types
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
9
Annex C: Testing in different life cycle models
The development activities performed are more or less the same in all life
cycle models; the main differences lie in the definition of their scopes, the
amount and nature of documentation produced, and the frequency with
which they are repeated during the course of the development life cycle.
 Agile
 Evolutionary
 Sequential (i.e. the waterfall model)
Scrum is not a development methodology (i.e. it does not tell you the best
method for describing requirements or how to write code) and is better
described as a project management framework within which an agile
methodology is applied by the developers (XP is often used).
 A Scrum project comprises a number of iterations called sprints, with
each sprint normally resulting in new functionality that can be delivered to
the users (Figure).
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
10
29119 Part 4, Test techniques
Ossi Taipale,
Lappeenranta University of Technology
Part 4, Test techniques: Introduction
 The purpose of ISO/IEC 29119-4 Test Techniques is to provide an
International Standard that defines software test design techniques
(also known as test case design techniques or test methods) that can
be used during the test design and implementation process of ISO/IEC
29119-2 Test Processes.
 The test design techniques presented in ISO/IEC 29119-4 can be used to
derive test cases that, when executed, can be used to collect evidence
that test item requirements have been met and/or that defects are
present in a test item (i.e. that requirements have not been met).
Activities:

Derive Test Conditions (TD2),

Derive Test Coverage Items (TD3), and

Derive Test Cases (TD4).
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
12
Introduction: Figure 1, ISO/IEC 29119-2 Test Design and
Implementation Process
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
13
Introduction
 Experience-based testing practices like exploratory testing and other test
practices such as model-based testing are not defined in ISO/IEC
29119-4 because this standard only describes techniques for
designing test cases. Test practices such as exploratory testing are
described in ISO/IEC 29119-1.
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
14
Test Design Techniques
 ISO/IEC 29119-4:
 In specification-based testing, the test basis (e.g. requirements,
specifications, models or user needs) is used as the main source of
information to design test cases - "black-box testing“.
 In structure-based testing, the structure of the test item (e.g. source
code or the structure of a model) is used as the primary source of
information to design test cases - "white-box testing" (or "clear-box
testing").
 Both - "grey-box testing"
 In experience-based testing, the knowledge and experience of the
tester is used as the primary source of information during test case
design.
 In practice they can be used interchangeably.
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
15
Test design techniques presented in ISO/IEC 29119-4
 Figure 2, page 7
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
16
Equivalence Partitioning
Derive Test Conditions (TD2)
 Equivalence partitioning (BS 7925-2:1998; Myers 1979) uses a model of
the test item that partitions the inputs and outputs of the test item
into equivalence partitions (also called “partitions” or “equivalence
classes”), where each equivalence partition shall be defined as a test
condition.
 These equivalence partitions shall be derived from the test basis, where
each partition is chosen such that all values within the equivalence
partition can reasonably be expected to be treated similarly (i.e. they
may be considered "equivalent") by the test item. Equivalence partitions
may be derived for both valid and invalid inputs and outputs.
Derive Test Coverage Items (TD3)
 Each equivalence partition shall also be identified as a test coverage
item
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
17
Equivalence Partitioning
Derive Test Cases (TD4)
 Test cases shall be derived to exercise the test coverage items (i.e. the
equivalence partitions).
Approach for selecting test coverage items:
 One-to-one, in which each test case is derived to cover a specific
equivalence partition. The number of test cases derived using one-to-one
equivalence partitioning equals the number of test coverage items.
 Minimized, in which equivalence partitions are covered by test cases such
that the minimum number of test cases derived covers all
equivalence partitions at least once.
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
18
Example: Equivalence Partitioning
Test basis:
 The component receives as input an exam mark (out of 75) and a
coursework (c/w) mark (out of 25), from which it outputs a grade for the
course in the range 'A' to 'D'. The grade is calculated from the overall
mark, which is the sum of the exam and c/w marks, as follows:




greater than or equal to 70 - 'A'
greater than or equal to 50, but less than 70 - 'B'
greater than or equal to 30, but less than 50 - 'C'
less than 30 - 'D'
 Where a mark is outside its expected range then a fault message ('FM') is
generated. All inputs are passed as integers.
 Page 42
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
19
Annex A1: Testing Quality Characteristics
 This annex contains examples of how quality characteristics defined in
ISO/IEC 25010 (ISO/IEC 25010:2011) can be mapped to the test design
techniques defined in ISO/IEC 29119-4.
Ohjelmistotestauksen standardi ISO/IEC 29119 kurssi 12-13.2.2014, Innopoli 2, Espoo
20