Abd Azis Bin
Bon
Cao Jing
Edward Bauer
Herman Bin Md
Tahir
Jongkwan Lee
Adalberto
Crespo
Celestina
Bianco
Eric Gauthier
Hohaeng Cho
Alistair Walker
Cerys Giddings
Fernando Gebara
Manus
Filho
Hyung jin Yoon Jung-hwan Kim Chakraborty
Juichi
Takahashi
Muhammad
Risto
Magnus Ohlson Azam Bin Alias Nevalainen
Susan Burgess
Manikandan K
N Swaminathan Rolf Ziegler
Sylvia
Veeraraghavan Tristan Brown
Nathalie van
Delft
Tafline Ramos
(Murnane)
Satomi
Yoshizawa
Ton van
Bergeijk
Tsuneo
Yamaura
Cerys
Alphonse Philipe Willoughby
Francois Coallier Ian Hurst
Kazuo Yabuta
Amiruddin Jaafar
Sidek
Chai Afeng
Francois
Darphin
Ine Lutterman
Keerthishankar Mastura Abu
KN
Samah
Cheng (Kevin)
Anne Mette Hass Geng
Gary Roedler
James Moore
Keum-Ran Lee
Arun Kumar
Melkote
Grantham Daniels Javier Tuya
Klaudia DussaZieger
Matthias Daigl Patricia Huyos Soo-yeon Lee
Thomas George Yasuhara Nishi
Atsushi Mitomi Cheryl George
Guizhi (Tracy) Jean-Louis
Yang
Michel
Kouichi
Akiyama
Matthias
Rasking
Paula Angeieri Steve Wilsher
Thomas
Kurihara
Yu-Whoan Ahn
Barbara Huisman Cheryl Jones
Gustavo
Valencia
Jim Moore
Kwangik Seo
Michael Crerar
Qin Liu
Stuart Reid
Tim Archer
Yuji Shinoki
Jon Hagar
Kwanho Kim
Moohee Han
Radoslaw
Hofman
SuJin Choi
Tohru
Matsuodani
Yumoto
Tsuyoshi
Leon Gerber
Ricardo
Magnus Ohlson Cristalli
Sunny Kwon
Tom Wissink
Liliana Gomez
Arenas
Cheong Jeong
Seo
Beatrix
Barafort
Dan-Hyung Lee Hack-Youp Noh
Boris Pozin
Dnyanada
Wagh
Jonathan
Hareton Leung Earthy
Mariye Akkaya Ogawa Kiyoshi
Matt Mansell
Satoshi Masuda Tan Liqun
Ursula Parker
Ola Perman
Shankaranarayan
an G A
Terry Rout
Vijay Sampath
Ossi Taipale
Shinji Iwata
Wonil Kwon
Theresa Hunt
Alistair Walker, Alphonse Phillipe, Anne Mette Hass, Jon Hagar, Matthias Daigl, Qin Liu
Stuart Reid, Sylvia Veeraraghavan,Tafline Ramos (Murnane),
Tom Wissink, Ursula Parker, Wonil Kwon
PROCESS
ISO/IEC 29119 –Structure
Part 1
Concepts & Vocabulary
BS 7925-1
Part 4
Part 2
Part 3
Testing
Techniques
Processes
Documentation
BS 7925-2
‘Part 5’
Process
Assessment
IEEE 829
Actually ISO/IEC 33063
Part 1 Software Testing Concepts
UPDATED PART 1
Scope, Conformance, Normative References
SOFTWARE TESTING CONCEPTS
Static Testing
Static
Analysis
Reviews,
Inspections,
etc.
REF: IEEE 1028
TESTING IN DIFFERENT LIFE CYCLE MODELS
ROLES AND RESPONSIBILITIES IN TESTING
ANNEXES – Metrics, Examples, Bibliography
Part 1 view on heuristics
“5.1.3 Testing as a Heuristic
In engineering (and software engineering) a heuristic is
an experience-based (trial and error) method that can
be used as an aid to problem solving and design.
However, while heuristics can be used to solve problems,
they are fallible in that sometimes they may not solve, or
only partially solve, a problem.
“Recognition that the manner in which testing is
undertaken may be fallible allows us to mitigate the risk
of an ineffective test strategy by employing multiple test
strategies.”
Part 2 – Test Process
PART 2 - BEFORE
PART 2 - AFTER
ORGANIZATIONAL TEST
PROCESS
ORGANIZATIONAL
TEST PROCESS
TEST MANAGEMENT
PROCESSES
TEST MANAGEMENT
PROCESSES
X
STATIC
TEST
PROCESSES
DYNAMIC
TEST
PROCESSES
DYNAMIC TEST
PROCESSES
Part 3 – Test Documentation
PART 3
Scope,
Conformance,
Normative References
TEST
DOCUMENTATION
ANNEXES
EXAMPLES
Agreed text
based on
IEEE 829
(making it
normative)
Part 3 Test Documentation
Part 3 Test Documentation
Part 3 Test Documentation
• Part 5 Keyword Driven Testing
• Part 6 ISO/IEC 33063
Process Assessment
Model for
Software Testing
• Agile extension?!
Washington 2010
•
•
•
•
•
•
•
•
•
•
Vote 1: Static testing is a part of
testing.
For: 6 (countries voted for:
Australia, China, Denmark, India,
South Korea, Sweden)
Against: 4 (countries voted against:
Canada, Japan, South Africa, USA)
Abstain: none
Majority is for. Static testing is a
part of testing.
Vote 2: Inspections are a separate
approach (not part of testing).
For: 4 (countries voted for: Canada,
Japan, South Africa, USA)
Against: 5 (countries voted against:
Australia, China, Denmark, India,
South Korea)
Abstain: 1 (Sweden)
Majority is against. Inspections are
not a separate approach (i.e.
inspections are a part of testing).
ISO Liaisons
•
•
•
•
•
•
•
•
•
V&V Study Group
WG4 (Tools NWIP)
WG7 (12207/Static Testing)
WG10 (Test Process Assessment)
WG42 (Architecture Evaluation NWIP)
DIN (Keywords NWIP)
WG6 (Metrics/Definitions)
Framework for Product Quality Achievement SG
System Integration SG
Liaison with TMMi
Benefits of a liaison with TMMi:
• Free marketing of the 29119 standard by
TMMi, which would be a new market for
WG26
• The opportunity for sales of the 29119
standard to members of the TMMi community
and users of TMMi material
• Access to experts from TMMi to contribute to
and comment on 29119standards
Other Liasons
• European Telecommunications Standards Institute
– Methods for Testing Specifications
– Free marketing of the 29119 standard by ETSI to a
market of security testers who may not have
visibility of the standard at present
• ISTQB
– Request, by Rex Black, in November 2008 but not
finalised then. Re-proposed in November 2014
Petition
To the President of the International Organization for
Standardisation,
We, the undersigned, hereby petition the International
Organization for Standardisation to suspend publication of
ISO/IEC/IEEE 29119-4 and ISO/IEC/IEEE 29119-5, and to
withdraw ISO/IEC/IEEE 29119-1, ISO/IEC/IEEE 29119-2 and
ISO/IEC/IEEE 29119-3.
It is our view that significant disagreement and sustained
opposition exists amongst professional testers as to the
validity of these standards, and that there is no consensus
(per definition 1.7 of ISO/IEC Guide 2:2004) as to their
content.
The working group discussed the „Stop
29119‟ campaign
• The working group agreed that the article that
Stuart published on the
softwaretestingstandard.org website rebutting
the arguments of the 29119 campaign was a
suitable response to the campaign.
• The working group agreed that it would be
beneficial to upload links and documents to the
softwaretestingstandard.org website that debunk
the myths that are being spread on the standards
and on the working group by the campaigners.