Laatuominaisuuksien standardin ISO25010
ja testausarviointistandardin ISO33063
esittely
Risto Nevalainen, Senior Advisor, FiSMA ry
Sisältö



Hieman FISMAsta
SC7 standardoinnista
WG6 ja SQUARE, ISO/IEC 25000
 Standardit ja mallit, lähinnä 25010 sarja
 Esimerkkejä
 Mitä seuraavaksi

WG10 ja Testing SPICE, ISO/IEC 33063
 Standardit ja mallit, 330xx sarja
 33063, esimerkkejä
 Mitä seuraavaksi

Keskustelua

HUOM: esittelykalvot ovat englanniksi, helpompi esittää standardit niin ja
on valmista materiaalia. Käännöstöitä ei nyt harkinnassa.
FiSMA 2012
2
FiSMA organisation 2012 (changes possible)
Annual meeting
Board
6 members
CP: Tuukka Haarni
Managing Director
Erkki Savioja
Senior Advisor
Risto Nevalainen
FiSMA SPIN
CP: Jukka
Märijärvi
Testing Standards
Standards and
and Models
IT service mgmt
Models
CP: Heikki
CP: Jyrki Lahnalahti,
CP: Kimmo Vaikkola,
Uusitalo,
Inspecta
Fujitsu
Endero
Senior Advisor
Pekka Forselius
Scope Manager
Forum
CP: Hannu
Lappalainen
FiSMA 2012
3
SWG 1
Business Planning
Group
JTC 1 SC7 Structure
LCPHAG
SWG 5
Standards Management
Group
SWG22
Secrétariat
WG2
WG10
Systems & Software
Documentation
WG4
Process
Assessment
WG19
Tools and
Environment
WG6
Techniques for
Specifying IT Systems
WG20
Software Product
Measurement and
Evaluation
Software Engineering
Body of Knowledge
WG21
WG7
Life Cycle
Management
Life Cycle Process
Harmonization Advisory
Group
SC7
Software
Asset Management
Vocabulary
Maintenance
WG23
Systems Quality
Management
WG24
SLC Profiles and
Guidelines for VSE
WG25
IT Service
Management
WG26
Software Testing
WG27
IT Enabled Services
(BPO)
WG28
CIF Usability
WG40
IT Governance
WG42
Architecture
ISO/IEC JTC1/SC7/WG6 Responsibility and Structure
Title:
 Software Product and Systems Quality
Scope:
 Development of Standards and Technical Reports for Software
Product and System Quality Requirements, Measurement and
Evaluation
Organizational Structure:
 SQuaRE Series (ISO/IEC 25000 ~ 25099) => WG6
 Functional Size Measures => WG6/FSM SG
 CIF (ISO/IEC 25060 ~ 25069) => JWG => WG28
FiSMA 2012
5
Organization of SQuaRE series of International Standards
FiSMA 2012
6
Quality model division, 2501x
 This division states the general requirements for a quality model,
recommended model, and guides to customize and use the model.
 This division consists of two parts:
 ISO/IEC 25010: System and software quality models
 A quality in use model composed of five characteristics (11 subc.).
 A product quality model composed of eight characteristics (31 subc.).
 ISO/IEC 25012: Data quality model
 The data quality model defined in this International Standard
categorizes quality attributes into fifteen characteristics (22 views)
considered by two points of view: inherent and system dependent.
 New: ISO/IEC 25011 Service Quality
 Professor Markku Tukiainen is nominated as co-editor by Finland
FiSMA 2012
7
SQuaRE Architecture and projects
ISO/IEC 2503n: Quality
Requirement Division
25030: Quality
Requirements (IS)
ISO/IEC 2501n: Quality Model Division
25010: System and software quality models (IS)
ISO/IEC 2504n:
Quality Evaluation Division
25040: Quality Evaluation
Process (IS)
25012: Data Quality Model (IS)
ISO/IEC 2500n: Product Quality General Division
25000: Guide to SQuaRE (IS)
25000 Rev.: Guide to SQuaRE (WD)
25041: Evaluation Guide for
Developers, Acquirers and
Independent Evaluators (DIS)
25001: Planning and Management (IS)
ISO/IEC 2502n: Quality Measurement
Division
25020: Measurement Reference Model (IS)
2504n: Evaluation Module (NYA)
25021: Quality Measure Elements (TR)
25021: Quality Measure Elements (DIS)
25045: Evaluation Module for
Recoverability (IS)
25022: Measurement of Quality in Use (WD)
25023: Measurement of Sys & SWP Quality (WD)
25024: Measurement of Data Quality (WD)
ISO/IEC 25050 ~ 25099: SQuaRE Extension Division
25051: Requirements for quality of COTS
software product and instructions for testing (IS)
25051 Rev.: Requirements for quality of COTS
software product and instructions for testing (WD)
ISO/IEC 25060 ~ 25069:
Common Industry Format for Usability
JWG JTC1/SC7 and ISO/TC159/SC4
ISO/IEC
Hyderabad
Report
ISO/IECJTC1/SC7/WG6
JTC1/SC7/WG6 Paris
Meeting Meeting
Report - 2011
-05
8
FiSMA 2012
8
8
Overview of using SQuaRE
Customer
User
Development Organization
provide
guidance
Product Quality
Requirements
supports
Acquirer
Evaluator
Developer
determines
guidance
perform
Product Quality
Evaluation
supports
Product Quality
Model and Measurement
FiSMA 2012
9
Structure used for the quality models
Note: also 330xx will use the same structure in near future
FiSMA 2012
10
Quality in the lifecycle
Process
Effect of software
product
Software product
influences
Process
quality
influences
influences
Internal
properties
depends on
External
properties
depends on
Quality
in use
depends on
Contexts
of use
Process
measures
Internal
measures
External
measures
Quality in use
measures
FiSMA 2012
11
Quality in use model in 25010
FiSMA 2012
12
Product quality model (system, software) in 25010
FiSMA 2012
13
Next steps in SQUARE
 25051 ballot for renewal, CD is closing 15.10.2012
 25020 series is now started, will add details in quality
models by defining set of metrics per model
 25020, 25021, 25022, 25023, 25024
 So far 9126 is still valid
 Professor Markku Tukiainen is nominated as co-editor by Finland in
25022
 25040 series is also under work, mainly in CD phase
FiSMA 2012
14
Ohjelmistotuotteen laadun mittarit, 2 kpl FiSMA suositus

Suositeltu metriikka: A.1 Käyttäjän työn tehostuminen
 Tyyppi: Johdettu mittari
 Oleellinen sisältö: Käyttäjätehtävien aste, jota ohjelmisto tukee verrattuna
kaikkiin käyttäjätehtäviin. Suositeltu metodi on tapaustutkimus.
 Mitä mittari kertoo: Kuinka hyvin ja kattavasti ohjelmisto on toteutettu
täyttämään käyttäjän tarpeita.

Suositeltu metriikka: A.2 Käyttäjän tyytyväisyys
 Tyyppi: Suora mittari
 Oleellinen sisältö: Käyttäjän kokema tyytyväisyys (user experience). Voidaan
jakaa osa-alueisiin. Suositeltu metodi on Net Promoter.
 Mitä mittari kertoo: Kuinka käyttäjä kokee ohjelmiston toimivan hänen
kannaltaan, esim. käytettävyys ja esteettömyys.
FiSMA 2012
15
ISO/IEC JTC1/SC7/WG10 Responsibility and Structure
Title:
 Process Assessment
Scope:
 Development of Standards and Technical Reports for Software Product
and System Quality Requirements, Measurement and Evaluation
Organizational Structure:
 SPICE Series, Published (ISO/IEC 15504 Parts 1 -10)
 SPICE Series, Under development (ISO/IEC 3300xx)
 Benchmarking, ISO/IEC 29155 (several parts)
FiSMA 2012
16
ISO/IEC 330xx Plan
FiSMA 2012
17
ISO/IEC 33063 exemplar process assessment model for
software testing
 One of the assessment models.
 Other identified models are software, systems and services
assessment model.
 Started almost two years ago
 WD in 2011
 CD1 during winter 2012
 Lot of comments, also from SYTYKE/Testaus OSY
 Finland voted against: too complex, too much overlap
 Current draft is CD2
 Simplified structure: 8 processes
 Guidance on how the planning and scoping of an assessment is done
with this process assessment model for software testing (Annex B)
 NOTE: Also other test assessment models are developing!
FiSMA 2012
18
Overall architecture
FiSMA 2012
19
Processes in ISO/IEC 33063
FiSMA 2012
20
Assessment indicators
FiSMA 2012
21
Example: Test Completion process (Normative part)
FiSMA 2012
22
Example: Test Completion Process (practice indicators)
FiSMA 2012
23
Example: Test Completion Process (work product
indicators)
FiSMA 2012
24
Capability indicators, capability levels in 33063
 Same structure as in draft CD3 ISO/IEC 33020 ”Process
measurement framework for assessment of process capability and
organizational process maturity”




9 process attributes
6 capability levels, ordinal value
NPLF rating for process attribute rating
Additional indicators as in ISO/IEC 15504-5:2012 model (software)
 No maturity scale
 No definition or adaptation of assessment by different classes
 Note: Possibility that testing (and V&V) are in future also in some
other standards in 330xx family
 Note: Possibility that someone proposes other kind of process
quality characteristics than process capability
FiSMA 2012
25
Next steps in 33063
 CD ballot is open, closes 21.11.2012
 FiSMA meeting is 1.11.2012 to handle comments
 Also comments from Testaus OSY are welcome!
 Expected future: DIS ballot winter 2013(if ISO/IEC 33020 moves also to
DIS), publication late 2013
 Note: many other parts in 29119 are also now under ballot
 29119-4 DIS ballot closes 21.11.2012
 Note: NWIP proposal in topic ”Keyword driwen testing” is open,
closes 22.12.2012. Any volunteer from Finland to participate?
FiSMA 2012
26
Guidance in using 33063 at more detailed level
 Each process can be assessed at more detailed level
 Example: Test Management process can be rated separately for
Unit testing, System testing and for Reliability testing
 Need for aggregation: higher level rating is combination of
several lower level ratings. What is the aggregation rule?
 Also processes from other models can be included in
the assessment scope
 Example: DEV.5 Software Integration from ISO/IEC 15504:5
(2012)
FiSMA 2012
27
Training for FiSMA members in testing standards during
2013
 Two courses are planned for 2013, no details yet:
 2-day 29119 course
 2-day Testing SPICE (33063) course
 No policy how Testaus OSY members can participate, any ideas?
 Note: One SPICE training week is also planned, focuses
on highly critical applications (Nuclear SPICE etc)
 Note: Safety Manager course will be offered for FiSMA
members in February 2013, will include heavy practices
for V&V. Automotive and industrial automation are main
domains. Participation is free also for Testaus OSY.
FiSMA 2012
28