GQM – an example from
Fjellanger - Widerøe
Hans J. Lied
Tor Stålhane
IDI / NTNU
Environment
• Medium sized company making tailor-made software
systems
• Fixed price projects with little room for rework
caused by introduction of errors
• Two development projects, same project team and
process model.
– Project 1: standard with a given requirements specification
– Project 2: prototyping without requirements specification
The projects
• Project 1 had
– structure given from the start
– customer defined quality assurance requirements
– requirements specification from the customer
• Project 2
– depended on prototyping.
– started with an idea from the customer
– the requirements specification was a coproduction between customer and developers.
Methods
We used a combination of three methods:
• GQM, to decide what and how to
measure
• Pareto with a robustness analysis, to
analyse and assess collected data
• RCA: brainstorming and Ishikawa
diagrams to identify root causes
GQM method – 1
GQM abstraction sheet without the lower part
Analyse
for the
purpose of
Focus
Q1:
Q2:
What do we believe
Q1:
Q2:
with respect to as seen from
in the context of
Environment
Qa:
Qb:
Qc:
How does the environment influence
focus
Increased Qa values will reduce Q1
Low Qb values will increase Q2
Comments
Interesting info that surfaces during the GQM process but is not directly relevant for
the current GQM goal
.
GQM method – 2
The GQM abstraction sheet was used to:
• manage and structure the brainstorming
sessions
• identify problem areas for the RCA
• benefit from its strong focus on developer
participation to get a set of basic root causes
to analyse
GQM method – 3
The GQM method helps us to obtain:
• reliable data. The developers are more
interested in obtaining correct data if they felt
that they had ownership to the goal and
purpose of the data collection
• more interest in the results. One of the factors
that can kill an SPI initiative is lack of interest.
By involving the developers from day one, we
hoped to create interest and keep the
improvement program alive.
GQM - process
• Two hour workshop to introduce the method
• Establishing the GQM goal:
– Analyse the reported failures in order to
understand causes for failures seen from the
developers in the X project.
• Produced a GQM abstraction sheet, which was
converted to a data collection form
GQM - Data collection form
WPnr.:
One report pr. failure
Please specify:
Data collection form for failure causes in project 1/ project 2
Focus
Main cause:
Variation factors
Cross:
1 Incorrect use of cut-and-paste from old code
2 Incorrect reuse of old code
3 Incorrect use of language features:
3,1 -pointers
3,2 -char '0' terminating strings og var.
3,3 -other
4 Incorrect use of library components
5 Incorrect attempts to make the code general
6 Attempted reuse by introducing global variables
7 High component complexity
7,1 -module
7,2 -application
7,3 -design
8 Large, unanticipated variation in input data
9 Incomplete or bad test data
10 Too large code components
11 Missing or incomplete configuration management
12 Incorrect component integration
13 Insufficintly detailed requirements specification
14 Wrong code logic
15 Errors in hardware - software communication
Secondary cause:
Cross:
1 Time pressure during development
2 Missing competence in use of
2,1 -language
2,2 -development tools
3 Uncontrollable, external disturbances 3,1 -noise
3,2 -fire fighting old systems
3,3 -support on other products
4 Errors in development tools
5 Errors in libraries and subsystems supplied by subcontractors
6 Lack of user interaction
7 Lacking or missing motivation
7,1 -salary
7,2 -time off
8 Incomplete project model, for instance too few check points
Failure consequence:
Degree of seriousness: ( 1 is not serious, 5 is very serious )
1
2
3
Amount of person-hours spent correcting the failure:
4
5
Pareto – 1
• The 20/80 distribution, known as Pareto’s Law,
is applicable in software development
• Pareto diagrams is a way of visualising the
main problem areas
• We used Pareto analysis on our collection of
data to identify areas of improvement
Pareto – 2
Rework
Rework
16
14
25
20
15
A
10
5
Person hours
Person hours
30
12
10
A
8
6
4
2
0
0
3
4
14
7
2
1
6
13
13
15
14
1
Focus cause
Focus cause
Pareto identified the following main problem areas:
Project1 – standard project:
Project2 – prototyping :
• Incorrect use of language
features (3)
• Incorrect use of library
components (4)
• Wrong code logic (14)
• Incomplete or insufficiently
detailed req. spec. (13)
• Errors in HW-SW
communication (15)
• Wrong code logic (14)
RCA - Brainstorm
We presented the development team with the
results from the Pareto analysis and introduced
them to RCA.
We made an Ishikawa diagram, using the categories
from the data collection form as a starting point.
The strong point of a brain-storming session is that
it maximises the use of available knowledge
The RCA diagram
Incomplete
environment
description
Incomplete QA
activities
Incomplete functional
requirements
Time pressure
Lack of
knowledge
Lack of
overview
Routines not
followed
Missing
routines
Other
misunderstandings
Incomplete
understanding of
customer needs
Incomplete
requirements
Insufficient
customer contact
Too complex
description
Incomplete
functiona
nonlrequirement
s
RCA - Results
Improvement actions identified from this session:
• The need for sharing competence internally
• Checklist on Intranet
• Common routines/solutions for standard tasks
(best practice experience)
• Common dynamic document templates
• Start the planning, design and development of an
experience database
Conclusions
• Individual experience for the team members
• Measurements gave knowledge of how things
really are in the company
• Common understanding of what we learned
from this knowledge
• Explicit shared knowledge about the process
that the company can reuse