Towards the Integration of Quality
Attributes into a Software Product
Line Cost Model
Andy Nolan1, Silvia Abrahão2 Paul Clements3,
John McGregor4, Sholom Cohen3
1Rolls-Royce,
UK
Politécnica de Valencia, Spain
3Software Engineering Institute, USA
4Clemson University, USA
2Universidad
©2011 Rolls-Royce plc
The information in this document is the property of Rolls-Royce plc and may not be copied or communicated to a third party, or used for any purpose
other than that for which it is supplied without the express written consent of Rolls-Royce plc.
This information is given in good faith based upon the latest information available to Rolls-Royce plc, no warranty or representation is given
concerning such information, which must not be taken as establishing any contractual or other commitment binding upon Rolls-Royce plc or any of
its subsidiary or associated companies.
2
Rolls-Royce
Rolls-Royce
3
Rolls-Royce
The need for Software Product Lines
4
 The Control Systems department is
responsible for the Engine Electronic




Rolls-Royce
Controllers (EECs) for a range of small
and large gas turbine engines for the
aerospace industry.
The software is developed to DO-178B
Level-A standards
The company has been developing
high integrity software for over 20
years and has extensive data on its
processes and productivity.
We have the largest order book in
history, new engine development
places greater demand on the software
team (shorter time scales and lower
costs)
Since 2008, Rolls-Royce has
developed a PL which has potential for
both the software and hardware
aspects of engine design.
The need for Cost Models
5
 Since 2004, we have invested in cost
models to predict software cost
based on COCOMO II.
 The cost models have led to an
improvement in stability and
productivity, with an average of
around 11% productivity gain
 This benefit has occurred because a
project has a better understanding of
the Quality Attributes that affect cost
and risk.
 When the SPL initiative was
launched in 2008, the development
of a reliable and comprehensive
estimation model was seen as critical
to making the right decisions to
develop core assets that would bring
the greatest business value.
Rolls-Royce
6
Recap of
SPLC 2009
Rolls-Royce
7
Overview of the Model
T ra d itio n a l
PL
C om m on
L ib ra ry
S te p 4 P ro je c t
R e u s e P la n
S te p 1
D e te rm in e
A s s e t S ize
S te p 2
D e te rm in e
A s s e t V a lu e
S te p 3 A s s e t
to P ro je c t
M a p p in g
P P ro c e s s
M odel
S te p 5 P rL
Asset
D e ve lo p m e n t
Rolls-Royce
S te p 1 0 C o s t
B e n e fit
S te p 6 P L
D e p lo ym e n t
P la n
S te p 7 P L
P ro je c t C o s ts
S te p 8 P L
O rg a n is a tio n
C o s ts
S te p 9 P L
E n viro n m e n t
8
Recap of
SPLC 2010
Rolls-Royce
9
Cost = Size * Environment
Environment
 Team capability
 Team experience
Size isn't
 Team complexity and location
everything!
 Process capability
 Management capability
 Tool & IT capability
Size
 Lines of Code
 Complexity
 Project complexity
 Integration complexity
 Development approach
 Development standard
 Development domain
Rolls-Royce
10
10
cost
COCOMO II
Size & Complexity
COCOMO
Quality Attribute
Size, Complexity, Testability
Maintainability
Maturity, Reusability,
Portability, Variability......
SPL
Rolls-Royce
cost
The SPLC 2010 paper proposed that the
existing cost model had to be extended
with other Quality Attributes to get a better
understanding of costs & benefits
The evolution of cost models
 [2004] First generation of the Rolls-Royce cost model was
based on COCOMO II

The software product is represented as a single size measure (SLOC).
Limited use of the architecture or characteristics of the product being
developed
 [2008] Second generation of the model allowed us to
estimate the cost to develop and deploy individual core
assets

The model estimates the cost-benefit of each core asset based on
size, complexity, volatility, and difficulty to develop and deploy
 [2011] To understand the relevant Quality Attributes of the
core assets and the relationship between them

Rolls-Royce
To understand and integrate Quality Attributes into the cost model
11
12
Selecting
Quality
Attributes
SPLC 2011
Rolls-Royce
Objectives
 Look at quality attributes and their impact on the costbenefit of Software Product Lines
Every factor (question) a cost model considers is another
opportunity to: Refine your understanding, make trade
decisions and optimise the project
 Incorporate quality attributes into the Rolls-Royce Cost
Model to analyze:
The cost/benefit of having core assets with certain qualities
The impact on quality attributes of design decisions
 Use the cost model to derive design principles for the
core asset team
For example how much variability should there be and when ius
there too much or too little?
Rolls-Royce
13
Maturity
 Maturity of a core asset has been
defined here as the “degree to which an
asset is free from further modification”.
 A low maturity asset is likely to be
exposed to changes and depending on
when they manifest, this can lead to
high levels of effort to fix defects
 Maturity becomes a sensitive issue for
SPLs especially if products are using
low maturity assets.
 Maturity can be estimated from process
exposure and project exposure
Rolls-Royce
14
Testability
 It is an important quality attribute for
any type of product, and in particular
for safety-critical products.
 A non-SPL safety-critical product
invests around 52% of its total
development effort on some form of
verification and validation.
 In a SPL at Rolls-Royce, data shows
that RELATIVELY up to 72% of a
product’s overall effort will be spent in
some form of V&V
 Testability can be estimated from the
number of test cases (decision points)
required to exercise the complete
asset
Rolls-Royce
15
Variability
 The selection of a specific variation mechanism for a core asset can
have an impact on the final product development & deployment
cost.
 The cost of variability in a core asset is calculated by multiplying the
cost of deploying the asset (in a specific process) by the cost of
using the different variation mechanisms
Future Work
Need estimation tool
based on both the
number AND type of
variations
Rolls-Royce
16
Reusability
 The reusability of core
assets largely determines
the success of product line
projects.
 Higher reusability of core
assets indicates the
potential for a higher
return-on-investment.
This can be measured
retrospectively.
Future Work
Need to develop a
pre-process measure
of reusability
Rolls-Royce
17
Maintainability
 Maintainability of core assets is of
great importance. It is also
important to understand the
degree of maintainability that a
core asset should have to be
cost-effective to the business.
 Measured by the effort to
maintain an asset (normalised by
size).
Future Work
Need to develop a
pre-process measure
of maintainability
Rolls-Royce
18
19
Trade off
analysis
SPLC 2011
Rolls-Royce
Quality Attribute Trade-offs
Firstly, we created a hypotheses
diagram to show the relationship
between the Quality Attributes
Rolls-Royce
20
Quality Attribute Trade-offs
Then we measured the relationship
between the attributes (strength and
direction)
Rolls-Royce
21
Quality Attribute Trade-offs
Now we can derrive information from the
model
?
is it better to
have 2 simpler assets or 1 larger asset,
to have selectable assets rather than
configurable, constrain the requirements
or build in variability?
Rolls-Royce
22
23
SPLC 2012?
Rolls-Royce
24
Next Steps
 The Rolls-Royce
estimation model will
need to be updated to
take into account the
other quality attributes
selected (i.e., maturity,
variability and
testability)
 Other quality attributes
will be evaluated and
the model developed
further
Volatility
-L
Variability
-H
-H
Testability
-M
L
H
M
-H
Maturity
M
H
H
L
H
Reusability
H
H
M
H
M
Cost
H
Benefit
H
H
Model v.2.
Without derived
relationships
M
H
M
M
Customer Sat.
M
H
M
H
ROI
Rolls-Royce
Maintainability
M
Project Sat.
M
L
Business Sat.
25
1st International Workshop on Quantitative Methods in
Software Product Line Engineering (QMSPLE 2011)
QMSPLE 2011
Rolls-Royce