Software Quality and Metrics
Software Quality
• What is Quality ?
• Quality Characteristics
• Cost evolution : Software vs Hardware
• How to solve it ?
Software Quality
• What it is not !
– Zero errors
– A good GUI
• What it is !
– Meeting the user ‘s needs
ISO Definition of Quality
• ‘The totality of characteristics of an entity
that bear on its ability to satisfy stated and
implied needs’
Software Product
Should fullfill
has
Is related to
requirements
characteristics
(contributes to the fulfillments)
ISO 9126 Characteristics
• The objective of this standard is to provide a
framework for the evaluation of software
quality. ISO/IEC 9126 does not provide
requirements for software, but it defines a
quality model which is applicable to every
kind of software. It defines six product quality
characteristics and in an annex provides a
suggestion of quality subcharacteristics.
ISO 9126 Characteristics
•
Functionality is the set of attributes that bear on the existence of a set of functions and
their specified properties. The functions are those that satisfy stated or implied needs.
•
Reliability is the set of attributes that bear on the capability of software to maintain its
level of performance under stated conditions for a stated period of time.
•
Usability is the set of attributes that bear on the effort needed for use, and on the
individual assessment of such use, by a stated or implied set of users.
•
Efficiency is the set of attributes that bear on the relationship between the level of
performance of the software and the amount of resources used, under stated
conditions.
•
Maintainability is the set of attributes that bear on the effort needed to make specified
modifications.
•
Portability is the set of attributes that bear on the ability of software to be transferred
from one environment.
Sub Characteristics
Characteristics
Subcharacteristics
Suitability
Accurateness
Functionality
Interoperability
Compliance
Security
Maturity
Reliability
Fault tolerance
Recoverability
Usability
Definitions
Attributes of software that bear on the presence and appropriateness of a set of
functions for specified tasks.
Attributes of software that bear on the provision of right or agreed results or
effects.
Attributes of software that bear on its ability to interact with specified systems.
Attributes of software that make the software adhere to application related
standards or conventions or regulations in laws and similar prescriptions.
Attributes of software that bear on its ability to prevent unauthorized access,
whether accidental or deliberate, to programs or data.
Attributes of software that bear on the frequency of failure by faults in the
software.
Attributes of software that bear on its ability to maintain a specified level of
performance in case of software faults or of infringement of its specified
interface.
Attributes of software that bear on the capability to re-establish its level of
performance and recover the data directly affected in case of a failure and on the
time and effort needed for it.
Understandability
Attributes of software that bear on the users’ effort for recognizing the logical
concept and its applicability.
Learnability
Attributes of software that bear on the users’effort for learning its application.
Operability
Attributes of software that bear on the users’effort for operation and operation
control.
Sub Characteristics
Characteristics
Efficiency
Subcharacteristics
Time behaviour
Resource behavior
Analyzability
Maintainability
Changeability
Attributes of software that bear on response and processing times and on throughput rates
in performances its function.
Attributes of software that bear on the amount of resource used and the duration of such
use in performing its function.
Attributes of software that bear on the effort needed for diagnosis of deficiencies or causes
of failures, or for identification of parts to be modified.
Attributes of software that bear on the effort needed for modification, fault removal or for
environmental change.
Stability
Attributes of software that bear on the risk of unexpected effect of modifications.
Testability
Attributes of software that bear on the effort needed for validating the modified software.
Adaptability
Portability
Definitions
Installability
Conformance
Replaceability
Attributes of software that bear on the opportunity for its adaptation to different specified
environments without applying other actions or means than those provided for this
purpose for the software considered.
Attributes of software that bear on the effort needed to install the software in a specified
environment.
Attributes of software that make the software adhere to standards or conventions relating
to portability.
Attributes of software that bear on opportunity and effort using it in the place of specified
other software in the environment of that software.
Quality model
• The relative weight of the characteristics
depends on the type of software
– Prototype
– Critical (ex : nuclear power plant)
– Long live (ex : Telecom switch)
–…
Quality model
• What should be the ranked list of Quality
characteristics (most important first) for a
software used in a space shuttle ?
• Give them a weight.
NAVETTE HERMES
• QUALITY CHARACTERISTICS IN DECREASING
ORDER :
–
–
–
–
–
–
–
–
–
RELIABILITY
ROBUSTNESS
AVAILABILITY
CONFORMANCE
INTEGRITY
USABILITY
ERGONOMY
EFFICIENCY
TESTABILITY
CHANGEABILITY
FLEXIBILITY
PORTABILITY
REUTILISABILITY
ADAPTABILITY
Software Metrics
• The Quality needs to be quantified
Maintainability
Changeability
Testability
Cyclomatic complexity
Stability
Max nber of interleaving
Analyzability
Nbre of input/output
Metrics versus measurement
• A measure f is a function f:A->B where for
each object a belonging to A there exists a
formal object (measure’s value) f(a) belonging
to B
• A function f is called a metric iff it has the
three following properties:
– f(x,y)=0 with x=y
– f(x,y)=f(y,x) for each x and y
– f(x,z)<=f(x,y)+f(y,z) for each x, y, z
Metrics versus measurement
• A measure should satisfy 4 criteria
– Being in relation with well understood and significant
attributes
– Being supported by formal models or abstractions
that capture the attributes
– Respecting the relations and the existing order
between objects (determined by the attributes of the
models
– Being mapped to the numeric system in order to
maintain the order relationship.
• Remark: if it doesn’t respect the three first
criteria, it is a metric
Type of measurement
nominal
ordinal
interval
ratio
absolute
Type of measurement (examples)
• Nominal: number on the back of a football
player
• Ordinal: Richter scale
• Interval: f(x)=ax+b. Temperature is of interval
type
• Ratio: f(x)=ax. A length is a ratio
• Absolute: f(x)=x. Number of people working in
a company
Building of measures
• Non terminal measures
– Raw value
Ex:
=>
normative value
v(G) < 15 => v(G) = 2
15<v(G)<=30 => v(G) = 1
v(G)> 30 => v(G) = 2
• Terminal measures
Mes_1 = (mes_2 + 4*mes3)/5
• Qualitative expression:
[0;0.5 ]
=> Mes_1 = 0
NOK
]0.5;1.5]
=> Mes_1 =1
Average
]1.5;2]
=>Mes_1= 2
OK
Software Metrics
• Different levels of measurement
– Architecture
• Call Graph
– Module
• Control Graph
– Textual
Software Metrics
• Architecture
– Number of paths
– Hierarchical complexity
– Structural complexity
– Component assessibility
– Path testability
– System testability
– System entropy
Software Metrics
• Structural complexity
– Cyclomatic number
– Max number of degrees
– Max number of interleaving
• Textual
– Halstead Software Science measures
• Programming Norms
Object-Oriented Software Metrics
• Depth of inheritance tree (DIT)
– DIT is defined as the maximum number of steps
from the class node to the root of the inheritance
tree
• What it measures: Since in OO design, the application domain
is modelled as a hierarchy of classes represented as a tree, the
DIT metric is a measure of how many ancestor classes can
affect a particular class. The depth of inheritance tree metric
is an indication of the potential for reuse.
Should be between 0 and 4
Object-Oriented Software Metrics
• Number of children (NOC)
– NOC is defined as the number of immediate
descendants of a class in the hierarchy. (1..4)
• What it measures: NOC is an indicator of the
potential influence a class can have on the
design and on the system
Object-Oriented Software Metrics
• Coupling between object classes
– : This is a count of the number of other classes to
which a given class is coupled.
• What it measures: Also known as message delivery channels,
this metric counts the number of associations in a class and
attributes whose parameters are of the class type. Messages
can only be sent when an object of a class holds a reference
to another object of a class
Should be between 1 and 4
Object-Oriented Software Metrics
• Number of the associations linked to a class
– The number of associations including aggregations
is counted.
• What it measures: This metric is useful for
estimating the static relationships between
classes.
Object-Oriented Software Metrics
• Number of the elements in the transitive
closure of the subclasses of a class
• What it measures: This is potentially useful for
predicting the classes whose changes might
affect this class.
Object-Oriented Software Metrics
• Number of the superclasses of a class
– This counts the direct parents of a class
• What it measures: In a single inheritance
implementation like Java, the value of this
metric is either 0 or 1, whereas under multiple
inheritance scheme it is greater than or equal
to 0.
Object-Oriented Software Metrics
• Number of use cases in a model
• What it measures: A use case represents a
coherent unit of functionality provided by a
system, a subsystem, or a class.
Object-Oriented Software Metrics
• Number of system classes associated with a use
case
– This metric counts the number of classes whose
objects participate in the scenario of a use case.
• What it measures: NSCU is good for estimating
the impact of a requirement change onto the
system. Any changes of use cases spread to
classes and the interactions of their objects, and
vice versa
Object-Oriented Software Metrics
• Number of messages associated with a use case
– This metric counts the number of messages
comprising the scenario of a use case.
• What it measures: A use case is further refined through
its scenario. In UML, there are two scenario diagrams, i.e.
sequence diagram and communication diagram. These
two kinds of scenario diagrams are completely
isomorphic meaning one kind of diagram can be
automatically replaced with another kind without the
loss of information contained in it. This metric is useful
for tracing requirements into design-level elements.
Object-Oriented Software Metrics
• Number of actors in a model
– This metric is the number of actors in a model,
where an actor is a special class whose stereotype
is “Actor”.
• What it measures: This metric computes the
number of actors in a model.
Object-Oriented Software Metrics
• Number of actors associated with a use case
– : This metric is the number of actors that are
associated with a use case.
• What it measures: Normal system classes are
not counted for this metric because this
metric concerns the interactions between a
system and its stakeholders.
Object-Oriented Software Metrics
• Number of classes in a model
• What it measures: A class in a model is an
instance of the metaclass “class”. This metric
is comparable to the traditional LOC (lines of
code) for estimating the size of a system. This
metric can be used to compare sizes of
systems.
Object-Oriented Software Metrics
• Number of inheritance relations in a model
• What it measures: This metric counts the
number of generalisation relationships
between classes existing in a model.
Object-Oriented Software Metrics
• Number of packages in a model
• What it measures: Package is a way of
managing closely related modelling elements
together. Also by using packages, naming
conflicts can be avoided.
Object-Oriented Software Metrics
• Number of objects in a model
• What it measures: In a similar manner that a
class is an instance of the metaclass “Class”,
an object is an instance of a class.
Object-Oriented Software Metrics
• Number of associations in a model
– This metric measures the number of connections
between classes
• What it measures: An association is a
connection, or a link, between classes. This
metric is useful for estimating the scale of
relationships between classes.
Object-Oriented Software Metrics
• Number of aggregations in a model
• What it measures: An aggregation is a special
form of association that specifies a whole-part
relationship between the aggregate (whole)
and a component part
Object-Oriented Software Metrics
• Number of messages in a model
• What it measures: A message is an instance of
the metaclass “Message”. Messages are
exchanged between objects manifesting
various interactions
Object-Oriented Software Metrics
• Number of methods (NOM)
– NOM is defined as the total number of methods in a
class, including all public, private and protected
methods.
• What it measures: It has been suggested that this metric
is a useful indication of the classes that may be trying to
do too much work themselves; i.e., they provide too
much functionality. NOM cannot be viewed as an
indicator of the effort to develop a class, since a class
containing a single large method may take as long to
develop as a class containing a large number of small
methods.
Object-Oriented Software Metrics
• Number of methods added (NMA)
• What it measures : The number of operations added plays a
role in the specialization of the class and must be maintained
in a proportion which continues to justify inheritance,
otherwise known as generalization.
Too many added operations signify too big a difference with the
parent class. The inheritance would then make less sense.
The more added operations there are, the more the class must
be redeveloped, and the less the inheritance is justified.
Object-Oriented Software Metrics
• Number of Methods Inherited (NMI)
• What it measures: For a class, this metric gives the
percentage of the number of non-redefined operations
with regard to the number of operations inherited.
The percentage of operations inherited should be high. This is
the opposite of the Number of Methods Overridden (NMO)
threshold. A low percentage of inherited operations
indicates poor sub-classing.
The maximum of 100 % is ideal, but is never attained, given
the fact that we often need to adapt inherited services.
Object-Oriented Software Metrics
• Number of Methods Overridden (NMO)
• What it measures: The number of redefined operations plays a role
in the specialization of the class and must be maintained in a
proportion that continues to justify inheritance.
Too many redefined operations implies too big a difference with the
parent class and inheritance then makes less sense.
For a class, this is the count of the number of inherited operations that
are redefined by the class.
A class which inherits services must use them with a minimum of
modifications. If this is not the case, the inheritance loses all
meaning and becomes a source of confusion.
Object-Oriented Software Metrics
• Response for a class
– This is the size of the set of methods that can
potentially be executed in response to a message
received by an object.
Class A
A::m1() calls B::m1(), B::m2()
A::m2() calls A::m1(),C::m1()
A::m3()
RFC = 7
Object-Oriented Software Metrics
• Lack of cohesion in methods
– This is a measure of the number of disjoint sets
formed by the intersection of the sets of instances
variables used by methods.
Object-Oriented Software Metrics
• Lack of cohesion in methods (another one)
The LCOM value for a class C is defined as
LCOM(C) = 1 - |E(C)| / (|V(C)| x |M(C)|),
where V(C) is the set of instance variables,
M(C) is the set of instance methods, and E(C)
is the set of pairs (v,m) for each instance variable
v in V(C) that is used by method m in M(C).
Object-Oriented Software Metrics
• Number of attributes (NOA)
– NOA is defined as the total number of attributes in a
class.
• What it measures: The ratio of private and
protected attributes to total number of attributes
indicates the effort required by that class in
providing information to other classes. Private
and protected attributes are therefore viewed
merely as data to service the methods in the class
Object-Oriented Software Metrics
•
•
•
•
•
•
•
•
Number of public attributes
Number of protected attributes
Number of private attributes
Number of class attributes
Number of public methods
Number of protected methods
Number of private methods
Number of class methods
Software Analysis Process and Report
• Architectural Level
• Structural Level
• Report
Architectural Level
• General view of the call graph
– Depth, width, hierarchy, levels jump
• Singularities
– Isolated components, roots, strongly connected
components, recursivity
Call Graph examination
Symptom
diagnosis
decision
Large number of calls
No hierarchy in tasks
Splitting of the component
Large number of
recursivities
Very high risk: problem of
ending recursivity
Careful reading of code
Intensive testing of
components
Components largely used
Essential resource of
architecture => high risk
Careful reading of code
Intensive testing of
components
Isolated component
Doesn’t belong to this
version
Deleting the component
Root for a very limited
number of paths in the
graph
Auxiliary Entry point (ex:
debugging)
Deleting of this root
(doesn’t feature a normal
case)
Call Graph
Call Graph
Call Graph
Call Graph
Call Graph
Call Graph (analyzed components)
Call Graph (root = ana_appel)
Call Graph (root = display)
Control Graph examination
• General view of the control graph
– Sequences, nesting, pending nodes
• Singularities
– Isolated code, graphical similarities, intern calls
Control graph examination
Symptom
Diagnosis
decision
Destructuring back call
Bad maintainability, infinite
loop risk, bad
understandability
Back to the algorithm to
rewrite the component
Non reachable code
Many changes done, no test,
bad readability
Delete the dead code
Illogical structure, many
auxiliary exits
Poor programming. Bad
readability
Delete the exits but one =>
homogenisation
Several complex structures in
sequence
Very height intern complexity.
Hard to test
Split the component in several
ones
Several nested complex
sequences
Lack of modularity
Organize these structures in
components and create a new
level in the call graph
Graphical similarities
Non factorized code
Factorization of code –
Creation of a new component
Auxiliary entry point
Lack of modularity. Poor
maintainability
New component. Delete the
entry point
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Control Graph
Quality Report
• Quality model
– Quality characteristics (from the most to the least
important) for the project
– Sub characteristics
– Measures related to these characteristics and sub
characteristics (upper and lower bounds)
• Global quality results
– Kiviat, histograms, …
• Detailed analysis
– Architecture
– Components (accepted, rejected, …)
Example
Maintainability
Changeability
Testability
Cyclomatic complexity
Stability
Max nber of interleaving
Analyzability
Nbre of input/output
Testability
• Testability
– Cyclomatic complexity v(G)
• V(G) <= 15 => OK
• V(G) > 15 => NOK
2
0
– Max_nb_interleaving
• Max_level <=4 => OK
• Max_level > 4 => NOK
2
0
– Nb_input/output
• Nb_IO =2 => OK
• Nb_IO >2 => KO
2
0
Testability=(4*V(G)+4*Max_level+2*Max_IO)/10
V(G)
Max_Level
Max_IO
OK
OK
OK
OK
OK
OK
Diagnosis
2 = Accepted
1.6 = restructure
OK
OK
1.2 = to split
0.8 = to split
OK
OK
OK
1.2 = to split
0.8 = to split
OK
0.4 = to rewrite
0 = to rewrite
Testability
V(G)
Upper bound
15
Lower bound
1
1
Max_level
4
2
V(G) = 12
Max_Level =3
Max_IO =2
2
Max_IO
Testability
V(G)
15
1
1
Max_level
4
2
V(G) = 17
Max_Level =5
Max_IO =2
2
Max_IO
Maintainability
V(g)
fcom
Testability
Nb_Ins
Changeability
Max_level
Max_IO
V(g)
Nb_Ins
Nb_Ins
V(g)
Max_level
Stability
Analysability
Max_IO
fcom
Max_IO
Example in OO Programming
• Quality Model
• Measures and bounds
Quality Model
• Reliability
– WMPC
– CBO
– FD
– RFC
• Efficiency
– CBO
– AC
– LCOM
• Maintainability
– ASC
– DIT
– AC
– RFC
- AR
- ComR
• Portability
– NMCI - DIT
– AR
- EH
– ER
- PR
Class Level Measures
• ComR (Comment Rate)
• ER (Encapsulation Rate =nber of private
variables/total number of variables)
• NCI (Nber of Classes)
• NMCI (Average Nber of Methods per class)
• WMPC (Weighted Methods per Class)
• LCOM (Lack of COhesion in Methods)
• DIT (Depth of inheritance Tree)
Inheritance Measures
• EH (Entropy of inheritance Tree)
• AR (Abstraction Rate = Nber of abstract
classes/Total nber of classes)
• PR (Polymorphism Rate = (Nber of virtual
methods /total number of methods)
Object interactions Complexity measures
• AC (Association Complexity = R-NCI +2P with R
nber of interactions between objects and P
nber of disconnect parts in the graph)
• FD (Flow Density = sum of FI *p(start node
exists) with FI, number of invoked methods
between 2 objects and start node, object
invoking methods)
Object interactions Complexity measures
• CBO (Coupling between Objects)
• RFC (Response for a class)
Measures
LCOM
Lack of COhesion in Methods
<3
CBO
Coupling between objects
<10
EH
Entropy of inheritance tree
<2
AR
Abstraction Rate
>0.2
PR
Polymorphism Rate
>0.2
AC
Association Complexity
<15
WMPC
Weighted Methods Per Class
<15
RFC
Response for a Class
<10
FD
Flow Density
<100
ASC
Average Size per Class
<24
DIT
Depth of inheritance tree
<5
ComR
Comment Rate
>0.2
NCI
ER
Number of classes
Encapsulation Rate
<20
>0.9