Product Metrics
An overview
What are metrics?
“ A quantitative measure of the degree to
which a system, component, or process
possesses a given attribute.” [IEE93]
•
Attributes of metrics - Simple,
persuasive, consistent, Independent and
effective
Software Measures
• Processes are collection of software related activities
• Products are any artifacts, deliverables or documents
that result from a process activity.
• Within each measure we identify two types of attributes
– Internal Attributes are those that can be measured in terms of
the entity itself.
– External Attributes are those that can be measured only with
respect to how the product relates to its environment.
Types of Metrics
• Metrics for Analysis
• System Size
• Functionality Delivered
• Specification Quality
• Metrics for the Design
• Architectural Metrics
• Component Level
• Interface Design
• Metrics for the Code
• Halstead Metrics
• Complexity Metrics
• Length Metrics
• Metrics for Testing
• Defect metrics
Analysis Model Metrics
• Function – Based Metrics
– Function point metric
• Variables
• Formula
• Questions
• Metrics for Specification Quality
– Specificity
– Completeness
Design Model Metrics
•
•
•
•
Architectural Design Metrics
Object – Oriented Design Metrics
Component – Level Design Metrics
User Interface Design Metrics
Architectural Metrics
• Focus on characteristics of the architecture with
emphasis on the structure and effectiveness of modules
or components within the architecture.
– Structural Complexity
• S(i) = [fout(i)] ^2
– Data complexity
• D(i) = v(i) / [fout(i) + 1]
– System complexity
• C(i) = S(i) + D(i)
Where fout is defined as the number of modules directly
invoked by module i. v(i) = number of input and output
variables in or out of i.
• Design Structure Quality Index
Object Metrics
• Size:
– Population, Volume, Length, Functionality
•
•
•
•
Complexity
Completeness
Cohesion
Coupling - an indication of “connectedness” of
a module with others, global data and
environment
• Volatility
Object-Oriented
• System size
– Number of function calls and objects.
• Class or method size
– Size of classes and methods
• Coupling and inheritance
– Interdependence of classes. Number and
complexity of relationships.
• Class or method internals
– Complexity and level of documentation
9
User Interface Metrics
• Layout
• Absolute and relative position of entities
• Frequency of use
• “Cost of transition”
• Cohesion
• Relative connection between on – screen to other on –
screen content
• Time
• Time to achieve an specific action
• Time to recover from error
• Time to achieve a use case
Source Code Metrics
• Lines of code metric
– Lines of code used to develop a program
– Set limits on coding lines
• McCabe’s cyclomatic metrics
– Path control
– Logical decision statements and operators
• Halstead Metrics
– Depend on the following measures:
•
•
•
•
Number of direct operators
Number of direct operands
Number of operator occurrences
Number of operand occurrences
– Predicts
•
•
•
•
Length
Volume
Time, effort
Number of errors
Testing Metrics
• Information for testing metrics can be gathered thru
various sources:
– Function – based metrics
– Cyclomatic complexity
• Halstead Metrics for testing
• Metrics for OO Testing
– Metrics used in design provide an indication of design
quality
– The metrics consider aspects of encapsulation and inheritance
Web Projects
Since web projects can are more dynamic
than most other types of projects it can be
harder to measure them.
• Number of static/dynamic pages
• Number of internal page links
• Number of executable functions (scripts,
applets, etc)
13
Product Metrics Future
• Application Domain Complexity Research
• Automated Product Metric Tools
– ESQUT (Evaluation of Software Quality from
User’s viewpoinT)
– WEBMETRICS
• Mathematical Product Metrics
What are process metrics?
Process metrics really encompasses
both process and project metrics.
– Process metrics are used to measure the
effectiveness of a development process
over multiple projects
– Project metrics are used to evaluate
accomplishment of a single development
effort
15
Process Metrics
Process Metrics are generally used at
the project management level.
– Strategic in nature
– Main goal is process improvement
– Metrics from individuals combined to
provide group/project metrics
– Care must be taken to keep individual
metrics private
16
Measurement Methods
Due to the variation in software, and the
lack of standards, there are many types
of metrics that can be used
• Size-Oriented
• Function-Oriented
• Object-Oriented
• Use-Case Oriented
17
Size-Oriented
• Metrics are normalized based on the size
of the software
• Focus on LOC (Lines of Code)
– Errors Per KLOC (thousand lines of code)
– Defects per KLOC
– $ per LOC
– Page of documentation per KLOC
18
Function-Oriented
• Metrics are normalized based on program
functionality
• Most commonly used metric is the FP
(function point)
– Errors per Function Point
– Defects per FP
– $ per FP
– Pages of documentation per FP
– FP per person-month
19
Object-Oriented
• number of classes reused
• number of classes with defects
20
Metrics Tools
• SynQuest
• NEXTRA
• AMI
• SPQR/20
• SOFT-ORG
• SQUID M-Base
http://irb.cs.uni-magdeburg.de/sweng/us/CAME/CAME.tlist.shtml
21
Why use Metrics?
"If you can not measure it, you can not
improve it.“ – Lord Kelvin
•
•
•
•
Characterize - Increase Understanding
Evaluate
Predict - Managing Software
Improve - Guiding Improvements
22
Why use Metrics - Benefits
Although it can be hard to generate metrics
from software, there are many benefits
• Manage complexity
• Process improvement
• Quality tracking
• Cost estimation
23
Why use Metrics - Obstacles
"Not everything that can be counted counts,
and not everything that counts can be
counted.“ - Albert Einstein
• Takes time and effort away from the
project
• Simple project may not benefit
• Projects may vary substantially
• Indicators must be interpreted
24
Why use Metrics – Bottom
Line
Provides the best option to standardize the
development process to ensure quality
and reliability
25
Establishing a Metrics
Program
Software Productivity Center, Inc. suggests an 8
step approach:
1.
2.
3.
4.
5.
6.
7.
8.
Document the Software Development Process
State the Goals
Define Metrics Required to Reach Goals
Identify Data to Collect
Define Data Collection Procedures
Assemble a Metrics Toolset
Create a Metrics Database
Define the Feedback Mechanism
26
Establishing a Metrics
Program
Establishing a program takes planning and
time to be effective.
• Need historical data from past projects to
establish indicators
• Goals have to be well defined to be
meaningful
• Those collecting measures need to be
vested in the program
27
CONCLUSIONS
• Problems in products in each phase can be detected and
removed at an early stage by using metrics.
• Metrics provide a quantitative and predictive view of
potential problems. Thus, they are a powerful tool for
product development.
• Metrics are still in a young stage. Research is needed to
set a ground for prediction of problems.