Quantitative Measurement of Software Quality in Terms of
«Reliability yet Flexibility»
Abbas Rasoolzadegan, Ahmad Abdollahzadeh Barforoush
Intelligent Systems Laboratory (http://ce.aut.ac.ir/islab)
Information Technology and Computer Engineering Faculty
Amirkabir University of Technology (Tehran Polytechnic)
{rasoolzadegan, ahmad}@aut.ac.ir
Abstract—Reliability and flexibility are two interrelated aspects of software quality. A specific development
process, including particular methods, mechanisms, and tools should be used to meet the ever-increasing need
for reliable yet flexible software. Such a process should ensure software development with the desired reliability
and flexibility. Quantifying software quality is increasingly becoming necessary. Software quality directly
depends on the quality of the corresponding development process. In this paper, «reliability yet flexibility» is
regarded and defined as a new quality requirement. A systematic mechanism is then proposed to quantify the
«reliability yet flexibility» of software by measuring the «reliability yet flexibility» of the corresponding
development process.
Keywords—Reliability, flexibility, quantitative measurement, formal methods, design patterns.
1.
INTRODUCTION
Reliability is defined as the probability of failure-free software operation for a specified period of time in a specified
environment [1]. Contrary to hardware, software does not wear out or deteriorate, i.e., its reliability does not
decrease with time due to physical depreciation. Moreover, software generally enjoys reliability growth during
testing and operation since software faults can be detected and removed when software failures occur. However,
Software may experience reliability decrease due to abrupt changes of its operational usage or incorrect
modifications to the software. Unlike hardware failures which result mainly from physical faults, software failures
are caused by design faults, which are harder to detect and correct. In the context of software reliability, the term
design refers to all software development steps from the requirements to implementation [1]. Design faults result
mostly from human error in the development process or maintenance. Formal modeling methods (FMMs) are
broadly defined as notations with accurate and unambiguous semantics. They are supported by various tools. FMMs,
along with formal refinement and formal verification mathematically prove the consistency and completeness of
activities during software development. Such proofs help detect the existing faults before they turn into failures. As
a result, highly reliable software development is facilitated [2-3], [40].
Flexibility can be defined as the ability of a system to respond to potential internal or external changes affecting
its value delivery, in a timely and cost-effective manner. Thus, flexibility for an engineering system is the ease with
which the system can respond to uncertainty in a manner to sustain or increase its value delivery. Uncertainty is a
key element in the definition of flexibility. Uncertainty can create both risks and opportunities in a system, and it is
with the existence of uncertainty that flexibility becomes valuable. An artifact is hardly flexible in absolute terms [67]. Instead, it may be flexible towards a specific class of changes and inflexible towards another one. Predicting the
class of changes is the key to understanding software flexibility. Moreover, an artifact Α is more flexible than
another artifact Β towards a particular evolution step if the number of changes required for Α is less than those
required for Β. ‘Evolution step’ is regarded as the unit of evolution with relation to a particular class of changes in
design or implementation [8-9].
The results of literature review show that the need for both reliability and flexibility is increasingly becoming
important among the various classes of software applications. These results are summarized as follows:
The increasingly necessity of flexibility: Rapid technological developments pervade every aspect of daily life,
having a direct effect on the software we use. Every element of the software’s operational environment is in a state
of constant flux: Frequent changes in the hardware, operating system, cooperating software, and client’s
expectations are motivated by performance improvements, bug-fixes, security breaches, and attempts to assemble
synergistically ever more sophisticated software systems [3], [27]. Classic and contemporary literature in software
design recognizes the central role of flexibility in software design and implementation. Structured design, modular
design, object-oriented design, software architecture, design patterns, and component-based software engineering,
among others, seek to maximize flexibility. Textbooks about software design emphasize the flexibility of particular
choices, thereby implying the superiority of the design policy they advocate. But despite the progress made since the
earliest days of software engineering, from the ‘software crisis’ through ‘software’s chronic crisis’, evolution
(formerly ‘maintenance’) of industrial software systems has remained unpredictable and notoriously expensive,
often exceeding the cost of the development phase. Flexibility has therefore become a central concern in software
design and in many related aspects in software engineering research [11].
The increasingly necessity of reliability: Software permeates our daily life. There is probably no other humanmade material which is more omnipresent than software in our modern society. It has become a crucial part of many
aspects of society. Size and complexity of software systems have grown dramatically during the past few decades,
and the trend will certainly continue in the future. Because of this ever-increasing dependency, software failures can
lead to serious, even fatal, consequences in safety-critical systems as well as in normal business [1]. The demand for
complex software-hardware systems has increased more rapidly than the ability to develop them with highly desired
reliability [2], [4]. When the requirements for and dependencies on such systems increase, the possibility of crises
from software failures also increases. The impact of these failures ranges from inconvenience (e.g., malfunctions of
home appliances) to economic damage (e.g., interruptions of banking systems) to loss of life (e.g., failures of flight
systems or medical software) [25], [28-30].
The increasingly necessity of both flexibility and reliability: We demonstrate this issue from two aspects:
1. As mentioned above, there are several evidences that explicitly confirm the increasingly important role of
either reliability or flexibility. These evidences together implicitly imply that the need for both reliability
and flexibility is increasingly becoming important.
2. There are some references that explicitly confirm the necessity of both of them [3], [31-34]. There are two
main reasons for such requirements: 1) software applications increasingly pervade every critical aspects of
daily life and 2) each element of the software’s operational environment is in a state of constant flux. These
evidences were added to the text in an abridged way.
Developing reliable yet flexible software is a challenging issue and needs a specific model-driven development
process with a particular emphasis on requirements engineering [3-4]. Although modeling methods enjoy a lot of
advantages, the use of just one of them, in many cases, may not guarantee the development of reliable and flexible
software. In this respect, a specific software development process has been recently introduced [5], which is referred
to as SDPFlex&Rel in the rest of this paper. In SDPFlex&Rel, formal (Object-Z specifications) and visual (UML class
diagrams) models are transformed into each other using a set of bidirectional formal functions. In this process,
Object-Z, as a dominant formal specification language, is used to formally specify, refine, and verify requirements –
which, in turn, prevent and remove probable faults. Fault prevention and fault removal are two main approaches to
the development of reliable software systems. UML diagrams facilitate the interactions among stakeholders who are
not familiar enough with the complex mathematical concepts of formal modeling methods. Applying design patterns
to visual models improves the flexibility of software. The transformation of formal and visual models into each
other through the iterative and evolutionary process, proposed in [5], helps develop the software applications that
need to be highly reliable yet flexible. The workflow of SDPFlex&Rel is illustrated in Fig. 1. The iterative and
evolutionary process illustrated in Fig. 1 continues until a final product with a desired quality (in terms of
«reliability yet flexibility») is achieved.
As illustrated in Fig. 1, SDPFlex&Rel consists of the following phases:
 Reliability Assurance Phase (RAP) which supports formal specification and refinement in Object-Z.
 Visualization Phase (VP) which transforms Object-Z models into UML ones.
 Flexibility Assurance Phase (FAP) which revises UML models from the viewpoints of design patterns and
polymorphism.
 Formalization Phase (FP) which transforms UML models into Object-Z ones.
The ability of formal refinement and formal verification to ensure the reliability of software as well as the effect
of design patterns on improving the flexibility of software has already been confirmed by the related literature and
empirical studies [1-4], [11]. SDPFlex&Rel provides the ground for the use of formal refinement, formal verification,
and design patterns. This logically leads to the development of reliable yet flexible software.
Quantifying software quality is increasingly becoming necessary [5-7], [11], [39], [41]. However, the quantitative
measurement of software quality is a challenging issue [24], [35-38]. In order to quantitatively measure the quality
of software, the quality of the corresponding development process can be measured because of the direct
proportional relationship between them [21], [22-24].
Figure1. The Workflow of SDPFlex&Rel Using UML Activity Diagram
This paper addresses an open problem according to the aforementioned investigations. The problem is to quantify
software quality in terms of «reliability yet flexibility». To do so, firstly, «reliability yet flexibility» is regarded and
defined as a new quality requirement according to the interrelationship of reliability and flexibility. Then, a
systematic mechanism is proposed to quantify the quality of SDPFlex&Rel as a representative of reliable yet flexible
software development processes – which, in turn, determines the «reliability yet flexibility» of those software
developed by SDPFlex&Rel.
The rest of this paper is organized as follows: Section two presents the definitions of a set of key terms that are
necessary to communicate with the scope and contributions of this work. These terms are failure, fault, error,
evolution step, evolution cost metric, and evolution complexity. A brief description of the main approaches to the
achievement of reliability as well as the major classes of reliability assessment is also presented in section two. In
section three, by focusing on the interrelationship between the reliability and the flexibility of software, software
quality is quantified in terms of «reliability yet flexibility» through quantitatively measuring the quality of the
corresponding software development process. Finally, section four discusses the conclusions.
2.
KEY CONCEPTS AND DEFINITIONS
2.1. Failure
A failure occurs when the user perceives that the system ceases to deliver the expected service [1]. The user may
choose to identify several severity levels of failures, such as: catastrophic, major, and minor, depending on their
impacts on the system service. The definitions of these severity levels vary from system to system. Failure behavior
directly depends on the environment and the number of faults present in the software during execution. Let 𝑇 denote
a random variable representing the system failure time. Failure density 𝑓(𝑇) corresponds to the probability
distribution function of 𝑇. Failure probability 𝐹(𝑡) is the probability that the failure time is less or equal to time 𝑡:
𝑡
𝐹(𝑡) = 𝑃𝑟𝑜𝑏(𝑇 ≤ 𝑡) = ∫ 𝑓(𝑢). 𝑑𝑢
0
Reliability 𝑅(𝑡) is the probability that the system delivers the expected services in the time interval:
∞
𝑅(𝑡) = 1 − 𝐹(𝑡) = 𝑃𝑟𝑜𝑏(𝑇 > 𝑡) = ∫ 𝑓(𝑢). 𝑑𝑢
𝑡
2.2. Fault
A fault is uncovered when either a failure of the software occurs or an internal error (e.g., an incorrect state) is
detected within the software [25]. The cause of the failure or the internal error is said to be a fault. It is also referred
to as a bug. Software faults arise mostly from design issues. The sources of software faults include:
 Incorrect requirements, even though the implementation may match them.
 Implementation (software design and coding) deviating from (correct) requirements.
 Uncontrolled or unexpected changes in operational usage or incorrect modifications.
In summary, a software failure is an incorrect result with respect to the specification or an unexpected software
behavior perceived by the user at the boundary of the software system, while a software fault is the identified or
hypothesized cause of the software failure. When the distinction between fault and failure is not critical, defect can
be used as a generic term to refer to either a fault (cause) or a failure (effect).
2.3. Error
The term error has two different meanings [1]:
1. A discrepancy between a computed, observed, or measured value or condition and the true, specified, or
theoretically correct value or condition. Errors occur when some part of the software produces an undesired
state. Examples include exceptional conditions raised by the activation of existing software faults and an
incorrect system status due to an unexpected external interference. This term is especially useful in faulttolerant computing to describe an intermediate stage in-between faults and failures.
2. A human action that results in software containing a fault. Examples include omission or misinterpretation of
user requirements in a software specification, and incorrect translation or omission of a requirement in a
software design. However, this is not a preferred usage, and the term mistake is used instead to avoid the
confusion.
2.4. Reliability achievement
The development of a reliable software system calls for the combined utilization of a set of methods and techniques
which can be classified into [3]:
 Fault prevention: how to prevent fault occurrence or introduction. The interactive refinement of the user’s
system requirement, requirements engineering (RE), the use of sound design methods, and the encouragement
of writing clear codes are the general approaches to prevent faults in the software. Formal methods develop
and refine requirement specifications correctly using languages and tools with sound mathematical bases in
order to achieve the following goals: 1) executable specifications for systematic and precise evaluation, 2)
proof mechanisms for step-by-step verification using incremental refinement, and 3) every intermediate
artifact is a subject to mathematical verification for correctness and appropriateness.
 Fault removal: how to reduce the presence (number and seriousness) of faults. Fault removal uses techniques
such as testing, inspection, verification, and validation to track and remove faults in software. Formal
inspection is a practical fault removal scheme which is widely implemented in industry. Formal inspection is a
rigorous process focused on finding faults, correcting faults, and verifying the corrections.
 Fault tolerance: how to ensure a service capable of fulfilling the system’s function in the presence of faults.
Software fault tolerance is achieved by design diversity in which multiple versions of software are developed.
These multiple versions, which are functionally equivalent yet independent, are applied in the system to
provide ultimate tolerance to software design faults.
 Fault forecasting: how to estimate the present number, future incidence, and consequences of faults. Fault
forecasting involves formulation of the fault/failure relationship, an understanding of the operational
environment, the establishment of reliability models, the collection of failure data, the application of reliability
models by tools, the selection of appropriate models, and the analysis and interpretation of results.
2.5. Reliability assessment
The three major classes of software reliability assessment are:
 Black box reliability analysis: Estimation of the software reliability based on failure observations from testing
or operation. These approaches are called black box approaches because internal details of the software are
not considered.
 Software metric based reliability analysis: Reliability evaluation based on the static analysis of the software
(e.g., lines of code, number of statements, complexity) or its development process and conditions (e.g.,
developer experience, applied testing methods).
 Architecture-based reliability analysis: Evaluation of the software system reliability from software component
reliabilities and the system architecture (the way the system is composed out of the components). These
approaches are sometimes called component-based reliability estimation (CBRE), or grey or white box
approaches.
2.6. Reliability measurement
Measurement of software reliability includes two types of activities: reliability estimation and reliability prediction
[25]. Estimation determines current software reliability by applying statistical inference techniques to failure data
obtained during system test or during system operation. This is a measure regarding the achieved reliability from the
past until the current point. Its main purpose is to assess the current reliability and determine whether a reliability
model is a good fit in retrospect. Prediction determines future software reliability based upon available software
metrics and measures. Depending on the software development stage, prediction involves different techniques:
1. When failure data are available (e.g., software is in system test or operation stage), the estimation techniques
can be used to parameterize and verify software reliability models, which can perform future reliability
prediction.
2. When failure data are not available (e.g., software is in design or implementation stages), the metrics obtained
from the software development process and the characteristics of the resulting product can be used to predict
reliability of the software.
Data collected during the test phase is often used to estimate the number of software faults remaining in a system
which in turn is often used as input for reliability prediction. This estimation can either be done by looking at the
numbers (and the rate) of faults found during testing or just by looking at the effort that was spent on testing. The
underlying assumption when looking at testing effort is “more testing leads to higher reliability”.
2.7. Evolution step
As illustrated in Fig. 2, software evolution can be described as the process during which changes occur in an old
problem, which entail changes in the corresponding design/implementation. To distinguish between changes in
problems and changes in the corresponding designs/implementations, we refer to the former as shifts and to the
latter as adjustments, jointly represented as an evolution step [11].
Figure2. An evolution step
Let us represent the set of problems as ℙ and the set of designs/implementations as 𝔻𝕀. An evolution step can be
represented as a mapping of the combination of the old problem 𝑝𝑜𝑙𝑑 ∈ ℙ, the shifted problem 𝑝shifted ∈ ℙ, and the
old design/implementation 𝑑𝑖𝑜𝑙𝑑 ∈ 𝔻𝕀 into the adjusted design/implementation 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 ∈ 𝔻𝕀. This mapping can
thus be represented as a mathematical function ℰ, called the evolution function. This function maps each tuple <
𝑝𝑜𝑙𝑑 , 𝑝shifted , 𝑑𝑖𝑜𝑙𝑑 > to 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 , such that:
ℰ: ℙ × ℙ × 𝔻𝕀 → 𝔻𝕀, ℰ(𝑝𝑜𝑙𝑑 , 𝑝shifted , 𝑑𝑖𝑜𝑙𝑑 ) = 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑
where the old design/implementation 𝑑𝑖𝑜𝑙𝑑 realizes 𝑝𝑜𝑙𝑑 , and 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 realizes 𝑝shifted . Therefore, an
evolution step can be formulated as:
ℰ =≪ 𝑝𝑜𝑙𝑑 , 𝑝shifted , 𝑑𝑖𝑜𝑙𝑑 >, ℰ(𝑝𝑜𝑙𝑑 , 𝑝shifted , 𝑑𝑖𝑜𝑙𝑑 ) >
2.8. Evolution cost metric
𝜇
We can measure flexibility in terms of the cost of the evolution process. ‘Evolution cost metric’ ( 𝐶𝑀𝑜𝑑𝑢𝑙𝑒𝑠 )
measures the cost of executing an evolution step ℰ =≪ 𝑝𝑜𝑙𝑑 , 𝑝shifted , 𝑑𝑖𝑜𝑙𝑑 >, 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 > in terms of the
software complexity 𝜇(𝑚) of each module 𝑚 affected by the adjustments [11-12]:
𝜇
𝐶𝑀𝑜𝑑𝑢𝑙𝑒𝑠 (ℰ) ≜ ∑
𝑚∈∆𝑀𝑜𝑑𝑢𝑙𝑒𝑠( 𝑑𝑖𝑜𝑙𝑑 , 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 )
𝜇(𝑚)
(1)
where 𝜇 can be any software complexity metric such as LoC (lines of code) or CC (Cyclomatic Complexity), and
∆𝑀𝑜𝑑𝑢𝑙𝑒𝑠(𝑑𝑖𝑜𝑙𝑑 , 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 ) designates the symmetric set-difference between the set of modules in 𝑑𝑖𝑜𝑙𝑑 and the
set of modules in 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 , namely:
(𝑀𝑜𝑑𝑢𝑙𝑒𝑠(𝑑𝑖𝑜𝑙𝑑 )\𝑀𝑜𝑑𝑢𝑙𝑒𝑠( 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 )) ∪ (𝑀𝑜𝑑𝑢𝑙𝑒𝑠( 𝑑𝑖𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 )\𝑀𝑜𝑑𝑢𝑙𝑒𝑠( 𝑑𝑖𝑜𝑙𝑑 ))
2.9. Evolution complexity
To quantify flexibility, a notion called ‘evolution complexity’ can be used. The complexity of an evolution step
measures how inflexible the design/implementation is being evolved towards a particular class of changes. The
fewer changes required, the more flexible it is. The evolution complexity of a design/implementation 𝑑𝑖 towards an
𝜇
evolution step (ℰ) is formulated as 𝑂( 𝐶𝑀𝑜𝑑𝑢𝑙𝑒𝑠 (ℰ)). If the evolution complexity of 𝑑𝑖 towards ℰ is fixed and
𝜇
independent of its size (𝑂( 𝐶𝑀𝑜𝑑𝑢𝑙𝑒𝑠 (ℰ)) = 1), 𝑑𝑖 is flexible towards ℰ, but if the evolution complexity of 𝑑𝑖 directly
or indirectly grows as a function of the size of 𝑑𝑖, it is inflexible towards ℰ. It is worth mentioning that evolution
complexity does not measure the actual cost of the evolution processes required, but how it grows.
3. QUANTITATIVE MEASUREMENT OF «RELIABILITY YET FLEXIBILITY»
As previously mentioned in section 1, software reliability is mainly threatened with design faults. Although software
reliability does not decrease with time due to physical depreciation, it decreases due to abrupt changes of its
operational usage or incorrect modifications to the software. Therefore, the reliability of a software system becomes
time-independent if the system is made robust against possible kinds of change. In other words, the reliability of a
software system is time-independent if the system is flexible towards potential classes of change. Indeed, the
reliability of a flexible software system is fairly fixed and depends mainly on its design faults. According to the
mentioned interrelationship of the reliability and the flexibility of software systems, a new quality requirement
entitled «reliability yet flexibility» is defined as: “the probability of failure-free software operation in a specified
environment”. As expected, this definition does not include the expression “for a specified period of time” in
comparison to the definition of software reliability. Such expectation arises from the impact of flexibility on
reliability in the new term «reliability yet flexibility» abbreviated as 𝑅𝑦 𝐹. Indeed, 𝑅𝑦 𝐹 is equivalent to reliability in
the presence of flexibility but in the absence of time, i.e., 𝑅 instead of 𝑅(𝑡).
Developing reliable yet flexible software needs a specific development process. As previously mentioned, a
proper alternative to quantifying various aspects of software quality (such as 𝑅𝑦 𝐹) is the quantitative measurement
of the quality of the corresponding development process because there is a direct relation between the quality of
software and the quality of the corresponding development process. In the rest of this section, the quality of
SDPFlex&Rel (as a representative of reliable yet flexible development processes) is quantified in terms of 𝑅𝑦 𝐹. Fig. 3
illustrates the process of the quality measurement of SDPFlex&Rel in terms of 𝑅𝑦 𝐹. As illustrated in Fig. 3, this process
includes three activities elaborated in subsections 3.1, 3.2, and 3.3, respectively.
Activity1:
Measuring the
flexibility of
SDPFlex&Rel
Activity2: Evaluating the
transformation mechanism of
SDPFlex&Rel in terms of soundness
and completeness
Activity3:
Measuring
the 𝑅𝑦 𝐹 of
SDPFlex&Rel
Figure3. The process of the quality measurement of SDPFlex&Rel in terms of 𝑅𝑦 𝐹
3.1. Measuring the flexibility of SDPFlex&Rel
As illustrated in Fig. 3, before measuring 𝑅𝑦 𝐹, we should quantitatively measure the flexibility improvement
promised by SDPFlex&Rel. In the phase FAP of SDPFlex&Rel, the flexibility of the software being developed improves
using design patterns. Each design pattern allows some aspect of the system structure to vary independently of other
aspects, thereby making a system more robust to a particular kind of change. The flexibility of the software
developed using SDPFlex&Rel is directly proportional to the flexibility of those design patterns used in the phase FAP
during the different iterations of the development process. Therefore, to quantify software flexibility, we can
quantitatively measure the flexibility of the design patterns used during software development process [3].
A non-trivial case study, called the multi-lift system, has been taken as a test bed to evaluate the feasibility of
SDPFlex&Rel [5]. This system includes parallel, distributed, embedded, and real-time software. A detailed report of this
empirical study has been presented in [10]. Such a system needs high reliability and flexibility. For instance, the
dispatching strategy should be continuously updated for each lift according to some criteria such as manager policies
and traffic modes, which dynamically change. These variable factors increase the necessity of designing a flexible
controller having the potential to change the control strategy dynamically. In the process of developing the multi-lift
system using SDPFlex&Rel, the Observer, Strategy, and Mediator design patterns [26] have been used during the phase
FAP to improve the system flexibility. In this section, we investigate the usability of the evolution cost metric for
corroborating informal claims on the flexibility of these design patterns. The applications, benefits and the structure
of the Observer, Strategy, and Mediator design patterns, along with their usage during the development of the multilift system has been elaborately discussed in [5]. We can quantify the flexibility of each design pattern towards
specified evolution steps by calculating the corresponding evolution complexity.
The Observer pattern defines a one-to-many dependency between one object named subject and its dependent
objects, referred to as observers. All observers are notified and updated automatically once the state of the
corresponding subject changes. According to relation (1) and the structure of the Observer design pattern [5],
1
CClasses
is used to measure the flexibility of the Observer pattern. In other words, we assume that the cost of adding,
removing, or changing each modular unit (𝑚) is equal to 1 (𝜇(𝑚) = 1). Moreover, ‘class’ is considered as the
modular unit. Thus, the evolution cost metric is estimated by calculating the number of the classes that are added,
removed, or adjusted as a result of the evolution. The results of this analysis are summarized in Table 1.
Table1. The complexity of evolving the Observer pattern vs.
traditional design towards shifts in observers and subjects
Evolution step Change/Add/Remo Change/Add/Remove
ve observer
Design policy
subject
Observer pattern
Traditional design
(‘anti-pattern’)
O(1)
O(1)
O(|Subjects|)
O(|Observers|)
The results show that the complexity of evolving the Observer pattern or each Observer-based design towards
shifts in observers (ℰ1 ) and subjects (ℰ2 ) is fixed (O(1)). Therefore, we can conclude that the Observer pattern as
well as each design based on this pattern is flexible towards both ℰ1 and ℰ2 . As shown in Table 1, the evolution
complexity of a traditional design towards ℰ1 and ℰ2 is proportional to the number of subjects (|Subjects|) and
observers (|Observers|), respectively. As a result, it is inflexible towards both ℰ1 and ℰ2 .
The Strategy pattern configures a class named context with one of several behaviors , referred to as strategies. To
measure the flexibility of the Strategy design pattern, we assume that the cost of adding, removing, or changing a
modular unit 𝑚 is proportional to the number of those statements of 𝑚 that are added, removed, or adjusted as a
result of the evolution (𝜇(𝑚) = 𝐿𝑜𝑆 (Lines of Statements)). Moreover, ‘class’ is considered as the modular unit.
Thus, the evolution cost metric is estimated by calculating the number of the statements that are added, removed, or
𝐿𝑜𝑆
adjusted as a result of the evolution ( 𝐶𝐶𝑙𝑎𝑠𝑠𝑒𝑠
). The results of this analysis are summarized in Table 2 according to
the structure of the Strategy design pattern [5].
Table2. The complexity of evolving the Strategy pattern vs.
‘Switch’ or ‘Multiple conditional’ statements towards shifts in strategies
Evolution step
Design policy
Strategy pattern
‘Switch’ or ‘multiple conditional’ statements
Add/Remove
Strategy
𝑂(1)
𝑂(|𝑆𝑡𝑟𝑎𝑡𝑒𝑔𝑖𝑒𝑠|)
The results show that the complexity of evolving the Strategy pattern as well as each Strategy-based design
towards shifts in strategies (ℰ) is fixed (O(1)). Therefore, we can conclude that the Strategy pattern or each design
based on this pattern is flexible towards ℰ. As shown in Table 2, the evolution complexity of a traditional design
towards ℰ is proportional to the number of strategies (|𝑆𝑡𝑟𝑎𝑡𝑒𝑔𝑖𝑒𝑠|), so it is inflexible towards ℰ.
The Mediator pattern defines an object named mediator. This object encapsulates how a set of objects, referred to
1
as colleagues, interact. We use CClasses
to measure the flexibility of the Mediator design pattern. The results of this
analysis are summarized in Table 3 according to the structure of the Mediator pattern [5]. The results show that the
complexity of evolving the Mediator pattern or each Mediator-based design towards shifts in behavior (ℰ1 ),
colleagues (ℰ2 ), and relationships (ℰ3 ) is fixed (O(1)). Therefore, we can conclude that the Mediator design pattern
or each design based on it is flexible towards ℰ1 , ℰ2 , and ℰ3 . As shown in Table 3, the evolution complexity of a
traditional design towards these three evolution steps (ℰ1 , ℰ2 , and ℰ3 ) is directly proportional to the number of
colleagues (𝜃 × |Colleagues|), so it is inflexible towards ℰ1 , ℰ2 , and ℰ3 .
Table3. The complexity of evolving the Mediator pattern vs.
traditional design towards shifts in behavior, colleagues, and relationships
Design policy
Evolution step
Mediator
pattern
Traditional design (‘anti-pattern’)
Change behavior
𝑂(1)
𝑂(𝜃 × |Colleagues|); 0 < 𝜃 ≤ 1
Change colleague
𝑂(1)
𝑂(𝜃 × |Colleagues|); 0 < 𝜃 ≤ 1
Extend relationships
between colleagues
𝑂(1)
𝑂(𝜃 × |Colleagues|); 0 < 𝜃 ≤ 1
The results of the aforementioned analyses show that the revision of the initial class diagram of the multi-lift
system using the Observer, Strategy, and Mediator patterns during the phase FAP of SDPFlex&Rel makes the system
flexible towards the mentioned changes. The flexibility of the software developed by SDPFlex&Rel (such as the multilift system) is directly proportional to the flexibility of those design patterns used during its development. The
flexibility of each design pattern is quantified using the evolution cost metric through calculating the complexity of
evolution steps. In other words, the flexibility improvement claimed by design patterns can be corroborated by the
evolution cost metric. There is a direct relationship among the value of the evolution cost, the evolution complexity,
and the flexibility of a design towards a particular evolution step.
3.2. Evaluating the transformation mechanism of SDPFlex&Rel
As illustrated in Fig. 3, the second activity that should be done before measuring 𝑅𝑦 𝐹 of SDPFlex&Rel is the evaluation
of the transformation mechanism of SDPFlex&Rel. The model transformation mechanism used in SDPFlex&Rel includes
several formal functions that transform modeling constructs of the UML class diagram and Object-Z into each other
at the meta-level. Indeed, each of the proposed functions transforms a meta-element of the source language (UML or
Object-Z) into a corresponding meta-construct of the target language (Object-Z or UML). The meta-models of the
UML class diagram and Object-Z have been presented as Object-Z specifications in Appendices 2 and 3 of [5].
Moreover, the transformation functions are presented in Appendix 1. In this appendix, the lines of each function are
numbered to be used during its soundness evaluation in Appendix 2. It is worth mentioning that the formal
transformation functions, along with the corresponding comments, are presented in Table 2 of [5].
The two main criteria for evaluating a model transformation mechanism are: correctness (soundness) and
completeness [13-20]. A model transformation mechanism is correct if it transforms a source model into a
semantically equivalent target model. A model transformation mechanism is complete if each modeling construct in
the source language is transformed into a corresponding modeling element in the target language. As the
transformation mechanism of SDPFlex&Rel transforms modeling elements of the UML class diagram and Object-Z
into each other at the meta-level, using the formal functions, we can conclude that:

Instead of verifying the correctness of individual target models, we put stress on the correctness of the
transformation functions. In other words, the transformation mechanism of SDPFlex&Rel is correct, if each of
the transformation functions is correct. Each of these functions takes some (one or more) meta-elements of
the source language as input and returns some (one or more) meta-constructs of the target language as
output. Therefore, each of these functions is correct if it transforms some meta-elements of the source
language into semantically equivalent meta-models of the target language. Appendix 2 presents the
correctness evaluation process of the transformation functions.

The proposed mechanism is complete if there is a function for transforming each of the source metaelements into a corresponding target meta-construct. The transformation functions of SDPFlex&Rel cover all
shared modeling elements of the UML class diagram and Object-Z. These elements are primary attributes,
derived attributes, constants, operations, visibility, user-defined types, multiplicities, and initialization which
constitute class, inheritance, generic inheritance, unidirectional association, bidirectional association,
aggregation, composition, and dependency which are different kinds of relationships, and association class
and polymorphism. Tables 4 and 5 verify the completeness of the proposed mechanism.
Table4. Model transformation functions from UML to Object-Z
Meta Constructs of UML
Meta Constructs of Object-Z
Main Transformation Function
UML Meta-Type
Object-Z Meta-Type
convUMLTypeToOZ (Appendix 1, Part 1)
UML Meta-Attribute
Object-Z Meta-Attribute
transUMLAttributeToOZ (Appendix 1, Part 1)
UML Meta-Class
Object-Z Meta-Class
transUMLClassToOZ (Appendix 1, Part 1)
UML Meta-Generalization
Corresponding Object-Z Meta-Specification
transUMLGenToOZ (Appendix 1, Part 3)
UML Meta-Dependency
Corresponding Object-Z Meta-Specification
transUMLDepToOZ (Appendix 1, Part 9)
UML Meta-Association
Corresponding Object-Z Meta-Specification
transUMLAssocToOZ (Appendix 1- Part 5)
UML Meta-Association Class
Corresponding Object-Z Meta-Specification
transAssocClassToOZ (Appendix 1, Part 7)
Table5.Model transformation functions from Object-Z to UML
Meta Constructs of Object-Z
Meta Constructs of UML
Main Transformation Function
Object-Z Meta-Type
UML Meta-Type
convOZTypeToUML (Appendix 1, Part 2)
Object-Z Meta-Attribute
UML Meta-Attribute
transOZAttributeToUML (Appendix 1, Part 2)
Object-Z Meta-Class
UML Meta-Class
transOZClassToUML (Appendix 1, Part 2)
Generalization as Object-Z Meta-Specification
UML Meta-Generalization
elicitUMLGenFromOZ (Appendix 1, Part 4)
Dependency as Object-Z Meta-Specification
UML Meta-Dependency
elicitUMLDepFromOZ (Appendix 1, Part 10)
Association as Object-Z Meta-Specification
UML Meta-Association
elicitUMLAssocFromOZ (Appendix 1, Part 6)
Association Class as Object-Z Meta-Specification UML Meta-Association Class
elicitUMLAssocClassFromOZ (Appendix 1, Part
8)
3.3. Measuring the 𝑹𝒚 𝑭 of SDPFlex&Rel
After investigating the ability of SDPFlex&Rel to ensure the desired flexibility (subsection 3.1), 𝑅𝑦 𝐹 is measured
according to the fact that 𝑅𝑦 𝐹 (reliability in the presence of flexibility) is equivalent to 𝑅 (reliability in the absence
of time) instead of 𝑅(𝑡) (reliability). Fig. 4 illustrates the details of an iteration of SDPFlex&Rel.
Figure4. A schematic view of an iteration 𝑖 of SDPFlex&Rel
According to the details of an iteration of SDPFlex&Rel, the total 𝑅𝑦 𝐹 of SDPFlex&Rel is calculated as:
𝑘
𝑛𝑖
𝑘
𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 = 𝑅SDPFlex&𝑅𝑒𝑙 = ∏ ∏ 𝑅𝑅𝐴𝑃 (𝑖, 𝑗) ∗ ∏
𝑖=1 𝑗=1
𝑘
𝑛𝑖
𝑚𝑖
𝑅𝑅𝐴𝑃 (𝑖, 𝑗) = 𝑅𝑦 𝐹𝑅𝐴𝑃 (𝑖, 𝑗)
𝑅𝑉𝑃 (𝑖)= 𝑅𝑦 𝐹𝑉𝑃 (𝑖)
𝑅𝐹𝐴𝑃 (𝑖, 𝑗) = 𝑅𝑦 𝐹𝐹𝐴𝑃 (𝑖, 𝑗)
𝑅𝐹𝑃 (𝑖) = 𝑅𝑦 𝐹𝐹𝑃 (𝑖)
𝑅SDPFlex&𝑅𝑒𝑙
𝑖=1
𝑘
𝑚𝑖
𝑘
𝑅𝑉𝑃 (𝑖) ∗ ∏ ∏ 𝑅𝐹𝐴𝑃 (𝑖, 𝑗) ∗ ∏
𝑖=1
𝑖=1 𝑗=1
(2)
𝑅𝐹𝑃 (𝑖)
Number of iterations in the development process proposed by SDPFlex&Rel.
Number of formal refinement steps during RAP in the iteration 𝑖 of SDPFlex&Rel.
Number of revision steps during FAP in the iteration 𝑖 of SDPFlex&Rel.
Time-independent reliability of the 𝑗𝑡ℎ formal refinement step in RAP during the 𝑖𝑡ℎ iteration.
Time-independent reliability of the 𝑖𝑡ℎ formal transformation from Object-Z into UML in VP.
Time-independent reliability of the 𝑗𝑡ℎ revision step in FAP during the 𝑖𝑡ℎ iteration of SDPFlex&Rel.
Time-independent reliability of the 𝑖𝑡ℎ formal transformation from UML into Object-Z in FP.
Total time-independent reliability of SDPFlex&Rel which is equivalent to total 𝑅𝑦 𝐹 of SDPFlex&Rel.
In the current version of SDPFlex&Rel, all activities of the phases RAP, FP, and VP are performed formally with
sound mathematical bases. The formal transformation functions used in the phases FP and VP make it possible to
transform UML class diagrams and Object-Z specifications into each other without any fault during the phases FP
and VP (subsection 3.2). Moreover, formal refinement, along with formal verification guarantees the correctness of
the activities performed during the phase RAP. As previously mentioned, formalism ensures the absence of faults
and consequently provides the maximum amount of reliability. Therefore, the reliability of each of those activities
performed during the phases RAP, FP, and VP equals maximum (∀𝑖, 𝑗, 𝑅𝑅𝐴𝑃 (𝑖, 𝑗) = 𝑅𝑉𝑃 (𝑖) = 𝑅𝐹𝑃 (𝑖, 𝑗) = 𝑚𝑎𝑥 ≈
1). However, in the current version of SDPFlex&Rel, during the phase FAP, designers apply the required design
patterns to the class diagram of the system being developed without any formal systematic control. This may cause
the syntactic or the semantic structure of the class diagram to become inconsistent. Therefore, the reliability of every
activity in the phase FAP does not equal maximum (∀ 𝑖, 𝑗, 𝑅𝐹𝐴𝑃 (𝑖, 𝑗) ≠ 𝑚𝑎𝑥). However as illustrated in Fig. 1, the
required checks are carried out on the output of the phase FAP to guarantee the lack of inconsistency between the
input and the output of the phase FAP. Therefore, the reliability of the output of the phase FAP equals maximum.
Relation (2) is then simplified as:
𝑚𝑎𝑥≈1
𝑛
𝑚
𝑚
∀𝑖, 𝑗, 0 < 𝑅𝐹𝐴𝑃 (𝑖, 𝑗) ≤ 𝑚𝑎𝑥
𝑖
𝑖
𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 = ∏𝑘𝑖=1 ∏𝑗=1
𝑚𝑎𝑥 ∗ ∏𝑘𝑖=1 𝑚𝑎𝑥 ∗ ∏𝑘𝑖=1 ∏𝑗=1
𝑅𝐹𝐴𝑃 (𝑖, 𝑗) ∗ ∏𝑘𝑖=1 𝑚𝑎𝑥 ⇒
𝑖
𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 = ∏𝑘𝑖=1 ∏𝑗=1
𝑅𝐹𝐴𝑃 (𝑖, 𝑗)
(3)
A formal mechanism can be proposed to make the class diagram of the software being developed be formally
revised when a design pattern is applied to it in FAP. As a result, applying design patterns to the class diagram of
software not only improves the flexibility of the software but also preserves the syntactic and the semantic structure
of the class diagram – which, in turn, leads to consistency preservation during the revision process of the class
diagram in FAP. To do so, a set of formal functions can be defined using model refactoring based on graph
transformation at the meta-level of the UML class diagram to make it possible to add/remove/change a modeling
element to/from/in a class diagram without making its syntax and semantics become inconsistent. Designers are then
allowed to change a class diagram just using the defined functions in order to revise it based on a design pattern.
Therefore, the reliability of every activity in the phase FAP equals maximum (∀ 𝑖, 𝑗, 𝑅𝐹𝐴𝑃 (𝑖, 𝑗) = 𝑚𝑎𝑥). Relation
(2) is then simplified further as:
𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 = (𝑚𝑎𝑥) ∗ (𝑚𝑎𝑥) ∗ (𝑚𝑎𝑥) ∗ (𝑚𝑎𝑥) ≈ 1
(4)
As previously mentioned, the «reliability yet flexibility» of software developed using a development process is
directly proportional to the reliability of the development process. Therefore, the «reliability yet flexibility» of the
software developed using SDPFlex&Rel is obtained according to relation (3), but in the future, by proposing a formal
mechanism for supporting the revision process of the phase FAP, the «reliability yet flexibility» of the software
increases to 𝑀𝐴𝑋 according to relation (4).
Generally, a software development process includes several (𝑛) activities. Based on the assumption that these
activities do not enjoy a sound mathematical (formal) basis, the reliability of each of them does not equal maximum
(∀ 𝑖, 𝑅(𝑖) ≠ 𝑚𝑎𝑥). If the process meets the required flexibility, «reliability yet flexibility» of the process can be
formulated as:
𝑅𝑦 𝐹𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠 = ∏𝑛𝑖=1 𝑅(𝑖)
∀𝑖, 0 < 𝑅(𝑖) ≤ 𝑚𝑎𝑥
(5)
In order to simply compare 𝑅𝑦 𝐹𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠 with 𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 , relation (3) is reformulated as follows:
𝑘
𝑘′
𝑚𝑖
𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 = ∏ ∏ 𝑅𝐹𝐴𝑃 (𝑖, 𝑗) = ∏ 𝑅𝐹𝐴𝑃 (𝑖 ′ , 𝑗 ′ )
𝑖=1 𝑗=1
(6)
𝑝=1
such that:
∀ 𝑖 ′ , 𝑗 ′ , 0 < 𝑅𝐹𝐴𝑃 (𝑖 ′ , 𝑗 ′ ) ≤ 𝑚𝑎𝑥,
𝑘 ′ = ∑𝑘𝑖=1 𝑚𝑖 ,
𝑛′
∀𝑝, 0 ≤ 𝑛′ < 𝑘 ∙ ∑
𝑗=1
𝑛′ +1
𝑚𝑗 ≤ 𝑝 ≤ ∑
𝑗=1
𝑛′
𝑚𝑗 ⇒ 𝑖 ′ = 𝑛′ + 1 ⋀ 𝑗 ′ = 𝑖 − ∑
𝑗=1
𝑚𝑗
The following assumptions are made according to relations (5) and (6):
1) In 𝑅𝑦 𝐹𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠 , ∀𝑖, 0 < 𝑅(𝑖) ≤ 𝑚𝑎𝑥,
2) In 𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 , ∀ 𝑝, 𝑖 ′ , 𝑗 ′ , 0 < 𝑅𝐹𝐴𝑃 (𝑖 ′ , 𝑗 ′ ) ≤ 𝑚𝑎𝑥,
3) ∀ 𝑖 ′ , 𝑗 ′ , 𝑖, 𝑅𝐹𝐴𝑃 (𝑖 ′ , 𝑗 ′ ) ≮ 𝑅(𝑖), because: 1) the input materials of FAP are correct and fault-free, 2) During
FAP, the input materials are just revised using design patterns and polymorphism with low possibility of fault
occurrence, and 3) the lack of semantic inconsistency between the input and the output of the phase FAP is
guaranteed by the existing formal analysis techniques (such as initialization theorem and precondition
investigation) and the various formal verification mechanisms that support Object-Z.
4) 𝑘 ′ ≪ 𝑛,
With respect to these assumptions, we can conclude that:
𝑘′
𝑛
∏ 𝑅𝐹𝐴𝑃 (𝑖 ′ , 𝑗 ′ ) ≫ ∏ 𝑅(𝑖) ⇒ 𝑅𝑦 𝐹SDPFlex&𝑅𝑒𝑙 ≫ 𝑅𝑦 𝐹𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠
𝑝=1
𝑖=1
The above-mentioned analysis shows that the «reliability yet flexibility» of SDPFlex&Rel is greater than the
«reliability yet flexibility» of a generic flexible software development process. The main conclusion is that the more
widespread the use of formalism including formal specifications, refinement, and verification throughout a software
development process, the more reliable the software development process will be. Therefore, supposing that some
(𝑚) of the activities of the software development process 𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠 are performed formally, the «reliability
yet flexibility» of 𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠, previously formulated as relation (5), is reformulated as relation (7):
∀𝑖,1≤𝑖≤𝑚, 𝑅(𝑖)=𝑚𝑎𝑥
𝑛−𝑚
𝑅𝑦 𝐹𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠 = ∏𝑛𝑖=1 𝑅(𝑖) = ∏𝑚
𝑖=1 𝑅(𝑖) ∗ ∏𝑖=1 𝑅(𝑖) ⇒
𝑅𝑦 𝐹𝐺𝑒𝑛𝑒𝑟𝑖𝑐𝑃𝑟𝑜𝑐𝑒𝑠𝑠 ≈ ∏𝑛−𝑚
𝑖=1 𝑅(𝑖)
∀𝑖, 0 < 𝑅(𝑖) ≤ 𝑚𝑎𝑥
(7)
With respect to the fact that the «reliability yet flexibility» of a software product is directly proportional to the
«reliability yet flexibility» of the development process used, the more «reliable yet flexible» the software
development process, the more «reliable yet flexible» the software product. According to the aforementioned
analyses, the «reliability yet flexibility» of software developed using SDPFlex&Rel is greater than the «reliability yet
flexibility» of software developed using a generic flexible software development process.
4.
CONCLUSIONS
In this paper, «reliability yet flexibility» abbreviated as 𝑅𝑦 𝐹 is regarded and defined as a new quality requirement
according to ever-increasing need for meeting and measuring both reliability and flexibility as two interrelated
aspects of software quality. A systematic mechanism is then proposed to quantify the 𝑅𝑦 𝐹 of software by measuring
the 𝑅𝑦 𝐹 of the corresponding development process, because there is a direct relation between the quality of software
and the quality of the corresponding process. In this respect, we quantitatively measure the «reliability yet
flexibility» improvements promised by SDPFlex&Rel, as a representative of reliable yet flexible software development
processes. A process including three activities is defined to quantify the 𝑅𝑦 𝐹 of SDPFlex&Rel. As the first activity, the
flexibility of SDPFlex&Rel is measured. Software flexibility is directly proportional to the flexibility of those design
patterns used during its development process. Therefore, to quantify the flexibility of software, the flexibility of each
of the design patterns used during the development process of the software is quantitatively measured by calculating
the complexity of evolution steps through the evolution cost metric. As the second activity, a systematic mechanism
is proposed to investigate the soundness and the completeness of the transformation functions of SDPFlex&Rel. This
activity is a required step toward measuring the 𝑅𝑦 𝐹 of SDPFlex&Rel. Finally, as the third activity, 𝑅𝑦 𝐹 of SDPFlex&Rel
is measured according to the fact that 𝑅𝑦 𝐹 (reliability in the presence of flexibility) is equivalent to 𝑅 (reliability in
the absence of time) instead of 𝑅(𝑡) (reliability). It is worth mentioning that the proposed mechanism quantitatively
measures the 𝑅𝑦 𝐹 of software developed by SDPFlex&Rel. Indeed, the limitation of the proposed approach is that it
cannot measure the 𝑅𝑦 𝐹 of software built using existing components or libraries.
References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
H. Pham, System Software Reliability, Springer, 1st ed., 2006.
A. Pandit, “A Framework-based Approach for Reliability & Quality Assurance of Safety-Critical Software,” Int. Journal
on Computer Science and Eng., vol. 2 (9), pp. 2874-2879, 2010.
H. B. Christensen, Flexible, Reliable Software: Using Patterns and Agile Development, Chapman and Hall/CRC; 1st ed.,
2010.
D. Bjørner, Software Engineering III: Domains, Requirements, and Software Design, Springer, 2006.
A. Rasoolzadegan, A. Abdollahzadeh, “Reliable yet Flexible Software through Formal Model Transformation (Rule
Definition), Journal of Knowledge and Information Systems (KAIS), vol. 40 (1), 2014.
J. Niu, “A Measurement Method of Software Flexibility Based on BP Network,” in Proc. Int. Workshop on Intelligent
Systems and Applications (ISA), pp. 1-4, 2009.
S. Peng, “User-Oriented Measurement of Software Flexibility,” In Proc. World Congress on Computer Science and
Information Engineering, pp. 629-633, 2009.
R. Martinho, “A Two-Step Approach for Modeling Flexibility in Software Processes,” in Proc.
23rd IEEE/ACM Int. Conf. on Automated Softw. Eng., Italy, pp. 427-430, 2008.
H. Oliver, O. Philipp, and B. Udo, “Improving Software Flexibility for Business Process Changes,” Business & Information
Systems Eng., vol. 2 (1), pp. 3-13, 2010.
A. Rasoolzadegan and A. Abdollahzadeh, Specifying a Parallel, Distributed, Real-Time, and Embedded System: Multi-Lift
System Case Study, Tech. Rep., Information Technology and Computer Eng. Faculty, Amirkabir Univ. Technology,
Tehran, Iran, 2011.
H. Eden, T. Mens, “Measuring Software Flexibility,” IEE Software, vol. 153 (3), pp. 113-126, London, UK: The Institution
of Engineering and Technology, 2006.
G. H. Z. Li, “Research on Flexibility Metrics in Software Architecture Level,” Computer Science, vol. 35 (4), pp. 259-264,
2008.
D. Varró, G. Varró, A. Pataricza, “Designing the automatic transformation of visual languages,” Science of Computer
Programming, vol. 44 (2), pp. 205-227, 2002.
D. Varró, “Towards Formal Verification of Model Transformations,” PhD Student Workshop of Formal Methods for Open
Object-Based Distributed Systems (FMOODS), The Netherlands, 2002.
B. Hoffmann, M. Minas, “Defining Models – Meta Models versus Graph Grammars,” in Proc. the Ninth International
Workshop on Graph Transformation and Visual Modeling Techniques (GT-VMT), vol. 29, 2010.
B. Schätz, “Verification of Model Transformations,” in Proc. the Eighth International Workshop on Graph Transformation
and Visual Modeling Techniques (GT-VMT), vol. 18, 2009.
S. Andova, M. Brand, and L. Engelen, “Reusable and Correct Endogenous Model Transformations,” Theory and Practice
of Model Transformations, Lecture Notes in Computer Science, vol. 7307, pp. 72-88, 2012.
F. Orejas, E. Guerra, J. Lara, and H. Ehrig, “Correctness, Completeness and Termination of Pattern-Based Model-to-Model
Transformation,” in Proc. the 3rd international conference on Algebra and coalgebra in computer science (CALCO'09),
pp. 383-397, 2009.
L. Baresi, V. Rafe, A. T. Rahmani, and P. Spoletini, “An Efficient Solution for Model Checking Graph Transformation
Systems,” in Proc. 3rd Workshop on Graph Transformation for Concurrency and Verification, vol. 213, pp. 3-21, 2008.
T. Mens, P. V. Gorp, “A Taxonomy of Model Transformation,” Electronic Notes in Theoretical Computer Science, vol.
152, pp. 125-142, 2006.
R. Van Solingen, E. Berghout, “Causal Relationships between Improvements in Software Development Processes and Final
Software Product Quality,” The Electronic Journal Information Systems Evaluation, vol. 11, Issue 1, pp. 1- 10, 2008.
R. S. Pressman, Software Engineering-A Practitioner’s Approach-Required, 7th ed. McGraw Hill, 2009.
I. Sommerville, Software Engineering, 9th ed. Addison Wesley, 2011.
H. Jung, S. Kim, and C. Chung, “Measuring software product quality: A survey of ISO/IEC 9126,” IEEE Software, vol.
21(5), pp. 10–13, 2004.
[25] J. D. Musa, Software reliability engineering: more reliable software, faster and cheaper. Tata McGraw-Hill Education,
2004.
[26] E. Gamma, R. Johnson, and J. Vlissides, Design Pattern: Elements of Reusable Object-Oriented Software, 5th ed. AddisonWesley.
[27] T. Mens and A. H. Eden, “On the evolution complexity of design patterns,” Electronic Lecture Notes in Computer Science,
vol. 127 (3), pp. 147–163, Amsterdam, Elsevier, 2004.
[28] M. R. Lyu and X. Cai, Fault-Tolerant Software, Encyclopedia on Computer Science and Engineering, Wiley, 2007.
[29] M. R. Lyu, Handbook of Software Reliability Engineering. McGraw-Hill publishing, 1995.
[30] M. Jazayeri, Web Application Development: The Coming Trends, Future of Software Engineering, IEEE-CS Press, 2007.
[31] A. McKenzie, M. Thiessen, and D. Putnam-Pite, Building Software Flexibility and Reliability into Automated Systems,
Half-Day Technical Seminar, Owens Design Company, Fremont, CA, 2012.
[32] M. Rodríguez, L. V. Ivanchenko, and V. Berzins, “Reliability and Flexibility Properties of Models for Design and Runtime Analysis,” Monterey Workshop, Paris, France, 2006.
[33] A. H. Bagge, “Constructs & Concepts: Language Design for Flexibility and Reliability,” PhD thesis, Research School in
Information and Communication Technology, Department of Informatics, University of Bergen, Norway, 2009.
[34] P. A. Urbano, T. Wagner, P. Gohner, U. Katzke, and B. Vogel-Heuser, “Introducing Reliability and Real-time Features in
Flexible Agent-oriented Automation Systems,” in Proc. 2nd IEEE International Conference on Industrial Informatics
(INDIN '04), pp. 89-94, 2004.
[35] D. Spinellis, Code Quality: The Open Source Perspective, Addison Wesley, Boston, MA, 2006.
[36] H. Kan, Metrics and Models in Software Quality Engineering, Addison-Wesley, Boston, MA, 2nd ed., 2002.
[37] R. L. Glass, Building Quality Software, Prentice Hall, Upper Saddle River, NJ, 1992.
[38] C. W. Chang, D. Horng, and H. Lin , “A Quantity Model for Controlling and Measuring Software Quality Based on the
Expert Decision-Making Algorithm,” Intelligent Information Management, vol. 1, pp. 81-88, 2009.
[39] M. Gleirscher, D. Golubitskiy, M. Irlbeck, and S. Wagner, “Introduction of static quality analysis in small- and mediumsized software enterprises: experiences from technology transfer,” Software Quality Journal, 2013, DOI: 10.1007/s11219013-9217-z.
[40] M. Adham Isa, D. N. A. Jawawi, and M. Z. M. Zaki, “Model-driven estimation approach for system reliability using
integrated tasks and resources,” Software Quality Journal, 2013, DOI: 10.1007/s11219-013-9209-z.
[41] G. Zhang, H. Ye, and Y. Lin, “Quality attribute modeling and quality aware product configuration in software product
lines,” Software Quality Journal, 2013, DOI: 10.1007/s11219-013-9197-z.
Appendix 1: Bidirectional formal mechanism for transforming Object-Z specifications into UML class diagrams and vice versa
Part 1. Required functions for transforming UML meta-class into Object-Z meta-class
Function 1
Function 2
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Line 15
Functions 3 & 4
Function 5
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Line 15
Line 16
Line 17
Line 18
Line 19
Line 20
Line 21
Line 22
Line 23
Line 24
Line 25
Line 26
Line 27
Line 28
Line 28
Line 30
Part 2. Required functions for transforming Object-Z meta-class into UML meta-class
Function 1
Function 2
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Function 3
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Line 15
Line 16
Line 17
Line 18
Line 19
Line 20
Line 21
Line 22
Line 23
Line 24
Line 25
Part 3. Required function for transforming UML meta-generalization into Object-Z meta-specification
Line 1
Line 2
Line 3
Part 4. Required function for eliciting UML meta-generalization from Object-Z meta-specification
Line 1
Line 2
Line 3
Line 4
Part 5. Required functions for transforming UML meta-association into Object-Z meta-specification
Function 1
Function 2
Function 3
Function 4
Function 5
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Line 15
Line 16
Line 17
Line 18
Line 19
Line 20
Line 21
Line 22
Line 23
Line 24
Line 25
Line 26
Line 27
Line 28
Line 28
Line 30
Line 31
Line 32
Part 6. Required functions for eliciting UML meta-association from Object-Z meta-specification
Function 1
Function 2
Function 3
Function 4
Function 5
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Line 15
Line 16
Part 7. Required function for transforming UML meta-association class into Object-Z meta-specification
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Line 15
Line 16
Line 17
Line 18
Line 19
Line 20
Line 21
Line 22
Line 23
Line 24
Part 8. Required function for eliciting UML meta-association class from Object-Z meta-specification
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
Line 12
Line 13
Line 14
Line 15
Line 16
Part 9. Required function for transforming UML meta-dependency into Object-Z meta-specification
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Part 10. Required function for eliciting UML meta-dependency from Object-Z meta-specification
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Appendix 2: Correctness evaluation of the transformation mechanism of SDPFlex&Rel
Correspondence between UML meta-attribute and Object-Z meta-attribute
UML meta-attribute and related modeling elements
3
4
2
1
12
13
6
5
7
11
10
8
9
OZ meta-attribute and related modeling elements
3
4
2
1
12
13
6
5
14
7
8
15
11
9
10
Line 7
Line 1
Line 4
Line 3
Lines 12-15
Line 2
Line 6
Line 5
Lines 8- 9
Lines 10-11
Transforming UML meta-attribute into Object-Z meta-attribute using the function transUMLAttributeToOZ (Appendix 1, Part 1)
Line 10
Line 1
Line 3
Line 4
Lines 11-14
Line 2
Lines 8-9
Lines 5-7
Transforming Object-Z meta-attribute into UML meta-attribute using the function transOZAttributeToUML (Appendix 1, Part
2)
Correspondence between UML meta-class and Object-Z meta-class
2
UML meta-class and related modeling
elements
1
4
3
5
7
6
8
9
12
11
10
13
15
14
16
18
17
11
Object-Z meta-class and related modeling
elements
2
5
10
12
9
18
13
8
3
19
14
15
7
16
1
17
6
4
Line 1
Lines 15-19
Line 13
Line 14
Lines 11-19
Lines 7-30
Line 9
Line 10
Lines 20-27
Line 2
Line 3
Lines 4-6
Lines 28-30
Transforming UML meta-class into Object-Z meta-class using the function transUMLClassToOZ (Appendix 1, Part 1)
Line 1
Lines 17-21
Line 15
Line 16
Lines 13-21
Line 10
Line 11
Line 12
Lines 8-25
Line 2
Line 3
Lines 4-7
Lines 22-25
Transforming Object-Z meta-class into UML meta-class using the function transOZClassToUML (Appendix 1, Part 2)
Line 2
Line 1
Line 3
Transforming UML meta-generalization into corresponding Object-Z meta-specification using transUMLGenToOZ (Appendix 1, Part 3)
Line 3
Line 4
Lines 1-2
Eliciting UML meta-generalization from corresponding Object-Z meta-specification using elicitUMLGenFromOZ (Appendix 1, Part 4)
Line 1
Lines 21-29
Line 5
Line 15
Line 17
Line 16
Line 20
Line 30
Lines 18-32
Lines 3-17
Line 32
Line 31
Lines 6-14
Line 2
Transforming UML meta-association into corresponding Object-Z meta-specification using transUMLAssocToOZ (Appendix 1- Part 5)
Line 5
Line 9
Line 10
Line 13
Line 14
Lines 1-4
Lines 11-16
Line 7
Line 8
Line 15
Line 16
Line 6
Eliciting UML meta-association from corresponding Object-Z meta-specification using elicitUMLAssocFromOZ (Appendix 1, Part 6)
Line 1
Line 2
Lines 5-6
Line 3
Lines 6-9
Transforming UML meta-dependency into corresponding Object-Z meta-specification using transUMLDepToOZ (Appendix 1, Part 9)
Line 2
Line 12
Line 14
Line 15
Line 16
Line 17
Line 14
Line 9
Line 7
Line 6
Line 9
Line 1
Line 5
Line 5
Line 11
Line 10
Line 20
Line 18
Line 20
Line 21
Line 22
Line 23
Line 3
Transforming UML meta-association class into corresponding Object-Z meta-specification using transAssocClassToOZ (Appendix 1, Part 7)
Line 5
Line 6
Lines 1-4
Line 7
Lines 8-9
Eliciting UML meta-dependency from corresponding Object-Z meta-specification using the function elicitUMLDepFromOZ (Appendix 1, Part 10)
Line 7
Line 14
Line 9
Lines 1-5
Line 6
Line 9
Line 10
Line 11
Line 13
Line 12
Lines 15-16
Line 8
Eliciting UML meta-association class from corresponding Object-Z meta-specification using elicitUMLAssocClassFromOZ (Appendix 1, Part 8)