Method Engineering
Assignment 1 –
Method Description
Rick Barneveld - 3344754
This is a description of the paper: On the Perception of
Software QualityRequirements during the Project
Lifecycle, by Neil A Ernst and John Mylopoulos. It is the
first assignment for the Method Engineering course at
Utrecht University.
2/15/2013
Table of Contents
Introduction............................................................................................................................................. 2
Authors ................................................................................................................................................ 2
Method ................................................................................................................................................ 2
Example ................................................................................................................................................... 3
Deliverable: List of Signifiers ................................................................................................................... 3
Related literature .................................................................................................................................... 4
References ............................................................................................................................................... 5
Introduction
The title of the paper is: On the Perception of Software Quality Requirements during the Project
Lifecycle. In this paper, the authors try to find the answer to two questions, namely:
•
•
Is there an increase of discussion about quality requirements as the project evolves?
Whether different software projects have similar levels of interest about quality
requirements.
The paper is based on a Ph. D dissertation by one of the authors (N. A. Ernst, 2012). From this
dissertation several conference proceedings are distilled.
Authors
The authors of the paper are Neil A Ernst and John Mylopoulos. Both the authors have an academic
background. They have written several publications together, and this is not their first one. Ernst
currently is doing post-doc research at the University of British Columbia in software engineering.
However, the cooperation between the authors began at the University of Toronto, where Ernst did
his Ph. D study, which was about Software Evolution. Mylopoulos is currently still involved at the
University of Toronto, where he is leading research projects. Mylopoulos actually supervised Ernst
during his Ph.D. research.
Method
The research is based on the use of quality attributes as defined in ISO/IEC 9126, so called F-R-U-E-MP (International Standards Organization, 2001). These stand for: Functionality, Reliability, Usability,
Efficiency, Maintainability and Portability. The paper uses eight different parts from an open source
software project for its research, namely Gnome. Gnome is extensively discussed by German (2003).
The steps undertaken to determine how often a certain quality attribute is discussed by developers
and users of a product during the project lifecycles is described in the beginning of the paper. First, a
number of sources where the discussions should be captured from were determined. These were the
developers email conversation of a project, the bug list of the project, etc. All the messages were
handled separately (so email threads were divided into distinct messages). Then, a catalogue of
words associated with each of the six quality attributes was made, to be used in an analysis of all the
messages. Two catalogues were made, “WN” and “ext”. One based on Wordnet, and another list that
extended the first one. WordNet is an online lexical database designed for use under program control
(Miller, 1995). It contains synonyms, meronyms and word relations. The “ext list” extended the “WN
list” with several terms specific to software engineering. After the lists were constructed, the
messages were analyzed with the word catalogues to see how often a certain quality attribute was
mentioned over the project lifetime. The last step of the process was to check how precise the
analysis was, and eliminate errors.
Example
Say a company wants to know if the perspectives of several development teams that work on a
project differ in terms of quality attributes. To do this, the company can use the method described in
the introduction. They can use either an external consultant, or use one of their own employees to
do the research. In this case, we will assume consultant is hired to do the job. The consultant first
needs to gather data to do the research on. He decides to use the commit logs of the project, as well
as the email conversation (from all teams) about the development. The next step is to make a list of
signifiers for the quality attributes. For this, he uses a Wordnet, to discover synonyms and
meronyms. In addition, he interviews some of the developers of every team, to see what terms and
words they associate with the quality attributes. After this, the third step can begin, which is to
analyze the gathered data with help of the list of signifiers. The next step is to figure out how the
false positive ratio is. When this is done, a comparison between the teams can be made. The
consultant can present the results to the board.
Deliverable: List of Signifiers
One of the deliverables of the case described above is a well-defined set of signifiers. This is one of
the most important deliverables, because when the list is incomplete, not every occurrence of a
quality attribute gets noticed, which impairs the research. Of course, the data which contains
occurrences of the quality attributes is also very important. A good starting point for the list of
signifiers is to choose standardized quality attributes, like the before mentioned ISO/IEC standard FR-U-E-M-P. The next thing to do is to get synonyms for all of these quality attributes. The researcher
should also try to find meronyms. These are words or concepts that are a part of some bigger
concept (i.e. room is a meronym for house). The last set of words that needs to be added to the list is
the set of words that are related to the quality attributes (e.g. speed for Efficiency). These relations
can be found via a website like Wordnet, but also by doing interviews with the developers of the
product that is the subject of the research. This last way will probably result in a better list of
signifiers, because now the signifiers that the developers associate with the quality attributes are
added.
Related literature
The paper is more or less an extension of another paper written by the authors (N. a. Ernst &
Mylopoulos, 2007), where they describe the past and the future of requirements for designing
software. To get a better view on how quality requirements may change during the project lifecycle,
another research was conducted. The method in the paper discussed in this description borders on
the field of requirements engineering, as is a method that investigates how the requirements of a
project may change over time. Requirements engineering is an important field in software
development, as there needs to be a good set of requirements in order to develop a good software
product (Sommerville & Sawyer, 1997).The results of the research however, are not conclusive.
Therefore they cannot be used in mainstream Requirement Engineering practice. That this is a
common problem is described by Kaindl et al. (2002).
Other methods to gather data about non-functional requirements are described by (Cleland-huang,
Settimi, Zou, & Solc, 2006). However, this method is more an automated algorithm applied to
sources of data (like unspecified requirements documents and unstructured notes, memos and
meeting minutes). A more in-depth explanation about non-functional requirements is given by
Chung, Cesar, & Leite (2009). As requirements in an open source project like Gnome are not
generated by small team, but by all people that are collaborating in the project, they might be hard
to determine (Scacchi, 2001).
References
Chung, L., Cesar, J., & Leite, P. (2009). On Non-Functional Requirements in Software Engineering, 363–379.
Cleland-huang, J., Settimi, R., Zou, X., & Solc, P. (2006). The Detection and Classification of Non-Functional
Requirements with Application to Early Aspects 2 . Existing NFR classification methods.
Ernst, N. A. (2012). Software Evolution: a Requirements Engineering Approach.
Ernst, N. a., & Mylopoulos, J. (2007). Tracing software evolution history with design goals. Third
International IEEE Workshop on Software Evolvability 2007, 36–41. doi:10.1109/SE.2007.10
German, D. M. (2003). The GNOME project: a case study of open source, global software development.
Software Process: Improvement and Practice, 8(4), 201–215. doi:10.1002/spip.189
International Standards Organization. (2001). ISO/IEC 9126-1:2001 - Software engineering -- Product
quality -- Part 1: Quality model. Retrieved from
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=22749
Kaindl, H., Brinkkemper, S., Bubenko Jr, J. a., Farbey, B., Greenspan, S. J., Heitmeyer*, C. L., Leite†, J. C. S. D.
P., et al. (2002). Requirements Engineering and Technology Transfer: Obstacles, Incentives and
Improvement Agenda. Requirements Engineering, 7(3), 113–123. doi:10.1007/s007660200008
Miller, G. a. (1995). WordNet: a lexical database for English. Communications of the ACM, 38(11), 39–41.
doi:10.1145/219717.219748
Scacchi, W. (2001). Understanding Requirements for Developing Open Source Software Systems, (July).
Sommerville, I., & Sawyer, P. (1997). Requirements engineering: a good practice guide. John Wiley & Sons,
Inc.