Ref. Ares(2017)13571 - 03/01/2017
Quasar Science Resources, S.L
Gabriel García Márquez 4
Las Rozas de Madrid
28232 Madrid (Spain)
DOCUMENT
Software Quality assurance Plan
(SQAP)
Quasar Science Resources hardware and software proposed infrastructure for the
StarFormMapper H2020 project
Prepared by
Reference
Issue
Revision
Date of Issue
Status
Document Type
Distribution
José María, Quasar Science Resources, S.L
SFM-SW-DD-0013-1.0
1
0
01/12/2016
Approved
Document Deliverable (DD)
Quasar Science Resources (QSR)
Quasar Science Resources, S.L
APPROVAL
TITLE Software Quality assurance Plan (SQAP)
Issue 1
Revision 0
Author José María
Date 01/12/2016
Approved by Aitor Ibarra
Date 01/12/2016
CHANGE LOG
Reason for change
Issue
Revision
Date
Revision 0
Pages
Paragraph(s)
CHANGE RECORD
Issue 1
Reason for change
Date
Page 2/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Table of contents:
TABLE OF CONTENTS: ............................................................................................................................................. 3
1. INTRODUCTION ............................................................................................................................................. 4
1.1. SCOPE ............................................................................................................................................................ 4
1.2. APPLICABLE DOCUMENTS .............................................................................................................................. 4
1.3. ACRONYMS ................................................................................................................................................... 4
2. SOFTWARE STANDARD (ECSS-E-40) ............................................................................................................... 7
2.1. INTRODUCTION ............................................................................................................................................. 7
2.2. ECSS-E-40....................................................................................................................................................... 8
2.2.1. Processes.............................................................................................................................................................. 9
2.2.2. Reviews .............................................................................................................................................................. 11
2.2.3. Software documentation ................................................................................................................................... 12
2.2.4. Software life cycles ............................................................................................................................................ 13
3. CONTINUOUS INTEGRATIONS ...................................................................................................................... 13
3.1. PRACTICES OF CI .......................................................................................................................................... 14
3.1.1. Maintain a single source repository ................................................................................................................... 14
3.1.2. Automate the Guild............................................................................................................................................ 14
3.1.3. Automate unit test ............................................................................................................................................. 14
3.1.4. Everyone commits to the mainline every day .................................................................................................... 15
3.1.5. Every commit should build the mainline on an integration machine ................................................................ 15
3.1.6. Fix broken builds immediately ........................................................................................................................... 15
3.1.7. Keep the build Fast............................................................................................................................................. 15
3.1.8. Other practices................................................................................................................................................... 16
3.2. ADVANTAGES .............................................................................................................................................. 17
3.3. JENKINS ....................................................................................................................................................... 17
3.3.1. Features ............................................................................................................................................................. 18
4. SOFTWARE METHODOLOGY ........................................................................................................................ 20
4.1. AGILE DEVELOPMENT: SCRUM ..................................................................................................................... 22
4.1.1. Roles................................................................................................................................................................... 22
4.1.2. Components of the Scrum ................................................................................................................................. 23
4.1.3. Working meetings .............................................................................................................................................. 24
4.2. SCRUM WITH JIRA SOFTWARE ..................................................................................................................... 25
4.3. UNIT TESTING .............................................................................................................................................. 27
4.3.1. Unit test structure .............................................................................................................................................. 28
4.3.2. Advantages......................................................................................................................................................... 28
4.3.3. Tools ................................................................................................................................................................... 29
4.4. CODE COVERAGE ......................................................................................................................................... 30
4.4.1. Sonarcube .......................................................................................................................................................... 30
5. SOFTWARE CONFIGURATION MANAGEMENT .............................................................................................. 32
5.1. GIT............................................................................................................................................................... 34
5.2. JIRA ............................................................................................................................................................. 35
5.3. DEVELOPMENT AND PRODUCTION ENVIRONMENT ..................................................................................... 35
5.3.1. Eclipse ................................................................................................................................................................ 36
5.3.2. NetBeans ............................................................................................................................................................ 37
5.3.3. MS Visual Studio ................................................................................................................................................ 37
6. DOCUMENT MANAGEMENT CONFIGURATION CONTROL ............................................................................. 38
6.1. CONFLUENCE ............................................................................................................................................... 39
6.2. DOCUMENT MANAGEMENT AT JIRA / CONFLUENCE.................................................................................... 40
7. CONCLUSIONS ............................................................................................................................................. 41
8. REFERENCES ................................................................................................................................................ 41
Page 3/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
1. Introduction
1.1.
Scope
This document is the deliverable “Software Quality assurance Plan (SQAP)” for the EU H2020
(COMPET-5-2015 – Space) project “A Gaia and Herschel Study of the Density Distribution and
Evolution of Young Massive Star Clusters” (Grant Agreement Number: 687528), with
abbreviated code name StarFormMapper (SFM) project. The structure and content is based on
[AD1].
The document is organised in the following sections:
•
•
•
•
•
•
•
•
Section 1 – Introduction defining the purpose and scope of this document.
Section 2 – Software Standard (ECSS-E-40).
Section 3 – Continuous integrations.
Section 4 – Software Methodology.
Section 5 – Software Configuration Management.
Section 6 – Document management configuration control.
Section 7 – Conclusions.
Section 8 – References.
1.2.
Applicable documents
The following documents contain requirements and references, which are applicable to the
work:
•
•
[AD1] SFM proposal: A Gaia and Herschel Study of the Density Distribution and
Evolution of Young Massive Star Clusters. Reference 687528. Reference: Space2015StarFormMapper-Parts1-3, Issue: 1.1, Date: 04/08/2015.
[AD2] SFM-MAN-SDP-1.0: SFM System Developmen Plan.
1.3.
Acronyms
ACID Atomicity, Consistency, Isolation, and Durability
AD Applicable Document
ADQL Astronomical Data Query Language
AJAX Asynchronous JavaScript And XML
API Application Programming Interface
AR Annual Review
AVI Added Value Interface
CARD Cardiff University
CI Configuration Item
Page 4/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
CDR Critical Design Review
CDS Centre de Données astronomiques de Strasbourg
CLR Common Language Runtime
CSS Cascading StyleSheets
CSS3 Latest version of CSS
CU Coordination Units
DD Documents Deliverable
DDF Design Definition File
DEAVI Dynamic Evolution Added Value Interface
DJF Design Justification File
DMP Data Management Plan
DOM Document Object Model
DPAC Data Processing and analysis Consortium
DPACE interface for DPAC to the scientific community and to ESA and industry partners.
DPC Data Processing Centre
DR1 Data Release 1
ECSS European Cooperation for Space Standardization
ESA European Space Agency
FAT Factory Acceptance Test
FCL Framework Class Library
FIR Final Internal Review
GAVIP Gaia Added Value Interface Platform
GREN Univertité Jospeh Fournier, Grenoble
GWT Google Web toolkit
HEALPix Hierarchical Equal Area isoLatitude Pixelization
HiPS Hierarchical Progressive Surveys
HTML HyperText Markup Language
HTML5 fifth version of HTML
HVM Hardware Virtualization
IDE International Development Environment
IT Information Technology
IVOA International Virtual Observatory Alliance
JS Java Script
JSON JavaScript Object Notation
JSP JavaServer Pages
KoM Kick-off Meeting
LEEDS Leeds University
MLA MultiLateral Agreement
MVT Model View Template
NoSQL Not only SQL
ORR Operational Readiness Review
OS Operating System
PDL Parameter Description Language
PDR Preliminary Design Review
PHP PHP: Hypertext Preprocessor
Page 5/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
PL/SQL Procedural Language / Structured Query Language
PM Progress Meeting
PSAT Preliminary SAT
PSS Procedures, Specifications &, Standards
PV Paravirtualization
QA Quality Assurance
QR Qualification Review
QUAS/QSR Quasar Science Resource S.L.
RA Radiative Transfer
RB Reference Baseline
RD Reference Document
RISEA Remote Interface to the Stellar Evolution Algorithm
REST Representational State Transfer
SAT Site Acceptance Test
SCM Software Configuration Management
SFMP Software Development Management Plan
SFM StarFormMapper
SIAP Sloan Digital Sky Survey
SIMBAD Set of Identifications, Measurements and Bibliography for Astronomical Data
SQAP Software Quality Assurance Plan
SQL Structured Query Language
SSAP Simple Spectral Access Protocol
TAP Table Access Protocol
TBC To Be Confirmed
TBD To Be Determined
TBS To Be Specified
TDD Test Driven Development
TS Technical Specification
URD User Requirements Document
UWS Universal Worker Service
VM Virtual Machine
VO Virtual Observatory
WBS Work Breakdown Structure
WP Work Package
XML eXtensible Markup Language
XP Extreme Programming
Page 6/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
2. Software Standard (ECSS-E-40)
2.1.
Introduction
In June 1994, the ESA Council adopted a resolution to confirm the Agency’s commitment to the
transfer of the PSS (Procedures, Specifications &. Standards) system of ESA space standards to
the new set of standards prepared by the European Cooperation for Space Standardisation [1].
The structure of the ECSS standards is shown in Figure 1. As can be seen, it has three main
branches:
•
•
•
The management standards (M-series): These provide a uniform approach to a number
of common management issues.
The engineering standards (E-series): Addressing a broad range of key Space engineering
disciplines.
The product assurance standards (Q-series): Providing both general coverage, and
specific coverage for specific disciplines.
Figure 1: Structure of the ECSS standards [1].
Page 7/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
The ECSS system is organized into three levels that are interconnected [2]. In this way, the third
level is inside the second and the second is inside the first. The first one is "Policy and principles".
This level encompasses the general statements of policy and principles for standardisation in
that domain. For example in Table 1, the E-00 is the nomenclature for the engineering domain.
The second one is "Requirements". The requirements for the specific disciplines in that domain
are described in this level. Continuing with the previous example, E-40B is the nomenclature for
software. Finally, the third level is "Normative documents and handbooks" where are the
guidelines to interpreting the requirements for specific applications. In the case of our example,
it could be E-40-03 for the case of ground segment software.
Level 1
E-00
Policy and Principles
Level 2
E-40B
Software
Level 3
E-40-03
Ground Segment software
Table 1: Example of the three levels in the ECSS system.
The software standardisation within ECSS spans over the Engineering branch (the E-40 family)
and the Quality branch (the Q-80 family) 1. ECSS software standards, in particular ECSS-E-40
(Software Engineering) and ECSS-Q-80 (Software Product Assurance), have been in use since
1999 and 1996 respectively and were reissued in improved "B" and "C" (ECSS-E-ST-40, 2007)
versions [3]. They are complemented by ECSS-E40-07, a specific standard on ‘software
simulation’, and by the following level three handbooks:
•
•
•
•
•
•
•
•
•
ECSS-E-40-01 “Space Segment Software”.
ECSS-E-40-03 “Ground Segment Software”.
ECSS-E-40-04 “Software Life-cycle”.
ECSS-E-40-05 “Software Verification, Validation and Testing”.
ECSS-E-40-06 “Software development for reuse”.
ECSS-Q-80-01 “Guidelines for the Reuse of Pre-developed Software”.
ECSS-Q-80-02 “Guidelines on Software Process Assessment and Improvement”.
ECSS-Q-80-03 “Guidelines for Software Dependability and Safety Methods and
Techniques”.
ECSS-Q-80-04 “Guidelines for Software Metrication Programme Definition and
Implementation”.
Throughout the next section, a brief review of the ECSS-E-40 family will be conducted. For this
purpose, [1] will be used as the base of the content.
2.2.
ECSS-E-40
QSR has set the use of ECSS-E-40 in this project. ECSS-E-40 is based on the customer–supplier
model [4, 5]. This model begins with customer requirements and emphasizes active measures
1
http://www.esa.int/TEC/Software_engineering_and_standardisation/TECMKDUXBQE_0.html
Page 8/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
for customer satisfaction. Thus, the feedback is provided and the process is improved. As shown
in Figure 2, this concept may be applied recursively.
Figure 2: Customer–supplier relationship, [1].
Several processes, reviews and documentation are required in ECSS-E-40. These are described
below.
2.2.1. Processes
System Engineering: This is carried out by the “customer” and involves such activities as:
•
•
•
System requirements engineering
System integration
System validation.
The software is a part of a system that put requirements on it – the surrounding system could
be onboard hardware and other software systems, and need not necessarily include a human
user.
Software Requirements Engineering: This is carried out by the “supplier” and in essence
involves:
•
•
Software-requirements analysis.
Software top-level architectural design.
The related review is the Preliminary Design Review (PDR).
Software Design Engineering: This is also carried out by the supplier and involves:
Page 9/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
•
•
•
Designing of software items.
Coding and unit testing.
Integration.
Validation with respect to the technical specification.
The related review is the Critical Design Review (CDR).
Software Validation and Acceptance: This comprises:
•
Validation with respect to the requirements baseline: the milestone is the Qualification
Review (QR), which is carried out in the supplier’s environment and is often referred to
as the “Factory Acceptance Test (FAT)”. Milestones are tools used in project
management to mark specific points along a project timeline and baselines are defined
as a point in the software life cycle in which the configuration control is applied to a
specific configuration element.
•
Software delivery and installation.
•
Software acceptance: the milestone is the Acceptance Review (AR) and is carried out in
the operational environment. This is also referred to as the Site Acceptance Test (SAT),
and may be preceded by a Preliminary SAT (PSAT). Acceptance is carried out by the
customer.
Software Operations Engineering. This comprises:
•
Preparation of software operations procedures.
•
Preparation of plans for operational testing (i.e. of new releases coming from the
maintenance process).
•
Software operations proper.
•
User support, including what is usually called "first-line support", e.g. help desk.
Software Maintenance. This comprises:
•
Software problem analysis.
•
Software problem correction (software modification).
•
Re-acceptance (i.e. validation of corrections).
•
Software migration.
•
Software retirement.
Page 10/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
In Figure 3 the processes of ECSS-E-40 is shown.
Figure 3: Processes of ECSS E-40..
2.2.2. Reviews
In Table 2 the reviews required by ECSS-E-40 are collected.
Name
Acronym
Operational Readiness Review
ORR
System Requirements Review
Preliminary Design Review
Critical Design Review
Qualification Review
Acceptance Review
SRR
PDR
CDR
QR
AR
Associated process
System engineering
Requirements engineering
Design engineering
Validation and acceptance
Validation and acceptance
Software operations
engineering
Table 2: Reviews required by ECSS-E-40.
Page 11/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
2.2.3. Software documentation
The main categories of documentation are shown in Figure 4. It is arranged in "folders", into
which the various output documents are aggregated. The main folders are:
•
•
•
•
Requirements Baseline (RB)
Technical Specification (TS)
Design Definition File (DDF)
Design Justification File (DJF).
The contents of these folders are built up in the course of the project, as shown in Figure 4.
Figure 4: Main categories of ECSS-E-40 documentation.
Page 12/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
2.2.4. Software life cycles
The software life cycle defines the sequencing and dependencies of the processes. No particular
life-cycle model is imposed, but its selection is an essential management activity. The supplier
must document this choice in the Software Development Management Plan (SDMP).
3. Continuous integrations
According to Martin Fowler, the Continuous Integration (CI) is a software development practice
where members of a team integrate their work frequently. Usually each person integrates at
least daily - leading to multiple integrations per day. Each integration is verified by an automated
build (including test) to detect integration errors as quickly as possible [6]. QSR has set the use
of CI in this project. An scheme of CI is shown in Figure 5.
Figure 5: Scheme of CI.
Page 13/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.1.
Practices of CI
To carry out a good continuous integration some practices are necessary [6]. These are
discussed below.
3.1.1. Maintain a single source repository
The code repository is a fundamental tool that allows the team to work in a totally synchronized,
but at the same time simple way [7]. Each one works on his own. So it is possible to update the
changes without other developers having to wait for them and vice versa. At present there are
many free quality solutions 2, such as Apache Subversion (SVN), Git [8]. This is a fundamental
piece and it will be virtually impossible to perform continuous integration if it is not available. In
our case, the GIT repository was proposed by QSR, Section 5.1.
3.1.2. Automate the Guild
Any developed project must be build on any machine without the need of complex
environments. Not all the machines will have the development environment, even in some cases
they may have relatively expensive licenses, so it would not be a viable option. The condition is
that it must be a very simple tool that can be run on any machine and that can automate the
compilation process, the execution of the tests of the application and even the deployment in
the application server. There are many different solutions depending on the programming
languages. For example, in Java they are Ant [9] or Maven [10], in Ruby is Rake [11], etc.
3.1.3. Automate unit test
In order to verify that the project works automatically unit tests (often even integration tests)
are necessary. The fundamental premise is that all new functionality must have a set of tests
that verify that its behavior is correct.
There are many different ways to include tests in the project. In particular the rise of Extreme
Programming (XP) [12] and Test Driven Development (TDD) [13]. Over the last few years the rise
of TDD has popularized the XUnit family of open-source tools which are ideal for this kind of
testing [14]. This point is dealt with in Section 4.3.
2
https://en.wikipedia.org/wiki/Comparison_of_version_control_software
Page 14/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.1.4. Everyone commits to the mainline every day
Not all requirements are based on having a certain type of tool. Some are based on knowing
how to use a particular tool. Once the version control software is chosen, ideally a developer
should make even several uploads to the code repository a day. If the project has a code
repository, but each developer raises their changes every two weeks, the same problem is
appeared. The integration of the project is delayed during all that time, losing any option to
obtain information immediately. In addition, delaying the integration generates more conflicts
between the files edited by the developers. Because with each update of the repository a
complete project integration is done, every minimum change that does not destabilize the
project must be uploaded to verify that everything is still working correctly.
3.1.5. Every commit should build the mainline on an integration machine
There are two main ways: using a manual build or a continuous integration server [6]. The first
one it is similar to the local build that a developer does before the commit into the repository.
The developer goes to the integration machine, checks out the head of the mainline and kicks
off the integration build. A continuous integration server acts as a monitor to the repository.
Every time a commit against the repository finishes the server automatically checks out the
sources onto the integration machine, initiates a build, and notifies the committer of the result
of the build.
QSR has chosen the second option. Assembling an integration server, such as Jenkins (which is
chosen by QSR, see Section 3.3), or Cruise Control 3, is free and as simple as deploying a file on a
server. On the contrary the advantages are very important. On the one hand, the process is fully
automated, which avoids human error. On the other hand, it reduces the amount of work, since
a prefabricated solution is taken without having to create complex customized solutions.
3.1.6. Fix broken builds immediately
The goal is to automate the simplest tasks, to detect the most basic faults as soon as possible.
The longer the time elapses from the production of the error to its detection, the more difficult
it is to solve it.
3.1.7. Keep the build Fast
This is one of the first tasks to define, to know how the development is, what is the criterion for
the code to promote from one environment to another, and what is done in each environment.
3
http://cruisecontrol.sourceforge.net/
Page 15/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
To implement continuous integration, a pipeline, a set of stages, phases through which the
software passes and are automated is usually defined. An example is shown in Figure 6.
Figure 6: Example of pipeline in Jenkins.
3.1.8. Other practices
With the steps shown so far, a fairly complete process and that will improve the quality of the
software is described. However, it is recommended the use of the integration server to do
certain relatively complex tasks. For example, it is possible to configure the server to do static
code analysis, so that you can search the entire code for suspicious blocks, duplicate blocks or
references that are not used, etc. There are many options such as FindBugs [15] or PDM [16].
Another thing could be to include tasks that allow the application to be deployed on a server.
Thus, a demo server with the latest version of the project is always available. Another operation
that we can automate is to perform system tests on the application. It possible to create tests
with a navigation recording tool, such as Selenium 4, and launch them with every construction
that the server does [17].
Finally, other simple practices help ensure the proper functioning of the software such as to
have a clone of the production environment, the latest executable available for everyone and
scripts that allow to deploy the application into any environment easily.
4
http://docs.seleniumhq.org/
Page 16/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.2.
Advantages
The advantages of the use of IC can be summarized in the following points [18]:
•
•
•
•
•
Prevention and reduction of errors.
Generation of analysis and reporting on the quality of the code.
Eradication of extensive installation manuals.
Reduction of risk.
Reduction of overheads throughout the development and deployment process.
3.3.
Jenkins
As mentioned in the previous section, Jenkins, https://jenkins.io/, is the proposed software by
QSR for CI. It is a cross-platform, continuous integration and continuous delivery application 5. It
is possible to use Jenkins to build and test software projects continuously making it easier for
developers to integrate changes to the project, and making it easier for users to obtain a fresh
build.
Figure 7: Jenkins logo. https://jenkins.io/
This tool is inherited from another similar called Hudson 6, devised by Kohsuke Kawaguchi 7, who
worked at Sun. A few years after Oracle bought Sun, the Hudson community decided to rename
the project to Jenkins, migrate the code to Github and continue the work from there. However,
Oracle has continued to maintain and work on Hudson.
https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins
http://hudson-ci.org/
7
https://en.wikipedia.org/wiki/Kohsuke_Kawaguchi
5
6
Page 17/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.3.1. Features
•
Continuous integration and continuous delivery: As an extensible automation server,
Jenkins can be used as a simple CI server or turned into the continuous delivery hub for
any project.
Figure 8: Continuous integration with Jenkins.
•
Easy installation 8: It is a self-contained Java-based program, ready to run out-of-the-box,
with packages for Windows, Mac OS X and other Unix-like operating systems.
Figure 9: Installation of Jenkins.
8
https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins
Page 18/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
Easy configuration: It can be easily set up and configured via its web interface, Figure
10, which includes on-the-fly error checks and built-in help.
Figure 10: Web interface of Jenkins.
•
Plugins: With hundreds of plugins in the Update Center, https://updates.jenkins-ci.org/,
Figure 11, It integrates with practically every tool in the continuous integration and
continuous delivery toolchain.
Figure 11: Plugins home site of Jenkins.
•
Extensible: It can be extended via its plugin architecture, providing nearly infinite
possibilities for what Jenkins can do.
Page 19/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
Distributed: It can easily distribute work across multiple machines, helping drive builds,
tests and deployments across multiple platforms faster.
4. Software Methodology
The methodology for software development is a systematic way of realizing and managing a
project to carry it out with high chances of success. In other words, within a methodology for
software development are the processes to systematically follow to design, implement and
maintain a software product from the time the need for the product arises until we fulfill the
objective for which it was created [19]. Nowadays there are different methodologies. Taking
into account the philosophy of development of methodologies, those with greater emphasis on
project planning and control, in precise specification of requirements and modeling, receive the
appellation of heavyweight methodologies [20]. These traditional methodologies impose a work
discipline on the software development process, in order to achieve a more efficient software.
For this, emphasis is placed on the overall planning of all the work to be done and once it is all
detailed, the software product development cycle begins. On the other hand, a lightweight (or
Agile) methodology, Section 4.1, has only a few rules and practices which are easy to follow.
Agile processes emphasizes both the rapid and flexible adaptation to changes in the process, the
product, and the development environment [21]. The main differences between both are
collected in Table 3 [20]. The choice of one or the other is made according to the characteristics
of the project [22].
Approach
Success measurement
Project size
Management style
Perspective to change
Culture
Documentation
Emphasis
Cycles
Domain
Upfron planning
Return on investment
Team size
Agile methodologies
Adaptive
Business value
Small
Decentralized
Change adaptability
Leadership-collaboration
Low
People-oriented
Numerous
Unpredictable/exploratory
Minimal
Early in project
Small/ creative
Heavy methodologies
Predictive
Conformation to plan
Large
Autocratic
Change sustainability
Command-control
Heavy
Process-oriented
Limited
Predictable
Comprehesive
End of project
Large
Table 3: Differences between traditional and Agile methodologies [20].
Among the most used of both types are waterfall development [23], prototyping [24], iterative
and incremental development [25, 26], spiral development [27], rapid application development
Page 20/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
[28], etc. The scheme of some of these methodologies are shown in Figure 12. Throughout the
development of this project, QSR has set the use of an agile methodology [29], specifically Scrum
which will be explained below.
Figure 12: Examples of heavy and agile software development methodologies schemes. (a) Waterfall, (b)
prototyping, (c) iterative and incremental development, (d) spiral and (e) rapid application.
Page 21/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
4.1.
Agile development: Scrum
There are certain development projects that cannot be completely defined at the beginning of
the project because they require a constant revision and modification process. Our project is
located in this case so the methodology most indicated is the Agile. It allows maintaining
sufficient flexibility to adapt to a changing context maintaining a set of general rules that all
participants must know. In this sense, Scrum can be considered as the paradigm of Agile
development methodology, defining the way to approach a software development process in
an agile and light way, through the description of a set of roles, components and organization
of daily activity [30]. The fundamental basis of this methodology consists in the division of the
complete work (Product Backlog) in different sections or blocks that can be approached in short
periods of time (1-4 weeks) that are denominated Sprints. This organization of the process of
creation of software allows strengthening the following aspects:
•
Agile: The division of labor into small functional units (sprints) allows maintaining a
policy of frequent software deliveries that offer a clear view of the state of the process
and allows the introduction of modifications.
•
Simple: It focuses especially on facilitating rapid development, so its complexity (for
example from the point of view of the documentation to be generated or the
organization of equipment) has been tried to reduce to the maximum.
•
Flexible: All development is contemplated as a cycle of continuous development
iterations, which facilitates the introduction of modifications in real time, continuously
improving the process.
•
Collaborative: The approach, from the point of view of the organization of the team, is
quite horizontal giving the members of the development team a high degree of
autonomy and self-organization of their work.
4.1.1. Roles
As shown in Figure 13, different roles are defined within the Scrum framework:
•
Product Owner: It is the person responsible for transmitting to the development team
the vision of the product to be created, providing the business perspective. It represents
the rest of people of the project (stakeholders, customers, managers, etc.) in product
development. The Product Owner is responsible for defining the Product Backlog,
prioritizing them, and finally validating them.
•
Stakeholders: A group of people who are not directly involved in the development
process, but who should be taken into account, because they are people interested in
Page 22/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
it, such as directors, managers, commercials, etc. The Product Owner will be responsible
for collecting the opinions and suggestions and deciding whether to apply them to the
Product Backlog, as well as to decide if to invite any of these people to the process of
reviewing deliverables.
•
Users: Like stakeholders, they are not part of the creation process directly (they could
be in the revision phase of deliverables if deemed necessary). They are the final
recipients of the application to be developed, the target audience of the application.
•
Scrum master: Person who guarantees that the work team does not find impediments
or obstacles to approach their tasks within the project.
•
Scrum team: Team responsible for developing and delivering the product. It maintains
a horizontal organization in which each member of the team self-manages and organizes
freely in the definition and execution of the different sprints
Figure 13: Roles in Scrum.
In the particular case of our project, the product owner is the set of consortium members and
the stakeholders are the scientists who will provide their opinions and suggestions for
interacting with their algorithms. At the end of the project, anyone will be able to use the final
software therefore any person receives the role of user. Finally, both master and team Scrum
roles are related to the QSR team.
4.1.2. Components of the Scrum
•
Product Backlog: It is the definition of the project. It consists of a document that collects
the set of requirements that are associated with the project. It is the responsibility of
the Product Owner to make this definition and establish the priorities of each
Page 23/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
requirement. It is a high level document, containing generic (not detailed) descriptions,
and subject to modifications throughout development.
•
Sprint Backlog: It is the definition of the sprint. A sprint should be understood as a
subset of requirements, taken from the product backlog, to be executed during a period
of between 1 and 4 weeks of work. The sprint backlog would be the document that
describes the tasks that need to be performed to address the said subset of
requirements.
•
Running the Sprint: It would be the period of 1 to 4 weeks (period previously defined
based on the tasks collected in the sprint backlog) during which the work team would
address the corresponding development tasks. Once the execution of a defined sprint
has begun, it cannot be modified, and in case of being necessary to make changes these
will be done once the period ends through the definition of another sprint backlog.
•
Delivery: Once the execution of the sprint is completed, a portion of the potentially
definitive application will be available.
•
Burn down: It is a document that reflects the status of the project, indicating the volume
of requirements that are currently being addressed (in the product backlog), the
requirements that are currently being developed (sprint backlog) and the requirements
whose development has already been completed in its entirety.
4.1.3. Working meetings
•
Sprint planning meeting: It is done at the beginning of each sprint cycle, and it is
directed to select the set of requirements of the product backlog that will be addressed,
the work team that will be necessary and the estimated time (between 1 and 4 weeks)
for its development.
•
Daily scrum: This is done at the beginning of each day that the sprint is running. It is a
short meeting (no more than 30 minutes) in which the team members answer the
following questions: “What have you done since the last meeting”, “What problems did
you encounter to get the job done”, “What do you plan to do before the next meeting”.
•
Sprint review meeting: Once the sprint cycle is completed, a meeting is held which
defines which part of the planned work has been completed and which part remains
pending. As for the work completed, a review of the same is done to the product owner
and other users who may be involved.
•
Sprint retrospective: A meeting in which all team members make an assessment of the
work done in the last sprint, identifying points of improvement for the next to perform.
Page 24/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
The main objective is to introduce a continuous improvement component into the
process.
In Figure 14 a scheme of the Scrum framework is shown.
Figure 14: Scrum framework.
4.2.
Scrum with Jira software
Jira software has been chosen by QSR to develop the project by means of Scrum methodology.
Atlassian, software owner, provides a tutorial which is shown below 9:
•
Create a Scrum project: Once a JIRA Software account is started and logged in, it is
possible to create a project. Subsequently, the project type and the Scrum methodology
must be selected. A Scrum card is created automatically with a Scrum software
development project and it possible to view the empty backlog of the Scrum card.
•
Create user stories or tasks in the backlog: In JIRA, the tasks, and bugs "issues" are
called user stories 10. There is a quick create option on the backlog as is shown in Figure
15. Once it is created 6 or more user stories, the priorization the stories in the backlog
is allowed.
Figure 15: Creating a new story.
https://es.atlassian.com/agile/how-to-do-scrum-with-jira-software
In an agile framework, user stories are the smallest units of work. As a (type of user), I want (goal) so
that I (receive benefit).
9
10
Page 25/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
Estimate your stories: Before to begin the first sprint, it is important to estimate the
stories in the backlog and to determine the execution time. Estimates will also help
gauge how much work it should add to the next sprint based on the number of team
members. In JIRA Software, it possible to estimate issues by going to the backlog and
entering a number in the Estimate field for each issue.
•
Create a sprint: To create the first sprint in the backlog, Figure 16, and to organize issues
into sprints. Before starting a sprint, a sprint planning meeting must be held. Afterwards,
the stories are dragged to the sprint created. Finally, name the sprint and to add the
duration and start and end dates of the sprint. In Figure 17, an example the sprint in Jira
is shown.
Figure 16: Creating a sprint.
Figure 17: Sprint in Jira.
Page 26/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
View the burndown Chart. A good idea to check the Burndown Chart during a sprint. A
Burndown Chart, Figure 18 shows the actual and estimated amount of work to be done
in a sprint. The horizontal x-axis in a Burndown Chart indicates time, while the vertical
y-axis indicates issues. In JIRA, the Burndown Chart shows the actual and estimated
amount of work to be done in a sprint. To view this chart, click Reports from the sidebar,
and then select the Burndown Chart from the reports dropdown.
Figure 18: Burndown chart.
•
Complete the sprint. At the end of the sprint, a situation similar to in Figure 19 is shown.
After you complete the sprint, reports should be done. QSR has proposed the
Confluence software for this, Section 6.1.
Figure 19: Complete sprint.
4.3.
Unit testing
The rapid correction of errors is one of the premises of the Agile methodology. In this sense, it
is convenient to have a method to systematize this process. QSR has set the creation and use of
Page 27/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
unit tests. There are many definitions about what really is a unit test. However, some are
ambiguous and others have a particular focus according to the situation in which they are used
or developed. Unit testing is the testing of individual hardware or software units or groups of
related units where a unit is a software component that is not subdivided into other components
[31]. Unit testing is often simple and quick to code. In addition, they must have the following
characteristics [32]:
•
•
•
•
•
•
Unitary. It tests only small amounts of code.
Independent. This should not depend on or affect other unit tests.
Tests public methods. Otherwise the test would be fragile to changes in implementation
and could not be used in regression tests.
Automatizable. La prueba no debería requerir intervención manual.
Repeatable and predictable. This should not affect the order and the times that the test
is repeated. The result should always be the same.
Professionals. The tests should be considered the same as the code, with the same
professionalism, documentation, etc.
4.3.1. Unit test structure
Unit test usually have a structure composed of three parts:
•
•
•
Arrange: This is the part of the unit test where all the code is configured to run the test.
Act: This is the phase of the unit test where the code to be tested is executed.
Assert: It is the section of the unit test where the result of the test is checked.
4.3.2. Advantages
Unit testing seeks to isolate each part of the program and show that the individual parts are
correct, providing some advantages:
1. Encourage change. Unit testing facilitates code restructuring (refactoring). It allows testing
on changes and verifying that the modifications have not introduced errors (regression).
2. Simplified integration. They allow reaching the integration phase by ensuring that the
individual parts work correctly. In this way, integration tests are facilitated.
3. Document the code. The tests themselves can be considered documentation, since they are
a reference implementation of how to use the code.
4. Separation of interface and implementation. The only interaction between the test cases
and the units under test are the interfaces of the latter. It is possible to change either one
without affecting the other through, for example, the use of mock objects [33].
5. Less errors and easier to locate. Unit testing reduces the number of errors and time to locate
them.
6. They can improve the design. The use of Test-Driven Design and Development practices
(TDD) [13], allows defining the expected behavior in a step prior to coding.
Page 28/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
7. It may be the simplest way to verify operation in situations such as the development of an
API or a component that provides services that are not yet available to a customer.
4.3.3. Tools
Currently there are a variety of tools to perform unit tests. Among the most outstanding are the
following:
•
•
•
•
•
•
•
•
•
Junit, http://junit.org/junit4/: It is a simple framework to write repeatable tests. It is an
instance of the xUnit architecture for unit testing frameworks.
HttpUnit, http://httpunit.sourceforge.net/: Written in Java, HttpUnit emulates the
relevant portions of browser behavior, including form submission, JavaScript, basic http
authentication, cookies and automatic page redirection, and allows Java test code to
examine returned pages either as text, an XML DOM, or containers of forms, tables, and
links. When combined with a framework such as JUnit, it is fairly easy to write tests that
very quickly verify the functioning of a web site.
DbUnit, http://dbunit.sourceforge.net/: It is a JUnit extension (also usable with Ant)
targeted at database-driven projects that, among other things, puts your database into
a known state between test runs. This is an excellent way to avoid the myriad of
problems that can occur when one test case corrupts the database and causes
subsequent tests to fail or exacerbate the damage.
PHPUnit, https://phpunit.de/: It is a programmer-oriented testing framework for PHP.
CppUnit, http://sourceforge.net/projects/cppunit: It is the C++ port of the JUnit
framework for unit testing.
NUnit, http://www.nunit.org/: It is a unit-testing framework for all .Net languages.
FoxUnit, https://vfpx.codeplex.com/: tool to facilitate using unit tests for their Visual
FoxPro code base.
TestNG, http://testng.org/doc/index.html: It is a testing framework inspired from JUnit
and NUnit but introducing some new functionality that makes it more powerful and
easier to use.
Selenium, http://docs.seleniumhq.org/: It is is a suite of tools to automate web browsers
across many platforms.
Page 29/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 20: Unit testing tools.
4.4.
Code coverage
Another quality measure that QSR proposes to implement is the Code Coverage. It was one of
the first methods invented for software testing [34]. It is loosely defined as the percentage of
some code structure or artifact which is executed or covered by at least one test [35]. This data
can be used to determine how many lines of code have been executed by the tests and the lines
of code that have yet to be tested. Code coverage analysis allows [36]:
•
•
•
•
Find areas of a program that were not run by a set of tests.
Create additional test cases to increase embracement.
Determine a quantitative measure of the amount of code covered or tested, which is an
indirect quality measure.
Identify redundant test cases that do not increase embedding.
4.4.1. Sonarcube
There are different tools to perform Code coverage:
•
•
•
•
Cobertura, http://cobertura.github.io/cobertura/: It is a free Java tool that calculates
the percentage of code accessed by tests. It is based on jcoverage.
Visual Studio, https://www.visualstudio.com/: It is an integrated development
environment from Microsoft.
Atlassian Clover, https://es.atlassian.com/software/clover: It provides Java and Groovy
code coverage.
JaCoCo, http://www.eclemma.org/jacoco/: It is a free code coverage library for Java,
which has been created by the EclEmma team based on the lessons learned from using
and integration existing libraries for many years.
Page 30/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
Jcov, https://wiki.openjdk.java.net/display/CodeTools/jcov: is a pure java
implementation of a code coverage tool which provides a means to measure and
analyze dynamic code coverage of Java programs. JCov provides functionality to collect
method, linear block and branch coverage, as well as showing uncovered execution
paths. It is also able to show a program's source code annotated with coverage
information.
However, QSR has chosen SonarQube, http://www.sonarqube.org/.
Figure 21: SonarQube logo.
It is an open platform to manage code quality. As such, it covers the 7 axes of code quality:
Figure 22: Code quality in SonarQube.
The main features are:
•
•
•
•
•
•
It is possible to extend its operation with plugins.
More than 20 programming languages are covered through plugins including Java, C#,
C/C++, PL/SQL, Cobol, ABAP…
It is a web-based application. Rules, alerts, thresholds, exclusions, settings… can be
configured online.
Analyses can be easily integrated into any continuous integration server to fully
automate the process.
Developers can keep their new code clean with a focus on the Leak in the IDE, in pull
requests, and in SonarQube itself.
Source code can be checked before pushing changes to the SCM directly in the IDE with
SonarLint for Eclipse, IntelliJ IDEA, and Visual Studio or on the command line with
SonarLint for Command Line.
Page 31/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
•
The platform provides the ability to track quality on new code: recently introduced
issues, lack of test coverage on new code, etc. Once found, issues can easily be assigned
for mop-up.
It is possible the integration with other software such as Maven, Ant, SVN, Git,
Mercurial, JIRA, Mantis, Google Analytics, Piwik, Fortify…
Figure 23: SonarQube software.
5. Software Configuration Management
The computer infrastructure of this project is too broad and complex to be managed by people
or with systems of manuals, spreadsheets, etc. QSR has set the use of the Software Configuration
Management (SCM). SCM is an organizational framework for managing the evolution of
computer systems throughout all stages of systems development [37]. The configuration
management facilitates the maintenance of the application, providing precise information on
the impact of the requested changes and reducing the time of implementation of a change, both
evolutionary and corrective. It also allows the control of the product throughout its
development, obtain reports on the status of the project and reduce the number of errors of
adaptation of the system, which results in an increase in product quality. Its main objective is to
Page 32/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
maintain the integrity of the versions throughout the development of the product, ensuring that
no uncontrolled changes are made.
Figure 24: SCM in the project context.
The SCM practices that will be carried out by QSR are [38]:
•
•
•
Control and audit changes [39]. Change control is a mechanism for evaluating and
approving changes made to elements of software configuration during the life cycle.
Identify and store elements in a secure repository.
Require Change Requests and Change Packages according to the established procedure.
Figure 25: Example of change request workflow.
•
•
•
Organize versioned elements into versioned components.
Record and track requests for change.
Create baselines at project milestones.
Page 33/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
•
•
•
•
•
•
Organize and integrate consistent sets of versions using activities.
Support simultaneous changes in elements and components.
Maintain stable and consistent workspaces.
Ability to do continuous build and deployments.
Integrate early and frequently.
Ensure absolute reproducibility for all elements.
Then the tools chosen by QSR for this purpose will be described.
5.1.
Git
Git, https://git-scm.com/, is a free and open source distributed version control system designed
to handle everything from small to very large projects with speed and efficiency. It is easy to
learn and has a tiny footprint with lightning fast performance. It outclasses SCM tools like
Subversion, CVS, Perforce, and ClearCase with features like cheap local branching, convenient
staging areas, and multiple workflows.
Figure 26: Git logo.
Its main features are:
•
•
•
•
•
11
Branching and Merging: Git allows and encourages you to have multiple local branches
that can be entirely independent of each other.
Distributed: One of the nicest features of any Distributed SCM, Git included, is that it's
distributed. This means that instead of doing a "checkout" of the current tip of the
source code, you do a "clone" of the entire repository.
Data Assurance: The data model that Git uses ensures the cryptographic integrity of
every bit of the project. Every file and commit is checksummed and retrieved by its
checksum when checked back out.
Staging Area: Unlike the other systems, Git has something called the "staging area" or
"index". This is an intermediate area where commits can be formatted and reviewed
before completing the commit.
Git comes with built-in GUI tools for committing (git-gui) and browsing (gitk), but there
are several third-party tools for users looking for platform-specific experience 11.
https://git-scm.com/downloads/guis
Page 34/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
5.2.
Jira
JIRA, https://es.atlassian.com/software/jira, is a proprietary issue tracking product, developed
by Atlassian. It provides bug tracking, issue tracking, and project management functions.
Figure 27: Jira logo.
Its main features are:
•
•
•
•
•
Plan: Create user stories and issues, plan sprints, and distribute tasks across a software
team.
Track: Prioritize and discuss in full context with complete visibility.
Release: Information is always current.
Report: Improve team performance based on real-time, visual data.
Compatible with Scrum methodology as shown in Section 4.2.
Note: An independent database Server has to be configured in order for Atalassian SW to run.
Figure 28: Jira software.
5.3.
Development and production environment
Page 35/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
An Integrated Development Environment (IDE) is a program composed of a set of tools to
facilitate the development of software [40]. This is a programming environment that has been
packaged as an application program, ie it consists of a code editor, a compiler, a debugger and
a graphical GUI interface builder. IDEs provide a friendly framework for most programming
languages such as C ++, Java, C #, Basic, Object Pascal, etc. Among the main existing tools, QSR
will use of any of the following discussed below.
5.3.1. Eclipse
Eclipse, https://eclipse.org/, is a software platform comprised of a set of multiplatform open
source programming tools to develop software.
Figure 29: Eclipse logo.
Its current version, Eclipse Luna, includes official support for Java 8 in the Java development
tools, Plug-in Development Tools, Object Teams, Eclipse Communication Framework, Maven
integration, Xtext, Xtend, Web Tools Platform, and Memory Analyzer.
Figure 30: Eclipse IDE.
Page 36/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
5.3.2. NetBeans
NetBeans, https://netbeans.org/, is the official IDE for Java 8. With its editors, code analyzers,
and converters, it can quickly and smoothly upgrade applications to use new Java 8 language
constructs, such as lambdas, functional operations, and method references.
Figure 31: NetBeans logo.
Batch analyzers and converters are provided to search through multiple applications at the
same time, matching patterns for conversion to new Java 8 language constructs.
Figure 32: NetBans IDE.
5.3.3. MS Visual Studio
MS Visual Studio, https://www.visualstudio.com/vs/, is the IDE of Microsoft. Visual Studio
enables to write code accurately and efficiently without losing the current file context. It possible
easily zoom into details such as call structure, related functions, check-ins, and test status. Also
it has a functionality to refactor, identify, and fix code issues.
Page 37/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 33: MS Visual studio logo.
Figure 34: MS Visual Studio IDE.
6. Document management configuration control
When several authors collaborate in a shared space as in this project, they may need to edit and
revise common documents repeatedly. Keeping track of the changes made by the different
authors means an additional effort. That is why it is necessary to use specialized tools similar to
the case of the software development, Section 5. In this case QSR has chosen the Confluence
tool that is compatible with any established procedure such as the show in Figure 35. It has also
established the use of the coding of document names described in [AD2].
Page 38/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 35: Example of document management work flow.
6.1.
Confluence
Confluence, https://www.atlassian.com/software/confluence is the collaborative platform of
the company Atlassian. It allows connecting teams, and creating and sharing content and
knowledge to develop a better and faster work.
Figure 36: Confluence logo.
Its main features are:
•
•
•
•
•
•
•
Powerful editor: Confluence gives the power to create anything - meeting notes,
project plans, product requirements, etc. thanks to a simple, but powerful editor.
Feedback in context: It is possible to leave feedback on the work itself with inline
comments on any Confluence page.
File collaboration: It works with files every day – images, PDFs, spreadsheets, and
presentations. It can give feedback directly on the files and it keeps tracks of versions
automatically.
Organized workspaces.
Blogs and discussions: Share news and announcements and it is possible to have active
discussions with comments, @mentions and likes.
User and content permissions.
Page and file versioning: Track every version and change that you make to a page. Files
are automatically versioned.
Page 39/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 37: Confluence software.
6.2.
Document Management at JIRA / Confluence
Each project present in Jira, Section 5.2, has a space reserved to save the documentation in
Confluence. The project documentation is structured by default as is shown in Figure 38:
Figure 38: Structure of the documentation.
The association between the JIRA project and the Confluence space is done automatically
(Creating space in confluence). This configuration can be edited from the project management.
Page 40/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
7. Conclusions
This document discusses the techniques, procedures, and methodology that will be
implemented by QSR for software development and that should be adopted by the rest of the
consortium. Given the characteristics of the project, QSR proposes the adoption of the ECSS-E40 standard and the Scrum methodology for software implementation and development. To do
this, we have chosen a set of tools, which include Jenkins, Sonarcube, Git, Jira and Confluence.
These tools will also ensure a good interaction with the rest of the consortium as they can be
made accesible to everyone, and imposes a good development praxis beneficial for the success
of the SFM project.
8. References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
M. Jones, E. Gomez, A. Mantineo, and U. Mortensen, "Introducing ECSS softwareengineering standards within ESA," ESA bulletin, 132 (2002).
M. Jones, U. Mortensen, and J. Fairclough, "The ESA software engineering standards:
Past, present and future," in Software Engineering Standards Symposium and Forum,
1997. Emerging International Standards. ISESS 97., Third IEEE International, (1997), 119.
J. Eickhoff, "OBSW Development Process and Standards," in Onboard Computers,
Onboard Software and Satellite Operations (Springer, 2012), pp. 165.
K. R. Tuli, A. K. Kohli, and S. G. Bharadwaj, "Rethinking customer solutions: From product
bundles to relational processes," Journal of Marketing 71, 1 (2007).
P. Brereton, "The software customer/supplier relationship," Communications of the
ACM 47, 77 (2004).
M. Fowler and M. Foemmel, "Continuous integration," Thought-Works) http://www.
thoughtworks. com/Continuous Integration. pdf, 122 (2006).
F. J. Ruiz-Bertol and F. J. Zarazaga-Soria, "El Control de Versiones en el aprendizaje de la
Ingeniería Informática: Un enfoque práctico," Actas de las XIII Jornadas de Enseñanza
Universitaria de Informática (2007).
C. M. Pilato, B. Collins-Sussman, and B. W. Fitzpatrick, Version control with subversion ("
O'Reilly Media, Inc.", 2008).
N. Lesiecki and R. Hightower, "Java tools for eXtreme Programming: mastering open
source tools including, Ant, JUnit, and Cactus," (2001).
S. McIntosh, B. Adams, and A. E. Hassan, "The evolution of Java build systems," Empirical
Software Engineering 17, 578 (2012).
A. B. Hamou, "Extending ruby: A fistful of gems," Practical Ruby for System
Administration, 177 (2007).
K. Beck, Extreme programming explained: embrace change (addison-wesley
professional, 2000).
K. Beck, Test-driven development: by example (Addison-Wesley Professional, 2003).
G. Meszaros, xUnit test patterns: Refactoring test code (Pearson Education, 2007).
N. Ayewah, D. Hovemeyer, J. D. Morgenthaler, J. Penix, and W. Pugh, "Using static
analysis to find bugs," IEEE software 25, 22 (2008).
P. Louridas, "Static code analysis," IEEE Software 23, 58 (2006).
Page 41/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
A. Holmes and M. Kellogg, "Automating functional tests using selenium," in AGILE 2006
(AGILE'06), (2006), 6 pp.
C. Kaner, J. Bach, and B. Pettichord, Lessons Learned in Software Testing: A ContextDriven Approach (Wiley, 2011).
H. Zhu, Software Design Methodology: From Principles to Architectural Styles (Elsevier
Science, 2005).
M. Awad, "A comparison between agile and traditional software development
methodologies," University of Western Australia (2005).
A. I. Khan, R. J. Qurashi, and U. A. Khan, "A comprehensive study of commonly practiced
heavy and light weight software methodologies," arXiv preprint arXiv:1111.3001 (2011).
R. N. Burns and A. R. Dennis, "Selecting the appropriate application development
methodology," ACM Sigmis Database 17, 19 (1985).
K. Petersen, C. Wohlin, and D. Baca, "The waterfall model in large-scale development,"
in International Conference on Product-Focused Software Process Improvement, (2009),
386.
P. Tavolato and K. Vincena, "A prototyping methodology and its tool," in Approaches to
prototyping (Springer, 1984), pp. 434.
R. Victor, "Iterative and incremental development: A brief history," IEEE Computer
Society, 47 (2003).
C. Larman, Agile and iterative development: a manager's guide (Addison-Wesley
Professional, 2004).
B. W. Boehm, "A spiral model of software development and enhancement," Computer
21, 61 (1988).
T. S. Jones and R. C. Richey, "Rapid prototyping methodology in action: A developmental
study," Educational Technology Research and Development 48, 63 (2000).
P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta, "Agile software development
methods: Review and analysis," (VTT Finland, 2002).
K. Schwaber, "Scrum development process," in Business Object Design and
Implementation (Springer, 1997), pp. 117.
J. Radatz, A. Geraci, and F. Katki, "IEEE standard glossary of software engineering
terminology," IEEE Std 610121990, 3 (1990).
P. Runeson, "A survey of unit testing practices," IEEE software 23, 22 (2006).
T. Mackinnon, S. Freeman, and P. Craig, "Endo-testing: unit testing with mock objects,"
Extreme programming examined, 287 (2001).
J. C. Miller and C. J. Maloney, "Systematic mistake analysis of digital computer
programs," Communications of the ACM 6, 58 (1963).
H. Zhu, P. A. Hall, and J. H. May, "Software unit test coverage and adequacy," Acm
computing surveys (csur) 29, 366 (1997).
S. Desikan and G. Ramesh, Software Testing: Principles and Practice (Pearson Education
Canada, 2006).
J. Keyes, Software configuration management (CRC Press, 2004).
S. P. Berczuk and B. Appleton, Software Configuration Management Patterns: Effective
Teamwork, Practical Integration (Addison-Wesley, 2003).
E. F. Ferraby and L. Ferraby, Change control during computer systems development
(Prentice Hall, 1991).
M. Nagl, Building Tightly Integrated Software Development Environments: The IPSEN
Approach (Springer, 1996).
Page 42/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0