ASSURING THE QUALITY OF EXTERNAL
PARTICIPANTS' CONTRIBUTIONS
Introduction
It is impossible to Imagine a project without evaluation. Therefore it is all the more important to understand
how to ensure the quality of the evaluation. Having said that, it is a difficult enough process to evaluate the
quality of contributions by internal projects and tasks. This report has ben written in regards to the quality of
work done for parts of the project, by an non-internal entity. This is a much more difficult task to comprehend,
however if one properly implements SQA (software quality assurance) activities as described later in the
report, then problems behind many software failures or delays could have been avoided, and the prevention
of such troubles is the subject of this report.
Types of external participants
In the past, partners to software development were seen as the only participants in a given project. The term
“partners” is refers to two organisations. Firstly the “customer”, is defined as the organisation whom has
desired to acquire the software system. Secondly the “contractor” the organisation of whom the customer
has given the responsibilty of completing the project. Other organisations that contribute to the project
through a set of structured agreements with the contractor can be seen as “external participants”. This
definition excludes the contactor’s partners. It is said that the larger and more complex the project, the
increased likeliness of aquiring the assistance of external participants. Galin (2004) states that external
participants can be divided into three main categorys.
1.
1.
2.
Subcontractors
Suppliers of Commercial off the shelf software (COTS) and previously used modules
The customer may contribute to the project
Risks and benefits associated with the introduction of external participants
Risks for the contractor and customer usually correspond with each other. It is obvious that when work done
by an external particpant is delayed, it causes a ‘knock-on’ effect and consequently brings further, much
larger delays with regards to the completion of the project as a whole. This problem however is not common
when relying on COTS suppliers and can be seen as one of the many advantages of using such software.
Infact a shorter project schedule is just one of the benefits for the contractor, obtained from involving
external participants generally within the project. Invariably it reduces cost and releases high-impact
problems such as staff shortages to other organisations. These external organisations are also taken
advantage of when requiring expertise is specialised areas and provide sufficient advantages to the
contractor.
To take the focus away from benefits to the contractor and concentrate on risks for both parties again, there
may be occasions when faced with a gifted salesman, one can be convinced of the benefits of using the
‘latest’ software package that is yet to be released. In actual fact there can be significant risks involved in
doing so. It is common that the pending release of this new package suffers an on-going delay which in turn
affects customers project completion date.
Another problem is general low quality coding on the parts of the project supplied by external participants,
with the root causes being anything from defects in the code to the programmer adopting non-stardard
coiding practices. There are two difficult maintenance situations that often occur, both arising from the simple
fact that many organisations collaborate to develop the project but essentially only one of them is wholly
responsible. The first difficult situation being that the contractor is responsible for maintenance, however the
contractor may be faced with the problem of low quality work as described above leading to low quality
maintenance and eventually costing the contractor more money. Secondly, and quite to the contrary, is when
maintenance services are supplied by more than one organisation (supplier of COTS, subcontractors) and a
failure is discovered, the contractor must determine which individual organisation is to be held responsible.
The loss of control over project parts is another typical downfall. It is often the case for external participants
to under estimate the overall project outlook. If a problem occurs, as it inevitably does, and the contractor is
not informed immediately then the likelihood of finding an effective solution decreases.
There are benefits for the customer of course. As well as cost reductions, the utilisation of external
partiipants provides a way of protecting the customer’s commercial secret by means of legal agreement.
Moreover, the provision of employment to internal software development department, and aquisition of
project understanding for self-supplied maintenance (Galin, 2004).
Before entering into these agreements, the contractor must assess the relevent benefits and risks of
introducing external participants in a project (Galin, 2004).
Objectives
Having looked extensively at the risks involved of involving external participants in parts of the project, it is
clear what the objectives of implementating any SQA tools are. According to (Galin, 2004)
1. To prevent the task from being completed later then scheduled, and ensure early notification of pending
delay.
2. Assure the quality of work done is satisfactory and receive notification of any failure to meet the quality
requirements.
3. Assure enough documentation is available to serve the maintenance team.
4. Assure complete control over external participant’s performance.
SQA tools utilised
Any organisation collaborating with an external participant should make certain that they have their own SQA
systems for ensuring there own product’s quality levels which should be applied before and during
integration (Galin, 2004). Having said that, most of the SQA tools which are applied to the external
participants are defined in the contracts signed between the relevant parties. The main goal of this process is
to assure the quality of the work supplied by the external participants.
Requirements documents reviews
This process is departmental process performed by the members of the contractor and subcontractors. It has
direct effects on the quality of the software products because failures in the
project proposal and project contract are liable to pass to the other project
development life-cycle phases and their output products (Korkmaz, 2007). It is very important to start
the error detection in this very early stage of the project development. Contracts are not always adequete
enough to prevetn all problems, and it is thought that a stricter contract containing more clauses should be
out into effect. Often it is disovered that the subcontractor has subcontracted their assigned work out to
another firm. The contractor obviously loses control over the project at this stage which then leads to delays.
Choice of external participant
This process requires the collection and systematic evalutaion of information about the suppliers and
subcontractors based on previous experience in order to induce the contracotr towards a particluar
organisation. The contractor will have access to a file which records past performance in projects.Teams of
committees then evaluate the reports with a designated commitee member who is responsible for following
the project up. Regular users of whom have previously found bugs in the software also contribute to the
reporting. The problem which this however is that the same contractor is likely to have failed to care about
completing similar reports about their own experiences. Also, the reports often are written in an unstructured
manner which leads to difficulties when evaluating. The supplier often encourages auditing to stimulate
accpetance on their QA system. This serves a different purpose to other processes as it does not aim to
increase the quality of the software directly, rather to control the performance of SQA activities. Finally, and
perhaps most obviously a survey is conducted to gain inisght into opinions from outside companies.
Project co-ordination and joint control committee
The role of the committee is to co-ordinate and keep track on the progress of the project. There a number of
different activities of which are common to the committees schedule. Ofcourse, these varie depending on the
contributions of the external participant. It is the commitee’s task is to serve as main moving force of the
SQA system and its application. Some of the main activities according to Galin (Galin, 2004) are as follow:
(a) Ensure the timeline of the project is acceptable and that milesontes are met
(b) Conduct the relevent follow ups
(c) In severe situations, meetings and conferences must be arranged
(d) Desicion making about timetable/resource problems that were identified during the follow up
(e) Desicion making regarding solution of problems identified in design reviews and software tests
(f) Solving conflicts regarding contract implementation
Participation in design reviews
The main goal of design reviews is to realise the approval of documents produced in the design phase, so
the developer can continue to the next stage of develepmoent process.
Participation in software testing
When required it should use all the stages of the testing process: design reviews of planning and design of
the tests, evaluation of the results and follow up meetings for correction and regression testing (Galin, 2004).
Galin also said that shortcomings of the testing phase were one of the main causes of software errors.
Specialised procedures
Usually adaptions of processes already applied and discussed in this report but supported by templates,
checklists and forms. Standardising the format by using a template makes it easier for the SQA team to
review and run the tests. The main objectives according to Galin are as follows:
(a) Preparation of requirements documents for external participants
(b) Choice of external participant
(c) Audit of subcontractors in SQA system
(d) The supplier’s file
(e) Appointment of commitee for project parts carried out by external participants and preparations of
instructions
(f) Progress reporting requirements for project parts
Certification of external participants’ team leaders and other staff
Certification standards focus on the external issues of the project and are vital to a good contractual
relationship. Many aspects of contractual requirements can be partially violated by the subcontractor
violating the original agreement. For exmaple, frequently subcontractors under pressure replace qualified
members of a team by a non-qualified or even splitting a persons time between two given projects. This
breaches one of clauses that were set up from the outset. A solution to this problem is that the certification of
staff should be listed as a contractual requirement and changes made should first be approved by the
contractor and this procedure should be regualrly deployed.
Progress Reports
It is the job of the co-ordination and joint progress control committee to collect progress reports from the
various external participants of whom are contributing to the project. These reports address the postition of
affairs concerning risks identified in previous reports and their possible solutions. They also focus on
activities behind schedule and discuss current strategies. Furthermore, they propose new ideas that can be
approved by the commitee to minimise delays. Two other issues to be overed in progress reports are the
follow up of the usage of budget and resources. (Galin, 2004). These issues are predominately an immidiate
concern for any given subontractor especially when the task is agreed at a fixed-cost beforehand. However it
is obvious that should these issues develop, then the problem will soon become a concern for the contractor
as the project quality could become compromised.
Review of deliverables and acceptance tests
Acceptance tests are an effective way to obtain confirmation from the customer of the system, thus it
ensures that the system meets mutually agreed upon requirements. It is carried out by the contractor and is
one of the final stages of a project. It often occurs before the customer accepts a new system. If you
combine this process along with a thorough review of the deliverables (development documentation), often
using a software deliverables requirements form (SDRF) by the contractor then an independent review has
been conducted.
Conclusion
The role of quality assurance regarding external participants is vital across different industries. In the
software world, essentially every contractor utilises COTS suppliers which highlights the importance of
assuring the quality of this work. From my research, it seems that the advantages gained from introducing
external participants far outweigh the drawbacks. Evidence suggests that should all of the SQA tools be
applied, the likelihood of maintaining complete control over the project, and achieving the remaining
objectives is increased.
References
Galin, D. (2004) Software Quality Assurance: From Theory to Implementation, Addison Wesley
Ang, M. & Eich, B. (2000), “A look at the Mozilla technology and architecture” [online at
http://mozilla.org/docs/ora-oss2000/arch-overview/intro.html accessed: 11 Nov 2009].
Mozilla. (2009), “Quality Assurance page” [online at http://www.mozilla.org/quality/ accessed: 11 Nov 2009].
Gouge, I. (2007) Shaping the IT Organization - The Impact of Outsourcing and the New Business Model
(Practitioner Series),Springer
Korkmaz, O. (2007), "System simulation for software Quality Assurance (SQA)", The Graduate School of
Natural and Applied Sciences, Atilim University, Turkey
Accessed: 2009 http://www.library.atilim.edu.tr/tezler/1027397.pdf
Testa, L, M. (2009) Growing Software : Proven Strategies for Managing Software Engineers