European Experiences with Software Process Improvement
Fran O’Hara
Insight Consulting Ltd.
114 Granitefield
Dun Laoghaire
Co. Dublin, Ireland
Ph: +353-1-2854510
fran.ohara@insight.ie
http://www.insight.ie
ABSTRACT
This paper/presentation will provide a brief overview of the
status of Software Process Improvement (SPI) in Europe –
its history, current situation and future direction.
Four case studies will then be presented covering a diverse
range of business domains, organisational sizes and
approaches to SPI.
The author has worked closely with each of the
organisations involved in support of their SPI programmes.
The case studies will show the starting position of each
company, the approach taken, results achieved and lessons
learnt. A number of themes such as assessment approach
used, cultural/people issues, etc. will be used to explore the
experiences of the various companies. The four case studies
are

NewWorld
Commerce
Technologies),

Motorola Cork,

Silicon and Software Systems and

Allied Irish Bank.
(formerly
Cunav
Assessment models used include SPICE (ISO/IEC TR
15504) [1] and Software Engineering Institute’s CMM 1 [2]
(one organisation also achieved ISO9001 certification).
Keywords
Software Process Improvement, CMM, SPICE
"Permission to make digital or hard copies of part or all of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for
components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, to republish, to
post on servers or to redistribute to lists, requires prior specific permission
and/or a fee."
1
Capability Maturity Model is a service mark of Carnegie
Mellon University and CMM is registered in the U.S.
patent and trademark office.
 2000 ACM
1 SPI – THE EUROPEAN PERSPECTIVE
In Europe there has been quite a significant interest in and
uptake of Software Process Improvement. Historically, the
quality management system approach using ISO9001
certification was the approach of choice for many
companies in Europe. Schemes such as the UK TickIT
scheme provided additional software emphasis to the
certification. However, there was a growing interest in
incremental software process improvement based at least in
part on the significant uptake of the Capability Maturity
Model [2] in the U.S.
Unlike the U.S. strategy of providing funding for the
development of tools to support SPI (i.e. the CMM and
associated deliverables from the SEI), the European
Commission chose a different strategy to stimulate and
support the adoption of SPI. They provided funding
directly to companies under the European Software
Systems Initiative (ESSI) scheme. ESSI funded many
programmes including, for example,

awareness and training actions,

direct funding to over 200 SMEs (small and medium
enterprises) to support Process Improvement
Experiments (PIEs – see [3] for the PIE case study
repository).

dissemination actions
The focus was on subsidising organisations to adopt best
practices with the hope that this would stimulate further
improvements in those organisations and more widely in
the software industry. However, the current status of SPI in
Europe is that although the ESSI initiative achieved a great
deal and SPI is coming more into the mainstream, the
breadth of uptake of SPI across organisations is still
somewhat fragmented. Local content at European SPI
conferences (e.g. European SEPG, EuroSPI) is now of a
very high quality but SPI has not yet become ingrained into
the culture of the software industry in the manner that it
now is in the U.S.
Awareness of the SEI CMM is however high in Europe and
there are many organisations who use it as a ‘toolbox’ for
improvement rather than using it as the basis for a formal
process improvement programme with the associated
formal CMM assessment. Indeed, there are a number of
factors that may indicate an increasing widespread adoption
of CMM…

availability of experienced SPI personnel from larger
CMM-based organisations moving to other
organisations and facilitating effective practical
improvements using the CMM

increasing availability of CMM lead assessors (both in
the SEI’s assessment method, CBA-IPI, and also in
the SEI-accredited assessment method from Compita
Ltd., PPA for CMM)

more experience and data on benefits achieved from
CMM improvement programmes resulting in a market
driven CMM emphasis based on expected competitive
advantage on one hand and it being a required supplier
certification on the other
SPICE (15504) is expected to become an ISO standard and
should be of interest to those who want to either focus on a
few processes to improve or those who want to widen the
scope of their improvement program beyond software
development (which is the focus of CMM). However, the
simple benchmarking levels in the CMM will remain
attractive to management in many organisations.
ISO9000:2000 will contain added focus on process
improvement which may help address the somewhat
declining interest in the standard.
2 CASE STUDIES – BACKGROUND
The intent here is not to provide an exhaustive treatment of
the experiences of the four organisations. Instead a number
of interesting themes of software process improvement will
be discussed by using the experiences of one or more of the
organisations involved.
NewWorld Commerce (formerly Cunav Technologies):
This is a software systems development and consulting
company, which provides IT resources and solutions to
customers operating in a variety of application areas, with a
focus on web-based development. NewWorld Commerce
had approximately 20 staff when they participated in the
EU funded SPIRE project [4] that supported focused
process improvement projects in small organisations. This
case study relates to their experience with SPIRE in 1998
and their experiences since that date.
Motorola Cork
Motorola established the software centre in Cork in 1990 to
develop analog switching software and GSM
telecommunication systems. There are now well over 400
staff involved in software development. The organisation
has had a strong software process improvement programme
in place since 1993 (see [5]).
Silicon and Software Systems:
Silicon & Software Systems (S3) have been providing
design services in silicon, software and hardware design
since it was established in 1986. Within the software
division of approximately 100 software engineers (now 150
software
engineers),
application
areas
include
telecommunications, consumer electronics, Internet and
digital broadcasting. The company embarked on an
improvement program in 1994 (see [6]).
AIB Bank I.T. Department:
Allied Irish Bank (AIB) has an IT department comprising
about 300 personnel providing IT systems and services to
the rest of the bank. They embarked on an improvement
programme in 1998.
3
STARTING
POSITION
AND
BUSINESS
DRIVERS
NewWorld Commerce performed a review of previous
project post-mortems and found some difficulties with
managing customer expectations on some projects and
excessive amounts of rework due to misunderstanding of
initial requirements. Meeting real customer needs and
improving project estimates and visibility with the
customer were to be the key drivers for the software
process improvements. As a small organisation,
improvements needed to minimise impact on resources and
yet maximise the return on any investments by aligning
them with the key business drivers.
Motorola develop systems with high reliability and
availability requirements. Given the system requirements
and the competitive marketplace, errors and downtime must
be kept to a minimum. The telecommunications technology
area is constantly evolving, with new products and services
continually offered. Time to market is also therefore a key
consideration. Motorola’s development process is explicitly
documented and is evolving as the company follows its
program for continuous improvement, which it expects will
ultimately lead to an improved CMM rating (Motorola
places much emphasis on satisfying the concepts of the
CMM).
S3 had a large amount of documentation in their quality
management system that required streamlining and a reevaluation as to the extent that this actually helped people
do their jobs. From a business point of view S3 initially
stated time to market as the key business driver for the SPI
programme. However, it became apparent that the first goal
should really be to measure their time to market. This
change coincided with an evolving change in mindset re
using measurement to guide their improvements. It could
also of course not be ignored that achieving a level on the
CMM has clear commercial/marketing value for S3
especially since customers were asking for their CMM
rating. However, it was made clear that improvement was
the goal, not certification to a level on the CMM for its own
sake. This distinction proved to be highly significant and
beneficial to the entire improvement program.
AIB recognised the importance of managing all changes
(including IT changes) as business changes. IT and the
business personnel identified a number of key
principles/drivers that were to shape the improvement
effort:

IT and the Business were to work together in
partnership

The quality of the product needed to be built-in during
development

Speed to market is a key business driver but it must not
compromise the quality of the product and the ability
to change into the future
These were combined with specific goals and concerns
from the various stakeholders of the improvement
programme (including IT staff themselves) to provide
direction and focus for the improvement efforts.
4 THE ROLE OF THE SPONSOR
The role of the sponsor was of course crucial in each
organisation. Without management commitment to the SPI
programme the chances of failure are high. Commitment
does not simply mean giving approval. It means providing
direction, having a good understanding of what is being
undertaken and why, providing visible active support and
encouragement.
Motorola’s management commitment and involvement was
demonstrated very well to me on one occasion when we
had improved the inspection process with the involvement
of a significant number of engineering staff. A key concern
raised by the group was whether inspectors would have
sufficient time to check adequately for faults given the
pressures of project deadlines. The department managers
responded by e-mailing all staff that they should notify the
managers personally if they found themselves in an
inspection where this occurred.
Interestingly, AIB’s improvement team initially
underestimated the real extent of management commitment
that existed. The lesson they learnt was to test the level of
commitment early. That way when the commitment is
forthcoming it ‘kick-starts’ the improvement programme
with the knowledge that it will be supported by
management if/when difficulties arise. (In this case the
commitment was very strong. However, if the commitment
is not forthcoming it raises the issue early so it can be dealt
with as appropriate i.e. halt the programme or work on
gaining the commitment!). Similarly maintaining a strong
level of management commitment throughout
improvement programme is crucial to its success.
the
5 ASSESSMENT APPROACH
NewWorld Commerce participated in the EU funded
SPIRE project that provided nominal funding for a focused
six month process improvement pilot. An SPI mentor was
provided to support the organisation during the pilot. The
first task of the mentor was to help identify the process area
to improve based on business drivers and the results of a
facilitated SPICE self-assessment. This facilitated selfassessment involved using an assessment tool to gather data
on their processes based on round table discussions with
key personnel. It also involved an SPI questionnaire to
gather, from a wider group, the attitude towards SPI before
and after the improvement pilot. The self-assessment took
one day and had minimal impact on resources. It was by no
means comprehensive but gave a reasonable indication of
strengths and weaknesses for the purpose of the pilot.
Motorola used SEI CMM assessments and its own
extension of the assessment approach accredited by the
SEI. Between these external assessments, they also
performed internal (local) assessments. Interestingly, these
local assessments involved comparing the results from the
local team’s assessment with the results determined by each
process area team’s assessment of their own process area.
This helped ensure a consistent understanding of CMM
requirements.
S3 initially achieved ISO9001certification and then used
two approaches to CMM assessment. The initial approach
used on two occasions was effectively a facilitated selfassessment. This ‘interim maturity evaluation’ used a
questionnaire and consisted of discussions based on the
questionnaire and guided by an external consultant. The
training value of these sessions was also described as
excellent. An external assessment based on the Process
Professional Assessor for CMM assessment method was
then performed (this is similar to the SEI’s CBA-IPI
assessment method)
AIB performed a business review of IT competencies
which was a high level assessment of the IT department as
a whole rather than a process focused assessment. It then
held a number of internal workshops to identify process
and organisational issues/problems as perceived by IT staff.
Each workshop had a process-related theme or area for
discussion that had been identified as a ‘hot spot’ - this
served to direct the discussions. The idea was to identify
the ‘pain’ from the bottom up and correlate this with the
key business drivers as identified by the business review.
Additionally, the CMM was used as a toolbox or reference
point to raise questions on the strategy that was planned for
improvements. However, this improvement program did
not follow a formal assessment approach or improvement
model.
6 IMPROVEMENT STRATEGY
AIB are have been so successful in moving new projects to
a Rapid Application Development approach (i.e. using a
customised DSDM framework) that they are now using the
principles of DSDM on their own improvement project and
finding it very useful. These include timeboxing
improvement activities (typically one month timeboxes),
prioritising SPI requirements and actions within each
timebox, using facilitation techniques for SPI workshops,
etc.
7 PROGRAMME MANAGEMENT
All four companies followed some form of improvement
lifecycle based on a Plan, Do, Check, Act cycle.
Motorola used a variation on the IDEAL2 improvement
model from the SEI. This is a recommended set of steps to
be followed in any improvement programme.
8 INFRASTRUCTURE
S3 managed to reduce their average time to evaluate and
introduce a process change from 533 days prior to the SPI
project down to 112 days now using managed pilot
projects. A significant factor contributing to this
improvement was an infrastructure change (see figure 1.)
decided to run with were injected directly into their
projects where they could assess and monitor the effect of
the changes.
9 TRAINING
S3 used a practical approach to training. Initially a oneday
overview of the updated quality system was provided
which also covered the rationale for changes. This involved
a lot of discussion and improvement suggestions were fed
back into the system. Process mentors were assigned to
each project to support the introduction of new processes.
A series of software project management workshops were
held that proved highly effective – so much so that it is now
done for every new project with a facilitator from the
process group. Two project teams participated in each
workshop. An introduction to the project was given by the
project leader. This was followed by an introduction from
the process mentor on the processes that needed to be
used/planned (i.e. CMM level 2 processes such as
requirements management, project planning itself,
configuration management, etc.). Each team then spent the
majority of the workshop developing the project plan based
on this information. Roles such as project leader were
allocated and rotated. This approach introduced people to
the quality system, gave the team an understanding of the
project leader role and how they could support him/her,
was useful as a team building exercise and new projects
were doing real work by planning their project!
In terms of external support, S3 used three different
consultants during the project that proved useful in
providing different perspectives.
Figure 1. S3’s
Infrastructure
Software
Process
Improvement
The Process Management Group consisted of the SPI
programme manager, project leaders and key process area
team leaders (called task forces in S3 – not depicted in
figure 1). The infrastructure change was in response to rollout difficulties (slow, dissemination of process changes not
effective enough, etc.). The key addition was a divisionspecific project leader process forum. These were key
people who are more likely to realise the effect on their
projects of any process changes. Improvements that they
2
IDEAL is a service mark of Carnegie Mellon University
Motorola recognise the importance of training and
development of a capable workforce. This also helps attract
and retain staff in the current climate of high attrition rates.
There is a policy of having 40 hours training for each
member of staff in Motorola. Motorola made good use of
customised training workshops covering a range of topics
from CMM overviews to testing and inspections and
addressing best practice and the relationship to in-house
processes. A key element was the use of in-house projects
and documentation for exercises and practicals during
training to ensure the training material was related to each
participant’s own situation.
AIB also used customised training (including AIB’s flavour
of DSDM, facilitation, Test Management, Peer reviews,
etc.) and often held follow-up sessions with the trainer
three months after the initial workshop to follow-up on the
implementation of the training material thereby enhancing
the effectiveness of training.
10 CULTURAL/PEOPLE ISSUES
Winning the hearts and minds of people is crucial to a
successful process improvement programme. S3 noted the
value of informal communication, especially early in the
SPI project, in achieving this – i.e. selling SPI at an
individual level. The SPI programme manager coined a
key approach used for this… TSDM – the Tea Station
Dissemination Mechanism! The key issue, though, in S3’s
change management and winning people over was having a
driving philosophy of achieving real improvement and
helping people do their jobs better rather than seeking
compliance to a model. Involvement is another key issue –
about 70-80% of staff were involved in some manner in the
SPI project (e.g. from active involvement in a task force to
just acting as a reviewer of proposed changes)
AIB have found that having a strong project focus with
direct hands-on support and mentoring of projects by the
improvement team has been very effective at enabling
effective change. Providing improvement personnel to act
as facilitators and mentors on the projects has been the key
to this partnership and to viewing process improvement and
best practices as a supportive force for the project team.
Another key issue has been driving improvements, as least
in part, from a bottom up approach whereby improvements
are addressing the issues and pain of staff at all levels. At
the same time identifying all the stakeholders in the
improvement project and ensuring everyone achieves some
benefit and has their key needs met greatly increases your
chances of success.
One lesson learnt from earlier improvement actions in AIB
is the importance of feedback – the loop needs to be closed
with people who helped initiate improvements so they see
the fruit of their work and have visibility on progress.
Motorola experience is that process ownership and
development are best placed with those closest to the
process. Empowering staff to define and tailor processes is
key to achieving ownership. An example of this was the
approach adopted to improving their inspection process [7].
The improvements were defined and agreed in a one-day
workshop involving about 35 senior engineers from all
departments. This resulted in a great deal of buy-in for the
changes.
11 RESULTS OBTAINED
NewWorld Commerce firstly achieved immediate benefits
from the pilot projects with the requirements process
improvements. This was indicated from data they collected
for requirements-related rework and time/budget estimates.
However, customers also provided direct feedback on how
impressed they were with improved ability to deliver what
they wanted. Similarly, the improvements in the project
planning and tracking processes (especially in relation to
risk management and estimation) resulted in a significant
high risk project being delivered successfully.
Motorola achieved level 4 on the CMM scale in 1997. This
contributed to the investment in the Cork development
group and to its ability to perform highly despite significant
and rapid growth.
S3 achieved level 2 on the CMM scale (with a number of
level 3 Key Process Areas also satisfied) in 1999. A
number of customers (including a CMM level 4
organisation) have performed audits/evaluations on S3
recently and the results have been very positive and have
validated the internal improvements undertaken. S3 have
even found themselves in the position of being asked by
customers to provide advice to them on key process areas
that have been seen to be highly effective! From an internal
point of view, a significant change in attitude towards SPI
has resulted in S3 staff raising numerous improvement
requests – they now see what is possible and realise they
have the power to make change and improvement happen.
AIB have performed a number of benchmarking exercises
including determining productivity (function points/week),
delivery rates (time taken from initial request to delivery to
production/function point) and quality (defects/function
point). In the words of the SPI programme manager ‘ there
has been a huge improvement in each of these three
measures’. As significant is the improved way in which
people are working together and co-operating.
12 LESSONS LEARNT
S3 believe that the philosophy they adopted in relation to
SPI was crucial to their success. They stressed
improvement not CMM compliance. The SPI programme
manager is convinced they would not have achieved level 2
(and done so well at level 3) if the goal had been level 2
compliance! Indeed although CMM is a good model, it is
just a model and its scope may not be wide enough for a
given organisation’s purpose. Another key lesson was the
importance of transitioning improvements through a
division/department-specific project leader forum to
accelerate the rollout of improvements.
Motorola are providing greater emphasis now on using SPI
more directly to support the business objectives. In this way
even greater gains can be achieved from the use of SPI and
leveraging off the progress made in attaining CMM level 4
certification. In terms of improving visibility and tracking
progress one must view process improvement as a project –
this means applying more structured planning and tracking
mechanisms to the SPI project itself in a manner similar to
product development projects. Another lesson learnt was
the importance of goal-driven metrics and aligning this to
business goals.
ACKNOWLEDGEMENTS
The author would like to acknowledge the kind support of
each of the organisations involved in the case studies. The
contributions of the following for their help in the
preparation of this paper are particularly noted: Bill
Culleton from S3, Hugh Ivory from AIB, Tom O’Kane
from Motorola and John McEvoy from NewWorld
Commerce. Thanks also to a number of people who
contributed to the European perspective on SPI: Colin
Tully of European Software Process Improvement
foundation (ESPI), Tony Elkington of ESPI and Robert
Cochran of the Centre for Software Engineering.
REFERENCES
1. Rout, ‘SPICE: A framework for Software Process
Assessment, Software Process Improvement and
Practice’ August 1995, pp 56-66, ISSN:1077-4866
2.
Paulk et. al., ‘Capability Maturity Model for Software,
Version 1.1’, Report CMU/SEI-93-TR-24, Software
Engineering Institute, Pittsburg, 1991.
3.
VASIE Best Practice Repository, The European
Comission, DG III Industry, EPSRIT program
<http://www.esi.es/VASIE> This contains ESSI
programme results i.e. information on improvement
experiments performed in the European software
industry, the results achieved and lessons learnt.
4.
SPIRE project and handbook, The
Commission,
ESPRIT/ESSI
http://www.cse.dcu.ie/spire
5.
Fitzgerald and O’Kane, A longitudinal study of
Software Process Improvement, IEEE Software,
May/June 1999, pp 37-45
6.
Kelly and Culleton, Process Improvement for Small
Organisations, IEEE Computer, October 1999, pp 4147
7.
O’Hara, O’Kane and Smith, ‘Contradictions within an
evolving inspection process’, EuroSTAR conference,
Barcelona, 1999
European
23873,