EXPERIENCES FROM USING ICMM IN INSPECTION PROCESS
ASSESMENT
Sami Kollanus
Department of Computer Science and Information Systems
P.O.Box 35 (Agora), FI-40014 University of Jyväskylä
Finland
sami.kollanus@jyu.fi
KEY WORDS
Software inspections, maturity model, software process
improvement, case studies
EXTENDED ABSTRACT
Michael Fagan [1] published his first experiences with
software inspections in IBM over 30 years ago. Over the
years inspections have been acknowledged as an
important part of software development practices. Several
studies have reported remarkable savings when using
inspections systematically to find defects as early as
possible (e.g. [2][3][4]).
Basically, usefulness of inspections is widely known in
industry. However, it is also well know there is a huge
gap between this knowledge and the real state of the
practice. There is very little systematic research made
about the state of the art, but a couple of studies give
some ideas about it. Ciolkowski et al. [5] found that about
40 % of the respondents of their survey regularly review
requirements, which is the most typical document type to
review. Johnson [6] reported that 80 % of the respondents
in his small survey use inspection irregularly or not at all.
Also our previous work supports these results in the small
sample of six organizations [7]. In addition, we found that
there may be significant weaknesses in the process even
when inspections are regularly practiced. The gap
between the knowledge and practice brings out a need to
better understand deployment and improvement of
inspection practices.
CMM/CMMI [8] is the best known and the most
common reference model, which supports assessment and
improvement of the whole set of software processes. The
traditional stage model has been experienced as a good
path to process improvement (e.g. [9]). However, the
model itself is too general when specific support is needed
in improving individual process areas. Based on this
limitation, several CMM kind of maturity models have
been created for individual software process areas like
testing [10], project management [11] and maintenance
[12].
Inspection Capability Maturity Model (ICMM) [13] is
a maturity model for software inspection practices. It aims
to support assessment and improvement of inspections
practices in a software organization. It includes five
stages, from initial to optimizing, which follow the idea of
the corresponding stages in CMMI. Each maturity level
includes some key process areas and all the defined
requirements have to be satisfied to reach the level.
The first maturity level is the first (Initial) level, which
doesn’t have any requirements. The second (Practising)
level requires an organization to simply practise
inspections regularly. The third (Defined) level requires
an organization to have well defined inspection process.
The focus on the fourth (Managed) level is on inspection
process improvement and defect prevention instead of
only finding and fixing defects. The fifth (Optimizing)
level is focused on continuous improvement of costeffectiveness.
In the first phase of our research, the original version
of ICMM was created based on a literature review to be
evaluated and improved with empirical study in industry.
In the next phase, the model was used to assess inspection
practices in eight case organizations in Finland. This
paper discusses these first experiences from using ICMM
in practice. It analyses usefulness of the model structure
and the specific requirements in the model. Additionally,
some improvements are suggested to the model.
The case studies were conducted interviewing software
professionals in eight software supplier units within seven
Finnish companies. These companies produce and tailor
software products for their customer organizations. The
sample represented different kind of units. The case
studies included three interviews in six unit and two
interviews if the two smallest organizations. The total
number of interviewees was 22. All the interviewees were
experts (quality managers, project managers and software
developers) with average 11 years of SE-experience.
The interviews had two main goals related to the
involved organizations: 1) to find out how inspections (or
less formal reviews) are practiced, and 2) to find out what
are the faced inspection related problems. In the
beginning, the interviewees were asked to estimate in
scale from 1 to 5 (5 is the best), the quality of the applied
inspection practices in their unit. First main part of the
interviews charted the currently applied inspection
practices. This part was mainly based on ICMM. Another
main part focused on the experienced inspection
problems.
In our earlier work [7][14], we have already discussed
the problems and weaknesses of the current practices. In
this paper we focus on evaluating ICMM model, which
was used in assessing the current practices in the case
organizations. Our results include the findings presented
in the next couple paragraphs.
Originally ICMM aimed to provide support for
organizational assessment and improvement of inspection
practices. In the case organizations, ICMM work well as a
framework in assessing the current practices and it helped
to identify the weaknesses. On the other hand, the model
did not distinguish the organizations as well as we
expected. Regularly, it appears be hard to design a model,
which is good both in improvement and assessment
purposes. In the future it might be an option to focus on
improving ICMM to provide better support for SPI.
All the assessed case organizations were on the first
ICMM level, but many of them also fulfilled several third
level requirements. The higher levels were not currently
relevant for the organizations. So, we could analyze only
usefulness of the second and the third level requirements.
Most of them appeared to be well defined and some
lacking practices had clear connection with faced
problems. For example, no one of the organizations had
arranged inspection training, although several faced
inspection problems could be treated by improving
general knowledge.
References:
[1] M.E. Fagan, Design and code inspection to reduce
errors in program development, IBM Systems Journal,
15(3), 1976, 182-211.
[2] G.W. Russell, Experience with inspection in ultra
large-scale developments, IEEE Software, 8(1), 1991, 2531.
[3] E. Doolan, Experience with Fagan's inspection
method, Software - Practice and Experience, 22(2), 1992,
173-182.
[4] R.B. Grady & T. Van Slack, Key lessons in achieving
widespread inspection use, IEEE Software, 11(4), 1994,
46-57.
[5] M. Ciolkowski, O. Laitenberger & S. Biffl. Software
reviews, the state of the practice, IEEE Software, 20(6),
2003, 46-51.
[6]
P.M.
Johnson,
Reengineering
inspection.
Communications of the ACM 41(2), 1998, 49-52.
[7] S. Kollanus & J. Koskinen, Software Inspections in
Practice: Six Case Studies. Proc. of the PROFES 2006
Conference, Amsterdam, June 12-14. 2006, Springer
LNCS 4034, 377-382.
[8] SEI, Capability Maturity Model Integration version
1.2, Software Engineering Institute, 2002. (URL:
http://www.sei.cmu.edu/cmmi/)
[9] F. MacGarry & B. Decker 2002. Attaining Level 5 in
CMM Process Maturity. IEEE Software 19(6), 87-96.
[10] I. Burnstein, A. Homeyen, T. Suwanassart, G.
Saxena & R. Grom, A testing maturity model for software
test process assessment and improvement, Software
Quality Professional, 1(4), 1999, 8-21.
[11] Kerzner, H. 2002. Strategic Planning for Project
Management Using a Project Management Maturity
Model, John Wiley & Sons.
[12] April, A., Abran, A. & Dumke, R. 2004. SMcmm
Model to Evaluate and Improve the Quality of the
Software Maintenance Process, Proc of the 8th European
Conference on Software Maintenance and ReEngineering, Tampere, Finland , 2004 , pp. 243-248 .
[13] S. Kollanus, ICMM – Inspection Capability Maturity
Model. Proc. of the IASTED International Conference on
Software Engineering, Innsbruck, Austria, 2005.
[14] Kollanus S. 2005. Issues in software inspection
practices. Proc. of the PROFES 2005 Conference, Oulu,
June 13-18. Springer LNCS 3547, 429-442.