Success and failure factors
in ICT projects
by
Dr. ir. Aart J. van Dijk EMITA RE
9 November 2010
ICT ZORG CONGRES 2010
Engineering Doctorate
Middlesex University London
School of Engineering and Information Sciences
Middlesex University – EngD – Aart van Dijk – November 2010
Middlesex University – EngD – Aart van Dijk – November 2010
Research?
Than you have a question
and you want an answer!
Middlesex University – EngD – Aart van Dijk – November 2010
The Research Question
Objective
(An academic exercise in) finding out
(a contribution to) the true success and
failure factors used in ICT practice
(SUFFIs = SUccess and Failure Factors in ICT projects)
Middlesex University – EngD – Aart van Dijk – November 2010
The Research Question
Definition of the problem
How were the ICT projects the author worked on
managed with regard to success and failure factors?
• the portfolio of projects: ICT projects the author worked on
including IT projects audited by the author
• the key here is the author’s observations and experiences
How do they agree or disagree with
• what others say happens with regard to success and
failure factors and
• the procedures in Professor Abdel-Hamid’s work
on Software Project Management
(reflection analysis of cases / ex post review of cases)
Middlesex University – EngD – Aart van Dijk – November 2010
Relevance
Middlesex University – EngD – Aart van Dijk – November 2010
?
Relevance
One may ask the question, whether it is relevant to look at
success and failure factors in ICT projects
1982 - Professor Jan Oonincx (The Netherlands)
(Why are information systems still failing?)
2002 - John Smith (United Kingdom)
(The 40 root causes of troubled IT projects)
2003 - The American “Standish Group”
(only 34% are successful, 51% does not go according
to plan but ultimately does lead to some result and
15% of the projects fail completely)
….
- A lot of other publications
Middlesex University – EngD – Aart van Dijk – November 2010
Relevance
One may ask the question, whether it is relevant to look at
success and failure factors in ICT projects
September 2010
• Logica verslikt zich in belastingsysteem
(Forse schadepost dreigt voor Waterschapshuis)
• Justitie trekt stekker uit ERP-systeem Cajis
(Budget: 13 miljoen. 12 miljoen is daarvan al is verbruikt)
• Professor Chris Verhoef (VU):
Overheid verspilt nog steeds miljarden door
mislukte ICT-projecten
Conclusion: SUFFIs still are very topical.
Middlesex University – EngD – Aart van Dijk – November 2010
What is understood by a project failure?
We can find different definitions
(Capers Jones, John Smith, Peter Noordam, Darren Dalcher, etc.)
For this thesis a project failure has one or more
of the following characteristics:
• it does not comply with the functionality agreed to in advance
• it exceeds the planned time-scale by more than 50%,
• it exceeds the build cost by more than 50%
====
A successful project satisfies this three factors: it complies with
the functionality agreed to in advance, it is delivered on time and
it is delivered within the agreed budget [Noordam et al. 2007].
Middlesex University – EngD – Aart van Dijk – November 2010
What others say happens
International publications
•
•
•
•
•
•
The 40 root causes of troubled IT projects (John Smith, 2002)
Large Software System Failures and Succ. (Capers Jones, 1996)
Major Causes of Software Project Failures (Lorin May, 1998)
Critical Success Factors In Software Projects (John S. Reel, 1999)
Seven Char. of Dysfunctional Software Projects (Evans et al, 2002)
Critical failure factors in information system proj. (K.T. Yeo, 2002)
• The procedures of Tarek Abdel-Hamid and Stuart Madnick in:
“Software Projects Dynamics – An Integrated Approach” (1991)
Middlesex University – EngD – Aart van Dijk – November 2010
What others say happens
Dutch publications
• Why are information systems still failing?
(Professor Jan Oonincx, 1982)
• Success and failure factors in complex ICT projects
(Nico Beenker, 2004)
• ICT project management on the road to adulthood:
Success factors for ICT projects
(Peter Noordam et al, 2007)
Middlesex University – EngD – Aart van Dijk – November 2010
Macroeconomic Environment
Project Conception
Competitive Environment
(6)
Technological Environment
Organisational Environment
Project Initiation/
Mobilisation
1
Time
6
5
2
Typical
Contracting
Points
(11)
System Design
3
(9)
4
System Operation,
Benefit Delivery,
Stewardship & Disposal
Time
(3)
System Development
(7)
System Implementation
(4)
The Project Life Cycle (John Smith)
Middlesex University – EngD – Aart van Dijk – November 2010
Software project outcomes by size of project
(Capers Jones)
1 FP =
125 C
statements
Probability of Selected Outcomes
Early
On time
Delayed
Cancelled
Sum
1 FP
14.68%
83.16%
1.92%
0.25%
100.00%
10 FP
11.08%
81.25%
5.67%
2.00%
100.00%
100 FP
6.06%
74.77%
11.83%
7.33%
100.00%
1,000 FP
1.24%
60.76%
17.67%
20.33%
100.00%
10,000 FP
0.14%
28.03%
23.83%
48.00%
100.00%
100,000 FP
0.00%
13.67%
21.33%
65.00%
100.00%
Average
5.53%
56.94%
13.71%
23.82%
100.00%
Middlesex University – EngD – Aart van Dijk – November 2010
Professor Jan Oonincx
“…
Information systems, which are set up too ambitiously, too
isolated or without proper planning, stand a very large chance
of failing. Insufficient involvement of future users in the
development of information systems or a passive attitude of the
top management also often lead to disappointing results.
…
25 August 1982 -
ir. Aart J. van Dijk”
Middlesex University – EngD – Aart van Dijk – November 2010
Some Success / Failure factors
Author
No
Description
PN
04
Unfamiliarity with scope and complexity
(Peter van Noordam, 2007)
PN
06
The use of a business case results in a higher degree of
satisfaction with the project, whilst the satisfaction with
the project is very low when no business case is used
PN
10
Technical knowledge is certainly an important skill for
project managers to have
JRR
02
The designer designs and not the method
(Jaap van Rees, 1982)
TG
01
Don’t believe blindly in any one method; use your
methods and common sense to measure the reality
against your needs (Tom Gilb, 1988)
Middlesex University – EngD – Aart van Dijk – November 2010
Success and failure factors
Input
1
John
Smith
2002
Process
Output
7
Jan
Oonincx
1982
8
Nico
Beenker
2004
9
Peter
Noordam
2007
2
Capers
Jones
1996
3
Michael
Evans
2002
4
K.T. YEO
2002
Eliminating
duplicates
John
Smith
Middlesex University – EngD – Aart van Dijk – November 2010
Others
10
Methods
2007
5
Lorin
May
1998
6
John
Reel
1999
John Smith
40 Root Causes + 7 Public Root Causes
Others
Capers Jones
16
Michael Evans
5
K.T. Yeo
10
Lorin May
4
John Reel
2
Jan Oonincx
8
Nico Beenker
3
Peter Noordam
9
Methods
2
-----------------------------------Total
59
Together: 47 + 59 = 106
Middlesex University – EngD – Aart van Dijk – November 2010
The procedures in Tarek Abdel-Hamid’s work on
Software Project Management:
An Integrated Approach*)
• based on systems thinking / system dynamics
in relation to Project Management
• 20 Chapters (264 pages)
• I found 82 (TAH) SUFFIs
• I studied the book many times
• it was a very heavy job to tease out the TAH SUFFIs
from the text
• I separated the TAH SUFFIs in category A (28) (most important)
and category B (54)
*) Englewood Cliffs, NJ: Prentice Hall, 1991
Middlesex University – EngD – Aart van Dijk – November 2010
Some TAH SUFFIs
• TAH/09: systems complexity grows as the square of
the number of systems elements
• TAH/14: the relationship between cost and system size is not linear.
In fact, cost increases approximately exponentially as size
increases
• TAH/65: different distribution of estimated effort among a project’s
phases creates a different project
• TAH/52: different estimates on a software project create
different projects
Middlesex University – EngD – Aart van Dijk – November 2010
Success/failure factors that are mentioned the “Big Hitters” *)
Success/failure factors that are
mentioned the “Big Hitters”(BH)
C
J
M
E
K
Y
L
M
J
R
J
O
N
B
P
N
Poor project management (BH01)
+
+
+
+
+
+
+
+
+
+
Deadlines are unrealistic (BH02)
+
Poor communication (BH03)
+
+
Incomplete/weak definition
requirements (BH04)
+
+
+
Insufficient involvement of future
users (BH05)
+
+
+
+
+
+
Total
8
4
+
4
+
4
4
*) Big Hitters: are the most important (most common, often mentioned)
success and failure factors.
(John Smith introduced the name Big Hitter [Smith 2001])
Middlesex University – EngD – Aart van Dijk – November 2010
Jan
Oonincx
1982
Nico
Beenker
2004
Peter
Noordam
2007
Jaap van Rees
Tom Gilb
1982/1988
Tarek
Abdel-Hamid
1991
Big
Hitters
(8)
(3)
(9)
(2)
(28)
(5)
John
Smith
2002
Capers
Jones
1996
Michael
Evans
2002
K.T. YEO
2002
Lorin
May
1998
John
Reel
1999
(47)
(16)
(5)
(10)
(4)
(2)
(..) = number of SUFFIs
Total number = 139
SUccess and Failure Factors in ICT projects (SUFFI Chart)
Middlesex University – EngD – Aart van Dijk – November 2010
One leg in theory and one leg in practice
Theory
Denken
Middlesex University – EngD – Aart van Dijk – November 2010
Practice
Doen
Portfolio of projects
9 Projects  related to 12 project-based publications
in Dutch journals*)
4 ICT project audits
•
•
•
•
Case:
Case:
Case:
Case:
Multihouse versus Nutsbedrijven (public utilities) (1997)
SYSA (GOVERN) (2004)
ACCINT (PUBLIC) (2004)
SOX (FINANCE) (2006)
*) of course other author’s projects could have been chosen or added
Middlesex University – EngD – Aart van Dijk – November 2010
Portfolio of projects
9 Projects  related to 12 project-based publications
in Dutch journals
•
•
•
•
•
•
•
•
•
Case:
Case:
Case:
Case:
Case:
Case:
Case:
Case:
Case:
Composition of bibliographies (DUT) (1971)
Traffic Data Collection (DUT) (1975)
Library Book Request system (DUT) (1979)
General Information Retrieval (GIRAF) (DUT) (1984)
Fin. info. system (building/housing) (OKAPI) (UoA) (1994)
Telephony (new PABX and so on) (DUT) (1994)
Charging method (services based) (GAK) (1998)
Interfacing appl. (EAI) (KOLIBRIE) (KPN Telecom) (2001)
RBAC SAP R/3 (POTVIS) (Police Agency) (2004)
Middlesex University – EngD – Aart van Dijk – November 2010
Results Telephony project
Telephony
Score
Complies with functionality
agreed
Yes
On time
Yes
Within the agreed budget
No*)
*) the project did not exceed the build cost by more than 50%
Middlesex University – EngD – Aart van Dijk – November 2010
Results case Multihouse versus Nutsbedrijven
(public utilities) (1997)
Multihouse
Score
Complies with functionality
agreed
No
On time
No
Within the agreed budget
No
Middlesex University – EngD – Aart van Dijk – November 2010
Big Hitters in relation with the discussed cases
Apply to
Results cases
Score
Big
Hitter
1
Big
Hitter
2
Big
Hitter
3
Big
Hitter
4
Big
Hitter
5
Funct.
On
time
Within
Budget
Case 1: POTVIS project (KLPD)
No
No
No
No
No
Yes
Yes
Yes
Case 2: Kolibrie project (KPN Telecom)
No
No
No
No
No
Yes
Yes
Yes
Case 3: Charging method project (GAK)
No
No
No
No
No
Yes
Yes
Yes
Case 4: Telephony project (DUT)
No
No
No
Yes
No
Yes
Yes
No
Case 5: OKAPI project (UoA)
No
No
No
No
No
Yes
Yes
---#
Case 6: GIRAF project (DUT)
No
No
No
No
No
Yes
Yes
---#
Case 7: AUBID project (DUT)
No
No
No
No
No
Yes
Yes
---#
Case 8: VDV project (DUT)
No
No
No
No
No
Yes
Yes
---#
Case 9: BIBLIOSYSTEM project (DUT)
No
No
No
No
No
Yes
Yes
---#
------------------------------------------------------
-------
-------
-------
-------
-------
-------
------
--------
Case 10: Audit Multihouse
Yes
Yes
Yes
Yes
---+
No
No
No
Case 11: Audit SYSA (GOVERN)
Yes
Yes
Yes
Yes
Yes
No
No
No
Case 12: Audit ACCINT (PUBLIC)
Yes
Yes
Yes
Yes
Yes
No
No
No
Case 13: Audit SOX (FINANCE)
No
No
Yes
No
No
Yes
Y/N*
Y/N*
+) unknown
#) no specific budget available
Middlesex University – EngD – Aart van Dijk – November 2010
*) Yes or No, depends on the project
Case “Netherlands Court of Audit”
Report “Lessons from government ICT projects”, November 2007
• I analysed the report several times:
I found 39 success/failure factors
• advisors/experts gave their comments/opinion in different ways
• analysing the comments:
I found 58 remarks/recommendations
Middlesex University – EngD – Aart van Dijk – November 2010
Case “Netherlands Court of Audit”
Report “Lessons from government ICT projects”, November 2007
Conclusion:
• it is possible to apply the SUFFI Chart in the
“Netherlands Court of Audit” case
• based on this case,
the SUFFI Chart does not need to be extended
• SUFFIs are well known but unpopular
Middlesex University – EngD – Aart van Dijk – November 2010
Big Hitter 6:
Lack of senior management involvement and commitment
(Jan Oonincx, John Smith)
Big Hitter 7:
Lack of professionalism (Tarek Abdel-Hamid, Chris Verhoef et al)
EX49: There is a gross lack of professionalism in the world of ICT.
Only a very small section of people have actually qualified
in informatics
EX50: The government should really just work with accredited
information scientists and not with self-educated people
EX52: Universities should train people better in managing and
executing large ICT projects
Middlesex University – EngD – Aart van Dijk – November 2010
7 Big Hitters
•
Poor project management (BH01)
•
•
•
•
Deadlines are unrealistic (BH02)
Poor communication (BH03)
Incomplete/weak definition requirements (BH04)
Insufficient involvement of future users (BH05)
•
Lack of senior management involvement
and commitment (BH06)
Lack of professionalism (BH07)
•
Middlesex University – EngD – Aart van Dijk – November 2010
Conclusion
• The results of the research as represented in this thesis
are partly recorded in the SUFFI Chart above
• Both experienced as well as starting project managers
can reap the immediate benefits (immediately usable)
• Spending a few hours in advance on studying the mapped
SUFFIs will help them avoid a number of pitfalls
• The SUFFI Chart seems to apply many more areas than just
software engineering
Middlesex University – EngD – Aart van Dijk – November 2010
Gouden regels voor opdrachtgeverschap *)
1. Begin met een heldere business case en blijf (aan) deze
business case toetsen
2. Eis productgerichte planning en rapportage
3. Geef stuurgroepleden verantwoordelijkheid voor levering
of gebruik van het resultaat
4. Wees kritisch bij de keus van een projectleider en moedig
hem aan om kritisch te zijn
5. Sta alleen scopewijzigingen toe met autorisatie door de
stuurgroep
*) VROM – Liesbeth Edelbroek
Middlesex University – EngD – Aart van Dijk – November 2010
Questions
Middlesex University – EngD – Aart van Dijk – November 2010
Thesis: 520 pages
Costs: € 78,- incl. mailing costs
ING bank account: 150248
Avédé-Beheer BV
Zoetermeer
The Netherlands
Delivery address!
Middlesex University – EngD – Aart van Dijk – November 2010
Thank you very much
for your attention
and your questions
aart.vandijk@planet.nl
Middlesex University – EngD – Aart van Dijk – November 2010