Information Systems and
Organisations
Lecture 5:
Evaluating IS
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.2
Scope and Coverage
This topic will cover:
• Tangible and Intangible benefits
• Formal-rational evaluation
• Wider evaluation approaches
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.3
Learning Outcomes
By the end of this topic students will be able to:
• Understand typical tangible and intangible benefits
of IS
• Describe a variety of approaches to IS evaluation
• Critically analyse evaluation approaches
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.4
Contents
•
•
•
•
Tangible and intangible benefits
Relevance of cost benefit measures to evaluating IS
Aspects of evaluation
Interaction of evaluation and organisational contexts
• Emphasis – Technology and Organisations
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.5
Costs of IS (Tangible)
• Costs of the technology:
- Capital cost of development or purchase
- Cost of implementation (project costs)
• Costs of the resources to run it:
-
V1.0
Hardware
Network
Software
Service and support
© NCC Education Limited
Evaluating IS Lecture 5 - 5.6
Benefits of IS (Tangible)
• Benefits (related to IS/IT)
- Reduce fixed costs of IS/IT service
- Reduce management overhead
• Benefits to the organisation
- Increased efficiency in processes (automation)
- Increased quality with reduced costs
- Increased sales (activity) per unit of resource
• (essentially PRODUCTIVITY benefits)
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.7
Fixed and Variable Cost
In general, managers may
prefer VARIABLE costs in many
types of operations because
they provide a built in method of
reducing costs if activity falls
and coping with higher activity if
it should rise.
IS and IT can help reduce
FIXED COSTS and in some
cases, through appropriate
process changes, replace them
with VARIABLE COSTS.
Meaning that the organisation is
viable at lower levels of
activity..e.g. as in diagram.
V1.0
Reducing fixed costs
will lower the breakeven
point
© NCC Education Limited
Evaluating IS Lecture 5 - 5.8
IS Driving Efficiency
• Some years ago, the following estimates of banking
transaction costs were made
-
Traditional branch based transaction $1.07
Telephone transaction $0.54
ATM transaction $0.27
Internet (online)transaction $0.13
• In fact online transactions have sometimes been
estimated as low a $0.01 – the exact figure will vary
depending in complexity of the transaction
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.9
ERP Benefits
Firm Infrastructure
Human Resource Management
55%
Technology Development
Procurement
Inbound
Logistics
Operations
Outbound
Logistics
13%
Marketing
and Sales
Service
32%
ERP Benefits % - Source AMR Research April 2004
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.10
IS Effect is Very Variable
• There is no simple relationship between IS/IT
investment and productivity
• In some cases (e.g. healthcare, social work)
productivity is difficult to measure in any event
• In some cases (e.g. electronic design, retailing)
measuring productivity is easier but outcomes can
be difficult to interpret
• A late 90’s study of banks in the USA documented a
large variability in return on IT investments
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.11
IT Spending and Profitability
24%
Wells Fargo
Return on Equity
First Tennessee
Nations Bank Services
Northern Trust
First Chicago
12%
Bank of America
Fleet Financial
6% $1000
$10,000
$100,000
1995 Information Technology Spending per employee
(Banking Sector)
Source: Strassman (1997)
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.12
Costs of IS (Intangible)
• Opportunity costs?
- Should we be doing something else?
- Overhead of running IS/IT
• Management
- Management time and attention
- Governance
• Risk
- Obsolescence not predictable
- ‘Game changing’ innovations
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.13
Benefits of IS (Intangible)
• Benefits to the organisation
-
Improved communications
Staff morale
Reputation (branding benefits)
Increased customer satisfaction (branding..)
Flexibility and agility
Organisational learning....
• Intangibles – implicitly ‘un-measurable’
• A management judgement is needed to weigh
intangible benefits
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.14
Intangible and Tangible Benefits
• Should we ensure that there are always significant
tangible benefits when we make an investment
decision?
• Should we make investment decisions with no way
of measuring potential benefits ?
• Are intangible benefits simply benefits that we
cannot yet measure ?
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.15
... Should We Even Worry ?
• Historical project experience shows that significant
numbers of organisations often never seek to
seriously measure benefits following
implementation of a new IS
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.16
Evaluation Perspectives
• Formal and rationale all very well – but:
• What can be evaluated ?
• Who is evaluating ?
• What is the purpose of the evaluation ?
• Success results from a combination of aspects
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.17
Main Dimensions of IS Success
Development and Process
Process
Costs
Use Process
User
Information
Satisfaction
SUCCESS
Quality of the IS Product
Product
Benefits
Impact of IS on Organization
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.18
Who Evaluates?
•
•
•
•
•
•
V1.0
Business managers
IS/IT managers
IS/IT users
External consultants
External authorities (regulatory bodies perhaps)
Auditors
© NCC Education Limited
Evaluating IS Lecture 5 - 5.19
Purpose of Evaluation
• Perspectives and Aims
-
To measure tangible costs and benefits
To endorse success
To investigate failure
To identify good practice
To improve implementation process
• Overt and covert motivations may exist
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.20
IS Success Model - Research
• DeLone and Mclean - a large meta study (1992 and
2003)
• What factors lead to IS success ?
• What is the relationship between these ?
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.21
DeLone McLean Model
System
Quality
Intention
to Use
Use
Information
Quality
Service
Quality
V1.0
Net Benefits
User
satisfaction
© NCC Education Limited
Evaluating IS Lecture 5 - 5.22
References
• Boddy, D., Boonstra ,A., Kennedy, G. (2008)
Managing Information Systems : strategy and
organisation 3rd ed. FT Pearson. ISBN-13: 9780273 -71681-5XXX
V1.0
© NCC Education Limited
Evaluating IS Lecture 5 - 5.23
Lecture 5 – Evaluating IS
Any Questions?
V1.0
© NCC Education Limited