Matakuliah
Tahun
: A0304 - Evaluasi Efektivitas Sistem Informasi
: 2006
Why evaluate information technology
investments?
Pertemuan 03-04
1
Introduction
• Highest ongoing capital expenditure but
economically justifiable
• Need to prioritize different IT projects
• Formal evaluation process is a learning
experience.
Ex
:
for
computer
(requirement, budget, expertise)
• Judgments only based on perceptions
2
Different approaches to evaluation
1. Ex-ante and ex-post evaluation
• Ex-ante : Evaluations to forecast and
evaluate the impact of future situations.
Example : financial estimate
Purpose : to support IT investment justification
• Ex-post : Assess the value of existing
situations
Purpose : to assess the value of IT investment
and support operational decisions
3
Different approaches to evaluation
2. Formative and summative evaluation
Formative : improving program performance, take
place while program is in operation, rely to
large extent of qualitative data and responsive
to the needs of program owner and operators.
Summative : monitoring the process and products
of system, gathering user feedback for further
development, assessing the effectiveness of
the system and overall performance
4
Establish the costs
• It can be difficult to allocate costs to specific
projects when a product will be utilized
throughout organization
• Investment cost sometimes escalates
during the development/implementation
phase
• Ongoing cost can be difficult to estimate
due to the fact that lifespan of IT investment
is unknown
5
Problems of benefit evaluation
• Types of benefit that a firm can expect from
its application (range, forms, task,
understanding)
• Hard to link the intangible benefit to
profitability
• Perform IT benefit evaluation as part of
business process. Ex : JIT
6
Estimating Benefits
• Estimating Benefits presents several challenges.
– In a Displacement Model, the costs being replaced,
are known.
– They are real and can easily be quantified.
• If the system will replace 10 workers in the Paint
booth at $18.00, then you will save $180 a day in
direct costs.
• Indirect costs would include, Fringe, G&A,
benefits, etc.
• These costs become savings or benefits to the
corporation.
7
Estimating Benefits
– In a Cost Avoidance Model, the costs being
avoided, can usually be estimated with a high
degree of certainty.
– If a system is being deployed to avoid hiring
additional staff, then the cost of the staff can also be
estimated with a good confidence level.
– There is additional risk with this model in that the
costs are probably accurate, but there is no
quantifiable data on which to base the estimate.
• As with the displacement model, these costs also
become savings or benefits to the corporation.
8
Estimating Benefits
• The most difficult benefit to accurately estimate is
those resulting from an IT Investment that is either a
new solution or product unto itself.
– Examples of this are a “.com”, a B2B or B2C portal
type of solution.
• The problems, except for those that have done exactly
the same thing previously, are as follows:
– If a similar solution has been deployed by another
company, they won’t tell you how they made it work
and how many benefits that are receiving, much
less quantify it. You don’t really know what will
9
happen.
Estimating Benefits
• Use data from similar sources or projects.
– It may be appropriate to retain some consultants
who have done this type of project before.
– Consultants can often provide guidance that would
not be obtainable elsewhere.
– Be cautious in the selection process. They may not
be as good as they say that they are.
• Quantify as much as possible.
– Support it with extensive market research.
– Ask for justification of each estimate.
– Use a reasonableness test. Does it make sense.
10
Estimating Benefits
• Backup estimates.
– The company that was going to sell $65,000 worth of Hot
Wheels a month was just plain wrong. It isn’t going to
happen.
– Question all the estimates that you receive.
• Use benefit numbers that are very conservative.
– If you are wrong and things are better, great. If they are
worse, you’ll know sooner.
• Make sure that the benefits projected identify any
associated costs that would offset the benefits.
11
Estimating Benefits
• Each solution and the firms that is deploying it at
any specific time will receive different benefits.
• Be as complete as you can in your analysis.
• Apply financial principles to the benefits estimates
as you would to the cost estimates.
– Both change over time.
– Look outside the solution that is being estimated. Risk
may be external to the solution.
• Regulations, conflicts and changing demographics all
impact costs and benefits. Consider everything that
affects the company in your analysis.
12
Estimating Benefits
• Estimating intangible benefits, uses one of
two approaches.
– Approach One: Negotiation with the operating
units to assess the value of the intangible.
• This is usually done by asking what is the value to the
benefit to stakeholders. Answers will vary but ask
enough and a common value can be obtained.
– Approach Two is Imputation.
• This is the practice of 'filling in' missing data with
plausible values and applying a test of the value.
13
Estimating Benefits
• In estimating intangible benefits, consider
that the very nature of this type of benefit.
– Through real, it is difficult to apply quantifiable
values to it.
– The values applied can all ways be questioned
and challenged.
• Realizing these difficulties, many firms as
part of their estimating methodology,
consciously choose to exclude intangible
benefit valuations from their calculations.
14