Management of
Computer System
Performance
Chapter 2
Why Evaluate IT Investment
Why Evaluate IT Investment

Agenda


Chapter 2
Objectives
 Students should be able: to mention the different
approach to evaluation of technology investment.
2
Chapter 1 Summary





Business Exist for one reason, optimizing
shareholders wealth.
IT projects are critical to accomplishing this.
The selection of optimal IT projects are key to
meeting this goal and allowing management to
succeed.
Large, diverse firms may have as many as 8-900 IT
projects going on at any one time and many more
identified as needing funding.
Selection of the optimal product is critical.
3
Chapter 1 Summary

Ideally Benefits should be identified and
quantified before an IT project is
implemented and costs incurred in its
development and/or deployment.


The primary benefits are usually quickly
identified. The secondary and tertiary benefits
make the gross benefit more nebulous to
quantify.
Consistent approaches to valuation are
critical to project selection.
4
Chapter 1 Summary





The concept of IT Value is a combination of
factors.
Benefits are both tangible and intangible.
Benefits are evolutionary.
Cost benefit analysis remains a key tool for
managerial decision making.
Stakeholders must be recognized within the cost/
benefit analysis. Their information is critical.

They are the ones who will receive the benefit and will
pay the cost,
5
Chapter 2: Why Evaluate
Information Technology Investments





The need is based upon the efficient allocation of
capital resources.
IT Investment may equate to as much as 50% of
a firms entire capital budget.
IT investments do not appear over night. They
require time and resources to roll out.
The larger the firm, the more these types of
investments can affect the corporations financial
viability.
To calculate value, several formulas are required.

This is unavoidable.
6
The Evaluation of IT Investment

IT Investment evaluations are driven by several
factors.


The price of IT services and solutions are usually the
most expensive indirect capital investment made by
any firm.
 A loaded Sun E6500, Workgroup Server can top $200K
WITHOUT the application software.
The return on an investment such as this should
be substantial but is often not quantifiable.

How would it be used? What efficiencies could be
expected for the corporations benefit? When would
they be realized? How much will be the return?
7
Approaches to IT
Investment Evaluation

There are several approaches to addressing IT
investment evaluations. These are usually
categorized as either:



Predictive (or ex-ante) evaluations and,
 Predictive is used for pre-investment justification
(includes Proposal supporting data).
Post-implementation (or ex-post) evaluations.
 Post Implementation is used for confirmation, validation
or repudiation of IT investment.
These can be used neutrally or to defend ones
position

i.e. don’t assume that the evaluation will be consistent with
your views.
8
Predictive Approaches to
IT Investment Evaluation

Predictive evaluations use predominately
financially based estimates.


These include single point/element or range
estimates for costs and benefits.
Return is calculated as a forecast for one or
more of the following:




NPV (Net Present Value)
IRR (Internal Rate of Return)
Payback Period
ROI (Return on Investment),
9
Approaches to IT Investment
Evaluation

There are two approaches to IT investment
evaluation.


Predictive comes first, is used for justification and
estimates as many variables as possible to determine
the viability of the project. This usually takes the form
of a cost benefit analysis.
Post implementation involves reviewing actual cost
and benefit data (after all cost and benefit data is
identified) and compares this against the predictive
estimates. This includes work in process evaluations
often referred to as Earned Value calculations.
10
Approaches to IT Investment
Evaluation

Political issues and uses of these
approaches aside, the performance of
both types of evaluations can and should
be done.

Predictive is critical for justification. It is used
not only for internal investment justification but
in the bid and proposal process as well.


This later element assumes that the firm is in the
IT services business.
Post Implementation evaluations should be
done, if for no other reason, than to make the
predictive process better and more accurate.
11
Approaches to IT Investment
Evaluation

A more accurate predictive process as the result
of post implementation evaluations requires that
the predictive process be subjected to error
analysis.



This requires each step of the predictive evaluation
process to be reviewed and its assumptions validated.
This should be done either by individuals or a team
made up of different people other than those who did
the predictive analysis initially.
 Anyone see political issues with this?
Each of the underlying variables that were the basis of
the assumptions that were made in the predictive
evaluation should be reviewed in this process.
12
Types of Analytical
Approaches
Project Life Cycle
700,000
600,000
500,000
Costs and 400,000
Benefits 300,000
200,000
100,000
0
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
Time
Start
Project Life Cycle
Questions:
• What are the costs (purple)?
• What are the benefits (cream)?
• At what level of project success are they realized?
• Do benefits exceed costs and if so, when?
13
Predictive and Historical
Analytical Methodologies

Predictive (ex-ante) and historical (ex-post)
methodologies refer to performing benefits
and costs of a project.


Predictive is, as it implies, a forward estimate
with little or no firm, quantitative or empirical
data on the current project.
Actual cost data is missing.
14
Predictive and Historical
Analytical Methodologies

Ex-post analysis refers to using actual cost
data from an existing project for the
calculations as to the cost and benefit for
that project.

In both cases, the approach is to establish
the costs of a project in the most
quantitative means possible.
15
Predictive Methodologies

Predictive or Ex-ante Methodologies
are commonly used.


These approach the quantification of
benefits and costs using estimates or
historical actuals from other projects.
It requires that executive management
make authorization decisions on those
predictions.
16
Predictive Methodologies

There are risks associated with this approach in
some circumstances. These include items such as:



The historical data was based on projects with dissimilar
traits that can make many elemental difference and,
The basis of estimates as to efficiencies achieved in the
execution of the project are just plain wrong.
Non-tangible benefits will be quantified incorrectly.
 These include gains in efficiency in indirect departments,
etc.
17
Predictive Approaches to
IT Investment Evaluation

The conservative approach towards a predictive
evaluation of an IT Project (Investment) is to
aggressively document the estimated value of
the project.


By documenting your estimates that are used in the
justification process, you are establishing a reference
point that will be used to validate your process in the
future.
Depending on the organization, there could be career
risk in this activity.

Its part of your job but realize that it could come
back an haunt you so make it as accurate as
possible.
18
Ex-post Methodologies

Post-implementation (or ex-post) evaluations
projects use historical data from the existing project
to determine if the project is viable.



The main liability in this approach is that the money is
already spent and the analysis can only tell you what you
missed.
This is important, if not critical, to prevent a repeat of
mistakes.
Remember,
 30% of IT projects never reach fruitful conclusion
 51% exceed budget by 189% and
 deliver only 74% functionality
19
Post-implementation
Approaches to IT Investment Evaluation

This analytical approach can be equally
challenging because the analysts required
to:



quantify estimates that were developed in the
earlier phases of the project as predictive
estimates,
determine if the deliverable product met the
functionality promised in the estimating phase
of the activity,
look for other benefits or costs that were
missed during initial estimating activity.
20
Post-implementation
Approaches to IT Investment Evaluation

In addition to the “newly discovered
costs and benefits”, the Postimplementation process must be
coupled with investment time lines.

In most projects that I have worked, the
time lines change as the project
progresses.

Military axiom: No plan survives intact,
the first contact with the enemy.
21
Post-implementation
Approaches to IT Investment Evaluation

This makes the analyst job more challenging in
that progress and schedules between estimates
must be synchronized as well to acquire actual
costs.

Regardless of the position in relation to the
investment, good accounting and financial
practices require apples to apples
comparisons.

In addition to the quantifiable elements of IT
investment evaluations, there is also a concept of
formative and summative evaluations.
22
Formative and Summative
Evaluation



These attempt to assign value to costs and
benefits of an IT project that are less
quantitatively inclined. It addresses user
satisfaction and stakeholder satisfaction.
These elements are frequently part of the
feed back loop process of IT Project
Management.
These are the approaches to project
valuation.
23
Formative Evaluation


Formative evaluation – Addresses
program performance while program is in
progress.
Views an evaluation internally.


Focuses on feed back iterative approaches.
Relies on qualitative data



Rebuilds and reworks
Focuses on process and product development.
May result in program changes in response to
the needs of the owners and sponsors.

Has the potential for significant scope creep.
24
Summative Evaluation

Summative evaluation – assess the
viability of the results of the program (the
deliverable system) while in process and
addresses:



The impact of the system to the organization.
The usability of the system to the user
community.
The effectiveness in it meeting its
specifications and goals as well as overall
performance.
25
Perspectives of Analytical
Approaches
Project Life Cycle
700,000
600,000
500,000
Costs and 400,000
Benefits 300,000
200,000
100,000
0
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
Time
Start
Project Life Cycle
Ex-Ante
Ex-Post
Formative
Summative
• Each provides a different perspective on the same project.
• Each requires a different set of metrics to be collected.
26
Summary

Ex-ante analysis requires predictive
methodologies.



These must be quantifiable where ever
possible.
Ex-post analysis can be accomplished with
actual data, assuming that the needed
data was collected
Use caution on this collection approach.
27
Possible Test Questions

What is an ex-ante evaluation?


Predictive is used for pre-investment justification
(includes Proposal supporting data).
What is an ex-post evaluations?

Post Implementation is used for confirmation,
validation or repudiation of IT investment.
28
Individual Paper Due

What are Ex-post and Ex-ante evaluations?

Explain the approaches and concepts.
29