INFO 630
Evaluation of Information Systems
Prof. Glenn Booker
Week 5 – Chapters 1-3
INFO630 Week 5
1
www.ischool.drexel.edu
Introduction and Foundations
Chapters 1-2
Slides adapted from Steve Tockey – Return on Software
INFO630 Week 5
2
www.ischool.drexel.edu
Introduction and Foundations
Outline
•
•
•
•
Goals
Harsh realities
Science vs. engineering
Relevance of engineering economy to
software
• Business on purpose
• Roadmap
INFO630 Week 5
3
www.ischool.drexel.edu
Goals
• Understand the importance of addressing
the business consequences of your
technical decisions
• Know, and be able to use, sane and
rational decision-making techniques for
your technical decisions
• Understand the implications of “software
engineering”
INFO630 Week 5
4
www.ischool.drexel.edu
Harsh Realities
• 23% of software projects fail to deliver any
working software at all
• Of projects that do deliver, they average
– 63% late
– 45% over budget
– 67% of the features and functions delivered
• 40% of commercial applications of
computers have been uneconomical
Reference: [Standish01], [Kidder81]
INFO630 Week 5
5
www.ischool.drexel.edu
Harsh Realities (cont)
• Annual software budget in the US is about
$275 billion
– $63 billion/year in cancelled projects alone
– As much as $149 billion/year in net moneylosing projects
Reference: [Standish01]
INFO630 Week 5
6
www.ischool.drexel.edu
The Cost of Bad Decisions
• Poor software project performance can almost always be
traced back to bad decisions (whether accidental or
intentional), either by the customer, the development
staff, or both
–
–
–
–
–
–
–
–
–
Which projects to do
Not getting good requirements
Not giving good requirements
Using inappropriate technology
Choosing the wrong design or architecture
Not giving the project team adequate resources
Not planning and/or managing the project
Not paying attention to quality
…
INFO630 Week 5
7
www.ischool.drexel.edu
Not a New Issue
“Railway location obviously is a field in which many
alternatives are likely to be available. Nevertheless,
Wellington observed what seemed to him to be an
almost complete disregard by many locating engineers
of the influence of their decisions on the prospective
costs and revenues of the railways. In his first edition
(1877) he said of railway location, “And yet there is no
field of professional labor in which a limited amount of
modest incompetency at $150 per month can set so
many picks and shovels and locomotives at work to no
purpose whatsoever”
Reference: [Grant90]
INFO630 Week 5
8
www.ischool.drexel.edu
Not a New Topic
• Recognized engineering degree programs
usually require a course in Engineering
Economy
– Applied micro-economics
• Fundamental question of software professionals
– “Is it in the organization’s best interest to
invest its limited resources in this technical
endeavor, or would the same investment
produce a higher return elsewhere?”
INFO630 Week 5
9
www.ischool.drexel.edu
Science vs. Engineering
• Definition: Science
“a department of systematized knowledge as an object of study; knowledge
or a system of knowledge covering general truths or the operation of
general laws esp. as observed and tested through the scientific method”
• Definition: Engineering
“the profession in which a knowledge of the mathematical and natural
sciences gained by study, experience, and practice is applied with judgment
to develop ways to utilize, economically, the materials and forces of nature
for the benefit of mankind”
Reference: [Webster94], [ABET00]
INFO630 Week 5
10
www.ischool.drexel.edu
Science vs. Engineering (cont)
• Demarco defines engineering as:
– “Finding the balance between what is
technically feasible and what is economically
acceptable”
• Wellington defines engineering as:
• “The art of doing that well with one dollar
which any bungler can do with two”
• In general
– Science – pursuit of knowledge
– Engineering – application of that knowledge
Reference: [DeGarmo93], [Wellington1887]
INFO630 Week 5
11
www.ischool.drexel.edu
Science vs. Engineering (cont)
• So relationship between science and
engineering is:
Engineering = Scientific Theory +
Practice +
Engineering Economy
INFO630 Week 5
12
www.ischool.drexel.edu
Economics: the Science of Choice
“… software economics has often been
misconceived as the means of estimating
the cost of programming projects. But
economics is primarily a science of choice,
and software economics should provide
methods and models for analyzing the
choices that software projects must make”
Reference: [Levy87]
INFO630 Week 5
13
www.ischool.drexel.edu
Software Engineering vs.
Computer Science
Software Engineering = Computer Science +
Practice +
Engineering Economy
INFO630 Week 5
14
www.ischool.drexel.edu
Software Engineering vs.
Computer Science (cont)
• Computer Science
“a department of systematized knowledge about computing as an
object of study; a system of knowledge covering general truths or
the operation of general laws of computing esp. as observed and
tested through the scientific method”
• Software Engineering
“the profession in which a knowledge of the mathematical and
computing sciences gained by study, experience, and practice is
applied with judgment to develop ways to utilize, economically,
computing systems for the benefit of mankind”
INFO630 Week 5
15
www.ischool.drexel.edu
Relevance of Engineering
Economy to Software
• Every professional developer is affiliated with some
public or private enterprise
• Every day, we make decisions that impact that
enterprise
–
–
–
–
Staffing
Internally-developed software
Leased or purchased software
Operation of software systems
• As professionals, we had better be concerned with the
impacts of our decisions
– “… a limited amount of modest incompetency …”
INFO630 Week 5
16
www.ischool.drexel.edu
Business on Purpose (Ch-2)
Why are companies in business?
Because it’s fun? Educational? A way to
have a positive impact on society?
No: companies are in business to make a
profit for the owners
Where does the money come from?
Where does the money go?
INFO630 Week 5
17
www.ischool.drexel.edu
Where Does the Money Go?
Gros s
Revenue
Cost of Goods Sold
M at erial
Labor
Operating
Income
Operat ing Expenses
Selling expenses
General and administ rat ive expens es
Taxes (ot her than income)
Investment-related Expenses
Int erest on loans
Depreciation
Income Taxes
Federal
State
Local
Return On Equity
Cash dividends on s tock
INFO630 Week 5
Net Income
Before Taxes
Net Income
After Taxes
Retained
Earnings
18
www.ischool.drexel.edu
PeopleSoft, 2003
$2,267M (100.0%)
Gros s
Revenue
Cost of Goods Sold
M at erial
Labor
$868M (38.3%)
Operating
Income
Operat ing Expenses
Selling expenses
General and administ rat ive expens es
Taxes (ot her than income)
$117M (5.2%)
$1,282M (56.6%)
Investment-related Expenses
Int erest on loans
Depreciation
Net Income
Before Taxes
$139M (6.1%)
Net Income
After Taxes
$85M (3.8%)
Retained
Earnings
$85M (3.8%)
-$22M (-0.9%)
Income Taxes
Federal
State
Local
$53M (2.3%)
Return On Equity
Cash dividends on s tock
$0M (0.0%)
INFO630 Week 5
19
www.ischool.drexel.edu
Profit Margin
• A key indicator of the health of a company
ProfitMarg in 
NetIncomeA fterTaxes
GrossReven ue
• Typically 2% to 10%
• How expensive is software?
– Full Time Equivalent (FTE) is usually between $150k and $300k
(in year 2002)
• Fully burdened salary
INFO630 Week 5
20
www.ischool.drexel.edu
Decision-Making in For-Profit Organizations
“All other things being equal, the role of
the technical person in a for-profit
company should be to choose—from the
set of technically possible solutions to a
problem—the solution that maximizes the
organization’s objective: profit”
INFO630 Week 5
21
www.ischool.drexel.edu
Decision-Making in Not-For-Profit
Organizations
“All other things being equal, the role of a
technical person in a not-for-profit
organization should be to choose—from
the set of technically possible solutions to
a problem—the solution that maximizes
the organization’s objective: providing the
greatest benefit at the least cost”
INFO630 Week 5
22
www.ischool.drexel.edu
Decision-Making in Your Own Personal
Finances
• You probably don’t have an unlimited
supply of money
– You work hard for the limited income you get
– You want to get the most you can out of it
INFO630 Week 5
23
www.ischool.drexel.edu
Key Points
• Software projects don’t perform very well
• Science and engineering are related but
distinct
• Companies don’t have a lot of money to
play around with
– Neither do you
INFO630 Week 5
24
www.ischool.drexel.edu
Fundamental Concepts of
Business Decisions
Chapter 3
Slides adapted from Steve Tockey – Return on Software
INFO630 Week 5
25
www.ischool.drexel.edu
Fundamental Concepts of Business Decisions
Outline
• Concepts to align technical decisions with
business decisions.
– Proposal
– Cash-flow instance
– Cash-flow stream
– Cash-flow diagram
– Developing cash-flow streams
– Business case for agile projects
INFO630 Week 5
26
www.ischool.drexel.edu
Proposal
• A single, separate option that’s being
considered
• Examples
– Do project Alpha
– Enhance product Beta
– Use the NIFF algorithm in the Pack module
– Use a doubly linked-list data structure for
messages
INFO630 Week 5
27
www.ischool.drexel.edu
Proposal Con’t
• Each proposal represents a binary choice
– Carry out or
– Reject
• Purpose of decision making is to
– Figure out, given current business
circumstances, which should be carried out
and which ones shouldn’t.
• Each proposal should be evaluated from a
business perspective
INFO630 Week 5
28
www.ischool.drexel.edu
Cash-Flow
• Cash-flow describes the business perspective of a proposal
• Cash-flow instance
– A specific amount of money flowing into or out of the organization at a
specific time as a direct result of a proposal
• Dev staff payroll in 3rd month of the Beta project
• Separate payment on a loan
• Cash-flow stream
– The set of cash-flow instances, over time, which would be caused by
carrying out some given proposal
• Basically the complete financial picture of a proposal.
• The financial view of the Beta project
INFO630 Week 5
29
www.ischool.drexel.edu
Categories of Cash Flow
•
Initial investment
–
•
Operating and maintenance costs
–
•
Direct income generated by the proposal
• Net increased sales on the enhanced Beta product
Cost avoidance
–
•
Costs occurring after the proposal is started that continue through its retirement
• Maintenance fees on the development environment for the Beta enhancement
Sales income
–
•
One-time, non-recurring costs associated with starting up a proposal
• Buying a new development environment so we can enhance product Beta
Indirect income, caused by a reduction in costs instead of an increase in sales
• Reduced software maintenance costs on the enhanced Beta product
Salvage value
–
Recoverable value in assets at the end of the proposal
• What we could sell the development environment for at the end of the Beta project
INFO630 Week 5
30
www.ischool.drexel.edu
Cash-Flow Diagram
• Picture of a cash-flow stream
– Gives quick overview of financial picture
• Characteristics
–
–
–
–
–
–
Time runs left to right
Income shown above time-line
Expense shown below time-line
Arrow length proportional to $ amount
By convention, net cash-flow per period shown
Initial investment shown “at the end of period
zero”
INFO630 Week 5
31
www.ischool.drexel.edu
Example Cash-Flow Diagram
$8150
$5900
$3650
$2900
$1400
$650
0
1
2
3
4
5
6
-$850
Income
7
Expense
Time 
Initial investment
-$10,000
INFO630 Week 5
32
www.ischool.drexel.edu
Cash-Flow Diagram Con’t
• Cash flow stream (cash-flow diagram)
– Based on the specific perspective of the
organization
– Changing perspective will change the cash
flow stream
INFO630 Week 5
33
www.ischool.drexel.edu
Cash-Flow Diagram Perspectives
INFO630 Week 5
34
www.ischool.drexel.edu
Developing Cash-Flow Streams
• The Work Breakdown Structure (WBS) is
a useful tool
– Typically a hierarchical decomposition of the
proposal
– Decompose only until reasonable timing and
cost estimates can be made
– All work necessary to complete the proposal
needs to be included
– Only work necessary to complete the
proposal should be included
– Income needs to be represented
See: [PMI02], [DoD98], or [Fleming00]
INFO630 Week 5
35
www.ischool.drexel.edu
A Simple WBS Example
•
My Software Project
– Construct software [Big task #1]
• Requirements [Smaller task #1.1]
– Part A Requirements [Tiny task #1.1.1]
– Part B Requirements [Tiny task #1.1.2]
– ...
• Design [Smaller task #1.2]
– Part A Design [Tiny task #1.2.1]
– …
• Construction [Smaller task #1.3]
– Part A Code [Tiny task #1.3.1]
– …
– Software quality assurance [Big Task #2]
• Peer reviews [Smaller task #2.1]
• Testing [Smaller task #2.2]
• ...
INFO630 Week 5
36
www.ischool.drexel.edu
An Example Project: Automated Test
Equipment (ATE)
•
One person-year = $125k
•
Initial investment
– $300k for test hardware and development equipment (Year 0)
– 20 person-years of software development staff (Year 1)
– 10 person-years of software development staff (Year 2)
•
Operating and maintenance costs
– $30k per year for test hardware and dev equipment (Years 1-10)
– 5 person-years of software maintenance staff (Years 3-10)
•
Sales income
– None
•
Cost avoidance
– $1.3 million in reduced factory staffing (Years 2-10)
•
Salvage value
– Negligible
INFO630 Week 5
37
www.ischool.drexel.edu
Automated Test Equipment (cont)
Year
0
1
2
3
4
5
6
7
8
9
10
Dev Staff
0
-$2.5M
-$1.25M
-$625K
-$625K
-$625K
-$625K
-$625K
-$625K
-$625K
-$625K
Equipment
-$300K
0
0
0
0
0
0
0
0
0
0
O&M
0
-$30K
-$30K
-$30K
-$30K
-$30K
-$30K
-$30K
-$30K
-$30K
-$30K
INFO630 Week 5
Savings
0
0
$1.3M
$1.3M
$1.3M
$1.3M
$1.3M
$1.3M
$1.3M
$1.3M
$1.3M
Total
-$300K
-$2.53M
$20K
$645K
$645K
$645K
$645K
$645K
$645K
$645K
$645K
38
www.ischool.drexel.edu
Cash-Flow Diagram for ATE
$645K
0
1
$20K
2
3
4
5
6
7
8
9
10
-$300K
-$2.53M
INFO630 Week 5
39
www.ischool.drexel.edu
Business Case for Agile Projects
• Agile methods assume the project’s goals
and/or requirements are inherently
unstable
– How can you build a WBS?
• An Agile project’s business case involves
estimating how much business value can
be returned (and when it’s returned) given
the level of investment the customer is
willing to make
INFO630 Week 5
40
www.ischool.drexel.edu
Key Points
• A proposal is a single, separate option that’s being
considered
• A cash-flow instance is an amount of money at a given
time caused by a proposal
• A cash-flow stream is the financial view of a proposal
• A cash-flow diagram is a picture of a cash-flow stream
• The WBS can be a useful tool for developing cash-flow
streams
• The business case for agile projects is different
INFO630 Week 5
41
www.ischool.drexel.edu