CS 510 Final Exam Q&A
Fall 2015
Barry Boehm, USC
COCOMO Estimation and Sizing
As a developer who is trying to improve my skill at estimating how long it will take to do my
work, I'm wondering what is the best approach for me to take. For me, the hard part about
using COCOMO would be estimating the SLOC (I don't know an accurate way of doing
this). Another approach I have in mind would be to use story points to estimate each story,
and then to keep track of how long each of them takes me, using those results to help with
future estimations. Do you have any advice on what kind of approach I should take?
If you are using agile methods, story points are the best fit. The CSSE web site has a
COCOMO version called Agile COCOMO (at
http://sunset.usc.edu/cse/pub/research/AgileCOCOMO/AgileCOCOMOII/Main.html) that
enables you to take your last project’s velocity in terms of #of ideal person weeks per
iteration (number of story points) and adjust it for any differences in the COCOMO cost
drivers on your next iteration or project. You can then adjust this if your next iteration has
a different number of story points.
If you are sizing a project in SLOC, you can try the function point option in COCOMO II, and
adjust it as you do parts of the project as prototypes. If a prototype takes 200 SLOC and
covers about 20% of the functionality, a good estimat would be 1000 SLOC
COCOMO III
Throughout the course there have been references to a COCOMO III model as being in
development, are there any major changes in the model from COCOMO II worth
mentioning? Also, do you have an estimate of when that would be published?
COCOMO III is still getting defined. A key research activity is defining appropriate
sizing parameters for various types of project (agile, cloud-services-based, COTS
integration, other ICSM special cases, hybrid) as in the previous question.
Business-Value-Based Earned Value
In EP-3 (Business Case Analysis), figure 14-6 describes a process for keeping Earned
Value estimates accurate by relating them to the business case. The diagram wasn't
covered in lecture, but EV-related issues crop up from time to time in failure stories.
Can you describe what this diagram means, and how it can be used to keep EV
"honest"?
The intent is to get away from value-neutral EV methods and relate value earned to
progress toward business objectives.. The next chart provides an example.
Customer Satisfaction
ITV
Ease of Use
7.0
12.4
1.7
1.0
1.8
3/31/04
427
20
72
7.0
12.4
1.7
1.0
1.8
7/31/04
1050
7/20/04
1096
9/30/04
1400
9/30/04
1532
2.4* 1.0* 2.7*
3500
3432
12/31/04
4000
20
80
8.0
0.0
-1.0 11.4
3.0
2.5
3.0
12/20/04
4041
22
88
8.6
0.6
-.85 10.8
2.8
1.6
3.2
3/31/05
4500
300
9.0
3.5
3.0
3.5
3/30/05
4604
324
7.4
3.3
1.6
3.8
Full Op’l Capability
CCD
7/31/05
5200
1000
7/28/05
5328
946
Full Op’l Capability
Beta
9/30/05
5600
1700
9/30/05
5689
1851
12/31/05
6000
2200
22
106 12.2
3.2
-.47
7.0
4.0
3.5
4.0
12/20/05
5977
2483
24
115 13.5
5.1
-.15
4.8
4.1
3.3
4.2
6/30/06
6250
Responsive IOC
Full Op’l Capability
Deployed Release 2.1
Increased COTS ITV risk,
fallback identified.
Using COTS ITV fallback; new
HW competitor; renegotiating
HW.
2.7* 1.4* 2.8*
9/30/04
Deployed IOC
Risks/Opportunities
Late Delivery %
72
ROI
20
Cum. Profits
400
Op’l Cost Savings
3/31/04
10/11/04
Hardware IOC
03/27/07
Annual Profits ($M)
Software Initial
Op’l Capability (IOC)
Annual Sales ($M)
Core Capability
Demo (CCD)
Market Share %
Life Cycle Architecture
Cost ($K)
Milestone
Schedule
Value-Based Expected/Actual Outcome Tracking Capability
$200K savings from
renegotiated HW.
3.5* 2.5* 3.8*
New COTS ITV source
identified, being prototyped.
New COTS ITV source initially
integrated.
3.8* 3.1* 4.1*
©USC-CSE
5
Value-Based vs. Value-Neutral Defect Fixing
In the slide "The Economics of Systems and Software Reliability Assurance", page
6,7,8. We compare three methods, value-based testing, value-neutral and value-neutral
defect fixing. The result is value-based provides more net value and value-neutral
defect fixing is worse. In the graph, the value-neutral and value-neutral defect fixing
would have the equal value in the end. The different is the when to become efficient. In
the customer type over 10, value-neutral defect fixing complete 80% value. Whether
there is a situation that value-neutral defect fixing test is better than other method?
The slide (shown next) just shows “in the end” as when all of the defects have been
fixed. In most projects not all defects get fixed, and one wants to prioritize to get the
most business value for the ones that do get fixed, so that if you only fix 90% of the
problems, you end up with most of the business value
Value-Neutral Defect Fixing Is Even Worse
Pareto 80-20 Business Value
100
Automated test
generation tool
- all tests have equal value
80
% of
Value
for
Correct
Customer
Billing
60
40
Value-neutral defect fixing:
Quickly reduce # of defects
20
5
10
15
Customer Type
09/23/2015
©USC-CSSE
7
Value-Based Defect Fixing
I am confused about the graph on EC-21,slide 8. Is it saying that in terms of business
value, testing only about 20% of the cases is sufficient for 80% of the business value
and then for defect testing, testing the easiest 20% almost has no effect on % of Value
for Correct Customer Billing?
Slide 8, just shown, is addressing defect fixing and not defect testing. Fixing the
easiest 20% almost has no effect on % of Value for Correct Customer Billing.
Build vs. Buy
We have solves many cost evaluations problems using COCOMO II for example in HW3,
Midterm I & II. There's a situation that we may face, there's a two way to achieve a
features or a system, one is to develop it by the company's own developer team, the
other is to buy license to use other company's service. Let suppose that the total cost
we use COCOMOII to get is less than the total license purchase cost. Before making the
decision, is there any consideration we need to take? In another word, is there any
possibility that we may still choose to buy license in this situation, and what could it be?
Buying a license gets you the capability right away vs. waiting for it to be developed.
The maintenance fees are generally similar to the maintenance costs for a build-yourown system, and you get upgrades without having to develop them. On the other hand,
for a build-your-own system, you determine and create what upgrades you need most,
rather than hoping that the vendor will choose to do them.
ROI and Present Value
TIn calculating ROI, how far into the future should we go for expected revenue? What
if we are comparing two different projects at two different times? For example, project
A starts now, costs $100K and gives us $20K/year. Project B starts 3 years from now,
costs $80K and gives us $25K/year. Would we just compare the ROI of A at year 1
(PV(20,1,r)/100) with the ROI of B at year 4 (PV(25,4,r)/PV(80,3,r)), or would we compare
both at year 4 as a series of cash flows across 4 years for A and 1 year for B (which
doesn't seem right since B has a lot less time to generate revenue), or something
else? (EC7)
These days, 5 years is about the longest that one can count on being reasonably
accurate, given the pace of change in technology, the marketplace, and the
competition. Comparing projects starting at different times is best done by giving
each the same length of payoff period, unless there is some reason that the payoff
period should be longer for one of the projects.
Benefits, Costs, and Net Value
In EC5 Business Case Analysis page27, Return-On-Investment (ROI) formula is
describe as ROI=(Benefits-Costs)/Costs. But in EC21 page3, ROI=NV/PV (present value
divided by net value). Why are those two both correct?
Net Value = Benefits – Costs, making both ROI formulas the same thing.
Similarity of Risk Examples
Question I: In EP-7 Chapter 19 page 281 Table 19-2, the table use NV relative to Option
A as payoff. What should be consider as payoff when there is no Option A, like the
questions in homework #7, or when Option A is some kind of totally different kind of
approach which is incomparable to Option B or cannot be quantified. Moreover, here if
Option BB doesn't work, the team will have to reprogram the system using the
conservative techniques (last paragraph at page 279), so that the overall payoff would
be -$60K + $30K = -$30K. However, in homework 7 solutions, the pay off of approach B
failure is -$100K, which doesn't consider "reprogram the system using conservative
approach". Why different calculation on those two similar situations? Could you
explain more on when to consider with reprogram and when don't to?
The example in the book includes the time and cost spent finding out that the Bold
option is not going to succeed, followed by doing the Conservative approach. The
Homework 7 example does not assume that the Conservative approach would be used
if Bold failed, but just indicates that the outcome would be a $100K loss in obtaining
the capability.
Utility Functions
In EC-10 and EP-7, we discussed utility functions. Other than using surveys, what are
other ways to develop utility functions? Where else can values be obtained? Also, EC6 leads me to believe that 'value' and 'time' are the main qualities of a utility function;
however, EP-7 and EC-10 identify other qualities such as 'utility' and 'payoff'. What
qualities identify a utility function?
Most ways end up looking like surveys, although sometimes they will just survey the
chief decisionmaker. Value and time are the most frequent quantities, although value
can be multidimensional. For example, sometimes it is worth developing a
moneylosing feature if it pleases a key customer.
Risk-Based Decisions
Assume, a project needs to be finished in a short span of time. Given, that the testing system needs a
graphical user interface, and developers are free to use any framework or technology to develop the system
with basic core functionality that data should be accessed from the firm database and the result should be
immediately generated. Consider, a user interface is provided to the client to enter user specific data and a
firm database is maintained to securely hold the data. The UI is to be tested as to review it and generate a
visible error immediately if there are some errors. The developers will be maintaining documentations for
the development and error handling.
So would you suggest that here we should not use ICSM principles. If we use ICSM practices and principle,
wouldn't that delay the project due to amount of phases and risk analysis that is needed to be performed.
So, will it be correct to directly jump on the development phases? If, yes, how do we account for the
skipped phases?
Reference: The Incremental Commitment Spiral Model: Principles and Practices for Successful Systems and
Software, Addison Wesley, 2014
The ICSM indicates that decisions are made based on the stakeholders’ willingness to accept risk. If a
rapid-fielding approach is risky but the stakeholders agree that the risks are negligible, then the project can
skip thorough definition phases and jump into development.
Other Life Cycle Models
For Chapter 5 of the ICSM book, when comparing ICSM to other life-cycle models,
do we have to understand a little bit of details of some other typical models?
Since the course material does not cover these in detail, you won’t need to know the
details of other models.