Demonstrating Early Value in Software
System Projects
Using Software and System Cost Models
Anca-Juliana Stoica, IT Dept, UU
26th International Forum on COCOMO and
Systems/Software Cost Modeling
University of Southern California, Los Angeles, CA
November 2-4, 2011
Agenda
Informationsteknologi
•Research Background
• Value-Based Software Engineering
• Software and System Cost Models
• Balanced Scorecard Method
•Algorithm for Applying VBSE
•Collecting Project Information and
Questionnaire
•Experience Database
•Case Study
•Final Remarks and Future Work
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
What is value?
- Value: worth in usefulness or importance to
the possessor; utility or merit
- Value of a system/software project: elicited
by the project stakeholders’ values
- Early value: value generated early in the
project lifecycle
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Value-Based Software
Engineering
- Traditional Software Engineering (SE) is not enough to make
projects succeed
- Value-Based Software Engineering (VBSE) is the integration a
software system’s stakeholder value propositions into the
system’s definition, design, development, deployment, evolution
•
Includes aspects not covered by SE that are critical to the system’s
success
- VBSE integrates value considerations into current and emerging
software engineering principles and practices while developing an
overall framework in which these techniques compatibly reinforce
each other.
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
VBSE Key Elements
Key components:
-
Benefits Realization Analysis
Stakeholder Value Proposition Elicitation and Reconciliation
Business Case Analysis
Continuous Risk and Opportunity Management
Concurrent Software and System Engineering
Value-Based Monitoring and Control
Change as Opportunity
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Problem/Question/Goal
- Validating VBSE compared to SE by
demonstrating early value in software
system projects
- Can early value be demonstrated in
software system projects, if so, how?
- Demonstrate value
 Using data collection and projects
developed in academic environment for
educational and commercial users
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
COCOMO II Model
- Helps planning and reasoning about the cost and
schedule implications of a software project
•
Software investment decisions
•
Project budgets and schedules
•
Cost/schedule/performance tradeoffs
•
Software improvement decisions
– Reuse, tools, process maturity, outsourcing
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
COSYSMO Model
- COnstructive SYStems engineering cost MOdel
- COSYSMO helps plan the systems engineering
costs of a project
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Balanced Scorecard Method
- How do customers see us? (Customer perspective)
- What must we excel at? (Internal perspective)
- Can we continue to improve and create value? (Innovation
and learning perspective)
- How do we look to shareholders? (Financial perspective)
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Algorithm for Applying VBSE
1.
Elaborate the Results Chain based on Benefit Realization Approach
2.
Identify success-critical stakeholders, their roles and value propositions
3.
Analyze the Model Clash Spider Web based on the previous two steps
4.
Verify that the project uses the prototyping techniques mentioned and/or a
relevant process model
5.
Identify milestones according to the process model used ( like: iterative
process model with: i) major and minor milestones; ii) objectives to be tested
against at each milestone; iii) risk-driven).
6.
Estimate costs using COCOMO II and/or COSYSMO models for
software and/or systems
7.
Make market projections for the next time horizon (ASA, APR) (years)
8.
Calculate Expected Benefits (EB)
9.
Calculate Cumulative Expected Benefit (CEB) and Return On
Investment (ROI)
10.
Compare expected and actual values using the measures presented
above in steps 6-9
11.
Identify the critical risks and try to figure out how much risk is
enough by applying for example sweet points, utility functions
and people factors
12.
Use the Results Chain and the milestones in order to monitor the
project progress and see if the project is progressing according to plan.
Goal-Question-Metrics (GQM) or the Balanced Scorecard Method (BSM)
can also be used. Apply corrective actions in case value is not realized
13.
Apply architecture-based and refactoring techniques can be used in order
to validate Change as Opportunity
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Experience Database
- In the database all the information gathered from the
projects in order to perform the analysis is stored
- Online access via website
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Database Model
Project
Stakeholder
Role
x
x
ID
Name
1
Institutionen för informationsteknologi | www.it.uu.se
1
ID
Name
Location
Description
1
ID
Name
x
x
Deliverable_ty
pe
Project
metrics
ID
Name
Project_id
x
x
ID
Name
Deliverables
1
Id, name,
filename,
project_id,
deliverabletype
_id, keyword
Informationsteknologi
Case Study:
Web-Based Time-Reporting
System
•Developed for a company that
provides other companies with
consultants
•Online, anytime, from
anywhere
•Keeps track of consultants
working with several customers at
the same time
•Security issues- no one could
break it from outside
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Step 1 – Results Chain
based on BRA
Implement an Online
System for Consultants’
Time Reporting
Saves time for
all system
users
Saves
operational costs
OUTCOME
OUTCOME
INITIATIVE
Contribution
Reduces time
to work
Institutionen för informationsteknologi | www.it.uu.se
ASSUMPTION
Contribution
Increased system
efficiency
Implementing a
new online IT
System is an
important efficiency
criterion
Step 2 – Stakeholders
Informationsteknologi
Step 4: Process Model
Stakeholders:
-
Acquirers (Users) - R2M
Mentors
Project Group
KTH
Institutionen för informationsteknologi | www.it.uu.se
Process Model:
- The Rational Unified Process
(RUP)
 Iterative
 Major and minor milestones
 Formal objectives to be
tested against
 Risk-driven
 Prototyping technique: HTML
prototype of the system
Informationsteknologi
Step 3 – Model Clash Spider
Web
Mentors and KTH
PC/PC
Acquirers/Use
rsrs
Applications compatibility
PD/PD
PD/PD
Many features
PP/PD
Early availability
PD/S
Project cost/effectiveness
PP/PD
PP/S
PD/S
PP/S
Changeable requirements
PC: Process
PD: Product
PP: Property
S: Success
Institutionen för informationsteknologi | www.it.uu.se
PD/PD
Development visibility and control
Limited development budget,
schedule
RUP Process
PC/PC
Project group
Freedom of choice process
Ease of meeting budget and
schedule
Freedom of choice COTS/reuse
Stable requirements
Informationsteknologi
Step 5 – Major Milestones
Milestone
Due date
Milestone details
Inception readiness
Oct 1, Year1
-
Life Cycle Objective (LCO)
Oct 26, Year1
Win- Win Negotiation, Vision, Glossary, Software and System
Requirement Specification, Use Case Specification,
Supplementary Specification
Life Cycle Architecture (LCA)
Dec 21, Year1
System Architecture, Software Architecture Document,
Rose Analysis and Design Models, Data Base Model, User
Interface Prototype (HTML), Architectural Prototype
Initial operational capability (IOC): hardware
Core capability drivethrough (CCD)
Dec 21, Year1
System up and running
Feb 1, Year2
Major Use Cases up and running
IOC: software
Feb 25, Year2
All preliminary Use Cases up and running
Developed IOC
March 1, Year2
-
Responsive IOC
March 5, Year2
-
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Step 6 – Budget, COCOMO, COSYSMO
- Overall estimated development budget: 22 PM (11 persons: 7
programmers, 4 networking engineers; 2 full-time months)
- COCOMO II results for the project
- COSYSMO results
•System Engineering Person-Months Estimated: 26
•System Engineering Hours Estimated:3951
•Total Normalized SE Size (expressed as EREQ): 553
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Step 7 – Market Projections
APR
0
.1
*
ASA
ASA
ASI
ASH
j
j
j
j*
j
Yea
r
ASIj
ASHj
ASAj
APRj
MSEK
%
MSEK
MSEK
0
400 000
0
0
0
1
500 000
0.1
50
5
2
600 000
0.5
300
30
3
700 000
1
700
70
4
800 000
1.5
1 200
120
5
900 000
2
1 800
180
6
1 000 000
2.3
2 300
230
7
1 100 000
2.3
2 530
253
(Annual market SIze)
Institutionen för informationsteknologi | www.it.uu.se
(Annual market SHare)
(Annual market SAles)
Annual market PRofits
Informationsteknologi
Step 8 – Expected Benefits
CPR
APR

APR
OCS
CPR
j
j
,
new
j
,
oldEB
j
j
j
Year
APRj,new
APRj,old
CPRj
OCSj
EBj
MSEK
MSEK
MSEK
MSEK
MSEK
(Annual market PRofits)
(Annual market PRofits)
(Change in PRofits)
(Operational Cost Savings)
(Expected Benefits)
0
0
0
0
0
0
1
5
0
5
22.5
27.5
2
30
0
30
22.5
52.5
3
70
0
70
22.5
92.5
4
120
0
120
22.5
142.5
5
180
0
180
22.5
202.5
6
230
0
230
22.5
252.5
7
253
0
253
22.5
275.5
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Step 9 – Cumulative Expected
Benefits and ROI
CEB
CC
j
j
ROI

j
CC
j
Year
k
CEB
EB

k
k
j
1
EBj
CEBj
CCj
ROIj
MSEK
MSEK
MSEK
%
(Expected Benefits)
(Cumulative Expected Benefits)
(Cumulative Cost)
(Return On Investment)
0
0
0
40
-1
1
27.5
27,5
55
-0.47
2
52.5
80
75
0.067
3
92.5
172,5
100
0.725
4
142.5
315
130
1.423
5
202.5
517.5
165
2.136
6
252.5
770
205
2.756
7
275.5
1 045.5
250
3.182
CEBk = 1045.5 MSEK
ROIk = 318.2%
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Return on Investment
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Step 10 – Expected and Actual Values
- Expected and Actual Values are compared using the
measures presented in steps 6-9
- System up and running
- Analyses performed show good ROI, COSYSMO and
COCOMO results
- Future expansion has a lot of possibilities if they were
exploited
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Step 11 – Risks/Opportunities at
Milestones
Major risks:
- Team member withdrawal
- Inadequate resources, personnel
- Personnel competence
- Time management during exams
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Risk Examples
Institutionen för informationsteknologi | www.it.uu.se
Steps 12 and 13 – Project progress and
corrective actions
Informationsteknologi
To monitor project progress, the deadlines for milestones,
their pass-fail criteria, and risk management are reviewed
Milestone
Inception readiness
Life Cycle Objective (LCO)
Life Cycle Architecture (LCA)
Initial operational capability (IOC): hardware
Core capability drivethrough (CCD)
IOC: software
Developed IOC
Responsive IOC
Institutionen för informationsteknologi | www.it.uu.se
Due date
Risks Eliminated
Oct 1, Year1
Oct 26, Year1
Dec 21, Year1
R1,R2, R6, R7, R9
Dec 21, Year1
R11
Feb 1, Year2
R8
Feb 25, Year2
R10
March 1, Year2
March 5, Year2
R3, R4, R5, R12, R13
Informationsteknologi
Step 14 – Architecture
Is only applicable if there is a high-level architectural
design like patterns and views
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Discussion
-Algorithm and method for value-based software
and system monitoring and control applied for a
large number of commercial and educational
applications developed in an academic environment
-Projects data were collected in the Experience
Database
-Value-based questionnaire was designed and the
results were analyzed
-Questionnaire results not included, but were used
for validation purposes
-Projects developed for commercial applications like
the presented case study showed larger ROI than
those for educational applications
-Applying the above approach helped demonstrating
early value in software and system projects
motivating the need for further research in the
area.
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Conclusion
Our approach for demonstrating early value in software and
system projects is based on:
- Survey questionnaire
- Real data collection via market research
- Algorithm for applying VBSE
 Software and system cost estimation models
 Other models / metrics
- Case studies
- Experience database
- Website
Institutionen för informationsteknologi | www.it.uu.se
Informationsteknologi
Future work
-
Monitoring value creation in modern software processes
Applying patterns and views
Extended metrics set
Strategic IT decision-making
Models and model integration
Use project follow-up
Further case studies
Education and communication of scientific and practical
results
Institutionen för informationsteknologi | www.it.uu.se
References
Informationsteknologi
Biffl, S., Aurum A., Boehm, B., Erdogmus, H., Gruenbacher P.(Eds.), “Value-Based Software Engineering”,
Springer, 2006
Boehm, B.,“Value-Based Software Engineering”, ACM SIGSOFT Software Engineering Notes vol 28 no 2,
2003.
Boehm, B., and Huang, L., “Value-Based Software Engineering: A Case Study”, IEEE Computer, March, 2003.
Jacobson, I., Booch, G., and Rumbaugh, J., “The Unified Software Development Process”, Addison Wesley,
1999.
“RUP 2000: Rational Unified Process”, Rational Software, 2000.
Stoica, A.J., “IT Project Management, Methods and Tools”. Lecture Notes, KTH, Stockholm, Sweden, 2002.
Stoica, A.J., “An Internet-Intranet Solution for Software System Estimation with Use Cases”, Proceedings of the
18th International Forum on COCOMO and Software Cost Modeling, 2003.
Stoica, A.J., “Software Engineering and Security Architecture”. Lecture Notes, KTH-DSV, Stockholm, Sweden,
2004.
Stoica, A.J., “Aspects of Value-Based and Strategic Software Engineering Research”. Invited technical
presentation, SINTEF Research Institute, Oslo, Norway, Sept. 2005.
Stoica, A.J., “Value-Based Software Engineering”. Lecture Notes, KTH-DSV Stockholm, Sweden, 2007.
Institutionen för informationsteknologi | www.it.uu.se