University of Southern California
Center for Systems and Software Engineering
Rapid-Fielding
Software-System Development
Supannika Koolmanojwong
koolmano@usc.edu
USC CSSE Annual Research Review
March 9, 2010
University of Southern California
Center for Systems and Software Engineering
Outline
• The Incremental Commitment Model & Rapidfielding projects
• Process Pattern Decision Driver
• Preliminary Results
03/09/2010
USC CSSE ARR 2010
2
University of Southern California
Center for Systems and Software Engineering
Motivation
• Growing diversity of software systems
– No one-size-fits-all process for the full range of software
systems
• Rapid Fielding is a very important Software System
Objective
– To fit into market windows, respond to competition
– Several possible process patterns can be used
• Some process models provide specific evidence-based
and risk based decision points
– ICM is the most thoroughly elaborated
03/09/2010
USC CSSE ARR 2010
3
University of Southern California
Center for Systems and Software Engineering
NDI/NCS Growth Trends
CBA Growth Trend in USC e-Services Projects
0.8
0.7
0.6
Percentage
%
80%
*
0.5
0.4
0.3
0.2
0.1
0
60%
50%
57%
35%
40%
19%
20%
0%
1997
1998
1999
2000
2001
2002
Year
USC e-services project data shows increase in
number of projects using COTS from 28% in
‘97 to 70% in ’02 - Similar results (54% in
2000) were found in the Standish Group’s
2000 survey [Yang 2006].
Fa'06 - Sp'07 Fa'07 - Sp'08 Fa'08 - Sp'09 Fa'09 - Sp'10
Year
USC e-services project data shows increase in
number of projects using NCS from 19% in
2006 to 57% in 2009
Programmableweb.com :
-3 new Mashups listed /day
-Total 4436 Mashups listed
[Programmableweb.com
accessed 11/09/09]
03/09/2010
USC CSSE ARR 2010
4
University of Southern California
Center for Systems and Software Engineering
4 focused ICM Common Process Patterns
Process Patterns
Market –Driven,
Services- Driven,
NDI-Driven
03/09/2010
Use Single Non-Developmental Item (NDI)
Agile
Architected Agile
Formal Methods
HW with embedded SW component
Indivisible IOC
NDI- intensive
Hybrid agile/ plan-driven system
Multi-owner system of systems
Family of systems
Brownfield
Services- Intensive
USC CSSE ARR 2010
5
University of Southern California
Center for Systems and Software Engineering
Different Risk Patterns Yield Different Processes
Architected Agile
E.g. Business data processing
Use Single NDI
E.g. Accounting System
NDI-Intensive
E.g. Supply Chain Management
Services-Intensive
E.g. Community Services
03/09/2010
Risk?
Risk?
USC CSSE ARR 2010
Risk?
Risk?
Risk?
6
University of Southern California
Center for Systems and Software Engineering
03/09/2010
USC CSSE ARR 2010
7
University of Southern California
Center for Systems and Software Engineering
Definitions of NDI / NCS
• NDI or Non-developmental Item is an item that is previously developed
and available to use. It includes commercial-off-the shelf, open source
product, reuse library and customer-furnished package.
– Two main categories of NDI are
• platform-based NDI such as Internet Explorer, Eclipse, MySQL, Apache
Tomcat, JDK/JSP
• application-based NDI such as Microsoft office, Quickbook,
BusinessWorks, Barcode Generator, JMP and primopdf.
• NCS or Net-Centric Services is an online service available to be
accessed over the Internet such as Google services, Yahoo services,
Google map, Twitter, Ning.com, Gmail, Facebook, Amazon payment,
online currency converter and online dictionary.
– Net-Centric Services is known as web service, web application, online
application, cloud computing, and software-as-a-service.
03/09/2010
USC CSSE ARR 2010
8
University of Southern California
Center for Systems and Software Engineering
NDI, NCS characteristics
Characteristics
NDI
NCS
Platform Independent
Yes / No
Yes
Required Internet Access
Yes / No
Yes
Common Standard
No
Yes
Option of rejecting next release*
Yes
No
Change / upgrade control
Client /Server’s site
Server’s site
End user has the latest version
Yes / No
Yes
Database Ownership
Yes
Yes/No
* Will you be able to freeze the version you are using?
03/09/2010
USC CSSE ARR 2010
9
University of Southern California
Center for Systems and Software Engineering
Existing Process Guidelines on Process Patterns
Process Patterns
Architected Agile
Use Single NDI
NDI- Intensive
Services- Intensive
Pure Agile
03/09/2010
Current process guidelines
• None
• Early user-programming guidance [Scaffidi 2009]
focuses on tailoring the selected NDI
• Various process guidelines
• USC CSSE COTS-Based Application Development
Guidelines (2005)
• CMMI-COTS-Based Systems
• None
• CMMI-SVC focuses on how to develop web services; not
how to apply the available services
• Scrum, XP, Crystal, others
• Not a good fit for turnkey systems
USC CSSE ARR 2010
10
University of Southern California
Center for Systems and Software Engineering
Process Pattern Decision Driver
Decision Criteria
Importance
Architected
Agile
Alternatives
More than 30% of features available in NDI/NCS
Has a single NDI/NCS that satisfies a complete solution
Very unique/ inflexible business process
Life Cycle
Need control over upgrade / maintenance
Rapid Deployment; Faster time to market
Architecture
Critical on compatibility
Internet Connection Independence
Need high level of services / performance
Need high security
Asynchronous Communication
Access Data anywhere
Resources
Critical mass schedule constraints
Lack of Personnel Capability
Little to no upfront costs (hardware and software)
Low total cost of ownership
Not-so-powerful local machines
Use Single
NDI-Intensive
NDI
ServicesIntensive
0–1
0–1
2–4
2–3
4
0–1
3–4
2–3
0–1
3–4
2–3
0–1
2–4
0–1
0–1
4
0–1
2- 4
0–1
2–3
2–4
0–4
0–4
2–4
0–4
0–4
3–4
0–4
0–3
0 -4
0–4
0–4
1–3
0–4
0–3
0–4
0–4
0–4
2–4
0
0–2
0–2
0
4
0–1
0–2
0–2
0–1
1–4
3–4
3–4
2–4
0–3
1–3
2–3
2–4
2–4
0–3
0–4
2–4
2–3
3–4
3–4
3–4
Note: Decision importance scale varies from project to project
Decision Criteria Rating Scale; 0:Very Low; 1:Low; 2: Medium; 3:High; 4:Very High
Importance Rating Scale: 1:Low; 2: Medium; 3:High
03/09/2010
USC CSSE ARR 2010
11
University of Southern California
Center for Systems and Software Engineering
An Example of a team that follows the Architected Agile Process Pattern :
Shields For Family Project
– Develop various reports for LA city-based Family Housing Project
Decision Criteria
Alternatives
More than 30% of features available in NDI/NCS
Has a single NDI/NCS that satisfies a complete solution
Very unique/ inflexible business process
Life Cycle
Need control over upgrade / maintenance
Rapid Deployment; Faster time to market
Architecture
Critical on compatibility
Internet Connection Independence
Need high level of services / performance
Need high security
Asynchronous Communication
Access Data anywhere
Resources
Critical mass schedule constraints
Lack of Personnel Capability
Little to no upfront costs (hardware and software)
Low total cost of ownership
Not-so-powerful local machines
03/09/2010
USC CSSE ARR 2010
Importance
Architected
Agile
1
1
2
1
0
2
3
1
3
1
3
3
2
2
2
3
2
1
3
2
3
3
2
2
2
2
2
1
2
2
3
2
12
University of Southern California
Center for Systems and Software Engineering
An Example of a team that follows the Architected Agile Process Pattern
Use single NDI
Architected Agile
NDI-Intensive
Services -Intensive
High importance level
Low importance level
Project Status
03/09/2010
USC CSSE ARR 2010
13
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Model – Electronic Process Guide
(ICM EPG)
Using IBM Rational Method Composer to build a software development
process guideline mainly describing roles, responsibilities and artifacts.
03/09/2010
USC CSSE ARR 2010
14
University of Southern California
Center for Systems and Software Engineering
Preliminary Results
• 28% of projects are able to deliver in one semester, instead
of two semesters comparing to 12.5% from previous year
• Clients are highly satisfied with the project results
• Less effort spent in project development
Guidelines \
Average Effort (hours)
Learning
Documentation
Total Effort
03/09/2010
Paper-based process
guidelines
15.2
69.2
209.57
USC CSSE ARR 2010
ICM EPG
11.9
46.6
175.8
15
University of Southern California
Center for Systems and Software Engineering
Process Adoption Statistics
# of Teams Results
8 /14
Selected the right process from the beginning
3/14
•Selected wrong process due to unclear project
scope
•Selected right process after exploration, more
prototyping, found available NDI/NCS
2/14
Changed process due to minor project scope
changes
1/14
Changed process due to infeasible project
implementation
03/09/2010
USC CSSE ARR 2010
16
University of Southern California
Center for Systems and Software Engineering
Conclusion
• For rapid-fielding software projects, this research focuses
on developing and evaluating the effectiveness of process
guidelines for 4 process patterns.
• The approach is being used to validate and refine the
content of the process patterns
03/09/2010
USC CSSE ARR 2010
17
University of Southern California
Center for Systems and Software Engineering
Differences between NDI and NCS (1/2)
Category
Payment


Platform


Integration




Changes





03/09/2010
Non-Developmental Item
[ includes open source, customer-furnished
software]
Non-commercial items usually have no monetary
cost
Expensive initial costs, moderate recurring fee,
training fee, licensing arrangement-dependent
Specific and limited to specific platform / language
Generally supported on a subset of platforms or
multiple platforms but with different editions
Generally more tightly coupled
Not very flexible on existing legacy systems when
proprietary standard is used
Difficult when platform dependent and different
technologies involved
detailed documentation and on-site extensive
support
Able to freeze the version, under user control
Designed for specific use so costly for
customization and change
Change on server side doesn’t impact the client
side
Major releases once in while
Requires end user intervention to upgrade
Net-Centric Services
 Not all services are free
 Low initial costs, moderate marginal cost, duration
depending license




Platform and language independent
Server and client can work on different platform
Interaction between machines over a network
More loosely coupled,
Common web standards,
flexible to integrate
 Requires internet access
 Support forums and API documentation available
 This integration could be done merely in code, without
additional installation of external components




Changes are out of developers’ control
Not easy to predict change, cannot avoid upgrade
The end-user has the latest version of the service
Change on server side result in client side, not require
user intervention
 Minor releases frequently (through patching)
USC CSSE ARR 2010
18
University of Southern California
Center for Systems and Software Engineering
Differences between NDI and NCS (2/2)
Category
Extensions



Evaluation
Criteria

Support
Services
Data












03/09/2010
Non-Developmental Item
[ includes open source, customer-furnished
software]
Only if source is provided and the license permits
Delivered to the end-user by the producer
may not be portable across COTS or compatible
with future releases
Maintenance, extensibility, scalability, reliability,
cost, support, usability, dependency, ease of
implementation, maintainability, upgrades, size,
Access to source and code-escrow considerations
Upfront costs opposed to subscription
Platform compatibility, Feature controllability
sometimes available for a fee
Help topics or FAQs would likely not be updated
after installation
Upgrades/Patches and data migration support
Sometimes can be customized for specific user
Upgrade through purchasing new releases
Data often stored locally. Backups by the user
Data access is generally fast
Possible variety of proprietary formats, Platformdependent
May be inflexible for change but more secure
Can process data offline
Net-Centric Services
 Extension is limited to data provided by the web services
 In-house extension such as wrapper or mashup
 Little control over performance overhead
 Reliability, Availability, Cost, Available Support, Speed,
Predicted longevity of the service provider, release cycle,
Bandwidth
 Recurring costs and future functionality offered
 Standards compatibility
 Feature and data controllability
 generally not available
 Help topics would generally be frequently updated; selflearning
 Usually not customized for specific user
 Patching on service provider’s side; mostly does not
require installation on client side
 Data stored on service host’s servers. Backups by the
provider. Introduces privacy and data-retention
 Data access could be slower since internet based,
Process data online
 Common XML using web standard protocols
 Data from different web services can be used by a single
client program
USC CSSE ARR 2010
19
University of Southern California
Center for Systems and Software Engineering
An Example of a team that follows the “NDI- Intensive” Process Pattern :
SPC website Project
– Develop a website for an organization by using “wordpress” and some additional
custom code
Decision Criteria
Alternatives
More than 30% of features available in NDI/NCS
Has a single NDI/NCS that satisfies a complete solution
Very unique/ inflexible business process
Life Cycle
Need control over upgrade / maintenance
Rapid Deployment; Faster time to market
Architecture
Critical on compatibility
Internet Connection Independence
Need high level of services / performance
Need high security
Asynchronous Communication
Access Data anywhere
Resources
Critical mass schedule constraints
Lack of Personnel Capability
Little to no upfront costs (hardware and software)
Low total cost of ownership
Not-so-powerful local machines
03/09/2010
USC CSSE ARR 2010
Importance
NDI- Intensive
3
2
1
4
3
1
2
3
1
3
1
1
1
1
2
3
2
1
2
2
1
3
2
3
3
3
3
3
2
3
3
3
20
University of Southern California
Center for Systems and Software Engineering
An Example of a team that follow the NDI-Intensive Process Pattern
Architected Agile
NDI - Intensive
Use Single NDI
Services - Intensive
High importance level
Low importance level
Project Status
03/09/2010
USC CSSE ARR 2010
21