University of Southern California
Center for Systems and Software Engineering
Integrating Systems and Software
Engineering:
Technical Aspects Workshop
30 October 2007
University of Southern California
Center for Systems and Software Engineering
IS&SE Workshop Objectives: Identify
• Biggest issues and opportunities?
– Technical aspects
– Management aspects
– Complex systems aspects
• Inhibitors to progress and how to overcome them?
• Ability of six principles to improve IS&SE?
• Ability of Incremental Commitment Model to
improve IS&SE?
• What else is needed?
–
–
–
–
Technology/management research
Education and training
Regulations, specifications and standards
Other?
October 2007
©USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Attendees
• Rick Selby, NGC/USC (Chair)
• Anthony Peterson, Raytheon
• Karen Owens, Aerospace
Corporation
• George Huling, LA Spin
• JinBo Chen, NGC
• Wilson Rosa, Air Force Cost
Analysis Agency
• Rich Turner, Stevens Institute
of Technology
• John Miller, MITRE
October 2007
• Elliot Axelband, USC/RAND
• Jared Fortune,
Aerospace/USC
• Shawn Rahmani, Boeing
• Daniel Winton, Aerospace
• Sue Koolmanojwong, USC
(Scribe)
• Winsor Brown, USC
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects : Summary
• SE/SW integration
– Systems engineering is doing an commendable job in incorporating
software, hardware, and human factors as it has for years however
improvements are needed
– A lot of initiatives are pushed by software people, because they suffered
from the consequences of inadequate SE (integration)
– Significant benefits can be achieved (or are possible) by greater
integration and synergy between systems and software engineering in
many areas, such as communication, education, research, cross
training, integrated life cycle models
• Personnel/Education/Communication
– Too few “software engineers” have adequate engineering background,
domain expertise, or physics understanding / exposure
– At the engineering executive and chief engineering level, few come from
software engineering background
• Research, initiatives, guidance, policy
– Prototype technical policy
– Need major DoD initiative to fund research centers for SE/SW
October 2007
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects: Issues
• Earned value is a rear-view mirror metric; does not adapt to added work
over time, not reflect the rework
• Commitment and accountability: how to make sure/ verify the
commitment and accountability (more than powerpoint)
• For project implementation, it is difficult to predict schedule;
requirement volatility drives schedule unpredictability
• Lack of standardization within DoD for technical standards such as
software requirement specifications, capability design documents
• Include mechanism for monitoring cost and technical performance
throughout the life cycle of the systems
• One-sided systems and software engineering marriage attempt
– Integration attempt pushed by software engineering and not adequately
pulled by system engineering (e.g. CMM and CMMI, system and software
consortium – SSCI, UML and SysML, software technology conference and
systems and software technology conference, center for software
engineering and center for systems and software engineering)
– Need system engineering pushed/pulled integration initiatives for an
acceptable marriage
October 2007
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects :
Biggest Opportunities(1)
• Define integrated measures for systems and software
engineering
• Define a common core set of metrics (common across
acquisition organization and prime contractor) for software
development that visible at system level (build on ASN RDA
initiative)
– Need Software-specific items within the system WBS to a level
to support the metrics
– Include software-specific criteria into checklist and entry/exit
criteria at system engineering technical review (SETR)
• Augment technology readiness assessment (TRA) deskbook
with a second set of software technology readiness levels
(TRL)
• Leverage the attention and focus provided by OSD-AT&L on
integration of systems and software engineering desire to
October
2007
provide
center of excellence ©USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects :
Biggest Opportunities(2)
• Use true technical reviews to communicate between
different disciplines, requiring the small set of the right
people to be there, small set of people
• Ask systems engineers to develop test procedures (test
group does not have the capacity to develop test
procedure)
• Model-based engineering
• Prototype policy using a true IRT
• For systems engineering requirements and architecting
processes, define and implement a quantitative set of
quality attributes and checklist for coverage of software
engineering
October 2007
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects :
Biggest Opportunities(3)
• Develop tools for automated measurement of systems sizing
metrics
• Use requirement document to construct end-to-end threads
to cover every requirement such as not just users but
sensors, in-bound radar data (JPL experience)
• Have systems engineers write test requirements, such as
“What do we have to show to sell off this requirement”
• Combine earned value systems with agile development
approach
• Systems engineering to co-own software engineering risks
• Explore automated generating code because we are unable
to keep up with the systems/software size growth trend
– Needs domain specific languages and domain specific models
– Need to validate the results of the translation; compilers are not
perfect
October 2007
©USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects: Inhibitors to progress
and how to overcome them (1)
Inhibitor
How to Overcome Inhibitor
“Never changing” earned value is required (reflects
spent costs)
An agile rebaselining process
Lack of communication between system
engineering and other types of engineering and
software engineering due to lack of overlapping of
educational background
Define and deliver 90-day “crash courses” on
software to non-software engineer and on
physics/math/etc. to software people; small group
of key people
Culture: company
1.
2.
Verifiable demonstrated prototype evaluated by
a true IRT
Properly selected and trained program
manager who appreciate software, hardware
and people; must be managed from the system
perspective
Culture: government; a history of predicating policy
on unvalidated pilot, e.g., acquisition reform, CMM,
CMMI, and DoD BMP (business management plan) ;
Prototype policy using a IRT
Culture: congress; Not interested in open, e.g., cost
plus or incremental contract (want fixed price)
Demonstrated success and a selling campaign to
explain same
Culture: engineers
Define new lexicon of terminology; same words
mean different things to SE and SW
Communication; lack of prioritization, priorities
commonly
poorly
October
2007 communicated
©USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects: Inhibitors to
progress and how to overcome them (2)
Inhibitor
How to Overcome Inhibitor
Large differences in technical background,
scope, focus, and priority
Do the old fashioned way mentoring
TRA Deskbook software TRLs deal with
domains and algorithms but do not address
the development of software with respect to
systems
Develop system applicable
TRL and TRA for software (and an
assessment process)
Separate, overly complex, poorly
synchronized software engineering and
systems engineering life cycles
Replace traditional system life cycle, e.g.,
1521 with an integrated systems and
software life cycle
Failure by OSD to include software-specific
guidance in their system engineering plan
(SEP) guidance
Traditional waterfall acquisition model is not
as flexible as would be optimal for
opportunities such as ICM
October 2007
Prototyping different acquisition process
©USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
Process Model Principles (0)
1.
2.
3.
Commitment and accountability
Success-critical stakeholder satisficing
Incremental growth of system definition and
stakeholder commitment
4, 5. Concurrent, iterative system definition and
development cycles
Cycles can be viewed as sequential concurrentlyperformed phases or spiral growth of system
definition
6.
October 2007
Risk-based activity levels and anchor point
commitment milestones
©USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects :
Ability of six principles to improve IS&SE?
•
•
•
•
•
•
Step 1 through 5 are what we do now
Step 1 through 5 are things that we do now, but I question the level of rigor,
such as original Fagan Inspections required a high level of rigor
Risk-focused anchor point decisions are a great value-added concept of
ICM. Unfortunately, most programs perform risk identification and
management activities poorly
Software has multiple, overlapping risk reduction possibilities but systems
engineering risk mitigation plans have sometimes lacked integration; they
do not reflect the overlap
Risk-focused anchor point decisions duplicate what is already present and
practice in current DoD risk management policies. What is lacking is a
rigorous adherence to establish entry and exit criteria
General concern that the six principles are already in place but lack rigor
–
•
ICM could introduce/bring about/reinforce a greater degree of rigor and formalism
The rational that lead to the six principles is based on 19 years of misuse of
using spiral model, as arrive by expert group responding to explaining the
intended use of the spiral to the general accounting office (GAO)
October 2007
©USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
Process Model Principles (1)
1.
Commitment and accountability
•
•
2.
3.
4, 5.
Should be supported, not prohibited, by RFP
Should accommodate changes driven by risks
Success-critical stakeholder satisficing
Incremental growth of system definition and stakeholder commitment
Concurrent, iterative system definition and development cycles
Cycles can be viewed as sequential concurrently-performed phases or spiral
growth of system definition
6.
October 2007
Risk-based activity levels and anchor point commitment milestones
©USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
Process Model Principles (2)
1.
Commitment and accountability
2.
Success-critical stakeholder satisficing
•
•
•
•
•
•
•
3.
4, 5.
Require access to success critical stakeholder and frequently they
are in short supply
Potential concern that emergent behavior by success critical
stakeholder or non success critical stakeholder can be lost. Final
system may not do what really needs to be done.
Satisficing is a condition that may not be achievable; therefore
• Kill the program early
• Create conditions where satisficing is achievable
• Push forward regardless e.g. eliminate success critical
stakeholder or criteria or knowingly proceed on the path to
partial success
Concept of satisficing requires Openness and honesty regarding
win conditions
Success critical stakeholder are milestone dependent
Need to build in flexibility accommodate/ anticipate the changing
mind of success critical stakeholders
Keep stakeholders involved and continue to identify new ones
Incremental growth of system definition and stakeholder commitment
Concurrent, iterative system definition and development cycles
Cycles can be viewed as sequential concurrently-performed
phases or spiral growth of system definition
October 2007
©USC-CSSE
6.
Risk-based activity levels and anchor point commitment milestones
14
University of Southern California
Center for Systems and Software Engineering
Process Model Principles(3)
1.
2.
Commitment and accountability
Success-critical stakeholder satisficing
3.
Incremental growth of system definition and stakeholder
commitment
•
•
4, 5.
Improve method for quantifying the system/software size at
the concept exploration phase
Define representations to capture integrated systems /
software artifacts such as specifications/architectures
/implementations
• Imagine a future where there are executable artifacts
delivered early in the lifecycle (other than code )such as
executable specifications and executable architectures
• Executable specification captures nominal behavioral
properties but not necessarily / off-nominal or realistic
efficiency/fault-tolerance needs
Concurrent, iterative system definition and development cycles
Cycles can be viewed as sequential concurrently-performed phases or spiral
growth of system definition
6.
October 2007
Risk-based activity levels and anchor point commitment milestones
©USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
Process Model Principles(4&5)
1.
2.
3.
Commitment and accountability
Success-critical stakeholder satisficing
Incremental growth of system definition and stakeholder commitment
4, 5. Concurrent, iterative system definition and
development cycles
Cycles can be viewed as sequential concurrentlyperformed phases or spiral growth of system
definition
– Impossible to do concurrent, iterative
definition/development unless you have got
stakeholders involved concurrently and iteratively
– Requires agile program management method
6.
October 2007
Risk-based activity levels and anchor point commitment milestones
©USC-CSSE
16
University of Southern California
Center for Systems and Software Engineering
Process Model Principles(6)
1.
2.
3.
4, 5.
Commitment and accountability
Success-critical stakeholder satisficing
Incremental growth of system definition and stakeholder commitment
Concurrent, iterative system definition and development cycles
Cycles can be viewed as sequential concurrently-performed phases or spiral growth of
system definition
6.
Risk-based activity levels and anchor point
commitment milestones
•
•
•
•
•
October 2007
Need to have greater emphasis on feasibility rationale
Tom Schroeder’s presentation on difficulty of finding
independent experts
Milestone definition and feasibility rationale are
independent;
Feasibility rationale could be used to measure
progress
Define additional metaphors (engage, marriage, kids,
and gambling) ©USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects:
Ability of ICM to improve IS&SE? (1)
•
•
The rubber will hit the road when you need to define the specific
deliverables capabilities in the first increment, how long it takes to get
there?
Many people may feel that there is nothing new here and “we are already
doing this”
–
–
–
•
•
•
•
They often overlook on minimizing the risk-driven aspects
Seldom have explicit evidence of feasibility
Main point of ICM is that feasibility rationale became a first class citizen
Define a detailed plan for prototyping/piloting, evaluating, and deploying
(initial, medium, large scale) ICM including success criteria and a business
case analysis
general concern of premature focus on ICM as a point solution as opposed
to the other way to implement the six principles
Is ICM actually being proposed as a point solution or is this just a way to try
something new on a small scale (need strategic positioning on ICM)
To make ICM work, it requires the funding agency and the LSI / prime
contractor to do much more than they are currently doing such as con ops
development, testbed development, frequent evaluation of program status
and making adjustments, gathering information to evaluate
subcontractors/suppliers, and team building
–
LSI/prime contractor must engage the developer earlier in the process, so they do
not end up with a poor con ops or design
October 2007
©USC-CSSE
18
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects:
Ability of ICM to improve IS&SE? (2)
•
•
•
Could/should feasibility rationale be developed in the same way that the
safety and assurance cases ?
Can the guidance for those support the feasibility rationale development?
Could the FR be seen as uber-case that organizes and gathers the
information from both systems and software specific attribute cases as well
as the business and stakeholder satisfaction cases?
October 2007
©USC-CSSE
19
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects:
What else is needed? (1)
• Technology/management research
– Create a catalog of successful integrated systems and software
architectural patterns, will help get systems engineering to think
about what they need to do with software
– Model-driven approach for integrated systems and software
engineering
– Simulation-based acquisition approaches, e.g., processes,
methods, tools for integrated systems and software engineering
– Launch a major DoD funded initiative for creating joint
university-industry-government of research centers to address
integrated systems and software engineering to catalyze
research breakthroughs
– Apply, and further research where necessary social science to
overcome cultural barrier and facilitate team interaction, e.g.,
common training, symbolic language
October 2007
©USC-CSSE
20
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects:
What else is needed? (2)
• Education and training
– Need hands-on manager as a part of the education / training
process, e.g., six sigma training “too abstract and not applicable
to the work” to fine tune the process
– Need for personnel that have expertise in multiple domain such
as software engineer that also know
aerospace/autos/automobile/telecom
– The regular engineer need exposure to software engineering
– At what level is the expertise needed (MS level? )
• Awareness would be an improvement
October 2007
©USC-CSSE
21
University of Southern California
Center for Systems and Software Engineering
IS&SE for Technical Aspects:
What else is needed? (3)
• Regulations, specifications and standards
– Real consequences for a non-compliance with policy or
incentive structures (carrot vs 2x4)
– Relieve those who are forced to follow policy that leads to
failure from the consequences of the failure (JPL failure for 2
MARS missions that were failure as a consequence of faster
better cheaper policy)
– “None before their time”, e.g., before understood and their value
demonstrated
• Other?
– Prototype technical policy
– Senate staffers at the sub committee level reduced the FCS
program proposed fiscal year 08 budget by $ 635 M primarily
from system engineering and management accounts. WA HA
– Need root cause analysis and action plan to prevent for future
October 2007
©USC-CSSE
case
22