Managing the Engineering of
Systems
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA 15213
Brian P. Gallagher, Director
Acquisition Support
29 October, 2007
© 2007 Carnegie Mellon University
Contents
Engineering Management
Software Engineering is Systems Engineering
Software Engineering is Different
Back to Basics: Principles of Effective Engineering Management
Three Important Things
Conclusion
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
2
Engineering Management
Managing complex engineering projects requires:
•
Ability to manage customers
•
Ability to understand technical complexities
•
Ability to understand team complexities
•
Ability to inspire and lead
•
Ability to negotiate
•
Ability to deliver
•
Ability to plan, re-plan, re-start, and close-out
•
Situational awareness
•
Luck
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
3
Knowledge Evolves: Acquirer’s View
MS B
AoA
Plan
Tech
Devel
Strat
Develop
Acquisition Strategy
Acq.
Strategy
v1.0
Stakeholders
Refine
Acquisition Strategy
Acq.
Strategy
v2.0
Stakeholders
Improve
Process
Evaluate
Incremental
Progress
Refine
Acquisition Strategy
Improve
Process
Execute
Acquisition
Strategy
Evaluate
Incremental
Progress
Execute
Refined
Acquisition
Strategy
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
4
Knowledge Evolves: Developer’s View
Traditional Approach
(Waterfall Development)
Evolutionary Approaches
System
Context
System
Context
Marketplace,
Re-use
• COTS products
• NDI
Known Rqmnts
requirements
cost
schedule
business processes
operational
procedures, etc.
Simultaneous
Definition
and Tradeoffs
Architecture &
Design
Implementation
•
•
•
•
•
Architecture &
Design
• Strongly
influenced by
products and
standards
Buy, Re-use, Build, Integrate, Refresh
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
5
Complex Operational Environments
Require Complex System Solutions
Manned Ground Vehicles (MGV)
Mounted Combat
System (MCS)
XM1202
Infantry Carrier
Vehicle (ICV)
Unmanned Aerial Systems (UAS)
Command and
Control Vehicle
Class I UAV
(XM156)
(C2V) XM1209
XM1206
Class IV UAV
(MQ-8B)
Centralized
Controller
Reconnaissance
APS
Unattended Ground Systems (UGS)
And Surveillance
U-UGS
U-UGS
(AN/GSR-10 (T)) (AN/GSR-9 (U))
Vehicle (RSV)
XM1201
Common
Non-Line of
Chassis
Sight Mortar
(NLOS-M)
XM1204
Tactical and Urban
Unattended
Non-Line of Sight
Launch System
Ground Sensors
(NLOS-LS) XM501
Non-Line of
Sight Cannon
Unmanned Ground Vehicles (UGV)
(NLOS-C)
XM1203
MULE-C
MULE-T
Medical Vehicle
FCS Recovery and
Small UGV (SUGV)
Treatment (MV-T)
XM1208
Maintenance
Vehicle (FRMV)
XM1205
Medical Vehicle
Evacuation (MV-E)
XM1207
Armed Robotic
Vehicle – Assault
(Light) (ARV-A-L)
Multifunction Utility/
Logistics
and Equipment
Countermine and
Transport
17 Jul 07
Voyages of Discovery, not Well-Defined Programs
Software Engineering is
Systems Engineering
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
9
What is a System?
… a collection of different things which together produce results
unachievable by the elements alone
•
The Art of System Architecting, 2nd edition, Maier and Rechtin
… an integrated composite of people, products, and processes that
provide a capability to satisfy a stated need or objective
•
Systems Engineering Fundamentals, Defense Acquisition University Press
… a set of complementary, interacting parts with properties, capabilities
and behaviors emerging both from the parts and from their interactions
•
Hitchins, Derek, http://www.hitchins.co.uk/WCES.html
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
10
Systems Engineering is …
…an interdisciplinary, collaborative approach that derives, evolves, and
verifies a life-cycle balanced system solution which satisfies customer
expectations and meets public acceptability
•
(Reference: IEEE P1220)
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
11
Software Engineering is …
…the design, development, and documentation of software by applying
technologies and practices from computer science, project
management, engineering, application domains, interface design,
digital asset management and other fields
•
(Reference: Wikipedia)
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
12
Systems Engineering: “I want” to “I got”
Operational
Need
Systems Engineering
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
13
Software Engineering: “I want” to “I got”
Operational
Need
Software Engineering
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
14
Effective Engineering Managers Employ Software-Aware
Systems Engineering: “I want” to “I got”
Operational
Need
Software-Aware
Systems
Engineering
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
15
Software as a “Product”; Software Engineering as
a “Discipline”
Software-Aware
Systems Engineering
Software Products
Software Engineering
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
16
Software is Different
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
17
Realities of Software
The flowchart might correspond to a 100
LOC module with a single loop that may be
executed no more than 20 times.
There are approximately 1014 possible paths
that may be executed!
For any but the smallest programs, complete
path coverage for defect detection is
impractical.
Loop < 20 times
Adapted from Pressman, R.S., Software Engineering: A Practitioner’s Approach,
Third Edition, McGraw Hil, 1992
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
18
Challenges of Software
Typical Industry Software Quality at Delivery
• A 1,000 line-of-code (1 KLOC) program listing has about 20
pages of executable code
• For industrial software, typical shipped quality levels are 5 to
10 defects per KLOC or 1 defect in 2 – 4 pages
• A 1 million line-of-code (1 MLOC) printed listing stands
roughly 5’7” and contains between 5,000 to 10,000 defects
when shipped
For DoD acquisition programs, these realities are often
ignored resulting in unrealistic schedules and unplanned
test/fix cycles inserted to grow the reliability of low quality
software.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
19
What Does 17M ESLOC Look Like?
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
20
How do we Grow the Reliability?
Do we expect between 85,000 and 170,000 defects to still be present after
FQT?
Do we plan to discover, fix, re-test or do we plan to “prove functionality”
(Green Light Integration and Test)?
Do systems engineers without a foundation in software engineering
understand this unique aspect of the complexities of software-intensive
systems and latent defects that will be present?
What are the consequences?
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
21
V-22 Osprey
A V-22 Osprey crashed on December
11, 2000.
• Four marines were killed.
• The problem was traced to a software
defect.
V-22 software had been exhaustively
tested.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
22
Software is Challenging
The complexities, the interactions, the flexibility, and our inability to grasp
how difficult building software-intensive systems places us at a crossroads
New techniques are needed but are years away
We refuse to accept this and grasp at “Silver Bullets”
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
23
Back to Basics:
Principles of Effective Management
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
24
The Buzzword Quagmire and Quest for the “Silver
Bullet”
Open Systems
Interoperability
DoDAF
FEAF
ATAM
Acquisition Reform
Total System Performance Responsibility
Time-Certain Development
Agile Acquisition
Win-Win Spiral
Evolutionary Acquisition
Extreme Programming
Capability-Based Acquisition
Lean Six Sigma
CMMI
Team Software Process
Insight versus Oversight
Net-Centric Warfare
Service-Based Acquisition
Open Architecture
Service-Oriented Architecture
Incremental Commitment Model
Earned-Value
Architecture-based Development
Lean Acquisition
Systems Engineering Revitalization
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
25
How to make sense? Principle-Based Decisions
“Principle” Defined:
The collectivity of moral or ethical standards or judgments: a
decision based on principle rather than expediency.
Decisions to pursue a given management approach should be grounded
on underlying principles designed to increase the effectiveness of
acquiring, developing, and deploying systems to the end user
The following describes the Seven Principles of Effective Engineering
Management
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
26
Seven Principles of Effective Engineering
Management
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
27
The Core Principle: Open
Communication
Encouraging free flowing information at
and between all stakeholders.
Enabling formal, informal, and
impromptu communication.
Using consensus-based processes that value the
individual voice (bringing unique knowledge and
insight to evolving mission capabilities).
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
28
The Three Sustaining Principles
Team Risk Management
Continuous Process Improvement
Continuous Product Improvement
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
29
Team Risk Management
Evolving the warfighter’s capabilities
by continuously mitigating
operational, development,
and acquisition risks.
All stakeholders participating in managing
the project by managing the risks.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
30
Continuous Process Improvement
Maturing the acquisition, development,
and operational processes to meet
the warfighter’s objectives.
Employing a common process improvement
framework and language to align and enhance
process capability.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
31
Continuous Product Improvement
Enhancing the warfighter’s mission
through evolutionary delivery
of enhanced capabilities.
Delivering an initial capability on the first
promise date, with the demonstrated capability
to deliver improved or updated capability on a
regular, dependable schedule.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
32
The Three Defining Principles
Forward-Looking View
Global Perspective
Shared Product Vision
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
33
Forward-Looking View
Seeing a common tomorrow against
which all stakeholders can measure
potential breakthroughs and risks.
Managing project resources and activities while
anticipating uncertainties.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
34
Global Perspective
Sharing a single mental model of project
success that crosses all boundaries
between acquirer, developer,
and operator.
Viewing enhancements within the
context of the operational
mission.
Recognizing both the potential value of
opportunity and the potential impact of adverse
effects.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
35
Shared Product Vision
Developing and sustaining a common
conception of the product being built one that can be stated simply and
briefly, and is founded on common
purpose, shared ownership, and
collective commitment among the
stakeholders.
Focusing on results.
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
36
Three Important Things:
Scope, Focus, Deliver!
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
37
Three Important Things
Scope the problem – “What”
Focus the team – “How”
Deliver value – “When and Where”
Everything else is just noise!
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
38
RUP/ICM Anchor Points Enable Concurrent Engineering
Scope
Focus
Deliver
Conclusions
Management of engineering project has become more complex as
demand grows and technology tries to keep up
Software and Systems Engineering employ defined processes to get from
“I want” to “I got”
Managers need to ensure their early Systems Engineering efforts are
informed by Software Engineering issues (Software-Aware Systems
Engineering)
Software has some unique properties including logical complexities that
lead to an inherently defect laden product
Software and Systems Engineering professionals need to be wary of
“Buzzwords” and “Silver Bullets”
All effort should help Scope the problem, Focus the team, and Deliver
value
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
40
Contact Information
Air Force
John Foreman, jtf@sei.cmu.edu
Brian Gallagher
Director, Acquisition Support Program
Software Engineering Institute
4500 Fifth Ave.
Pittsburgh, PA 15213-3890
(412) 268-7157
bg@sei.cmu.edu
Army
Cecilia Albert, cca@sei.cmu.edu
Navy
Rick Barbour, reb@sei.cmu.edu
Intelligence Community
Rita Creel, rc@sei.cmu.edu
Civil Agencies
Steve Palmquist, msp@sei.cmu.edu
http://www.sei.cmu.edu/programs/acquisition-support
SEI Presentation (Basic)
Author, Date
© 2007 Carnegie Mellon University
41