Managing the Engineering of Systems Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Brian P. Gallagher, Director Acquisition Support 29 October, 2007 © 2007 Carnegie Mellon University Contents Engineering Management Software Engineering is Systems Engineering Software Engineering is Different Back to Basics: Principles of Effective Engineering Management Three Important Things Conclusion SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 2 Engineering Management Managing complex engineering projects requires: • Ability to manage customers • Ability to understand technical complexities • Ability to understand team complexities • Ability to inspire and lead • Ability to negotiate • Ability to deliver • Ability to plan, re-plan, re-start, and close-out • Situational awareness • Luck SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 3 Knowledge Evolves: Acquirer’s View MS B AoA Plan Tech Devel Strat Develop Acquisition Strategy Acq. Strategy v1.0 Stakeholders Refine Acquisition Strategy Acq. Strategy v2.0 Stakeholders Improve Process Evaluate Incremental Progress Refine Acquisition Strategy Improve Process Execute Acquisition Strategy Evaluate Incremental Progress Execute Refined Acquisition Strategy SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 4 Knowledge Evolves: Developer’s View Traditional Approach (Waterfall Development) Evolutionary Approaches System Context System Context Marketplace, Re-use • COTS products • NDI Known Rqmnts requirements cost schedule business processes operational procedures, etc. Simultaneous Definition and Tradeoffs Architecture & Design Implementation • • • • • Architecture & Design • Strongly influenced by products and standards Buy, Re-use, Build, Integrate, Refresh SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 5 Complex Operational Environments Require Complex System Solutions Manned Ground Vehicles (MGV) Mounted Combat System (MCS) XM1202 Infantry Carrier Vehicle (ICV) Unmanned Aerial Systems (UAS) Command and Control Vehicle Class I UAV (XM156) (C2V) XM1209 XM1206 Class IV UAV (MQ-8B) Centralized Controller Reconnaissance APS Unattended Ground Systems (UGS) And Surveillance U-UGS U-UGS (AN/GSR-10 (T)) (AN/GSR-9 (U)) Vehicle (RSV) XM1201 Common Non-Line of Chassis Sight Mortar (NLOS-M) XM1204 Tactical and Urban Unattended Non-Line of Sight Launch System Ground Sensors (NLOS-LS) XM501 Non-Line of Sight Cannon Unmanned Ground Vehicles (UGV) (NLOS-C) XM1203 MULE-C MULE-T Medical Vehicle FCS Recovery and Small UGV (SUGV) Treatment (MV-T) XM1208 Maintenance Vehicle (FRMV) XM1205 Medical Vehicle Evacuation (MV-E) XM1207 Armed Robotic Vehicle – Assault (Light) (ARV-A-L) Multifunction Utility/ Logistics and Equipment Countermine and Transport 17 Jul 07 Voyages of Discovery, not Well-Defined Programs Software Engineering is Systems Engineering SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 9 What is a System? … a collection of different things which together produce results unachievable by the elements alone • The Art of System Architecting, 2nd edition, Maier and Rechtin … an integrated composite of people, products, and processes that provide a capability to satisfy a stated need or objective • Systems Engineering Fundamentals, Defense Acquisition University Press … a set of complementary, interacting parts with properties, capabilities and behaviors emerging both from the parts and from their interactions • Hitchins, Derek, http://www.hitchins.co.uk/WCES.html SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 10 Systems Engineering is … …an interdisciplinary, collaborative approach that derives, evolves, and verifies a life-cycle balanced system solution which satisfies customer expectations and meets public acceptability • (Reference: IEEE P1220) SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 11 Software Engineering is … …the design, development, and documentation of software by applying technologies and practices from computer science, project management, engineering, application domains, interface design, digital asset management and other fields • (Reference: Wikipedia) SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 12 Systems Engineering: “I want” to “I got” Operational Need Systems Engineering SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 13 Software Engineering: “I want” to “I got” Operational Need Software Engineering SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 14 Effective Engineering Managers Employ Software-Aware Systems Engineering: “I want” to “I got” Operational Need Software-Aware Systems Engineering SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 15 Software as a “Product”; Software Engineering as a “Discipline” Software-Aware Systems Engineering Software Products Software Engineering SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 16 Software is Different SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 17 Realities of Software The flowchart might correspond to a 100 LOC module with a single loop that may be executed no more than 20 times. There are approximately 1014 possible paths that may be executed! For any but the smallest programs, complete path coverage for defect detection is impractical. Loop < 20 times Adapted from Pressman, R.S., Software Engineering: A Practitioner’s Approach, Third Edition, McGraw Hil, 1992 SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 18 Challenges of Software Typical Industry Software Quality at Delivery • A 1,000 line-of-code (1 KLOC) program listing has about 20 pages of executable code • For industrial software, typical shipped quality levels are 5 to 10 defects per KLOC or 1 defect in 2 – 4 pages • A 1 million line-of-code (1 MLOC) printed listing stands roughly 5’7” and contains between 5,000 to 10,000 defects when shipped For DoD acquisition programs, these realities are often ignored resulting in unrealistic schedules and unplanned test/fix cycles inserted to grow the reliability of low quality software. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 19 What Does 17M ESLOC Look Like? SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 20 How do we Grow the Reliability? Do we expect between 85,000 and 170,000 defects to still be present after FQT? Do we plan to discover, fix, re-test or do we plan to “prove functionality” (Green Light Integration and Test)? Do systems engineers without a foundation in software engineering understand this unique aspect of the complexities of software-intensive systems and latent defects that will be present? What are the consequences? SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 21 V-22 Osprey A V-22 Osprey crashed on December 11, 2000. • Four marines were killed. • The problem was traced to a software defect. V-22 software had been exhaustively tested. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 22 Software is Challenging The complexities, the interactions, the flexibility, and our inability to grasp how difficult building software-intensive systems places us at a crossroads New techniques are needed but are years away We refuse to accept this and grasp at “Silver Bullets” SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 23 Back to Basics: Principles of Effective Management SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 24 The Buzzword Quagmire and Quest for the “Silver Bullet” Open Systems Interoperability DoDAF FEAF ATAM Acquisition Reform Total System Performance Responsibility Time-Certain Development Agile Acquisition Win-Win Spiral Evolutionary Acquisition Extreme Programming Capability-Based Acquisition Lean Six Sigma CMMI Team Software Process Insight versus Oversight Net-Centric Warfare Service-Based Acquisition Open Architecture Service-Oriented Architecture Incremental Commitment Model Earned-Value Architecture-based Development Lean Acquisition Systems Engineering Revitalization SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 25 How to make sense? Principle-Based Decisions “Principle” Defined: The collectivity of moral or ethical standards or judgments: a decision based on principle rather than expediency. Decisions to pursue a given management approach should be grounded on underlying principles designed to increase the effectiveness of acquiring, developing, and deploying systems to the end user The following describes the Seven Principles of Effective Engineering Management SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 26 Seven Principles of Effective Engineering Management SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 27 The Core Principle: Open Communication Encouraging free flowing information at and between all stakeholders. Enabling formal, informal, and impromptu communication. Using consensus-based processes that value the individual voice (bringing unique knowledge and insight to evolving mission capabilities). SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 28 The Three Sustaining Principles Team Risk Management Continuous Process Improvement Continuous Product Improvement SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 29 Team Risk Management Evolving the warfighter’s capabilities by continuously mitigating operational, development, and acquisition risks. All stakeholders participating in managing the project by managing the risks. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 30 Continuous Process Improvement Maturing the acquisition, development, and operational processes to meet the warfighter’s objectives. Employing a common process improvement framework and language to align and enhance process capability. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 31 Continuous Product Improvement Enhancing the warfighter’s mission through evolutionary delivery of enhanced capabilities. Delivering an initial capability on the first promise date, with the demonstrated capability to deliver improved or updated capability on a regular, dependable schedule. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 32 The Three Defining Principles Forward-Looking View Global Perspective Shared Product Vision SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 33 Forward-Looking View Seeing a common tomorrow against which all stakeholders can measure potential breakthroughs and risks. Managing project resources and activities while anticipating uncertainties. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 34 Global Perspective Sharing a single mental model of project success that crosses all boundaries between acquirer, developer, and operator. Viewing enhancements within the context of the operational mission. Recognizing both the potential value of opportunity and the potential impact of adverse effects. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 35 Shared Product Vision Developing and sustaining a common conception of the product being built one that can be stated simply and briefly, and is founded on common purpose, shared ownership, and collective commitment among the stakeholders. Focusing on results. SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 36 Three Important Things: Scope, Focus, Deliver! SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 37 Three Important Things Scope the problem – “What” Focus the team – “How” Deliver value – “When and Where” Everything else is just noise! SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 38 RUP/ICM Anchor Points Enable Concurrent Engineering Scope Focus Deliver Conclusions Management of engineering project has become more complex as demand grows and technology tries to keep up Software and Systems Engineering employ defined processes to get from “I want” to “I got” Managers need to ensure their early Systems Engineering efforts are informed by Software Engineering issues (Software-Aware Systems Engineering) Software has some unique properties including logical complexities that lead to an inherently defect laden product Software and Systems Engineering professionals need to be wary of “Buzzwords” and “Silver Bullets” All effort should help Scope the problem, Focus the team, and Deliver value SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 40 Contact Information Air Force John Foreman, jtf@sei.cmu.edu Brian Gallagher Director, Acquisition Support Program Software Engineering Institute 4500 Fifth Ave. Pittsburgh, PA 15213-3890 (412) 268-7157 bg@sei.cmu.edu Army Cecilia Albert, cca@sei.cmu.edu Navy Rick Barbour, reb@sei.cmu.edu Intelligence Community Rita Creel, rc@sei.cmu.edu Civil Agencies Steve Palmquist, msp@sei.cmu.edu http://www.sei.cmu.edu/programs/acquisition-support SEI Presentation (Basic) Author, Date © 2007 Carnegie Mellon University 41