University of Southern California Center for Systems and Software Engineering Course Overview and ICSM Principle 1 CS 510 Software Management and Economics Fall 2015 Barry Boehm, USC Lecture delivered by Jim Alstad, USC University of Southern California Center for Systems and Software Engineering Outline • Course objective – Help you learn to be a successful software manager – For a career lasting through the 2050’s. • Software management learning objectives – What does a successful SW manager need to deal with? • Overview of Course – Programmatics, schedule, academic integrity • Current and future software and management challenges • ICSM Overview – Principle 1: Stakeholder value-based guidance • Enterprise Success Theorem • Enterprise Success Realization Theorem • This Week’s Assignment 8/24/2015 © USC-CSSE 2 University of Southern California Center for Systems and Software Engineering What Does A Successful Software Manager Need to Deal With? • People: customers, users, architects, designers, programmers, testers, lawyers, venture capitalists, suppliers, politicians, … • Products: requirements, designs, code, documentation, plans, tools, data, facilities, equipment, … • Projects: proposals, presentations, contracts, deliverables, budgets, schedules, milestones, … • Resources: time, money, space, communications, skills, … • Technology: software, hardware, domain technology, COTS, OSS, clouds, apps, widgets, big data… • Organizations and Cultures: top management, marketing, sales, development, finance, customer/user organizations, … • Changes in all of the above 8/24/2015 © USC-CSSE 3 University of Southern California Center for Systems and Software Engineering Outline • Course objective – Help you learn to be a successful software manager – For a career lasting through the 2050’s. • Software management learning objectives – What does a successful SW manager need to deal with? • Overview of Course – Programmatics, schedule, academic integrity • Current and future software and management challenges • ICSM Overview – Principle 1: Stakeholder value-based guidance • Enterprise Success Theorem • Enterprise Success Realization Theorem • This Week’s Assignment 8/24/2015 © USC-CSSE 4 University of Southern California Center for Systems and Software Engineering Comparison of CS 510 and CS 577a CS 510 CS 577a • VBSE Theory, Practice • COCOMO II Extensions • Microeconomics • ICSM Principles and Practices – Decision Theory • WinWin • Agile and Rapid – Risk Management Development •Planning & Control • People Management – COCOMO II • Business Case • 2 Midterms, Final Analysis 8/24/2015 © USC-CSSE • S/W - System Architecting • Operational Concept & Rqts. Definition – Winbook System – Prototyping • OO Analysis & Design – Visual Paradigm •Team Project (DEN: IV&V) 5 University of Southern California Center for Systems and Software Engineering CS 510 Course Schedule Overview • Aug 24 – Sep 21 • • • • • ICSM Principles and Practices, COCOMO II, Cost Estimation, Business Case Analysis Sep 23 Midterm Exam I Sep 28– Nov 4 Software Microeconomics, Risk Management, ICSM Stages and Phases Nov 9 Midterm Exam II Nov 11 – Dec 2 Contracting, Ethics, Maturity Models, Past and Future Trends Dec 9 Final Exam 8/24/2015 © USC-CSSE 6 University of Southern California Center for Systems and Software Engineering CS 510 Programmatics - I Basis of grade. Final Exam, 30%; 2 midterms: 20%; Homework exercises: 50%. Text. Boehm, Lane, Koolmanojwong, Turner, The Incremental Commitment Spiral Model, Addison Wesley, 2014 Instructor. Prof. Barry Boehm, SAL 328, (213) 740-8163, Fax (213) 740-4927; boehm@usc.edu Office hours: Monday and Wednesday, 11am -12noon or by appointment. Teaching Assistants and Office Hours. Jim Alstad, MW 10-11am or by appointment Anandi Hira, Thursday 2-4pm or by appointment Email: csci510@usc.edu Web page: http://csse.usc.edu/classes_wp/csci-510-fall-2015/ 8/24/2015 © USC-CSSE 7 University of Southern California Center for Systems and Software Engineering CS 510 Questionnaire and Acknowledgement Please fill out and return. Name: _________________________________________________ Student ID #: ___________________________________________ Dept./Degree Program: __________________________________ Job, Employer: _________________________________________ Software Work Experience (years): _______________________ Phone, fax numbers: ____________________________________ E-mail Address: ________________________________________ Acknowledgement: I acknowledge the importance of USC's academic integrity standards (with respect to plagiarism, referencing others' work, etc.), and agree to abide by them. Signature: ______________________________________________ 8/24/2015 © USC-CSSE 8 University of Southern California Center for Systems and Software Engineering Academic Integrity Acknowledgement • Single most-serious offense: Plagiarism – Using other people’s work without crediting them – Homework, exams, class exercises, individual assignments • Minor first offense: You lose one grade level – E.g., B+ instead of A- • Major first offense, or second offense: F for the course 8/24/2015 © USC-CSSE 9 University of Southern California Center for Systems and Software Engineering We are Serious About Plagiarism – And experienced in finding it ID 32 8 46 39 30 36 44 42 35 23 8/24/2015 Critique Individual Total Team Total 182 453.25 715.4 160 454 708.875 185 450 708.875 182 430.55 715.4 140 428.5 708.875 186 461.75 674.4 0 136 674.4 165 456.875 674.4 190 452.08 674.4 185 433.25 674.4 © USC-CSSE Total Final Grade 1168.65 B+ 1162.875 B+ 1158.875 B+ 1145.95 B+ 1137.375 B+ 1136.15 B+ 810.4 F 1131.275 B+ 1126.48 B+ 1107.65 B 10 University of Southern California Center for Systems and Software Engineering Outline • Course objective – Help you learn to be a successful software manager – For a career lasting through the 2050’s. • Software management learning objectives – What does a successful SW manager need to deal with? • Overview of Course – Programmatics, schedule, academic integrity • Current and future software and management challenges • ICSM Overview – Principle 1: Stakeholder value-based guidance • Enterprise Success Theorem • Enterprise Success Realization Theorem • This Week’s Assignment 8/24/2015 © USC-CSSE 11 University of Southern California Center for Systems and Software Engineering Current and Future Process Challenges-I The Four D’s: Dynamism, Dependability, Doubt, Diversity • Dynamism: Rapid pace of change – In competition, mission priorities, technology, widgets, apps, Commercial Off-the-Shelf (COTS), cloud services – Need incremental development to avoid obsolescence – Need concurrent vs. sequential processes – Need both prescience and rapid adaptability • Dependability: Always-on, never-fail systems – Need well-controlled, high-assurance processes – Need to synchronize and stabilize concurrency – Need to balance assurance and agility 8/24/2015 © USC-CSSE 12 University of Southern California Center for Systems and Software Engineering Dynamism: Increasing Pace of Change Technology change Related infrastructure and services change Marketplace dynamics Competition dynamics Organizational change Software is critical User agility aids critical 8/24/2015 © USC-CSSE 13 University of Southern California Center for Systems and Software Engineering Dependability: Cost of Downtime Survey • • • • • • • • • • Industry Sector Lost Revenue/Hour: 2000 Energy $2.8 million Telecommunications $2.0 million Manufacturing $1.6 million Financial Institutions $1.4 million Information Technology $1.3 million Insurance $1.2 million Retail $1.1 million Pharmaceuticals $1.0 million Banking $996,000 • Source: IT Performance Engineering & Measurement Strategies: Quantifying Performance Loss, Meta Group, October 2000. 8/24/2015 © USC-CSSE 14 University of Southern California Center for Systems and Software Engineering Current and Future Process Challenges-II The Four D’s: Dynamism, Dependability, Doubt, Diversity • Doubt: Emergence and human-intensiveness – – – – Requirements not pre-specifiable Budgets and schedules not pre-specifiable Need for evolutionary growth Need to manage uncertainty and risk • Diversity: Enterprises and systems of systems (SoS) – Integrated supply chain: strategic planning, marketing, merchandising, outsourcing, just-in-time manufacturing, logistics, finance, customer relations management – Over 50 separately evolving external systems or services – Need to satisfice among multiple stakeholders – Wide diversity of needed capabilities – No one-size-fits-all solutions or processes 8/24/2015 © USC-CSSE 15 University of Southern California Center for Systems and Software Engineering Doubt: The Cones of Uncertainty – Need incremental vs. one-shot development Uncertainties in competition, technology, organizations, mission priorities 8/24/2015 © USC-CSSE 16 University of Southern California Center for Systems and Software Engineering Diversity: The Proliferation of Choices Not just a million apps. A million great apps. Shopping the App Store is a great experience because it’s easy to find the apps you want — and to discover new apps you didn’t even know you wanted. Browse freely by category. Or shop collections of apps and games handpicked by experts. Apple reviews everything on the App Store to guard against malware, so you’re buying and downloading from a trusted source. 8/24/2015 © USC-CSSE 17 University of Southern California Center for Systems and Software Engineering Diversity: Avoid Procrustean Beds One-size-fits all policies and standards • Procrustes: Greek Mythology – Rogue smith and bandit – Hostel with one-size-fits-all bed – Guests too small: stretch them to fit – Guests too large: lop off the offending parts 8/24/2015 © USC-CSSE 18 University of Southern California Center for Systems and Software Engineering Software Procrustean Beds • Pure Waterfall, Vee: Fixed Price and Spec Contract – Lop off needed changes as requirements creep • Pure Agile: Easiest First; Dedicated On-Site Customer – Later scalability and assurance problems; single-failure point • Voice of the Customer: Accept All “Requirements” – Gold-plating; neglect voices of acquirer, developer, owner • Piling on Incompatible Constraints: No Way Out – Project Example: Waterfall, COTS, Ada, GOTS Reuse • Inflexible Standards: No Choice But Tailoring Down – MIL-STD-498: choice of 23, 6, or 1 DID denied • Overconstrained Maturity Models: Excluding Expertise – Software CMM: Exclude software group from system rqts. 8/24/2015 © USC-CSSE 19 University of Southern California Center for Systems and Software Engineering Current System Acquisition Methods Too easy to misinterpret as one-size-fits-all • V-Model1 • Spiral Model2 High level guidance assumes that acquirers have extensive acquisition experience... Without experience, too easy to misinterpret and auger in with disastrous results... 1 2 http://en.wikipedia.org/wiki/V-Model 8/24/2015 © USC-CSSE http://en.wikipedia.org/wiki/Spiral_model 20 University of Southern California Center for Systems and Software Engineering Procrustean Example: DoD Acquisition Process 8/24/2015 © USC-CSSE 21 University of Southern California Center for Systems and Software Engineering Outline • Course objective – Help you learn to be a successful software manager – For a career lasting through the 2050’s. • Software management learning objectives – What does a successful SW manager need to deal with? • Overview of Course – Programmatics, schedule, academic integrity • Current and future software and management challenges • ICSM Overview – Principle 1: Stakeholder value-based guidance • Enterprise Success Theorem • Enterprise Success Realization Theorem • This Week’s Assignment 8/24/2015 © USC-CSSE 22 University of Southern California Center for Systems and Software Engineering What is the ICSM? • Risk-driven framework for determining and evolving best-fit system life-cycle process • Integrates the strengths of phased and riskdriven spiral process models • Synthesizes together principles critical to successful system development – Stakeholder value-based guidance – Incremental commitment and accountability – Concurrent multidiscipline engineering – Evidence and risk-driven decisions Principles trump diagrams… Principles used by 60-80% of CrossTalk Top-5 projects, 2002-2005 8/24/2015 © USC-CSSE 23 University of Southern California Center for Systems and Software Engineering The Incremental Commitment Spiral Model Cumulative Level of Understanding, Product and Process Detail (Risk-Driven) Concurrent Engineering of Products and Processes OPERATION2 DEVELOPMENT3 FOUNDATIONS4 OPERATION1 DEVELOPMENT2 FOUNDATIONS3 DEVELOPMENT1 FOUNDATIONS2 FOUNDATIONS RISK-BASED STAKEHOLDER COMMITMENT REVIEW POINTS: VALUATION EXPLORATION 6 5 4 3 2 1 Opportunities to proceed, skip phases backtrack, or terminate Risk-Based Decisions Evidence-Based Review Content - A first-class deliverable - Independent expert review - Shortfalls are uncertainties and risks Acceptable Negligible Risk High, but Addressable 8/24/2015 Too High, Unaddressable 24 © USC-CSSE 1 Exploration Commitment Review 2 Valuation Commitment Review 3 Foundations Commitment Review 4 Development Commitment Review 5 Operations1 and Development2 Commitment Review 6 Operations2 and Development3 Commitment Review 24 University of Southern California Center for Systems and Software Engineering Incremental Commitment in Gambling • Total Commitment: Roulette – Put your chips on a number • E.g., a value of a key performance parameter – Wait and see if you win or lose • Incremental Commitment: Poker, Blackjack – Put some chips in – See your cards, some of others’ cards – Decide whether, how much to commit to proceed 8/24/2015 © USC-CSSE 25 University of Southern California Center for Systems and Software Engineering The Incremental Commitment Spiral Process: Phased View Anchor Point Milestones Synchronize, stabilize concurrency via FEDs Risk patterns determine life cycle process 8/24/2015 © USC-CSSE 26 University of Southern California Center for Systems and Software Engineering ICSM Activity Levels for Complex Systems 8/24/2015 © USC-CSSE 27 University of Southern California Center for Systems and Software Engineering Anchor Point Feasibility Evidence Descriptions • Evidence provided by developer and validated by independent experts that: If the system is built to the specified architecture, it will – Satisfy the requirements: capability, interfaces, level of service, and evolution – Support the operational concept – Be buildable within the budgets and schedules in the plan – Generate a viable return on investment – Generate satisfactory outcomes for all of the success-critical stakeholders • All major risks resolved or covered by risk management plans • Serves as basis for stakeholders’ commitment to proceed • Synchronizes and stabilizes concurrent activities Can be used to strengthen current schedule- or event-based reviews 8/24/2015 © USC-CSSE 28 University of Southern California Center for Systems and Software Engineering 8/24/2015 © USC-CSSE 29 University of Southern California Center for Systems and Software Engineering Outline • Course objective – Help you learn to be a successful software manager – For a career lasting through the 2050’s. • Software management learning objectives – What does a successful SW manager need to deal with? • Overview of Course – Programmatics, schedule, academic integrity • Current and future software and management challenges • ICSM Overview – Principle 1: Stakeholder value-based guidance • Enterprise Success Theorem • Enterprise Success Realization Theorem • This Week’s Assignment 8/24/2015 © USC-CSSE 30 University of Southern California Center for Systems and Software Engineering Principle 1 and Enterprise Success Theorem Stakeholder value-based guidance Theorem: Your enterprise will succeed if and only if it makes winners of your success-critical stakeholders • Proof of “if”: Everyone that counts is a winner. Nobody significant is left to complain. • Proof of “only if”: Nobody wants to lose. Prospective losers will refuse to participate, or will counterattack. The usual result is lose-lose. 8/24/2015 © USC-CSSE 31 University of Southern California Center for Systems and Software Engineering Win-lose Generally Becomes Lose-lose Proposed Solution “Winner” Loser Quick, Cheap, Sloppy Product Developer & Customer User Lots of “bells and whistles” Developer & User Customer Driving too hard a bargain Customer & User Developer Actually, nobody wins in these situations 8/24/2015 © USC-CSSE 32 University of Southern California Center for Systems and Software Engineering Enterprise Success Realization Theorem Theorem: Your enterprise can realize success if and only if 1. You identify and involve all of the success critical stakeholders (SCSHs) – 2. Dependency theory You determine how the SCSHs want to win – 3. Utility theory You help the SCSHs determine and commit to a win-win course of action and solution – 4. Decision theory You adaptively control the course of action to continue to realize a win-win solution – 8/24/2015 Control theory © USC-CSSE 33 University of Southern California Center for Systems and Software Engineering Case Study: Personnel Assignment • You are running a project to develop a supply chain management system for a company in Boston – Need to select a system engineer (SE) to work with the client people in Boston. – Two primary candidates, Ann and George. – Both are equally capable , and very much want the job. • Here is how a Theory X, Y, or Z manager would likely decide: – Theory X. I'm the boss. George was a good friend and classmate at USC. I'll give him the job. – Theory Y. I'll ask them for their most creative suggestions for doing the job, and pick the most creative. – Theory Z. We are a team, and don't want any favoritism. Each should have an equal chance. I'll flip a coin to decide. • All three produce win-lose outcomes. How could you come up with a win-win approach? 8/24/2015 © USC-CSSE 34 University of Southern California Center for Systems and Software Engineering Personnel Selection: A Win-Win Approach • Step 1. Identify the success-critical stakeholders (SCSHs): Ann and George • Step 2. Determine how they want to win – Ann: I'd like a career path to marketing, and the SE job would be a good step in that direction. – George: my daughter is just starting college in the Boston area, and the Boston job would be an ideal way to keep in touch with her, along with being professionally satisfying. • Step 3. Help the SCSHs find a win-win situation – Find a comparable marketing job for Ann, and a comparable Boston job for George – Determine which selection works best for both Ann and George 8/24/2015 © USC-CSSE 35 University of Southern California Center for Systems and Software Engineering Principle 1 Failure Story: The Too-Good Road Surface Assessment Robot • Success-Critical Stakeholders – Roadbuilding Company • Top Management: Profitability, Reputation • Roadbuilders: Efficiency, Quality • Quality Assurance: Quality, Reputation – Carnegie Mellon U. Robotics institute • Strong robotics research, transition to usage • Improvement Opportunity – Manual road assessment expensive, inaccurate – Robotic technology could cut costs, catch defects 8/24/2015 © USC-CSSE 36 University of Southern California Center for Systems and Software Engineering Road Surface Assessment Project • Year 1. Alternative Operational Concepts, Designs – Best design: likely 100:1 cost savings, improved accuracy • Year 2: Selection of Components, Detailed Design • Year 3: Robot Development and Test – 100:1 cost and time savings, all deviations caught – 100x more deviations; 99% non-threatening – Requirement to report all deviations threatened reputation; unacceptable to quality assurance, top management • Net result: Cancellation of production project – Huge technical success; sociotechnical failure – Kept in storage for 5 years; revived under new criteria 8/24/2015 © USC-CSSE 37 University of Southern California Center for Systems and Software Engineering Principle 1 Success Story: Symbiq Medical Infusion Pump Winner of 2006 HFES Best New Design Award Described in NRC HSI Report, Chapter 5 8/24/2015 © USC-CSSE 38 University of Southern California Center for Systems and Software Engineering Symbiq IV Pump ICSM Process - I • Exploration Phase – – – – Stakeholder needs interviews, field observations Initial user interface prototypes Competitive analysis, system scoping Commitment to proceed • Valuation Phase – – – – – Feature analysis and prioritization Display vendor option prototyping and analysis Top-level life cycle plan, business case analysis Safety and business risk assessment Commitment to proceed while addressing risks 8/24/2015 © USC-CSSE 39 University of Southern California Center for Systems and Software Engineering Symbiq IV Pump ICSM Process - II • Foundations Phase – – – – – – Modularity of pumping channels Safety feature and alarms prototyping and iteration Programmable therapy types, touchscreen analysis Failure modes and effects analyses (FMEAs) Prototype usage in teaching hospital Commitment to proceed into development • Development Phase – – – – Extensive usability criteria and testing Iterated FMEAs and safety analyses Patient-simulator testing; adaptation to concerns Commitment to production and business plans 8/24/2015 © USC-CSSE 40 University of Southern California Center for Systems and Software Engineering Principles Satisfaction: Symbiq IV Pump 1. Stakeholder value-based guidance – Extensive involvement of multiple stakeholders – Extensive use of prototyping, safety analysis methods 2. Incremental commitment and accountability – Expanding system definition and evidence elaboration – Decision to start with composable 1- and 2-channel pumps 3. Concurrent multidiscipline engineering – Concurrent evaluation of display, alarm, pump suppliers – Concurrent definition, evaluation of safety and business cases 4. Evidence and risk-driven decisions – Evidence-based reviews of technical and business feasibility – Outstanding risks covered by next-phase risk mitigation plans 8/24/2015 © USC-CSSE 41 University of Southern California Center for Systems and Software Engineering First Homework: Due Wed. Sept. 2 • Identify the multiple success-critical stakeholders of the Hospira Infusion Pump project, and good examples of Principle 1 practiced in the project. • Identify approaches for avoiding the failure of the Too Good Robot project – Concise bulleted recommendations acceptable 8/24/2015 © USC-CSSE 42 University of Southern California Center for Systems and Software Engineering References - I • • • • • • • • • • • • • • Beck, K., Extreme Programming Explained, Addison Wesley, 1999. Boehm, B., "Some Future Software Engineering Opportunities and Challenges," In Sebastian Nanz (Ed.): The Future of Software Engineering, Springer Berlin Heidelberg, 2011, pp. 1-32. Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of SoftwareIntensive Systems of Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004. Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century SoftwareIntensive Systems of Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006. Boehm, B., and Lane, J., “Using the ICSM to Integrate System Acquisition, Systems Engineering, and Software Engineering,” CrossTalk, October 2007, pp. 4-9. Boehm, B., and Lane, J., “A Process Decision Table for Integrated Systems and Software Engineering,” Proceedings, CSER 2008, April 2008. Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000. Boehm, B. and Lane, J., "Evidence-Based Software Processes," New Modeling Concepts for Today's Software Processes, Springer Lecture Notes in Computer Science, 2010, Volume 6195/2010, pp. 62-73. Boehm, B., Lane, J., Koolmanojwong, S., and Turner, R., “An Evidence-Based SE Data Item Description,” Proceedings, CSER 2013, Elsevier, www.sciencedirect.com Boehm, B., Lane, J., Koolmanojwong, S., and Turner, R., The Incremental Commitment Spiral Model: Principles and Practices for Successful Systems and Software, Addison Wesley, 2014. Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2nd ed., 1999). Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System Highsmith, J., Adaptive Software Development, Dorset House, 2000 Huang, L., Boehm, B., Hu, H., Lv,J., and Qian, C., “Applying Value-Based Software Process: an ERP Example,” Intl. J. Software and informatics, July 2008, pp. 1-15. 8/24/2015 Copyright © USC-CSSE © USC-CSSE 43 University of Southern California Center for Systems and Software Engineering References -II •International Standards Organization, Information Technology Software Life Cycle Processes, ISO/IEC 12207, 1995 •ISO, Systems Engineering – System Life Cycle Processes, ISO/IEC 15288, 2008. •Krygiel, A., Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33 •Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp. 23-32, 2007. •Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of IEEE Systems, Man, and Cybernetics Conference, 2005. •Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development", USC CSSE Technical Report USC-CSSE-2006-623, 2006. •Maier, M., “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267-284). •Maier, M., “System and Software Architecture Reconciliation,” Systems Engineering 9 (2), 2006, pp. 146-159. •Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software Engineering Institute, 2006. •Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New Look, National Academy Press, 2007. •Rechtin, E. Systems Architecting, Prentice Hall, 1991. •Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC Integrating Systems and Software Engineering Workshop, http://csse.usc.edu/events/2007/CIIForum/pages/program.html, October 2007. •USC CSSE, ICSM Electronic Process Guide, http://greenbay.usc.edu/IICMSw/index.htm#publish.icm.baseusc/customcategories/icm_welcome_page_D99DA7B2.html 8/24/2015 Copyright © USC-CSSE © USC-CSSE 44 University of Southern California Center for Systems and Software Engineering List of Acronyms B/L C4ISR CD CDR COTS DCR DI DoD ECR EVMS FCR FED FMEA FRP GAO GUI 8/24/2015 Baselined Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance Concept Development Critical Design Review Commercial Off-the-Shelf Development Commitment Review Development Increment Department of Defense Exploration Commitment Review Earned Value Management System Foundations Commitment Review Feasibility Evidence Description Failure Modes and Effects Analysis Full-Rate Production Government Accountability Office Graphical User Interface Copyright © USC-CSSE © USC-CSSE 45 University of Southern California Center for Systems and Software Engineering List of Acronyms HMI HSI HW ICSM IOC IRR IS&SE LCO LRIP MBASE NDI NRC OC OCR OO&D OODA O&M 8/24/2015 (continued) Human-Machine Interface Human-System Interface Hardware Incremental Commitment Model Initial Operational Capability Inception Readiness Review Integrating Systems and Software Engineering Life Cycle Objectives Low-Rate Initial Production Model-Based Architecting and Software Engineering Non-Developmental Item National Research Council Operational Capability Operations Commitment Review Observe, Orient and Decide Observe, Orient, Decide, Act Operations and Maintenance Copyright © USC-CSSE © USC-CSSE 46 University of Southern California Center for Systems and Software Engineering List of Acronyms PDR PM PR PRR RUP SoS SoSE SSE SW SwE SysE Sys Engr S&SE USD (AT&L) VCR V&V WBS WMI 8/24/2015 (continued) Preliminary Design Review Program Manager Public Relations Product Release Review Rational Unified Process System of Systems System of Systems Engineering Systems and Software Engineering Software Software Engineering Systems Engineering Systems Engineer Systems and Software Engineering Under Secretary of Defense for Acquisition, Technology, and Logistics Validation Commitment Review Verification and Validation Work Breakdown Structure Warfighter-Machine Interface Copyright © USC-CSSE © USC-CSSE 47