Model-Based System Integration (MBSI) An Instructional Approach Dr. Paul Montgomery Associate Professor of Systems Engineering Naval Postgraduate School May 15, 2012 1 Perspectives Calibration 2 Developmental SE SI – Different perspective from SEs DoD – Developing complex systems Academia – SE professor MBSE – Evolving and maturing MBSI – An idea (MBSE from SI’s perspective) BLUF Integration • Integration begins at design Modeling • Don’t try to integrate the system until you successfully integrate the system model Instruction • Integration must be experienced, not merely studied 3 Essential Concepts of I&Q1 Integration = ensuring the system comes together • Interfaces (connectivity and flow) • Interactions (also interoperability) Qualification = ensuring the system is acceptable to the customer (aka ‘acceptance’) • Building the system correctly (aka ‘verification’) • Building the correct system (aka ‘validation’) 1 4 I&Q = Integration and Qualification What’s the Problem? Many system developments fail at integration & qualification (I&Q) … and fail badly Added cost, schedule, and needed redesign 1 • Hershey Foods Corp. PROJECT: IBM-led installation and integration of SAP, Manugistics Group Inc. and Siebel Systems Inc. software…Hershey sales fell 12% in the quarter after the system went live — down $150.5 million compared with the year before • Norfolk Southern Corp. PROJECT: Systems integration with merger target Consolidated Rail Corp…Norfolk Southern lost more than $113 million in business during its 1998/1999 railroad merger with Conrail. Custom logistics software wasn’t tested properly and a dispatcher mistakenly fed bogus test data into the system • Tri Valley Growers PROJECT: Oracle Corp. ERP and application integration…Tri Valley bought at least $6 million worth of ERP software and services from Oracle in 1996. None of the software worked as promised; some of it couldn’t even be installed on Tri Valley’s DEC Alpha hardware, the co-op claimed in a $20 million lawsuit filed in February. 5 From: “Top 10 Corporate Information Technology Failures” Where are we (DoD) Going? - DoD and SoS/LSI (Gansler) SoS acquisition and engineering is the norm in DoD SoS design, integration and qualification (I&Q) is highly complex DoD engineering workforce not well aligned to LSI responsibilities 6 • Government oversight of LSI has been complicated with contractual ambiguities • Delineation of “inherently governmental functions” for LSI needs more clarity • Private LSIs have inherent conflicts of interests without specific controls • SoS integration requires a strong, centralized LSI If SE is Well Defined, Why is I&Q a Challenge? What’s wrong with this picture? ID Needs Deployment O&M Verification & Validation Define Concept Preliminary Design Detailed Design System I&T Component I&T Component Build 7 Production / Mfg MBSI INTEGRATION BEGINS AT DESIGN 8 What is a System Model? Functional Decomposition (Hierarchy) Behavior Diagram (Sequence) CORE Model Generic Physical Block Diagram 9 Functional Flow Model (FFBD) Functional Process Model (IDEF0) Interface Diagram (N2) MBSI – The SI’s MBSE Perspective Design Environment Integration Environment Modeling Environment MBSE? 10 MBSI? Qualification Environment SE Activities Should Produce System Definition/Model Operational Model 11 Interface Physical Model Model Functional Behavioral Model Model System Definition (“Model”) System Model Underpins I&Q Activities 12 System Modeling Don’t try to integrate the system until you integrate the model 13 Progressive Integration Different teams in diverse locations 14 Integration and Qualification Considerations from Functional Analysis Complex flows/connectivity may indicate complex interactions and bears special attention for integration and qualification focus (or possible redesign) 15 Integration and Qualification Considerations from Behavior Analysis High behavioral interaction activity bears special attention for integration and qualification focus (or possible redesign) 16 Integration and Qualification Considerations from N2 Analysis Large number of interface content (complex interactions) can warrant special integration and qualification focus (or possible redesign) Function 1 Function 2 Function 3 Function 4 Function 5 17 I&Q Instructional Methods INTEGRATION MUST BE EXPERIENCED, NOT MERELY TAUGHT 18 Process Approach Popular Approaches to Teaching I&Q SE Fundamentals SE Integration Test and Eval Shortfalls: Non-tangible experience Hard to develop I&Q instincts Disjointed learning experience End-to-End Approach “Toys” Approach Shortfalls: Cannot design components Interfaces are fixed Interface design may be hidden Concept Design Integrate Qualify Build System Model System model is essential for project success 19 Shortfalls: Not enough time Not enough student skills Set up for failure MBSI Instructional Example A PROJECT 20 Overview of Class Project SOH Submarine Detection using Fire Scout (STRAIT SCOUT) 21 Customer Problem Statement Problem In the Persian Gulf, we do not have a reliable system to detect submarines that egress and ingress through the SOH by hiding in tanker wakes. Research Questions Can a combination of BAMS and one FireScout be used to provide a high Pd of the submarine behavior above? What is shipping traffic density vs. Pd performance of such a system? What are some FireScout search strategies for such a system deployment? 22 Primary System Assets BAMS - Persistent surveillance over AOR with surface search Radar Fire Scout •Speed = 0 – 90kts FireScout Sensor = LIDAR (Light Detection and Ranging) •Scanning •Sub – inch resolution 23 Top-Level STRAIT SCOUT Architecture Concept BAMS 1 FireScout 5 6 3 6 Red Team 7 1 Sensor 4 8 Test Parameters (Parameters, Scenarios, Results) 1 – Ships and sub position data 2 – N/A 3 – Surface track data 4 – Sensor data 5 – Flight path commands 6 - Position data 7 – Environment parameters 8 – Test results 24 C2 8 Team Roles & Responsibilities LSI • Primary stakeholder negotiations • Top-level architecture • Taxonomy and structure – External systems interfaces – Intra-subsystem interfaces – Functional naming conventions • Conop • Integration and qualification strategy – Integration strategy – Acceptance goals, objectives, and agreements • Leadership Subsystem teams • Subsystem derived requirements • Subsystem Design – Functional, interfaces, and generic physical • Subsystem integration and qualification Instructor = Primary customer/stakeholder 25 SE Design 26 Define the problem Develop functional architecture Develop physical architecture Develop operational architecture Develop interface architecture Define integration, test, V&V strategy Simulation Concept Tanker track and direction FireScout sensor scan field Wake (with sub) Detection? time = t Wake (no sub) time = tn 27 Subsystem A Excel™ VBA Subsystem B VBA “Dictator” Project I&Q Environment Excel™ Excel™ VBA System Subsystem C Excel™ VBA 28 Subsystem Teams LSI Team Advantages: Readily available “Office” tools Concept-to-design Interface visibility Team integration = subsystem integration LSI integration = system integration Disadvantages: VBA is not innate SE skill Too much to do in time alloted Integration can still be undisciplined MBSI Environment “Design” “I&Q” Implementation 29 Functional Modeling Physical Modeling Cell formulas and VBA Sheets and cells Interface Modeling Behavioral Modeling Qualification Modeling modeling Conop Needs Mission Constraints Assumptions Goals Objectives Primary Class Project Phases 1 30 System design Model integration (CORE1) System development (code) Subsystem & System integration System verification (test) System validation (demonstrate) System acceptance (grade) CORE 8 (University) Service pack 3 MBSI Instructional Example THE SI PERSPECTIVE 31 SI Challenge Questions Do you understand your problem and what your subsystem needs to do? Do you understand enough about your subsystem behaviors to define functions? How many functions are in your subsystem? Are the functions “modular” and simple? How many interactions do you expect? How many external interfaces do you need to define? How many internal interfaces do you need to define? Have you thought of which functions need to be integrated first? What are the integration and qualification risks that are starting to emerge? 32 Simplified Strait Scout Sequence Diagram? LSI Red BAMS C2 FS Sensor Setup / Run loop Locations Target Data Flight Cmds Sub detected Record Terminate 33 Fly/ Location Strait Scout Functional Context Systems Control flow is linear? 34 N2 Interface complexity? 35 IDEF Many interfaces? 36 Sequence Triggers? Responses? 37 Student Discoveries Early requirements clarification is important Early architecture design imperative (especially functional and interface) Rushing to development prior to model definition wastes time and effort Early model integration drives out: Functional gaps and overlaps Interface inconsistencies and discontinuities System behavior misunderstandings Inter and intra-system interface problems SI involvement in design can reduce risk during I&Q Project would have failed without MBSE/MBSI methods 38 MBSI – An Instructional Approach LESSONS LEARNED 39 Value of MBSI Successful I&Q requires: • Strong LSI / SI • Detailed system definition (particularly interfaces and functional interactions) • Early taxonomy and structure definition • Early SI influence with I&Q success perspective • Modeling in order to discipline design efforts • Model integration prior to system integration to reduce I&Q risks • Diverse and integrated SE/SI support system (i.e. tool sets, etc.) MBSE tools not yet MBSI tools “Teach” I&Q using MBSI applied to experiential project 40 References Handbook of Systems Engineering and Management, Sage and Rouse (ed.), Wiley and Sons, 1999, Chapter 14 Systems Engineering Guide for Systems of Systems, Ver 1.0, Aug 2008, Director, Systems and Software Engineering, DUSD (Acq and Tech), OSD (AT&L) The Role of the Lead System Integrator, Gansler, et.al., NPS-AM-09-005, Jan 2009 Top 10 Corporate Information Technology Failures, www.computerworld.com/computerworld/records/images/pdf/44NfailChart. pdf http://cdn.business2community.com/wpcontent/uploads/2012/04/customer-experience-focus.jpg http://www.fas.org/man/dod-101/sys/ac/f-18-ef-usnhornt.jpg http://www.behavioradvisor.com/sbPuzzled.jpg http://www.blogspot.com/-hSnr-sGBKko/T2_WfiI8DI/AAAAAAAAD88/tzcvl7wxgRA/s1600/acellphone.gif 41