Headquarters U.S. Air Force Integrity - Service - Excellence Air Force Systems Engineering Focus NDIA Systems Engineering Division Meeting 18 Aug 2010 Col Shawn P. Shanley SAF/AQRE 703.254.2487 shawn.shanley@pentagon.af.mil 1 Agenda What WSARA & Related OSD Initiatives Mean to AF SE Leadership Perspective / Expectations / Challenges AF SE Conference What AF is Doing to Focus Our Practitioner Community Process Focus Development Planning / Early System Engineering AF Program Support Review (PSR) (integrated reviews) Policy Focus Recent Policy Changes Near Term Policy Changes Specialty Areas (RAM, HSI, ESOH, etc) Specs/Standards & RAM People Focus Required Resources Integrity - Service - Excellence 2 Bottom Line Up Front – What WSARA Means to the AF AF understands the need to improve our acquisition approach SECAF/CSAF initiated AF Acquisition Improvement Plan prior to WSARA Forward-looking AF response to OSD’s WSARA-based questions Grounded in acknowledgement that “all is not well” Focused on what we are doing about it “Business as usual” policies and processes won’t solve the problems Innovative policies / processes in work through various AF initiatives Additional workforce alone is wrong approach Need right number of personnel with the right skill sets to support policies and processes Change doesn’t happen overnight Continued leadership support and commitment to improvements and visible ROI over time helps ensure we realize the improvements $295 Billion in Overruns & Major Schedule Slips – How SE Can Help to Solve the Problem! Integrity - Service - Excellence 3 Leadership Perspective Robust SE contributes to successful programs Front end trades Cost budget analysis Performance vs. cost and/or schedule SE focus areas to increase chances of success Development Planning: early SE to inform MDD Risk Management: identify all technical risks and manage to closure — full and open discussion with PMs, PEOs, and SAE Competitive prototyping: reduce risk for key technologies; increase competition ECPs: work collaboratively with industry and stakeholders; mitigate requirements creep (non-Class 1 changes); use the CSB Specs and standards in contracts: Reviews and Audits, Systems Requirements Document, Corrosion Prevention, ESOH, Reliability Integrity - Service - Excellence 4 Expectations of the SE Community Robust, disciplined SE; sound engineering/technical management Policy Processes People Honest broker on performance/technical risks Thorough understanding of programmatic technical issues Competent insight into contractor technical planning and execution To meet these expectations, we must: (1) Institutionalize technical rigor and discipline across the life cycle of acquisition programs (2) Revitalize the AF SE workforce Integrity - Service - Excellence 5 Challenges Planning: lack of sufficient technical input to make informed decisions Little up-front and early technical mentoring, advice, and assessment Program start decisions made with insufficient information — lack of concept/solution work prior to MDD Programs pass milestones without adequate look at technical risk Execution: technical issues/risks not discovered and addressed at the right time and at the right level Beware of the “Sea of Green” — “I’m OK because I’ve got a plan” Immature technologies and integration issues continue to drive cost and schedule impacts — we need to step up our game ECPs are still rampant — impacts to AF enterprise unknown; must use CSBs more effectively Maintain discipline of SE processes — don’t trade away rigor Integrity - Service - Excellence 6 AF SE Conference 3-5 Aug 2010 THEME: “Systems Engineering: A Key Enabler to Development, Acquisition, & Sustainment” Highlighted activities associated with improving the practice and management of SE across the Air Force 224 attendees, principally representing the AF Engineering community Product, Logistics, and Test Centers Concept Development organizations (XR) AFRL HQ AFMC (Requirements, Engineering, Sustainment) HAF/SAF Invited presentations from DDR&E/SE NDIA Reps Integrity - Service - Excellence 7 AF SE Conference 3-5 Aug 2010 2 Key takeaways Robust SE contributes to successful programs Implementing standard SE practices, processes, and policies fosters effective and efficient integrated life cycle management Must take a balanced approach to reducing risk Improved development and management of the technical workforce are critical to the future success of SE throughout the acquisition and sustainment life cycle Accurate product and design data essential to maintain and improve operational safety, suitability, and effectiveness (OSS&E) – key to sustainment Feedback was overwhelmingly positive SAF/AQR, AF Center for Systems Engineering, and AFMC/EN have agreed to co-sponsor next year’s conference in Aug 11 at WPAFB Integrity - Service - Excellence 8 Process Focus Areas of Strength Development Planning/CCTDs: drives disciplined, repeatable concept definition; supports MDA at MDD Air Force Program Support Review: standardized structure for specialty reviews & technical planning; includes PMO mentorship and assistance Systems Engineering Assessment Model (SEAM): AFMC policy; identifies key SE processes & products Common Tools: RI3, TDTS, LHA/ILA, … Area for Improvement Need to ensure new processes are fully implemented across our programs and adjustments made as necessary following implementation Integrity - Service - Excellence 9 Recapturing Technical Excellence – Two Initiatives JCIDS Strategic Guidance Joint Concepts 5000.02 ICD CBA MS (-18 to -12) MS (-17 to-11) Materiel Solution Analysis MDD MS (-9 to -6) MS (-2) MS (-1) DP 3 AFRB MS A MS B MS C Technology CDD Engineering and CPDProduction and Manufacturing Deployment Development Development MS Interim Feedback to PM Guidance & Planning mtg DP 1 DP 2 Conduct DAPS assessment Address preliminary findings Identify CTEs Assess CTEs’ TRLs DAB OIPT Identify critical processes Complete MRA MS (-18 to -12) MS (-17 to-11) MS (-9 to -6) MS (-2) MS (-1) DP 3 AFRB MS Interim Feedback to PM Review logistics readiness Prepare/review/complete PESHE documentation Guidance & Planning mtg Prepare/review/complete technical documents (TDS, SEP, …) DP 1 DP 2 Conduct DAPS assessment DAB Address preliminary findings OIPT Conduct risk assessment Monitor risk buy-down Assess CTEs’ TRLs Identify CTEs Identify critical processes Complete MRA MS (-18 to -12) MS (-17 to-11) MS (-9 to -6) MS (-2) MS (-1) DP 3 AFRB MS Interim Feedback to PM Review logistics readiness Prepare/review/complete PESHE documentation Guidance & Planning mtg Prepare/review/complete technical documents (TDS, SEP, …) DP 1 DP 2 Conduct DAPS assessment Address preliminary findings Identify CTEs Assess CTEs’ TRLs DAB OIPT Conduct risk assessment Monitor risk buy-down Identify critical processes Complete MRA MS (-18 to -12) MS (-17 to-11) MS (-9 to -6) MS (-2) MS (-1) DP 3 AFRB MS Interim Feedback to PM Review logistics readiness Prepare/review/complete PESHE documentation Guidance & Planning mtg Prepare/review/complete technical documents (TDS, SEP, …) DP 2 DP 1 Conduct DAPS assessment DAB Address preliminary findings OIPT Conduct risk assessment Monitor risk buy-down Assess CTEs’ TRLs Identify CTEs Identify critical processes Complete MRA Review logistics readiness SEP TRA TDS Prepare/review/complete PESHE documentation Prepare/review/complete technical documents (TDS, SEP, …) Conduct risk assessment Outcomes Statutory / regulatory document development, review, and approval Development Planning Process (DP) S, T & E Engagement Concept/Solution selection influence SEP influence TDS influence T&E risk assessment TRL data evaluation Monitor risk buy-down Standardized OSD/AF reporting Integrated risk assessment AF Program Support Review Process (AF PSR) SE Engagement up front and early with PMs to mentor/assist AF Standard for Tech Review Process Synchronized data requests to reduce program office & contractor impacts Identification of technology maturation/integration risks Collaborative with OSD to prevent surprises Integrity - Service - Excellence 10 MDD Information Needs – Forthcoming DDR&E Policy ... To support those decisions, the DoD Components shall provide evidence at the MDD Review that will facilitate the determination that: a. The candidate materiel solution approaches have the potential to successfully address the capability gap(s), associated dependencies, and operational attributes. b. There exists a range of technically feasible solutions generated from across the entire solution space, as demonstrated through early prototypes, models, or data. c. Consideration has been given to near-term opportunities to provide a more rapid interim response to the capability need. d. The plan to staff and fund analytic, engineering, and programmatic activities supports the proposed milestone entry requirements. AF is assessing issues associated with industry engagement during DP / Early SE Conflicts of interest, protection of proprietary information, solutions masquerading as concepts, contracting terms and conditions, numerous others ... Integrity - Service - Excellence 11 Process DP #1—Organize, train, equip -Mentor/advise program office -TRA: ID independent review team -Early risk identification -Gain program understanding DP #2—Provide early status -Provide preliminary report to PM -TRA: approve CTEs -Address risks -Cross flow information DP #3—Finalize products -Out brief findings to PM -TRA: approve TRLs -Capture open risks -Complete MS deliverables Post Review -Support AFRB/DAB -Conduct lessons learned hot-wash Notional Timeline MS (-18 to -12) MS (-17 to-11) MS (-9 to -6) MS (-2) MS (-1) MS DP 3 AFRB DAB Interim Feedback to PM Guidance & DP 1 Planning mtg DP 2 Leverage tech reviews DAPS (OSD-led) Conduct DAPS assessment Address preliminary findings Technology Identify CTEs Assess CTEs’ TRLs Manufacturing Identify critical processes Complete MRA Logistics Review logistics readiness ESOH Review ESOH Technical docs Prepare/review/complete technical documents (TDS, SEP, PESHE, …) Integration Risk Conduct risk assessment OIPT Monitor risk buy-down Early technical engagement to seamlessly mentor/advise, independently assess & capture risks…so PMs can act * Statutory/regulatory process Integrity - Service - Excellence Policy Focus Areas of Strength Acquisition Improvement Plan: acquisition workforce revitalization and improved requirements generation Development Planning: focus of early systems engineering Program Technical Assessments: collaborative, independent advice to senior decision makers Standards revitalization: restoring standard practices Areas for Improvement Improve SE authority, independence, credibility/trust and visibility: Development Planning, AF Program Support Review processes right first steps Reliability, Availability, and Maintainability: Guidance memo; explore potential use of standard tools, e.g., Risk Identification, Integration, and Illities (RI3) Guide Integrity - Service - Excellence 13 Recent Policy Changes Reliability, Availability, and Maintainability (RAM) Requires continuous reliability improvement, reliability growth plans Guidance Memo published 20 Jul 10 Program Technical Assessments Establishes AF Program Support Review Establishes responsibilities for Center-Level Technical Authority and PEO Chief Systems Engineers Released over SAF/AQ signature, 10 Aug 10 Development Planning (DP) Guide Released over AFMC/A5 and AFSPC/A5 signatures, Jul10 Integrity - Service - Excellence 14 Near-Term Policy Changes Development Planning (changes to AFI 63-101 and AFI 63-1201) Establishes Concept Characterization and Technical Description (CCTD) as early SE artifact to support Materiel Development Decision and Analysis of Alternatives Adjudicated comments from Technical/Functional Coordination Starting Mandatory Coordination Aug 10, publishing goal Oct 10 CCTD Guide In final review; expect release over AQR signature Aug 10 Early Systems Engineering Guide Content to be updated and folded into revision to DP Guide, start 1QFY11 Corrosion Prevention & Control F-22/F-35 issues driving increased emphasis Evaluating adequacy of current policy and guidance Integrity - Service - Excellence 15 Specs and Standards Acquisition program execution problems (technical, cost, and schedule) have a common root cause: Blanket government removal of specs/standards (early 90s) This led to Absence of well-defined product characteristics Inconsistent application of necessary SE processes Specs and standards compliance becoming trade space in a highly cost-competitive environment Plan for AF Acquisition Improvement requires … Tailorable standard practices for SE processes (balanced) Policy to mandate use in solicitations and contracts (TBD) Integrity - Service - Excellence 16 RAM Improvement Include RAM considerations in Early SE (pre-Milestone A) activities Incorporate Lessons Learned Ensure realistic schedules and funding Plan for longer operational use (extended service life) of systems and components Utilize ManTech, CSE, and other resources Increase training and awareness Reconstitute AF PQM expertise Increase emphasis on RAM in market surveys and source selections Implement reliability growth programs Integrity - Service - Excellence 17 Personnel Focus Areas of Strength Revitalization of product center EN functional offices: centralizes SE career development and personnel management Science, Technology, Engineering, & Mathematics (STEM): executing “Bright Horizons” strategic plan Areas for Improvement Acquisition Workforce Training: training capacity at DAU; in-process workforce growth Identification of SE and DP resources across Air Force: functions and activities widely distributed Integrity - Service - Excellence 18 STEM – Revitalizing the AF Workforce AF STEM Definition The Air Force professionals having degrees in Science, Technology, Engineering, or Mathematics (STEM). Goal Areas: AF Population 465k STEM Cognizant Size TBD STEM Degreed 53k STEM Inventory 25k Requirements/Inventory Funding/Resources Force Management Continuum of Learning Outreach Evaluation/Analytics Integrity - Service - Excellence 19 Required Resources Initial major resource required: TIME Time to complete plans and processes Time to implement plans and processes into policy; institutionalize Time for the field and HQ to respond with higher fidelity resource requirements after policy implementation Once numbers are in, need funding and billets to implement policy: PEOPLE SE, DP, and SMEs manning Supporting resources and personnel Training sources; quality OSD assistance to help enforce/implement the SE discipline desired through policy, processes, reviews, senior leadership message SE is a key enabler of solutions to Acquisition priorities Collaboration with all stakeholders -- industry, users, decision-makers -is imperative to solve the problems Integrity - Service - Excellence 20 WHY IT’S IMPORTANT Integrity - Service - Excellence 21 BACKUP Integrity - Service - Excellence 22 Weapon System Acquisition Reform Act “The key to successful acquisition programs is getting things right from the start with sound systems engineering, cost estimating, and developmental testing early in the program cycle. The bill that we are introducing today will require the Department of Defense to take the steps needed to put major defense acquisition programs on a sound footing from the outset. If these changes are successfully implemented, they should help our acquisition programs avoid future cost overruns, schedule delays, and performance problems.” –Senator Carl Levin, Chairman, Senate Armed Services Committee “The Weapon System Acquisition Reform Act of 2009 is an important step in efforts to reform the defense acquisition process. This legislation is needed to focus acquisition and procurement on emphasizing systems engineering; more effective upfront planning and management of technology risk; and growing the acquisition workforce to meet program objectives.” –Senator John McCain, Ranking Member, Senate Armed Services Committee Integrity - Service - Excellence 23 SECDEF Guidance “Chief among institutional challenges facing the Department is acquisition.” Integrity - Service - Excellence 24 Recapture Acquisition Excellence Clear Priorities Reinvigorate AF Nuclear Enterprise Win Today’s Fight Develop & Care for Airmen & Families Modernizing Aging Air & Space Inventories Recapture Acquisition Excellence Acquisition Improvement Plan (AIP) Revitalize the Air Force acquisition workforce Improve requirements generation process Instill budget and financial discipline Improve Air Force major systems source selections Establish clear lines of authority and accountability within acquisition organizations Systems Engineering: Key enabler to recapture Acquisition Excellence Integrity - Service - Excellence 25 Air Force Systems Engineering Conference Tuesday, 3 August 2010 Day 1 Theme: Need for Disciplined SE (Policy) 0730-0815 0815-0830 Check-in Room 118/121/123;Overflow Rooms 111,112,116 Welcome/Conference Administration Leadership Expectations of Systems Engineering (SE) Room 118/121/123; Overflow Rooms 111, 112, 116 0830-0900 Lt Gen Mark Shackelford, SAF/AQ 0900-0930 Dr. Steven Butler, AFMC/CA 0930-1000 Mr. Blaise Durante, SAF/AQX Keynote Address Room 118/121/123; Overflow Rooms 111, 112, 116 Mr. David Van Buren, Air Force Service 1000-1030 Acquisition Executive 1030-1050 Break SE in Acquisition (WSARA) Panel MODERATOR: Dr Steven Walker, SAF/AQR Room 118/121/123; Overflow Rooms 111, 112, 116 Mr. Stephen Welby, DDR&E, Systems 1050-1110 Engineering 1110-1130 Mr. Robert Boulware, SAF/AQXA 1130-1150 Col Shawn Shanley, SAF/AQRE 1150-1210 Panel Q&A 1210-1330 Lunch Technical Sessions Room 1A: SE Policy (incl. JCIDS, 118/121/123 DoDI 5000.02 & Early SE) 1B: Revitalization of Room 111/112 1330-1500 Specs and Standards 1C: Systems Engineering Room 116 Research Break 1500-1530 Room 118 1D: Modeling & Simulation 1E: Architectural Room 121/123 1530-1700 Frameworks 1F: Human Systems Room 111/112 No-Host Social, 1700-1900 Integration Hamlet Restaurant Wednesday, 4 August 2010 Day 2 Theme: Importance of Standardized SE Across the Life Cycle (Where SE is applied and by whom) 0730-0815 Check-in 0815-0830 Room 118/121/123;Overflow Rooms 111,112,116 Conference Administration SE Importance Across the Acquisition Life Cycle MODERATOR: Mr G Richard Freeman, AFIT/SY Room 118/121/123; Overflow Rooms 111, 112, 116 0830-0850 Dr. Kenneth Barker, AFRL/XP 0850-0910 Mr. Gerald Freisthler, ASC/CA 0910-0930 Maj Gen David Eichhorn, AFFTC/CC 0930-0950 Mr. Grover Dunn, AF/A4I 0950-1010 Panel Q&A 1010-1030 Break S&E Workforce Development Panel MODERATOR: Mr G Richard Freeman, AFIT/SY Room 118/121/123; Overflow Rooms 111, 112, 116 1030-1050 Dr. Donald Gelosh, DDR&E 1050-1110 Lt Col Michelle Trigg, SAF/AQH 1110-1130 Mr. Albert Boulter, AFPC/DPIBE 1130-1150 Panel Q&A 1150-1330 Lunch Technical Sessions 2A: Integrated Master Plan Room & Integrated Master 118/121/123 1330-1500 Schedule Room 111/112 2B: Reliability Room 116 2C: ESOH 1500-1530 Break Room 121/123 2D: Test and Evaluation 1530-1700 2E: Life Cycle Supportability Thursday, 5 August 2010 Day 3 Theme: SE Applied 0730-0815 0815-0830 (How SE is applied and measured) Check-in Room 118/121/123;Overflow Rooms 111,112,116 Conference Administration SE Application and Assessment Panel MODERATOR: Col Shawn Shanley, SAF/AQRE Room 118/121/123; Overflow Rooms 111, 112, 116 0830-0850 Mr. Robert Martin, AF PM&AE 0850-0910 Dr. Marvin Sambur, Univ. of Maryland 0910-0930 Mr. Russell Howard, AFMC/EN 0930-0950 Col David Swanson, SMC/EA 0950-1020 Panel Q&A Break 1020-1040 Technical Assessments Panel MODERATOR: Col Shawn Shanley, SAF/AQRE Room 118/121/123; Overflow Rooms 111, 112, 116 1040-1100 Mr. James Thompson, DDR&E 1100-1120 Mr. Joseph Shearer, SDB II Program 1120-1140 Lt Col Scott Brown, SAF/AQRE 1140-1200 Panel Q&A 1200-1215 Closing Comments & Adjourn 116 K I T C H E N 123 121 118 Room 118 2F:will Coolbe Tools (ARM,to SEAM, All briefings posted the AF SE 111/112 Room RTT, ConferenceRI3, CoP byetc) 12 August https://rso.my.af.mil/afknprod/ASPs/Reg/ GroupAdmin.asp?Filter=23843&EventID=1464 7 RECEP TION DESK 111 112 Session Descriptions – 3 August 1A: SE Policy (including JCIDS, DoDI 5000.02, and Early SE). Rooms 118/121/123, 1330-1500. Moderators: Mr. Bob Reifenberg, ASC/ENS, and Mr. Jeff Loren, SAF/AQRE. This session will address recent and coming policy changes that impact Early SE, including WSARA and its impacts to DoDI 5000.02; requirements community interfaces; governance; resources; the Early SE “V” and CCTD; support for AoAs; and integrated Development Plans/roadmaps. 1B: Revitalization of Specs and Standards. Rooms 111/112, 1330-1500. Moderator: Mr. Scott Kuhnen, AFMC/ENS. Presentations will address current AF policy initiatives and on-going efforts at the MAJCOM/AF/DoD levels to revitalize specifications and standards. 1C: Systems Engineering Research. Room 116, 1330-1500. Moderator: Dr. David Jacques, AFIT/ENV. This session will discuss the unique nature of SE research, new directions being pursued, and potential benefits for the Department of Defense. Three leading researchers will provide their perspectives and examples of DoD-related SE research. 1D: Modeling & Simulation. Room 118, 1530-1700. Moderator: Lt Col John Simeoni. M&S is a valuable tool that can support and enhance the acquisition process. M&S helps identify key ops and design factors early and supports rapid design, fabrication, and testing. Targeted application of M&S has the potential to reduce risk, shorten timelines, and provide data for making sound managerial and technical decisions. The AF must plan for the use of M&S early in the life cycle as an integral part of the SE process. 1E: Architectural Frameworks. Rooms 121/123, 1530-1700. Moderator: Dr. John Colombi, AFIT/ENV. Architecture has been long used as a tool to understand and communicate complexity in order to enable structured decision-making. This session will focus on ways to make AF architecture more useful and relevant. Presentations will provide attendees with new ideas, policy, and perspectives on architecting to better support acquisition and preacquisition activities. 1F: Human Systems Integration (HSI). Rooms 111-112, 1530-1700. Moderator: Col Larry Kimm, AFHSIO. Systems engineers will learn how HSI is a life cycle process and applies from before MS A (supporting development planning and early systems engineering) through operations and sustainment, with information on available guidance, tools and resources. Session Descriptions – 4 August 2A: Integrated Master Plan (IMP) & Integrated Master Schedule (IMS). Rooms 118/121/123, 1330-1500. Moderator: Mr. Sam Johns, AF PM&AE. This session is targeted toward increased understanding of the importance of IMP & IMS and the associated role of the Systems Engineer. It will cover current Air Force IMP/IMS policy; engineering's role in IMP development and construction; an IMP/IMS integration tutorial; and the IMP/IMS in the Integrated Baseline Review process. 2B: Reliability. Rooms 111/112, 1330-1500. Moderator: Mr. Sam Brown, SAF/AQRE. Presentations will address ongoing OSD efforts to improve reliability guidance and implementation across the Services, and challenges being faced by Air Force programs as they implement reliability programs. Attendees will be able to share their thoughts and ideas following the presentations. 2C: Environment, Safety and Occupational Health (ESOH). Room 116, 1330-1500. Moderator: Mr. Ken Dormer, SAF/AQRE. This session will address the integration of ESOH design and technical risk management considerations across the life cycle as a part of a disciplined systems engineering approach. Speakers will discuss OSD initiatives to improve ESOH-engineering integration, the implementation perspective of one AF Center, and DoD efforts to update MIL-STD-882D. 2D: Test and Evaluation (T&E). Rooms 121/123, 1530-1700. Moderator: Dr. Michael Deis, HQ AFMC/A3. This session will provide perspectives on the implementation of WSARA systems engineering and developmental testing requirements. A question and answer period will follow. 2E: Life Cycle Supportability. Room 118, 1530-1700. Moderator: Mr. Edward Durell, SAF/AQRE. Session will cover "Logistician Expectations of Systems Engineering" as well as "Supportability Perspectives from Opposite Ends of the Weapon System Life Cycle." A panel discussion on life cycle supportability will follow presentations. 2F: Cool Tools (ARM, SEAM, RI3, RTT etc). Rooms 111/112, 1530-1700. Moderator: Ms. Karen Bausman, AFIT/SY. This session will include two presentations focused on enlightening the workforce to the strategic planning activities for SE tools. Presentations will address the overarching AF plan for SE tools, as well as the use of SE tools throughout AFMC. Q&A will follow with a focus on coordination among different tools efforts. Air Force Systems Engineering Conference 3-5 August 2010 Systems Engineering: A Key Enabler to Development, Acquisition, & Sustainment SAFTAS Conference Center 1550 Crystal Drive Arlington, Virginia