Mr. Edward Jopson
Assistant Director
MDA/QSI
CQSDI March 27 2007

KI
C2BMC
GM AB
Developmental Systems morphs to Operational System – Reliability standards are inherently more stringent
Heritage Programs – Different mindsets, focuses and cultures, integration of these Heritage Programs into a BMDS
TH
STSS AL
2

Spiral Development - Qualification baseline changes and lack of test data to validate models
Acquisition Reform - Performance-based specifications and “Best
Practices” replacing MIL-STDs
3

MDA/QS DESCRIPTION
• Started October 2002
• Currently encompasses 134 personnel in 28 locations
• Supplier facilities, Test sites, Government facilities
• Direct Report to MDA Director
ROLES AND RESPONSIBILITIES
• Develop and maintain within MDA a world-class Safety, Quality and Mission
Assurance (SQMA) organization
• Provide BMDS Elements with robust and practical policy, guidance and expertise
• Develop an integrated BMDS SQMA team across MDA, the program elements, and the Ballistic Missile Defense industry
• Provide MDA personnel and resources to assist Elements and Industry in SQMA activities
• Terminate any operation that presents an immediate and unacceptable risk to personnel, property, or mission operations
4

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assistant Director
Assurance Integration (QSI)
Assistant Director
Safety (QSS)
Assistant Director
Mission Assurance (QSA)
Assistant Director
Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
5

Ft. Greely, AK
Ft. Greely, AK
ATK, Magna, UT
ABL, Kirtland AFB, NM
Lockheed, Denver, CO
JNIC, CO
BSOs (6)
Eagle Picher, Joplin, MO
Kodiak Facility, AK Raytheon, Woburn, MA
Raytheon, Andover,
MA
Lockheed, Moorestown, NJ Aerojet, Sacramento, CA
Lockheed, Sunnyvale, CA
ATK, Elkton, MD
Vandenberg AFB, Lompoc, CA
Northrop Grumman,
Redondo Beach, CA
Boeing, Anaheim, CA
Edwards AFB, CA
MDA HQ
Orbital, Chandler, AZ
Spectrum Astro, Gilbert, AZ
Raytheon, Tucson, AZ
Raytheon, Camden, AR
Honeywell, Clearwater, FL
KI, Huntsville, AL
GMD, Huntsville, AL
THAAD, Huntsville, AL
GMD, Huntsville, AL
GMD/IM & CTF, Huntsville,
AL
Black = QSL
Blue = MAR
Green = BSO
Red = Safety
6

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
DCMA Letter of Delegation
MDA Assurance Representatives System/Range Safety
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
7

• A standardized set of 14 SQMA provisions, requirements, and processes for mission and safety critical items:
- Management
- Design and Development
- Software and Firmware
- Technical and Mission Assurance Reviews
- Reliability, Maintainability, and Availability
- Parts and Materials Control Program
- Integrated Test and Evaluation Program
- Test, Measuring and Diagnostic Equipment and Standards
- Interface Management
- Configuration Management
- Control of Nonconforming Items and Materials
- Fabrication and Quality
- Supplier Management
- Safety
8

•
Provide methods to measure, verify, and validate mission success through:
- Collection and analysis of metrics
- Risk Assessment, technical evaluations, independent assessment, reviews
- Conducting tests
•
Promote continuous process improvement to reduce cost, improve productivity, and assure SQMA
• Specific applicability is defined in Mission Assurance
Implementation Plan (MAIP) – Verification Matrix
MAP is on/almost on all MDA contracts
9

COOPERATIVE APPROACH IMPLEMENTATION
Near-Term Mid-Term Long-Term
MDA
Assurance
Provisions
Evaluate /
Understand
Element MA
Practices
Improve Element MA Programs
Build Integrated MA Team Throughout MDA
Define / Implement Mission Assurance Program For The BMDS
Refine And Improve MAP And MDA MA Processes
Mandate Use Of “Golden Round” Philosophy – Test Like You Operate
Flow Down MA Requirements To
Contractors And Vendors
Formally Incentivize Mission Success
MAP As A Core Standard On All New Contracts
Perform Independent Reviews
Matrix QS Personnel Into Elements
• Perform Gap Analysis
• Collect / Share / Apply Lessons Learned
• Prioritize High Risk Items
Independent Assessment
•
•
QS Personnel Matrixed Throughout MDA Elements
Provide Comprehensive “Second Look”
10

Design And
Development
Manufacturing
Element
Integration &
Ground Test
Flight
Testing &
Verification
IDO
Design &
Development
Manufacturing
Ground Test
Flight
Testing &
Verification
Block 06
Design &
Development
Manufacturing
Ground Test
Time
Flight
Testing &
Verification
Block 08
11

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP) DCMA Letter of Delegation
MDA Assurance Representatives
System/Range Safety
Unfettered Access Parts, Materials and Processes Program
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
12

MARs provide MDA with deployed Mission Assurance resources for direct and unfiltered supplier insight, with the leverage to effect value added change and mitigate risk in a timely fashion, and serve as MDA’s Mission Assurance ambassadors to its Supplier community, associated agencies and organizations
Responsibilities:
- Rapidly identify, report and resolve technical issues on both hardware and software that impact the safety and reliability of the Missile Defense program
- Immediately report to the supplier’s program office and to MDA/QS any safety concerns/issues
- Immediately report to MDA/QS significant hardware/software issues impacting upcoming missions
- Report and resolve safety, quality, and reliability issues/concerns impacting MDA hardware and software
13

• Responsibilities (cont’d):
Represent QS Directorate and Cognizant Program Office
Provides DCMA and contractor direct access to MDA
Provides MDA Director unfiltered insight into program operations in real-time
Participates in design integration, test, qualification activities and issue resolution
Ensures that all applicable safety, quality and mission assurance provisions are being followed
Disseminates lessons learned to counterparts at all BMDS components and
DCMA
On-site authority for QS LOD to DCMA
14

MARs Mission Assurance
Dedicated MDA Resource (Agency Focus)
MA Ambassador (Leadership & Influence)
Direct Feedback
Pre-Emptive Role (Initiate Risks)
BMDS Perspective
System & Subsystem Cognizance
System & Subsystem Test Adequacy
Adequacy Of Requirements
Acceptability Of Issue Resolution
Adequacy of Quality Process
Adequacy Of Process & Procedure
Cross Element Comm/Coord
DCMA Quality Adherence
Matrixed, Non-Dedicated, Contract Focus
N/A
Non-Direct Feedback
N/A
Facility/Product Specific
N/A
Test Surveillance
Requirement Compliance
Surveillance
Mandatory Inspections
Procedure Adherence
N/A
15

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
16

17

Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic
Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
18

•
Product Design
•
Product Requirements
•
Processes
“Comparison”
• SQMA Requirements
•
MDA/Industry Best-
Practices
(MDA Assurance Provisions
& Other MIL-STDs/Guidelines)
• Contractor Plans, Procedures,
Practices and Records
19 19

• Validate that end items at a selected level conform to engineering design requirements
•
Validate end item at a selected level is qualified
•
Compare fabrication methods and acceptance test levels against
MDA & industry standards as appropriate for MDA use
• Validate production readiness
• Assess Contractor/Government
Mission Assurance practices, procedures, and implementation
20

• Management Systems
• System Design
• Integrated Test and Evaluation
•
Supplier Management
•
Configuration Management
•
Safety
•
Reliability, Maintainability, and Availability
• Software
• Parts, Materials, and Processes
• Fabrication and Quality
• Quality Management System – AS9100
• Other Safety, Quality, & Mission Assurance Practices
21

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
22

The MDA Assurance Group (MAG) is used to ensure industrywide understanding and implementation of the MDA Quality,
Mission Assurance and Safety requirements, as set forth by the
MDA/QS Director
MAG membership includes key Government and aerospace industry executives knowledgeable of quality enterprise strategic planning, Quality Management System (QMS) requirements / implementation, mission assurance principles, and knowledge of their companies Missile Defense programs
23

• Primary responsibilities of the MAG are to:
- Review and discuss the MDA Quality, Mission Assurance and Safety policies and strategies set forth by the MDA Director to make certain implementation facilitates mission success
- Identify, discuss and resolve significant, cross-cutting systemic issues affecting MDA programs
- Understand the activities of other Aerospace Industry groups/associations like SQIC, AAGQ, AIA, etc with purpose to eliminate redundant activity and to gather their lessons learned
•
MAG Meetings were held on 15 March 2005 and 28
February 2006
•
Key MAG Members: MDA, NASA, Boeing, Aerojet,
Northrop Grumman, Raytheon, Lockheed Martin, ATK
• Ex: Spawned “Stakeholder Initiatives”
24

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
25

•
MDA Assurance Advisories (MAA) provide information about problems, potential problems and failure experience data on parts, components, materials, manufacturing processes, specifications, computer software, test equipment and safety
• Advisories are exchanged to improve quality, reliability, and delivery schedules and to reduce costs
•
Implemented in October 2003, QS has published 23 Advisories. Distribution to the MDA program elements, Industry,
NASA, NRO and SMC
26

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
27

• Agreement between the Missile Defense Agency
Director and the Defense Contract Management
Agency Director…
- Establishes quality oversight requirements for mission critical items
- Defines MDA expectations and priorities
- Empowers DCMA to provide hands on quality insight/oversight
- Establishes quality assurance insight/oversight requirements throughout the supply chain
- Provides commonality and consistency for quality assurance implementation
- Provides a government influence on mission success
28

•
DCMA Quality Assurance oversight requirements for:
- Quality assurance surveillance strategy & plan
- Process surveillance activities
- Contract management offices reporting and notification
- Material review board activities
- Mandatory Government Inspections (MGIs):

Critical lifts and operations

Test setup (i.e., equipment and crew certifications, documentation verification)
 Final inspection
 Pre-shipment and shipment activities (i.e., black box closeout,
DD250, etc.)
29

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
30

•
State of MDA PMP Today
- Non-standardized PMP management
- COTS Technology and Obsolescence management lacks definition
- MDA lacks a comprehensive product engineering process
•
MDA/QS PMP
- MDA-wide PMP Management Plan
- MDA Parts, Materials and Process Board/Control Boards
- MDA Preferred Parts List
- Pilot Program developed for GMD Product Lines
31

• PMAP meets two of the Director’s goals:
Support the operations and sustainment of capabilities fielded to the warfighter
– Eliminates variability in screening and qualification of
PMP
– Improves the probability of mission success
– Ensures repeatability in the qualification baseline
Develop an integrated capability for Block 2006 and beyond
– Through a fully integrated Agency PMP program
Uniformity of Requirements Across All Elements
32

Industry State of Affairs
• Overseas Plants
Limited processes and material change knowledge
Military parts built on comm. manufacturing lines
Easy access: foreign countries reverse engineer military technology
• Unauthorized Distributors
No established relationship with part vendors, poor quality, no recourse – traceability issues
Inconsistency in vendor lot to lot parameters
Counterfeit fasteners - rivets, nuts, bolts, washers
• Lead-free Initiative
Pandemic for Hi-Rel systems
No accepted method for tin-whisker mitigation
Government - military and space application
• Obsolete Parts
Consumer
16%
Communication
21%
• EEE parts represent decreasing market share
Automotive
8%
Commercial Driven Industry = Greater Controls Required
Industrial
7%
Government
<1%
Computer
47%
33

Counterfeit Components
• Unauthorized Manufacturers
Build fake parts, selling cheap or defective devices at brand name prices
Buy low-power die for high-power packages
Parts marked with “high-rel” part no.
Fabricate parts to substandard processes
• China # 1 Seller
$1 - 10 billion per year (estimate)
35% of world market by 2010
Fab sites expanding rapidly, little govt control
Patent infringement
• Companies Counterfeited
Motorola - On Semi
Toshiba - National
PMC-Sierra
Counterfeit Genuine
Counterfeit capacitors with substandard electrolyte
Counterfeit Genuine
Note the difference in part numbers
34

• Introduction
Scope - Purpose
• Applicable Documents
Government, Commercial
• PMP Management Structure, Roles, and Responsibilities
PMP Control Board
• PMP Requirements
Program categories and definitions
Management of PMP selection & Quality Requirements
Requirements Applicability Matrix
PMP Procurement
Rad Hard, GFE, GFM
PMP Qualification
Non-Conforming Materials and Parts
Failure Analysis
ESD Control, Environmental Controls, Handling Controls
Preservation, Packaging and Storage Requirements
• Appendices
Derating
PEMS
COTS
Radiation Hardness
Prohibited Parts & Materials
35

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDa Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Software Acquisition Process Improvement
Test/Operational Safety (BSOs)
Supplier Roadmap
36

• Supplier Roadmap and Supply Chain Database
-
453 Suppliers in 39 states and 2 foreign countries
•
Can be searched by:
-
Congressional District
-
State
-
Supplier
-
Program
37

38

MDA/QS
Director
Quality, Safety And Mission Assurance (QS)
Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ)
QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical
Ballistic Missile Defense System.
MDA Assurance Provisions (MAP)
MDA Assurance Representatives
Unfettered Access
Mission Assurance and Safety Audits
MDA Assurance Group
MDA Advisory Process
DCMA Letter of Delegation
System/Range Safety
Parts, Materials and Processes Program
Govt/Industry Lessons Learned Program
Test/Operational Safety (BSOs)
Supplier Roadmap
39

“The frustrating part about it is we have not had an incident that, in my view, can be classified as a failure of the technology. . .
Someone didn’t perform a process right, somebody left out a piece of material where they shouldn’t. . .”
Former MDA Director, Lt Gen Ronald Kadish, December 2003
Roadshow Targets all audiences: Program Offices, President,
Vice President, Director, Chief Engineer, Systems Engineering
Manager, IPT Leads, Engineers, Quality, Safety and Mission
Assurance Personnel and Technicians
Discusses Lessons Learned on IFT-13c and IFT-14 Tests
40

Test
Test 1
Root Cause
EKV failed to separate due to an ASIC lead fatigue failure in the laser firing unit on the
EKV
Fundamental Issue
Inadequate laser firing unit qualification test. Stiffening foam was removed from a board. The random vibration test was conducted without power-on so the problem could not be detected on the ground.
Test 2
Countdown automatically aborted due to a builtin-test anomaly that detected excessive drops of
Electronic Control Unit (ECU) messages
Roundoff error propagation caused a timing mismatch. When the SIGI and FC clocks crossover, timing jitter occurs and occasionally data is sent to the ECU when it is busy, resulting in no responses. To avoid a recurrence, need to raise the tolerance level for missed messages -changing one line of computer software code.
Test 3
One of the last steps in the launch sequence is to open the silo doors and retract the horizontal stabilizers that hold the missile in place. Sensors indicated that one of the three lateral support group mechanisms had not retracted, causing the missile to abort launch.
Corrosion and dissimilar metals (galvanic corrosion) caused the failure. One mechanism failed to latch due to wrong crush blocks.
41

MDA LESSONS LEARNED ROADSHOW (CONT’D)
Company
Northrop Grumman
Northrop Grumman
Lockheed Martin
Raytheon
Raytheon
THAAD
Lockheed Martin
Lockheed Martin
Lockheed Martin
AF SPO
Raytheon
Spectrum Astro
Northrop Grumman
Aerojet
Group
KEI
MA Leads
Aegis
RMS
IDS
Program Office
THAAD & MKV
THAAD
BV+ Courtland
ABL
RMS
NFIRE, STSS
STSS
MDA
Location
Fair Lakes, VA
Reston, VA
Moorestown, NJ
Woburn, MA
Tewksbury, MA
Huntsville, AL
Sunnyvale, CA
Huntsville, AL
Elkton, MD
Albuquerque, NM
Tucson, AZ
Gilbert, AZ
Los Angeles, CA
Sacramento, CA
42

• Increasing Quality Assurance rigor by:
Organizing for Mission Success
Changing Culture
Onsite MDA Mission Assurance Representatives
(MARS)
MDA Assurance Provisions (MAP) & PMAP
Defense Contract Management Agency (DCMA)
Delegation of Authority Letter
System and Range Safety Oversight
Safety and Occupational Health
Getting Back to the Basics
43

• Change Customer and Contractor Cultures
- Corporate Leadership and UNIVERSAL, CONTINUING Commitment
• Increase Process Discipline – Supplier Management
- Management and Engineering Process Rigor
- Establish and Maintain Basic Set of SYSTEM Standards (MDA MAP –
Core Standards)
- Ensure basics are CONTINUOUSLY accomplished by BOTH
Customer and ALL Supplier Tiers
- Ensure Systems Engineering, Disciplined Test Management, Quality
Management, Risk Management, Configuration Management
• Organize for Mission Success
- Clear, continuing organizational accountability
- Robust engineering organization
- Independent, capable SQMA organization and personnel
44