Process Improvement
in the Aerospace Industry
CMMI and Lean Six Sigma
USC CS510
Rick Hefner, Ph.D.
Northrop Grumman Corporation
Rick.Hefner@ngc.com
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
2
NDIA Top 5 Systems Engineering Issues
(2006)
• Key systems engineering practices known to be effective are not
consistently applied across all phases of the program life cycle.
• Insufficient systems engineering is applied early in the program life cycle,
compromising the foundation for initial requirements and architecture
development.
• Requirements are not always well-managed, including the effective
translation from capabilities statements into executable requirements to
achieve successful acquisition programs.
• The quantity and quality of systems engineering expertise is insufficient
to meet the demands of the government and the defense industry.
• Collaborative environments, including SE tools, are inadequate to
effectively execute SE at the joint capability, system of systems (SoS),
and system levels.
3
Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by Mr. Robert Skalamera
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
4
Heritage of Standards for Systems Engineering
2002
2002
ISO/IEC
15504
ISO/IEC
19760
(FDIS)
(PDTR)
1998
1994
EIA / IS
632
1994
1974
1969
Mil-Std499
Mil-Std499A
Mil-Std499B
(Not Released)
EIA
632
(Full Std)
1994
(Trial Use)
ISO/IEC
15288
(FDIS)
1998
(Interim Standard)
IEEE
1220
2002
1998
IEEE
1220
(Full Std)
EIA/IS
731
SE CM
(Interim Standard)
2002
CMMISE/SW/IPPD
Legend
Supersedes
Source for
5
Standards for Systems Engineering, Jerry Lake, 2002
The Frameworks Quagmire
Sarah A. Sheard, Software Productivity Consortium
6
http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp
Two Complimentary Approaches
to Process Improvement
Data-Driven (e.g., Lean Six Sigma)
Model-Driven (e.g., CMMI)
• Clarify what your customer wants
(Voice of Customer)
– Critical to Quality (CTQs)
• Determine the industry best practice
– Benchmarking, models
• Determine what your processes can do
(Voice of Process)
– Statistical Process Control
• Identify and prioritize improvement
opportunities
– Causal analysis of data
7
• Anticipate your customers/ competitors
(Voice of Business)
– Design for Six Sigma
• Compare your current practices to the
model
– Appraisal, education
• Identify and prioritize improvement
opportunities
– Implementation
– Institutionalization
• Look for ways to optimize the processes
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
8
What is the Capability Maturity Model Integrated?
• The CMMI is a collection of industry best-practices for engineering,
services, acquisition, project management, support, and process
management
– Developed under the sponsorship of DoD
– Consistent with DoD and commercial standards
Three Constellations sharing common components and structure
• CMMI for Development - used by engineering organizations
• CMMI for Acquisition - used by buyers (e.g., govt. agencies)
• CMMI for Services - used by service providers (e.g., help desk)
9
Basic Building Blocks – 22 Process Areas
Implemented by
the organization
Implemented by
each project
Project Management
• Project Planning
• Project
Monitoring and
Control
• Supplier
Agreement
Management
• Integrated
Project
Management)
• Risk
Management
• Quantitative
Project
Management
10
Engineering
• Requirements
Development
• Requirements
Management
• Technical
Solution
• Product
Integration
• Verification
• Validation
Support
• Configuration
Management
• Process and
Product Quality
Assurance
• Measurement
and Analysis
• Decision Analysis
and Resolution
• Causal Analysis
and Resolution
Process Management
• Organizational
Process Focus
• Organizational
Process
Definition
• Organizational
Training
• Organizational
Process
Performance
• Organizational
Performance
Management
Expected Practices Provide Guidance
for Implementation & Institutionalization
Project Planning – Implementation
SG 1 Establish Estimates
SP 1.1 Estimate the Scope of the Project
SP 1.2 Establish Estimates of Work Product
and Task Attributes
SP 1.3 Define Project Lifecycle Phases
SP 1.4 Estimate Effort and Cost
SG 2 Develop a Project Plan
SP 2.1 Establish the Budget and Schedule
SP 2.2 Identify Project Risks
SP 2.3 Plan Data Management
SP 2.4 Plan the Project’s Resources
SP 2.5 Plan Needed Knowledge and Skills
SP 2.6 Plan Stakeholder Involvement
SP 2.7 Establish the Project Plan
SG 3 Obtain Commitment to the Plan
SP 3.1 Review Plans That Affect the Project
SP 3.2 Reconcile Work and Resource Levels
SP 3.3 Obtain Plan Commitment
11
Project Planning - Institutionalization
GG 2 Institutionalize a Managed Process
GP 2.1 Establish an Organizational Policy
GP 2.2 Plan the Process
GP 2.3 Provide Resources
GP 2.4 Assign Responsibility
GP 2.5 Train People
GP 2.6 Manage Configurations
GP 2.7 Identify and Involve Relevant
Stakeholders
GP 2.8 Monitor and Control the Process
GP 2.9 Objectively Evaluate Adherence
GP 2.10 Review Status with Higher Level
Management
GG 3 Institutionalize a Defined Process
GP 3.1 Establish a Defined Process
GP 3.2 Collect Improvement Information
Practice Ratings for the Organization/Projects
12
How is the CMMI Used for Process
Improvement?
IDEAL
Model
www.sei.cmu.edu/ideal/
13
Typical CMMI Benefits Cited in Literature
• Reduced costs
– 33% decrease in the average
cost to fix a defect (Boeing)
– 20% reduction in unit
software costs (Lockheed
Martin)
• Faster Schedules
– 50% reduction in release
turnaround time (Boeing)
– 60% reduction in re-work
following test (Boeing)
• Greater Productivity
– 25-30% increase in
productivity within 3 years
(Lockheed Martin, Harris,
Siemens)
• Higher Quality
– 50% reduction of software
defects (Lockheed Martin)
• Customer Satisfaction
– 55% increase in award fees
(Lockheed Martin)
14
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
15
What is Lean Six Sigma (LSS)?
• Lean Six Sigma is a
powerful approach to
improving the work we do
• LSS improvement projects
are performed by teams
• Teams use a set of tools
and techniques to
understand problems and
find solutions
16
+
• Lean Six Sigma integrates
tools and techniques from
two proven process
improvement methods
Six Sigma
• A management philosophy based on meeting business
objectives by reducing variation
– A disciplined, data-driven methodology for decision making and
process improvement
• To increase process performance, you have to decrease
variation
• Greater
Too early
Defects
Too late
Defects
Delivery Time
Spread of variation
too wide compared to
specifications
17
Too early
Too late
Reduce
variation
Delivery Time
Spread of variation
narrow compared to
specifications
predictability in
the process
• Less waste and
rework, which
lowers costs
• Products and
services that
perform better
and last longer
• Happier
customers
DMAIC Roadmap
Define
Measure
Improve
Control
Define
project
scope
Identify
needed
data
Explore
data
Identify
possible
solutions
Define
control
method
Establish
formal
project
Obtain
data set
Characterize
process &
problem
Select
solution
Implement
Evaluate
data quality
Summarize
& baseline
data
[Hallowell-Siviy 05]
18
Analyze
Update
improvement
project scope
& scale
Document
Implement
(pilot as
needed)
Evaluate
DMAIC Toolkit
Define
Benchmark
Contract/Charter
Kano Model
Voice of the
Customer
Voice of the
Business
Quality Function
Deployment
Measure
GQIM and
Indicator
Templates
Data Collection
Methods
Measurement
System
Evaluation
Analyze
Improve
Control
Cause & Effect
Diagrams/ Matrix
Design of
Experiments
Statistical
Controls:
Failure Modes &
Effects Analysis
Modeling
Control
Charts
Statistical
Inference
Tolerancing
Reliability
Analysis
Root Cause
Analysis,
including 5 Whys
Hypothesis Test
ANOVA
Time Series
methods
Robust Design
Systems
Thinking
Decision & Risk
Analysis
PSM Perform
Analysis Model
Non-Statistical
Controls:
Procedural
adherence
Performance
Mgmt
Preventive
measures
19
Design for Six Sigma (e.g., DMADV)
Define
Define
project
scope
Establish
formal
project
Measure
Identify
customers
Research
VOC
Benchmark
Quantify
CTQs
20
Analyze
Explore
data
Design
solution
Predict
performance
Design
Verify
Develop
detailed
design
Evaluate
pilot
Refine
predicted
performance
Develop
pilot
Scale-up
design
Document
Lean
• Series of tools and techniques refined by Toyota and called
the “Toyota Production System”
– Called “Lean” by Womack, Jones and Roos in The Machine That
Changed the World
• Focused on increasing efficiency by eliminating non-value
added process steps and wasteful practices
• Being adopted world-wide by both manufacturing and
transactional based organizations
• Utilizes tools like “Value Stream Mapping,” “Just in Time” and
“Kaizen”
LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME
21
Wastes in Production
CORRECTION
•
WAITING
Repair or
Rework
Any non-work time
waiting for tools,
supplies, parts, etc..
PROCESSING
Doing more work than
is necessary
Types
of
Waste
INVENTORY
MOTION
Any wasted motion
to pick up parts or
stack parts. Also
wasted walking
OVERPRODUCTION
Producing more
than is needed
before it is needed
CONVEYANCE
Maintaining excess
inventory of raw mat’ls, Wasted effort to transport
materials, parts, or
parts in process, or
finished goods into or
finished goods.
out of storage, or
between
processes.
22
Organizational Adoption:
Roles & Responsibilities
• Champions – Facilitate the leadership,
implementation, and deployment
• Sponsors – Provide resources
• Process Owners – Responsible for
the processes being improved
• Master Black Belts – Serve as mentors for Black Belts
• Black Belts – Lead major Six Sigma projects
– Typically requires 4 weeks of training
• Green Belts – Lead minor Six Sigma teams, or serve on
improvement teams under a Black Belt
– Typically requires 2 weeks of training
23
A Typical Lean Six Sigma Project
in Aerospace
The organization notes that systems integration has been problematic
on past projects (budget/schedule overruns)
A Six Sigma team is formed to scope the problem, collect data from
past projects, and determine the root cause(s)
The team’s analysis of the historical data indicates that ineffective peer
reviews are leaving significant errors to be found in test
Procedures and criteria for better peer reviews are written, using best
practices from past projects
A pilot project uses the new peer review procedures and criteria, and
collects data to verify they solve the problem
The organization’s standard process and training is modified to
incorporate the procedures and criteria, to prevent similar problems on
future projects
24
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
25
Northrop Grumman Approach:
Mission Success Requires Multiple Approaches
Risk Management
Systems Engineering
Independent Reviews
& Cost Estimates
Dashboards for
Enterprise-Wide
Measurement
Program
Effectiveness
Training, Tools, &
Templates
Process
Effectiveness
Mission
Assurance &
Enterprise
Excellence
Communications &
Best-Practice Sharing
Operations
Effectiveness
CMMI Level 5 for
Software, Systems, and
Services
ISO 9001 and AS-9100
Certification
26
Six Sigma
Robust Governance Model
(Policies, Processes,
Procedures)
Organizational Infrastructure Required for CMMI
Level 3
Process Group
Training Program
Measurement Repositories
Predictive Modeling
Best-Practice Libraries
Audits & Appraisals
Defects per component
Policies, Processes,
Templates & Tools
Process Improvement
Communications
25
20
15
UCL
10
_
X
5
0
1
11
21
31
41
51
61
71
Component #
Developing and maintaining mature processes requires
significant time and investment in infrastructure
27
Northrop Grumman Approach:
Institutionalizing Our Improvements
We systematically analyze quality and
process data and trends to determine
how to improve our processes
We improve our process assets
based on internal and external best
practices
Disposition
Analysis
ISO/AS9100
Findings
CMMI Appraisal
Findings
• Systems/
Software
Engineering
Process Group
Customer
Comments
• QMS Working
Group
Independent
Audits
Lessons Learned
& Metrics
Tools
28
Increasing
program
efficiency
• Program
Management
Advisory Board
Industry
Standards
Internal
Best Practices
Configuration
Control Board
Information
Deployed
to
programs
Policy
Process
Six Sigma
Projects
Procedures
External
Best Practices
Checklists and
Guides
Templates and
Examples
eToolkit
StartIt!
msCAS
My MS Portal
PAL
Workbench
PCDB
Northrop Grumman Approach:
Lessons Learned
• Multiple improvement initiatives helps encourage a change in behavior as
opposed to “achieving a level”
– Reinforces that change (improvement) is a way of life
• Benefits results from institutionalizing local improvements across the
wider organization
– CMMI establishes the needed mechanisms
• CMMI and Lean Six Sigma compliment each other
– CMMI can yield behaviors without benefits
– Lean Six Sigma improvements based solely on data may miss innovative
improvements (assumes a local optimum)
• Training over half the staff as Lean Six Sigma Green Belts has resulted in
a change of language and culture
– Voice of Customer, data-driven decisions, causal analysis, etc.
– Better to understand/use tools in everyday work than to adopt the “religion”
29