Solution Identification and
Verification of Effectiveness
ASQ PALMETTO SECTION
JUNE 9, 2009
Focus of this presentation
• 3 possible solutions
for each root cause
• Solution selection
methodology
• Change management
as it relates to
solution
implementation
• 3 levels of verification
– Evaluation of solution
effectiveness
– Team verification
– Independent
verification
• Measuring solution
effectiveness
• Verification tools
In our previous presentations. . .
• CREI statement for communicating
problems
• Isolating problems to their process of
origin and related tools, (process flow,
timeline, Is/Is Not analysis)
• Four levels of root cause investigation and
related tools, (control barrier analysis,
fishbone, 5 why, system cause analysis)
Visual Definition of Problem
• Gap between current
condition, (what is),
and the desired
performance level,
(what must be, should
be or could be)
• This gap can exist in a
process, product or
service
• The gap can be actual,
potential or “generated”
How is Problem Stated?
• Concern – what is; what was observed,
detected, etc.; what is the nonconforming
condition
• Requirement – what should be; refer to
defined requirement/specification in detail
• Evidence – what was observed that indicates
there is a gap between what is and what
should be; the more detailed the data, the
easier problem definition will be, (when,
where, how many, etc.)
• (Impact) – initial estimation of how the
problem may affect the organization; used in
establishing the priority/need for disciplined
problem solving; best stated financially
Correction vs. Corrective Action
• Correction is action to
eliminate
nonconformity
• Typically action is applied
only at location where
nonconformance was
identified
• Is not designed to
prevent the
nonconformance from reappearing elsewhere
• Corrective action is action
to eliminate the cause
of a detected
nonconformity
• By applying appropriate
corrective action,
recurrence of the
nonconformance is
typically prevented
Problem Type Considerations
Problem
Type
Process of
Origin
Method
Considerations
Just do it
Known
Troubleshooting;
rework
Seen before; can
live with impact
when problem
recurs
Dig Deeper
Unknown
Root cause
analysis
Data-driven
investigation to
determine actual
factors causing
problem condition
Unknown
Fire-fighting
Taking action
possibly on wrong
process; not using
data to confirm
root cause
Main Functions of Problem Solving
• Define Gap between “what
is” and “what should be”
• Identify process of origin
from which gap is
originating
• Study the process of origin
to determine which process
factor(s) are causing the
gap
• Analyze the relationship
between process causal
factors and system factors
to identify all levels of root
causes
Process View
Products/Services = output of producing Processes
Producing Processes to accomplish Plans
Planning Processes apply System
to fulfill customer requirements
System Processes = Policies, Objectives & Practices
(how an organization does business)
The Secret to Solving
Problems
• The source of every problem is a process: typically
the gap is found in the output of the process
• The cause of every problem is one or more process
factors not behaving as they should
• Understanding the relationship between process
factors and process outputs is important to effective
problem solving
• Data about the process and the problem is required to
gain enough understanding to effectively solve any
problem
• The result of any problem solving effort is increased
knowledge about processes and their outputs
Describe Problem
• Symptoms are only starting point in problem
definition
• What is (actual) – What should be = Problem
• Use quality and investigation tools: process
flow, timeline, interview
• Is/Is not analysis (splitting the dictionary)
Output:
• Operational definition – precise explanation
of problem specifying the process of origin to
study for root cause
Failure Modes & Effects Analysis
(FMEA) – Problem Solving Clues
Process
Function
Requirements
Potential
Failure
Modes
Potential
Failure
Effects
Potential
Failure
Causes
Current
Product &
Process
Controls
Process of
origin
Technical
definition of
problem
Symptom
Process
factors = root
causes
Interim
actions
Process Flow & Timeline
• Process mapping
process flow
• End point of flow
reflects where problem
was found
• Flow extends back to
process where
problem feature is
initially created or
changed
• Process of origin of the
problem will be
somewhere within these
process flow
boundaries
• Begin when the
problem was found
• Data may help identify
when problem
originated
• With traceability data,
may be able to
recognize time-related
pattern of problem
• Both of these tools will
also help with
containment and
application of interim
action
Is/Is Not Analysis
• Also known as Stratification Analysis
• Provides further detail about the problem so
process of origin can be identified and a
complete operational definition of the problem
can be formulated.
• Used at this stage as well as in applying
interim/containment actions and
implementing/verifying permanent actions.
• “Splitting the dictionary” or “20 questions to
the answer” demonstrates this idea of
problem convergence
Components of Operational
Definition of Problem
• Basis for root cause investigation
• Indicate process from which problem
originated/generated
• Indicate direction of problem related to
requirement
• Define extent of problem
• Possibly isolates problem to a certain
timeframe
• Include refined information re: impact
• Problem statement must be clear, concise
and understandable by anyone
4 Levels of Root Cause
Defect/Detection Cause = Product level
Process Root Cause = at Process of Origin
Actual Root Cause = previous process factor
contributing to Process Root Cause, (planning)
System Root Cause = management system
policy/practice contributing to Actual Root Cause
Root Cause Analysis Levels
Root Cause
Consideration
Tools
Other
(Wide)
Product
Defect/Detection
cause
Condition of
controls to
detect problem
Control
Barrier
Analysis
What other
products have
similar
controls?
Process
Direct process
cause, (trigger at
process of origin
Factors at
process of
origin triggering
problem, (5Ms)
Fishbone,
(cause &
effect)
What
processes
have similar
trigger cause?
Plan
Actual root cause,
(led to trigger
cause)
Linkage to
planning
processes that
trigger cause
5 Why with
Hypothesis
testing
What other
processes
affected?
System
“weakness” in
mgt. policies or
practices
Linkage of mgt.
system to
actual cause
System
Cause
Analysis
Level
(Deep)
Other affected
mgt. policies
Control Barrier Analysis
(Defect/Detection Root Cause)
• How did the problem
escape the process
and/or organization?
• Was the problem
anticipated in advance?
• Were controls defined
to recognize and
contain the problem?
• At which process are
the planned controls
applied?
• Were the planned
controls in place?
• Were the planned
controls working?
• What is the capability
of these controls?
• Assists in identifying
appropriate interim
actions as well as
identifying the
defect/detection root
cause
Process Cause at Process of Origin
• Relates one or more factors of the affected
process, (process of origin), not “behaving”
as required to obtain the desired output result
at that process
• Use Cause & Effect diagram, (fishbone
technique)
• Direct process causes, (trigger causes), are
the starting point for identifying root cause
• Some action may be required to address the
direct process/trigger cause but actions
should not be taken until actual root cause is
known
Actual Root Cause
• Explains why condition exists at the process of
origin of the problem
• Typically found in previous “planning” processes
• Use 5 why analysis and hypothesis testing
• Usually only one over-riding cause that when
addressed, can significantly reduce the
problems impact on the organization
• Very complex problems may have interacting
causes but these are typically viewed as isolated
problems that only repeat infrequently, (often
managed as Just Do It), until resources allow
necessary time to discover interaction through
data collection, analysis and experimentation
System Causes
• What in the system allowed this problem/cause
to occur
• Identifies why the process root cause occurred
based on current management policies/practices
• Often not readily measurable
• Data obtained through interview
• By identifying system causes, systemic
improvement can be made in order to prevent
recurrence of problem in other similar processes
• Typically addressed once process root cause of
problem is known
Plan/Implement Solutions
• Inputs:
– Confirmed process root
causes
– Confirmed system root
causes
– SMART goals
– Solution criteria
• Methods:
– Brainstorming solutions
– Decision-making process
– Planning
• Outputs:
– Solutions for process of
origin, (if appropriate)
– Solutions addressing
process root cause
– Solutions addressing
system root cause
– Error/mistake-proofing
– Solutions implementation
plans
– Contingency plans
– Action plans
Plan, Implement
& Verify Solutions
Brainstorm
possible solutions
for each confirmed
root cause
Permanent
solution
implementation
4/8/2007
Establish solution
selection criteria
Evaluate results of
permanent
solution
Evaluate possible
solutions vs.
solution criteria
Remove interim
actions
Develop action
plan to implement
selected solutions
Team verification
of solution vs.
goals
Evaluate solution
risks and impact
on other
processes
Independent
verification of
problem solving
effort
Develop
contingency plan
for solutions
Finalize problem
solving report,
lessons learned
Establish solution
effectiveness
measures
Team celebration
and disbanding of
problem solving
team
Trial plan for
solution
implementation
Evaluate trial plan
results
Revise solution
implementation
plan as necessary
3 Possible Solutions
• Eliminate root cause – preventive control; often
referred to as error-proofing; eliminates causal factor
leading to problem condition
• Control root cause – process detective control;
implement actions to monitor cause condition so
action can be taken on process factor before problem
occurs
• “Do nothing”/Live with it – reactive control;
continue monitoring for problem condition; defect
detection solution; may be required when root cause
can’t be eliminated or controlled economically or
technically; this solution may include accepting
interim action as permanent solution
Problem Solutions
• There are always at
least 3 possible
solutions related to
each level of cause
• Therefore, at least 12
possible solutions could
be identified for a
problem investigation if
all levels of cause are
investigated!
• Management provides
solution selection
criteria as basis for
evaluating possible
solutions
Possible Solutions Matrix
Root
Cause(s)
Defect:
Direct:
Actual:
System:
Confirmed
Eliminate
Control
Detect Gap
Process Solutions
• Address process root
cause
• Impact control or
elimination of process
factor(s) identified to
be root cause
• Consider errorproofing, (elimination
of cause), or mistakeproofing, (control of
cause)
Error-Proofing vs. Mistake-Proofing
•
•
•
•
Error-proofing
Product or process features
designed such that a
potential failure mode and/or
its cause can not occur
Eliminate the possibility for
the failure mode and/or
cause to occur
Also includes self-correcting
cause detection
mechanisms, (e.g.
regulators)
E.g. polymer bonding of
teeth
•
•
•
•
•
Mistake-proofing
Detection mechanisms
designed into the process to
identify the cause of a
potential failure mode
prevent manufacture of
nonconforming product
Also referred to as pokayoke
Installed at the process
where defect could occur
Mechanisms which detect
incorrect cause conditions
and alert the operator to
these conditions so remedial
action may be taken
E.g. automatic shut-off on
iron
System Solutions
• Address system root
cause
• Create new policy
• Change existing policy
• Monitor application of
current policy
• Apply process solution to
other similar processes
which could experience
the same problem
Real Problem Solving Example
Background: Company X manufactures heavy-duty support brackets for
transport vehicles. Company X recently received a complaint from one of
their customers that the brackets they were receiving were stuck to
cardboard packaging spacers between each layer of product.
Concern: brackets stuck to cardboard spacers
Requirement: per customer packaging specification, “product packaging
shall not damage product quality nor shall packaging affect assembly
process”
Evidence: customer complaint
Impact: disruption to assembly process, rework of brackets $$$, potential
safety issue
Process of origin: line 3 Pack
Operational Definition: 50% of brackets randomly
distributed throughout the container, packed in past 2
weeks were found sticking to cardboard spacers beginning
at the Line 3 Pack process for Customer X.
Containment & Interim Actions: review of stock at Pack, Storage,
Shipment and Customer
Defect/Detection Root Cause: no inspection currently in place to
recognize condition of parts sticking to cardboard spacers at Pack
operation
Direct Process Root Cause: conveyor speed increased
Actual Root Cause: 50% increase in customer demand from Customer X
Possible Solutions Matrix - Example
Root
Cause(s)
Confirmed
Detection:
No inspection
for condition
of parts
sticking to
cardboard
Direct:
Conveyor
speed too fast
Eliminate
Control
Detect Gap
add operator
check of parts
Redesign
process to
eliminate
conveyor
Single speed
conveyor drive
Lock
conveyor
speed control
box
Alarm when
conveyor
speed
changed
Check
conveyor
speed during
shift
add operator
check of parts
add operator
check of parts
add operator
check of parts
Actual:
Increase in
customer
demand
Limit
production to
customers
Level
customer
production
demand
Add an extra
line
Add an extra
shift
Adjust line
speeds
add operator
check of parts
System:
No procedure
for evaluating
impact of
change in
customer
demand
Create
procedure to
address all
types of
changes in
customer
requirements
Create
procedure to
address
change in
customer
demand
Wait for
another
customer
complaint!
Solution Selection
• Allow brainstorming of possible solutions at
all levels of confirmed causes and the 3
possible categories of solutions
• Then apply solution selection criteria provided
by management to evaluate each possible
solution as well as refine the brainstormed
ideas
• Have data available re: actual costs
associated with problem, (initial impact,
revised impact based on data
collection/analysis, anticipated future impact if
no action is taken)
CRITERIA MATRIX
SOLUTIONS
A
CRITERIA
1
MUSTS
2
3
4
n*
1
WANTS
2
3
4
n*
RATING TOTALS
* reflects any number of variables that are appropriate to include in the analysis.
B
C
D
E
n*
CRITERIA MATRIX
SOLUTIONS
A
CRITERIA
MUSTS
1
No new equipment
2
No additional work shifts
3
Meet increased customer demand
4
n*
WANTS
1
Flexible; can be reversed if customer demand
changes
2
Review condition of finished product
3
4
n*
RATING TOTALS
* reflects any number of variables that are appropriate to include in the analysis.
B
C
D
E
n*
Implementing Solutions
• Actions to eliminate
and control causes
require change
• Also solutions that
continue to monitor
for problem condition
• Change management
tools should be
applied when
implementing
solutions
Change Management
Tools
• FMEA
• Risk assessment
• Resource planning
• Contingency
planning
• Training
• Evaluation
• Verification
Apply Quality Planning Process to
Implement Solutions
• Determine if selected solutions will resolve problem
• Test selected solutions prior to full-scale
implementation
• Use decision analysis tools - consensus, criteria
rating, etc.
• Evaluate adverse affects caused by solution; use
FMEA
• Consider solution’s impact on other processes
• Develop contingency plans & countermeasures
• Prepare action plan to manage verification activities
• Verify that customer is satisfied with solution
Implement Permanent Solutions
• Permanent actions should answer “why did this
problem occur?”
• Eliminates concern without creating other
problems
• Establish an action plan
• Define on-going process controls
• Statistical plan to measure effectiveness of
corrective actions
• Identify contingency actions
• Controls for monitoring long-term effectiveness
• Document changes
• Provide training re: changes
Evaluation of Results
• Inputs:
– Solutions
implementation
plans
– SMART goals
• Methods:
– Trials of solutions
– Monitoring/data
collection of
implemented
solutions
• Outputs:
– Data for evaluation of
solutions effectiveness
– Other opportunities
Key Questions to Ask
• Is objective data available to prove that actions taken
work?
• Has enough time elapsed to prove the Issue is truly
fixed?
• Since solutions were implemented, has the Issue
been seen again?
• What efforts have been made to track and report
recurrence of the Issue?
• Is sufficient evidence available in a presentable
format to prove solutions are working?
• Has the team looked for proof that the fix is working
as planned?
Evaluate Solutions
• Establish measures associated with problem
cause to determine if implemented solution is
effective
• This step is data collection of solution results to
support verification of effectiveness
• Data collection may be done by problem solving
team or process owners
• Monitor application of contingency actions
• Time needed for evaluation is dependent on
problem process of origin’s business cycle, (how
often that process occurs and generates data to
support evaluation)
Verification
• Inputs:
– Data from evaluation
of implemented
solutions
– SMART goals
• Methods:
– Problem solving team
verification
– Independent
verification
• Outputs:
– Objective evidence
reflecting solutions
effectiveness in
fulfilling SMART goals
for problem solving
effort
– Evidence of continual
improvement
– Other opportunities
Verifying Solutions
• Verification plan defined
• Confirm effectiveness of solution in
eliminating or controlling root cause
• Statistical basis
• Data collected to support verification
• Interim actions should not be removed
until permanent solution is verified
• Verification plan should contain action,
statistical confidence and results
Problem Identification:
Item
Detail
CREI
Concern clearly
statement
stated as failure
mode or effect
Documented
requirement
referenced
Evidence includes
available details re:
where, when, etc.
Impact addresses
cost and
performance
Problem type
Just Do It or Dig
Deeper?
External
Immediate actions
problem
taken?
Priority
Management has
established
problem priority
Management has
assigned a team
champion
Management has
established SMART
goals for this
problem
Other
Step 1: Establish Team:
Item
Detail
Team
Team champion
members
has identified core
team
Adjunct team
members identified
Team
Team leader
structure
selected
Result
Comments
Result
Comments
Independent Verification
• Possibly performed by internal auditor or someone
not directly involved in problem solving effort
• Review of problem solving effort and results
• Checklist of questions prepared based on results
• Problem solving team members and others affected
interviewed
• Data collected to support answers to questions
• Results of team problem verification reviewed
• Related indicators reviewed
• Conclusion re: implementation and effectiveness of
problem solving effort
How Verification Audits are Performed
• Auditor reviews problem solving information
prepared by problem solving team
• Auditor prepares checklist of questions based on
problem solving information, (fix it actions, how root
cause was identified, actions taken to address
cause, how results of actions were evaluated, etc.)
• Auditor performs audit to identify evidence which
supports that each step of problem solving process
has been completed and that implemented actions
have eliminated/controlled the causes to prevent
recurrence of the problem
• Auditor also reviews information collected
demonstrating results of implemented actions and
data reflecting whether the problem has recurred
Results of Verification
• Record results of verification audit; formal audit
report not required when only reporting results of
verification audit
• If results of verification audit demonstrate that
problem solving is not complete or not effective,
communicate this to manager of area and
Management Representative; the problem solving
effort can not be closed until specified actions have
been taken and data exists to demonstrate that
problem solution was effective in preventing
recurrence of the problem
• Verification audits should also be performed for
preventive and improvement actions; same process
except no “fix it” actions would be required
Problem Closure
• Inputs:
–
–
–
–
Team plan
SMART goals
Results of Verification
Records of problem solving
effort
• Methods:
– Evaluation
– Presentation
– Celebration
• Outputs:
– Problem solving report
– Listing of additional
opportunities identified
– Lessons learned
Problem Closure & Congratulate Team
• Acknowledge significance and value of problem
solution
• Recognize team’s collective efforts in solving the
problem as well as individual contributions
• Document what was learned in solving the
problem; lessons learned not only about the
problem which was solved but also about the
problem solving process
• Consider investigating other potential causes as
preventive actions
• Write case study reports
Importance of Documenting
Problem Solving Effort
• Provides explanation of pathway used in solving
problem
• Captures the data collected and analyzed
• Clearly states decisions made and solutions
implemented
• Sufficient information should exist for
understanding the problem solving effort and its
results; “tells the story”
• May refer to other documents which support
results of problem solving effort
A Key Outcome of Every Problem
Solving/Root Cause Investigation. . .
Expansion of Knowledge
An Anonymous Quote
“Within each problem
lies a disguised
opportunity. . . But it
is the art of
unmasking the
disguise that
distinguishes between
the two.”