Global Procurement Supplier Quality
Introduction to MSA
Lise Robert SQS (Supplier Quality Specialist)
Rev. 01 – Dec 17th 2018
Introduction to Measurement System Analysis (MSA)
2
 Everyday our lives are being impacted by more and more data. We have become
a data driven society.
 In business and industry, we are using data in more ways than ever before.
 Today manufacturing companies gather massive amounts of information through
measurement and inspection. When this measurement data is being used to
make decisions regarding the process and the business in general it is vital that
the data is accurate. If there are errors in our measurement system we will be
making decisions based on incorrect data. We could be making incorrect
decisions or producing non-conforming parts. A properly planned and executed
Measurement System Analysis (MSA) can help build a strong foundation for any
data based decision making process.
What is Measurement System Analysis (MSA)
3
 MSA is defined as an experimental and mathematical method of
determining the amount of variation that exists within a measurement
process. Variation in the measurement process can directly contribute
to our overall process variability. MSA is used to certify the
measurement system for use by evaluating the system’s accuracy,
precision and stability.
What is a Measurement System?
4
 Before we dive further into MSA, we should review the definition of a
measurement system and some of the common sources of variation.
A measurement system has been described as a system of related
measures that enables the quantification of particular characteristics.
It can also include a collection of gages, fixtures, software and
personnel required to validate a particular unit of measure or make an
assessment of the feature or characteristic being measured.
What is a Measurement System?
5
Variation
Think of Measurement as a
Process
What is a Measurement System?
6
Measurement
The assignment of numbers to material things to represent the relationships
among them with respect to particular properties.
C. Eisenhart (1963)
What is a Measurement System?
7
 The sources of variation in a measurement process can include the
following:
 Process – test method, specification
 Personnel – the operators, their skill level, training, etc.
 Tools / Equipment – gages, fixtures, test equipment used and their
associated calibration systems
 Items to be measured – the part or material samples measured, the
sampling plan, etc.
 Environmental factors – temperature, humidity, etc.
What is a Measurement System?
8
 All of these possible sources of variation should be considered during
Measurement System Analysis. Evaluation of a measurement system
should include the use of specific quality tools to identify the most
likely source of variation. Most MSA activities examine two primary
sources of variation, the parts and the measurement of those parts.
The sum of these two values represents the total variation in a
measurement system.
Why Perform Measurement System Analysis (MSA)
9
 An effective MSA process can help assure that the data being
collected is accurate and the system of collecting the data is
appropriate to the process.
 Good reliable data can prevent wasted time, labor and scrap in a
manufacturing process.
Why Perform Measurement System Analysis (MSA)
Example
10
 A major manufacturing company began receiving calls from several of
their customers reporting non-compliant materials received at their
facilities sites. The parts were not properly snapping together to form an
even surface or would not lock in place.
 The process was audited and found that the parts were being produced
out of spec. The operator was following the inspection plan and using the
assigned gages for the inspection. The problem was that the gage did
not have adequate resolution to detect the non-conforming parts.
 An ineffective measurement system can allow bad parts to be accepted
and good parts to be rejected, resulting in dissatisfied customers and
excessive scrap. MSA could have prevented the problem and assured
that accurate useful data was being collected..
How to Perform Measurement System Analysis (MSA)
11
 MSA is a collection of experiments and analysis performed to evaluate a
measurement system’s capability, performance and amount of
uncertainty regarding the values measured. We should review the
measurement data being collected, the methods and tools used to collect
and record the data.
 Our goal is to quantify the effectiveness of the measurement system,
analyze the variation in the data and determine its likely source. We need
to evaluate the quality of the data being collected in regards to location
and width variation. Data collected should be evaluated for bias, stability
and linearity.
How to Perform Measurement System Analysis (MSA)
12
 During an MSA activity, the amount of measurement uncertainty must
be evaluated for each type of gage or measurement tool defined
within the process Control Plans.
 Each tool should have the correct level of discrimination and
resolution to obtain useful data. The process, the tools being used
(gages, fixtures, instruments, etc.) and the operators are evaluated for
proper definition, accuracy, precision, repeatability and reproducibility.
How to Perform Measurement System Analysis (MSA)
13
Data Classifications
 Prior to analyzing the data and or the gages, tools or fixtures, we
must determine the type of data being collected. The data could be
attribute data or variable data.
 Attribute data is classified into specific values where variable or
continuous data can have an infinite number of values.
How to Perform Measurement System Analysis (MSA)
14
The Master Sample
 To perform a study, you should first obtain a sample and establish the
reference value compared to a traceable standard. Some processes
will already have “master samples” established for the high and low
end of the expected measurement specification.
How to Perform Measurement System Analysis (MSA)
15
The Gage R&R Study
 For gages or instruments used to collect variable continuous
data, Gage Repeatability and Reproducibility (Gage R & R) can be
performed to evaluate the level of uncertainty within a measurement
system.
How to Perform Measurement System Analysis (MSA)
16
 To perform a Gage R & R, first select the gage to be evaluated.
 Then perform the following steps:









Obtain at least 10 random samples of parts manufactured during a regular production run
Choose three operators that regularly perform the particular inspection
Have each of the operators measure the sample parts and record the data
Repeat the measurement process three times with each operator using the same parts
Calculate the average (mean) readings and the range of the trial averages for each of the
operators
Calculate the difference of each operator’s averages, average range and the range of
measurements for each sample part used in the study
Calculate repeatability to determine the amount of equipment variation
Calculate reproducibility to determine the amount of variation introduced by the operators
Calculate the variation in the parts and total variation percentages
How to Perform Measurement System Analysis (MSA)
17
 The resulting Gage R & R percentage is used as a basis for accepting
the gage. Guidelines for making the determination are found below:
 The measurement system is acceptable if the Gage R & R score falls below
10%
 The measurement system may be determined acceptable depending upon
the relative importance of the application or other factors if the Gage R & R
falls between 10% to 20%
 Any measurement system with Gage R & R greater than 30% requires
action to improve
 Any actions identified to improve the measurement system should be
evaluated for effectiveness
How to Perform Measurement System Analysis (MSA)
18
 When interpreting the results of a Gage R & R, perform a comparison
study of the repeatability and reproducibility values.
 If the repeatability value is large in comparison to the reproducibility
value, it would indicate a possible issue with the gage used for the study.
 The gage may need to be replaced or re-calibrated.
 Adversely, if the reproducibility value is large in comparison with the
repeatability value, it would indicate the variation is operator related.
 The operator may need additional training on the proper use of the gage
or a fixture may be required to assist the operator in using the gage.
How to Perform Measurement System Analysis (MSA)
19
 Gage R & R studies shall be conducted under any of the following
circumstances:




Whenever a new or different measurement system is introduced
Following any improvement activities
When a different type of measurement system is introduced
Following any improvement activities performed on the current
measurement system due to the results of a previous Gage R & R study
 Annually in alignment with set calibration schedule of the gage
How to Perform Measurement System Analysis (MSA)
20
 Attribute Gage R & R







Attribute measurement systems can be analyzed using a similar method.
Measurement uncertainty of attribute gages shall be calculated using shorter method
as below:
Determine the gage to be studied
Obtain 10 random samples from a regular production run
Select 2 different operators who perform the particular inspection activity regularly
Have the operators perform the inspection two times for each of the sample parts and
record the data
Next, calculate the kappa value.
When the kappa value is greater than 0.6, the gage is deemed acceptable
 If not, the gage may need to be replaced or calibrated
How to Perform Measurement System Analysis (MSA)
21
 Attribute Gage R & R
 The attribute gage study should be performed based on the same criteria
listed previously for the Gage R & R study.
 During MSA, the Gage R&R or the attribute gage study should be
completed on each of the gages, instruments or fixtures used in the
measurement system. The results should be documented and stored in a
database for future reference. It may be required for a PPAP submission to
the customer.
 Furthermore, if any issues should arise, a new study can be performed on
the gage and the results compared to the previous data to determine if a
change has occurred. A properly performed MSA can have a dramatic
influence on the quality of data being collected and product quality.
Key terms and definitions
22
 Attribute data – Data that can be counted for recording and analysis (sometimes referred to as go/
no go data)
 Variable data – Data that can be measured; data that has a value that can vary from one sample to
the next; continuous variable data can have an infinite number of values
 Bias – Difference between the average or mean observed value and the target value
 Stability – A change in the measurement bias over a period of time
 A stable process would be considered in “statistical control”
 Linearity – A change in bias value within the range of normal process operation
 Resolution – Smallest unit of measure of a selected tool gage or instrument; the sensitivity of the
measurement system to process variation for a particular characteristic being measured
Key terms and definitions
23
 Accuracy – The closeness of the data to the target or exact value or to an
accepted reference value
 Precision – How close a set of measurements are to each other
 Repeatability – A measure of the effectiveness of the tool being used; the
variation of measurements obtained by a single operator using the same tool to
measure the same characteristic
 Reproducibility – A measure of the operator variation; the variation in a set of data
collected by different operators using the same tool to measure the same part
characteristic
Key terms and definitions
24
 Accuracy – The closeness of the data to the target or exact value or to an
accepted reference value
 Precision – How close a set of measurements are to each other
 Repeatability – A measure of the effectiveness of the tool being used; the
variation of measurements obtained by a single operator using the same tool to
measure the same characteristic
 Reproducibility – A measure of the operator variation; the variation in a set of data
collected by different operators using the same tool to measure the same part
characteristic
25
Measurement Systems
Analysis
Measurement Systems Analysis
26
 Basic Concepts of Measurement Systems
 A Process
 Statistics and the Analysis of Measurement Systems
 Conducting a Measurement Systems Analysis
 ISO - TC 69 is the Statistics Group
 Ensures high ‘Data Quality’ (Think of Bias)
Course Focus & Flow
27
 Measurement as a Process
 Mechanical Aspects (vs Destructive)
 Piece part
 Continuous (fabric)




Features of a Measurement System
Methods of Analysis
Gauge R&R Studies
Special Gauging Situations
 Go/No-Go
 Destructive Tests
Place Timeline Here
28
The Target & Goal
Continuous Improvement
29
Production
Pre-Launch
Prototype
LSL
USL
Key Words
30
 Discrimination
Ability to tell things apart
 Bias [per AIAG] (Accuracy)
 Repeatability [per AIAG] (Precision)
 Reproducibility
 Linearity
 Stability
Terminology
31
 Error ≠ Mistake
 Error ≠ Uncertainty
 Percentage Error ≠ Percentage Uncertainty
 Accuracy ≠ Precision
Measurement Uncertainty
32
 Different conventions are used to report measurement uncertainty.
 What does ±5 mean in m = 75 ±5?
 Estimated Standard Deviation: 
 Estimated Standard Error: m = /√N
 Expanded Uncertainty of ± 2 or 3
Sometimes ± 1 (Why?)
 95% or 99% Confidence Interval
 Standard Uncertainty: u
 Combined Standard Uncertainty: uc
Measurement Uncertainty
33
 Typical Reports
 Physici
Measurement as a Process
34
Basic Concepts
 Components of the Measurement System
 Requirements of a Measurement System
 Factors Affecting a Measurement System
 Characteristics of a Measurement System
Features (Qualities) of a Measurement Number




Units (Scale)
Accuracy
Precision (Consistency or Repeatability)
Resolution (Reproducibility)
Measurement Related Systems
35
Typical Experiences with
Measurement Systems
Basic Concepts
36
 Every Process Produces a “Product”
 Every Product Possesses Qualities (Features)
 Every Quality Feature Can Be Measured
 Total Variation
= Product Variation + Measurement Variation
 Some Variation Inherent in System Design
 Some Variation is Due to a Faulty Performance of the System(s)
The Measurement Process
37
What is the ‘Product’ of the Measurement Process?
What are the Features or Qualities of this Product?
How Can We Measure Those Features?
Measurement Systems Components
38
 Material to be Inspected
Piece
Continuous





Characteristic to be Measured
Collecting and Preparing Specimens
Type and Scale of Measurement
Instrument or Test Set
Inspector or Technician
AIAG calls these ‘Appraiser’
 Conditions of Use
Where Does It Start?
39
During the Design (APQP) Stage:
The engineer responsible for determining inspections and tests, and for
specifying appropriate equipment should be well versed in measurement
systems. The Calibration folks should be part of the process as a part of a
cross-functional team.
Variability chosen instrument must be small when compared with:
Process Variability
Specification Limits
Typical Progression
How will the data be
used?
Determine ‘Critical’
Characteristic
Product Engineer
Determine Required
Resolution
Product Engineer
Consideration of the Entire
Measurement System for
the Characteristic
(Variables)
Cross-Functional
Determine What
Equipment is Already
Available
Metrology
40
Measurement Systems Variables
Material
Inspector
Sample
Collection
Methods
Test Method
Training
Sample
Preparation
Workmanship
Parallax
Practice
Fixture
Eyesight
Air Pressure
Air Movement
Fatigue
Samples
Reproducibility
Ergonomics
Standards
Measurement
Discrimination
Vibration
Bias
Repeatability
Calibration
Linearity
Instrument
Temperature
Lighting
Humidity
Environment
These are some of the variables in a measurement
system. What others can you think of?
41
Determining What To Measure
42
External
Requirements
Convert To
Internal
Requirements
 Voice of the Customer
You Must Convert to Technical Features
 Technical Features
 Failure Modes Analysis
 Control Plan
Voice of the Customer
43
 External and Internal
Customers
 Stated vs Real and
Perceived Needs
 Cultural Needs
 Unintended Uses
 Functional Needs vs.
Technical Features
Customer may specify causes
rather than output
Convert to Technical Features
44








Agreed upon Measure(s)
Related to Functional Needs
Understandable
Uniform Interpretation
Broad Application
Economical
Compatible
Basis for Decisions
Y
Functional Need
Z
Technical Feature
Failure Modes Analysis
45




Critical Features are Defined Here!
Design FMEA
Process FMEA
Identify Key Features
Identify Control Needs
Automotive FMEA
Process Fai lure Mo de And Effects Analysi s
Process :
Prim ary Process Re spon sibi lity:
Eng inee r:
Mode l Ye ar/Ve hicle(s):
Othe r Div. Or Peo ple Involved :
Schedu led Productio n Rele ased :
PFMEA Da te:
Qual ity As suran ce Ma nage r
1
Wrong M aterial
Fragmented Container
Insufficient Supplier Control
Unp redictable Deployment
Improper Handling
Storage Area
M aterial Certification
2
18
3
10
3
90
Required With Each
M isidentified M aterial
Ship ment
Release Verification
Out Of Spec
M aterial
Contaminated
Fragmented Container
Supp lier M aterial
Visual Inspection
1
9
7
63
Fragmented Container
Engineering Change
Release Verification
1
10
7
70
Unp redictable Deployment
Supp lier Change
Fragmentation
Untrained LTO
5
10
1
50
Unp redictable Deployment
M aterial
Change
2
M ove To
Approved
Storage
Unreleased
Periodic Audit Of
Open Boxes
Fragmented Container
M aterial
Comp osition
Supp lier Process Control
Unp redictable Deployment
Green "OK" Tag
Customer Notification
Untrained P ersonnel
Check For Green "OK"
Tag At Press
Trace Card
Check List
Training
Leading to MSA. Critical features are determined by the FMEA
(RPN indicators) and put into the Control Plan.
RPN
9
Actio ns
Ta ken
Dete cti on
1
Recomme nded
Actio ns And
Sta tus
Severity
Curren t Controls
Occured
Potenti al Cau se Of Fail ure
RPN
Take TP PE
M aterial Held In
Potenti al Effects Of
Fa ilure
Dete cti on
SIR
Containe r
Potenti al Failu re
Mode
Sen ior Advisor
Severity
Process Fun cti on
Rev.
Qual ity As suran ce Engin eer
Opera tion s Mana ger
Part Name
Opera tion
Numb er
1 - 10
Part Numbe r:
Occured
Ap provals :
Low - High
Outs ide Suppl iers Affe cte d:
Resp onsi ble
Activity
Control Plan / Flow Diagram
47








Inspection Points
Inspection Frequency
Instrument
Measurement Scale
Sample Preparation
Inspection/Test Method
Inspector (who?)
Method of Analysis
GM Process Flow Chart
Process Flow Diagram
Inspect
Store
Move
Step
Fabrication
Part Number:
Part Description:
Prepared By:
Date:
Rev. :
4/5/93
C
Approved By:
QA Manager
Operations Manager
Senior Advisor
QA Engineer
Operation Description
Item #
Key Product Characteristic
Item #
Key Control Characteristic
1
Move "OK" Vinyl Material
From Storage Area and
Load Into Press.
1.0
Material Specs
1.0
Material Certification Tag
2
Auto Injection Mold Cover
In Tool #
2.0
Tearstrip In Cover
2.1
2.2
Tool Setup
Machine Setup
3.0
Hole Diameter In Cover
2.1
2.2
Tool Setup
Machine Setup
4.0
Flange Thickness In Cover
2.1
2.2
Tool Setup
Machine Setup
5.0
Pressure Control Protrusions
Height
2.1
2.2
Tool Setup
Machine Setup
6.0
Pressure Control Protrusions
Filled Out
2.1
2.2
Tool Setup
Machine Setup
3
Visually Inspect Cover
Standard Control Plan Example
49
Control Plan Number
Key Contact / Phone
Date (Orig.)
Part No./ Latest Change No.
Core Team
Customer Engineering Approval/Date
Part Name/Description
Supplier/Plant Apoproval/Date
Customer Quality Approval/Date
Other Approval/date (If Req'd)
Other Approval/date (If Req'd)
Supplier/Plant
Supplier Code
Characteristics
Part/
Process Name/
Process
Operation
Number
Desc ription
Machine,
Device,
Jig, Tools
for Mfg.
No.
Product
Date (Rev.)
Methods
Process
Special
Char.
Class
Product/
Process
Spec/
Tolerance
Evaluation
Measurement
Technique
Size
Frequency
Control
Method
This form is on course disk
Reaction
Plan
Ford’s Dimensional Control Plan (DCP)
50
Measurement as a System
51
 Choosing the Right Instrument



Instrument Calibration Needs
Standards or Masters Needed
Accuracy and Precision
 Measurement Practices


Where
How Many Places
 Reported Figures


Significant Figures Rule
 2 Action Figures
 Rule of 10
Individuals, Averages, High-Lows
Measurement Error
52
Measured Value (y)
=
True Value (x) + Measurement Error
Deming says there is no
such thing as a ‘True’ Value.
Consistent (linear)?
Sources of Measurement Error
53
 Sensitivity (Threshold)
Chemical Indicators





Discrimination
Precision (Repeatability)
Accuracy (Bias)
Damage
Differences in use by Inspector (Reproducibility)
Training Issues
 Differences Among Instruments and Fixtures
 Differences Among Methods of Use
 Differences Due to Environment
Types of Measurement Scales
54
 Variables
 Can be measured on a continuous scale
 Defined, standard Units of Measurement
 Attributes





No scale
Derived ‘Unit of Measurement’
Can be observed or counted
Either present or not
Needs large sample size because of low information content
How We Get Data
55
Inspection
Measurement
Test
Includes Sensory (e.g..: look,
touch, smell…etc)
Magnitude of Quality
Operational Definitions
56
 Is the container Round?
 Is your software Accurate?
 Is the computer screen Clean?
 Is the truck On Time?
Different Method = Different Results
57
Method 1
Method 2
Out of
Spec
In Spec
Measurement System Variability
58
 Small with respect to Process Variation
 Small with respect to Specified Requirements
 Must be in Statistical Control
Measurement IS a Process!
Free of Assignable Causes of variation
Studying the Measurement System
59
 Environmental Factors
 Human Factors
 System Features
 Measurement Studies
Environmental Factors
60
 Temperature
 Humidity
 Vibration
 Lighting
 Corrosion
 Wear
 Contaminants
Oil & Grease
Aerosols
Where is the study performed?
1. Lab?
2. Where used?
3. Both?
Human Factors
61
 Training
 Skills
 Fatigue
 Boredom
 Eyesight
 Comfort
 Complexity of Part
 Speed of Inspection (parts per hour)
 Misunderstood Instructions
Human Measurement Errors
Unaware of
problem
 Sources of Errors
62
Inadvertent Errors
 Attentiveness
 Random

Training
Issue
Good
Accept
OK!
Good Mistake-Proofing Target
Bad
beta
Technique Errors
 Consistent
Reject
alpha OK!
Wilful Errors (Bad mood)
 Error Types (Can be machine or human)
Type I - Alpha Errors [ risk]
Type II - Beta Errors [ risk]
Process in
control, but
needs
adjustment,
False alarm
Measurement System Features
63
 Discrimination
Ability to tell things apart
 Bias [per AIAG] (Accuracy)
 Repeatability [per AIAG] (Precision)
 Reproducibility
 Linearity
 Stability
Discrimination
64
 Readable Increments of Scale
 If Unit of Measure is too course: Process variation will be lost in
Rounding Off
 The “Rule of Ten”: Ten possible values between limits is ideal
Five Possible Values: Marginally useful
Four or Less: Inadequate Discrimination
Discrimination
65
Range Charts & Discrimination
66
Indicates Poor
Precision
Bias and Repeatability
67
Precise
Imprecise
Accurate
Inaccurate
Bias
You can correct for Bias
You can NOT correct for Imprecision
Bias
68
Standard
Value
Bias
Measurement Scale
 Difference between average of
measurements and an Agreed
Upon standard value
 Known as Accuracy
 Cannot be evaluated without a
Standard
 Adds a Consistent “Bias Factor” to
ALL measurements
 Affects all measurements in the
same way
Causes of Bias
69
 Error in Master
 Worn components
 Instrument improperly calibrated
 Instrument damaged
 Instrument improperly used
 Instrument read incorrectly
 Part set incorrectly (wrong datum)
Bias
70
 Bias - The difference between the observed Average of
measurements and the master Average of the same parts using
precision instruments. (MSA Manual Glossary)
 The auditor may want evidence that the concept of bias is
understood. Remember that bias is basically an offset from ‘zero’.
Bias is linked to Stability in the sense that an instrument may be
‘zeroed’ during calibration verification. Knowing this we deduce that
the bias changes with instrument use. This is in part the concept of
Drift.
Bias
71
 I choose a caliper (resolution 0.01) for the measurement. I measure a
set of parts and derive the average.
 I take the same parts and measure them with a micrometer
(resolution 0.001). I then derive the average.
 I compare the two averages. The difference is the Bias.
Repeatability
72
 Variation among repeated
measurements
 Known as Precision
 Standard NOT required
 May add or subtract from a given
measurement
 Affects each measurement randomly
Repeatability
Measurement Scale
5.15
= 99%
Margin of Error
Doesn’t address Bias
Repeatability Issues
73
 Measurement Steps
 Sample preparation
 Setting up the instrument
 Locating on the part
 How much of the measurement process should we repeat?
Using Shewhart Charts I
74
Repeatability
Using Shewhart Charts II
75
Evaluating Bias & Repeatability
76
 Same appraiser, Same part, Same instrument
 Multiple readings (n≥10 with 20 to 40 better)
AIAG
 Analysis




Average minus Standard Value = Bias
5.15* Standard Deviation = Repeatability
or +/- 2.575  [99% repeatability]
or +/- 2  [95% repeatability]
 Histogram
 Probability
Repeatability Issues
77
 Making a measurement may involve numerous steps
 Sample preparation
 Setting up the instrument
 Locating the part, etc.
 How much of the measurement process should we repeat? How far
do we go?
Bias & Repeatability Histogram
Never include assignable cause errors
Linearity
79
 The difference in the Bias or Repeatability across the expected
operating range of the instrument.
Plot Biases vs. Ref. Values
80
Linearity = |Slope| * Process Variation = 0.1317*6.00 = 0.79
% Linearity = 100 * |Slope| = 13.17%
Causes of Poor Linearity
81
 Instrument not properly calibrated at both Upper and Lower extremes
 Error in the minimum or maximum Master
 Worn Instrument
 Instrument design characteristics
Reproducibility
82
 Variation in the averages
among different appraisers
repeatedly measuring the
same part characteristic
 Concept can also apply to
variation among different
instruments
Includes repeatability which must be accounted for.
Reproducibility Example
83
Calculating Reproducibility (I)
84
 Find the range of the appraiser averages (R0)
 Convert to Standard Deviation using d2*
(m=# of appraisers; g=# of ranges used = 1)
 Multiply by 5.15
 Subtract the portion of this
due to repeatability
Calculating Reproducibility
85
People variance
Times done
Trials
Stability
86
 Variation in measurements
of a single characteristic
 On the same master
 Over an extended period
of time
 Evaluate using Shewhart charts
Evaluate Stability with Run Charts
87
Stability
88
Both gages are stable, but.....
Importance of Stability
89
 Statistical stability, combined with subject-matter knowledge, allows
predictions of process performance
 Action based on analysis of Unstable systems may increase Variation
due to ‘Tampering’
 A statistically unstable measurement system cannot provide reliable
data on the process
90
Methods of Analysis
Analysis Tools
91
 Calculations of Average and Standard Deviation
 Correlation Charts
 Multi-Vari Charts
 Box-and-Whisker Plots
 Run charts
 Shewhart charts
Average and Standard Deviation
92
Correlation Charts
93
 Describe Relationships
 Substitute measurement for desired measurement
 Actual measurement to reference value
 Inexpensive gaging method versus Expensive gaging method
 Appraiser A with appraiser B
Substitute Measurements
94
 Cannot directly measure quality
 Correlate substitute measure
 Measure substitute
 Convert to desired quality
Comparing Two Methods
Magnetic
 Two methods
 Measure parts using both
 Correlate the two
 Compare to “Line of No Bias”
 Investigate differences
95
Line of Perfect Agreement
Line of Correlation
Stripping
Measurements vs. Reference Data
96
Measurements vs. Reference Correlation
97
Disparity
Comparing Two Appraisers
98
Run Charts Examine Stability
99
Multiple Run Charts
More than 3 appraisers confuses things...
Multi-Vari Charts
High Reading
101
 Displays 3 points
 Length of bar; bar-to-bar; Bar cluster to cluster
 Plot High and Low readings as Length of bar
 Each appraiser on a separate bar
 Each piece in a separate bar cluster
Average Reading
Low Reading
Multi-Vari Type I
102
 Bar lengths are long
 Appraiser differences small
in comparison
 Piece-to-piece hard to detect
 Problem is repeatability
Multi-Vari Type II
103
 Appraiser differences are
biggest source of variation
 Bar length is small in
comparison
 Piece-to-piece hard to detect
 Problem is reproducibility
Multi-Vari Type III
104
 Piece-to-piece variation is
the biggest source of
variation
 Bar length (repeatability)
is small in comparison
 Appraiser differences
(bar-to-bar) is small in
comparison
 Ideal Pattern
Multi-Vari Chart Example
105
Normalized Data
Multi-Vari Chart, Joined
106
Look for similar pattern
Using Shewhart Charts
107




Subgroup = Repeated measurements,, same piece
Different Subgroups = Different pieces and/or appraisers
Range chart shows precision (repeatability)
Average chart “In Control” shows reproducibility
If subgroups are different appraisers
 Average chart shows discriminating power
If subgroups are different pieces
(“In Control” is BAD!)
Shewhart Charts
108
This is not a good way to plot this data
Too many lines
Shewhart Chart of Instrument
109
110
Gage R&R Studies
Gauge R&R Studies
111
 Developed by Jack Gantt
 Originally plotted on probability paper
 Revived as purely numerical calculations
 Worksheets developed by AIAG
 Renewed awareness of Measurement Systems as ‘Part of the
Process’
Consider Numerical vs. Graphical Data Evaluations
Terms Used in R&R (I)
Minimum of 5.
2 to 10 To
accommodate
worksheet factors
 n = Number of Parts [2 to 10]
Parts represent total range of process variation
Need not be “good” parts. Do NOT use consecutive pieces.
Screen for size
 a = Number of Appraisers
Each appraiser measures each part r times
Study must be by those actually using
 R - Number of trials
 Also called “m” in AIAG manual
 g = r*a [Used to find d2* in table 2, p. 29 AIAG manual]
1
2
3
4
5
1 Outside Low/High
1 Inside Low/High
Target
112
Terms Used in R&R (II)
113
 R-barA = Average range for appraiser A, etc.
 R-double bar = Average of R-barA, R-barB
 Rp = Range of part averages Process Variation
 XDIFF = Difference between High & Low appraiser averages
Also a range, but “R” is not used to avoid confusion
 EV = 5.15 = Equipment variation (repeatability)
 EV = 5.15 = Equipment variation (reproducibility)
 PV = Part variation
 TV = Total variation
R&R Calculations
114
Left over
Repeatability
Remember Nonconsecutive
Pieces
Left over
Repeatability
Measurement
System Variation
Product Process
Variation
Accumulation of Variances
115
Evaluating R&R
116
 %R&R=100*[R&R/TV] (Process Control)
 %R&R=100*[R&R/Tolerance] (Inspection)
 Under 10%: Measurement System Acceptable
 10% to 30%: Possibly acceptable, depending upon use, cost, etc.
 Over 30%: Needs serious improvement
Analysis of Variance I
117
 Mean squares and Sums of squares
 Ratio of variances versus expected F-ratio
 Advantages
Any experimental layout
Estimate interaction effects
 Disadvantages
Must use computer
Non-intuitive interpretation
Analysis of Variance II
118
 The n*r measurements must be done in random sequence [a good
idea anyway]
 Assumes that EV [repeatability] is normal and that EV is not
proportional to measurement [normally a fairly good assumption]
 Details beyond scope of this course
Special Gauging Situations
119
 Go/No-Go
 Destructive Testing
If Gauges were Perfect
120
But Repeatability Means We Never Know The Precise
Value
121
So - Actual Part Acceptance Will Look Like This:
122
The Effect of Bias on Part Acceptance
123
Go/No-Go gauges
124
 Treat variables like attributes
 Provide less information on the process, but...
 Are fast and inexpensive
 Cannot use for Process Control
 Can be used for Sorting purposes
“Short” Go/No-Go Study
125
 Collect 20 parts covering the entire process range
 Use two inspectors
 Gage each part twice
 Accept gauge if there is agreement on each of the 20 parts
* May reject a good measuring system
Destructive Tests
126
 Cannot make true duplicate tests
 Use interpenetrating samples
 Compare 3 averages
 Adjust using √n
Destructive Tests: Interpreting Samples
127
AIAG does not address
128
Summary
Measurement Variation
129
 Observed variation is a combination of the production process PLUS
the measurement process
 The contribution of the measurement system is often overlooked
Types of Measurement Variation
130
 Bias (Inaccuracy)
 Repeatability (Imprecision)
 Discrimination
 Linearity
 Stability
Measurement Systems
131
 Material
 Characteristic
 Sampling and Preparation
 Operational Definition of Measurement
 Instrument
 Appraiser
 Environment and Ergonomics
Measurement Systems Evaluation Tools
132
 Histograms
 Probability paper
 Run Charts
 Scatter diagrams
 Multi-Vari Charts
 Gantt “R&R” analysis
 Analysis of Variance (ANOVA)
 Shewhart “Control” Charts
Shewhart Charts
133
 Range chart shows repeatability
 X-bar limits show discriminating power
 X-double bar shows bias
(if a known standard exists)
 Average chart shows stability
(sub-groups overtime)
 Average chart shows reproducibility
(sub-groups over technicians/instruments)
Conclusion
134
 Rule of Ten
 Operating Characteristic Curve
 Special Problems
Go/No-Go Gages
Attribute Inspection
Destructive Testing