Phase 2
Process Measurement
(Class Exercise)
Process Measurement – Classroom Exercise
Table of Contents
Page
Measurement Capability
ACI: Tuner Alignment ............................................................................... 3
Tasks ................................................................................................. 6
Manual Assembly: Reversed Parts ............................................................ 11
Tasks ............................................................................................... 11
Solder: Solder Quality ............................................................................. 13
Tasks ............................................................................................... 13
List of Figures
Fig 4: Possible Sources of Variation for a Process & It’s Measurement System . 7
List of Tables
Table
Table
Table
Table
Table
5:
6:
7:
8:
9:
Spring 2004
Original Measurement Study Data .................................................. 8
Revised Measurement Study Data .................................................. 9
Stability Data Taken After Revised Measurement Study .................. 10
Assessing Operator Ability to Detect Reversed Parts ....................... 12
Assessing Solder Quality on a Nominal Scale ................................. 14
Page 2 of 14
Process Measurement – Classroom Exercise
Measurement Capability
The next step in the overall measurement process is to analyze the capability of the
measurement system. We now have to decide what outputs and inputs to measure.
1. For the ACI, the only output (type of defect) requiring measurement by a
precision instrument is Tuner Misalignment. All the other most frequently
occurring and significant types of defects (outputs) as well as inputs would
need to be observed/measured visually.
2. For MA, all the most frequently occurring and significant types of defects
(outputs) as well as inputs would also need to be observed/measured
visually.
3. For Soldering, all the most frequently occurring and significant types of
defects can be observed/measured visually. However, the inputs would need
to be measured with instruments whose measurement capabilities may need
to be determined.
It was decided to first focus on Tuner Alignment, which is associated with the ACI
operation. Measurement capability studies on manual assembly and soldering would
be done afterwards.
ACI Tuner Alignment
The tuner is inserted into a designated slot on the PCB and has a current tolerance of
+ 8 microns from its target position. This tolerance was previously
+ 11 microns, but was recently changed because customers were experiencing
voltage irregularities resulting in excess component failures in the field, as well as
poor tuner performance. This was believed to be caused by allowing too great a
tolerance from target for tuner insertion. The insertion of the tuner on the PCB is
done by the ACI, and is based on a component placement algorithm written into the
software. A simple description of this process operation is as follows: (1) the tuner
is inserted into the component magazine known as a feeder rack, (2) an insertion
head moves from its home position to pick an appropriate component (tuner) from a
sleeve and then places it in the designated target slot on the PCB, (3) the deviation
of its insertion from the target is automatically measured and recorded, and (4) for
any deviation between placement and target, the PCB location is then adjusted for
such bias by the algorithm, for placement of the next tuner on the next PCB to
attempt to assure that it is on target. The quality characteristic of interest is the
deviation of the inserted tuner from the designated target, as measured by the sum
of the deviations squared from the x and y axes, i.e.,
D2 = x2 + y2
where
x-T = deviation of tuner placement from target on the horizontal axis
y-T = deviation of the tuner placement from target on the vertical
axis
T = target (=0)
Spring 2004
Page 3 of 14
Process Measurement – Classroom Exercise
If D2 > 64, the tuner placement is rejected (needs to be reworked or scrapped),
otherwise it is accepted and goes on to the next stage of assembly. To begin the
measurement study of the instrument monitoring tuner alignment, Pradeep thought
it prudent to review some measurement study fundamentals.
Measurement Study Fundamentals
Any information gathered about process behavior must first pass through a
measurement system. In other words, output from the process must pass through a
second process, which is measurement. The measurement process should be
managed and treated as an actual process requiring application of the DMAIC
strategy phases and associated tools of process mapping, cause & effect, etc. A
chart showing a breakdown of the possible sources of variation for a process and its
measurement system is given in Figure 4.
A measurement system can have both bias and variation. Measurement system
bias can be determined through a ‘Calibration Study.’ Measurement system
variability can be investigated through a ‘R & R’ Study (repeatability and
reproducibility). This can be stated more technically as follows:
μ observed (total) = μ actual (part) + μ measurement system
σ2 (total) = σ2 actual (part) + σ2 measurement system
where
μ is the mean
σ2 is the variance.
Observed values are the actual readings which are composed of the actual (true)
value for a part and that attributable to the measurement system.”
We define the following additional terms:
Spring 2004
o
Linearity…a measure of how the size of a part affects the accuracy of
the measurement system. It is the difference in the observed
accuracy values through the expected range of measurements.
o
Accuracy…a measure of the bias in the measurement system. It is
the difference between the observed measurement and a master
value.
o
Stability…a measure of how accurately a system performs over time.
It is the total variation obtained with a particular measurement
instrument, on the same part, when measuring a single characteristic
over time.
o
Precision…precision or measurement variation is the variation due to
a measurement system and can be broken down into the following two
components:
Page 4 of 14
Process Measurement – Classroom Exercise

Repeatability…The variation observed when the same
operator measures the same part repeatedly with the same
device.

Reproducibility…The variation observed when different
operators measure the same parts using the same instrument.
The bottom line is that we really want to answer the following questions:
1. How large is the measurement bias?
2. How big is the measurement variation?
3. What are the sources of bias and measurement variation?
4. Is the measurement instrument stable over time?
5. How capable is the instrument for measuring tuner alignment?
6. How can we improve the measurement system?
Minitab can perform such a measurement study as follows:

A Gage Run Chart, which is a plot of all observations by operator and part
number. A horizontal reference line is drawn at the mean, which can be
calculated from the data, or a value can be entered from prior knowledge of the
process. You can use this chart to quickly assess differences in measurements
between different operators and parts. A stable process would give you a
random horizontal scattering of points; an operator or part effect would give you
some kind of pattern in the plot.

A Gage R&R Study determines how much of the observed process variation is
due to measurement system variation. Either of two methods can be used to
perform this analysis, the latter being more informative: (a) X and R, or (b)
ANOVA (Analysis of Variance). The X and R method breaks down the overall
variation into three categories: (1) part-to-part, (2) repeatability, and (3)
reproducibility. The ANOVA method goes one step further and breaks down
reproducibility into its operator, part, and operator by part interaction.

A Gage Linearity and Accuracy Study tells how accurate the measurements
are through the expected range of the measurements. It answers the question,
“Does the gage have the same accuracy for all sizes of parts or objects being
measured?” A gage accuracy study examines the difference between the
observed average measurement and a reference or master value. It answers the
question, “How accurate is the measurement instrument when compared to a
master value?”
The metric that is often used to assess measurement capability is the Precision-toTolerance (P/T) Ratio. Namely, it is usually 5.15 * total gage R&R standard
deviation (obtained from the output of the Gage R&R Study) divided by the
tolerance, i.e.,
Spring 2004
Page 5 of 14
Process Measurement – Classroom Exercise
P /T 
5.15 * Sigma(Total _ Gage _ R & R)
USL  LSL
where
USL, LSL are the upper and lower specification limits respectively.
If the P/T ratio is <10%, the measurement instrument is considered acceptable; if it
is between 10% and 30%, it is marginal; and greater than 30% it is unacceptable.”
Armed with this knowledge, data was collected on the ACI tuner and the following
studies were performed (a) a gage run chart, (b) gage R&R study (crossed), and (c)
a gage linearity study. In performing the measurement study ten parts, two
operators, and two measurements per operator were collected, totaling forty
data points. The data are shown in Table 51.
Tasks
1. Perform a Gage R & R Study (crossed) and analysis. Comment on each of the
respective studies: (a) Runchart, (b) Gage R&R (ANOVA), and (c) Gage
Linearity. Is there bias in the measurements? What are the magnitudes and
sources of measurement variation in an absolute and relative sense?
2. Compute the measurement capability in terms of the P/T Ratio. What does
the ratio measure? Is the measurement system capable with respect to what
is being measured?
3. A second similar measurement capability study was subsequently performed
after some improvements to the measurement system were made. The data
for this study are shown in Table 6.
a. Compare the results obtained here to the original measurement study.
b. What is the new measurement capability in terms of the P/T Ratio with
respect to the improved measurement system? Is the measurement
system acceptable?
c. What possible improvements were made to the measurement system
that yielded the improvements in the second measurement study?
d. The stability of the measurement system was also assessed by taking
readings of the measurement system once per day for 30 days after
the second measurement study. The data is contained in Table 7.
1
Tables 5, 6 and 7 have been inserted into the electronic copy of this file as a Microsoft Excel Worksheet
Object. To analyze the data, simply double click anywhere within the table, select the range of data to be
analyzed, and copy the data into the preferred (Minitab) application
Spring 2004
Page 6 of 14
Process Measurement – Classroom Exercise
Figure 4: Possible Sources of Variation for a Process and Its
Measurement System
Parts
(Example)
Inputs
Process
Outputs
Measurement
Process
Inputs
Outputs
Observations
Measurements
Data
Observed Process Variation
Actual Process Variation
Long-term Process
Variation
Short-term
Process Variation
Measurement Process Variation
Variation
within sample
Repeatability
Spring 2004
Variation due
to gage
Calibration
Variation due
to operators
Stability
“Other” Sources
+ Environmental
+ etc…
Linearity
Page 7 of 14
Process Measurement – Classroom Exercise
Table 5: Original Measurement Study Data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Spring 2004
Part No. Operator Response
1
O1
9.01
1
O1
1.25
2
O1
6.25
2
O1
6.10
3
O1
12.52
3
O1
9.28
4
O1
4.36
4
O1
4.50
5
O1
13.78
5
O1
21.25
6
O1
9.64
6
O1
9.86
7
O1
9.04
7
O1
8.00
8
O1
12.29
8
O1
11.17
9
O1
18.72
9
O1
5.80
10
O1
3.77
10
O1
15.17
1
O2
7.46
1
O2
9.21
2
O2
10.25
2
O2
10.06
3
O2
6.66
3
O2
5.76
4
O2
13.25
4
O2
8.54
5
O2
8.00
5
O2
15.65
6
O2
10.25
6
O2
0.00
7
O2
8.65
7
O2
3.49
8
O2
8.01
8
O2
8.76
9
O2
6.74
9
O2
8.85
10
O2
12.49
10
O2
13.68
Master
5.0
5.0
6.0
6.0
10.0
10.0
5.0
5.0
15.0
15.0
10.0
10.0
8.5
8.5
12.0
12.0
12.0
12.0
9.0
9.0
5.0
5.0
6.0
6.0
10.0
10.0
5.0
5.0
15.0
15.0
10.0
10.0
8.5
8.5
12.0
12.0
12.0
12.0
9.0
9.0
Page 8 of 14
Process Measurement – Classroom Exercise
Table 6: Revised Measurement Study Data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Spring 2004
Part No. Operator Response A Master A
1
O1
7.30
7.00
1
O1
6.70
7.00
2
O1
7.99
8.10
2
O1
8.23
8.10
3
O1
8.47
8.50
3
O1
8.61
8.50
4
O1
8.15
8.10
4
O1
8.04
8.10
5
O1
16.93
17.05
5
O1
17.22
17.05
6
O1
7.11
7.05
6
O1
7.09
7.05
7
O1
8.00
7.98
7
O1
7.95
7.98
8
O1
9.89
9.95
8
O1
9.98
9.95
9
O1
9.36
9.40
9
O1
9.45
9.40
10
O1
11.31
11.35
10
O1
11.33
11.35
1
O2
7.40
7.00
1
O2
7.11
7.00
2
O2
8.01
8.10
2
O2
8.16
8.10
3
O2
8.77
8.50
3
O2
8.53
8.50
4
O2
7.98
8.10
4
O2
8.08
8.10
5
O2
17.33
17.05
5
O2
17.19
17.05
6
O2
6.98
7.05
6
O2
7.07
7.05
7
O2
7.97
7.98
7
O2
7.99
7.98
8
O2
10.05
9.95
8
O2
9.93
9.95
9
O2
9.47
9.40
9
O2
9.44
9.40
10
O2
11.27
11.35
10
O2
11.43
11.35
Page 9 of 14
Process Measurement – Classroom Exercise
Table 7: Stability Data Taken After Revised Measurement
Study
Day
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Spring 2004
Stability Data
8.37
5.69
9.87
12.71
12.50
13.97
3.20
8.56
12.21
6.21
7.30
4.55
4.12
6.51
7.07
3.38
7.64
8.09
9.57
12.89
0.57
3.07
7.13
12.12
15.34
9.98
15.43
3.87
14.74
7.26
Page 10 of 14
Process Measurement – Classroom Exercise
Manual Assembly: Reversed Parts
Reversed parts in manual assembly is found to be a major source of defects.
There is considerable concern that this may be due to poor design and/or the
inability of operators to recognize such a situation, which could be the result of
inadequate training. A study was this conducted to determine whether operators in
manual assembly could accurately recognize whether a part was installed properly or
was improperly reversed.
Two operators inspected 10 installed parts in random order. Then the parts were
reordered and inspected again. Responses were recorded and in shown below in
Table 8.
Tasks
1. How well do operators identify reversed parts? 2
2
Table 8 has been inserted into the electronic copy of this file as a Microsoft Excel Worksheet Object. To
analyze the data, simply double click anywhere within the table, select the range of data to be analyzed, and
copy the data into the preferred (Minitab) application.
Spring 2004
Page 11 of 14
Process Measurement – Classroom Exercise
Table 8: Assessing Operator Ability to Detect Reversed
Parts
Part
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
Spring 2004
Order
9
2
1
5
6
3
8
4
7
10
5
1
3
6
2
7
9
10
4
8
2
4
9
5
3
10
7
8
1
7
3
4
10
1
5
7
9
8
6
2
Operator
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Jeff
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Sarah
Response
ok
reversed
reversed
ok
ok
ok
reversed
ok
ok
reversed
ok
ok
reversed
ok
ok
ok
ok
ok
reversed
reversed
ok
ok
reversed
ok
ok
reversed
reversed
reversed
ok
reversed
reversed
ok
ok
ok
ok
reversed
reversed
reversed
ok
reversed
Standard
ok
ok
reversed
ok
ok
reversed
reversed
ok
ok
ok
ok
ok
reversed
ok
ok
reversed
reversed
ok
ok
ok
ok
ok
reversed
ok
ok
reversed
reversed
ok
ok
ok
ok
ok
reversed
ok
ok
reversed
reversed
ok
ok
ok
Page 12 of 14
Process Measurement – Classroom Exercise
Solder: Solder Quality
Solder quality is determined by visual inspection. Hence, there is an opportunity
for error in terms of misclassifying whether a proper amount of solder has been
added to the solder joints. The Six Sigma team is considering among two types of
assessment schemes to evaluate the quality of visual inspection:
(1)
(2)
Nominal… use of a classification scheme, namely, missing solder,
insufficient solder, proper solder, excessive solder, and solder bridge
(short) and,
Ordinal… rating the quality of a solder application on the basis of a 1 – 5
scale (1 = poor application, 5 = proper application).
Nominal Scale
Three operators were randomly selected. Each operator inspected 15 printedcircuit boards (PCB’s) for solder quality that were presented in random order. The
response was either (a) missing solder, (b) insufficient solder, (c) proper solder, (d)
excessive solder, or (e) solder bridge (short). Responses were compared to a
standard to evaluate the consistency and accuracy of each operator/inspector. The
results of the experiment are shown in Table 1.
Task
1. How well do operators assess the quality of the solder on the PCB? 3
Ordinal Scale
The three operators’ responses in the above experiment were then redefined on a
1 – 5 scale, (1 – missing, 2 – insufficient, 3 – proper, 4 – excessive, and 5 – bridge).
Task
2. How well do operators assess the quality of the solder when using this scoring
rule / rating scheme vis-à-vis the nominal scale?
3
Table 9 has been inserted into the electronic copy of this file as a Microsoft Excel Worksheet Object. To
analyze the data, simply double click anywhere within the table, select the range of data to be analyzed, and
copy the data into the preferred (Minitab) application.
Spring 2004
Page 13 of 14
Process Measurement – Classroom Exercise
Table 9: Assessing Solder Quality on a Nominal Scale
PCB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Spring 2004
Order
9
13
4
2
8
12
14
3
15
10
11
5
1
7
6
6
9
13
8
15
10
12
2
7
14
4
5
3
11
1
11
10
9
2
7
4
15
5
14
12
3
13
8
1
6
Name
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Karen
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Jim
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Sue
Response (N)
Excessive
Proper
Insufficient
Insufficient
Missing
Missing
Missing
Insufficient
Proper
Excessive
Excessive
Excessive
Insufficient
Bridge
Missing
Bridge
Excessive
Proper
Insufficient
Missing
Missing
Missing
Insufficient
Proper
Excessive
Insufficient
Proper
Insufficient
Bridge
Missing
Bridge
Proper
Proper
Insufficient
Missing
Missing
Insufficient
Proper
Proper
Bridge
Proper
Excessive
Insufficient
Bridge
Missing
Response (S)
4
3
2
2
1
1
1
2
3
4
4
4
2
5
1
5
4
3
2
1
1
1
2
3
4
2
3
2
5
1
5
3
3
2
1
1
2
3
3
5
3
4
2
5
1
Standard (N)
Bridge
Excessive
Proper
Insufficient
Missing
Missing
Insufficient
Proper
Excessive
Bridge
Proper
Excessive
Insufficient
Bridge
Missing
Bridge
Excessive
Proper
Insufficient
Missing
Missing
Insufficient
Proper
Excessive
Bridge
Proper
Excessive
Insufficient
Bridge
Missing
Bridge
Excessive
Proper
Insufficient
Missing
Missing
Insufficient
Proper
Excessive
Bridge
Proper
Excessive
Insufficient
Bridge
Missing
Standard (S)
5
4
3
2
1
1
2
3
4
5
3
4
2
5
1
5
4
3
2
1
1
2
3
4
5
3
4
2
5
1
5
4
3
2
1
1
2
3
4
5
3
4
2
5
1
Page 14 of 14