IE 361 Module 23 Design and Analysis of Experiments: Part 4 (Fractional Factorial Studies and Analyses With 2-Level Factors) Reading: Section 7.1, Statistical Quality Assurance Methods for Engineers Prof. Steve Vardeman and Prof. Max Morris Iowa State University Vardeman and Morris (Iowa State University) IE 361 Module 23 1 / 47 Large p Complete Factorials are Impossible in Practice In this module we discuss what can be done in a many-factor context where the number of possible combinations of levels of p factors is so large that doing a complete factorial experiment is not practically possible. For p factors, even 2p gets big fast, for example. for p = 10, 2p = 1024 So in practice, for p of any size, one must make do with information from only some fraction of a complete factorial in p factors. We consider rational approaches to design and analysis of fractional factorial studies, and here limit our discussion to fractions of 2p studies. We will …rst consider half fractions of these, and then other fractions that are powers of 1/2. Vardeman and Morris (Iowa State University) IE 361 Module 23 2 / 47 Large p Complete Factorials are Impossible in Practice Example 23-1 (From an article by Hendrix in 1979 Chemtech) A 15-factor chemical experiment had factors and levels as Factor Levels A-Coating Roll Temp 115 B-Solvent Factor vs 125 Levels J-Feed Air to Dryer Preheat Yes vs No Recycled vs Re…ned K-Dibutylfutile in Formula 12% vs 15% C-Polymer X-12 Preheat No vs Yes L-Surfactant in Formula .5% vs 1% D-Web Type LX-14 vs LB-17 M-Dispersant in Formula .1% vs .2% E-Coating Roll Tension 30 vs 40 N-Wetting Agent in Formula 1.5% vs 2.5% F-Number of Chill Roll 1 vs 2 O-Time Lapse 10min vs 30min G-Drying Roll Temp 75 P-Mixer Agitation Speed 100rpm vs 250rpm H-Humidity of Air Feed 75% vs 90% vs 80 and response variable y = a measure of product cold crack resistance A full factorial would require at least 215 = 32, 768 runs of the process! An obvious "solution" is to collect data for only some (a fraction) of all possible combinations of levels of the factors. Vardeman and Morris (Iowa State University) IE 361 Module 23 3 / 47 "Obvious" Limitations and Objective Qualitative points that ought to be “obvious” a priori if a fraction of a factorial is used as an experimental design are that there must be some information loss (relative to the full factorial), some ambiguity must inevitably follow because of the loss, and careful planning and wise analysis are needed to hold these to a minimum. Vardeman and Morris (Iowa State University) IE 361 Module 23 4 / 47 "Obvious" Limitations and Objective Example 23-2 (Hypothetical/unrealistic but instructive half fraction of a 2X2 Factorial) The full 22 factorial structure is shown in below. Here, if combination (1) is used, combination ab must be employed or else one learns nothing about the action of one of the factors. (If a is used, then b must be employed or one learns nothing about the action of one of the factors.) On the other hand, if there is a big di¤erence in response between the two combinations included in the half fraction, one doesn’t know whether to attribute this to one or the other or to the interaction e¤ect of the factors A and B. Figure: A 22 Full Factorial Layout Vardeman and Morris (Iowa State University) IE 361 Module 23 5 / 47 Potential and Problems Example 23-3 (Hypothetical) Suppose that 23 factorial e¤ects are µ... = 10, α2 = 3, β2 = 1, γ2 = 2, αβ22 = 2, αγ22 = 0, βγ22 = 0, αβγ222 = 0 The corresponding means are then as pictured below. Figure: A 23 Factorial With a "Good" Half Fraction Indicated With Circled Corners Vardeman and Morris (Iowa State University) IE 361 Module 23 6 / 47 Potential and Problems Example 23-3 continued Suppose further that one gets data adequate to essentially reveal the mean responses for combinations a, b, c and abc (the 4 corners circled on panel 6) but has no data on the other combinations. How then might one try to assess the 23 factorial e¤ects if presented with information from the circled corners on panel 6? Remember that with the 23 as pictured on panel 6, α2 = "right face average" "grand average" = 3 A "half-fraction version" of this might be α2 = "available right face average" = 13 10 = 3 "available grand average" Here α2 = α2 !!! Is this something for nothing? Can we learn about the A main e¤ect using data from only "4 corners"? Vardeman and Morris (Iowa State University) IE 361 Module 23 7 / 47 Potential and Problems Example 23-3 continued But note that a similar calculation for the C main e¤ect gives γ2 = "available back face average" = 14 10 = 4 and 4 = γ*2 6= γ2 = 2 "available grand average" ??? The general story here is that for this 23 1 fractional factorial α2 = α2 + βγ22 and γ2 = γ2 + αβ22 We were able to "recover" α2 using only α2 (based on only half of the corners of the cube) because βγ22 = 0. We were unable to "recover" γ2 using only γ2 (based on only half of the corners of the cube) because αβ22 = 2 6= 0. We can really only know α2 + βγ22 (and not α2 alone) and γ2 + αβ22 (and not γ2 alone) based on the half fraction. This is an example of confounding/aliasing/ambiguity that of necessity comes with use of only a fractional factorial data collection plan. Vardeman and Morris (Iowa State University) IE 361 Module 23 8 / 47 An Aside Example 23-3 continued As a bit of an aside at this point, notice that the choice of 4 corners on panel 6 has admirable symmetry. If one collapses the cube in any direction one is left with a complete factorial arrangement in the two other factors. That means that if, in fact, one of the 3 factors is "inert" doing nothing to a¤ect response, one has a full factorial in the 2 factors that matter. Vardeman and Morris (Iowa State University) IE 361 Module 23 9 / 47 Issues to Address The issues to be addressed in order to use 2p how to rationally choose 1 2q q fractional factorials are: out of 2p combinations for study, how to determine the corresponding aliasing/confounding pattern, and how to do data analysis. We proceed to consider these questions …rst in the context of half fractions (the q = 1 case), then for general 1/2q fractions. Vardeman and Morris (Iowa State University) IE 361 Module 23 10 / 47 Choice of a Half Fraction To choose what we will call a "standard" (a "good") half fraction of a 2p factorial, we will write out signs for specifying levels for all possible combinations of levels of the “…rst” p 1 factors, and then “multiply” these together for a given combination of the “…rst” factors to arrive at a corresponding level to use for the “last” factor. Vardeman and Morris (Iowa State University) IE 361 Module 23 11 / 47 Choice of a Half Fraction Example 23-4 (A Hypothetical Half Fraction of a 4-Factor Study) With 4 two-level factors A, B, C and D, one proceeds as in the following table. (One multiplies and + signs as if they were 1’s and the last column gives the indicated combination of factor levels for the row, using the special 2p naming convention introduced in Module 22.) A B C Product (used for D) + + + + + + + + + + + + + + Vardeman and Morris (Iowa State University) + + Combination (1) ad bd ab cd ac bc abcd IE 361 Module 23 12 / 47 Choice of a Half Fraction Example 23-5 R. Snee in a 1985 ASQC Technical Supplement discussed a 25 1 chemical process study. The factors and their levels were as in the following table. Factor A-Solvent/Reactant B-Catalyst/Reactant C-Temperature D-Reactant Purity E-pH of Reactant ( ) low .025 150 92% 8.0 vs vs vs vs vs (+) high .035 160 96% 8.7 In Snee’s study, the response variable was y = color index Vardeman and Morris (Iowa State University) IE 361 Module 23 13 / 47 Choice of a Half Fraction Example 23-5 Snee’s (unreplicated) data were as below. Combination e a b abe c ace bce abc y .63 2.51 2.68 1.66 2.06 1.22 2.09 1.93 Combination d ade bde abd cde acd bcd abcde y 6.79 6.47 3.45 5.68 5.22 9.38 4.30 4.05 These are data from half of all 32 combinations of 2 levels of each of the 5 factors (half of all possible labels of combinations based on the 5 letters a,b,c,d and e are given above, namely those involving an odd number of letters). In fact, Snee followed the standard recommendation for choosing this half fraction. Vardeman and Morris (Iowa State University) IE 361 Module 23 14 / 47 Aliasing Structure of a Half Fraction To understand the pattern of ambiguities one is left with upon using a standard half fraction of a 2p factorial, we will use a method of formal multiplication, beginning from a so-called “generator” that represents the way in which the half fraction was chosen. The generator is of the form name of “last” factor $ product of names of “…rst” factors The rules of formal multiplication are that any letter I$the same letter letter same letter$I (We will use the words "aliasing" and "confounding" interchangeably to refer to the ambiguities left by a fractional factorial study.) Vardeman and Morris (Iowa State University) IE 361 Module 23 15 / 47 Aliasing Structure of a Half Fraction Example 23-3 continued In the hypothetical 23 1 example, the generator is C $ AB We can multiply through by C to obtain the so called “de…ning relation” I $ ABC This …rst says that the ABC 3-factor interaction αβγ222 is aliased with the grand mean. That is, only µ ... + αβγ222 can be evaluated based on the half-fraction, not αβγ222 alone. Multiplying through the de…ning relation by any set of letters of interest produces a statement of what e¤ect(s) are “aliased with” the corresponding e¤ect. Vardeman and Morris (Iowa State University) IE 361 Module 23 16 / 47 Aliasing Structure of a Half Fraction Example 23-3 continued For example A $ BC (read “the A main e¤ect is aliased with the BC 2-factor interaction). Similarly C $ AB as was illustrated earlier. In fact, the whole alias structure is I $ ABC and A $ BC and B $ AC and C $ AB and we see that 23 factorial e¤ects are aliased in 4 pairs. The technical meaning of aliasing is that only sums of e¤ects can be learned from the half fraction study, not individual e¤ects. (This makes sense. With only 4 conditions studied, one should be able to resolve only 4 di¤erent quantities!) Vardeman and Morris (Iowa State University) IE 361 Module 23 17 / 47 Aliasing Structure of a Half Fraction Example 23-4 continued The hypothetical 24 1 study had generator D $ ABC So the de…ning relation is I $ ABCD From this we see, e.g., that the AB 2-factor interaction is aliased with the CD 2-factor interaction. In fact, the student is encouraged to write out the entire alias structure and see that the 24 factorial e¤ects are aliased in 8 pairs. Vardeman and Morris (Iowa State University) IE 361 Module 23 18 / 47 Aliasing Structure of a Half Fraction Example 23-5 continued Snee’s 25 1 study had generator E $ ABCD and hence de…ning relation I $ ABCDE From this one sees, e.g., that the AB 2-factor interaction is aliased with the CDE 3-factor interaction. (There are 16 pairs of aliased/ indistinguishable 25 factorial e¤ects. One can hope to learn only about the sum of a pair, not the individual e¤ects making up a pair.) Vardeman and Morris (Iowa State University) IE 361 Module 23 19 / 47 Data Analysis for a Half Fraction To do data analysis for a half fraction of a 2p study, one may initially temporarily ignore the “last” factor, treat the data as a full factorial in the “…rst” p 1 factors, and judge the statistical signi…cance and practical importance of estimates derived from the Yates algorithm, and then interpret these estimates in light of the alias structure (as estimates of appropriate sums of 2p e¤ects). For judging statistical signi…cance, where there is some replication (not all 2p 1 sample sizes are 1) con…dence intervals can be made for the (sums of) e¤ects. Lacking any replication, normal plotting of the output of the Yates algorithm (ignoring the “last” factor) is the only available method. Vardeman and Morris (Iowa State University) IE 361 Module 23 20 / 47 Data Analysis for a Half Fraction To be completely explicit about the making of con…dence intervals based on the output of the Yates, algorithm, we use r 1 1 b tspooled E 2p 1 ∑ ncomb where 2 spooled = 2 ∑ (ncombination 1) scombination ∑ (ncombination 1) and the appropriate degrees of freedom for t are ∑ (ncombination Vardeman and Morris (Iowa State University) 1) = n IE 361 Module 23 2p 1 21 / 47 Data Analysis for a Half Fraction Example 23-6 (Another Hypothetical Half Fraction of a 2X2X2 Study) Suppose na = 1, ya = 5, nb = 2, ȳb = 3, sb2 = 1.5, nc = 1, yc = 2.5, and 2 nabc = 3, ȳabc = 5.5, sabc = 1.8 Then listing the 4 combinations in Yates standard order as regards the "…rst" 2 factors A and B (i.e. ignoring the "last" factor, C), the (p 1 = 2 cycle) Yates algorithm is applied to the following table. Combination c a b abc ȳ 2.5 5.0 3.0 3.0 Vardeman and Morris (Iowa State University) IE 361 Module 23 22 / 47 Data Analysis for a Half Fraction Example 23-6 continued Con…dence intervals based on the output of the algorithm would be made using 0 + (2 1)1.5 + 0 + (3 1)1.8 2 spooled = 0 + (2 1) + 0 + (3 1) These have the form b E t3 spooled Vardeman and Morris (Iowa State University) 1 23 1 r 1 1 1 1 + + + 1 2 1 3 IE 361 Module 23 23 / 47 Data Analysis for a Half Fraction Example 23-5 continued Snee’s 25 1 study had no replication. Ignoring factor E temporarily, the (4-cycle) Yates algorithm can be applied to the 16 responses exactly as listed earlier (they are in Yates order as regards the …rst 4 factors). The result is the set of estimates below. Figure: Estimates for Snee’s Chemical Data Vardeman and Morris (Iowa State University) IE 361 Module 23 24 / 47 Data Analysis for a Half Fraction Example 23-5 continued A normal plot of the (last 15) Snee estimates is below and suggests that at most 4 sums of e¤ects are distinguishable from background variation. Figure: A Normal Plot of 15 Fitted Sums of E¤ects From Snee’s 25 1 Study Vardeman and Morris (Iowa State University) IE 361 Module 23 25 / 47 Data Analysis for a Half Fraction Example 23-5 continued Tentative engineering conclusions based on Snee’s study were that for uniform color index, attention must be paid to controlling/reducing variation in the following (in decreasing order of importance): Factor D, Reactant Purity Factor B, Catalyst/Reactant Ratio Factor E, pH of Reactant Factor A, Solvent/Reactant Ratio Vardeman and Morris (Iowa State University) IE 361 Module 23 26 / 47 Issues to Address The issues to be addressed in order to use 2p how to rationally choose 1 2q q fractional factorials remain: out of 2p combinations for study, how to determine the corresponding aliasing/confounding pattern, and how to do data analysis. The answers for smaller-than-half fractions are the natural generalizations of the half fraction answers just discussed. Vardeman and Morris (Iowa State University) IE 361 Module 23 27 / 47 Choice of a Fractional Factorial To choose a 1/2q fraction of a 2p factorial, we will write out signs for specifying levels for all possible combinations of levels of the “…rst” p q factors, and pick q di¤erent groups of the …rst p q factors and use the products of the signs corresponding to members of the groups to specify levels for the “last” q factors. Vardeman and Morris (Iowa State University) IE 361 Module 23 28 / 47 Choice of a Fractional Factorial Example 23-7 Hanson and Best in a presentation at the 1986 annual meeting of the American Statistical Association reported on an experiment for the development of a catalyst for producing ethyleneamines by the amination of monoethanolamine involving p = 5 factors. These factors and their levels were Factor A-Ni/Re Ratio B-Precipitant C-Calcining Temp D-Reduction Temp E-Support Used ( ) 2/1 (NH4 )2 CO 300 300 alpha-alumina vs vs vs vs vs (+) 20/1 none 500 500 silica alumina The response of interest was y = % water produced Vardeman and Morris (Iowa State University) IE 361 Module 23 29 / 47 Choice of a Fractional Factorial Example 23-7 continued The investigators decided against a full factorial, choosing instead to use a 25 2 design. That is, they chose to use q = 2 and a 14 fraction of the full 25 study. They ran 25 2 = 8 out of the 25 = 32 possible A, B, C, D, E combinations. The (somewhat arbitrary) choice was made to use ABC sign products to choose levels of D, and BC sign products to choose levels of E. (Other choices are possible and lead to di¤erent aliasing patterns that might for some other studies be preferred by the engineer in charge.) The choice used is summarized in the table on panel 31, whose last column speci…es the 8 combinations of levels of the 5 factors used in the study in the special 2p factorial notation. Vardeman and Morris (Iowa State University) IE 361 Module 23 30 / 47 Choice of a Fractional Factorial Example 23-7 continued A B C ABC Product (for D) + + + + + + + + + + + + + + Vardeman and Morris (Iowa State University) BC Product (for E) + + + + + + IE 361 Module 23 Combination e ade bd ab cd ac bce abcde 31 / 47 Choice of a Fractional Factorial Example 23-7 continued The data reported by Hanson and Best were as listed and summarized below. Combination e ade bd ab cd ac bce abcde y 8.70, 11.60, 9.00 26.80 24.88 33.15 28.90, 30.98 30.20 8.00, 8.69 29.30 Vardeman and Morris (Iowa State University) y 9.767 26.800 24.880 33.150 29.940 30.200 8.345 29.300 IE 361 Module 23 s2 2.543 2.163 .238 32 / 47 Aliasing Structure of a Smaller-Than-Half Fraction To understand the pattern of ambiguities one faces with a particular choice of a 21q fraction of a 2p factorial, we will use the formal multiplication, beginning from the q generators that represent the way in which the 21q fraction was chosen. To …nd the de…ning relation (the list of all products "equivalent to" I) we …rst convert the generators to statements of products equivalent to I, and then multiply these in pairs, then in triples, then in sets of four, etc. The letter I will ultimately have 2q 1 equivalent products, i.e. the 2p factorial e¤ects are aliased in 2p q di¤erent groups of 2q each. Vardeman and Morris (Iowa State University) IE 361 Module 23 33 / 47 Aliasing Structure of a Smaller-Than-Half Fraction Example 23-7 continued In the catalyst example, the generators were D $ ABC and E $ BC so I $ ABCD and I $ BCE Further, multiplying these two we get I I $ (ABCD) (BCE) i.e. I $ ADE So the whole de…ning relation for the catalyst study is I $ ABCD $ BCE $ ADE and therefore e¤ects are aliased in 8 groups of 4. For example, multiplying through the de…ning relation by A gives A $ BCD $ ABCE $ DE and we see that the A main e¤ect is aliased with the DE 2-factor interaction (among other things). Vardeman and Morris (Iowa State University) IE 361 Module 23 34 / 47 Data Analysis for a Smaller-Than-Half Fraction To do data analysis for a 2p q study, one may initially temporarily ignore the “last” q factors, treat the data as a full factorial in the “…rst” p q factors, and judge the statistical signi…cance and practical importance of estimates derived from the Yates algorithm, and then interpret these estimates in light of the alias structure (as estimates of appropriate sums of 2p e¤ects). For judging statistical signi…cance, where there is some replication (not all 2p q sample sizes are 1) con…dence intervals can be made for the (sums of) e¤ects. Lacking any replication, normal plotting of the output of the Yates algorithm (ignoring the “last” factor) is the only available method. Vardeman and Morris (Iowa State University) IE 361 Module 23 35 / 47 Data Analysis for a Smaller-Than-Half Fraction To be explicit, the form of con…dence intervals for the (sums of) e¤ects is r 1 1 b tspooled E 2p q ∑ ncomb where 2 spooled = 2 ∑ (ncombination 1) scombination ∑ (ncombination 1) and the appropriate degrees of freedom for t are ∑ (ncombination Vardeman and Morris (Iowa State University) 1) = n IE 361 Module 23 2p q 36 / 47 Data Analysis for a Smaller-Than-Half Fraction Example 23-7 continued In the catalyst example the 8 sample means, ȳ , listed before were in Yates standard order for factors A, B and C (the “…rst” p q = 3) ignoring D and E (the “last” q = 2). So the (p q = 3 cycle) Yates algorithm can be applied to them in the order listed. The following table shows the …rst two and last columns of the Yates table and then records what the estimates produced by the algorithm attempt to approximate. Combination e ade bd ab cd ac bce abcde y 9.767 26.800 24.880 33.150 29.940 30.200 8.345 29.300 Vardeman and Morris (Iowa State University) Estimate 24.048 5.815 .129 1.492 .399 .511 5.495 3.682 Sum Estimated grand mean+aliases A main e¤ect+aliases B main e¤ect+aliases AB interaction+aliases C main e¤ect+aliases AC interaction+aliases BC interaction+aliases ABC interaction+aliases IE 361 Module 23 37 / 47 Data Analysis for a Smaller-Than-Half Fraction Example 23-7 continued Since the original data had 3 sample sizes larger than 1, statistical signi…cance/detectability of these can be judged using con…dence limits for sums of e¤ects. First, 2 spooled = (3 1)(2.543) + (2 1)(2.163) + (2 1)(.238) = 1.872 (3 1) + (2 1) + (2 1) p So spooled = 1.872 = 1.368, and this can be used as a measure of background noise and as a basic ingredient of con…dence intervals for the sums of e¤ects. spooled has 4 associated degrees of freedom. So if, e.g., 95% con…dence intervals for the sums of e¤ects are desired, the “+/ part” of the con…dence interval formula becomes r 1 1 1 1 1 1 1 1 1 2.776(1.368) 3 + + + + + + + i.e. 1.195 2 3 1 1 1 2 1 2 1 Vardeman and Morris (Iowa State University) IE 361 Module 23 38 / 47 Data Analysis for a Smaller-Than-Half Fraction Example 23-7 continued So a margin of error to associate with any one of the values produced by the Yates algorithm is 1.195. We might therefore judge any estimate larger in absolute value than 1.195 to represent a sum of e¤ects clearly large enough to see above the background experimental variation. Then the “detectable” sums are (in decreasing order of magnitude): Sum Estimate α2 + βγδ222 + αβγe2222 + δe22 5.815 βγ22 + αδ22 + e2 + αβγδe22222 5.495 αβγ222 + δ2 + αe22 + βγδe2222 3.682 αβ22 + γδ22 + αγe222 + βδe222 1.492 Happily, the last of these is smaller in magnitude than the other 3, but there are at least 4 a priori equally plausible interpretations of the possibility that each one of these 3 are really driven by a single e¤ect. Vardeman and Morris (Iowa State University) IE 361 Module 23 39 / 47 Data Analysis for a Smaller-Than-Half Fraction Example 23-7 continued From this data analysis alone, it is equally plausible that there are important A main e¤ects, E main e¤ects, and D Main e¤ects, A main e¤ects, E main e¤ects, and AE 2-factor interactions, A main e¤ects, AD 2-factor interactions, and D main e¤ects, or DE 2-factor interactions, E main e¤ects, D main e¤ects. In fact, a follow-up study con…rmed the importance of the D main e¤ect (and made the …rst of these most attractive). If the A (Ni/Re ratio) main e¤ect, the E (Support Type) main e¤ect and the D (Reduction Temp) main e¤ect are indeed the most important determiners of y , and large y is desirable, the signs of the estimates indicate the need for “high A” (20/1 Ni/Re ratio), “low E” (alpha-alumina support) and “high D” (500 reduction temp). Vardeman and Morris (Iowa State University) IE 361 Module 23 40 / 47 Perspective By now it should be obvious that the larger is q, the larger the inevitable ambiguity of interpretation of the fractional factorial results and the more likely the need for follow-up study. Small fractions are really most useful as screening studies, to pick a few likely candidates out of many potentially important factors for subsequent more detailed study. We end with an extreme example involving a large q, i.e. a small fraction. Vardeman and Morris (Iowa State University) IE 361 Module 23 41 / 47 Example 23-1 continued The Hendrix chemical process study involved p = 15 factors A, B, C, D, E, F, G, H, J, K, L, M, N, O, P (the factor names and levels were given earlier). Here p q = 4, i.e., only 24 = 16 combinations were run !!!!! This was a 1 1 = 2048 fraction !!!! 2 15 4 The 11 generators used were: E $ ABCD,F $ BCD,G $ ACD,H $ ABC,J $ ABD, K $ CD,L $ BD,M $ AD,N $ BC,O $ AC,P $ AB These led to the 16 combinations and (ultimately) the data on panel 43. Vardeman and Morris (Iowa State University) IE 361 Module 23 42 / 47 Example 23-1 continued Figure: Hendrix 215 11 Data Vardeman and Morris (Iowa State University) IE 361 Module 23 43 / 47 Example 23-1 continued Pretty clearly it isn’t sensible to write out the whole de…ning relation here. E¤ects are going to be aliased in 16 groups of 211 = 2048 e¤ects. But for a most tentative interpretation, let’s see what we might glean if the physical system is so simple that only main e¤ects dominate. (Whether this is physically reasonable is a question that needs to be answered by a process expert.) The 16 observations are listed in Yates order for factors A,B,C and D (ignoring the rest). We therefore begin by running them through the Yates algorithm, with the results on panel 45. Vardeman and Morris (Iowa State University) IE 361 Module 23 44 / 47 Example 23-1 continued Figure: Result of the Yates Algorithm for the Hendrix Data Vardeman and Morris (Iowa State University) IE 361 Module 23 45 / 47 Example 23-1 continued There is no replication in this data set, so we’re driven to normal plotting in order to judge statistical signi…cance of these estimates. A normal plot of the (last 15) estimates is below. Figure: Normal Plot of 15 Estimated Sums of E¤ects from the 215 11 Chemical Process Study Vardeman and Morris (Iowa State University) IE 361 Module 23 46 / 47 Example 23-1 continued The plot shows that there are two …tted sums of e¤ects that are clearly statistically detectable. As each of these stands for a sum of 2048 e¤ects, no solid …nal conclusions can be made. But if one assumes that the "big" sums are primarily driven by the main e¤ects appearing in them, a plausible tentative interpretation is that the most important factors appear to be B (Solvent) and F (# of Chill Rolls) and for large cold crack resistance "high B" (re…ned solvent) and "high F" (2 chill rolls) appear best. Note that the analysis does point out what is in retrospect quite obvious, namely that it is those combinations in the data set with “high B” and “high F” that have the largest y ’s. Vardeman and Morris (Iowa State University) IE 361 Module 23 47 / 47