Strategies for Sequential Design of Experiments
by
Rizwan R. Koita
Submitted to the Department of Electrical Engineering and Computer Science
in partial fulfillment of the requirements for the degree of
Master of Science in Electrical Engineering and Computer Science
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
September 1994
®Massachusetts Institute of Technology 1994. All Rights Reserved.
This thesis may be reproduced and distributed for LFM partners' internal use only.
No other person or organization may reproduce this thesis without permission of the
copyright owner. Requests for permission can be addressed to the LFM Manager,
Communications.
A
u
th
o
r
...................................................
Author............
.....-.
Department of Electrical Engineering ad Computer Science
August 2, 1994
7....
. ... .
..........
by
...........
Certified
David . t elin
Professor of Electrical ngineering
Thesis Supervisor
q2
r1 .
Accepted
by........ ...........
A.
.
A.._
Chairma.n_ rena.rtmental COn mittee nn
------- ,7'1~~~-~~1
Fearker
ng
--
VVY----UYUV-
.............
F. R. Morgenthaler
radlliteP
Sq.tllPnt
YUI~U
Strategies for Sequential Design of Experiments
by
Rizwan R. Koita
Submitted to the Department of Electrical Engineering and Computer Science
on August 2, 1994, in partial fulfillment of the
requirements for the degree of
Master of Science in Electrical Engineering and Computer Science
Abstract
The growth of competitiveness in the manufacturing industry has resulted in an acute need
to improve product quality. Efficient methods for Design of Experiments (DOE) have thus
become more important than ever before. With increases in complexity of manufacturing processes and the cost of experimentation, it has become important to use all prior
knowledge of the processes, derived first from theory and experience and later from exper-
imentation, to determine subsequent experiments.
The objective of this work is to develop sequential DOE methods and software tools
which can be used by manufacturers with no prior knowledge in design of experiments.
Techniques for designing blocks of fractional factorial experiments have been developed
and implemented in computer software. In particular, given a block of factorial experiments
and a list of probable interactions, different sets of experiments are designed to augment
the original block of experiments and form new factorial designs with different confounding
patterns. The set of experiments which produces a new design with minimal confounding
between the variables and the probable interactions is the optimal set of experiments. Software has been implemented to generate the optimal Full-Block or Half-Block of experiments.
The fold-over design technique, which is popularly used in DOE, is a special case of the
Full-Block design strategy. The analysis routines have been extended to analyze multiple
blocks of experiments with different confounding patterns and to determine the effects of
the confoundinginteractions.
A One-at-a-Time design strategy has been proposed which uses the results of the previous experiments to design the optimal experiments one at a time. This strategy assumes
that there is only one significant variable or interaction on each significant column of the
design matrix. Therefore, this strategy is particularly useful when the number of significant
effects is sparse. A hypothesis is made that certain interactions are significant on the basis
of their variables. The optimal experiment for this hypothesis, i.e., the experiment which is
expected to yield the maximum output quality, is conducted. The result of this experiment
is compared with the predicted output quality of all possible plant models. The model
which gives the least prediction error is selected as the next hypothesis and the procedure
is repeated. The simulation results for this design methodology have been very promising.
Thesis Supervisor: David H. Staelin
Title: Professor of Electrical Engineering
Acknowledgments
First and foremost, I would like to express my sincere thanks to my advisor Professor
David Staelin for his excellent guidance, constant support and understanding.
He taught
me many things about research and life. Working with him was a privilege.
Thanks to my fellow group members: Ambrose, Carlos, Mark, Michelle and Mike for
their help and cooperation. I appreciate the discussionsI had with them and enjoyed the
group meetings.
I acknowledge the efforts of the EECS Department for making it possible for me to
rejoin MIT, particularly the staff of the EECS Graduate Office who have been so patient
and supportive through my difficult periods.
Special thanks to Asif Shakeel, Sankar Sunder, S. R. Venkatesh and T. A. Venkatesh.
Spending time with you all has probably been the most memorable part of my stay at MIT.
Thanks to Amit Kumar who has always been a source of immense help and encouragement
and to Firdaus Bhathena for helping me on numerous occasions.
I am very grateful to God for helping to me complete my masters degree at MIT and
making life such an interesting challenge.
Finally, I am deeply indebted to my grandmother, my parents, my sister Irfana, Zarina
Aunty and Rekha whose love and sacrifices have made a dream come true.
The work in this thesis was carried out at the Research Laboratory of Electronics and was
supported by the Leaders for Manufacturing program at MIT.
Contents
1
15
Introduction
1.1 Reseach Motivation .
...........
...
1.2 Background ..........................
1.3
1.2.1
Search Designs.
1.2.2
Response Surface Methods
Thesis Overview
.............
.......................
1.4 Notation ............................
. . . ..... . .
15
. . . ..... . .
17
. . . ..... . .
18
. . . ..... . .
18
. . . ..... . .
20
...... . 19
1.4.1
Factorial Fractional Designs .............
. . . ..... . .
21
1.4.2
Resolution of Factorial Designs ...........
. . . ..... . .
23
1.4.3
Basic Columns and Variables of Factorial Designs.
. . . ..... . .
23
1.4.4
Shifting with respect to a Columns .........
. . . ..... . .
25
1.4.5
Fold-Over Designs ..............
. . . ..... . .
25
2 Sequential Experimental Design
27
2.1 Non-Sequential Approach to DOE . . .
..
.
27
2.2
Sequential Design Philosophy ......
..
.
29
2.2.1
Sequential Block Design .....
.. .
30
2.2.2
One-at-a-Time Design ......
.. .
31
. ..
31
2.3
Assumptions on Manufacturing Models.
3 Sequential Block Design
33
3.1
Need for Sequential Block Design ...................
3.2
Sequential Design Methodology ...
3.3
. . . . .
33
..... ..34
Half-Block Designs ...........................
..... ..36
3.3.1 Algorithm to Determine the Optimal Half-BlockDesign . . ..... ..37
.................
7
3.3.2
Advantages and Disadvantages of Half-Block Designs ..
3.4
Full-Block
Designs
................ ... .
3.4.1
3.5
. . . ..... . .
.......
. ..
43
Algorithm to Determine the Optimal Full-Block Design . . . ..... . .
Completing Block Designs .....................
. . . ..... . .
4 Analysis of Sequential Experiments
4.1
4.2
41
43
46
49
Classical Analysis Techniques .....................
49
4.1.1
Analysis of Variance (Anova) ...............
49
4.1.2
Normal Probability Plots.
50
4.1.3
Lenth's Algorithm for Determining Significant Coefficients .
51
Closed Loop Technique for Determining Interactions
........
52
5 One-at-a-Time Designs
55
5.1
Need for One-at-a-Time Design Strategy ...............
... .
55
5.2
One-at-a-Time Design Strategy ....................
...
...
...
...
56
62
68
70
5.3 Mathematical Justification .
5.4
Advantages of One-at-a-Time Design Strategy ............
5.5 Partial Optimization ..........................
.
.
.
.
75
6 Conclusions and Future Work
6.1
The Complete DOE Software Tool .......................
75
6.2
Thesis Achievements.
77
78
6.3 Future Work.
81
A Binary Orthogonal Matrices
A.1 Definitions .
81
A.2 Properties of Binary Orthogonal Matrices ...................
82
B Designing a Half-Block of Experiments
85
C Designing a Full-Block of Experiments
91
C.1 Full-Block Design Strategy
...........................
91
C.2 Fold-Over Designs ................................
96
D Completing Fractional Factorial Blocks
99
8
E Computer Software Listings
105
Bibliography
159
9
10
List of Figures
1-1
Proposed Experimental Design Strategy ........
. . . ..... . .
16
2-1
Non-Sequential Approach to Experimental Design . .
. . . ..... . .
27
3-1
Pictorial Representation of Completing Block Design .
. . . ..... . .
47
4-1
Normal Probability Plot .................
. . . ..... . .
51
5-1
One-at-a-Time Design Strategy .............
5-2
Permissible range of values of ij ............
6-1
Overview of Complete DOE Software Tool
11
57
. . . . . . . . :. . :. .
73
......
76
12
List of Tables
1.1
25-2 Factorial Design Matrix.
22
1.2 Basic and Non-Basic Columns of a Binary Orthogonal Matrix .
22
1.3
Changes in Confounding Patterns .................
24
3.1
Fold-over Design Matrix for Example 3-1
35
3.2
Results of Fold-over Design of Example 3-2 ..................
39
3.3
Analysis of Design of Example
39
3.4
Half-Block Experiments and Results of Example 3-2 .............
40
3.5
Analysis of New Design of Example 3-2 ....................
40
3.6
Results of Fold-over Design of Example 3-3 ................
44
3.7
Analysis of Design of Example 3-3 .....
44
3.8
Full-Block Experiments for Example 3-3 ....................
45
3.9
Analysis of New Design of Example 3-3 ....................
45
5.1
Confounding pattern of the 27-3 factorial experiments of Examples 5-1, 5-2
and 5-3
3-2
...................
. . . . . . . . . . . . . . . . . . . . . . .
..................
..............................................
59
5.2
Results of One-at-a-Time Designs of Example 5-1. ..............
60
5.2
Results of One-at-a-Time Designs of Example 5-1 ...............
61
5.3
Results of One-at-a-Time Designs of Example 5-2. ............
63
5.3
Results of One-at-a-Time Designs of Example 5-2. .............
64
5.4
Results of One-at-a-Time Designs of Example 5-3. ..............
65
5.4
Results of One-at-a-Time Designs of Example 5-3. .
66
5.5
Results of Design of Example 5-4 .......
A.1 3 x 3 Binary Orthogonal Matrix
.......
13
...........
...
.................
70
82
B.1 Equivalence of Different Designs
.......................
89
B.2 Changes in Confounding Pattern
........................
89
C.1 Columns of the variables in the original and the new design .........
92
Table for Example C-1 ..............................
95
C.3 Initial Design of Example C-2 ..........................
97
C.4 Fold-over and Full-Block Design of Example C-2 ...............
98
C.2
D.1 Incomplete Block of Experiments ........................
102
D.2 Missing Block of Experiments ..........................
103
14
Chapter
1
Introduction
In this chapter we outline the motivation driving the research in this area. A brief overview
of the existing methodologies is given in Section 2 along with the outline of the thesis in
Section 3. Section 4 gives the notation used in the report.
1.1
Reseach Motivation
With the increase in competitiveness in the manufacturing industry the methods of Experimental Design are becoming increasingly important.
The need for efficient Design of
Experiments (DOE) stems from the desire to improve quality and productivity quickly and
economically. The concepts in Design of Experiments have been extensively researched
since Fisher [11] introduced the basic theory in 1926.
Although the basic ideas in the DOE are relatively simple, their application involves use
of comparatively advanced concepts in algebra and statistics. Hence, despite the extensive
literature, DOE has remained largely outside the purview of manufacturing and is used very
infrequently, if at all, in today's industry.
With the advent of high speed digital computers it is now possible to design and search
for optimal experimental designs and to use statistics without the need of the experimenter
to understand all the concepts. The goal of this thesis is to develop an experimental design
package for a user who is familiar with the plant/process to be optimized, but has no prior
experience with experimental design.
The basic proposed strategy for the DOE method developed here is shown in Figure 1-1.
An experimenter inexperienced with DOE can be expected to only perform the following
15
Block 1:
Select Variables
and guess InteractionsExeintMrx
~~Block
2: Design
Block 3: Conduct
Experiments
.
C
Block 4: Analyze
Results
I
Block 6: Select
Probable Interactions
bad
,
Block 5: Evaluate
Results
good
Block 7: Suggest
Optimum Setting
Figure 1-1: Proposed Experimental Design Strategy
tasks:
Block 1: Decide which parameters/ variables affect the process, and the approximate range
in which they can be varied for the normal operation of the plant.
Also, based
on the knowledge of the process, the experimenter can guess some of the probable
interactions.
Block 3: Perform the set of experiments as determined by Block 2 of Figure 1-1.
Traditionally, an experimenter was expected to perform the tasks of all the blocks.
But as discussed above, it is possible to develop software so that the experimenter can
successfully use DOE without being required to understand all the concepts used in the
implementation of Blocks 2, 4, 5, 6 and 7 of Figure 1-1. With such simplification, DOE can
be used by a large number of manufacturers who are presently unable to do so.
This thesis is the continuation of the work done by Paul Fieguth et al. [10] based on
the insights developed by Dr. Ashraf Alkairy [1]. The earlier work focused on developing
software for Blocks 2 and 4. The contribution of this thesis is in the development of the
theory and software for Blocks 5, 6 and 7 along with an improvement of Block 2.
The thrust of the thesis is on designing sequential experiments. The need for sequential
16
designs arises due to:
* Lack of a priori knowledge of the process/plant.
* Constraints of available resources on the size of experiments.
* Desire to find a good operating point even if unable to determine the best operating
point.
* Need to conduct the experiment in a systematic and efficientmanner.
All these factors are outlined in greater detail in the next chapter.
1.2
Background
The concept of Design of Experiments (DOE) was first introduced by Fisher [11]. Plackett and Burman [19] proved that in order for the parameters of the design matrix to have
certain optimal properties it was essential that the designs have an orthogonal structure.
This resulted in the use of Factorial Designs. In these designs, as the number of variables
controlling a given process grow, the number of experiments required to completely ana-
lyze the variables and their interactions grow exponentially and so become impractical to
perform.
Ideas of fractional factorial designs were developed to reduce the total number of ex-
periments and yet retain the beneficial properties of factorial designs. The reduction in the
total number of experiments leads to the problem of confounding.
Two variables ( or interactions ) Ti and Tj are said to be confounded with each other if
they vary in the same manner in all the experiments. That is, the value of the variables for
the mth experiment is given by,
Ti,m = k Tj,m
Vm
(1.1)
for some fixed k. In two-level factorial designs each of the variables are set at -1 or +1 level
in all the experiments. Therefore, Ti and Tj are confounded with each other if,
Ti,m = +Tj,m
17
Vm
(1.2)
or
Ti,m = -Tj,m
Vm
(1.3)
Given a process, there exists no well defined technique to select which variables/ in-
teractions are to be confounded. The selection is done on an ad hoc basis and hence an
element of subjectivity is introduced in the designs. After the experiments are carried out
all the significant variables/interactions may not be determined as there may be overlapping
between then.
In order to use DOE effectively it is very important that the exact dependence of quality on the variables and their interactions be established.
Hence, whenever two or more
variables or interactions are confounded some technique is required by which their separate
effects can be determined.
Some of the DOE techniques available to determine the unknown plant models are
described below. The discussion is intentionally concise and should only serve to familiarize
the reader with the techniques.
1.2.1
Search Designs
Search Designs is a technique used for determining important interactions. Search designs
were developed by J. N. Srivastava [21]. The basic strategy of search designs is to design
a set of experiments which has the capacity to estimate the effects of a set of variables
and interactions and a few unknown interactions of a specified order. Readers may consult
Ghosh [13] or Srivastava [22] for a more detailed discussion on Search Designs.
Although the methodology is interesting there are shortcomings.
* The technique is relatively complicated and there exists no general technique to solve
all kinds of problems.
* The technique does not use a posteriori information and hence there is no obvious
extension to sequential experimentation.
1.2.2
Response Surface Methods
The Response Surface Methodology (RSM) is an alternative to factorial DOE. Like factorial
DOE, experiments are designed on a process to determine the relationship between the input
variables and the response. But in RSM the variables are not constrained to lie at any fixed
18
levels. Each design technique has its own advantages and disadvantages. These issues are
discussed in greater detail in [17].
RSM is a sequential design procedure and there are many software packages which use
RSM for process optimization [14]. Some of the popular ones are:
* Simplex- Evolutionary Operations (Simplex - V) 1
* ULTRAMAX 2
The ideas used in these procedures are significantly different from Factorial DOE discussed in this report. Hence the readers interested in RSM techniques may find the references useful.
1.3
Thesis Overview
The thesis focuses on performing sequential experimental design using two different approaches:
1. Sequential Block Design Strategy
2. One-at-a-Time Design Strategy
Chapter 2 is based on these strategies and gives an overview. The need for sequential
experimentation is discussed and the basic assumptions on the models of the manufacturing
processes are enumerated.
In Chapter 3 the theory of Sequential Block Design is developed. The detail proofs are
given in the Appendices. The algorithm used in the computer software is discussed along
with the results of computer simulations.
Chapter 4 focuses on the Analysis of Sequential Block Designs. Classical techniques such
as Daniel Plots and Anova are discussed briefly. A closed loop technique for determining
significant interactions is described.
Chapter 5 deals with the One-at-a-Time Design Strategy. The basic concepts and advantages are presented in this chapter. Simulation results are included to justify the strategy.
'Silnplex - V is a tradename of Statistical Program, Houston, Texas
is a tradename of Ultramax Corporation, Cincinnati, Ohio
2UJltramax
19
Chapter 6 describes the Complete DOE Software Tool which being currently developed
at MIT. It also lists the achievements of the thesis and suggests possible future work on
sequential design of experiments.
1.4
Notation
In this report we will try to use consistent notation. Given a plant P, its output or quality
is denoted by Y. The performance of that plant is governed by input variables. These
variables are denoted by T1, T2,..., Tn. The model of the plant is assumed to be non-linear
and the output Y is assumed to be a linear function of the variables and their interactions.
-
The interaction between the variables Ti and Tj is denoted by Iij. Denoting the coefficients
of the variables and interactions by
we can represent the output as,
Y =,3 + 1T + 2T2 +
+
nT +
(1.4)
· + Ajhlj +
!
(
o
/1
/32
Y=
i T, T2 .
~..
,
...
+ 6
.
tn
3ij
\
20
!!
(1.5)
If there is more than one experiment then there will be one equation like Equation
1.5 for
each of the experiments. All such equations can be stacked together.
/
(
o
P1
Y,
)
i
... T
Y2
1
...
Iij,1
...
Iij,2
Cl
/32
+
E,2
. (1.6)
f3n
Y./
1
T1,m
T2,m
...
Tn,m ...
Iij,m
Em
ij
\
!
A
This can be represented as,
Y=Xf- +
(1.7)
where ]Y is the Output Vector, X is the Regression Matrix, / is the Coefficient
Vector and
1.4.1
is the Error Vector.
Factorial Fractional Designs
For a process, P, dependent on n variables, if the experiments are designed is such that
* In each of the experiments, the variables are at either one of two levels, denoted by
'+' and '-'.
* There are 2n different experiments.
then the experimental design is called a Full Factorial Design.
If the experimental design has half the number of experiments of a Full Factorial Design then
it is called a One-Half Factorial Design. Similarly, if it has a quarter of the experiments
of a Full Factorial then it is called a One-Quarter
fraction of a Full Factorial Design is called a
n - k
2
Factorial Design. In general, a 1/2 k
Factorial Design.
This report deals with DOE using Fractional Factorial Designs which are based on
Binary Orthogonal Matrices.
Appendix A discusses some of the relevant properties of
Binary Orthogonal Matrices. Table 1.1 is an example of a 25-2 Factorial Design based on
the 23x23 Binary Orthogonal Matrix shown in Table 1.2. The first column of of the matrix
21
Variable:
T1
T2
T3
T4
T5
Experiment
1
2
3
4
5
6
7
8
Vl
v2
v3
V12
v 23
+
±
+
+
-
+
+
+
+
-
+
+
+
+
-
+
+
+
+
+
+
+
+
Table 1.1: 25- 2 Factorial Design Matrix
Expt.
Vavg
VI
V£
V3
V12
v23
V31
V123
1
2
3
4
5
6
7
8
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+
+
+
+
-
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+
+
Table 1.2: Basic and Non-Basic Columns of a Binary Orthogonal Matrix
22
consists only of +1 and is called the Average Column (vavg).
1.4.2
Resolution of Factorial Designs
Resolution is a useful concept associated with factorial designs. The resolution of a design
is a measure of the ability of the design to resolve confounding between interactions.
A
2 n--k
factorial design is said to be of resolution R if no Pth order interaction is
confounded with another interaction of less than (R - P)th order. For example, a design
has a resolution III if no variable is confounded with another variable but at least one
second order interaction is confounded with a variable. Designs with higher resolution are
preferred.
In this report we will usually begin the sequential DOE procedure using a design of
resolution IV or higher.
Such a design has the property that none of the variables are
confounded with second order interactions.
1.4.3
Basic Columns and Variables of Factorial Designs
In this subsection we discuss the notation used to represent an factorial design. An factorial
design is viewed as a mapping of variables and interactions to a set of columns of a binary
orthogonal matrix.
As discussed in Appendix A, given a binary orthogonal matrix X of size 2m, we can
determine a set of m independent columns. These columns are called basic columns. The
other 2m - m columns are called non-basic columns. The basic columns are denoted by
the symbols vl, v2 , ... , Vm. Every non-basic column corresponds to a product of a unique
set of basic columns. A non-basic column is denoted by
of the columns vi, vj and
k.
Vijk
if it is generated by the product
An example of a 23 binary orthogonal matrix is given in
Table 1.2.
The set of experiments can be represented by assigning to every variable T1 , T2 , ... , T,
a column of the orthogonal matrix. For example, comparing Table 1.1 and Table 1.2, we
can represent the information of the experiments simply as shown in Design(I) of Table 1.3.
It is clear that if T1 lies on column v and T2 lies on column v2 , then the interaction of
these, represented by Il2, must lie on the column corresponding to the product of columns vl
and v 2 , namely column v12 . Therefore, all variables and their interactions must correspond
to some basic or non-basic columns. In general, there are more variables and interactions
23
Expts.
Vavg
V1
V2
V3
V12
V23
12 V23V31
Design(I)
T1
T2
T3
T4
T5
Interactions
I24
114 I25
I35
I12
123
Design(II)
T1
T2
I45
134 124
T1
T2
T3
I14
I24
I34
Interactions
Design(III)
T4
Interactions
T3
V123
113
I15
I45
134
114
T4
T5
112
123
113
I35
I15
I25
T5
I12
123
I13
135
115
125
I45
Table 1.3: Changes in Confounding Patterns
than there are columns. Hence, each column of the matrix may be associated with more
than one variable or interaction. All variables and interactions lying on one column are said
to be confounded
with each other. All variables and interactions which are confounded
with each other are indistinguishable from each other, with respect to the given experiment.
The orthogonal matrix used to design a
n-
k basic columns and
2
n-k-(n
2 n-k
factorial experiment in n variables has
- k) non-basic columns. The design of experiments
corresponds to assigning n- k variables to the basic columns and the remaining k variables
to the non-basic columns. The assignment of these non-basic variables govern the overall
confounding pattern of the design. A set of variables lying on a set of basic columns are
called a set of basic variables and the other variables are called non-basic variables.
Example 1-1
Suppose a process P depends on 5 variables T1 , ... , T 5 . A 25-2 Fractional Factorial
experiment is to be designed to study the process. As explained above, this is equivalent to assigning 3 variables to a set of basic columns and the other 2 variables to
any non-basic columns. If T1 , T2 and T3 are assigned to to vl, v2 and V3 respectively,
then T4 and T5 can be assigned to any of the non-basic columns. Different choices of
non-basic columns will lead to different confounding patterns.
In Design(I) of Table 1.3, T4 and T5 are assigned to columns vl2 and v2 respectively.
The corresponding positions of the second order interactions are shown in the Table.
When the assignments of T4 and T5 are changed to columns v23 and
24
V123
respectively,
the confounding pattern of the design changes as shown in Design(II) of Table 1.3.
Design(III) is an example in which T4 and T5 are assigned to Vavg and vl23 respectively.
Such a design is not very desirable as it confounds the average effect with the main
effect of T4 . Hence, both these important effects cannot be determined.
Design(I) and Design(II) have resolution III because there are second order interactions which confound with the variables but there are no variables which confound
with another variable or the average column. Design(III) has resolution I because
T4 confounds with the average column - which can be considered as a zero order
interaction.
In this example only T4 and T 5 have been shifted keeping the variables TI, T 2 and T3
fixed. If the basic variables are shifted from one column to another along with the
non-basic variables there are problems caused due to rearrangement of rows. This
issue is discussed in greater detail in Appendix B. ·
1.4.4
Shifting with respect to a Columns
This report deals with sequential experimentation.
Using sequential experimentation it is
possible to shift the variables and interactions to different columns of the design matrix and
reduce confounding. In order to mathematically define shifting, we introduce the concept
of Shifting with respect to a Column. A variable ( or interaction ) Ti is said to have shift
with respect to column v, if and only if the old column of Ti, v, and the new column of
Xi,
vty, are such that v, = v'vy.
1.4.5
Fold-Over Designs
Given a block of 2 n-k factorial experiments, if the signs of all the elements of this block
are reversed, a fold-over design is obtained.
Wilson
This technique is originally due to Box and
[6].
If the first block has resolution III, then it can be shown that the fold-over design
together with the first block results in an factorial design of resolution IV, that is one in
which none of the second-order interactions are confounded with the variables. Therefore,
using such a design it is possible to determine the effect of all the variables independent of
the effect of ally second-order interactions.
25
26
Chapter 2
Sequential Experimental Design
In this chapter the concept of sequential design of experiments is introduced.
Section 1
deals with the non-sequential approach to the DOE and the problems with it. In Section 2
the overall sequential approach is outlined.
Section 3 lists the assumptions made on the
manufacturing models.
2.1
Non-Sequential Approach to DOE
The concept of Sequential Experimental Design is not completely understood especially
when the variables are at three and higher levels. Therefore, most of the commercial
software available for factorial DOE support non-sequential experimental design1 . As shown
1A few of the commercial software products surveyed (PC-QPI and RS/Discover)
do have sequential
design features but they do not generate optimal experiments based on previous results. Rather the user has
to design the experiments and the software analyzes the results. Also, these commercial software products
do not support sequential block analysis.
Figure 2-1: Non-Sequential Approach to Experimental Design
27
in Figure 2-1 the non-sequential approach to DOE can be divided into the following steps:
1. Selecting variables and potential variable interactions which are perceived as being
most important in determining the performance of a given process.
2. Given the number of variables, designing a set of experiments which is to be performed.
3. Conducting the experiments and obtaining the results, i.e. values of the output parameters to be optimized, such as strength, variance, etc.
4. Analyzing the results to obtain the coefficients of the variables and their interactions
in the mathematical expression Equation 1.6 predicting the output parameters.
5. Using these coefficients to suggest new settings of the control variables, T, to improve
the process.
In order to use non-sequential experimental design, the experimenter has to guess the probable interactions. The commercial DOE software can be used only after the list of probable
interactions has been obtained. Some of the main features currently available in commercial
DOE software include the following:
* Designing factorial experiments based on various criteria of optimality such as Doptimality
2 [2].
* Analyzing results using different techniques. Some of the commonly used include:
1. Analysis of Variance (Anova)
2. Normal Probability Plots
3. Bayesian Estimation
Despite these advancements the commercial DOE software products are limited in their
application. Some of the main problems are:
* It is not possible to suggest a general design methodology which can be applied to all
processes. The concept of DOE depends a lot on the specific process to be optimized
and therefore a design that may be good for one process might not be so for another.
2
A design matrix is said to be D-optimal with respect to a set of matrices if it has the maximum
determinant in the set.
28
* The experimenter has to guess the interactions. In order to be safe it is necessary to
guess very conservatively. Thus, the effects of a much large set of interactions has to
be determined than is actually necessary. This results in the use of more experiments
than necessary.
* Irrespective of the sophistication of the design and analysis procedures, one cannot
overcome the fundamental limitations of confounding. That is if one variable/ interaction is confounded with another, then the effects of each cannot be evaluated
separately irrespective of the nature of the analysis.
* Typically, there is a limit to an experimenter's knowledge about the process he or she
wishes to optimize. Hence it is often true that having performed the experiment some
of the following situations might arise.
1. Some of the columns of the test matrix which were assigned to variable/ interaction(s) which the experimenter thought were unimportant were found to have
large coefficients.
2. The results of the experiments suggests the possible presence of interaction(s)
which are confounded with other variables/ interactions and hence could not be
evaluated.
3. Only a few variables and interactions are found to be important.
Hence the experimenter might be forced to ask: WHAT NEXT???
This is exactly the
question that we hope to address in the course of this thesis.
2.2
Sequential Design Philosophy
In this project we aim to address the issue of DOE in a systematic manner. Given a process
which needs to be optimized the experimenter should start by studying its physics. Based
on this, a group of variables and interactions should be selected. The software will then
use this information and will design a matrix which has minimum possible confounding
between these variables and interactions. After conducting the experiments and analyzing
the results it may happen that the experimenter is not completely satisfied due to one or
more of the reasons stated in Section 2.1. Thus in order to proceed the software needs to
29
consider all the possible conditions for which an experimenter may not be satisfied with
the results of the first set of experiments and suggest acceptable solutions for each of these
conditions.
It is difficult to determine which set of experiments the experimenter should perform,
in the most general sense. The choice of experiments depends not only on the specific
questions which the experimenter wishes to answer but also on the nature of the process at
hand. There are many factors which need to be considered.
* The level of noise in the experiments.
* The number of experiments that the experimenter wishes to perform.
* The accuracy required in estimating the parameters.
Fractional Factorial Designs have very interesting and important properties and are very
popular in experimental design techniques.
* The experiments are easy to perform.
* The analysis of the results yield uncorrelated estimates of the effects of the variables
and the interactions.
* Factorial designs support sequential experimentation.
* They can be applied to a large class of manufacturing processes.
Given these advantages, it is desirable to design a new fractional factorial experiment.
So the question is how to design a new block of experiments which will supplement the old
block of experiments and enable the experimenter to resolve many of the problems stated
above? The Sequential Block Design and the One-at-a-Time Design are developed with this
perspective. They are briefly described below.
2.2.1
Sequential Block Design
In this methodology first a fold-over block of experiments is designed. Since the design used
is at least of resolution IV, the effects of the variables can be determined independent of the
effects of second order interactions. Based on the assumptions on the models discussed in
Section 2.3, a list of probable significant interactions is formed. There may be confounding
between these interactions.
30
With the help of the Half-Block Design and the Full-Block Designs discussed in Chapter 3, an optimal set of experiments is obtained which 'best' disentangles the confounding
interactions.
Hence, the new set of experiments allows the determination of the effects of
the confounding interactions. It may happen that there are still a few interactions which
remain confounded, but the optimal set of experiments ensures that these are relatively
less important.
If desired, a second optimal design may be used to determine the effects of
these interactions. Hence using this method we can sequentially sort out the confounding
between interactions. Sequential block designs are discussed in detail in Chapter 3.
2.2.2
One-at-a-Time Design
In most manufacturing processes the number of significant variables and interactions affecting the quality is usually small. Suppose a fold-over block of experiments has been
conducted.
It is often reasonable to assume that at most one variable or interaction is
significant in any column of the design matrix.
Therefore, once the effects of the columns have been determined, only the significant
variable or interaction in each column needs to be determined. This is done by first making
a hypothesis about the significant variable or interaction in each column. Based on this
hypothesis an optimal operating point is computed. The actual output of the plant at that
operating point is obtained. The actual output is compared with the predicted output and
the hypothesis is verified. If the hypothesis fails, a new hypothesis is proposed.
There are several advantages of this approach. One-at-a-Time designs are discussed in
detail in Chapter 5.
2.3
Assumptions on Manufacturing Models
In this report we assume that the manufacturing systems that we deal with satisfy the
following assumptions. In case there are any deviations they will be noted in the report.
Al - Sparsity-of-Effect Principle: Most of the systems are dominated by the effects of
the variables and the low-order interactions. Most of the higher order interactions are
negligible. This assumption will also be referred to as the Sparsity Principle in the
report.
This assumption is widely used and is supported by experienced practitioners of DOE.
31
A2 - Minimal Complexity
of Models:
Most systems are governed by interactions that
are important whenever the main effects of some of their variables are important too.
This assumption will be called the Simplicity Principle in the report.
A3 - Uncorrelated
Noise Distribution:
The noises in the different experiments are un-
correlated and are approximately gaussian distributed with zero mean and constant
variance. This justifies the use of least-square techniques in estimating the manufacturing models.
A4 - Sequential Experimentation:
It is possible to combine the results of two or more
fractional factorial experiments to assemble sequentially a larger design to better
estimate the effects of the significant variables and interactions.
32
Chapter 3
Sequential Block Design
This chapter discusses the concept of Sequential Experiments using Block Designs. Section 1
discusses the need for such a methodology. Section 2 deals with the initial steps required
to do sequential experimental design. Section 3 discusses Half-Block Designs. A detailed
example is given to help the reader understand the technique.
Section 4 discusses Full-
I3lock Designs and Fold-over Designs. Section 5 illustrates a method for completing an
incomplete block of experiments. The mathematical proofs of the procedures are given in
the Appendices.
3.1
Need for Sequential Block Design
Consider a situation in which a manufacturer, inexperienced in DOE, desires to use experimental design to improve the quality of the product. Before starting experimentation, the
manufacturer must do the following:
* Choose variables and their operating ranges.
* Guess the important interactions affecting the response.
* Choose a response or quality variable which really provides useful information about
the process under study.
It is not desirable to do only one block of experiments or a large first block of experiments
since:
1. The levels of the variables may be incorrectly chosen, making the effects of some of
the variables dominate the results.
33
2. Since the significant variables and interactions are not known a priori, it is advantageous to use the information from the first block of experiments to design subsequent
ones.
3. There are always constraints of time, funds etc. on the total number of experiments
that can be performed. Therefore, it is desirable to conduct most experiments when
there is more information available about the process.
It is usually recommended that the experimenter invest no more than 25 percent of
the available resources in the first block of experiments [17]. Therefore, it is necessary to
use an experimental design technique which supports sequential experimentation.
Hence,
fractional factorial designs are often used. A detailed description of factorial designs can
be found in several books including [9], [17], [8]. In this report the reader is assumed to be
relatively familiar with the basic concepts of fractional factorial designs.
3.2
Sequential Design Methodology
Given a process P which depends on n variables Tl,...,T,,
a complete factorial design
requires 2 n experiments to determine the effects of all the variables and interactions. Even
for moderate values of n, the complete factorial design requires an infeasible number of
experiments to be performed.
experiments
For example, for n equal to 8, 9 and 10, the number of
required is 256, 512 and 1024 respectively.
Most manufacturing systems satisfy the assumptions given in Section 2.3 and therefore require far fewer experiments to completely determine the significant variables and
interactions. The basic steps involved in sequential block design are described below.
Step 1: Designing First Block of Experiments
We advocate that the first block of experiments should be a fold-over design. To obtain
a fractional factorial design which is also a fold-over design, first a 2 m factorial matrix is
designed, where m is the smallest integer for which 2 m > n. The n variables are then
assigned to unique columns of this matrix. The 2m factorial matrix is now folded i.e. a
matrix is formed whose elements are obtained by changing the sign of the elements of the
factorial matrix. The matrix is appended to the original factorial matrix and the new matrix
of
2
m+l
experiments is the desired fold-over design.
34
Variables
Expts.
1
2
3
4
1'
2'
3'
4'
------------
T1
& Interactions
B
L
K
1
F
0
L
D
T2
T3
T4
112
I34
I23
I31
114
124
Vl
V2
V3
V123
V12
V23
V3 1
+
+
+
+
+
+
+
-
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+
-
+
-
+
+
+
+
Table 3.1: Fold-over Design Matrix for Example 3-1
Example 3-1
* Consider a process P which depends on 4 variables T1, T2 , T3 and T4. To obtain a
fold-over design, a 23 factorial experiment needs to be designed. First a 22 factorial
matrix is designed as shown in Block 1 of Table 3.1. The four variables are assigned
to the four columns of this matrix. Then the matrix is folded over and appended to
Block 1. The overall matrix corresponds to the desired fold-over design. Note that
none of the variables are confounded with any second order interaction.
Step 2: Analyzing the Results
After conducting the fold-over experiments, the coefficients of the columns of the orthogonal
matrix are obtained. The fold-over design has resolution IV. Hence, if it is assumed that
the third and higher order interaction are insignificant, the estimates of the main-effects
of the variables are the coefficients of the columns on which they lie. Using the analysis
techniques described in the next chapter, a list of significant variables is obtained.
Step 3: Determining Probable Interactions
On the basis of the Minimal Complexity of Model assumption, a list of all second order interactions containing at least one significant variable is made. From this list, the interactions
which lie on columns with significant coefficients are selected. This group of interactions
are called the probable interactions. By assumption, all the true interactions must lie in this
group. It may happen that there may be many probable interactions which are confounded
with each other and hence their effects cannot be determined from the results of the first
block of experiments.
35
Step 4: Resolving Confounding between Interactions
In case there is confounding between the probable interactions it should be possible to
conduct more experiments to unconfound them and determine their separate effects. If we
view the variables and interactions as 'sitting' on the columns of the orthogonal matrix,
the DOE techniques correspond to 'placing' the variables on the columns of the matrix.
Once the variables have been assigned to the columns of the matrix, the positions of the
interactions get fixed.
Hence, in order to unconfound the interactions, the variables need to be shifted from
one column to another with the help of more experiments.
In order to unconfound the
interactions, we must exactly understand the mechanics by which the variables can be
shifted from their original positions. Once the mechanics is understood it is possible to
search through the design and determine the columns to which the variables can be shifted
to achieve maximum unconfounding.
Shifting a variable from one column to another is equivalent to designing an experiment
with a different confounding pattern. This idea is discussed in more detail in Appendix B.
Given that the first block of
2
n-k
factorial experiments has been conducted it is desired
that the second block of experiments leads to a fractional factorial design with a different
confounding pattern. There are essentially two options.
1. If possible, replace some of the rows of the first block of experiments by those of a
second block, and create a distinct fractional factorial design.
2. If possible, append the rows of the first block of experiments to the rows of a second
block of experiments to create a larger fractional factorial design having a different
confounding pattern.
Both these options are possible. The technique based on option 1) is called the HalfBlock design and the one based on option 2) is called the Full-Block design. These designs
are described in the next two sections.
3.3
Half-Block Designs
The focus of this section is to study the problem of unconfounding probable interactions.
It is assumed that a 2 n-k factorial experiment has been conducted on the process and the
36
effects of the variables are known. On the basis of this, a list of probable interactions is
obtained. The effects of all the probable interactions are known, except for those which are
confounded with each other.
We need to determine the effects of the confounded probable interactions. Therefore, a
fractional factorial experiment has to be designed which has a different confounding pattern.
In order to save resources it is natural to inquire if it is possible to design a new factorial
experiment in which many of the experiments of the first block are similar, rather than one
in which all the
experiments are different from those of the first block. This issue has
n-k
2
been discussed in Appendix B. The main result is given below. The readers interested in
the proof can refer to Appendix B.
Result:
Given a block of
n
2 -k
-' k - 1
1. At least 2
factorial experiments in n variables,
experiments of the block need to be changed to obtain a new block of
factorial experiments with a different confoundingpattern.
2. New blocks of factorial experiments which can be obtained by doing an additional set of
2
n- k - 1
experiments, can be generated by shifting the variables with respect to every
column of the design matrix.
3.3.1 Algorithm to Determine the Optimal Half-Block Design
To begin with, we need to assign each of the probable interactions a weight based on the
significance of its variables and the coefficient of its column. That is, an interaction of two
important variables should be weighted more than one in which only one of the variables
is significant. Also, an interaction lying on a column with a large coefficient should be
weighted more.
In this report, the weight of an interaction is obtained by taking the absolute value of
the products of the coefficients of its column and those of its variables. There are other
acceptable means of assigning the weights. Further work could be done to examine them
and determine if some are better than others.
The goodness of a design k is evaluated on the basis of four quantities:
1. Number of columns in which the variables confound with probable interactions, Nvik.
37
2. Sum of the weights of the interactions confounded in 1., Wvik.
3. Number of columns in which the probable interactions are confounded with each other,
Niik.
4. Sum of the weights of the interactions confounded in 3., Wiik.
For any two designs, design k and design 1, design k is said to be better than design I if
Nvik
< Nvil. If Nvi
= Nvil, then design k is better than design I if WVik < Wvil. If
terms 1. and 2. are equal for both the designs, then design k is better than design 1 if
Niik < Niil.
Eventually if terms 1., 2., and 3. are equal, then design k is better than
design 1 if Wiik < Wiil.
This method of comparing different designs ensures that:
* Designs with less confounding between variables and interactions are preferred.
* Designs which have confounding between probable interactions with high weights are
avoided.
* A design with no confounding is preferred over all other designs.
This method of comparison has been found to yield good results. Nevertheless, there are
other measures which would be acceptable too. Further research should be done to determine the existence of an 'optimal' comparison method which performs best over the
ensemble of manufacturing processes.
Given a block of experiments, the variables are shifted to with respect to different
columns and different designs are obtained. The search procedure follows the Tree Search
Algorithm.
Each time a better design is found it is called the current optimum design
and is stored in memory. The search procedure stops when either all four quantities given
above reduce to zero, or if no design is found which is better than the current optimum
design. The Tree Search algorithm ensures that all acceptable sets of
2
n - k -1
experiments
are searched.
The results given in Appendix B greatly reduce the search. Shifting any one variable
completely determines the permissible columns for the placement of all the other variables.
That is, once any one of the variables has been shifted, the other n -1 variables can each be
placed in only 2 of the 2 n-k columns. Whenever the variables are placed in the permissible
38
Experiment:
Result:
1
2
3
4
1'
2'
3'
4'
16.21
8.41
23.64
15.35
11.72
-6.22
16.57
0.84
Table 3.2: Results of Fold-over Design of Example 3-2
Column:
Vavg
Variable
& Interaction:
Coefficient:
10.84
VI
V2
V3
V123
V12
T1
T2
T3
T4
-0.31
5.11
-0.33
-2.22
V23
V13
112
I23
I13
134
-3.31
114
I24
0.24
6.19
Table 3.3: Analysis of Design of Example 3-2
columns, there exists a block of 2 n -
k-
1
experiments which along with
2
n - k-1
experiments
of the first block gives a new 2 n-k factorial design.
Once the variables have been placed in the permissible columns, the procedure for
obtaining the new experiments is simple. The new design is generated from the old one
l)y shifting some of the variables with respect to a column of the old design as described
in Appendix B. The new block of experiments are the
2 n-k-1
rows of the new design
matrix in which the elements of this column are -1. The rows for which the elements of the
column are +1 are common to both the new and the old block of experiments and therefore
these experiments need not be done again. Once the
2
n-k-1
new experiments have been
conducted, the new matrix can be formed and the effects of the probable interactions can
be dletermined.
Example 3-2
* Consider the process given in Example 3-1. Suppose that the true expression for
quality is given by,
y = 10.6 + 5.2T 1 - 2.5T 3
-
3.1112 + 6.1113
(3.1)
The first block of experiments is the fold-over design of Table 3.1. The results obtained
are shown in Table 3.2. The coefficients of the columns obtained from these results
are shown in Table 3.3.
From Table 3.3 it is clear that only columns Vag, vl,
39
V3,vl3
and vl2 are significant.
-
Experiment
1"
2"
3"
4"
T1
T2
T3
T4
Result
-
+
+-
+
+
+
+
10.31
-6.49
17.33
-0.36
Table 3.4: Half-Block Experiments and Results of Example 3-2
Column:
Vavg
Variable
& Interaction:
Coefficient:
10.57
V1
V2
V3
V123
V12
V23
V13
T1
T2
I34
T3
I24
I14
112
I23
T4
I13
5.37
-0.17
-2.32
0.07
-3.46
-0.15
6.31
Table 3.5: Analysis of New Design of Example 3-2
Therefore, the significant variables are T1 and T3. The possible significant interactions
are: 112,I13, I14, I23 and 134. Of these, only I12, I34 and I13 lie on significant columns
and constitute the set of probable interactions.
Since it is assumed that third and higher order interactions are negligible, the coefficients of the columns v,,g, vl and v correspond to the effects of the constant, T1 and
T3 respectively. Since 124 is assumed negligible, the coefficient of vl3 is an estimate
of the effect of I13. The effects of probable interactions I12 and I34 cannot be determined separately as the interactions are confounded with each other. Therefore, we
need to design a Half-Block experiment to unconfound the effects of I12 and I34. The
Half-Block algorithm suggests shifting T4 from column vl23 to v2s, that is, shifting T 4
with respect to column vl. Thus, the experiments in which the elements of column vl
are '-' need to be done. The sign of T4 is reversed in each of these experiments. The
Half-Block experiments are shown in Table 3.4. The results of theses experiments are
given in the same table.
The experiments of Table 3.4 along with Experiments 1, 2, 3 and 4 of the the first
block give a new orthogonal matrix which has a different confounding pattern. The
confounding pattern and the result of the analysis of the new matrix is shown in
Table 3.5.
From Table 3.5 it is observed that each column has only one significant variable or
40
probable interaction. Thus, the effect of each of them can be determined. Since the
column v2 in Table 3.5 has a non-significant coefficient, it implies that
significant.
I34
is not
The coefficients of columns of 13 and I12 have significant coefficients
which agree well in the result of the analysis of the old and the new design matrix.
Hence I13 and I12 are significant interactions.
The results of the analysis of the two designs can be improved by taking the average
of the corresponding effects in Table 3.3 and Table 3.5. Dropping the non-significant
effects, the model of the plant is given by:
y = 10.71 + 5.24T1 - 2.27T 3 - 3.3912 + 6.25I13
(3.2)
Equation 3.1 and Equation 3.2 agree quite well. Hence the Half-Block methodology
has been effectively used to estimate the model of this process.
In this example it is not possible to determine the optimal operating point of the
process before obtaining the model of the process because the conditions cited in
Section 5.5 are not satisfied.
The interactions I14 and I23 were dropped form the list of probable interactions because they were located on a non-significant column. If these interactions are significant the coefficients of the columns in Table 3.3 and Table 3.5 will not be consistent.
For example, if I23 is significant, the coefficients of the column I12 would not be similar
in both Table 3.3 and Table 3.5 since it includes the effect of I23 in Table 3.5. This
procedure can be used to verify the process model.
3.3.2
Advantages and Disadvantages of Half-Block Designs
The Half-Block designs procedure described in this section is suitable for a large number of
situations.
1. If the number of confounded probable interactions is small.
2. If the first block of experiments is large. The size of the design matrix increases
exponentially with the number of variables, n. Even in the worst case the number
of probable interactions (of second order) increase with n2. Therefore, as the number of variables increase, the number of columns increases and it becomes easier to
41
disentangle the probable interactions using Half-Block designs.
3. Given a block of
2
n-
k
factorial experiments, a Half-Block design uses
ments of this block to forms a new block of
2n
k
2
n-k-
1
experi-
factorial experiments. The analysis
of the new block yields uncorrelated estimates of the effects of the unconfounded
variables and interactions.
There are a few limitations of this design technique.
1. When a Half-Block design strategy is applied to a block of 2 n-k factorial experiments,
2
n-k-
1
experiments are determined which form a new block of
iments with a group of
2
n- k -1
2
n-k
factorial exper-
experiments from the first block. The new block has
a different confounding pattern. It is possible to estimate the effects of the variables
and the interactions by analyzing all the
been conducted.
2 n-k
+
2
n-k-
1
experiments which have
The estimates obtained from this analysis will be correlated with
each other because the columns of the regression matrix are not orthogonal to each
other.
Instead, if only the experiments of the new block of
2
n- k
factorial experiments are
analyzed, the regression matrix is orthogonal and the estimates will be uncorrelated.
These estimates will have a larger variance because the number of experiments used
in the analysis is reduced. Thus there is a trade-off between smaller variance and
uncorrelatedness of the estimates.
2. If the design is highly confounded, it may not be possible to unconfound all the
probable interactions using Half-Block designs.
This limitation is due to the fact that Half-Block designs can only restructure the
confounding pattern and not decrease the confounding. The optimal restructuring
ensures that the probable interactions get unconfounded at the expense of the nonsignificant interactions. Therefore, the only manner in which the overall confounding
can be reduced is to design a larger matrix which is less confounded. This is the issue
of the next section.
42
3.4
Full-Block Designs
This section deals with the theory and application of Full-Block Designs. The Half-Block
designs described in the last section are very useful but there may be situations where,
having conducted a
2 n-k
factorial experiment it may happen that:
1. The number of confounded probable interactions is large.
2. Uncorrelated estimates of low variance are required.
3. Resources permit conducting more than a Half-Block Design.
In such cases it is more desirable to have a design technique which simultaneously uses all
the experiments to determine the model of the process. The use of all the experiments is
advantageous on two counts. One, the larger design matrix leads to less confounding and
two, the estimates obtained from analysis have a smaller variance.
The basic problem can thus be stated as follows. Having done a block of
2 n-k
factorial
experiments on a process, which is the next block of experiments that need to be performed
so that the analysis of both blocks 'best' unconfounds the probable interactions?
This
question is the focus of Appendix C. The main result is stated below. The proof and an
illustrative example is given in Appendix C.
Result:
Given a block of 2 n-k factorial experiments in n variables,
1. A new block of 2 n-k experiments forms a block of 2 n-k+1 factorial experiments along with
the original block only if both the new and the original block have the same confounding
pattern.
2. There are 2k-1 possible blocks of 2 n-k
experiments which can form a block of
2
n-
k+1
factorial experiments with the original block. The new blocks can be generated by
reversing the signs of the columns of the non-basic variables in the original block.
3.4.1 Algorithm to Determine the Optimal Full-Block Design
In the Full-Block design algorithm, different blocks of
2 n-k
experiments are generated by
reversing the signs of all the possible subsets of the non-basic variables.
unique.
Each block is
The confounding pattern of the overall design is determined for each of these
43
Experiment:
Result:
1
18.59
2
18.43
3
13.89
4
14.60
1'
2.81
2'
-7.18
4'
8.58
3'
18.05
Table 3.6: Results of Fold-over Design of Example 3-3
Column:
Vavg
Variable
& Interaction:
Coefficient:
10.97
V1
V2
V3
V123
V12
v23
V13
T1
T2
T3
T4
112
I34
I23
I13
I14
I24
-2.81
0.17
2.36
5.41
4.94
-2.50
0.04
Table 3.7: Analysis of Design of Example 3-3
blocks. The Tree Search algorithm is used to determine the optimal new design using the
criterion described in the previous section.
The results of Appendix C lead to a considerable simplification in the search procedure.
The Tree Search algorithm ensures that all 2 k-1 block designs, which can be used with the
first block to give a 2 n-k+1 Fractional Design, are searched. Hence the search gives the true
optimum design.
The new block of experiments to be performed corresponds to the old design in which
the signs of certain columns of the non-basic variables are reversed in accordance with the
result of the search.
Once the new block of experiments have been carried out, the new design can be appended to the old one. The overall design corresponds to a
2
n-k+ l factorial design. The
algorithm used to generate the new block of experiments guarantees that the overall design
matrix has optimal confounding between the probable interactions.
Example 3-3
* Suppose the process in Example 3-1 is given by,
y = 10.6 + 5.2T1 + 4.9T 2 - 2.5T 3 - 3.1112 + 6.1113 - 37124
(3.3)
Let the first block of experiments be the fold-over design of Table 3.1. The results of
these experiments are given in Table 3.6. The coefficients of the columns of the matrix
are given in Table 3.7. It is observed that the variables T1, T2 and T3 are significant.
According to our assumptions the possible significant interactions are 112, 113, 114,
44
Experiment
1
2
3
4
5
6
7
8
T1
+
+
+
+
-
T2
+
+
+
+
T3
+
+
+
+
T4
+
+
+
+
Result
24.76
11.11
21.72
6.58
9.99
-14.78
25.29
1.14
Table 3.8: Full-Block Experiments for Example 3-3
Column:
V4
Vavg
V1
T1
T2
T3
T4
112
113
114
Coefficient:
10.85
5.36
4.90
-2.51
0.15
-2.88
6.04
0.14
Column:
V23
V24
V34
V123
V124
V134
V234
V1234
Var.&
Int.:
123
124
134
1123
1124
1134
1234
Coefficient:
0.03
-3.67
-0.11
0.01
0.07
0.05
1234
0.05
Var.&
Int.:
V2
V3
V12
V13
Table 3.9: Analysis of New Design of Example 3-3
45
V1 4
0.12
I23, I24 and I34. But the column of v2 3 is not significant. This implies that I23 and
not significant 1. Thus there are 4 probable interactions 112, 134, 113and 124-
114 are
The first two and the last two probable interactions are confounded with each other.
When a Half-Block design algorithm is applied to this problem, it is found that there
exists no Half-Block design which can unconfound all the probable interactions. Thus
we have to resort to a Full-Block design. The Full-Block design algorithm suggests
reversing the sign of the column of T4 keeping the other columns unchanged. Table 3.8
gives the Full-Block experiment. The results of the experiments are also given in the
same table.
The results of the analysis of all the 24 factorial experiments are given in Table 3.9.
Using Table 3.9 and selecting the significant terms, the estimate of the model is given
by:
y = 10.85 + 5.36T1 + 4.90T 2 - 2.51T 3 - 2.88112 + 6.04113 - 3.67124
(3.4)
This estimate compares very well with the true plant model,
y = 10.6 + 5.2T 1 + 4.9T2 - 2.5T3 - 3.1112 + 6.1113 - 3.7124
(3.5)
Notice that if our assumption that I23 and 114are not significant were wrong, the
coefficients of the column on which they lie would be significantly different in the
results of Table 3.9 and Table 3.7. Comparing the two results would permit the
estimation of the effects of these interactions.
3.5
Completing Block Designs
This section deals with the issue of completing an incomplete block of fractional factorial
experiments. Suppose a block of 2 n-k factorial experiment has been conducted followed by a
Half-Blockof experiments. The Half-Blockhas the property that it forms a
matrix with
2
n-k-1
2 n-k
orthogonal
experiments of the first block of experiments. There may be situations
where, after conducting a Half-Block, it is desired that a set of experiments is found which
1There is a rare possibility that I23 and 14 have effects which are similar in magnitude but opposite in
sign and therefore cancel each other. A check for this possibility is discussed later.
46
-_____I
-
Orthogonal
:Matrix
2 n-k
Orthogonal
Matrix:
Experiments
2
n-k +1
Experiments
I
I
I
I
II
Half- Block:
:
l
2 n-k-I
Expts.
I
:
I
I
Orthogonal
2
n-k
Matrix
Experiments
Figure 3-1: Pictorial Representation of Completing Block Design
together with all the experiments conducted previously, determine a orthogonal matrix of
size
2
n-k+
l.
This idea is represented in Figure 3-1. The procedure for completing a block
is equivalent to finding the missing block of 2 n-k-1
experiments shown in Figure 3-1.
The concept of completing blocks is discussed in detail in Appendix D. The procedure
is demonstrated with an illustrative example. The general procedure for completing blocks
is given below.
Procedure for Determining the Missing Block:
Given an incomplete block of m (2 n -
k
< m <
2n
- k+ l
) experiments of a block of
2
n-k +1
factorial experiments,
Step 1. Determine the basic variables and the confounding pattern of the incomplete block.
Step 2. Check which of the
2
n-k + l
combinations of +1 and -1 are missing from the columns
of the basic variables. There will be
2 n-k+l-m
such combinations.
Step 3. For each of the combinations determined in Step 2, determine the entries of the
non-basic variables of the matrix using the confounding pattern of the incomplete
block.
47
48
Chapter 4
Analysis of Sequential
Experiments
This chapter deals with the analysis techniques used to analyze sequential experiments.
Section 1 deals with classical techniques of analysis. Section 2 discusses the issue of closed
loop techniques for determining interactions.
4.1
Classical Analysis Techniques
In this section we deal with the classical analysis techniques that have been added to the
MIT computer software to analyze sequential block designs.
These techniques are well
known and are discussed very briefly.
4.1.1
Analysis of Variance (Anova)
It is probably the most important technique used in statistical inference. The name is
derived from a partitioning of the total variability of the experimental results into component
parts.
For a process P, dependent on n variables T1 , ... , T,
if Yi (i = 1,..., N) is the
result of the ith experiment and Y is the mean of the results, then the total sum of squares
N
SST =
(Y i=l
49
)2
(4.1)
-/0 ~OnTn
0l121
++...
'
is used as a measure of the overall variability of the results. Suppose the process is modeled
as,
=
+
1T1 + /2T2 + ... +
+
. . . +- ijlij
+
(4.2)
where the effects 1,, are determined using the least-square technique, then it can be shown
that
SST:= L i2 + E/3j 2 +
i
i,j
ci 2
(4.3)
i
The last term in Equation 4.3 is called the sum of squares of error, SSE, that is,
SSE = E i2
(4.4)
If there are a terms in the model of the plant shown in Equation 4.2, then SST and SSE
have N - 1 and N - a degrees of freedom, respectively. The F-ratio Fx, for a coefficient /,
is given by
SSE/(N - a)
(4.5)
and it represents the measure of significance of the coefficient. If it is assumed that the noise
has a gaussian distribution then Fx has an F-distribution. If Fx is large then the variable/
interaction of the coefficient is significant.
A more detailed discussion on Anova can be found in many books including [17],[8].
4.1.2
Normal Probability Plots
This method of analysis attributed to Daniel [7] provides a simple way to determine significant variables and interactions. Given the result of a factorial experiments, the coefficients
of the columns of the orthogonal matrix are calculated using the Least Squares technique.
In order to construct a normal probability plot', the N coefficients are arranged in
increasing order and the kth of these ordered coefficients is plotted versus the cumulative
probability point Pk = (k - 1/2)IN on the normal probability paper.
The coefficients which contain only the noise effects tend to be normally distributed
with a zero mean and a common variance. Thus they lie along a straight line on the plot.
The slope of this line is the estimate of the common variance.
1
A normal probability plot is a plot on a normal probability paper, a graph paper in which the ordinate
has been scaled so that the cumulative normal distribution plots as a straight line.
50
Normal Probability Plot
!'
!
!
0.98
_...:
..
..........
.......
.........
..........
0.95
l...,
..........................
: .
0.90
',*
,*'/
.....
_
...................
.
.
.
+
i,
U./D .. ... .. .. ... ... .. . . .. ... ... ...... .. .
. . . . . . . . . . . . . . .
.. . . . . . . . . . . .
>
X
...............................................
0.50 . . . . .. .. ... ... ...... .. ... ... ...... ... .
0
0.25
.
. . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . -
/ -I0.10
+
0.05
;
./'/
/'
0.02
I
-6
.
I
-4
-2
/I
I
0
2
4
6
8
Data
Figure 4-1: Normal Probability Plot
On the other hand, the coefficientswhich contain the effects of the dominant variables
will typically have non-zero means and lie significantly away from the straight line. Thus
the normal probability plot can be used to determine these dominant coefficients.
Example 4-1
* A group of 20 random numbers are generated on the computer. Of these 20 numbers,
15 are normally distributed with zero mean and unit variance.
The remaining 5
numbers have unit variance but non-zero means. As seen from Figure 4-1 the numbers
with zero means tend to lie on a straight line and can be distinguished from the other
numbers with non-zero means.
4.1.3
Lenth's Algorithm for Determining Significant Coefficients
In this section we describe the Lenth's algorithm [15] which has been used to determine the
significant coefficients in this report. The advantages of this algorithm are that it is quick
51
and easy to incorporate in computer software, and it does not require the user to use any
judgment in determining the significant coefficients. The algorithm is described below.
Consider a process on which a
2
n-k
factorial design is conducted and the results of the
experiments are analyzed. Let the coefficients of the design matrix be denoted by a,...,
(m =
2 n-k).
a.,
If the noise in the process is assumed to be uncorrelated and gaussian dis-
tributed, then the coefficients will have a normal distribution.
Let ai be independently
distributed with mean 'yi and variance a 2. Let,
So= 1.5x
median
ail
(4.6)
The pseudo standard error (PSE) is defined as,
PSE = 1.5x
median
m1il<2.So lail
(4.7)
It has been shown in [15] that when the assumption of Sparsity Principle (Section 2.3)
is valid, PSE is a fairly good estimate of a. Once the estimate of a is known, the coefficient
ai is significant if,
Iail
where t.97 5 ;m/
3
> t. 9 7 5 ;m/
3
X PSE
(4.8)
is the .975th percentile of a t-distribution of m/3 degrees of freedom. The
range of values of t.975;m/3 varies from 3.76 for m = 8 to 1.99 for m = 256.
4.2
Closed Loop Technique for Determining Interactions
In most manufacturing processes the third and higher order interactions are negligible.
Suppose a fold-over experiment has been conducted and analyzed. The significant columns
can be determined by using any of the techniques discussed in the previous section. Since
the fold-over design has resolution of IV, the coefficients of the columns with variables are
the estimates of the effects of the variables. The group of variables lying on significant
columns form the group of significant variables.
Thus, only the effects of the interactions remain to be determined. In case there is confounding, the assumption of Simplicity Principle discussed in Section sec-ass, helps reduce
the total number of interactions that need to be analyzed. The group of all the interactions which satisfy the assumption and lie on columns with significant coefficients are called
52
probable interactions and have been discussed in Chapter 3.
Thus further experimentation is needed to determine the effectsof the confoundedprobable interactions and to determine which of them are significant. The Half-Block and the
Full-Block designs of Chapter 3 were developed with this goal.
It is possible to view the problem of determining the significant probable interactions
from a different perspective. Instead of determining the effects of all the probable interactions, we can hypothesize that a subset of the probable interactions are significant and then
design experiments to check this hypothesis. This approach is developed in Chapter 5.
53
54
Chapter 5
One-at-a-Time Designs
The sequential block design strategy was discussed in Chapter 3. In this chapter we propose
an alternative One-at-a-Time Design strategy to determine the unknown probable interactions. Section 1 deals with the need to have such a method and the kinds of applications
where it will be useful. Section 2 outlines the algorithm and illustrates it with examples.
Section 3 presents a mathematical justification of this approach.
The advantages of the
One-at-a-Time approach are discussed in Section 4. Section 5 deals with the concept of
Partial Optimization.
5.1
Need for One-at-a-Time Design Strategy
Consider a manufacturing process for which the assumptions of Sparsity Principle, Simplicity Principle and Uncorrelated Noise Distribution discussed in Section 2.3 hold. Let this
process depend on n variables T1 , ... , Tn. Suppose a
2
n-k
factorial fold-over experiment
is conducted on this process and the significant variables and probable interactions are determined. In many processes the number of significant variables and interactions are small
and it is reasonable to assume that each significant column has only one significant variable
or interaction lying on it.
Therefore, under these assumptions, once a
2n
- k
factorial experiment is analyzed,
1. Significant variables and their effects are known.
2. Unconfounded probable interactions and their effects are known.
3. The effects of the confounded interactions are known although the interactions need
55
to be determined.
If the number of confounding probable interactions is small there should be no need to
design either a Half-Block or a Full-Block experiment to determine the true interactions.
Instead it should be possible to determine the interactions with only a few more experiments.
In the next section we describe such a technique.
5.2
One-at-a-Time Design Strategy
Consider the example of the previous section in which a 2 n-k factorial experiment is conducted and the significant variables and probable interactions are determined.
Let m
columns of the orthogonal matrix have more than one probable interaction and let the
number of probable interactions on each of these columns be denoted by n 1 , n2, ... , nm. If
it is assumed that each significant column has only one significant variable or interaction
then there are a total of NT possible models of the process, where NT is given by:
NT = n1 ' n2 . . . nm- 'rim
(5.1)
That is, there are NT models which are completely indistinguishable with respect to the
first block of
2
n-
k
experiments.
Of these NT possible models there is only one model
which corresponds to the true plant.
Hence, under the above assumptions, the problem
of determining the plant is equivalent to determining the correct model from a set of NT
possible models.
To begin with, the confounded probable interactions are rank ordered on the basis of
the significanceof their constituent variables and the significanceof the coefficientsof the
columns on which they lie. In this report the absolute value of the product of the effectsof
the variables and coefficient of the column of the interaction is used as the weight. Therefore,
for a given column, usually interactions in which both the variables are significant, are
ranked above those which have only one significant variable.
The highest ranked probable interaction from every column is selected and a hypothesis
is made that these interactions correspond to the true interactions.
On the basis of this
hypothesis an optimal experiment is designed which would give the highest quality if the
hypothesis is correct. If this experiment has already been carried out then the next high56
2 n-k Factorial
Experiments
P
I
I
!
l
Rank Confounded
Probable Interacations
Figure 5-1: One-at-a-Time Design Strategy
57
est ranked interactions are selected and an optimal experiment is designed. The optimal
experiment is conducted on the actual plant.
Since the new experiment is different from those carried out earlier, not all the NT
possible models predict the same result. The models are ranked in increasing order of their
prediction errors, that is the squared difference between the actual and predicted results.
If two or more models give the same least prediction error, then the one which has higher
ranked interactions is ranked higher. The highest ranked model, that is the model yielding
the lowest prediction error, is selected as the next hypothesis. Once again an experiment
is designed which would give the highest quality if the new hypothesis is correct. If this
experiment has been conducted before, then an experiment is designed such that it is the
optimal experiment for the model which gives the next least prediction error.
Having conducted the second experiment the prediction errors are computed for all the
models and the next hypothesis is made. The same recursive algorithm is implemented until
the hypothesis is verified by the results. Once the hypothesis is verified, ideally no further
experimentation is required since any further experimentation will lead to the selection of
the same hypothesis over and over again. In presence of noise it is advisable to do a few more
experiments to ensure that the results are consistent. The outline of the above procedure
is shown in Figure 5-1.
Example 5-1
* Consider a manufacturing process which depends on 7 variables T1, ... ,T7 and the
true model of the process is given by
y = 20.5 + 3.8T1 + 4.1T 2 + 9.0T 4 - 3.7134 + 2.3113 - 6.1115
(5.2)
A 27-3 factorial fold-over experiment is conducted and the results are analyzed. The
confounding pattern is shown in Table 5.1. From Table 5.2 (a) it is observed that
variables T1, T2 and T4 are significant. Also, the columns 9, 10 and 12 are significant.
The probable interactions lying on the significant columns are:
-
Column 9:
112
and
I34
- Column 10: I24 and I13
- Column 12: 115and I26
58
Column:
1
Interaction:
3
4
5
6
7
8
V1
V2
V3
V123
V4
V124
V134
T1
T2
T3
T4
T5
T6
T7
9
10
11
12
13
14
15
16
V12
V1 3
V23
V14
V24
V34
V1234
V234
112
113
114 I15
116
117
I45
I34
156
I24
I67
I26
I47
I46
I27
157
I23
I37
I25
I35
I36
Vavg
Variable:
Column:
2
Table 5.1: Confounding pattern of the 27-3 factorial experiments of Examples 5-1, 5-2 and
5-3
Thus, assuming that only one of the interactions in each of the columns is significant
results in NT = 8.
The initial guess of the process model is shown in Table 5.2(a). On the basis of this
hypothesis, Experiment 1 is designed. The model shown in Table 5.2(b) gives the least
prediction error and is the next hypothesis. The most significant interaction, I15, is
determined. Experiment 2 is designed to optimize the current hypothesis. Notice that
it is not possible to discriminate between the pairs of interactions (12, 134) and (124,
I13) using the first 2 experiments because both the interactions in the pairs take the
same value in both the experiments.
Since the hypothesis has not changed after the Experiment 2, Experiment 3 is designed
to optimize the next best hypothesis, that is, the model which gives the next least
prediction error. Experiment 4 is designed in the same manner as Experiment 3. By
now the true process model has been determined.
Interestingly, from Table 5.2(f),
we notice that the true optimal operating point of the process, Experiment 5, is the
same as Experiment 3. This means that the true optimal point of the process was
determined prior to the determination of the true process model.
Example 5-2
* Consider a process which is dependent on 7 variables T1, ... , Tn. A 27-2 factorial
experiment is conducted on the process. The experiments are the same as those of
Example 5-1. The confounding pattern of the block is shown in Table 5.1. The results
of the One-at-a-Time strategy are given in Table 5.3.
59
COLUMNWISE SUMMARY:
Col.
1
2
20.04
3.69
20.50
3.80
Avg.
T1
4.10
3
3.87
T2
-0.08
T3
5
9.26
T4
6
7
8
9
10
-0.05
1.09
-0.52
-4.09
1.67
T5
T6
T7
I12
I24
12
13
-5.77
0.67
I26
-----
0.05
-----
0.71
0.52
0.34
14
15
16
True Cf. True(V/I)
Pred(V/I)
Est. Cf.
4
11
COLUMNWISE SUMMARY:
Col.
Avg.
T1
1
2
Est. Cf.
Pred(V/I)
T2
3
-----
4
-0.08
T3
9.00
T4
5
9.26
T4
6
-0.05
T5
-3.70
2.30
------------I34
113
-6.10
I15
-----------------
-------------
3.87
1.09
----9.00
---------
T7
I12
I24
----I34
113
12
13
14
-5.77
I15
0.67
----0.71 -----
-3.70
2.30
-----
-----6.10
-----
0.52
-----
-----
16
0.34
-----
----
EXPERIMENT-WISE SUMARY:
Initial Guess of Process Model.
Expt.
1.
-
+
-
+
-
-
-
29.04
2.50
Pred(V/I)
3.87
T2
4
-0.08
T3
9.26
5
20.50
3.80
4.10
-0.52
-4.09
1.67
0.05
-5.77
T7
I12
124
----115
-------------
-------------
16
0.34
-----
--
-----
-----
13
14
15
-----
Experiment
Prediction
Expt.
Observation
1.
-
+
-
+
-
-
-
29.04
31.54
2.
-
+ -
+
+ +
-
42.68
42.51
RMSProd. Error:
Experiment
-3.70
2.30
-6.10
----134
113
---115
Prediction
Observation
1.
2.
- +- +- - - + - + ++ -
29.04
42.68
31.54
42.51
3.
+
41.96
43.86
+
-
+
-
RMS Pred. Error:
1.77
(c) Results after
0.67
0.71
0.52
-----
EXPZRIMENT-WISE SUMMARY:
EXPERIMZENT-WISESUMMARY:
Expt.
T4
---------
8
9
10
11
12
0.71 ---------
9.00
T3
T4
T5
T6
-6.10 I15
0.52
T1
T2
9.26
-0.05
1.09
I15
-----
T1
T2
Avg.
-0.08
------------134
113
3.69
3.87
3.80
4.10
5
6
7
-5.77
0.67
0.34
20.50
4
12
13
16
True Cf. True(V/I)
-----
T5
T6
T7
I12
I24
-----
-3.70
2.30
Pred(V/I)
Avg.
T2
9.00 T4
T4
20.04
2
3
-0.05
1.09
-0.52
-4.09
1.67
0.05
15
1
Avg.
T1
6
7
8
9
10
11
14
Col. Est. Cf.
True Cf. True(V/I)
Avg.
T1
3
Experiment 1
COLUMNWISE SUMMARY:
COLUMNWISE SUMMARY:
20.04
3.69
31.54
(b) Results after
(a) Results after Experiment 0
Observation
Prediction
Experiment
RMSPred. Error:
1
2
I15
15
EXPERIMENT-WISE SUMMARY:
Col. Est. Cf.
T4
T6
-0.52
-4.09
1.67
0.05
Avg.
T1
4.10 T2
T2
8
9
10
11
20.50
3.80
Avg.
T1
20.04
3.69
7
True Cf. True(V/I)
+ -
1.81
(d) Results after Experiment 3
Experiment 2
Table 5.2: Results of One-at-a-Time Designs of Example 5-1
60
COLIWISZ SIRY:
COLD1fWISE SUMKARY
Col.
Est. Cf. Pred(V/I)
True Cf. True(V/I)
Avg.
T1
1
2
20.04
3.69
3
3.87
T2
4
-0.08
T3
5
9.26
T4
20.50
3.80
4.10
9.00
Col.
Avg.
T1
1
-----
4
-0.08
T3
T4
5
9.26
T4
-0.05
T5
-0.05
T5
-----
6
1.09
-0.52
-4.09
T6
T7
134
--------134
7
10
1.67
113
11
0.05
-----
12
-5.77
115
13
14
15
0.67
0.71
0.52
16
-----
EXPERIMENT-WISE
Expt.
1.
-6.10
-------------
0.34
I13
----115
----------------
Prediction
Observation
-
+
-
+
-
-
-
29.04
31.54
2.
-
+
-
+
+
+
-
42.68
42.51
3.
4.
+
-
+
+
- + + + +
+
+
-
46.80
30.98
43.86
29.36
RMSPred. Error:
20.50 Avg.
T1
T2
1.09
3.80 T1
4.10
T2
----9.00
-----
T6
-0.52
-4.09
1.67
0.05
-5.77
T7
134
I13
-__115
16
0.34
-----
0.67
0.71
0.52
T4
-----3.70
2.30
-6.10
-------------
----134
I13
----115
----------------
ZXPERIMENT-WISE SUMMARY:
SUIUVARYs
Experiment
3.69
3.87
8
9
10
11
12
13
14
15
True Cf. True(V/I)
20.04 Avg.
T2
6
2.30
Pred(V/I)
2
3
7
8
9
-3.70
st. Cf.
Expt.
Prediction
-
+
-
+
-
-
-
29.04
31.54
3.
+
+
-
+
-
+
-
46.80
43.86
4.
-
+
+
++ +
-
30.98
29.36
5.
+
+
+
-
-
46.80
43.64
- + - ++ + -
+
42.68
(f)
Results
after Experiment 5
Table 5.2: Results of One-at-a-Time Designs of Example 5-1
61
42.51
2.35
RMS Pred. Error:
(e) Results after Experiment4
Observation
1.
2.
2.09
Experiment
The example indicates that the more significant interactions are determined before the
others. Also, the example demonstrates that the procedure determines the optimal
operating point of the process before it determines the true process model..
Example 5-3
* This example is a variant of Example 5-2 in which the crucial assumption that there
is only one significant variable or interaction per column of the design matrix, has
been violated. In Example 5-2, Column 9 has one significant interaction I34. In this
example the true plant model has two significant interactions I56 and 134 on Column
9. The coefficients of the two interactions are chosen to be effectively the same as
that of 134 in Example 5-2. Therefore, the results of the first block of experiments are
identical in both these examples. When the One-at-a-Time strategy is applied to the
plant the output behavior is very interesting. The results are shown in Table 5.4. In
the first few experiments, neither of the two interactions in Column 9 are selected by
the algorithm. But after Experiment 4, I34 is selected. Although the true model of
the plant cannot be determined, the experiments tend to improve the output quality.
Also, the interactions in the rest of the columns are determined correctly.
As expected, the RMS prediction error for this example is much larger than that for
the previous examples. The large RMS prediction error suggests that there may be
something wrong with the plant model that has been determined.
5.3 Mathematical Justification
In this section we give a justification of the One-at-a-Time strategy introduced in this
chapter. We show that under certain assumptions, the model which gives the least prediction
error is the Maximum Likelihood estimate of the the model of the plant.
Claim:
Suppose T One-at-a-Time experiments are conducted and the results are Y 1, ... , YT. If
the noise in these experiments is independent and gaussian distributed with zero mean and
constant variance A, then the model which minimizes the squared prediction error is the
Maximum LikelihoodEstimate of the plant.
62
COLUMNWISESMMARYs
COLUMNWISE SUMMARY:
Col. ESt. Cf.
Pred(V/I)
1
2
3
4
5
6
7
8
9
10
Avg.
T1
T2
T3
T4
T5
T6
T7
I12
113
22.51
5.45
6.24
2.95
0.72
-4.19
-0.53
0.31
-3.54
2.84
11
0.47 -----
12
0.26 -----
13
-4.10
135
15
16
-0.35
-0.46
---------
EXPERIMENT-WISE
1
2
3
22.50
5.30
7.30
3.10
Avg.
T1
T2
T3
-----4.90 T5
---------3.70 I34
2.90 I57
4
5
6
7
8
9
10
11
12
13
-----
0.26 -----
14
Col. Est. Cf.
True Cf. True(V/I)
-----4.30
14
15
16
117
----
Pred(V/I)
22.51 Avg.
5.45 T1
6.24 T2
2.95
0.72
22.50 Avg.
5.30 T1
7.30 T2
T3
T4
3.10
-4.19 T5
-0.53
0.31
-3.54
2.84
-4.90 T5
T6
T7
I12
I57
-3.70
2.90
0.47 ----0.26 ----0.26 -----4.10
-0.35
-0.46
Expt.
1.
I17
--------
Experiment
+ + + + - -+
-4.30
Prediction
32.41
True Cf. True(V/I)
Avg.
T1
T2
T3
22.50
5.30
7.30
3.10
T4
6
-4.19 T5
7
8
9
-0.53
0.31
-3.54
T6
T7
I12
-3.70
2.84
157
2.90
10
11
12
13
-4.10 117
15
-0.35
16
Col. Est. Cf.
-4.30
-----
1
2
3
4
-----
5
0.72
T4
--------134
6
7
8
9
-4.19
-0.53
0.31
-3.54
T5
T6
T7
156
157
10
------------I17
----
-0.46 -----
-----
EXPERIMENT-WISE SUMMARY:
Expt.
1.
2.
Experiment
++ + + - - +
+ ++ * - - -
RMS Pred. Error:
Pred(V/I)
Avg.
T1
T2
T3
-4.90 T5
0.47 ----0.26 ----0.26 -----
14
29.42
COLUMNWISE SUMMARY:
Et. Cf. Pred(V/I)
0.72
Observation
(b) Results after Experiment 1
COLUMNWISE SUMMARY:
5
117
----
3.00
(a) Results afterExperiment 0
22.51
5.45
6.24
2.95
--------134
157
-------------
RMSPred. Error:
1
2
3
4
T3
-----
ZXPfRIMENT-WISE SUMMARY:
UMMARY:
Initial Guess of Process Model.
Col.
True Cf. True(V/I)
22.51
5.45
6.24
2.95
Avg.
T1
T2
T3
-----4.90 T5
---------3.70
134
2.84
157
11
0.47
-----
----
12
13
14
15
16
0.26
0.26
-4.10
-0.35
-0.46
--------I17
---------
--------I17
----
Prediction
Observation
Expt.
32.41
45.67
29.42
45.98
1.
2.
3.
2.90
-4.30
I57
SUMMARY:
Experiment
+ + ++ - - +
+ ++ + - - + ++ + - +-
RMS Pred. Error:
(c) Results after Experiment2
22.50
5.30
7.30
3.10
Avg.
T1
T2
T3
EXPERIMENT-WISE
2.13
True Cf. True(V/I)
Prediction
Observation
32.41
45.67
51.70
29.42
45.98
48.82
2.41
(d) Results after Experiment 3
Table 5.3: Results of One-at-a-Time Designs of Example 5-2
63
COLUMNWISE SUMMARYI
COLUMNWISE SUMMARY:
Col.
True Cf. True(V/I)
Pred(V/I)
Est. Cf.
22.50
5.30
Avg.
T1
1
2
22.51
5.45
3
4
6.24
2.95
T2
T3
5
6
7
8
0.72
-4.19
-0.53
0.31
T4
T5
T6
T7
9
-3.54
134
10
11
12
13
14
15
16
2.84
0.47
0.26
0.26
-4.10
-0.35
-0.46
I57
------------I17
-__-----
7.30
3.10
-4.90
-3.70
2.90
-4.30
Col.
Avg.
T1
T2
T3
----T5
--------134
157
----
7
8
9
10
11
-0.53
0.31
-3.54
2.84
0.47
T6
T7
I34
157
-----
+
+
-
-
+
-
32.41
45.67
29.42
45.98
3.
+
+ +
+
+ +
-
+
-
-
44.61
4.
+
-
48.82
53.66
51.32
Expt.
Observation
+ + +
+ + +
1
-----4.30
Prediction
Experiment
Pred(V/I)
Est. Cf.
5.45
3
4
6.24 T2
2.95 T3
+
+ +
+
-
-
+
32.41
+
+ +
+
-
-
-
45.67
45.98
3.
+
+
+ +
-
-
+
-
-
44.61
4.
+
+
-
51.32
48.82
53.66
5.
+ +
+ -
-
-
46.85
49.79
-
2.86
22.50 Avg.
T1
5.30
T1
7.30 T2
3.10 T3
T4
T5
T6
T7
-4.90
----T5
---------
-3.54 134
2.84 157
-3.70 134
2.90 157
11
12
13
14
15
16
0.47
0.26
0.26
-4.10
-0.35
-0.46
------------I17
---------
--------117
----
-4.30
..................---------------------------......
EXPERIMENT-WISE
SUMMARY:
Prediction
Observation
Experiment
..---.---------------------------------------....
Expt.
2.
3.
+ +
+ +
+
+
+
29.42
(f) Results after Experiment5
Experiment 4
9
10
1.
Observation
1.
True Cf. True(V/I)
22.51 Avg.
0.72
-4.19
-0.53
0.31
117
----
2.
SUMMARY:
2
5
6
7
8
-3.70
2.90
--------134
I57
----
2.84
(e) Results after
Col.
Avg.
Ti
T2
T3
-----4.90 T5
22.50
5.30
7.30
3.10
----I17
---------
0.26
-4.10
-0.35
-0.46
RMS Pred. Error:
COLUNNWISE
True Cf. True(V/I)
0.26 -----
13
14
15
16
117
-----
1.
2.
RMSPred. Error:
Avg.
T1
T2
T3
T4
T5
EXPERIMENT-WISE SUMMARY:
Prediction
Experiment
22.51
5.45
6.24
2.95
0.72
-4.19
12
EXPERIMENT-WISE SUMMARY:
Expt.
Est. Cf. Pred(V/I)
1
2
3
4
5
6
-
-
+
32.41
-
-
45.67
29.42
45.98
48.82
+ +
+
+
-
+
-
44.61
4.
+ +
+
-
-
-
-
51.32
53.66
5.
6.
+ + + + +
-
-
-
-
46.85
51.32
49.79
54.93
RMS Pred. Error:
3.00
(g) Results after Experiment6
Table 5.3: Results of One-at-a-Time Designs of Example 5-2
64
COLUMN-WISE SUMMARY:
Col. Est. Cf.
Pred(V/I)
1
2
3
4
22.51
5.45
6.24
2.95
5
0.72
6
COLUMN-WISE
22.50
5.30
7.30
3.10
T4
-4.19 T5
T6
T7
I12
113
-----
13
0.26
-----
4.70,
-8.40
2.90
0.26 -----
14
-4.10
135
15
-0.35
-----
16
-------
-4.30
---
True Cf. True(V/I)
22.50 Avg.
5.30 T1
7.30
T2
T3
T4
3.10
-4.90
-4.19 T5
7
8
9
10
11
12
-0.53
0.31
-3.54
2.84
0.47
0.26
T6
T7
112
I57
---------
14
15
16
-4.10
-0.35
-0.46
I17
---------
13
117
----
-0.46 -----
2.95
0.72
6
--------I56, 134
157
----
Pred(V/I)
22.51 Avg.
5.45 T1
6.24 T2
4
5
-4.90 T5
-0.53
0.31
-3.54
2.84
0.47
Est. Cf.
1
2
3
Avg.
T1
T2
T3
-----
7
8
9
10
11
12
Col.
True Cf. True(V/I)
Avg.
T1
T2
T3
SUMWARY:
4.70,
0.26 -----
EXPERIMENT-WISE SUMMARY:
Initial Guess of Process Model.
Zxpt.
Experiment
+
+
+
-
-
Prediction
+
32.41
RMSPred. Error:
(a) Results after
COLUDM-WISZ
5
6
7
8
9
10
11
12
13
22.51
5.45
6.24
2.95
Avg.
T1
T2
T3
0.72
-4.19
-0.53
0.31
-3.54
2.84
T4
T5
T6
T7
I12
I57
-4.10
-0.35
16
Col.
True Cf. True(V/I)
22.50
5.30
7.30
3.10
-4.90
4.70,
-8.40
2.90
-4.30
117
-----
-0.46 -----
Avg.
T1
T2
T3
----T5
--------156, 134
157
-------------
1.
+ +
2.
+
+
+ +
+
Est. Cf. Pred(V/I)
1
2
3
4
22.51
5.45
6.24
2.95
Avg.
T1
T2
T3
5
6
7
8
9
10
0.72
-4.19
-0.53
0.31
-3.54
2.84
T4
T5
T6
T7
I12
157
0.26
-----
-4.10
-0.35
-0.46
I17
---------
11
34.43
13
I17
----
True Cf. True(V/I)
22.50
5.30
7.30
3.10
-4.90
4.70,
0.47 -----
12
-8.40
2.90
0.26 -----
14
15
16
-----
Experiment
Observation
(b) Results after Experiment1
EXPERIMENT-WISE SUMMARY:
Expt.
117
----
COLDN-WISE SUMMARY:
0.47 ----0.26 ----0.26 -----
14
15
--------156, 134
157
----
2.02
Experiment 0
SUMMARY:
Col. Est. Cf. Pred(V/I)
1
2
3
4
T5
-4.30
EXPERIMENT-WISE SUMMARY:
1.
-8.40
2.90
T3
-----
Avg.
T1
T2
T3
----T5
--------I56, 134
I57
---------
-4.30
117
----
XPZRIMZNT-WISE SUMMARY:
Prediction
Expt.
Observation
Experiment
Prediction
Observation
-
-
+
32.41
34.43
1.
+
+
-
- +
32.41
34.43
-
-
-
45.67
45.89
2.
+
+ +
+
-
-
-
45.67
45.89
3.
+
+
+
-
+
-
44.61
37.78
1.44
RMS Pred. Error:
(c) Results
+ +
RMS Pred. Error:
after Experiment2
4.12
(d) Results after Experiment 3
Table 5.4: Results of One-at-a-Time Designs of Example 5-3
65
COLUMN-WISE
Col.
Eat.Cf.
3
4
6.24
2.95
T2
T3
5
0.72
T4
6
-4.19
T5
7
8
9
10
-0.53
0.31
-3.54
2.84
T6
T7
134
I57
12
0.26
-----
-4.10
-0.35
117
-----
13
16
22.50
5.30
Avg.
T
22.51
5.45
14
15
True Cf. True(V/I)
Pred(V/I)
1
2
11
COLUMN-WISE
SUMMARY:
7.30
3.10
-4.90
4.70, -8.40
2.90
0.47 -----
Col.
-0.46 -----
22.51
5.45
T2
T3
3
4
6.24
2.95
T2
T3
-----
5
0.72
T4
T5
6
-4.19
T5
--------156, I34
157
7
8
9
-0.53
0.31
-3.54
T6
T7
134
4.70, -8.40
10
2.84
157
2.90
11
12
0.47 ----0.26 -----
-----
13
14
15
16
117
----
-----
EXPZRIMENT-WISE SUMMARY:
Expt.
Experiment
1.
2.
3.
4.
+ ++ + ++ + + + ++ + ++ + - -
- +- -
0.26
-4.10
-0.35
-0.46
EXPERIMENT-WISE
22.59 Avg.
5.30 T1
Avg.
T1
7.30
3.10
T2
T3
-4.90
T5
-----
Expt.
Experiment
32.41
45.67
44.61
51.32
34.43
45.89
37.78
66.29
1.
2.
3.
4.
5.
+ +. + + ++ + + ++ + + - ++- +-
-4.30
-+
- +- --
Prediction
Observation
32.41
45.67
44.61
51.32
46.85
34.43
45.89
37.78
66.29
59.33
9.28
(f) Results after
Experiment 4
Experiment 5
COLUMN-WISESUMMARY:
Pred(V/I)
True Cf. True(V/I)
...................................................
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
22.51
5.45
6.24
2.95
0.72
-4.19
-0.53
0.31
-3.54
2.84
0.47
0.26
0.26
-4.10
-0.35
-0.46
Avg.
T1
T2
T3
T4
T5
T6
T7
134
22.50
5.30
7.30
3.10
Avg.
T1
T2
T3
-4.90
T5
4.70, -8.40
2.90
I57
I56, 134
I57
.....
I17
-4.30
117
.....
.....
EXPERIZMENT-WISE
SUMMARY:
Expt.
Experiment
1.
2.
3.
4.
5.
6.
++ ++ + + ++ ++ ++ ++ + - ++ - +++ + - -
RMSPred. Error:
117
----
8.29
(e) Results after
Est. Cf.
157
SUNMARY:
Observation
RMSPred. Error:
Col.
--------156, 134
---------
----I17
---------
Prediction
RMSPred. Error:
True Cf. True(V/I)
Pred(V/I)
1
2
-----4.30
Est. Cf.
Avg.
Ti
--
0.26 -----
SUMMARY:
-+
- +- - - -
Prediction
Observation
32.41
45.67
44.61
51.32
46.85
51.32
34.43
45.89
37.78
66.29
59.33
63.59
9.84
(g) Results after
Experiment 6
Table 5.4: Results of One-at-a-Time Designs of Example 5-3
66
Proof:
Consider the model discussed in the previous section.
Let the NT possible models be
denoted by M1, ... , MNT
In the One-at-a-Time design methodology it is assumed that the coefficients of the
variables and interactions have been determined and only the confounded interactions need
to be determined.
Given the T observations, let YT= [Y1 , ... , YT]. The Maximum Likelihood Estimate is
defined by,
MML[YT]= argmax
P(Mi; YT)
i
=
MML[YT]
= argmax P(Mi) P(YT/Mi)
(5.3)
(5.4)
In case the probabilities of the different models P(Mi), are known, Equation 5.4 can be
simplified.. In absence of any such information, all models can be assumed to be equally
likely. That is, P(Mi) = 1/NT, and Equation 5.4 reduces to,
MML[YT]= argmax P(YT/Mi)
(5.5)
Let the predicted result of model Mi for the jth experiment be denoted by Yij. Since the
noise in the different experiments is assumed to be independent,
T
P(YT/Mi) =
l fe(Yj - Yij)
j=1
(5.6)
where fe() is the probability distribution of the noise corrupting the results. If this distribution is assumed to be gaussian with zero mean and constant variance A, then
P(YT/Mi) =
T
1
(1(1,
A
-
j=l
exp
(i
1
yj _ Yj) 2
(5.7)
T
T/2 .exp -2L(Y - j)2
(27rA)
(5.8)
Maximizing the likelihood function in Equation 5.5 is the same as maximizing its logarithm.
Thus.
MML[YT]= argmax logP(YT/Mi)
i
67
(5.9)
Using Equation 5.8 in Equation 5.9 we get,
MML[YT] = arg max log (2
)
T/2
-exp
T
L(2irA)
/
exp
argmax - log2(rA-#2-
j2
(5.10)
)j=
1
(Y,-Ij)2
(5.11)
1
argmin
= argmin
--
(Yj
Y
)2]]
j)2]
(5.12)
(5.13)
The expression on the right hand side of the equation is the total squared prediction error
of model Mi. Thus, the Maximum Likelihood Estimate of the plant, MML, is one which
minimizes the total prediction error. This proves the claim.
The One-at-a-Time strategy selects models based on minimum prediction error. Hence,
under the above assumptions, the results obtained using One-at-a-Time strategy converge
to the maximum likelihood estimate of the plant. The most critical assumption is that there
is only one unknown significant interaction per significant column of the orthogonal matrix.
5.4
Advantages of One-at-a-Time Design Strategy
The One-at-a-Time design strategy can be very useful particularly when the number of
confounded probable interactions is small. The main advantages of the One-at-a-Time
design strategy are:
1. Maximum Likelihood Estimate: The One-at-a-Time strategy uses the minimum
prediction error criterion to hypothesize the plant model. This criterion is equivalent
to the Maximum Likelihoodestimate under the assumptions discussedin the previous
section. These assumptions are justified in most situations.
2. Designing Optimal Experiment: Given the hypothesized plant model, the Oneat-a-Time strategy designs the optimal experiment for this hypothesis, i.e. the experiment which would maximize the quality of the output if the hypothesis is correct.
This leads to interesting behavior.
68
* In case the hypothesis is wrong and some very significant interactions have been
guessed wrongly, the optimal experiment for the hypothesis tends to produce a
very poor output. Although this experiment is far away from the true optimal,
it can be very useful. The prediction errors for such an experiment tend to be
very large for the wrong models. Hence, this experiment helps in discriminating
between different models.
* If the hypothesis is correct, the optimal experiment will produce a very good
quality output. The prediction error of the correct model will be much smaller
compared to the wrong model and the hypothesis will be verified.
3. Validation
of Assumptions:
When a block of factorial experiments is analyzed
some of the columns have large coefficients which correspond to the effects of significant variables and interactions of the plant. The non-significant coefficients can be
used to determine the variance of the plant.
Suppose that the One-at-a-Time design strategy is applied and the most probable
plant model is determined.
If it is found that the average prediction error is very
large, it could be due to the the violations of one or more assumptions made above.
In such a situation it is desirable that sequential block design strategy should be used
to determine the correct plant model.
4. Determination of the Most Significant Interactions: Suppose that there are
two columns, one of which has a large significant coefficient and the other has a
small significant coefficient. Suppose that each of these columns contain two probable
interactions. It is clear that models which contain the wrong probable interaction in
the column with the larger coefficient will give larger prediction errors than models
which have the wrong probable interaction in the column with smaller coefficient.
In the simulations it is observed that after a few One-at-a-Time experiments have been
carried out, all models having wrong probable interactions in the significant columns
with large coefficients give prediction errors which are more than an order of magnitude
greater than the error of the correct model. Hence, the probable interactions lying
on such columns can be determined within a few One-at-a-Time experiments and the
number of possible models is then considerably reduced.
69
Column:
Vavg
Variable
& Interaction:
Coefficient:
10.6
Vl
V2
V3
V1 2 3
V1 2
V23
V13
T1
T2
T3
T4
112
134
123
I13
114
124
7.2
0.1
-6.5
-0.2
-3.1
-0.3
2.3
Table 5.5: Results of Design of Example 5-4
It is shown in the next section that not all the unknown probable interactions need
to be determined to determine the optimal operating point of the true plant. Hence,
the One-at-a-Time strategy needs to be implemented only until the very significant
probable interactions have been determined.
5.5 Partial Optimization
The goal of the DOE procedure is to determine the plant model. But in many cases only
the optimal operating point, that is the operating point of the plant which produces the
best quality, needs to be determined. Hence, in such cases, the sole purpose of determining
the plant model is to determine the optimal operating point. In this section we will demonstrate that it is not always necessary to determine the complete plant model to determine
the optimal operating point. It may be possible to determine it only on the basis of the
significant variables and the known significant interactions.
Example 5-4
* Consider a process dependent on 4 variables T 1, T2 , T3 and T 4. Suppose a 24-1 factorial
experiment is conducted on this process and the results are as shown in Table 5.5.
Using Table 5.5, we can express the model of the process as
Y= 10.6 + 7.2T1 - 6.5T3 - 3.1
( 12 ) + 2.3
I34
1243
(5.14)
I24
where only one of the interactions expressed in (.) is actually significant. If the terms
involving T are arranged together, the model can be expressed as,
Y= 10.6 + T1 [7.5 + (-3.1T2) + (2.3T3 )] + terms not involving T1 ....
70
(5.15)
The terms within (.) might not be present in the true model. Since, 17.51> I - 3.11+
12.31,irrespective of which combination of interactions is really important, the quality
Y is maximized only if T1 is set at +1. Therefore the optimal setting of T1 is +1.
Similarly,
Y= 10.6 + T3 [-6.5 + (-3.1T4) + (2.3T1 )] + terms not involvingT3 ....
(5.16)
Since, - 6.51 > I - 3.11+ 12.31,the optimal setting for T3 is -1 irrespective of which
interactions are really important. It is not possible to determine the optimal settings
of T2 and T4 since their settings are dependent on the unknown interactions.
It is clear from this example that it may be possible to determine the optimal settings
of some (if not all) of the variables without knowing the exact plant model. In the above
example the optimal settings of the variables T1 and T3 could be determined because their
coefficients were dominant. The general analysis for such cases is given below.
1. Dominant Variable:
Consider a process in which a
2
n- k
factorial fold-over experiment has been carried
out. The results are analyzed and the model of the process is determined.
Y=30
+lT+-...+
nTn+
-+ijIij
+-l ^ + -+m
/~1
(5.17)
mpr
where /3i is the known coefficient of the ith variable, Pij is the known coefficient of the
known probable interaction Iij, and yk is the known coefficient of the kth significant
column with an unknown probable interaction. The probable interactions lying on
the kth column are ikl,
... , ikpk
Let Di be the set of all the columns which have one unknown probable interaction of
Ti. Namely, if keDi, then the columns contain Ii.g(i,k),the interaction between Ti and
Tg(i,k). The optimal setting of variable Ti can be determined if the expression
/3i+
ji
/3ijTj
,
+ E YkTg(i,k)
keDi
(5.18)
has a known sign irrespective of the unknown setting of the other variables.
If the setting of all the variables are unknown then the condition in Equation 5.18 can
71
be restated as,
IAI> Z4 Iiji +
j7 i
E hI
(5.19)
kcDi
If Equation 5.19 is true, the optimal setting of Ti is +1 if /i is positive and is -1 if
f3iis negative. Thus if a variable is dominant, its optimal setting can be determined
without the complete knowledge of the process.
2. Dominant Pair of Variables:
In many situations Equation 5.19 cannot be applied because one or more known
significant interactions are large. That is, one or more of the coefficients f 3ij are large.
In such situations it may be possible to determine the optimal settings of variables
taken two at a time. This procedure is explained below.
Suppose that /3ij is large. In the expression for quality given in Equation 5.17, the
terms containing Ti and Tj are separated as follows:
Y = Ti[Pi + interactions involving Ti]
+PijIij + Tj[3j + interactions involvingTj]
+ terms not involving Ti and Tj
(5.20)
Suppose that the terms corresponding to the coefficients of Ti and Tj in Equation 5.20
satisfy Equation 5.19. The range of these terms is,
\
17i,j
10i
keDi
kcDi
/mij
I
i,j
il 1: keDi
keDi
)
(5.21)
Rj
in,
ax] = [(1j
I
E
)
3iI
10j,'i
kcDj
,
\
+
IjlI+
1/ j,i
Z
keDj
1
I)
(5.22)
It can be shown that the optimal settings of Ti and Tj can be determined only if the
ranges Ri and Rj do not intersect and if 3ij lies in the intervals 1, 3 or 5 of Figure 5-2.
In case any of the above conditions are not satisfied, the optimal setting of Ti and Tj
72
2
i
Odiin
5
imax
ilin
0i
0j
in
Ej
ax
Figure 5-2: Permissible range of values of 3ij
cannot be determined. Further experiments are required to determine more unknown
probable interactions before the settings of Ti and Tj can be determined.
3. Dominant Group of Three or more Variables:
In principle it is possible to analyze groups of 3 or more variables to determine their
optimal setting. But the procedure becomes very complicated. Also the conditions
required to be able to determine the settings are fairly strong and will not be satisfied
very often.
Thus it is seen that if the plant contains some dominant variables or pairs of variables, then
their optimization may be possible without the complete determination of the interactions.
The setting of the other variables can be obtained by further experimentation. Usually the
variables that cannot be set easily using this technique are ones which are associated with
weak interactions and hence they do not affect the quality very much.
73
74
Chapter 6
Conclusions and Future Work
In this chapter the basic ideas of the thesis are integrated. Section 1 discusses the Complete
DOE Software Tool which is being currently developed at MIT. Section 2 outlines the
contributions of the thesis. The possible areas for future research are discussed in Section 3.
6.1
The Complete DOE Software Tool
In this section we show how the different ideas of Block Design and One-at-a-Time Design
are being integrated to develop a Complete DOE Software Tool. The underlying motivation
for developing such a DOE software tool is that it should aid an inexperienced user in DOE
to successfully improve the performance of his/her plant more rapidly than with existing
software tools.
Consider the flow-chart shown in Figure 6-1. Given a plant, an experimenter has to
select the variables affecting the performance and guess the important interactions. This
permits the experimenter to use the past knowledge about the plant which may be very
important.
Even if some of the interactions are guessed incorrectly, a few more experi-
ments will generally be required to estimate them, but nothing is lost. On the basis of this
information, an optimal fold-over factorial block of experiment is designed. Next, the experiment results are analyzed and the effects of the variables and the guessed interactions are
determined. The variables with significant effects are used to generate the list of probable
interactions - in accordance with the Simplicity Principle.
Usually there is confounding within the set of probable interactions and guessed interactions. Therefore further experimentation is required to determine the effects of all these
75
No
Figure 6-1: Overview of Complete DOE Software Tool
76
interactions. At this stage there are 2 possibilities.
Case 1: The number of significant columns is small compared to the number of unknown
interactions and it is reasonable to assume that there is only one unknown significant
interaction per significant column of the design matrix.
Case 2: The number of significant columns is not small and there may be more than one
unknown significant interaction per significant column of the design matrix.
If Case 1 is valid for the given plant, the assumptions of the One-at-a-Time Design
Strategy discussed in Chapter 5 are satisfied, and the One-at-a-Time Technique can be
effectively used to determine the plant model and optimize its performance.
If Case 1 is not valid, the Block Design Technique described in Chapter 3 can be used.
With the help of Half-Block or Full-Block Designs the confounding of the interactions can
be sorted either completely or to an extent that the One-at-a-Time Design Strategy can be
applied.
Presently, we are implementing the Complete DOE Software Tool. Much of the work has
been completed. The basic concepts of Block Design and One-at-a-Time Design Strategies
have been implemented and tested separately. Also, the analysis tools have been developed.
Techniques required to simultaneously analyze block designs with different confounding
patterns have been developed and implemented in computer software.
Some of the concepts that can be added to make the software more automated and less
dependent on the user are discussed in Section 3.
6.2
Thesis Achievements
The goal of the DOE project at MIT is to automate the entire process of DOE. At the time
this thesis began, the goal was well defined but the path to achieve it was a little unclear.
During the course of the thesis, the process of sequential DOE has been formulated in a
structured
fashion. We have identified the conceptual problems that need to be solved
before the goal of the Complete DOE Software Tool can be completely realized. Some of
these problems have been resolved and implemented. As for the other problems, they have
been formulated in a mathematically meaningful manner. Even if these problems are hard
to tackle, we know exactly what questions need to be addressed in the overall project.
77
The contributions of the thesis are enumerated below.
Block Designs:
The techniques of Half-Block and Full-Block Designs are implemented in
computer software. They have been found to be very useful in resolving confounding
interactions.
Completing
Block Designs: Algorithm for completing incomplete blocks of experiments
have been developed and added to the software.
Block Analysis:
The analysis routines of the DOE software have been improved to enable
it to simultaneously analyze confounding interactions in different blocks of experiments.
One-at-a-Time
Designs: The concept of One-at-a-Time Designs have been developed
and verified by simulation results. The results are very encouraging and we believe
that the One-at-a-Time Design Strategy can be generalized as explained in the next
section.
Complete DOE Software Tool: The basic ideas for the Complete DOE Software Tool
are developed and their implementation is being carried out.
6.3
Future Work
The following represents some of the more significant issues that could be added to the
computer software to make it more general and useful to various manufacturing processes.
* As seen in Figure 6-1 the Complete DOE Software Tool needs to make a decision whether only one unknown significant interaction per column is likely or not.
Presently, this decision is made by the experimenter using the software. It may be
possible to determine the probability of there being only one unknown significant
interaction per column. Using this probability and the level of risk that the experimenter is willing to take, the computer can make that decision which minimizes the
expected value of the overall cost of experimentation.
* Once the Block Design option is selected the experimenter has the following options:
1. Design a Half-Block of experiments for any of the previous blocks.
78
2. Design a Full-Block of experiments for any of the previous blocks.
3. Complete any of the previous incomplete blocks.
Therefore as the number of blocks of experiments conducted increases there are more
options. If the cost model for the experiments is known, the software could choose
between the above options based on:
- Total number of confoundedprobable interactions.
- Relative unconfounding achieved with each of the options.
* The present software can be used with any plant in which the variables are at 2 levels.
The two-level factorial designs have many strong advantages as outlined in Chapter 1.
But there may be situations in which the plant intrinsically has one or more of its
variables at 3 or more levels. Extension of the software is required to handle such
situations.
* It should be possible to extend the ideas of One-at-a-Time Design Strategy to be
able to design arbitrary numbers of optimal experiments based on the results of the
experiments conducted earlier.
* A Graphical User Interface (GUI) can be added to the software to make its use
easier. Also, adding various graphical features can help the experimenter get a better
understanding of the results.
79
80
Appendix A
Binary Orthogonal Matrices
In this section we enumerate some of the properties of Binary Orthogonal Matrices which
are required in the report.
The Binary Orthogonal matrices belong to a general class of matrices called Hadamard
Matrices [19], matrices in which each of the elements are either +1 or -1. The most general
Binary Orthogonal Matrices are the Plackett-Burman
Designs. These designs are based
on the Hadamard Matrices of size m x m where m is a multiple of 4. In the PlackettBurman Designs all the columns of the matrices are balanced (equal number of +1 and -1
elements) and pairwise orthogonal. When m is not a power of two, these designs have a
very complicated alias structure and should be used very carefully. A detailed discussion of
these designs is given in [19].
Orthogonal Binary Matrices in which m is a power of two are widely used and have very
useful properties. Hence only Binary Orthogonal Matrices in which m = 2n for some n, are
studied in the report. For simplicity of terminology the term Orthogonal Matrix is used to
denote such a Binary Orthogonal Matrix, as defined below.
A.1
Definitions
Orthogonal
Matrix:
An Orthogonal Matrix, M, of size 2nx2n is defined as a matrix
whose elements are either +1 or -1 and which satisfies the condition MTM = 2n xI, where
I is an identity matrix of size 2n x2 n . Table A.1 is an example of an Orthogonal Matrix of
size 23x2 3 .
81
Table A.1: 3 x 3 Binary Orthogonal Matrix
Dependent
Column:
A column ci is said to be dependent on a set of columns cj, ck,
... , cp if and only if ci is identically equal to the column obtained by taking an element-byelement product of the columns of cj,
Independent
ck,
Cp.
... ,
Column: A column ci is said to be independent of a set of columns cj, ck,
... , cp if and only if ci is not dependent on any group of columns taken from the set of
columns cj, Ck, ... , cp.
Independent
Set of Columns: A set of columns cj, ck, ... , cp are independent
if none
of the columns within the set is dependent on any group of other columns of the set.
Column c4 in Table A.1 is dependent on columns cl and c2, since column
C4
corresponds
to the column formed by the product of the column cl and column c2. Also, columns cl, c2
and C3form an independent set of columns.
A.2
Properties of Binary Orthogonal Matrices
Given below are some of the properties of an 2nx 2n binary orthogonal matrix, M.
1. One of the columns of the orthogonal matrix consists of only +1 elements and is called
the Average Column. This column is denoted as Cavgin Table A.1.
2. Every column other than the Average column has equal numbers of +1 and -1 ele-
ments.
82
3. A product of any two columns of the matrix equals a column of the matrix. That is,
the product of column ci and cj must equal ck for some k.
Cj
Ci
(A.1)
k
Using the fact that,
ci . ci = Cavg
Vi
(A.2)
it can be seen that,
Ci' cj = Ck
:='
CiCk =Cj
C Ck = Ci
Cj
"-
4. Any set of n independent columns can be used to generate the entire matrix. Hence,
any set of n independent set of columns completely characterizes the entire matrix.
Such a set of columns is defined as a set of basic columns.
The other columns are
called non-basic columns. The set of basic columns is not unique.
5. Every 2n x2 n binary orthogonal matrix, M, has n basic column and 2n-n non-basic
columns.
6. Any set of n basic columns is such that every row of these columns corresponds to a
unique combination of +1 and -1. There are two possible options for every element
in a row. Therefore, there are a total of 2n possible combinations, each corresponding
to a different row of the orthogonal matrix.
83
84
Appendix B
Designing a Half-Block of
Experiments
In this section we will try to illustrate the mechanism for shifting variables and interactions
from one column to another when an additional block of experiments is carried out to
augment the existing one.
Let us formulate the problem in a general manner.
Given
a block of 2 n-k factorial experiments, is it possible to replace some of the experiments
by other experiments, and obtain a new block of factorial experiments with a different
confounding pattern?
Claim:
Given a blockof 2 n-k factorial experiments in n variables,
1. At least 2 n - k - 1 experiments of the block need to be changed to obtain a new block of
factorial experiments with a different confoundingpattern.
2. New blocks of factorial experiments which can be obtained by doing an additional set
of 2 n-k-1 new experiments, can be generated by shifting the variables with respect to
every column of the design matrix.
Proof:
Consider a process P which depends on n variables T1 , T2 , ... , Tn. Suppose a 2 n -
k
factorial
experiment has been designed on it. Let the design be such that variables T1, T 2, ... , Tn-k
form a set of basic variables. Tn-k+l,
... , T are the corresponding non-basic variables.
85
It is desired that a set of experiments be designed which together with a set of experiments from the original block of experiments, determines a new block of
2
n-k
factorial
experiments with a different confounding pattern.
In order to change the confoundingpattern, the variables must be shifted to different
columns of the design matrix. In Part 1 we show that at least
2
n-
k-
1
new experiments are
required to produce a new factorial design. In Part 2 an algorithm is developed to generate
the different new designs which can be obtained by doing 2 n-k-1
new experiments.
Part 1:
In a factorial design the columns are balanced and orthogonal to each other. This implies
that any two distinct columns of a
2
n- k -1
n2
k
factorial design will agree and disagree in exactly
rows.
To change the confounding pattern at least one variable should be shifted from its
existing column to a new column. This requires that an experiment be carried out for every
row in which the elements of the existing column and new column differ with each other.
There are exactly
least
2 n-k
-1
n2
k-
1
rows in which the elements of the two columns differ. Hence, at
experiments of the original design must be changed to obtain a new one with
a different confounding pattern. This proves the first part of the claim.
Part 2:
In Part I it was demonstrated that
2 n-k-1
experiments can be used to shift one variable
from one column to another. Suppose that variable Ti is shifted from column vi to column
vI by doing 2n- k - 1 experiments, one for each row in which column vi and vI differ.
The question now arises whether it is possible to shift more than one variable using
the same experiments. If another variable Tj is to be shifted from vj to vj along with Ti,
then the columns vj and vj must differ in the same rows in which the columns vi and vI
differ. That is, the product of the columns vj and vj must be the same as the product of
the column vi and vl ( the product of two columns is a column which has a +1 element for
each row in which the elements of the two columns agree and a -1 element for each row in
which the elements of the two columns disagree ). That is,
vS = vi
VI =-- Vj . v
86
(B.1)
Using the notation introduced in Section 1.4.4, we can restate the condition in Equation B.1 as - the variables Ti and Tj can be shifted by doing only 2 n-k-1
new experiments
if and only if they shift with respect to the same column v, of the design matrix.
Clearly, any number of variables can be shifted to different columns provided they all
shift with respect to the same column of the design matrix. Moreover, the variables can
be shifted with respect any column of the design matrix. Therefore, we have shown that
by doing
2
n-k-1
more experiments any of the variables can be shifted with respect to any
column of the design matrix.
The new designs can be obtained by shifting all the possible combinationsof the variables
with respect to every column of the design matrix. This proves the second part of the claim.
Problems caused by rearrangement of rows
Although all the variables can be shifted with respect to any column of the design matrix,
it is not necessary that the design obtained are all distinct. In particular, it may happen
that given a column with respect to which the variables are being shifted, two different
combinations of variables may produce equivalent designs, i.e. identical designs in which
the rows are arranged differently. This point is best illustrated with the help of an example.
Example B-1 addresses this issue.
A problem with the procedure of shifting of columns, as described above, is that the
procedure is negated by changing the order of the rows of the designs. From the properties
of binary orthogonal matrices discussed in Appendix A, it is known that the set of basic
columns must have rows corresponding to all the permutations
of + and -.
Thus the
problem could be resolved if we ensure that any set of basic variables is not shifted. That
is, we shift only subsets of non-basic variables.
In the Example B-1, T1 , T 2 and T3 form a set of basic variables. Hence T 4 and/or T 5 can
be shifted by doing 4 more experiments. Also, T2, T 3 and T 4 form a set of basic variables,
so T1 and/or T 5 can be shifted keeping the other variables fixed. But, T2, T3 and T 5 do not
form a basic set. Hence we cannot move T1 and T4 without encountering the problem of
generating equivalent designs.
Therefore, we believe that an algorithm which generates different factorial designs by
shifting all possible subsets of non-basic variables with respect to all the columns of the
matrix will produce all possible designs where each will be distinct.
87
Although this idea
seems to be correct, the proof is not known to the author.
The MIT software currently generates new factorial designs by shifting all combinations
of variables.
Example B-1
* Suppose a process P depends on variables T 1, T2 , T3 , T4 and T5 . An initial design of
8 experiments is conducted as shown in Table B.1. The location of the variables and
their second order interactions are shown in Table B.2.
Given the Initial Design of Table B.2 four experiments (Expt. 3, 4, 7 and 8) are
designed to shift the variables with respect to column v2 . Table B.1 gives the actual
experiment matrix of the designs. In Design(I) T 4 is shifted from v
other variables are kept unchanged.
to v12s. The
In Design(II) both T 1 and T4 are shifted to
columns vl2 and vl23 respectively. In Design(III) only T 1 is shifted to vl2 and the
other variables are kept unchanged.
Examining Table B.2 we expect that the confounding patterns of the designs must
be very different. But this is not so. From Table B.1 we observe that Design(I) and
Design(III) are essentially equivalent to each other and only the order of the experiments are different. Hence, shifting T1 to vl 2 , keeping the other variables unchanged,
is exactly equivalent to shifting T 4 to vl12, keeping T1 , T 2, T3 and T5 unchanged.
In the same way, from Table B.1 it is seen that Design(II) is equivalent to the Initial
Design. That is, shifting two variables T 1 and T 4 has not changed anything, except
the order of the rows.
In the Initial Design and in Design(II) the variables T1, T2 and T 3 form a set of basic
variables. The variables T4 and T5 correspond to I13 and I23 respectively, in both these
design. Therefore the Initial Design and in Design(II) have the same confounding
pattern and are equivalent. In the same way, in Design(I) and in Design(III) T 1, T2
and T3 form a set of basic variables. The variables T4 and T5 confound with I123
and 23 respectively, in both these design. Therefore, Design(I) and Design(III) are
equivalent.
88
+
- +----.
-
Design(II)
Initial Design
T1
T2
T3
T4
Expt.
vl
v2
v3
V1i3
1
2
3
4
5
6
7
8
+
+
+
+
+
+
+
-
+
+
-
-
+
+
+
+
-
-
T1
T5
Expt.
23
+
1
2
+
+
3
V12
+
+
-
1
2
3
4
5
+
-
+
+
6
7
8
T2
T3
T4
T5
vl
+
+
+
v2
+
v3
V123
v2 3
+
+
+
+
-
+
+
+
+
-
+
-
-
-
_+
Expt.
1
2
3
4
5
6
7
8
+-
+
+
+
Initial
Design
Interactions
Vavg
VI
V2
V3
+
+
+
-
+
T1
v 12
T2
v2
T3
v3
T4
v 13
T5
v23
+
+
+
+
+
+
+
+
-
+
+
+
+
+
+
+
-
+
+
V23
T5
T4
123
113
T1
T2
T3
135
125
112
114
145
Design(I)
T1
T
2
T3
145
135
125
Design(II)
145
112
V
T5
123
113
134
I14
I24
T3
T1
T5
125
134
123
114
3
115
I15
T4
115
113
124
T2
T3
T1
T5
T4
135
125
145
114
115
I23
Table B.2: Changes in Confounding Pattern
89
12
T4
112
T2
I34
V
124
135
112
Design(III)
Interactions
V1 2
134
Interactions
Interactions
_
+
-+
Table B.1: Equivalence of Different Designs
Expts.
T5
Design(III)
T1
+
A-
5
Design(I)
Expt.
+
+
4
8
T4
V23
V2
V123 V3
+
+
_
V
+
_
7
T3
-
6
+
T2
'13
I24
+
+
+
90
Appendix C
Designing a Full-Block of
Experiments
In Appendix B a technique was demonstrated by which, a block of 2n- k - 1 experiments
experiments of the first block of 2 n-k factorial
could be designed which along with 2 n-k-1
experiments, forms a new block of 2 n-k factorial experiments. In Section 1 we will demonstrate the technique of selecting a block of
2
n-k
experiments of the first block, forms a block of
experiments which along with all the
2
n-k + l
2
n-k
factorial experiments. In Section 2
we show that the popular fold-over design technique is a special case of the Full-Block design strategy. The method of generating the fold-over design using the Full-Block design
strategy is demonstrated.
Full-Block Design Strategy
C.1
Claim:
Given a blockof 2 n-k factorial experiments in n variables,
1. A new block of 2 n - k experiments forms a block of 2 n- k +1 factorial experiments along with
the original block only if both the new and the original block have the same confounding
pattern.
2. There are 2 k- 1 possible blocks of 2 n-
k
experiments which can form a block of 2 n-k+1
factorial experiments with the original block. The new blocks can be generated by
reversing the signs of the columns of the non-basic variablesin the original block.
91
Variable
T1 T 2
OriginalDesign vl
NewDesign u 1
v2
u2
...
Tn- k
Vn-k
...
...
Un-k
T_k+l
V p n-k+l)
Unk+l
...
...
...
2
Tn
(n)
Un
Table C.1: Columns of the variables in the original and the new design
Proof:
Factorial designs are of size 2n . Hence it is clear that the number of experiments required
to be added to a block of 2 n-k factorial experiments to form a larger factorial design must
be at least 2 n-k +1 - 2 n-k = 2 n-k . In Part 1 of the proof we determine the constraints on
the new block of 2 n-k experiments. In Part 2 we prove the second part of the above claim.
Consider a process which depends on n variables T1, T2 , ... , Tn. A block of 2
-k
factorial
experiments is designed on this process. Let the design be such that variables T1, T 2, ... ,
Tn-k form a set of basic variables. The variables Tn-k+1, ... , Tn are the corresponding
non-basic variables. Let the columns of the basic variables T1 , T 2, ... , Tn-k be denoted
by vi, v2, ... , vn-k respectively. The column of a non-basic variable Ty corresponds to a
unique product of the columns of the basic variables and is denoted by vp(y) as shown in
Table C.1.
Part 1:
Suppose
2
n-k
new experiments are to be designed such that these experiments and the 2 n-k
experiments from the original design, form a
2
n - k+ l
factorial design. Let the columns of
the variables T1, T2, ... , Tn in the new design be denoted by ul, ... , un respectively as
shown in in Table C.1. The requirement that the overall design be a factorial design puts
severe constraints on the choice of the new design, that is on columns ul, ... , un, as shown
below.
Constraint 1: Every column, ui, must have equal numbers of +1 and -1 entries. This result
follows from the the fact that all columns of the original design have equal numbers of +1
and -1 elements and so must the columns of the overall design.
Constraint 2: The columns of the overall design will be orthogonal only if the columns of
the new design are orthogonal.
92
Proof: Consider any two distinct columns v and v' of the overall design. They are orthogonal only if,
[T
T
[
]
0
(C.1)
But, v and vy are orthogonal to each other. That is, vT .v = 0. Therefore the necessary
condition for Equation C.1 to hold is that,
UT U =
y
¥Z
(C.2)
Thus, the overall design will be orthogonal only if the new design is orthogonal.
Constraint 3: If a set of variables is independent in the original design, it is independent in
the new design too.
Proof: Let us assume that the constraint is not true.
dependent on variables Ti, Tj, ... , Tk (i,j,...
Ux = ui Uj . . uk or us = -ui
Let variable Tx (x < n - k) be
, k < n - k) in the new design. That is,
Uj ... Uk. The arguments for both these cases are identical,
so only the former case has been considered in the proof below.
The product of the columns of Ti, Tj, ... , Tk, and Tx in the overall design is given by,
Vij...kx
1
If
[
1
[
].
Ui
Uj
Uk
Vi
.
]k
1
= vk
(C.3)
Uij ... kx
Ux
1
In Equation C.3, vij...kx has equal numbers of +1 and -1 entries. Consequently, the
column v...kx will have more +1 than -1 entries. Hence, the column v...kz cannot be
part of an orthogonal matrix. Therefore the assumption that columns u is dependent on
columns ui, uj, ... , uk is wrong.
We can generalize the above result and claim that, the overall design will be orthogonal
only if, every set of independent variables of the original design, is independent in the
new design too. This implies that a set of basic variables of the original design must be
independent in the new design.
93
Constraint 4: Every non-basic variable in the new design must correspond to the interaction
of the same basic variables as it did in the original design. The sign of the interaction can
differ.
Proof: Consider a non-basic variable Ty (y > n - k). Its column in the original design is
denoted as vp(y). Let this variable be confounded with interaction ij...k of basic variables Ti,
Tj, ... , Tk (i, j,.
,k < n-k)
in the original design. That is, vp(y) = ij...k or vp(y) = -ij...k.
The arguments for both these cases are identical, so only the former case has been considered
in the proof below. The product of the columns of Ti, Tj, ... , Tk, and Ty in the overall
design is given by,
-
1
1
I k
[Ui
Uj
UiUi
uk
Uy
Uk
U
(C.4)
1
Uij...ky
If the overall matrix is orthogonal then there are only two possible cases.
1. Ij...ky
-
Vag.
That is all the elements of ij...ky are +1.
2. The sum of the elements of Ij...ky is zero. That is all the elements of
For any other choice of
Uij...ky,
the columns Ij
Uij...ky
are -1.
ky will not correspond to a column of a
orthogonal matrix. Therefore, the only way in which the overall matrix will be orthogonal
is if Uij...ky
=
+Vavg
or if Uij...ky = -avg.
This implies that uy = Vij...k or uy = - vij...k in
the new design. That is, the non-basic variable in the new design must correspond to the
interaction of the same basic variables as it did in the original design although the sign of
the interaction may differ.
Combining the above constraints, we conclude that the new design and the original
design must have the same basic variables, and the non-basic variables must correspond to
the same interactions of the basic variables. That is, the confoundingpattern of the new
design must be the same as that of the original design. This proves the first part of the
claim.
94
Variable
Expt.
Block
1
Expt. Block 2
T1
T2
T3
T4
T5
v
v2
v3
v1 2
v2 3
vl
v2
vs
±vl 2
±v 2 3
Table C.2: Table for Example C-1
Part 2:
In Part 1 we have established the following results:
* A set of basic variables of the original design must be a set of basic variables of the
new design.
* Every non-basic variable of the new design must correspond to the same interaction
of basic variables as in the original design, although the sign may differ.
Therefore, the only choice in selecting the new design is the signs of the interactions
confounding with the non-basic variables. That is, if a non-basic variable Ty corresponds to
interaction vp(y) in the first block of experiments, then it can correspond to either +Vp(y)
or to -Vp(y)in the second.
There are k non-basic variables in the original design. Hence there are 2 k possible choices
for selecting the signs of the columns. The choice corresponding to selecting all positive
signs results in the same design as the original one. Hence the overall design will have two
sets of identical rows and will not correspond to a 2 nremaining
2 k-1
k+ l
factorial design. Each of the
combinations corresponds to a distinct block design which can be used with
the original design to form a larger factorial design. Thus, the second part of the claim is
proved.
Using this result, we develop an algorithm which generates different block designs and
selects the one which produces the minimum confounding between the variables and the
probable interactions.
Example C-1
* Suppose a process depends on 5 variables T1 , ... , T 5 and 8 experiments have been
performed as shown in Table C.2. T1, T2 and T3 form a set of basic variables.
95
In the second block of 8 experiments T 4 and T 5 can be assigned to
±vl2
and ±v 23
respectively. Hence, there are essentially 4 choices. Choosing columns of T 4 and T 5 as
+v1 2 and +v 2 3 respectively in the second block, gives the same block of experiments
as in the first block. Therefore, this choice is not valid.
If we choose -v 12 for T4 , then, T4 along with T1, T2 and T 3 forms a set of basic variables
of the overall design. Choosing the column of T 5 as +v 2 3 will make T5 confound with
the column I23 in the overall design. If the -v 23 is selected then, T5 will be. confounded
with I34 in the overall design. In general, it is more desirable to confound T5 with
I134than with I23. ·
C.2
Fold-Over Designs
The procedure of folding factorial designs is commonly used to increase the resolution of
the designs and enable them to determine important interactions. In this section we show
that this procedure is a special case of the Full-Block design strategy. This result not only
justifies the use of Full-Block designs, but also establishes its superiority over the fold-over
designs in sorting out the important interactions.
Claim:
The fold-over design is a particular case of a Full-Block design. A fold-over design can be
obtained by the following procedure:
1. Determine the order of each non-basic variable, that is, order of the interaction of the
basic variables with which it is confounded. (For example, if T 5 is confounded with
the interaction of basic variablesT1, T2 and T3 then its order is 3.)
2. Design a Full-Blockof experiments by changingthe signs of the even-ordered non-basic
variables only.
Proof:
Consider a process dependent on n variables, T1, T2, ... , Tn, on which a
2 n-k
factorial
experiment has been designed. Let T1, T2 , ... , Tn-k be a set of basic variables and Tn-k+l,
T2, ... , Tn be the corresponding set of non-basic variables. A non-basic variable corresponds
96
--- ---
V/I
T1
T2
T3
T4
T5
Expt.
1
2
3
4
5
6
7
8
vl
+
+
+
+
v2
+
+
v3
+
+
+
vl2
±
+
v123
-
+
+
+
+
+
-
-
+
+
-
+
-
-
+
Table C.3: Initial Design of Example C-2
to a distinct interaction of the basic variables.
Hence, given the settings of the basic
variables, the settings of the non-basic variables can be uniquely determined.
From the property of orthogonal matrices discussed in Appendix A, we know that the
columns of the basic variables contain rows with all the possible combinations of +1 and
-1 elements. Therefore, for every row of the design, there exists a second row in which the
signs of all the elements of the basic columns are reversed. Clearly, in the second row the
signs of the elements of the columns of the odd-ordered non-basic variables will be reversed
too.
Comparing these two rows, we find that the signs of all the elements are reversed except
those which lie on the even-ordered non-basic variables. Suppose a new design is obtained
by reversing only the signs of the columns of the even-ordered non-basic variables. For
every row of the original design, there will be a row in the new design which has the sign
of all its elements reversed. Thus the new design is a fold-over design. Moreover, only the
signs of the columns of the non-basic variables have been changed. Therefore the design is
also a Full-Block design.
Thus we have determined a technique of designing the fold-over design using the FullBlock design strategy. This proves the claim.
Example C-2
* Consider the initial design shown in Table C.3. The fold-over design of the initial
design is shown in Table C.4. Table C.4 also shows the Full-Block design obtained
using the above procedure. Comparing the Full-Block design and the fold-over design
97
Fold-over Design
V/I
Expt.
T1
-vl
T2
T3
T4
Full-Block Design
T5
V/I
T1
vl
T2
T3
T4
T5
Expt.
v2
vs
-vl2
v123
1
2
3
4
5
6
7
+
+
+
+
-
+
+
+
+
-
+
+
-
+
-
+
+
+
+
+
-
8
-
-
-
-
-
-v 2
-v 3
-v12
-v 12s
1
-
-
-
-
-
2
3
4
5
6
7
8
+
+
+
±
+
+
+
+
+
+
+
+
+
+
+
-
+
+
+
+
+
+
+
+
Table C.4: Fold-over and Full-Block Design of Example C-2
we find that they are identical, except that the rows are arranged differently. From
this example, it can be seen that if the signs of the columns of the even-ordered
non-basic variables are changed, the fold-over design is obtained. ·
98
Appendix D
Completing Fractional Factorial
Blocks
In this section, we will study the techniques of completing a partially incomplete block of a
factorial design.
Claim:
Given an incomplete block of m (m >
2n
- k
) experiments of a complete block of a
2 n-k+
l
fractional factorial design,
1. The incomplete block must have the same confounding pattern as the complete design
block.
2. There exists a unique set of 2n-k+l-m
experiments which complete the block.
Proof:
In order to prove the above claim, it is assumed that the given set of m experiments indeed
corresponds to an incomplete factorial design. Later, using the properties of the above
claim, a procedure is developed to check if the incomplete block is part of a factorial design
or not. We shall prove the Claim 1 in two parts. In Part 1(a) we show that if any group of
variables are confounded in the complete block then they must confound in the incomplete
block too. In Part 1(b) we prove the converse result. The first part of the claim is proved
by putting together the results of Part 1(a) and 1(b). Part 2 of the proof proves the second
part of the claim.
99
Part (a):
Consider a block of 2 n-k + l factorial design. From this block any m (m >
2
n- k )
rows are
selected to form an incomplete block. The columns of the complete block are denoted by
ui. The columns of the incomplete block are indicated by prime and those of the missing
block by double primes. Therefore,
Ui =
:
U
i
Vi
(D.1)
Ui
For any column ui of the complete block which is dependent on some other columns uj,
Uk, ... , Up,
Ui
Ui
Ui= uj uk ' . ..- up
(D.2)
1=
(D.3)
[
[][U;P
]
X J.
Clearly, this relationship holds for the incomplete block too. Therefore, any group of variables which are confounded in the complete block, must be confounded in the incomplete
block too.
Part 1(b):
Next we need to show that if that any column u iz is dependent on columns U~7
u, k,
k,
... , u
,
Up
in the incomplete block, it must be so in the complete block too. In order to prove this, let
us assume that the statement is not true. Suppose,
Up
(D.4)
II ujII*UUk ....U
. Up
(D.5)
U~
z =UUkj
k
...
By assumption,
ui
k
In any orthogonal matrix if a column is not equal to another, it must be orthogonal to
it. Therefore, if the column ui of the complete block is not dependent on the column
corresponding to the product of columns uj, uk,..., up it must be orthogonal to it.
100
If the product of the columns Uj, Uk,...,
UT Ujk...p -
[ uk ...p
jul
iii - 'a/I. -
But from Equation D.4 u and
k
UT Ujk...p
denoted by Ujk...p, then
Up is
I IT
=UIT
UT...
+Ui
m +u
u
(D.8)
- k
> 2n
uIT . uI=j...
-=m
is 2n-k+l-m
Ul
(D.6)
(D.7)
T i
· Ujk...p
u'i' Uk...p= -m
lT
0
are identical. Therefore,
u:Ujk...p
=m
The dimension of u and u"p
Uk
(D.9)
and their elements are +1 or -1. Hence,
< 2 n-k+l
-
m < 2 n-k
(D.10)
Comparing Equations D.9 and Equations D.10, there is a contradiction. Therefore
the assumption that column ui of the complete block is not dependent on the column
corresponding to the product of columns uj,
k,..., up is incorrect. Hence the column ui of
the complete block is dependent on the columns uj, Uk,...,
up.
Therefore, if any column depends on a set of columns in the incomplete block, it must
do so in the complete block too. Consequently, if a variable is confounded with a group
of variables in the incomplete block, then it must be confounded with the same group of
variables in the complete block too.
From the results of Part 1(a) and Part 1(b) we conclude that the confounding patterns
of the complete and incomplete blocks must be identical.
Part 2:
In Part 1 it was shown that the confounding patterns of the complete and the incomplete
blocks are the same. Therefore, any set of basic variables of the complete block must be a
set of basic variables of the incomplete block and visa versa.
From the properties of Binary Orthogonal Matrices given in Appendix A, it is known
that the rows of the basic variables must be such that each row corresponds to a unique
101
Expt.
1(1)
2 (2)
3 (3)
4 (5)
5 (7)
T
+
+
+
-
T2
+
+
+
-
T3
+
+
+
+
T4
+
+
-
T5
+
+
-
Table D.i: Incomplete Block of Experiments
combination of +1 and -1. Also, the columns of the complete block contain all possible
combinations of +1 and -1.
Using this property of Binary Orthogonal Matrices along with the result of Part 1, the
incomplete block can be completed in the following steps:
Step 1. Determine the basic variables and the confounding pattern of the incomplete block.
Step 2. Check which of the
2
n - k +l
combinations of +1 and -1 are missing from the columns
of the basic variables. There will be 2 n-k+l-m
such combinations.
Step 3. For each of the combinations determined in Step 2, determine the entries of the
non-basic variables of the matrix using the confounding pattern of the incomplete
block.
Since each of the steps of the above procedure is deterministic and unique, we prove
Claim 2, i.e. there exists a unique set of 2 n - k+l - m experiments which completes the block.
Example D-1
* Consider a process P which is dependent on five variables T 1, T 2, T3 , T4 and T5 . Say
a 25-2 fractional factorial experiment is to be carried out on this process, as shown in
the Initial Design of Table B.1. Suppose that of these 8 experiments, 5 (m = 5 > 4)
have already been carried out as shown in Table D.1. Which 3 experiments need to
be carried out to complete such an incomplete block?
Step 1: As per the procedure described above, the first step is to determine the 3
(n - k + 1) basic variables. From Table D.1 it is observed that the following
combination of variables correspond to a basic set of variables:
102
Step 2
Expt.
1 (6)
2 (4)
3 (8)
T1
-
T4
+
+
+
Step 3
T5
+
+
T2 =1 14 5
+
-
T 3 =-I1 4
-
Table D.2: Missing Block of Experiments
(i) T 1 , T 2 , T 3
(ii) T 1, T 2 , T 4
(iii) T 1 , T 2, T 5
(iv) T 1, T 3 , T 5
(v) T 1, T 4 , T 5
(vi) T 2 , T 3 , T 4
(vii) T 2 , T 4 , T 5
(viii) T 3 , T 4 , T5
Since only one set of basic variables is required, it is not necessary to enumerate
all the sets of basic variables as done here. Let us select T1, T4 and T5 as the set
of basic variables.
Step 2: Scanning the columns of T1, T4 and T 5 the missing combinations of +1 and
-1 are determined. They are shown in Table D.2.
Step 3: From Table D.1, it is observed that T2 confounds with interaction
I145
and
T 3 confounds with I14. Using this information the missing columns of T2 and T3
are determined in Table D.2.
Comparing Table D.2 with the Initial Design of Table B.1, we conclude that
Expts. 1, 2 and 3 of Table D.2 correspond to the missing experiments 6, 4, and
8 of the Initial Design of Table B.1 respectively. Hence, the missing experiments
of the incomplete block have been determined. a
Determining Incomplete Factorial Blocks
All the results determined above assume that the the incomplete block of experiments
indeed corresponds to a complete
2
n-k+l
fractional factorial block. Therefore, if the set of
experiments does not correspond to a complete block, the results do not hold.
A block of experiments which do not correspond to a complete block will have either
more than n - k + 1 or less than n - k + 1 independent columns. Thus there will not be any
set of n - k + 1 basic variables. When the procedure for completing the block is applied to
such a block, the method will fail in Step 1. Thus the procedure of determining the basic
variables can be used to check if a block of experiments correspond to an incomplete block
or not.
103
104
Appendix E
Computer Software Listings
E.1 Startup Routines
1. Startup.C
2. Stritr.C
........................................
107
........................................
114
E.2 Block Design Routines
1. Block.C
........................................
2. Blkitr.C
116
........................................
2. Compblk.C
2. BlkAitr.C
123
........................................
125
........................................
129
E.3 One-at-a-Time Design Routines
1. OAAT.C ........................................
132
2. OAAT-itr.C
141
........................................
3. OAATopt.C
........................................
144
E.4 Common Routines
1. Plant.C
........................................
2. Sortint.C
3. Dispinfo.C
146
........................................
147
......................................................................
148
4. Database.C ........................................
150
5. Setmat.C
152
........................................
105
E.5 Header and Data Files
1. Conincl.H ........................................
153
2. Coninc2.H
154
........................................
3. TrueProcess.Dat
4. Interaction.Dat
........................................
........................................
156
156
106
C
O
C'-
Ho,
Co
A-;
-
Co
o
o
C)
C
f
*
2
.
Co
-4
*
U2
*
_
*
Q
~.-.-
*
-;
*
°,
Co
4
S
o =
~Co
P. .
Co N
-, I
>^
Q.
X >,C
^,
O2
I-
o
uI:
x
. -
#
#
#
Co
#4#
Co
Co
#
4*
#
#4-
Co
*'-
#
#-
o
#
g
C,
4#
4-
*,
#
~
4U
Co=X%
Co
X
,
UC^
o Co O5
>
*5
11
Co
uWO~
t
o
o
o
o
5
44-
zD-o
W U]C
Co
D
. . . . .
'.>','>','>',
#
o
ii_
.O
44-
Co
.=
e2
-
#
2
#-
£
,
,_'Q
- :3
g
cjN
#
Co
eo,
.5
Cou
4~
#
#
CN
- Q
._
0
._ . x
N
o
0I 51
g
.
Co
#
Co
Co
l 4,x='°
It
4
Co
.-
#
' Co D~ u
It
#
#
#
CoA-
*a
#
#
##-
#
x-
ICIo_l..
o
#
#
#
#
#
D
D~
#
o
#'..4
#
#
I
D
0
O
#
4-
4-#-
oCo
LCS
2'
2'2'
2'Co
.A
Z
Co
Co
0
LA
::
II
Co
z
.
II
II
*
o
Th
2'4-
Co
5Co
Co
0
Co
0
Co
4-
Co
oLA
2'
*
z
II
4-
#
o
4-
-
tC
O
C
.Co
Co;
Co
o
4
"N
XE<,i
E
4-
4-
5.
SCo
EE_ Co
CoX
Co
H-Co
C~
####
#C#
C
#o#o
CO
44#444444#44#4444#-4
*
z
CN
O
O
0UI
U
O
u 5 =
Qo Qo
*-
o--
~Co
Co
z;
-'
Z Z
5'
5.5
Co
-
ii
*
Co-
-
=
Co
*
Co
Co
*
=
Coz
*o I ;
0
Co
5
I
5LA
444#
#4444444#-
Co
*
II
Co
CoLA
*
V
ii
Z
4,44-
C
*
S
Co
'-
II
Co
'
4#-
II
Co
CoCoCo
Co
Co Co
>
-5
.2*
Co
4-
z
5'-
s
Co > 5
.5 S ''24Co Co.*
5 Co* *
Co Co
Co Co
>4-
Co
-
LA
Co
2'
2'
Co
ii
:
Co
Co
*
*
Co4-'
-
o
Co
O
X
'-o
Co
-
Co Co Co.Co 0
5
Co
Co
Co*
Co>Co
Co *
*
11
Co~.
z
-
*
*
4-
'
.C
0
4-
*
Q'*
=4 #
'2
0
--
0"
-·
N
N 0
'5 -.
+
En
_.
rF
'oS
-i> I:I
c0
0
< C+
°.
*
®
00--
N
*
*
g1+
='0 +
o.
- +'
ND
'
0
aq
0-
.
N
,
0l
0
N~-'S
'1
F
R~
0N
N
0000
5 2~
N,
N~l
~
0000
0vi
0-
000
0
,Z_1
Z
CO
T--
oI
*
±o
O
0R-I
*
*
0
,n9
0
*+~
*
00
s-~nI l~
N
*1
*
U)
N
*U
11
0
*t
-'
*
-
U)
0D
*
0
0
0
g-q"
'
*
0-
-
Is*
N
-
_-.
O
0P
N
·
r
0
N
t
N
0
o
o
VCD0
_
,N
0e
*
_1
c
~
II-.
0
,
o2o A a
Cn C
I
,-Z ~2.5
3C
_1.
'
11'
c
*
-
*
: CL
5
II II Ho
t
zI
11
1J
*
*
C'-
0
*11
m
2. . .
0
*
*
S
2. .
~
N
0'
.
0-.'
t-t*
*
2. i
**
*
*
*
.
P
cD
CD
-~
2-
2
,0
55
£
i. V\x V
Th
±
*
*
*
C) ..
ei
0
_
I
Nr
I
"
o 0F -
p
-
Io,
0
(
+
'r
P
0
X BF3
>
ae
O
0
(D
-,
0
0t
C
Co
c
o
0
"I
a
cn
~D
D
09
'
I
0
0
-~~o
-
0
a-il
0
, .I .
CD
q
0
*
*
6
' .
a
2.
011- i--
,
"
M
N
<C
N
-
x
'1
0
0
0
-0
oFN
*
pvi
_
X
S
r t
N
0
O
0C0
F
0
o
S= W Nw *
_.t~
H,
p
N
0o N* ; .
'"~ 0_.
0*D -. ,
Oc
.·
s
N* *CD
-a(.
NDO
OH _
oOj Z70
*
_.
C
C
O
cn
C
H-
S-
_ ._ 0
_,
.w*
.US
-~.H-. m
H.-R
o
CO
11
CAo
cq
c
'2.
4-'
0
I
o
0~
~~~~~
~~~~~'
0 '"I
a
0
o '
-r-
-0
I
c
C;
~~~~~~
4
"~,
0.·r
0
u
.clE:
-
-
0
I
,0.A4
.- ~
~~~~~~~~~~~~~~~~~
c'-
fO~a 0.3
u0
0d
r
* -
'u
*
(d**-C
dI
H~r
4-
o ID
4-
._~
u u u
0
0
0
40
·
IH
*c*
c,
4-
* '-r
a,iI
0
-*
,
*,
-0
A
'=
'El
000
-0
*
u
4
"
4-
0
.
-C)
.
n
,
*2C-4
~. W
:
ii
g
If, u
a
3=
-
,, ,L~~~1
+ u~~~E..
,,-
r= "I
~
Il
d :, >
-C~~~~~~~
~
U0
.0
fi
0~
11
I
~~)
d c emd
-,
--
-,,0
b ,
O
EW
0r
0
c
~,,C)
®
VV
,
w66
- 000
- 0-
d
-i
_, -F3d ZbOU
B
Q
-,
U
I
+.h
0
-'
4.
,,b II
· .. Q
"-4-
,
OC)
:> >;>
C)
4-
3
E
00
3'l Ed
~I
F
?
~--
o -
o
H
+F~
.
i
: -'-3
~ -n
0.
-
oi
C
-q
I
Q
_
I
¢-
0
OV
.
C)
C
C)
.
U
I0_
~
.
~~~~I0
= !t 4-
4-
~0OO
O
CDO~~
*
·
4-4-
N
NI
0
0
E
~~~~~~~~~~~~0.
#
#
4-
0 4-
4-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Q
C)
4-
i
#
<
*
4-
.44-
4-0
4-~
0 -
:~
.
* 0C).
~
'4,~
>~Z
~ ~~ ~~~
'.-~
.4-
4-,
*
"
-
0'
,N
~ ~~~
4-
II -~v**A -0
~S'',I
-~4'*'
>
*w0~
~~
~ 0.~~
.,-~
.
*
~
A
S~'4
o
0
~)C
*
0
*
0.
0
-=
0.
.0
-
~~< ~~
0 2
-' C) t
CN
.5
-
-0*040
0
0
#
,
In
'
IIC
(
)W
C
4-'
'.
O
**
w,: #,
*r';4 -4
*~ 0
*4#
* x
-04C)
#
0C)C
0.*
(.1
a,
0
0
U
'-
,e*
#
*
O
#
V1
~'
SSb3o
V
,
*l
#
u:
0
0
r_
*n
o
0E
1
O
g
*
#
eca
+? _
,
_
##
c':
;r
x
0
0
n0
II
o
F
n
+
II
#
*'
II
O
-
*
8
CD
o
e-
-'
*
II
.y.
a
IIo
0
o~~~~~~~~~~~~ .-#4,o
n
~0
*.##
· 00
o0
**#
M
m
*##
O g
##
1
r
I
00
g 0
iX
00'
0
0,
1
*#
-o
a*#
xo
0
O
-i 1
C3
e
#
oO
- 0 ,
0
01
(D
*
CD
rt
a
0
*c
*
0
-
0F
t
Ffi
8
Q©+
+
D
0
n
_.
o
PI
0
c-i
.
O
.
_
r·0
awPL't
*
*
*
Z-
I-
00
0d
O
0
Q
I1
-P
I
0C,
.S
0
00
0
OC
0
0-
-
0
qD
~°
L0
O
~I
(Do 0
0
o
0
d
0
'
° A
.
/\ II
('~
I
8';8SCO
_. O
p
o
-
o
*
c
Oo
0
0oo.
B
m
3- 7*
*
S
"
0 0-
-II
II
5o
A
c`
0eIl
®%,
l(D°.
cn
* 0
'1
r
II
fi
+
mI~ I
11
_.
I
C
0
n
I-0A
0
00
II
,
o
co
1~
d
±2-
Oil
0
-
o
:
*
*
*
*
r
o
+
'
00'
o
*
r7
rn· r·
~ ~ ~
+
*
*
*
_.
_.
a
C0a
00+CL
M
-
g
9
'"
+
0'"
_ 0 0,
v,0 - l
o.a
. ,>
0
1to
*
_. o
U)
D
oi o
2I, D 00 ±1
,
n,
o
C
In2m
.
o
o o o o o
n .n n_
0 0
II
-11
,C_.
O
0
0
0
0
I
0
0
10
*
o
*
C."
11
O
n .~
0 0
-Ao ~.% o io<
It +o II
*
'
O
0
10
0CD C
03~
0
0o
(
Oi
l p
0
0t
'O+
0
A
II~~
o
II
1
0
0
0
0..
-I
~
~
o
cII
OI'
0
1
1
o
-
CI
- .
0.
H-
'- - -
0
c0
0
I
0
II
w.
'1
-
I
00r
1
CD
*0
F
r
R°
.
1
C-I~~~~~~~~~~~~
a
c,
O
C4
rP
C
P,
*]
-0S
S
c I'
C Dm "CDCD0
CD
-
C*)
~
~
~
~
3
C
cq
0
..
u
o
- .5
+
U
0
0
U
._
*
*
.5
*
-u
-'.+
..,VIntt
0
2
#
*
0
S
-
o
u
,
',-r
4~
4##
#*
.-
Co u
cP
+
A
.-_
I
d>
E
0
#
E
#
4
.=1
o,~~2
cr
S
,Th
O
X
E
C
00
0Ut:
(a
"'0
u
-'0I
U
-0
#4
x
o
# ##
.·,
;
*
°2
o
v.
I
_
_
V-.
+,0
+oo
Ii-II
o+I ''V 5zI
,
X*
II
'0
O.
0
o,
II
E
0
0
co
Sd
*
.
-,
SYS -d
y
8 u
C
O
#i .-==~U o
U
1
o
Uo88t
>
UUC~
4
.-
v
0
UiE ~ · ,,,.
0
u
0
X
voU --,-,F3I
a
0a
o
#
II
U
E
*
*#
#l._
0
II II
n
0
4x
EX
+
U
A-
---
*
;e
U- 1*
+._
0
4
'o
:,
0-
vU
i.
_
0
0
0
0
+l
o
UV
OUUi
_U__
0In o
0
* _l.
I5
_
0
0
C
--
0o
.I
fo~
0;+
o
I Q
Io
U
O
oC
0
o
C1
02
o7
0
.,
a0
4
CJ
z
.
10
.
2:
4
4
4
4
4
4
#
#
o
4::
4
4
4
4
4
#
4
4
4
4
4
4
4
#
#
4
4
#
4
4
_
0
O>
4
4
4
4
4
#
#
4
#
4
4
4
4
4
4
4
4
#
4
4
4
4
4
4
4
4
4
#
X4
4
4
4
4
4
-
O
4 4 4"U
'
aI
0
;
#
*4
4
4
0
1I4
.,
In
F. 4
*4
-
#
4
0
#*
,
4
4
~ o ~ X
I
_
i.=
o o ,"
v
4
4
v
4
* *
*c
...... 4
4,
**
*
*_
,
> 2.c
It
*
4
.
4
I
=
+
01
#
4
4
4
4
4
4
4
4
4
Q
+
C.
,
)
40 4,0
4, 4,,
4, 4
Y~WW~WWWWWW~WWWW~W~
0:
4
0
0)
Q
-4
4
-#.-
Q-
0
0
r
.
n,
#
II
I I "I
DD
SC8a~i
O
.
- S :
(090
O
0000
~*~'
0
_ . _.
1
O.0
0o 11~ O
V'n
>:
.0
rvi
rr
PC-
8
0
0nt
-
IIII
I
II
GOfXXO
: '
_I
~
a~
_
0(0:.
I
;
o
+
x
*
n
_
.o
B'-*'-*
_ .3r
I)
x
WJ
I .
0S
D_
t,
.
.
+
n
8 '~ 8
Ap
0a
·+
_
o
'
II
0 9
0
-I
_-.
II
I
II 11 1 I
00
~11
m
11
_
n
11
n
aq
09
D7
*
tL
.
O
0
m
_.
-.
.
<
II
C-C
0
DCO
r
I_
t\p
II
11
0o
C
0
-I
()
5 *
II
=CO
0c
,
C
M
,
n
m0
o
II
+
I
o'
0+
II
3
*0t
I
I
<mr
< 'C0'
O
c-
0-01
I I
000
*1I1
0
FI
_
L-
0
+
_..
0 +±5
,0 p'
-'
II
0:
*
c
ro* 41
u
+'h
O
N
0
*n
I_
O
±0. N
_
0
0
_.
I-
x("
Cl
IC
*
g3t
%
%
4.
*
0a
*
iS
-p
Co e:5
00
*
05
#
l
4
4
#1
0c
0J
B
Or
II
0
O
o
*
**
0
00+
O
I
*
*
0
0
(
5
*#
a
pI
o
#
o
#
o
_
c-
_
#
W
*
11
11
w_.1
4.sr
0
II
I!
cn
o
0
o
+
0
o
m
*
cm
#Og
-S 0~ '70II
0
#
*
cm
**%
*
*
*
*
l#
0
-
=,O Z
o
#
0
<
0
(P
e
T:'
l
4
4
4
*4
*
II
a5
1
(D3o
I X
~0
*
4
*
#
<<-I
00'r
4
4
**
0
0
o
,W
S ~ a;
g'' c~
- o-
'X° %
4
4
ci'i
-o~
.
*4
*
*
'00<*
-
II
PI
-iO
#-
0
C
10
o.
©
Z
,--I
0
.:
0
._
I *0
ri
0
0
0
U
411
_c >
11
C
o
o
10
10
o
cc,
0t__
II f
·
L
-
+i,
0'+
4
m
1~~C
.
0
±-
O
*^
It
U,-
c,
'40
U
0.)
+
C
U
I
I
r
Q.
O
.,
0
U,
U.
m
z
1
o
. Ita
11)
0
U
¢
a
O
E
O
I'
E
.
,O
*
o
,;
Ut
VII
*
._
11I
H
_
U
)
.
V
f+
mII
.4-
o
O
+U I
o
.*
U
o
._
V;
Ea+
0
*O
o+*
3+
+
11a~~
^. w
I c1
I II
._
''
,
I
_._
_~
Oo
O
I
U)
d
U
,
,
V
O
_ r.
II
.( .c
a
0½.
3125
+ .
S0
O~f~
0~~I
II
1.1
D
aO
v
00 0
e'-
c4
o
.~
O
.t
VO
&V
--
III
.c
V0
1
-21
m
*
V
O
11 >
i
II
U
I
.__
- °C
I
0
0
E
II
m
*_
O
._
0
V
·
0L~)
.
+I I
-~ . V
_
0o-*
~~2311
a
+q
_/
+=m+
+
u
_
._
'>
0
s~s
)0
20
W
d
o
U
1
0 0
11
;
=
0
-'C-
,
59 =C-9D5
CC
C"
8
A
Z
ct
m.:
."
"C
N C g . ii
ICt
II
X rC BS
~~
m
N
3
' Fe "
?-O
~S
re
*N
v
o
0
+-CD
f
**
*
*
*
*
0
a
-9
0D
cn
*
*
H-
3 o
n.
0
"I
CD
*
*
*
*
*
oe Ct
*
*
*
*x.
v
*
Ct
o
-9-0
-9
-9
CCC
Cr
-9
0r
·
;-.CC
0
CV2
*
*
0.
CC- F
oC
*
Cr
-9-O
-9-U
*
0
vlC
(D II
p:2
-r
r
?r
-C-.
Ct if
_
-.
:2
CB
F9
-C
-9
0
.
=1
c s
0 Ct 0
re
*
PI
-
C
*
--
E
0 -
o
>
O
C7
NCS
ct
-- *o
Mo
ft
Ct CD0
ra
9
*
*
CtP
-.
-
*
,
--9-
I
-9
1
*
*
*
8 8 8v
1
IM
8
M
b
a10
8
+'P
Cr
onO
za
CC
*Cr~
-D
-9
F
1
=r
S
Cr~tSC
C
Ct
M
Cr
-
N
N
*c~*=l
-_.
8 *
o. oil
8
r. tr
D
1o
11
'r
*
*
*
x
0
*
*
*
0-9
w¶
*
In F
S
+I
Ss7
0
e~
cn
I_
2
CC
**hA
c--
0
C
*
9s
Cl
S
#
*
CHHnHH
0<
"I
e ~
tHI c
=1
10
$$
D
P,
vet
~.
Cr
~.o
d0 t 't
,_A
0o
CtI
0oif
cC
a
_.p.
A
t
U]
i-
-
.t
o
-- 9
IN
C
-'
t
D
t
0
t
~C
- -e
"CC
"t
(DD -D
-(D
CCO
O
CO
ft
000000
if
if
if
if
if
if
if
if
*
if
if
if
o
W
*
_
Cr
8
i.
8_8
I
I
I
g
P
"9O
v
o o:
M,
-CI
•.•-t
~
¢CC
C -
Cr tr
IwM
II II 5-
f
OtClt
-C""
C
o
o
o
09
C3,
OPI
=-
a
I
11111w
8 II 8 ii
11110o
_..
-
C
.
R_.. 2N
87-.
"t
-9,
if
S
5 A
O -s
0t
C
+
II I O/
+-+:
e+
'*
-9'
N
o
_
II
o
11
N
a
-9
-9.
8-
tu
C<0-9 d Cr
0+
aat1
i
I.
i1
o
,
-o
"
PI
_
a
;3
ie
11
(D
o
\._..
~.
_-
H
-
c
if
>S
SS;
cf
'-p
_9.
_-._7
Ot
Ct
CDUN
OC
N
'-'tcD
DJ
-9
8
aC
H- ·
I
N
Nj-~
c
C
II
S.
-U
H-
0-
C
o
+11
A
Ctt
II
0
'F-9
8-
-C
I I-
-O9'
o
I
_i
CO
O
+ o°
.
R°1.
I.
2
CC~ C
Ct8
0. os
'9CD
-9
-9
-
_CCC
-s
-
0
.t
1
o:
CCC
g plr
8-C
P.
0CD
a a\
o%
. _--.*Co
0
±
o
°
~~~
CO
o
t
a(D
a
--
-9
-.
C
I
8
C
0
0
(0
0
II
ii
0
0
0
A
0
0
0
0
.-
-1-
-
0
0
S
+
-
0
A
0
O*
*
0
0
0
0
S
0
ES
U)
. U
U) 3
+
Cv
0
I
.Q
*'Nv
S
0
II
0
t
*i-~
4
o-o
0
If
SV
I
CS
I
0
I
0
0
.~ r0
0C
0
0:z
rz
,g
0
0
0
*U
0En
*0Inu . _0
0
.2
,;
A
11 00
C!
-02 1
')
#'
+.
420
0
0_0
1:
0
2
.
0
0o,0
0._
InIo;
0
0
L _
.. -
A
IC
'
Qx,
0
;
,,A
.
_-I *
0z
Qf
It
Cl)
0
-
InIfO
00
CN
. ,- *
.,ro
0
0
-
i
F
Ci)
0
0
0..
)),-,
0
..
#0
0
'.'500
*
-.- 0)E
0
0
I
Id
-
._~~~
-
In
0E
01
C)
0
0-
1-
0
-C~'
0Q
;2
0
-w
'o
>
I
073 . _
0-=
Q -00I-I
2
--.
0
I
D
S
ii
In.
+
ou
A
In
1'
-0CN
11
0
E
.
0
+
~I
o
0v
0
-
o
+
#
If
#
0
-
0-
r
~
*0
0
o'~
00
)
00,
o
0
o
o
;>
._
A
. _
II
-
.o
'
If
If
N
If
#
#
._)
0
I
-).
In*
,
A
#
If-.
If
If
0g
0001
#
#
If
If
If
If
I
X
#
##f
If
If
If
#
#
#
#
*A
X
#).
If
IfIf
If#
If
If
If
If
If#
If
-IfIf
If
#f
If#
#
#If
If
00°
If
If
If
If
If
0
If
If
a)
>
w
2e
V;
0
-
0
0
0
0
c,-
0
0
.d
0
,..1,
.
*
0
11
0
0 II -- :'~
.
.
i)~Sw
,,,=
C~
o
C
I
I IdR
^ 0-1 0
"I
0,A
0 9le
II
If
If
,,, Uo
11 -.
I.
If
A 11 e 11
~"
'
.
C C
e e e
0
~
-I
0
0
0
.0
If
VI
d
Q. '
;>
If
If
1)
0IA1
.0
-0I
0~
A
A ,A· o~;;>
II
00.=e
Ae .A
0E
0
0
e0.+
,
~ A,
,,oAl> IIA I>
0
0
0
Q
U.E
0
.00
.
0>
0
-
0
If
If
If
If
0
0
CO
I0
0
-
0
0.
9
'
O
9
0ID
n
*
0#
0 u:
In<10
0M
_ 0 09
IO
1 0E
cn
0v
:P PPI
#
N Oq
tl Z
_
_.
D ,>o:
1
"
0
__~
#
w
E.
N
01P
I0
*
'
0
4.
~ ~ I,*
00ll
'1
I
~~q
90000
V~
3
_
C
8
Jr
O-q~~~~~~~~~~~~~~~,
^t
'
0
0
-~,'
< 00
000
00
V V V
00
01~
*
B1+0>110r
~'
~'
~'~_.
001.
+Z.
° s
c
0
0
0n
*
-
--
0-
m
m
+
~~~~~0
cn S
11
11
C
a
~~~~~
D ~
~
P
>
_
C
01'~-~
~ ~
_ .t
PF
S
n
010
,,
-V
0
_
I 11
a
nI
0-
+v
+
0
Ctn.
sOq
to
10
C
C
0
01
<i
<
0
-
----
*
.*0t
~-O ~.
<0o-
0
0-0
o'
9' 9
*'S
0
0
*
*
*
<
001
B
B
T
*
*
0n
0
0crq
<
~_
0
I
0<0
0-
0-
0# #
03~ 9
0o-
0
n
..
0"1
0tl
n n
0Z
Ii,
z
0-h'"
o
*
0-
0-.
#
*
*
*
II
0'o
00
010
09
0
0
02
oO0-
0
*
*
0'o
0-0
0Ii
II01 Z
11
C
'7
z
<-'I
0-
0
0
0-
c-f
II
z
0
0-
g
C
CD
10
CD
C
10
*
*
#
#
#
*
*
07-
*
0
#
*
-
#
*
*
-9 0
0-
9-
0
*
#
*
0
#
00
10
"Z
II01
*
.
'0
0
#
*
0
0
0
01,0
--
.
0
0
0
-
0
-
91
0
01
0-901
-9
Z
0
*
0
01
'0
0
#
#
*
o5,-
00
0o
-010IZ ,
o.
*
0. o-'0
*
*
II.
*
01010 o.. 0-
b-f
0
*
0
*
*
z
-
0-
0.
01
~r
0II
r-
C)0
0
#
*
00
o0
*
Ot
*
#
*0
4. 0
*
"-
00_0_
00
*
#
*
#
#
ca,
Q- 0
--
0
9
0
- H
0
11
Do
'
II -
Z
O7
0
PP0-
9
Zo
0
II
0O
*
.
_.
*09
001
*
0
o.9
0
o
¥
+
.5
U
+
+
0
[I
£
*-Ca
0 0 Ca
"'.0
0
0
%
Ca
%
0,:
C'
.0
II
IC
E
1.)
0
-_
v..:
0
U
.
v~-
._ 0
I
II
Ii
~' 4 X, =
1II >
-
0
0
*
'
; X c0
21 W0.5 v
0
j-
Ca
*
*
*
0.
0
*
*
*
*
Ca
*
*
*
*
*
*
*
*
*
*
-
'-.*
*
*
Z'
w
0
A
I
*-
+~+.o>
0
11 -,--
0l
+'*
0 *
"-Cl
0
Cl)
O~j*
.2I*
*
*
0+
>
...."0
._
C.)
.C-
0
..
=
~~,~'
._=A
oI -Q .
u0
;
0.-mi
,
~'''
.0U
.....
o
0
11 o 1
-
S u
H ~
II. ~'g
V.0
0
0
0
.5
C-s
0
~
~v
~4- -
-I-'
U)
C
C
C.
C
~1-
.~
V0ii C.ur-
0
0
*
*
*
.
+
=-
Ca
0
*
*
)
.0_*
0
..- '
*
0
,
+;
' U.
-
:5
o
0
0
I0
*
UOO
I
*
*
Ca
v0£a
0
.0
*
*
*
*
*
*
a)'-'
*
*
*
*
*
*
*
*
*
-
+
.,,.
*
*
*
*
*
*
*
*
0
o
>
C-
*
.
V..+
-
v
%
##
##
#
Ca
U
.
I%
I
-'-'U
C..
II .. : .
o
U
II
x
t-
N
0
o
C-
2
0
o
a)
-0.
I:I
.. '
I
^~ 0
-:
0
0bO
4-4
'4
0
I
h
'
.~
I
D
-A
O
00-
2
0
Cr
.
taD
.H
Ca
I 0
ODN
~
a)
0
Cl
I -42'
a]
O
=
2'
E]'
t
'^p>
>)0.
0o
C:
%<AR.
0
. .5
U
I
*
Ca;
>
0
Ca
,EO
_ 4>
PJ
"i'-
Ca.5
III
>'I...
50.5
000
I
0.0.0.I.0
000'-'
1/)
+'
-' -'
C
00
0.0 C- (0
'CC 0.
0000000
> >.1-
I
00
014
0-0vvCd)
a
00
10
CaCaO
,:
10Ca
O
2
(0
v
4
oa~~U
-
a
_.
.
D0CN
II
0~~~
N
.0.IC
O u
Ca-Or
Ca
'
XIO
-
0
0I .- I
11 -2
IO-
0.;
D*_
2
)
a
v
.0
IS>
'-'0
20 V
+
V2
-0
0
>
>
>
>
nD
4#
o ,-,
. -0
, -.
O~q
I
o:
<
4
o
0
*
0
0
0
-s
,(
0
o
..
0 0
II
1++o(1 00n
m
+--~11 I _.~
n
-.
4
I
-e
#
<
CD
I
vv
14
*~
0*
I
4
0
Cl.
4
4
#
5
J
C
00.
Z-
o
CL
0-
*1
41
00
V
o
o,
#
o~~.
P
~
CD
CD
4 .
N
m
0
-%
-COI
2r ++
.cr1 0c
~
0
V
CD<
~
t --
<
<++
0.
<.--
CD
0
~
~om
-i
t
0t
1
+ --'-+
I ~
,~o
t=
CDOO-
00
I,:
+
-C
7,~
Ov
-
~~~1
5*
I
II
4
#-
4S
+~
V'A-*
V
"I
It±
no
~~ri~
+
~'~A
O
<
I0
*
4
p
~~0 ~ ~*0
-4-~~,, ---
C
II II
I*
I0
*
-.
I
I
-C
4
0.
~O
O
*-
#
OD
0
_0
0-
-~
CD-'i x
_.o
0
Co
,.<.)(
-=
oC
Gn
04
_.
_.
*
-0o I
'-,-.
0~
-0 >
114'
I
n0<
-
M
0_.
4
na t
0
-t
P"
I m
-
01
'
4o .
\~ II
C'
-
Q
_ 9
P
Gn
.
0
IC~
0~.'
0 .-< O 8 -"
0411
0~II
: 00co0
-
0
*
I
+0 0
0_, -2 *
O '
nA
P
11
GG
1
0+
nC5
o
-
.
0
-4
4
P
4
4
4
4
Z2-
#
A<
I-
_*
4
Q
4
00 XD
-
1,
°
t
4
..
4
Q
Q
_.
II
W.
#x.
4
4
4
0-
_n_0
-
I
IQ.,
0.
I0
0
A
0
_. ~
+
1I0 *I
0
0CD
4
k+
4
~
+
I 0-
II Q
4
00
+O
0-
V
N
I
OP
#
0
CD
0
.
--CC
Gn
*
#
IT
S
4~~~~
0
_.
0
C
It
11
wGn
0
<
0
oCO
z
0
11I
II
O
DC~
DC.
C
Co
C
tCl
o
CD
O
C
CD
0'
N~
e0
±
+
0
©
0,
-z
rl0
U
U
0
0
Ii
-:
e
#
+
70
N
>
-
*
10
.,
0
dV
-
E
U
.Z
o
rt
4
HH
'~'
S j. c++
~.~A
C
>> 0
-C
0
UD O
-0
d
0
0
-
..
O
~=~1~
io vr-
O ·-
A
r.
0
o
-0
00
I
C
ooV
..
I -1
, u 0<
-H
v
#
#
ro
00 ·
1~ * I
II
4
4
4
4
4
4
4
4
~
o~~~~
05
s,
04
x
#
4
#
4
0
o
"I'
N'
#
o
U:
0
4
#
4
000r~
Es 0
-
~P000
C
4'
#
o
C
CO
*-00
5'>
IId
4
4
4
4
*
4
4#
4
-I C'.
0i
4
4
II
0
0
#
#
#
4
4
4
4
4
d
0d
*
4
4
4
4
4
4
4
4
,+
a *
UY
*
*
*
0
.0
* .
0>
0
4
*
#
#
4
4
4
4
_
r-0
C
*
*
#
0
0*O
#
#
#
II II
Ul
*
#
*
4
o
N0
CA
4
o
II
o
u
a
4'
¥
4
a, II
V~
0v
oF
o
II .~
1-
)
U1
O 0
%4
4*
°0
4-
L I I
_0
0
4
O
4#
4
0
* \;
0
4
.
05
4
~, ......
o
'-
0
*I
V
*
4
-
__I
I
~.,r
_I
>
0
IO
i
-
+
*
b
4
4
0
*-
II ~
E
I
40
*
00
1
*
o
-
o
'0
O
00
,
F:
>
.
O
·n
#
o
*
#
4
U
3
-5 _=
~
44Is
4.O
0
bO
0
U]
O
_
1-
._
4~~~~~~ 4
.-
o
.
o
H-
0ua,
-n
4
>4'-
a
ed -
.5-05
+0
H--U0,~ +o E ._i
II
>a4
La,
· ,mo
4
4
H
4
0
4
44
#
I
o~
*##~
>
Q
'+
+
-1
a
0O
A
3, E
,
o'-
t
eel r0 .cl
al
#
=
0
0
4'
II
4'
5-
0.0
0
S', ii6 o0
0-05
'
##--
-...
44
w
4')
4a
.
-;'0
o
U 0 1. 01
Il
>0
"
M
*
4
*
4
o
"- 0r
0
r
3
i0
,.,
PC
PC
r.
I
PCtP
4
or
e
-
I
t,
c~1 II
PC
Pc
4
4
III
1r1
P
~~R
II
0-0
11
4
PC
M.
11
9!
11I
44*
C
O.
0Al 1XI
1
4
o
"C
·D
(
~.~
v
#
%
4
4
4
nI·
r·
.0
r· 0 t5
aaaaaaD0-0-a
D
CD
O
PC
oCD(D1< <CD CD
I1
0QP
X PC
CCL o
C
#
4
4
0
0000I· Ha
C·
w.'
4
`X
*4
*4
*4
b- , c
oooooooooooooo
4
4
.#.4
4
4
4
4
4
4
*
4
s
_
_~
3D
CLCCC
-'
.~.~'
C .
.
PC
CI I 10
r·
r
0
CD0
' oO I PD 'oP
2
2
(0
Ii
L -
~
~
/>s
()09
4
CCC
La r·
i
P
~~
c\
a
I tp
_
.
4
4
c
4*
%
4
(D
4
*
#
4
o
4
c'
c-
I
Co
co
Cl
ocn
#
<
#
#
##
I
4
-C
c4
a#
#
#
0
Ct
4
4
4#
0
CD
x
H
#
#
#
M
Z2
;3
C,
"I
0-.
0l
4#
*
P
4#
0r
a#
#,#-C
#4
#
#
4
#
4
#
4
#
4
4
4#
4#
4
#
#
#
4#
PC
M
Fl
#
+ P,0
®~'
~.
1
o ,. o t
c+ 0o0
P o
, -e
0 to
DI0-,-5
2
O
Or O"
O <)
"c
,
O
l
B
'
C
S
C
O
P
0-C-~
o~X0O·
0 Ca
0 fa
cn
_.
o
-
I
4
4
C P9<,
p-t
13
CO
_
Ic-C
22
0
tJ _,
S
I
c
a
IC
0
*
+
~-*
l
_
c>.
X
.
_.
_
O
+
,
.a
0
o
c
P
0-
0
0
o
PrC
.
-
-
t,
_0
0
CO
.
-z .C
(D
W
1.
IC O .10
Cl cF
g-C *PC
+
.
0
o
._.
0
.
= T
0
..C
C-4
0
CC
_.
N
-L
on_
=CO
-
0
CO
30
±
on
.
N+
0
_
+\
-1
.4
0II
PC
N
ND
m
U'
CL CL
P
IP
-
2
I
*
0,F
0,~
-
023
i
II
OPCn
2O
±
_c.
'01
8.a
r.
1
0p
O
o D
a
Lz
Cf ,00,
0-,
0
ve
'Igx
0
)
=-E
9
<
3
- Sh-
.
H
2 ~
*
+,
B<
-
Iu
~,o
I !5 _.2
tIl c
_l
L'
1
PSo
- 9o
*
PC
2
PC
.O _
co
I
C
0-
4
4
0
1<
70
c+-l
r
II 0
2- 112L
A1< 2A<
++S'
+
2
l~
00
re
00
C
I
II
O
-F
dF
_ . _.
*~
=
0
_
-.
,
00c
O--~
0
-
-~
I
PC
A,0
-.
O
4
O
R
O
_-
,H
4
4
Cf
j
rX
#
o
_.
00
'.
#
o0'
0
o
.
0CD~
Z!
-1
CD
cC
-
0C
+
+.
0.
0
.--4
E
cq
-Y
I n
o
¥_
II
0-
a)
,,
_
#I
0~~~~~~~~~
'0
C"U r
**
**
*
%
**
.
F ,*
_4-=
2
a)>
#
~
,2
-~~
X'7-- P
2
#
Q
#**
*
4-
LH,
~
~
~~~~·z~~~~
~
ed~ ~
~ ~~rd
O~i
C~
~
a
,
-0..
**
. **
4-
0
~~ Or
~*
C.
*
~~
*
LI
2~
o
++
+
*
*O
-"
a)cI*
#S
4-
I.
--
Q'V
*
2
0
r.,
$j. V
--
e
l
*a
II)
0
0
cI
CD
C
a)'
±
L9+
a)
-I
a)
3
0
le
=
~
~
~
a)
2
a
,
+0O
0
IE
V)
-*
2a)
a
2
E
c~~~~~~
IS
_
oH
,
0
cdn gE
1
a)D
5,
rl
a
s
rx
-
a=) >
O
'C-
O2+
*-
O
eC
ac
10E
C-
a- C','-E
2
a
,
Cl
II II
-
I
I
o
C-a)~
.5
+
{
a)
4-
-a).
C.
w
-
4
*
~" '
I
1
t1
z
II
O
_~-;
0r
5
5>~
- i o-
O
S
0g ~%
g:1h'~
_.
II
o
0
o
-+C.
o
I
II
-.~-
U0 -'
II
.
NI
. .
P
*
0-
*%
#
*
A
2
a
_.
*#
#
0
;2
*
.~
+
*#
*
.o
Q33
Ja
cn
0o
**
I
#
0
+
i
+
_
.c-
#*
%
o
9:
O
m
0g
#
-
;;,.
0I
0v(D
II
0o
a' 4
*
*
I
*
(p
cI
0
I C00aI
.
o
CD
mo
o
#- %
*
0
ca
_.
0. _i
a iSI3
'7
U0
0o
0
*
1B
+:3
*#
<
.,
0
C-z'"
9'+_02o
o
0
oo
%
Q
o -
-. , ,-.
~
-X
,._7_
A-Ao
5
o
I0nI-
A
o
O
v
CD
I
(
O.
N -
o _.1
I_.
I I 11
00+~*
S
S.
I-I-
,
-2.-
.p
p
54
L
-
00
0
z.
9
II
CPcnr
71 wI~
':5'"
Cn
C
O
C~
I
E3
C
0
0
0
e
v
0
0
1
-4- 4D 4
II 5-
I-
0 'd0
x 0
-
~.
CD
I
I
o
-.
-
.
II
+-
IC
:
C
+ I1 CC
1I
r
-- 0
CD
0
~-0--
-·
S~Svl
-,
cn
0-
0
00k2
,
;t
IS
A
0
f
I0
21-0
O
nII IIn
-<
-o
4
D
rF
(D,
x
I0
II II
W0
II
I
I
In
00
44
t0
.-.
0
5
000R S
.
1_,3
I-
-
5
0lo
0-
m rn
0-
cr
fP
O
CD
O
8CD
0
A
II
>
A
-
'!Z
.
0II
U
>
*2
.
=
.. E
0
·
I
4
*
*
4
*
4
4
4
*
." w
"
cN.5
._
0-I
4
'~av.II
~~A~
E~
'
*
4
*
4
*
F4u
A_ "
L
.
t
>
S
~ .';=
.;t
_
U)
s,S
U)
U()
X
w
0
0,
*
U
. I
c
0
0
-
.
~ -Z ~ '~
I
It
CI
44
*
*
*
X.-
0 0 .)~
:5
4
-C!
>1
4
*
*
*
*
B
o0
..
r*
0I
4
+
% A <+
*
*
*
*
*
.;~~~
*
(
+
4
11
I
A
n
o''-
U
4
U-0 Ic
'-'-
04
C
*
U
U4204
.
4~
.
.5
4
~
c'^
Q
a. - > uz
4
*
*
*
*
II
_I .
C0
0
4
/~~~~~
11
-i
4
2~~
+
+-.-
09
0
4
*
00.
=
.
u4
+
0
00
U
-5
-,'-'
0 ~
U
-
02
0Z
0.
o
4 I '>00
00^
oU.
1 ~
o,
0
' 004. Z
0z.~;
EQ-a
0'-Z
-
z
C
S
0
U
A
0
0
U
..
II
V
=Z
UoQ
~1
4
4
#
4
#
4
QD w
0o
'3
4
U
4
%%#
4
#
4
#
4
U
0
W0
0
,
#4
#
4
#
4
.-#- 4
#I
0
S
*
o
o0 WU
-Z
US o:o
404##
W
;
4
:*
b~o
'-
*O
*
A .
>u
~~~~~~~~~~~~~~~~~~I>
_l
H0
=~
U
Z
Q
)
0
'*
-
II
4
_
?
4
--
#
4
#4
*
!!
0V
I,
0
._0
'v000
*
bO
-
O
4.
2
II
*~~~.
**~~~~
# ,4
0
#
- A
~;
-- . -
#
#
#
#
4
4
-;
_
.~
10
*
#
4
4
S.0
#
.tv;
0l1
>
#
#
0
=*0
0o
#
#
#
4
4
m
-4zc)i
_
#
#
#
4
4
4
#
#
#
03
4
*
0
4
~Q~
._
V
.
4
.=iiv *.SC
._
o
0
U
ce
A
t
4
4
0
10
-D
#
a;
0
#
4
4
1
0
I C4A
0;
0
0.
4
4
#4
L,0
C
4
_
._
._
o
*
-0-
-,,,
0
0
-
-q
00
-
*
0
0
20
(C
I(
8
(
<<00
0
i'22 2
vV
(C
0
*-
~-S:
0
0
2IA
0
0
0-
1
(IC
0
~~~~~~~I
,< -. +~+*
._x
0
0
0
(IC
0
0
2 ±
II
0
0-.
-
0
0
2'
00
-.
0
0
A0
0
i
i il
--
l <-
-- -)
0
8 8
(-I
II
_0_- II
0
(IC
(IC
0
-
II
0--l
2
2
0
0
-lI
II
-
0
11
11-
0
+
0-
0-
±
n
2
0
.111
II
2
-i
I
I
II g
n
0-
2
0
n
I)IIII '<
-o0.
-
'+
-.
-
(IC
2
±
-
0n
II
II
II
0
0
0
0
0
0
o
01
0
(C
'-0
o"'-'
-A
-. II 0-
II
Ho
0
0
2
2
+n
I-
-<
£-
---
0
0-
'
+
<<2
0
(IC
0
-
0m -
0
-
n
0nIIH
II
0
'-2--'
0
I' IIn
0
II
0
-F-.
-*\[
CC'+I
-'-C-''
I
I
(C
-
+
Ii
n~~~~~
~
00(C0
0-0-0
0
00
nn-i
2
o 0
0
00
o-
+
o0 0
<
0(1(0
4
00-
0
1
l
D
0
0
0
0
O
0-1
0
n
(D
0
0
0
C
O
o
'P
O
*
-r
FS --
2
02')---
-.
0
-.0
~-> <n ~r ~
.~o.
C/I
o'C
,Il II -I
0
V Q:
+
II2 ,
_.
,
CD c
m Q
S.
~0
±0-
-F
aItO
_.
(C
_.
on
C
a
_2-2-Q
P(
_
cn
_'.
ta
0
P
+I
_ I
11
V
I0-
0-0
l 0
00
P
I
11oi ~
-F.
". r
0<
U)I
F +
O
-U2
*
_
.
O
+ +
(-'(IC
'-'0
._.
00
n
I1
11
_~C'
Oo
-I
P
4-.
(IC
0
C*CC'-(IC
00
C-' CC-
0
o'+'-
<<00
00 000
-
0
(IC
(IC
0
nn
0011
(IC
0
0
'-'
'+---'(0
2-F
±±
j
2
-
-
0
0
-
<
'1''
o
'4
0
0
I
0
2
Z._
(: .Up
00
0
0-
11
0
0
n
0
0-/
0
o
0
_
*
2
±
0
00
0
00
+
00"-'
i'41
o-'
II
+T
0
0.
_ Z
<
0Z,
-F
0
0
2- - ii
2~
C
0
2
o
C
I-
C
0~
0
'31
'00
U
0
0
--. 31
s-.0
0
0>
0
i
I0
0
I.
'*
=
'
,1
_.4
,.
,~,-~,~
.H' _
I
.3
4
03,
4
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
C-.
t_
Q_
IIt
0
It
.
Iizi
~
J0
_I
.
3°
3)
43
132
Id
-
' 11
C
II
'
,x,
I .-
-C
- >
3nU]
.
_
7X
U,
a)
s
v.
0Qi
2-1
~1
M
.
.
I-
0
o.
_
c
I
U
310
0
5.5
4.34.3
*
*
o.
'-0
0
:z-
~e v
~' 0
.03 - -
0I
U.-3
-
.
V
>
-
0
00
3)
0
-
0
o
**
0
LI
3
8'
>
Vc
*'0
'0.
o 11io
4.3
~o1
,II
2I1o
"-
4.3-4.3
c
°
*
.- *
*
*
>
·
_
..
10
5
*
*
*
*
31
4-
U
4.3
*
*
*
*
*
*
*
*
*
*
390
.C..I
0
*
52
.
U*.
00
.
*
00.
0
*
*
-z
In
a)
31
3)
*
*
*
*
C6~~~~'3
04It
0
*
*
*
*
*
*
*
*
*
*
*
Id
10
3)
'.3
O~
liii
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
-0
'
Z;Z
'
*
*
-
*
N
U
-2
iiE
C_"
0..
C
C
o
Z
II
>1
*
z
a)z _Oi
II 0-. 01
-1
II
#
#
#
i
0
0
33
*
#*
a
~
#
.2. I
004
bt
*
#*
#*
C°
-
-
0
0
0
0
0
W
#
z
31
0
:
#*
-
#
#*I
#
#*1
'*#
#
#
#*1
#
0
0
31 0
0 -0
5050
0
0
0
0-
0
03'.o0o
3- o
31
0.
0
0
0'
.
#
3)
0
*
*
*
*
*
*
a.
2I
o.1
0
I
*O
1343;
*
.
u
*
.
(03-
-°1_;
1
3,1
;> ,_*
3
_2
=
=
v
U
Q
2'2i
r uN
0 0,
,
2J
-
*,
5-
43
/
II
2
321
5(
0
S,
o0 O
22
2
-
5
,,l
0*
_ _.)
=
002
._ .
_ 0 0_
*
I
113
*
*l
3-s $0
300
3).
H
I
13o
0
-S
E
13)
' "
E;2^
#
#
#
#
#
#
#
##
#*1
*1
#
-
0
U
33
1
'~X ,II
;a^~
#
.
133
Q
'.3 -.3
3,3,
33
*U
*
#
*
#
#
31 31
-o
°3)
*
#*
3100
-Sd 0
31*'
.
@'-:
0
~..,
i i..#
13 0
._
.U
- 433
o
*'
.
.
O~
0Z O
._
O
_
.
5
40114
11=
0
.
:
~
°
°
-<
O
'~
~
I I*
m
=00
tt'
t
<
D
0:
.8
0
1 > I 1°I
V
.o
n n, n
co
C**
a
?
>
1
I,
s-
r
tQO
*
+
+9++
~
~
-~iII
' ~,.
0
0
11
*.
<
0811Z
_, I
C)
0-
,
0PI
0
0
=
0
<
9I'l
#
±P 0D.. n
0~~~~~
0c
*
(D
°
_
.
.
--
1~1
~
PII
11
0
0
~ ~
0
a
9
CD
IPI
10
;.
_
0_t\
9I
00
a
-
0
I
o
0_.P
0
<~A
*
o-
a
0
ma
11
En
N
Nr
c-
t
1
g+ ot.
II
3~1 a+
·
1
ao
S
S
~~II±
a
-
_
_._
0
91
#
3.
L
0f
cL
+
II
0
II
r
I
C
C
1En
Oo1
r
1
0
°e
_.o
I
Z
nu0
+<0
y
Z
*
10
0-
*
0
(0o
*
#
C
.
0_
0 '-Z
OO
0
'-
0,
-8
<
*
*
0
9 9
o0
z~9 9
*
.
0-
10
0
10
0o
0
9
;
0
_'.
*
*#
9.
n
9
O
0+
V9
V 8
.
119
00'c
-*
_ .
+3
4
F0
I0
0
*
0
4
#
'- I F
-
±
*
Fr
o '9
It.).
0*
1 0,
0f
O
'
x
no
9
7-
'-0
IIA OOA
0
*
o
C/Il
II
0D -"-*
0
*
P~~
x
-0
c
*
*
*
*#
*0
9
-0
(0
_I
CD
*
Q
*
-0
*
*
*
*
#
0
* '--
*
0
0
c
2
1*
/1
xe
#
1i
0
*#
10
I0-·~
0
#
*
o
II
*
CS
I.D
Q
0
0
I
-
"
0
9
9
_.
,nI
0
0
,
0
0
0r
0
0UN
I
0f
n.
-1
o
oC~
cn
O
o
O
0
M
CD
ms
c
o
o
u
cq
>f
0U
Cu
N
c.
s:
S
0
ceO
*
+
>2
II
0.
S
*~~~~
c,
0Cu
0
lu
rJ
Cu
CD
~ t T-
-4
XJ
o
II
oCH
Cu
m
Cu
0
h
o
Q4+
_]
Ce u
-
~0
,,*
Cu0
-U
+.Ue
3 a
o
-
1,
4
0.
4C
o
0
"0
o
V . +~~~~~~~~~~~~~~~~~~~C
x
0
I
~0
H
OU
o
Cu
-
#
o
e
S
+
,E
e
C-a
00.
+
Cu5
V
Ce Ce +u
#
uC
e
*
I1+.+
+
. ,
+
W
*
bO
o
-I:,
.0
.
t~-5
0~
0
U
O
o
C~L
,, V
u
N
0
U
°
'
+
a,
C.
V,
o
0--i
1
)C'I
0e
0
0
-
+
I~-10~
II
II
0a
cu
~0
-C0
+(c
Y"U
0VE
II
0
oS
-o-, ~:o
c.
a
0'
e0
-r,
0o
0-~
"Sc
II
o
C0
0
_0
S
.'
-c
S
4
--
- .
-0
gl
·
i-
o
0D
cq
0o
N4
0
N
0
Q
.,
0
Ce
11
.11
..
.
0
0
2'
e
It
co
4-C
~
00
0
*;-1
Ce
a)
Cu
h
00
-C)C.)
.e_
.-
Q
_
t
"'
,
4
4
Sd
0~
C
0-eI
i
+
E7
1
*
1* 4*
F
0
4
0
4
0
0
0
4
4
5
4
4
4
4
C
4
.I
Cu
-
*
4
0
4
4
0
4
4
0.
0
4
4
4
4
.
0
0.
*C-C
4
*-4444
'
.
4
4
a)
0
0
'-'-'
-
*
0
N
C.
Ce
>
0
0
0.
4
0.
C
00
Cu
4
0
Cu
0'0
0
4
4
4
4
% UO6
.,-?.,-
o
0Q
N
C
:
4
,_
C.
Ce
0
0
II
i
X
3
0
:6
0
r.
4
0
4
:
0O
6
#
*
.-.
4
?r.
0~~~~~~~~~~~~~~~~
4
4
_
5~~~~~;
4
41
C
Z:
·
'U
200
,
-
,
'.0
0
C.C
--
-d.C
-
Cu
- W
0
00
.
o 12 ,O
-34 .0
g.O
.u
,,C. _._'x0
.(~30
0
r
C.
Id
._
E
#
:4
,1
n
C
. ..
I
1
na
e
-.
A
e
e
3.
_.
0
'-- *0OCD
CD
-,IDIDCD
DID,-----'
+~
-
0
80I
00
-4--
I;,p
*<o
l
oD
oCr
oX
Lpi3
O°
0~
-.CD
'C
+
±OOA
_
D
I
X
11
CD r
±+
a
--
0'CD'
I0=
-
S-
0
2aA
© -_~w.
c
*
D,,
*- *X
*
CI
<'
Q
"I
- 0
0-*
CD
O
#
0C
##
I
#
-
*
*
2.
C
n
* 0r
*
n
*
*
C
*
m
*
*
o
II
?
i
I
>j r
o
3
S.
S.
+
SF
.
_ a
DI
II II
OX
+
110
D
*
mW
I
0*
-z
II II
_.
#
*
*
*
'71
r
#
##I
0I
F
,-
NCD
CDC
1
i
-C
·
#
I
Om
_.
"+
*
*
0'
'71
S~I
O
*#
*
*
x
-
I "
00
S-
o
CD
|F
o
0'-
0
0F+
°
'i
.
cn cn
It
+
0
C,
<
°
c
CD
O
0
CD
CD
D
D
DICDCDCDCDC
D
Itt
DD
CD CD
CD,dCaCD dCD
.
.
.
CL
II
0
o
0 0 0 0 0 0 0 -'
*--. I
*
*
*
*
*
*
-x-*
*
*
**
rp
,,, x
.'
o
'
*
*
*
*
*
*0
JJ
5
c
1"ID
CD
*
*
C
'
·- sl~~~~~~-
*
< < < < < <
*
*
-I
C
o0
*
*
*
*
3I
*
0
CD'-'0-I
_ .
aM
ca
2.
Inn
i
11
CD
Q
'2
0
*
*
pr
*
Q.
0
*
0lz
*
ICD
0~
0-
*
_ .
DI
I
*
00
'0
I
c_.~i
~.1
~-~
_.
'0
*
*
*
*
*
*
*
*
**1
Z*
*~
`*
*~
D(D .IID] a
*
*
*
03
*
*
-X
--CD x'0
CD*
0CD.
I,S
0000000~~~~~~~~~~~~~~~~~
.I
DI
C
0F11
C
CDl
I
i:
CO
5
C
CO
C
C
In
C
Co
rC-
0
o
C
0
0
&
41
0
0
,,...t
a)
N
US
a)
0
O
'
..-
0
r H.
-
#
#
#
if
2
<i
Q -.
I
c
#
.-
O
%
if#f
t1 X0+0
A-11 I
4CN,
*
*
*
~l
of
H
C:
#
#
#
c~
c
o
.C
,~
H
#
#
1Ce
N
0-.0
i
iifci
i
if
#
0
0o-
S
i
*#ifi
I
reif"
t
if
C-
;
0
_
a
'-
0
~CoWr~~-"~)Ceif
V-C- .
-:~
"0
iQ
i
'e
CA
#
o
#**
if
ifif
¢
if
if
f
'-'-0I
0
if
#
O-~
1;
*
C-
0
+cCe:
C
._
*
°
i
V
b
w
O
"
-,
C
s
>r
if
i
O
24,S
"0
-0
C
~
t
ed
7
U
if
if
.:
S0 C
V
.-
a,00~~'-p
O
#f
if
if
if
·-,
C-
C-'
#
-2mc
x
if
-0
#
if
-
o
.:=
xx
_
Ce
-"0
#
*D
.--
#if
~ o> Ccg-m>
~~~~C
d
.,~~
~
0
.=
#
if
~ C-i [::)..
^
*#
-2
C
#
,*b -w
Q~~~C~i
II
S~~~St
if0
#
if·
if
#
if%If
H
Cf
O
C
06~
VZ0d~~
CriS~~
a S
o
U)
CN
Cq1
c
o
o
0]
o
:J
o0
:l
4
U)l
H
I.
-I
42II,
0.
.,bO
0x
0
.
0
o
ov
0
:
0011
z 'a
-~
oo0,
**~
-
U
0
i
C,
.
C
if
*o C-O
if
't
if
.;
*if~V
C
£i
.~*e*
. C
Cd
Ce.-
2-2
g>-0-
if
Cf
~
CCC
C,)
C-CA~
s'mfC
*
)&H
,
h
,
*
2 2* ***
~w
--
r~
x
0o
O
O
-
UC
Cm
UU
=H
>
3~ IZcH~(
;
l II
'
Cr-
C-.
C e0HZ
0
0
ec--'
""
a
,-2u
OC
0
H
c: .s L
l
CeoCC~CC
Q.
JH
2OH8a
-H
·
C-,
'-C +.;F 4-
0
CeE'
4
2--'
4L~
IC
I
..
H,,
O
*
~~~ Ia
- Sea
if
-) (-)
·
-
X
_~.
.
*if**
* :::I _, ~*
**
if
"I
-;4
0I
'
5)
,
bO
H
"
z
if1
.
0-3
*a
ZI~
X0
._UP) Z
"0·o0
Ce-ac
E .'
> > >
-
E
-
00~--
Ce~'Z
2
e
~a.
C"0'
V·- >er?
C EC
*
0 "
z
*
*
*X
*
00-.
O
*
O
n0
0
0
ce
*~
m
11 -*
o1
~', o o'
oq
po* Gqo*Oq
*
0
**
Q
I
o
?
*
*
N
**
_0
0
aCO
o
z II
-*
~
-
,~
~O
0
I
*
*
Se
*
<_ o°+
;l'
*1
0
<
.-
Q
0.
0
.
-'-
I
0
*
' :~ ~'11 ~
0
°
'c~
O
*,
II
*
o
0
**
*I(
o _*
0 _.
'""oCII'
t
c7'
*~
*X
mQ*I(
0
P
*X
_.0
0
P-O
z
*
~AAA£A
n
I0
0D-
0
a
0s 0 0 0 0
*-.
r
-I
O
7'
0-
_.
< 0-
S _ o
01
C-
.
o
-
IN
,0
0
0
-~
e-H
0
0
oe
-
0)
Oi_
I0I
o
~~~~
0;
0- 0
*1 UC
-.
N.,N
I1.
i-
z
o
-.00m e
*
0
s
0--
~
0
*
0
O
+0O
_.
-i
~
5*
0
_.
-P
0
0
0
n
o
00
0
0
0
0
0
0~;
,.E
°
3:- B
I
°- A
o
P
° I -° P p
'
. I
o
5
'li,
'
S
0
.)0
o.
II II II
o oDo
AAA
.
A - -1_
ur a
a
0
88S
©
o
0j
'.85
-.
*S
01,1
0Dr
+
11
, B
o
-A
I00
)
0D
~1 i~
I
0
11
-
.
.
0m
_.-
,-'.
-
00
oD
In
-
Fh-*'
o :
s
_.
-
2
s'
+0-0-0-r©±
o
0-
.ZoII
I
~a
0
.
0
*
-is
r-
or
2.
Ir
In
o o
o.00
>o
~_.
~o115.
A
'-1 I/ 1
~~
_1_
1, -
0
0
I
1
uz
_z S
0
*
:
5
+
0-.
o
11 11 11 11
p,
wo>zz
0
_0
*0
_
I
7-
I I_
._
+
.. .
_.
~o
+o
0
*
0
o
<o
-
0
_:-
0
*
N
N
N
0 0
0r__
I0
0
O 0
C'
g
0-' 0
o0g
n
*.
CD>
O±0©
o0Q0
0
0
C) p
N
0
vii.
0
.icI
0
0*
0
1
0
0
--
0
3
C
e
*
*
*
*
0
>_
*
*
*
*
*
*
*
*
000000
>
>
C- CC. C.
C
>C.
C.
X
0D
C
0
cq
Cl
0C
0_
O
o
-x.-
II
S
U)
.
.
-,
V
0-
+0
0
0
U Q))
0.
·[
-
0
0
0
·
0 _^
C
0-
.v0
41~~~~~~~_
-0
0
00
.
j,.
¥cb,
1.±
U
II
0
-
O
y l~
-
-~
-
0
C
X
s
_,
,C
o
0-0
+u
V0.0c
o,
O ..~
C
>
0
- I
4-
11
l
U
Cl0
C-
I
00._
0
40.O a)°
+ C.O
II
I
'0
>
.0C:aC
- t Ce Ci)= CiC
#
#
#
#
###
#
#
#
#
#
#
#
#
#
#
#
#
#
####
##
##
#
#
#
#-
#-
#
*
#
#
#-
4-
44-#
#
4-
CCC
#
IL)
>
CC
0.
S
.~_
._
E
5 o oX
*
*
*
*
*
*
0
a;
0
lI
U
#-
#
#
0
.
~.
##
4
4-#
_
4-
4-
44
o
>>> >g>
QC
C-C
CC
)
tD
Q
a
0.
0. 0. . 0. 0
*
*
*
0 0
0
*
*
,
0"e*
0e
n
°*< * a
* I0l *a0
-
M . 0
n
0 O
" a
-
0 n n
0
"
I
m
U)
0
a
*X
0
5 -r I 1011
II .O 0
r~ zc~
""00
0-
P
0*
U
*
0
Q
I
0
*X
*1
10
0
00
o. 0.
0-
93
0
0"
00-
00
o-,
s"-
0
0
ca
0
0
0~
0-
0-
I
_,II*r
0o''.
-
-.
9
*X
0
-i0
0!
.
z r
(D
1C
o
co
0
0
0
a
000
e
0v
o
0.
-i
0
-.
.
0
o00
0o
0
:0
0"0
0e
0D
"I
000
-i
*X
II
Z*
Zo0
0
ZD
#
10
.
0~
o,* *
*
#
o
0
0
j
.o
*
0-10
"I
.O
I
0
#
C
0-0
-0-jo
*X
-*0-0-
r0
,0
I
01
0
0
*0r
*
*
*
0:
'"-
-
r0
01
001*
z
0-
'
II
0-0
VI-
*
CO
~t
0
-
11
.
11
C
1*
00
~
II
o
I4I O
ql ~o
00
k-" *
*
_.
-o o "o
"0"0"0
0
e
0
i
~"1*.1
*
_.
*
0
*~~~~
°
CD oz)
c1'
a ~
~
* *#
o0
~
~
o o o
o0
_. o_. _. _
o
0_
**
0
o
0nC
*-°.OS '
0
0
-.
*
*
0
0
*90-
*#
0
P
rJ
(D
0
00
0
0
0
.
o
0_
#
1*
O::0
0:
Z P
*
7
I
r-
0
0n
0-o
o0.
.
0
_o=-*
0
1:
0
*
00
0-
C
-- 0
.
*
"0
r-
0
0.
000
*
P,
P, 1
C~I
0'-.~ '-0
o.
o. _.
ce
o
P
_.
*l F
*
E.
0
1+
- 1
0
,*
0;
o
* N
.
iI
B
N:
O*
o,>
.
_.
.*
co
CD
o
0
C
0
0
CD
bC
0.
2
C)
A
o
0
C-
0.
+ +
-F-F
0
.
a,
N
4-
2>.
*
*
*
*
*
a)
2
Ca t'-'
,.I
a,
-
0
;::,
'4
1
U
-
0
0--'
Ca
2
-I-
z
-
I-
.
'4-'
o
7
a)
0
0
CC
0
.- ,
a)
-
00
ii
,
0
0
*
*
~
V-
0*
0.04
.
55-~
)
1
0.*
i, o
-w
,
Ca
.
*
0
*
0.
*
0
2I
*
-
U
0
C'a)4-'
O-
0
E~2
2E I
U
,O^ 1C
_I.
-
o
EA
o
g4
'1
V
1
a,
0
o
a
I
o
z,'
'._.11b.O u +
0
a)
0
0
II
; _
I11
a)
tCc
I'
'U
0
u
0
E
-
I;,(D
UI'-
.E;
40.
Ca
-4-'
o
I-.
I
'
U. -
I D
.E
o
00I
r-
N
U°
)
.'oo
o
oa
>
wI
I C
N
0
.
+
0
*0
-,~
Ca
0
a)
.-,
,
*C"
II .-'
H
*-0
00.+.
·
~~.+
v-F
w
CU
J
1
° C
01
--.
*
44*
*
*
4*
444444-
0
U
1
II
0
'-
U
._
V
+
-6
0
*-2
0
*4-...
*
*
*
*
*
4*
*
*
4*
-*
2
-0 U1
* 2 ,,
4-,
'
C'l
U'A
.2
*3
Ca
4-
0
II
2 j
0
0
Ca
'-'
-'
0
4-
0
*
*
*
4*
*
*
*
a)
bO
C',
4.-,
444-
'.-,
*
*
*
*
44*4-
I*
U
o
4*
*
0u2 c
0,IL
U
0
.a)-
"
-
*
*
44*
4-
-a)
+I
-C
0
0
o
/
-
0
U
0
.
ul
.
-- 'a)
~0
4-
0
..
.E
0
a,
,
,*
2
o
a)
N
.
0
iiO
o
c~
-r
+ 0 I. ~
'.C
fE+
-4
04.-'
tI'
0.> C
0
4X x~~~0
+;
-E
Ib
0
U
.
_0
It
I
0
o
4-,I
,
Cv2
CaC
r.
H
E0
C
~'--"~W 0 0
. l-~0
0 ai
°
.4..-C O
=C
IS
-
I
0.
I4
-,
to
II
.0
o u
~,~
~
O:
Ca
-2 2.'
4-'
0
*- o
3
60
1
O
II.
--
1*
V
g
o
.
I
0o,-.. ,
o
4-
.
c
._
a)
a,
-0
":,-0I
I
E
.0
,
ooC
±>
ba
0
a)
~
'O
0
U
+
II
U:N
~l~..
,
U
2E
*0
2-'
,-
-w-i -)
-'-
00
*
a.,
02
E
0
4
0
E
E.
C*-
-
um °a)
:.
4 .
II
0
cn
o
'°
. 0.U
2~
-.
a~~~~~~~~~-
40,~~~~~
-CDL
n"a I~. c~ 0o
°9 A
I
L
- --
CD
C-
-*C CII
-I.~
C
)L
+ A"
0CD
I
A
P'~~
?.
I
CDCDU]A,
C
=
2
s
c~
O
P
It
I.
CDCJDCDCD
~o_r
oD I_
_.
,.
H
_ .
>-.-
CL
oH
=
m
m-
OC
oI
HCDH~~I R
~' .
+
_.~
.
w.S
9'C
'" oD
I ~
C
.-
G
<~CD±
D-
V
C
11
_.
+
a
m
~
tD
N
PI
0j
m
CD
a:
.
.:n
tn_
-i
0D
H
D
H
CD
-
+~+O~~
V
ct
n'-
x_..
n
o9-XA
+1.
_
Co
.
HO
.
- rt
_
CD'-
+ --
8
CDL
CD
a
_.
o
-
C
9
CD
-
>
o o
#D
S
9
0
D
CD-
9-C
CCC
CD-CD-.CD
C
C
-
H-°O
C<
VCDC-
t:t*
*
,a°
9CDCD<
CD< ''CDHCCD
H
'I
-
o
CD
t' -'
_.o
~L
-C
F
-t
H-CD5
CD
9
P
C'-C-CD
'¢*0¢
9CD
9'9
CD
HCD9
9
rHC-== ':q'
.C
CD' u
L
;P
0.
CD3
*
=L CDCDC-.-.-II
CL *5
*,
g
$
CD
a
-<
''-*C
*LCL-
CD
9 C CD''--
'CD-
a O.%
L )
CLr
'D CL
CD
XHCD
C
**
D
PCp
CD
C
*
PI
CD-
CD
O9CDt
-
-OH
CN
9r
I
-
"I 0
CD
CCC
N
CD
>4
NCD
5
H
9CD
x
-
:r'"
CD
,5CC
CD
CD
CD CD
CD
CD
CD
aD
CD-A
CD'
9
C
*~"
_.
F
C
I=
C
H
CD
5'
*
I
I--
.
~'o I......
"
c0 CD
~ ' '' : " HoC'=
i°. o
*
o
0
o
CL
.
-
-
CtD
I
11i Io
H>
-
O
C~
D -
o
-C~
.t- <
0-O C 0<-' V
fLik>.
O~
S
CD
CDr
L
aCL
CL.
L
9#~~~~~~C
# CDCJJ HOt"
CD.
"CD
o
oL
t0
_* ,
N
CD
0
-.
II
II
z1
_.
o
g-s'
CL~
~~~~o
~~~
~~~~~~
~~~~~~~~~P
~~~~~~~~
CL
..
C
_L
D
CD
CD
CD
C
CD
CD
CD
CD
H
H
CD
CD
CD 9-CD
I-iH-
*8CD CL
CD
CD
*
i
CD
CD
o
H
0
0
0
0
v:
co
II
0
.
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
0
0
0q
S
...+
O~~1.
DII
+
4
f4
IV
116
.11
.i
.
8 II
5~
.,
0*
*
I
00
,
-I
EtlI ft I
* *
*to * 0
+
4I
1>8
U)>
I:IIIw
.s
V
w
I,_
o
IIP.4
A
*
Ul
V
)
I-
I-.
i.e X
C
t\I
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
I.
eIs
**
0A I- -0
00
0
,Z;
Q
1 __
~~~~~
a1
~~~~l
.-
c:
£4-.
._-
w
' ~O o o o o o o o o o 0o o 0o o o0
>
>
coe
u t0
>.
.6
-
*- 0
o~~~~~~~~~~~~~
I . I0.I
*',I-,O
*'C;|I*)
'1 N44-:
la
0
>
U
. _
1_,
;
O
>
>
>
>
aX
z
a. 0.
>
>
>
>
>
>
0
>
,_.
._ e
. 0. 0. p.
.
0
C0
a. 12 0XC%0 0. 0.
m
C
0
0)
0)
$Cu
w
I. _
I
.11
4-
*
*
*
*
*
*
*
*
*
*
*
-
*
*
*
*
*
*
*
*
*
0.)
O
0 0_
-
N
V
U
11
.
*
0 0
$.0.
SSc
U
LI
LI
1 0 0
E
_
.
II II II°
ECu Cu
S.
.4.104.1
~~O
0-
00
U)N
V
0
O O °
u tU, U,
z
-e
oS
Q)
*0-
*
**
*
'-
;
=
4 , IrrS1 X^
z
Qc>.> -
0M
.-m
v-.
I
A11
0
e
0.
0.
N =
0
T
Cu
CS +
V
-.5
0.0
*
*
*
*
*
II)
.
0
._r
UV
V
00
0N
7;_
.00.)
*
*
-****
000
V
C)
*
0
w
o o0
~
Hu
'1.0
0
*
0
S
00._
*I,e
11zz_
%
*
*
0
1.
cn
.O
0
-L*.
T.
U)
0
4
.,
0
5
0
o
23
0 o r-+
0+
+
+.
*0
0g
II
0
0~*
0
-
0
o*
05
/+
0
0
-.
0
0
A
2
IQ ± )
Q0
-0 n
';-
onZ<
r
I-
T
I0
4-,4
o
<+
^
e
-
o~~- ~
QQ
0
0
,.
2.
#
0a
O
**
#
a
#
#
- f
*0+
_
.
o
CD
'-' c
<0~~~~~C
'-'
c~)C~h0
.
-
-x,
4
+
~:
##
%
-.
0*
00
CD
-
0+
00-*
*
*#
4
+I -
0
.
z
+
1*
*
*
0~
A o0
<R IS >
S.
t4i
.
%
%
-
0
0 0
*
*
0 . Zi
II
I
*
~
*
*
0
0
0
,.
II
00(
0
0
-.
I
~,
*
Z.#
CD
0
#
1
0
0
D
#
._
0
n
C
#
%
0)
0
Q
0.°
0
S
O
-
w
10
CO
oC
CD
S
0
*51 '+
I
o
-0
11
0
'"0
""'-0
0-.
-*0
0
o
-
'c+
A
P
~-
A
-- 0
_.-A
o*+
5* 2*
+ I-_
0
0
51c C
O
_.
I
I
*
, a4
I
0.
1+
i,
,~.
± A,.
,.- _±_
_
Q
0:
00''
++ C
±tQ
0
01
oo
00~~
U)
o'
2I-I
0
c
0.
010
0
U)
o
NO
CD-'
-
0
0
I
D+
0*
N
CD
0
0
"-'0
0
+
0
11
_.
II
0
0
A
CD-
-
-
V±
I-l><
I.
0
,-'-''
00
00
I'- ~ 0 I0 I0
1.
I
'0
'o
0
H
O
0
0
0
0
+ 1_, ooo
1_, 1_,
' II
1111
.I
1.1 .1
II
_ *
-
O
D
H
0
I0
_.
0.
0.
11
0
, .I
,III
0 o0
'. .- 11 m*
V - W
11
0
0
g-
.0
)*
0-
o
0
0
_.
c*
0
00
0)',
S I-.
H
_."
:n ic iUl-~
o S *'Z
00
I--S~"a-
I-
I :lo
_.
001
A
B
-10
00
0
5* 0
0
8
o
ca
0
0
00
04
*
0
*0
0
0
0nL
0
U)
15
e
.n . .~. _
0
o
0
,.
N*
0*
N
0
S..
o
'0'--'
-
0
C4
CD
T
..
o
B~~~~~~~~~~~~~~~~~~~I
0
'0
-F0'
0I
I
±
I
-
)
0
>
II
I0
.~-
C
0
C~
0
c,7
C~ C
Qf
0
0
~u
)
0
/
.0-I
0_~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~0
0-I
_02
O
_^
2
A-
-
_>
0e
0
#2
#2
*
*
4-*
.
0
0
o.
~~~~~
*-0
0
*
0
0
O2
02
O
~~~~~05
0-'I
'-0
00
'S
I
,biO
t
s>
bD
0
V-''
,0
c0
.~
>I
<2
**
2
O_
0
O
*r
-,o't
'8
0
0
01"
0
O
11
- -0
[41
O
o
~N
01
.- IC
0
-
0.-I
0 C0
0
.
02
0,
0
2>
0
,,
g
U
_
aC
I
*
c
.
.
ES
0l0
.o-
.
I
>
-,
-
0
.
I
5A
.
o
-~~~~~~~
0
x-.
-~~~~~~~~~~~~~~~~~~~U
~~~~~
~~~~~~~~~~~~~~~~
~~ ~~
~'
I
I
0*~
~~~~'-~
02 0202 02
0
02
~. O#
02'~
0
0
02
02
:0,
02
0 2
0 20
V
I
>.
,_
*-
0
0
02A
#.
_.
...
.-
, ,
.-
'
v U
3 _|
A!nnn
,,
o
#...**
-'1
-
-
-
02
.So
0
II
I Co"II
V11
. 11
'1
VVV V 11I 0
-' - --6y
- - ' - - - - -' - - - -
.\§Y,
-E
2 2
>-1
-
-
-
-
.
-
0
+
+
> > >
I
-
UI
-
O
~3
-
{/
I~
IOE
c'~
. ~
0
UI
II
A
' --
0A1
02~
..
2
$-
00
02
02~ '
0
o
02 ,
002
02
0
#
02- *
0
0
-
4.
v2 ·
0
0
A
C
I~.
I -
I-
,11
0
o~ ._
0,a)z
*k
X
QI
0x0
","_o
~~
z
*
1-
A
o
0
*
U
-- '
._
o
*
8g
O
^
~
z*
I
CD
0o
11~
,
._# 2
- . -
U'
-r
r r r02#r
*~~ Q
c0
S
J._ 0
0
02
II
Ii
.5-0
#
I
yII
0C
I1 11
1 10
... 0
-
-0
-
-
m
11.
oq
-;
aq
-
r
e
CDr
'
~-0-V
'I
"I~~
-
-
qn
CD~~C
;
<~I~
e
.
#
' -s CD
--
*
D
I #U1
*
<
0
*-
01
*
r
/
CDCD
>
*#
2-
o
2
4
=
II
-
11
A
II ,
C-
#
C
1.,0#
0-
CD%
1-~~
~~~~
0
#*
6,
.,.=
<*
,
±~~~~~~~~~~~~~~
CD
~
,_,
~
.
~~~~*
*I
<,-
*:#
*
I1
-
.,~
OI
- '
-.
:S
#
<
.
CD
S' oD ~
1<
-¢
o~~~~~~~~~
g
01
(..I
,
-
CD
a
o=F
~~~~~~~~~~~~~~~~~*
O
^
(D
n
(I
fD
H
zCDCDCD..CD
_I
_
*
1
<C
Q
>
S
01<
-
*
7
0
+C.
H
+
~~~~~~~~~~~~~~~~~iI
I
O
CD
_.
~~~ ~ ~
.
o
D0
~
~
oo
v'^
P, cm
D x,_.
-0D
..~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~C
-CD
AS.
M
~~~-* FC )P ,-
_
It
°
CDC
o
0 01
-
ID~
~
CD
°_
-,0
cr
~
.,-,
CD+-0
'-'C
CD
*
,
o S'c~
*',~?'~'
-
O
C'
-.
0CJ
5-cn
*'
IC
rD
(T0
'"......
|
'
p m>^O~
o(
C
Oi
IIz'Dl5M
-r
I .
z Oq
i °CD
_
o
,'u
D
-.
I_
0
A,,
-"
2
._
q,
-
1
--'CD
- D
CD
_O
'"h
DDC
,
+0 1
CD
.
,~~
~
i1~=~~~~~~D
5'
01
.-C
~'t
.
01'1±
......
r-
0H
I.~01oCD'
~-
~~[~
_
1-
p
0
CD
CD
=
~
.- ~'~..,_,
l'-=h
(D"
~
...
01
0.
*
~~~~~
~~ ~~~
-D
~
01
'-
3,F,
CD
0..
CD
CD*=
~~~~~~~~~~~~~~CoD
<
iO
CD
_O
CD.D
01
oD
oD
CD
_D
A
I
D
CDg
_
C
0
g
0
0t-
tH
0
.0
. _
0
4-
-o
4+
~:,
11
II
-I
-I
,
t-
O >
©
+
+
.
.,
O)
0
-
--
=
,
.0
1
A
'4+
,
-
+
4o
·)
a,
._
'ci
0
4I~
4>
_
-
4-'
5>
01 -
I
0
00
,,I,
~~-. ~.._ .~
,+I
g
= =
04I
X
X,
0
_
4-'o
3
X
,
sN,
It
I I ,._ _.
Or
-
0-
.>
-_
\_X
04
~t
o
-
z
I _z
,+o
4-'w
3
1T
4
0
II
oi', -
N
r-
I
4-'
Ca
+
4-'4
a,
.
_
>
4+i
,00.
0d
0
-4-
.,
.i~
O.
1F
O~~CO
-O
I-
O
',
+ji
to .
o
0
A1
-
I
, ._
O
0--
0
;
0_
0..
O
0
C
CD
CO0
IC
I
0U
0
0
¢;
0
.
0
D7
Sq
o-
4-..-
o
1-
0
u
o
a
H
.5
0
'0
54
O
i
a,
14
S
C)
._
:
1
h-
O, o
?
-' h
,ci,
#
*
#
0_
o-
#
0t --e
*
£
O,
0
*
*
ci,,
I'•4ci
0>-0
0J
r
4-
0
*
4-kp
# > r
# ,>;
x!
2
Q
£
,,
#
#
*
N
~00
*
#
4-
-
_
C')
0
o0
U.
a.
0. 4.
~-
t
C
C)
-
w
*5'
0,
_
-
i; C,; =
.
.S
0000
ci ci, ci,
>0'-'7
_,
._ =
4C'J
0
4 -
>
0
0
.'
>z>
4
0
._
SI4
0
i5
-- 4-CD
4-
*
*
3-
g0.
*
0.
O
*
*
>0
b
S
*
0
#
-
0
*
d0 ._
0-
+0
1- H
ci,
#
~,
0 >
- z>U>
#
._
0
~
-
0,
a,
4-<
O
>
0
4-#
#
000
Ca
u
C>
. ,
V
0
4+0
4-
0
0
Cc,
bO
a~CN >
=-i =
_CQ
0
_
X
0S
.) ci
Ia ci
u-
IIo
IIV ~. II4'
·
.000
a
4'-_
.
-'.
',
oY -.
4 + 4 + 4 + 4 +
-.
000.
Z..
-.i_
4 + 4;3
+ 4+,_3
e3
0
g 4-.
.~
C
CS
0
O
00C0,
t
4-
O 0
+0V_
-
I_.
-°
0
P:
d~t
A
~
~o
I
r ,
+.
~IQ
a#
*
X*
*
#
VF
U,
II
N
CD
°.
on
05
0-i(
0
_1
a
-,,-
*
.
m
C
*
p
-
*~
(
*
0a
a
R
a
o*
oi
0'
C
*
4
4
4
4
*
a0
It
lt.
+
*
00
.
0 -00:
+
HV
o rr0
ro
PI
~-
.
0=
aN
tn
C
0l
D
a*#
rr
0-
*
**#
*
0
CD
#
*#
X D
=
H
00·
0
0-
=00X
t_
CD
0
+
CD
X
*
_D
*
N #
_
#
.
II
-
II
o
_.j
#
4#
4
4
a
4
u~~~~
co
;a
CD
X0
0
P
m
0
·
-
W
o
0
Fh
O(D
0
C
~0
lb
D
0
CD
CD
(D
(n
Oct
3
-
6
0(D
.
0~~~~~~~I
0
4~~
-4
-4
C
0
C)
~1
0.
-S
11 t
0
2
PS
.
II
_
N
+.
C
D
-
c
II o +
cr CDttn
dD> o
-p
CDND'
N
X
P,
zz
0-
II
Ii
'
H
CDI
-'
I-.-
o
H,
I
D'-.
'11
11
o
I
CD I
U,
N
m,
+
0
0.
C
tn
1
0cn
0
0
H
Cr
0
0m
D
II II
I-
a,
P,
0-
Oi
ocLI I!
-(
C
o-
o
-
-.0
r.
0"
I.
_,0
N
N
r
L
0O
C-
CD
I-
I+
CO~Y
C
0
r:
C.
II
-0
-
0C)
-4
C0
o
C
CI,
o
C
10
g
00
#
0
+
+i
1-0
.S
o
C--
f
+
II I
'4
-
tz-j
1 0
T]
9
0 ,
r.
II
o0
0
0
-uo
+
+ II
0a- :
.+
,i~ 0
00a
±
0
0
,'
ol
*
o
0
0u
V
02
0
0
0
0
,
O
000
c.I c4 II
0
- t ):
SH- q
..
£
'
*
R,0si
.I
I
I~*u
0
0I'
I. 11
E
~11
-
o
Q
_2
O, =
O -i;z
#
o
o.-
ol
0e
*
I
ii
*c*.,
c
.- 1 3
000
I I0
C~~
o
#
O'"
O
0
O~
*
EI
-
0
o
NT
m
0
.
N
N
ii
0
.
I
0
I
0
*~~~~~~(
*
*
*
*
*
*
*
*
*
*
*
0)
u
IE
*
*
O
0
0-l
*
*
*#
I#
.5E I*#
*
*
A, 0
*
*
0
*
*
**
w E- <
a° £
¢l
H
#
*'· *,
*)
o#
*
*#
q
*
I
*
*
.-
-
0
.
a
*
#
*
*
00VU 00u
00
4
C
.+
I
r.
C)
C
C)
X
XX
X
Q~~~
C
C
XX
C
0
_ 4s
0/
z
0>
c,
0
0'
o a11
V
,
I
^
t~~
a*0
*
0
>
e,
#
°
Y
W.= V
°
C e;
I0 0
H
11°
m rl
;:!
0.
-
+._
55555,
00000
000-0
.-
°
~~
000000
~~
+ o
o .~
0.-.
u
*
o
C
,
IE .
*
*
~
0
*
CN
._0.
U0;*
) w
O
0
00-0
o
I
SI
i/i
IO
*
1*
I*
',-.
0
a
=+
011
*
*
*
*
.*0-EI 0
*
*
c'
*
I
I
*
I*
*
0
S
*
*
*
*
0
0 0 I
_ *
00000
00000
0 0
_ 0*
*
0I
OW
* °I^
oo
.
1.-
VV
1_
.I .
.
o-o
00 . 1_
._-
aT1
~~~~~~
3a ooo "I
P2.
O
5-=--
I
H
C uC
.CDC
2
*1 r
CDCD
CD0r
:
,~
Cc
*a~ CD1..,
' #
t
CD
5
's 1o,:I z; ICC
--
P *
---
+m0mm6
*
*-
-
'
1CD
~#
oooo
*
*
CCC
10
..
'-
*
*
0
i
0-~o
4
", " RR
c
0-0
-'~~~
<C1~
0
C,
a
-
II
H*
V
~~c CD
X
3-
#
3
=
°
*
*
*#· r·
mrm
.
*
·
.
r
C
CDU
*
t
>B
CD
CD
I
I
O0
O
~
4
CD
*
*
4
4
.- --,
4
4
CD-
CD-
-
CD
CDCD
-i,-,--
0'??
4
55
--o
I
*.
4
CD
CDCDCDCD
0
0
-C-i
5>
I
4
5
I
4
CD
CD
CD
CD
C0
=~
D' I
CD
mOC
-·~~~~~~~~~~~~~O
I
~.~
4
*
*
4
*
<8
s-'-
,o
CDCD-
4
='
,'
II
4
4
4
4
4
4
*
~~~
OCDCD'~C
.. C ~C-CD
aCD
~.,~ ,~~o
-
CDCD
0 C
CD
5
4
CD
+-
C~~~~~~~I
V
a-'
4
-
*
*
*
I4
0-.-iCD
c
5
CD
.10-0CD
--i
--
CD
I
,-.
CD
0CD-
CD
's-
DC
*
-0
i
O
C(~~~D~C
O
r
-
-i
CD-
Htct
1
_
'
-
9-
4
1e
a
P
,
O
~~~1
NOp
__.
*
*
-
V
4
4a
#
4
*
~~~~~~~~
~-~~ ~
*
C
~~
O
Ob
oi
c.
4
O
1o O
-. 0
m~t~
Qa.
,
I-
..
I
_10, D-
**
IId
a-
NCDi
CDCD
II
'C I I
. A ODC_
AI
Vt
9
*
3c
I
=.
-
II~~~~C
D
*
o
C .I
3
C
DC
*
***
_;
cn-
°~° °-.
0000
CD
~~~
CD
~~~~
o~~~
~~~c
8H
L'q
*
,
-
n mpO
OC O
0 0°
*
o
0+
o
CD-
(D
CD
-i
CD
-.
~'
D
(
5
.1-
CD
lCD
CD
CD
~'~
CD-
~~~
8
1 DI 'C
~C
~.
CD
.CI 00CD
CD-i'
~~.CD
5
, 0 =
DI C
8 CCD~~
i
oCD
C
D
0~>i
,
- CC
o=
C
I
o~
CDo
-C
-~~
00
oCD-
~
r
0
o
°
in~~~m
o
O
- I
iC
D' ~
X
~CDC
DC
CD
8
CX
-'
CC
C,
.
(C
CD ,
CD
V
-
CD,
+
oo
CD
~
~:
o
o
co
cq
UO
C
u
N
.
S:
0~
H
U)
C-
_
*
O
Ca
.
Ca
-C
a,
*
0~c
O
U
CA
o
..
(N=
U
CS
#
*
oo
~#
+
O
g
CT
.4
_
*
)
rv
oOe
.50
4
X
.t..-
4
4
a)
I_
-
-'-
12
..
I:
a
.r.
a
Ca
CI
.
4
V
e
ra
C
~~~~,,
o eC
U
.
So
_~
$
.
+
,
-
.a
U'
)
4
0*
*
C -
Ca
o
0 -
Ca0
C)-,.-
+2
W
4
--... *4
a.
C
C
C- .--
o
C
C:
U
-
a
o'
#a
ao
-z
a)
*
#
C-
'-C
4
#
-#
#)
C-
r'
#-
*
o0
C
¥a
-
a
0
0,
a
*
+
.a
aa
4~~~~~~~~~~~~-
II
rv
4
w
'H
n. I
I V .-
H
V-)
a Ca
x
00P
C-.
4
0,
Ca-
Ca
~H
U
4-.
I
II
0
H
Ha
0
o
*
O a
xl--
4
*
C- )
*
4,
a)a
r
Ca, 0)
.~
'a rt~, c a
.
- C,
E r.
0xU AZ
iI
!DCa
aZ
.~
CU
o;,
,I
h
II
0
0
-C
]
--.
~o
-, CS
._
H~C'
Ca
a
U0
(d
U
a).-Vo
,-
C
_.-
00
2,
au
H
14
C c.U
.,o
ra
_
N
~
o
0'~C
4)
U)
0
.H
4a1
(O
0
-E
U
- 0
4*44X
U) -
3~o
Ca
H
Va
a)
4-'
E-,
,
C-
+
C
zCa
Ca
4
*~a
,~~~~~~~_
a)
Ca
0
-:
U_.
.
.a
a'-- .2
-a
11*
._
-C
a
IC.
a
r
>~~~~~~v
,d<
I
.
r-
4
444
44
4
*
4
4
4
4
4
444
44
4
4
4
4#
4
4
Ca
-
-
;
#)
a
44
,
x a~~~~~~~~~0
U
Ca
.2
o Cn-r
u
*
#-
aL
'a
Z-~-
**#'
#-
a
,
4
4
4
;, Q
0
E"
ta)
O
X
a
4
*
Ca
''
,-
Ca
Ca-
Ud
**
4
4
~a
-u
a
*
-Ce'
O
2
4
4
o a
ou U
Z2 . 2.
Ca
Ca
-
-
C)
Ca
+dt
'
11
"I
U
O
.
-;
0
#
*
..*
U.a
*
.2 '
X
O
t
4
CC,
4
4
x
4
I.) _^
U~~C.
#
#
a,-a
#
,c
#
+
Cal
o- o
~
#
*
II
Ca
J
>C
1
A'C _ 1 1C-
O
oa
_a
·
.:
~o
rM
...II
o"-
z
- 1t
B2
°
'Dt
i- I
,
.,
o
=
lQo
3
0
09O~~~~~~O
+"~+
o
o
g? -~
e
-*
*3 0j3
X
*4
4
4
'
:
-
-
o,>
0
"a^. m g "R=
V
a
a
=b
~
V
"
"
#
X
M
CO
-;5-_' :
:~
a'-~ .o~.
a
m
0,
m>ru-
O #to
R
o
.
e~I
rU
3
o
'c
¢
,
3
I> O
_~C
-m~
~
sX;
O~93
n
rc- I. . .i'% ;3 o(
c~~~~-c~~
rsl
;3
~~
¢b
O>
C
JO
~=
O
_~
14
ma-·0
3
a r x, .
'-io~3
jR-
©
-~
mn m
2
¢
.
X
-*
"aP~xr
c-e
2 r~P ~
' * (9 ~a
~
c+ i; pr
a an
~o 1~~~~~~~~~~~~~~ <
*tb"
"
ca
C
a
°
~'
ii 8
rjCu
~
-a , g -9O 3" ln O ~...,
C
cn, ,. =fj
3.1>o a CCc~~
$~
3;dP9"
$
~X
ona c
09 -
C)
>e#<4
0
0
>
X
3:_8<
3 1t
o~~I
b
R
o
f_~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'
c3
o
0-
c:
oC
O
O
C
c"
#
-F
0
o
r:
'0>
f
.:0
,
a
fr
St+
*
+
o
A:
#
ci.
.-
+
iL·
+
O4
C b;
:0
;
+
.
v
0.0
0
* e
e
,$
bO
+
,
Z~~~~~~v
0
>
c
H
-5
0'
9!
.2o0
+o I
U
0
0
.SI~
d
II
_.
0
0
'V
*
I
EII
C
,'
ObE
d
a
_
.
.
.0
.5
O
0l
0
U
So>
-
;Z,
-1,
u
I
'-
* *
*
*
0'-"
i
'
C
oO
X
J
2 *
ao ,
II ¢Vo
-'
10
T_-So
0
.
0 O
11
113
-
-
OY Zy,,'
#0,.,
..
g
~i
V" xrI
.
d
>
>
00mtX
c
>
'>- s
o
C
*
*
0
L.
I -0 0
0::I
>2Ed
>In,
va
Q)
oU7
-1 in a,
>
oc10
-. :,
..~
C<.
x
II
V
O\
A
sc
>
'0
4-o
#
'i
U
6.
rs
r
-0
I-
-0
oQ,
rb
IE
rCe
0
II I
·
-F
'0'
o
11
0
o+
.3
-0 100
0
o0
;
V
i
O
V O-
Oi
s #
II O
_
0
11 1
1E 1O
,I
rE 40
#
o
-0:.C.)4-
*-Iu
-4
~v+ rva
;d
0
-~,~
.,~.S
v
-t~~~~~~-I-
-Z0
b.D
11
2w
'"
0
0
0
U
- o25
0-'S
0
i
0I -'^'
-E
*
I
'
:00
0
C
dO2s
V
e
I
F~
-~
V
°
+E
VI
*
'-
V
O
I I
CD-C < = = FL
o
'ifl
oA
r+
II n,
13
o2'a0
;a
CDA~
_.,
'
=
A
eF
11 __ Ii pII_
+
V1
_.+
c--
A
9-;
_.
CD
9
3 O
.
a:_t
II
vt 9-
°
-'<
II
II
2
II
~
CD
93,-
C+
p
0
CD
-eAA
~.~'
II
, _. C~A
0
0
nF
<
H
_ 4-.
I
11
0,
II
Z,*
-
C̀-
it
t
_.X
r
1
-m
0cz
n
C'n
9200
-00
'9
*
*6
*
*
*
*
*
*
*
C
Q
*
Co
*
0o,
.90
*
"
*)
c--cc
00
'00
CD
*
*C
0
*
0-.1
0I
Ca
Oz"
Ft
CD
b0 I"0
4
*4
#
*4
4
*
4
0
-
*
*
2
*
r+
P
*
*
cCDr~ 9-Oa
#*
*
0
*
n .
0rCD
*
**
*
P.P0
nOl
CDe
II
CD
92
L
C
Ia
* "
*
*
*
*
00
CD
m
CD
*
xit +
<
,2
h+
*
Ot
*
Co
CD
__.
.
-9-CDI
00"
1nCD
rcDn
1<
A
gCD ~
~
0
rz:
?
n ,.I
it
CD C
c-ret
P
co
LD
0
r-
C
pr
*D
rr
CD
r- O'. I,
- -
r·
nn
*
M
It -
t-c
r·
P
n
-;
,
11
0
--
"0
CD
r
os
000
CD
1CD
CD
'
0v.
o
en
*1
F
C
F
-
CD
C
o CD
U)
N
o
C
en
0
CD
_.
0
0
CD
S9
CC
C
C
r
C
04~
Da =
4
4
9
CDq -
Ifl
4
4
4
4
II
9-'0
*
-
<
001
C
"i
0
<
-
cD
*
,
0
4
#
0o
CD
0
CD
o_
0P
-e
C_.
l_
~CD
Q.0~
1
CD
~S~
tm
-
0t.9
o
'S
- A.
0
0
-
0'
cD
0
0
4
4
#1
#1
r H
p,
Al
N He
00(D en
H
o
CD
-
m
#H.
2Q
H
--
(D
CD
en
.
it
H
It
0
o
itm
:t
Fh
i-
:0
F
0
l
0
O
1. a
P
H
c
,HnL
H
CD
Fh
0
a
0
cD
..
\H
Q.
1
9
1~r
=
-
lDI
a-
ID
n,
Ij
0-
H
eo
r
.
P_
CD
El
aR
n
1
.
00 . .
<92
CD,
*,
_
<2~
CD
_
.D
0o
9
CD
CD
_0
Ccl,
CO
0
0
C
U
++ 0r
wi
I
11-
-
.,
Cr
sw_.
CD
Cr
+
ICD
D
1 +
4
4
#I#X
#1
C'
-
*C
-C~
t
.%4
0
0
H
*.
CD
CD-'
4b
.
=
i0
#
#
4
4
-
CDic
4 4411
4
4
4
4
4
44
4
4
4
.9Q
II
a
i'-
'0
P,
I
CD
o~!
+
#c
#
#
:ja
rD*
4
44
4
4
#
~-
0
M"0
en0
%
F~
C1
0-
i
_.
H.
PDCD
nP)
#-
C
*
O
C)l
O
ma
o
0
a)
4i
e
0
a)
.- '
0
0
a
%~~~
*
*
,
u*
*
0
W
,
_
#
-°
j
0..-'0
jy
U
05
*
'
*
o.E
#
'$:
t
dl
a, 2
-'-'I0'a^
oO
#
#
r r)
#4
44
#
4
4
:
U)>
#"a #U · N
Q
r
5
o
.o
.5
~~4V;. ## o ,o.O V
0
a).a'
t
a,D a)
s
2 a)
E
.0
I
1
u o
'
0
._
a)
>
0
C
4
4
uI
66
V~
a;
I
-0
44
~~~
*
4 4
P
Uz
00
°
.9
a) i
b
CO
0
C;
_;._
(dC
*
a, )
00)0
a
in.
C > 1;
os:_
0 . ..:
._F
4.)
=* V~
14H
.
a
,1
OLeE-
,
)a)
u o
'-'0
0O
-- 0
O
UJ) =
t--
05.V
42,
o
0'°
v
11
V
I
t
v
4,
4"
a4u
II
OU
0o
U,
4
4
O
UU
x
*
*
4-'
ua
cn
-Ea
0
*-E*
U
*
+
0
..
H
OU -SUs
;
.
-021
SJ
C)
.-s '- W
4 30 "
0.
3
-- -
*
C,
0
C
_p
4
#
._
'
#.
En
._
O
4
*
a)
0.
a,
._
%~~
4#4
vt
+
=
Q
P
-
a)
E'I
.0
0
*
+-
'I)I
0-~
x
4
#
#
II
D
C
o
m
0 (-C0
4
*
_E
x
O
a
a)
.
._
0I
(D
E
le-T
e
. _
.
0
0D
a_
a
4 40
a_
''
,
. 0:
U
00
1.-I3
*
0-
-
,
a
4
--ww
w
a
0
Id1
E
44
=
C=
Hs
4
a
0)
e
z
ca1
10(O
0i
4-
r
a
D
C
o
C
l I
Vj
-
*
11
k
a
'
O
a)
U
U
.,40
4
0
0-2-IC
a)C
6
d
0
U,
U
VC
a)
j
.0
c a)
,
a)
+
_s
a
CL3E
c
U
a)
u14
:r I2_
_
2
a
-:
11
)
4
.-
e
*
a)
._0 ._
V
Y0.'a
C
II
a
.-
e
C
'-
U
o
-'oa
O
U
*
.
n
0~
a)o
u
2)0·e
~0 ~
4-
>.[
'2
a)
2
-
a
E
I
a
-
_
i
V
E
.,,
h ,
0k
O
O
O
aZ
0
12
a
._ N 0-a)
*
S
bEoa
Id 0
0-
I
4c
s_
--
a); j._
Od
U . 22.d .
cHU+
.
-
0
a40
U
._
o.) .
o
U I
(0
921
U
,
H
0
-S
5
-
V
E
E
-
tn
o
7Oc
1.
1I
I PI
0
p
0
0
3D-0
0
0
01
11
0,
0
0o
+
PI
Ico
11
.
0'
-3
0m
I
0
<Q
+
0
0
11
o
0l
tO
o0
c
I5·
.
0
j
H1
+
-,~.
~~
.,
-p = = =
1-I
-'
1RC5
11O
1-1
11c
0*
1
~
~I0
I It
A IH
e
50
C.
s~
*
--
1 I -:4
'0
-<I-
r
-' *-
L
0--D
*
0
/
0+' 01
+
n
S
p
U)~
I
\c3
I
n
01
o
.
~
-1
r
'D
i.~h~~H
!,:
ID
P
.
~
IZ* o~
!~
I
**
*
*
'-.
y CG
O
-~~
~D
-.
-
0
CD
0 33
*
0,
a
D
I
HIl
ID
0-~
*
~~~~~~~
03 B
****#
-I
a*
*
**;
2ay
*
tt
I-
I- Z1 *
C,
#
a
*
01
*.
-
I*
C
*a
_C
*
a
4
*a
a
*
*
0
Ci 0
*
*
0
a,
*
*c
*
*
0o0
I
d
x~~~~~~
I;
ID
-. -
_
*
-
~
r
I HI
o
~~~~
*
_
*
O
CD
*r
*~~
r
c
o
a
0
Z
os
0S
#
:*:*
.
X- ='
I=.=, I-$ I- & oO0
- ,
C)
= = =.
2:"
#"In#
.~ 0- C,
-- ,~~~~~~~~~~~M
B~!*1 ?_ o x.. . C3=·
F
*#
-i-iC.
q8
m
I,
Q
'
H*
-'0
-i
-
0
*-
.,
*~~~~~
'00,
o
0
0
*
C
*
*
;t
._
*
*
*x
*
*
*
*
a~~~~
*~~=r
I
c·
-
o
0
O
O
0C
o
0
0
E
U
cO' I. - *^.:
4-
-,
._
0
0
F0
C4
.44
#
2X
*#
*
N
**#
or
a)
0
*._
*
0
#
0
r._
44
t
i
0O
**
,,,.,I
#
:^
0 0
44
3
#
O
#
3
a)._
044
X
._
bO
44...
._,..
% ,,
#O
#
#
0*(
0- 0
cn
· I
._2
0H ,.
In
LZ
U
O
#
>
EE
.
o
C
C
.0X
C
.
+
.
o
''7
C
?_
o
U _
o ,._
0Ha
a-
,.~
--
0
C) .,
,._C 0
4
-aN
U
0
*
=
V
.
A
+
2'> ~:~>
<tVo
U
-
f
~~
0
z
.1zU]
_O
-"II
x_U:;
2
+4*'
I-
**
*
I
*
*
o.
r
>*
.-
'I
-
~_
X
4o
-1-'
0 '4
k0
-
II
+4+4
0
0.
c,
'-0
+-
+ O"
O
',r
C;
+
;-
C
t
D
.2
*
I~
*
0.:
C)*
0)
44
#*
**
*
*
a
- .T1
d
4~
d
t
/
m a
·
be
O^
'
n
. X 5+TO
.EI X
I-i'
Z.I
**
*
4o
X,d
O
°C
°-L
"
m
I
h~~~~~~a
2
(-I
[
~0
mi
|
N
- +4
.4 4
*-1
QŽI
a i
+-
_
5
C,
,^-7
Y C.
.
P0)
o
**
*
-_
o~
_
0
#*
o
o1
O
0O
o
u
.
m
Iz
C
r
U
C
i
2I
-o
-e
0
0
._
a,
0.c .i
+
cl
mL
a~
:
i. --,
-
CD
CD
CD CD
i.
I,
kC r
.
N
t.
o
a-i
cl
~e
*b
*~~
*D
'
~-
lCr
~,
*CCC
ES
~
~
--
J
C~
'r
O
- .
--
£Bn
CD*"
CD
El
m
O
;p
O
*
r
t3
C
*
-jNcD
CO
0
-0.-DCC
tai-
*
h
DCDr5
a1**
D
5S
cCtaCt-C
5
Tv'~
S-AP
~~e
* r (
.
_o.
C
*
0
S't S
----CDrO
CD
ZC
~n Im
-t
a--
c~~~~~~~~~~~~~~~
c~
*-c0
,r
*-
o0 S Z
.
*
0 " -
*
CD
*
**
=,
'
C*CD
D
a-r
Cu
*1
*cC5D'
DC---
~,r % t)
11 .
Ct-C
--
**
'-
~~~~~J~~ ~
m
~~,
~~
*
*
*
m
*
*
*
0
II
s
*5
a
*
~
~
6~~~
'D
-
-d
DC
11
*p
p1
-~~r
CDC/
x
DI
c· 1
;"
nDo
H-
~
C
0
*C
t
Y
*
Ig
PJC
*
CD
tJL
lt
~
o
,,. -B
o .
>
~~~~
S~
C
m~
SO
0'r,.CD·
0,t
1.1
0
~I
x e
D
I~
nDC
*
_
,
*~
*
.
~
S c
'
x
IM))C
.
C
c
&
0.
*
c
*
-, **
·
)
**
-
CDDDD
CD 11
CD~~f
· r 'o- --
CD
S
-
CD
IIŽ-H
- x
*
--
*
*.
~
0 0 rr0
*-
Ds
P~~~~~~~~~~~
^o
'
* r~~~~~~~s~
*
*
D-
~
\
o
*
0
*
cr
u
*
* * 0 0
-
.
,
it-
. ,
C
.
C
T
C
~
D
13
T
D
noZ
,--'_.
p
N
-
-..
I
I
w
rl
,x
s
CCD
T~
t
C
-°
2
'
EnN
a
~
0
*r
Ft-+
IDd)~*
0
m
I
i-.
F
5- W W
-
-
0
O
c' Io
'-
,
II~
~ ~~
-
0a
0
0
'
1
O
1
CD
I
0'-'
o
CO
N
O
InQ
0
*
*
X
0
o
0
0-x,.
I1
-N
O--I _ ._
~--
.
.
_
0,
¥zo
I,
.
In5 *
_
O'-
So,
*
SII
_.,
OX
0
'0'0'0-0
v
In
II '>
a,'-
X-
ClK.
0
o-
s., .0
I
I
1-,
5,5,
1.
1-,
*
*
*
*
*
*
*
*
0
0'0*0-0
0000
0000
0000
I-,
5,
- ,-
Z
I
I,-
I..
ZO:5,w
0.0.0.0.
o
0
C)
2
-
0
=
m .
00
o
o
041
00
a)
O
'-
o
-r
'-
0. , 0 0
.
- :4-,
010
.
*
*.
-°"
5
CO 0 ..
R
0
0
1
In
-
0.
11
11
1 r0 CO
O 0_
cg-0~
0_i'~
0.,
,.a0,
s. s-
o-
Q)
O 0-
-
_.~eC
V
U2sN
0'X
u~ N
I
*
I
*
**
~~
*
._
ml
0
l
0.-
4
**-4 *j
I *In
._
0
*._
0000>
' >5'
V,~
VJ
_
10OO
_
_
_
0 0S)
.
._
00
a
*
*
*
*
c0
N
In
N
v
N
In3
Ns
V
4
00
n4
n
-v
I
0,Q
0z
._
0.0.0.0,
0 U e0
V V.V0
5'a
5'
._
._
*
*
*
*
._)
*
*
*
*
*0 0 0 0
0000_
*
101
1010ca
Vn
a
U_ U
*
-;
U .X;
N
,I 0
*
In
*._
00_s
X -C
0
aI
*'0
'4-
o., 76
0
W
0
0
I~
IN _. s.
-o
0
'"Vz
*
l-
*N+
0
.~'-.'-
0 0
.d
N- 0
O
*
N N
-0
O
-
.4.
In
.
.~
*
00
| *
.E z
C,I
~'
0
0
00V
4
4
4
4
#-1
0
_.
n
II
C=
0
0
n
t
11
Koz °
11
A
n
0
A
00
_
,"-e
/\
I
55
0
0
...
IIIl.
*
50
I-
+'
o~ .-
.
+
0
Q:
2B
0!
°
+
_...
;
+
o Pr
+
.
o
i5
0
pI
000
-
-.
IJ
*
0
_.000.
_.0
N71-6
0
0
0
0
0
-
*Ci
*
.
0
*
*
cjn
t
*
*
0
-
#
*0-
*
*b
*
100
#
#
0:-'
#
o
#
*
*
2--i
0
*
*
#
#
x.
00
X0
>
0
S
*
0
o
p
I
0
C
cn
0
,
no
.
0*O
*1
-
#
#
*
*
o.'
._
9
00-*
0
0
0
0
0
U.
00a
*
0
*-
-C
-C
5-
.
*
2. 0
*
0
0
*
m
+
II
t~O
*
-.o
#
e+i
0
*
0.
00
#
°=
0
I
I
o
00 a
-,
0a o
00
#
#
*
0
_0_
1
5
> m-
+
*
#
0 0
CD
0
0-I
o
*
*
M (D
0.
_.
~o
2.
- -
000
ii
0
+S
0.,o
C
0~
'0
,-..0
85
. F
2.
'0
01
g
S
o.
-E eL
II, , , F
5 11110o
o
'
>
E.
>- _
0
*0
5 5
.e*.0*
*
*
5
*
0
0
_.
0
0
_.
II
)
_
.I
IC
0Z
01
0
o
0
Po.
O
*
-e
_.
0L
r-0
e
<
'.~'
1-
0
O
t1
~.
0
II
_
+
-
0
0
0
-
0
C
0l-
*10<
*e
*0
$o
0
0
... 0*
A
..S'-0
SoOo
OA
*5
5 0
0
~'
A O'
II
0
"O
±_.-.
+ , +,-
P
0
0
o
0q
5
C 0
ic-c-
5
c-
0
0
,O,
_.- I ,~
So~A+ +1.
00.
~-0~~~
1
?
-,e~_
o ~~~~±
-5~~~~~~I..,
n+i
c-c0
PI I
--
0...0
II
a
I
Ap0
C
*
C
0
011
0o
O
0
*
0
0.
*
.
*,
-e
*.
C.
UP)
tD
:R
PI
I
III
I
-0 r
zCDC
_I
5
X
0
-O
04
S,
A
0
S
.1
c-
0
_.
0
o c°
+
B+*
0-
-t-
H
0
~0
0
I
o
R°
Se
(D
0
.
~I<
o
5
o-
*
1
_
o+I_+ 7
0
0v
>-rII
AA
5So0"-'
0.....
A0 _11P
5
0
0
*
, 00 Io
I
, -
0
0
0
0
0
O
0
*
4-C
4-
0
*
4-
>
*- -
>
Q)
,.S,
,. *
+
*
.
~
= O-**=
C1)
,e
i:
i
C)
-o~
a,
Cd
*-%2
- C)
eX
*
**
*
~
>)
*>
u
._
Ce
X :£>Xrg#
C)
t
VQ2Q
2t
.
S~~
~ ~
* "
._
g .)
*_*
*_
o
S;
=
E 000
;.
=,/
0
C
0
0>
.
4..
C.Cw
coC
,
> > >
0
0
g0
0
e
a)
a)
4C
a)
V
C)
0
J
A
05
00
0
#
H
4,
Q)
-a
A
A A \ A
C-4
_,~ bbt
_
1
-* -° -r
U
C/
C/
C
AA
)
C.
2 o
C
>
Id
V V V IV V V
I 'CSt
C:
\/ V V
z
zz.zz.zz.
z
w
=
w
z
w
l
Ce
w
C
z
.
C
C)
O Ce.--.'.Ce C..
er - rr
o
or
-r
)
)C)
Qw w
)
1O
C
-C
C)
a)
*0
5
I
rr r
)C)
C)
IC.
CC)
C1
,C
.
04
1 -
bO
-
.2C
Q)
V)
C. cC.)C.)
> >
-zZ
r~
)
CC
C)..
U.-
:
X
.)
C)
C) CO
0C
=3
4->,w
*- 4.
.
L.
,
C)
.w
.0
.6
0. >.
-1
'
4;
s
03 0 0
4-C
-
P.
4-C
C
>
*
*
o
~..m- cVF
-.dS0*
0 30
g t
*
,~'.
_
®
..
*8
~
0
<
. S*
o >
_. 3 -
3
0:
rm
10
c
cn
14$4V
I1-
0-
0 =
-.
b
0
II)
000002
0.
0o
01i
111 0
0
0' <
00
<
.- ,
03
-S
3
0.
0
000-0
0o
000
0K
Qi
.0.o00
QtQ-
0
0-
_~
o _D
- _
0-0-
0
00
0130101*
012
1
3
0
'~',~
' g
>ss~, B >
t:
'O
E
01
0
0
0
1WN
00
o
o
IX
3
I
0
WzI
a0
,~
0-
0
o
0
000
01
'01
~*0
0
0--.
o
o
000
0
2:
_
-.
0
.
0
0
=
-1
0-
*
0
+a
o08
01n
01C
n
CD
C)
0
+
1
A
0
ZQ0 ZX
V1
*
-'-t
0
_.
5C.
Q002.2.
0
0
0
001
01
001
+
+1
:
0
-
O0
0
C
CO,
C
II
0
0
2
.2
:: Ce *
M
C
*
-
-
0,
0
00
o-,
Ce
Me*-
*
Ce
*
II * 0
0,
00O
Q.
*U
0
-E
0
Ce
*-x.
.0
2
0
2
Ce
'Ce
.E
*
0
ce
0 o'
..
*
*
M
*
00
1
0,
0
=
0,
.2
._t~,
.
0
0
Ce
0
0*
0
.. C0,0*
"*
J:
2
"-a 0,
0*
0
0,
2
.0
0
N
*
0
0
C
C
20
*
I
0l
Ce
Io
,t
2
*- 0
0>
o:
2
*
C
D
C*O
0,
5
<
01
.2*-c
, ._
0
-
Ce
0
.ao
0=-x
.-
S.
'
*
*
0
-
o
o
00
0
221
-
*
0
*
0
0
0
>
-
D,,i-
0
+
+
*
0
0
CJC
0
0
IC
o
2 Cs .
-I
CO
*
C
.-.
C
00
0
.E
o
0
0.0
0
-
Id
1*
*~MI,4
:>CC*S
C e
2I
.4
0>
0
0
'-I
u
0,*
.2
'o0 .
C
I
N._ ._
*_
:
CC
0
U ._
0
C
.=
OCO
00
0==.
.000
-
0
0
0
Dl
-2
SI
C
0,*
N
M
o,...*
C
0,
'
0
I
0
-0
._
_ 0
.Ce*.
0
4.0
0
M
O X
_ Q0
O 00
o'-0 0
-0
0,-4.
M *
CO 0
-'--S
*
0
*-
2o
CO *
M *
N
-2
._Ce
o
0
C
D
0
2
*-0
.
._00
0
0
*M
*_00 *
5*
Ce
0
*
0
C
C.)
0
0
._
*-
riO
M
1-
-
C
0C
._
2
0
0:2
0>
0
±
0
*
-0
*.~
g
~_e
C-
C_
0 C.)
-
* # ##
"'
°
'
+
$ o~
0@ CCC
0
.2 .
z o
o o
.2?U
I
CD
00-
C
C
tD0t
.,004
..
CO
*,
S tD
0
.0
.
0
'0
0)
0
0
0
00,0#
Ot.t.t-0
#
c
0CC C
.C
0Sj
* .N
0
t
0.
j
i
tD--.
3
N
QD
CDtD
t
t
D o
tD
t
2
0
0
O
0
0
0
0
t 0 ;0
0
;00t0 ; t;
00D0D0D0t0t
t t
t0
t
-_COCOCOCOCOCOCOCOCOCOj
#0
5
COCO
°Ct
*,
tD tD
t0
0
c
0,(
tD
D
-t
0
0;t;t
tDtDt
u
.
C.
+*
0
o
0s
+
.0
CNo' 0
CC
o0,,
)-3
*
0o
ICC
-r
#0
C
0?
CC.0
-
- .
0
*D
tD t
tD
ri-C
00
.. ,0
s000x
-0
.0
0, 5
-# -E -E
0
-0
0
---CO
0
I.
001
ri-C
0
'6' ±
a aS*2,
ct>
oO0CCCO
C.)
0
0,
--
h
*0
0-;
0
0
0
_O4- _
0
-
.22t
0
Soo;
'-0
t
2 D
Ce
o>
O*
0
.O 2
2
0,-
0
'-.
0r
*-D
Ce
-C
C.)
0
C
*
42
0,
4e
0
0e
..I
C0
0
0
#
-
S0
-
0,0
oH
0
H H
0,
2
0,
000
00
000
0 CO CO
Z0o
*
*00
1$
*
4
*
,,- -AL
SD
: CD
ia
A
C.
co: *
C
tI
-
*
N
*-
*
CD
p
D
C)
I
C,)j
Al
I
c
It
10
cjl
z0
0-
4:
CO
0:0
t
C
C'
~-0:s
0'5
.
I
;
Cn
)n
N:
0
0o
0DO
Bibliography
[1] Ashraf Alkhairy. Optimal Product and Manufacturing Process Selection - Issues of
Formulation and Methods for Parameter Design. RLE Technical Report No. 572, 1992.
[2] A. C. Atkinson and A. N. Donev. Optimum Experimental Designs. Oxford Science
Publication, Oxford, 1992.
[3] Soren Bisgaard. A Method for Identifying Defining Contrasts for 2 k-p Experiments.
Journal of Quality Technology, 25, p28-35, 1993.
[4] G. E. P. Box, W. G. Hunter, and J. S. Hunter. Statistics for Experimenters.
John
Wiley, New York, 1978.
[5] G. E. P. Box and R. D. Meyer. An Analysis for Unreplicated
Fractional Factorials.
Technometrics, 28, p11-18, 1986.
[6] G. E. P. Box and K. B. Wilson. On the Experimental Attainment of Optimal Conditions. Journal Roy. Stat. Soc. Ser. B., 13, p1-45, 1946.
[7] C Daniel. Use of Half-Normal Plots in Interpreting Factorial Two-Level Experiments.
Technometrics, 1, p311-341, 1959.
[8] M. N. Das and N. C. Giri. Design and Analysis of Experiments. Wiley Eastern Limited,
New Delhi, 1991.
[9] Aloke Dey. Orthogonal Fractional Factorial Designs. Wiley Eastern Limited, New
Delhi, 1985.
[10] Paul Fieguth, M. S. Spina, and D. H. Staelin.
Conformal Design of Experiments:
An Automated Tool for Supporting Experimental Design and Analysis for Efficient
159
Improvement of Products and Processes. MIT Leadersfor Manufacturing Program
Publication, 1992.
[11] R. A. Fisher. The Arrangement of Field Experiments. Journal Min. Agri. 33, p503-513,
1926.
[12] R. A. Fisher. The Theory of Confounding in Factorial Experiments in Relation to the
Theory of Groups. Biometrika, Ann. Eugen. XI, p341, 1942.
[13] S. Ghosh. On some new Search Designs for 2 m Factorial Experiments.
Journal of
Statistical Planning and Inference, 5, p381-389, 1981.
[14] Arnon Hurwitz. Sequential Process Optimization with a Commercial Package. Sixth
National Symposium on Statistics and Design in Automated Manufacturing, 1993.
[15] Russell Lenth. Quick and Easy Analysis of Unreplicated Factorials.
31, p469-473,
Technometrics,
1989.
[16] T. J. Mitchell. An Algorithm for Constructing D-Optimal Design. Technometrics, 16,
p203-210, 1974.
[17] D. C. Montgomery. Design and Analysis of Experiments. John Wiley and Sons, New
York, 1991.
[18] Srivastava J. N. On the Inadequacy of Orthogonal Arrays in Quality, Control and
General Scientific Experimentation and the need for Probing Designs. Comm. Statis.
16, p2901-2941,
1987.
[19] R. L. Plackett and J. P. Burman. Design of Optimal Multi-Factorial Experiments".
Biometrika, 23, p305-325, 1946.
[20] E. G. Schilling. The Relation of Analysis of Variance to Regression. Journal of Quality
Technology, 6, p74-83, 1974.
[21] J. N. Srivastava. Design for Searching Non-negligible Effects. A Survey of Statistical
Designs and Models, Amsterdam, p507-509, 1975.
[22] J. N. Srivastava and S. Ghosh. Balanced 2
Factorial Designs of Resolution V which
allows Search and Experimentation of one Extra Unknown Effect, 4 < m < 8. Comm.
Statis. - Theory Methods A6, p141-166, 1977.
160
[23] G. Taguchi. Introduction to Quality Engineering. Asian Productivity Organization,
Tokyo, 1986.
[24] G. Taguchi. System of Experimental Design. UNIPUB/Kraus International Publication, White Plains, New York, 1987.
[25] C. F. J. Wu and Chen Youyi. A Graph-Aided Method for Planning Two-Level Exper-
iments when Certain Interactions are Important. Technometrics, 34, p162-174, 1992.
[26] W. J. Youden. Partial Confounding in Fractional Replications.
p353-358, 1961.
161
Technometrics, 3,