A System of Experimental Design

C. F. Jeff Wu
Georgia Institute of Technology
• System has four broad branches :
(i) regular orthogonal designs,
(ii) nonregular orthogonal designs,
(iii) response surface designs,
(iv) optimal designs.
• New opportunities in boarder areas.
– Interface between (ii) <=> (iii), (iii) <=> (iv).
– Space-filling designs.
New materials available in
“Experiments: Planning, Analysis and Parameter Design
Optimization” by Wu - Hamada (2000)
Regular orthogonal designs ( Fisher, Yates,
Finney, …): 2nk , 3nk designs, using
minimum aberration criterion
Nonregular orthogonal designs (Plackett-
Burman, Rao, Bose): Plackett-Burman designs,
orthogonal arrays
(factor screening, projection)
Response surface designs (Box) : fitting a
parametric response surface
Optimal designs (Kiefer): optimality driven by
specific model/criterion
 Fundamental Principles for Factorial Effects
• Effect Hierarchy Principle:
– Lower order effects more important than higher order
effects
– Effects of same order equally important
• Effect Sparsity Principle: Number of relatively
important effects is small
• Effect Heredity Principle: for an interaction to be
significant, at least one of its parent factors should
be significant
 Fractional Factorial Designs
Run 1 2 3 12 13 23 123
1 - - - + + + -
2 - - + + - - +
3 - + - - + - +
4 - + + - - + -
5 + - - - - + +
6 + - + - + - -
7 + + - + - - -
8 + + + + + + +

col “12” = (col 1)  (col2), etc.

• 4 factors: 1, 2, 3, 4 =12
(4 & 12 are said to be aliased)
24-1 design: I =124
• 5 factors: 1, 2, 3, 4 =12, 5 =13
25-2 design: I =124 =135 = 2345
(defining contrast subgroup)
• Resolution = shortest wordlength in the
defining contrast subgroup of a design
• Design of same resolution can be quite
different
d1 : I=4567=12346=12357
d 2 : I=1236=1457=234567
both are 27 - 2 design of resolution IV
but d1 is better (why?)
• Let Ai (d ) = no. of words of length i in the
defining contrast subgroup of design d
• Minimum aberration criterion (Fries-Hunter,
1980): sequentially minimizes the values A3,
A4, A5, …etc.
• Aberration criterion is an extension of
resolution criterion
• Ready-to-use tables of minimum aberration
(and related) 2k-p designs in WH
Thirty-Two Run Fractional Factorial Designs
 Extensions of Minimum Aberration to
Designs with Factor Asymmetry

• 2n-k designs in 2q blocks: “treatment defining

contrast subgroup”, and “block defining contrast
subgroup” are intertwined
• 2n-k parameter designs, control and noise factors
(control-by-noise interaction key to robustness):
modified effect hierarchy principle:
c, n, cn (1st group), cc, ccn, nn (2nd group), etc.
• 2n-k split-plot designs, n 1 whole-plot factors, n2
split-plot factors: wp effects, wp x sp effects, sp
effects treated differently, two variance
components
Two Types of Fractional Factorial Designs:
• Regular ( 2n k , 3n k designs):
columns of the design matrix form a group over a
finite field; the interaction between any two
columns is among the columns
 any two factorial effects are either
orthogonal or fully aliased
• Nonregular (mixed-level designs, orthogonal
arrays)
some pairs of factorial effects can be partially
aliased
 more complex aliasing pattern
 Orthogonal Arrays
• Two columns of a design matrix are orthogonal if
all possible level combinations of the two
columns appear equally often in the matrix
• An orthogonal array OA(N , s1m    skm ) of
1 k

strength two is an Nm matrix,

m  m1      mk in which mi columns have si
levels and any two columns are orthogonal
• 2n-k, 3n-k designs are OA’s
Cast Fatigue Experiment
Design Matrix OA(12, 27) and Lifetime Data
11
Full Matrix: OA(12, 2 )
 Partial and Complex Aliasing
• For the 12-run Plackett-Burman design OA(12, 2 11)
1
E i  i    jk
ˆ
3 j , k i
partial aliasing: coefficient  13
 10 
complex aliasing: 45 (    ) partial aliases
 2
• Traditionally complex aliasing was considered to
be a disadvantage
• Standard texts pay little attention to this type of
designs
 Useful Orthogonal Arrays
• Collection in WH
11
OA(12,2 ) OA(12,3124 ) OA(18,2137 )
OA(18,6126 ) OA(20,219 ) OA(24,31216 )
1 14 11 12 OA(36,37 63 )
OA(24,6 2 ) OA(36,2 3 )
8 3 11 12 OA(50, 21511 )
OA(36,2 6 ) OA(48,2 4 )
OA(54,21325 )
• Run Size Economy
OA(12,211) vs.16-run 2k -p designs, 8  k  11
7 vs. 27-run 3k- p designs, 5  k  7
OA(18,3 )
• Flexibility in level combinations
 Blood Glucose Experiment
1 7
OA(18,2 3 )
 Analysis Strategies

• Traditionally experiments with complex aliasing

were used for screening purpose, i.e., estimating
main effects only
• A paradigm shift: using effect sparsity/heredity,
Hamada-Wu (1992) recognized that complex
aliasing can be turned into an advantage for
studying interactions
• Analysis methods (frequentist and Bayesian)
allow two-factor interactions to be entertained (in
addition to main effects). Effective if the number
of significant interactions is small
 Examples

• Cast Fatigue Experiment:

Main effect analysis: F (R2=0.45)
F, D (R2=0.59)
HW analysis: F, FG (R2=0.89)
F, FG, D (R2=0.92)

• Blood Glucose Experiment:

Main effect analysis: Eq, F q (R2=0.36)
HW analysis: Bl, (BH)lq, (BH)qq (R2=0.89)

Bayesian analysis also identifies Bl, (BH)ll, (BH)lq,

(BH)qq as having the highest posterior model probability
 Further Analysis
• Success in the HW analysis strategy led to
research on the hidden projection properties of
nonregular designs. Commonly used arrays like
OA(12, 2 11), OA(18, 3 7), OA(36, 2 11312) have
desirable projection properties (i.e., for 4 - 8
factors, a number of interactions can be estimated
with good efficiency)
• This is achieved without adding new runs
• It has also inspired a new approach to response
surface methodologyFurther Analysis
 Central Composite Designs
A simple CCD is shown graphically below. It has three parts
(1) cube ( or corner) points, (2) axial (or star) points,
(3) center points.

A Central Composite Design in Three Dimensions (cube point (dot), star

point(cross), center point (circle))
An Alternative to Standard Response Surface
Methodology
• Standard RSM employs a 2-stage experimentation
strategy; this can be time consuming and
expensive.
• S. W. Cheng - Wu (2001) proposed a new strategy
to perform factor screening (1st order model) and
response surface exploration (2nd order model) on
the same experiment using one design, based on
new optimality criteria called projection -
aberration
 Optimal Designs
• D-,G-,I-optimality based on a single model,
performance not guaranteed over a variety of
models. Performance is highly model-dependent.
• Exact optimality more interesting than
approximate (continuous) optimality. Algorithms
make more impact than theory; generally
applicable to any models.
• Bigger impact when used in conjunction with or as
a supplement to a combinatorial or reasonably
uniform design (irregular design, follow-up
experiment, sequential designs).
• Generally useful as a benchmark.
dimensions using number-theoretic justifications. More gen

approaches: space-filling, minimax or maximin distance; us

fitting.
25 points of a Latin hypercube sample
25 points of a randomly centered randomized orthogonal array. For
any two variables, there is one point in each reference square.
25 points of an OA-based Latin hypercube sample
 Innovations in Bayesian Analysis for
Designed Experiments

• Choice of priors reflects the three principles

(hierarchy, sparsity, heredity)
• Model search strategy depends on nature of design
(much easier for regular designs; not so for
nonregular designs); strategy should exploit the
effect aliasing pattern
• Convenient for computer experiments.
 A flexible strategy for selecting a suitable design from amon

(allowing main effects and a flexible choice of interactions

(vii) Space-filling designs (Latin hypercube designs).