Chapter-3

ANALYSIS OF VARIANCE

Chapter 3 Design & Analysis of Experiments 1

Montgomery
 What If There Are More Than
Two Factor Levels?
• The t-test does not directly apply
• There are lots of practical situations where there are
either more than two levels of interest, or there are
several factors of simultaneous interest
• The analysis of variance (ANOVA) is the appropriate
analysis “engine” for these types of experiments
• The ANOVA was developed by Fisher in the early
1920s, and initially applied to agricultural experiments
• Used extensively today for industrial experiments

Chapter 3 Design & Analysis of Experiments 2

Montgomery
 An Example (See pg. 66)
• An engineer is interested in investigating the relationship
between the RF power setting and the etch rate for this tool. The
objective of an experiment like this is to model the relationship
between etch rate and RF power, and to specify the power
setting that will give a desired target etch rate.
• The response variable is etch rate.
• She is interested in a particular gas (C2F6) and gap (0.80 cm),
and wants to test four levels of RF power: 160W, 180W, 200W,
and 220W. She decided to test five wafers at each level of RF
power.
• The experimenter chooses 4 levels of RF power 160W, 180W,
200W, and 220W
• The experiment is replicated 5 times – runs made in random
order

Chapter 3 Design & Analysis of Experiments 3

Montgomery
Chapter 3 Design & Analysis of Experiments 4
Montgomery
 An Example (See pg. 66)

Chapter 3 Design & Analysis of Experiments 5

Montgomery
 • Does changing the power change the
mean etch rate?
• Is there an optimum level for power?
• We would like to have an objective
way to answer these questions
• The t-test really doesn’t apply here –
more than two factor levels

Chapter 3 Design & Analysis of Experiments 6

Montgomery
The Analysis of Variance (Sec. 3.2, pg. 68)

• In general, there will be a levels of the factor, or a treatments,

and n replicates of the experiment, run in random order…a
completely randomized design (CRD)
• N = an total runs
• We consider the fixed effects case…the random effects case
will be discussed later
• Objective is to test hypotheses about the equality of the a
treatment means

Chapter 3 Design & Analysis of Experiments 7

Montgomery
 The Analysis of Variance
• The name “analysis of variance” stems from a
partitioning of the total variability in the
response variable into components that are
consistent with a model for the experiment
• The basic single-factor ANOVA model is

 i  1, 2,..., a
yij     i   ij , 
 j  1, 2,..., n

  an overall mean,  i  ith treatment effect,

 ij  experimental error, NID(0,  2 )
Chapter 3 Design & Analysis of Experiments 8
Montgomery
 Models for the Data

There are several ways to write a model

for the data:

yij     i   ij is called the effects model

Let i     i , then
yij  i   ij is called the means model
Regression models can also be employed
Chapter 3 Design & Analysis of Experiments 9
Montgomery
 The Analysis of Variance
• Total variability is measured by the total
sum of squares:
a n
SST   ( yij  y.. ) 2

i 1 j 1

• The basic ANOVA partitioning is:

a n a n

 ij ..  i. .. ij i.
( y  y
i 1 j 1
)  [( y 
2
y )  ( y  y
i 1 j 1
)]2

a a n
 n ( yi.  y.. ) 2   ( yij  yi. ) 2
i 1 i 1 j 1

SST  SSTreatments  SS E
Chapter 3 Design & Analysis of Experiments 10
Montgomery
 The Analysis of Variance

SST  SSTreatments  SS E
• A large value of SSTreatments reflects large differences in
treatment means
• A small value of SSTreatments likely indicates no
differences in treatment means
• Formal statistical hypotheses are:

H 0 : 1   2   a
H1 : At least one mean is different

Chapter 3 Design & Analysis of Experiments 11

Montgomery
Chapter 3 Design & Analysis of Experiments 12
Montgomery
 The Analysis of Variance
• While sums of squares cannot be directly compared
to test the hypothesis of equal means, mean
squares can be compared.
• A mean square is a sum of squares divided by its
degrees of freedom:

dfTotal  dfTreatments  df Error

an  1  a  1  a(n  1)
SSTreatments SS E
MSTreatments  , MS E 
a 1 a(n  1)
• If the treatment means are equal, the treatment and
error mean squares will be (theoretically) equal.
• If treatment means differ, the treatment mean square
will be larger than the error mean square.
Chapter 3 Design & Analysis of Experiments 13
Montgomery
 The Analysis of Variance is
Summarized in a Table

• The reference distribution for F0 is the Fa-1, a(n-1) distribution

• Reject the null hypothesis (equal treatment means) if
F0  F , a 1, a ( n 1)

Chapter 3 Design & Analysis of Experiments 14

Montgomery
Chapter 3 Design & Analysis of Experiments 15
Montgomery
 ANOVA Table
Example 3-1

Chapter 3 Design & Analysis of Experiments 16

Montgomery
Chapter 3 Design & Analysis of Experiments 17
8E 2012 Montgomery
Chapter 3 Design & Analysis of Experiments 18
8E 2012 Montgomery
Model Adequacy Checking in the ANOVA
Text reference, Section 3.4, pg. 80

• Checking assumptions is important

• Normality
• Constant variance
• Independence
• Have we fit the right model?
• Later we will talk about what to do if
some of these assumptions are
violated
Chapter 3 Design & Analysis of Experiments 19
Montgomery
 Model Adequacy Checking in the ANOVA
• Examination of
residuals (see text, Sec.
3-4, pg. 80)

eij  yij  yˆij

 yij  yi.

• Computer software
generates the residuals
• Residual plots are very
useful
• Normal probability plot
of residuals

Chapter 3 Design & Analysis of Experiments 20

Montgomery
Chapter 3 Design & Analysis of Experiments 21
Montgomery
 Other Important Residual Plots

Chapter 3 Design & Analysis of Experiments 22

Montgomery
Chapter 3 Design & Analysis of Experiments 23
8E 2012 Montgomery
Chapter 3 Design & Analysis of Experiments 24
Montgomery
 Post-ANOVA Comparison of Means
• The analysis of variance tests the hypothesis of equal
treatment means
• Assume that residual analysis is satisfactory
• If that hypothesis is rejected, we don’t know which
specific means are different
• Determining which specific means differ following an
ANOVA is called the multiple comparisons problem
• We will use pairwise t-tests on means…sometimes
called Fisher’s Least Significant Difference (or Fisher’s
LSD) Method and Tukey Method

Chapter 3 Design & Analysis of Experiments 25

Montgomery
Chapter 3 Design & Analysis of Experiments 26
Montgomery
Chapter 3 Design & Analysis of Experiments 27
Montgomery
Chapter 3 Design & Analysis of Experiments 28
Montgomery
Chapter 3 Design & Analysis of Experiments 29
Montgomery
Chapter 3 Design & Analysis of Experiments 30
Montgomery
Chapter 3 Design & Analysis of Experiments 31
Montgomery
Chapter 3 Design & Analysis of Experiments 32
Montgomery
Chapter 3 Design & Analysis of Experiments 33
Montgomery
 Why Does the ANOVA Work?
We are sampling from normal populations, so
SSTreatments SS E
 2
a 1 if H 0 is true, and  a2( n 1)
2 2
Cochran's theorem gives the independence of
these two chi-square random variables
SSTreatments /(a  1)  a21 /(a  1)
So F0  Fa 1,a ( n 1)
SS E /[a(n  1)]  2
a ( n 1) /[ a( n  1)]
n
n i2
Finally, E ( MSTreatments )   2  i 1
and E ( MS E )   2
a 1
Therefore an upper-tail F test is appropriate.
Chapter 3 Design & Analysis of Experiments 34
Montgomery
 Sample Size Determination
Text, Section 3.7, pg. 105
• FAQ in designed experiments
• Answer depends on lots of things; including
what type of experiment is being
contemplated, how it will be conducted,
resources, and desired sensitivity
• Sensitivity refers to the difference in means
that the experimenter wishes to detect
• Generally, increasing the number of
replications increases the sensitivity or it
makes it easier to detect small differences in
means
Chapter 3 Design & Analysis of Experiments 35
Montgomery
 Sample Size Determination
Fixed Effects Case

• Can choose the sample size to detect a specific

difference in means and achieve desired values of
type I and type II errors
• Type I error – reject H0 when it is true ( )
• Type II error – fail to reject H0 when it is false (  )
• Power = 1 - 
• Operating characteristic curves plot  against a
parameter  where
a
n  i2
2  i 1
a 2
Chapter 3 Design & Analysis of Experiments 36
Montgomery
 Sample Size Determination
Fixed Effects Case---use of OC Curves
• The OC curves for the fixed effects model are in the
Appendix, Table V
• A very common way to use these charts is to define a
difference in two means D of interest, then the minimum
value of  2 is 2
nD
 2

2a 2
• Typically work in term of the ratio of D /  and try values
of n until the desired power is achieved
• Most statistics software packages will perform power and
sample size calculations – see page 108
• There are some other methods discussed in the text

Chapter 3 Design & Analysis of Experiments 37

Montgomery
 Example:
• EXAMPLE: Consider the tensile strength
experiment described earlier. Suppose
that the experimenter is interested in
rejecting the null hypothesis with a
probability of at least
• 0.90 if the five treatment means are
• µ1=11, µ2=12, µ3=15, µ4=18, and µ5=19
• She plans to use α = 0.01. In this case,
because Σµi = 75.
Chapter 3 Design & Analysis of Experiments 38
Montgomery
Therefore, the mean average = (1/5)75 =15 and,
T1 = µ1- µ = 11 - 15 = -4
T2 = µ2- µ = 12 - 15 = -3
T3 = µ3- µ = 15 - 15 = 0
T4 = µ4- µ = 18 - 15 = 3
T5 = µ5- µ = 19 - 15 = 4
Thus, Σi Τi2 = 50. Suppose the experimenter feels
that the standard deviation of tensile strength at
any particular level of cotton weight percentage will
be no larger than σ = 3 psi. Then, by using the
Equation, we have:

Chapter 3 Design & Analysis of Experiments 39

Montgomery
 Φ2= nΣτi2/ aσ2 =1.11n
We use the operating characteristic curve for a - 1 = 5 - 1 = 4 with,
N - a = a(n - 1) = 5(n - 1) error degrees of freedom and α = 0.01 (see
OC curves at Appendix). As a first guess at the required sample size,
try n = 4 replicates. This yields Φ2=1.11(4) = 4.44, Φ=2.11, and 5(3) =
15 error degrees of freedom. Consequently,
• from Chart V, we find that ß = 0.30. Therefore, the power of the test
is approximately
• 1-ß=1- 0.30 = 0.70, which is less than the required 0.90, and so we
conclude that
• n = 4 replicates are not sufficient. Proceeding in a similar manner,
we can construct the
• following display:

Chapter 3 Design & Analysis of Experiments 40

Montgomery
Chapter 3 Design & Analysis of Experiments 41
Montgomery
 n Φ2 Φ a(n -1) ß Power (1- ß)
4 4.44 2.11 15 0.30 0.70
5 5.55 2.36 20 0.15 0.85
6 6.66 2.58 25 0.04 0.96

Thus, at least n = 6 replicates must be run to obtain a

test with the required power.

Chapter 3 Design & Analysis of Experiments 42

Montgomery
The only problem with this approach to using the operating
characteristic curves is that it is usually difficult to select a
set of treatment means on which the sample size decision
should be based. An alternate approach is to select a
sample size such that if the difference between any two
treatment means exceeds a specified value the null hy
pothesis should be rejected.If the difference between any
two treatment means is as large as D, it can be shown that
the minimum value of Φ2 is: Φ2= nD2/ 2aσ2

Chapter 3 Design & Analysis of Experiments

Montgomery 43
To illustrate this approach, suppose that in the
tensile strength experiment Example, the
experimenter wished to reject the null hypothesis
with probability at least 0.90 if any two treatment
means differed by as much as 10 psi. Then,
assuming that σ = 3 psi, we find the minimum
value of Φ2 to be:
Φ2= n (10)2 / 2(5)(32) = 1.11 n
from the analysis in Example 3-11, we conclude
that n = 6 replicates are required to give the
desired sensitivity when α = 0.01.

Chapter 3 Design & Analysis of Experiments 44

Montgomery
 3.8 Other Examples of Single-Factor Experiments

Chapter 3 Design & Analysis of Experiments 45

Montgomery
Chapter 3 Design & Analysis of Experiments 46
Montgomery
 Conclusions?

Chapter 3 Design & Analysis of Experiments 47

Montgomery
Chapter 3 48
Design & Analysis of Experiments
Montgomery
Example:

Chapter 3 Design & Analysis of Experiments 49

Montgomery
Chapter 3 Design & Analysis of Experiments 50
Montgomery
 3.9 The Random Effects Model

• There are a large number of possible

levels for the factor (theoretically an infinite
number)
• The experimenter chooses ‘a’ of these
levels at random
• Inference will be to the entire population of
levels

Chapter 3 Design & Analysis of Experiments 51

Montgomery
 Variance components

Chapter 3 Design & Analysis of Experiments 52

Montgomery
Chapter 3 Design & Analysis of Experiments 53
Montgomery
 ANOVA F-test is identical to the fixed-effects case

Chapter 3 Design & Analysis of Experiments 54

Montgomery
 Estimating the variance components using the ANOVA method:

Chapter 3 Design & Analysis of Experiments 55

Montgomery
• The ANOVA variance component
estimators are moment estimators
• Normality not required
• They are unbiased estimators
• Finding confidence intervals on the variance
components is “clumsy”
• Negative estimates can occur – this is
“embarrassing”, as variances are always
non-negative

Chapter 3 Design & Analysis of Experiments 56

Montgomery
Chapter 3 Design & Analysis of Experiments 57
Montgomery
• Confidence interval for the error variance:

• Confidence interval for the interclass

correlation:

Chapter 3 Design & Analysis of Experiments 58

Montgomery
Chapter 3 Design & Analysis of Experiments 59
Montgomery
Chapter 3 Design & Analysis of Experiments 60
Montgomery
Chapter 3 Design & Analysis of Experiments 61
Montgomery
Chapter 3 Design & Analysis of Experiments 62
Montgomery
Problem. A manufacturing engineer was concerned about the density of bricks. He
conducted an experiment on the brick-manufacturing process, to determine the effects of
firing temperatures on the density of a certain type of brick.
Four specific firing temperatures were selected to be used in this experiment. This
experiment led to the following data:

Temperature
(oF) Density
100 15.3 15.3 15.2 15.3 15.4
130 15.7 15.4 15.5 15.5 -
160 15.9 15.8 15.8 15.6 15.5
190 15.9 15.7 15.8 15.7 -

Q1- Does the firing temperature affect the density of the brick? What the experimenter
should do to decrease the Type I Error? Analyze the residuals from the experiment.

Chapter 3 Design & Analysis of Experiments 63

Montgomery
Q1-Answer: Ho: µ100 = µ130 = µ160= µ190 and H1: Not.

ANOVA SS df MS Fo
Treatment 0.653111111 3 0.217704 15.01405
Error 0.203 14 0.0145
Total 0.856111111 17 F0.05,3,14: 3.34

Therefore, reject Ho.

Chapter 3 Design & Analysis of Experiments 64

Montgomery
Q2- Assuming that the normality assumption of ANOVA is unjustified, test
the null hypothesis that the ‘a treatments’ are identical? Did you obtain a
similar decision to the one obtained from ANOVA?

Q2- Answer: Let’s, first, sort the density from minimum to maximum.

Density i j Rank
15.2 100 3 1
15.3 100 1 3
15.3 100 2 3
15.3 100 4 3
15.4 100 5 5.5
15.4 130 2 5.5
15.5 130 3 8
15.5 130 4 8
15.5 160 5 8
15.6 160 4 10
15.7 130 1 12
15.7 190 2 12

Chapter 3 Design & Analysis of Experiments 65

Montgomery
 Density i j Rank
15,7 190 4 12
15.8 160 2 15
15.8 160 3 15
15.8 190 3 15
15.9 160 1 17.5
15.9 190 1 17.5

Chapter 3 Design & Analysis of Experiments 66

Montgomery
 There are ties.

Y1j R1j Y2j R2j Y3j R3j Y4j R4j SUMj(Rij^2)

15.3 3 15.7 12 15.9 17.5 15.9 17.5 765.5
15.3 3 15.4 5.5 15.8 15 15.7 12 408.25
15.2 1 15.5 8 15.8 15 15.8 15 515
15.3 3 15.5 8 15.6 10 15.7 12 317
15.4 5.5 15.5 8 94.25
SUMi: 2100

S^2= [1/(18-1)]*[SUMi SUMj Rij^2 -(18*(19)^2)/4]= 27.97

H= 1/S^2*(SUMi ((Ri.^2)/ni) - (18*19^2)/4)= 12.8787

Χ20.05,3=7.81
Since H= 12.8787 > 7.81 We reject Ho.

Chapter 3 Design & Analysis of Experiments 67

Montgomery