Chapter 6: Factorial Designs

 In this chapter, we discuss the analysis of experiments where the objective is to

study the effects of two or more factors on a response of interest.
 However, unlike Chapters 4 and 5, interaction effects between the main factors of
interest will also be explored.
 Thus, a primary objective in this chapter is to study the effects of the
combination of the levels of the factors that are of interest.
 Question: If there are two main factors of interest, factors A and B with and
levels, respectively, then how many treatments (factor level combinations) will be
investigated in the experiment?
 Answer: treatments in total
 In both Sections 6.3.1 and 6.3.2, we discuss the analysis of a two-factor factorial
design when in a completely randomized design or a randomized block design.
 In both cases, it is assumed that a fixed effects model is used. (It is possible to use a
random effects or mixed effects model. However, the analyses becomes more
complicated).

6.3.1 Analysis of Factorial Designs for a Completely Randomized Design (CRD)

 In this section, we will be considering the analysis of a two-factor factorial design (say,
factor A and factor B) in a completely randomized design (i.e. the design of the
experiment must satisfy the assumptions of a CRD).
 There are levels in factor A and levels in factor B.
 There are treatments in total and replicates for each treatment.
 Factor B

… Total

…
Factor A
…

Total …

 is the response to the level of factor A and the level of factor B.

 E.g. is the response to the level of factor A and the level of factor B.
 Some more notation…
𝑏 𝑟
𝑌 =
o 𝑖.. ∑ ∑ 𝑌 𝑖𝑗𝑘is the total of the responses for the level of factor A.
𝑗=1 𝑘=1

𝑎 𝑟

o 𝑌 . 𝑗. = ∑ ∑ 𝑌 𝑖𝑗𝑘is the total of the responses for the level of factor B.

𝑖=1 𝑘 =1

o𝑌 𝑖𝑗. = ∑ 𝑌 𝑖𝑗𝑘is the total of the observations for the treatment corresponding to level of factor A
𝑘 =1 and level of factor B.

𝑎 𝑏 𝑟

o𝑌 .. . = ∑ ∑ ∑ 𝑌 𝑖𝑗𝑘is the grand total of all the observations.

𝑖=1 𝑗=1 𝑘=1

o o is the total number of observations

Example 1:
The yield of a chemical process is being studied. The two most important variables that are thought to affect the yield of
the chemical process are pressure and temperature. Three levels of each factor are selected and a factorial experiment in
a CRD with two replications is performed. The yield data follow. The numbers in ( ) are the cell totals.

Pressure
200 215 230 Total

90.4 90.2 90.7 90.6 90.2 90.4

20 542.5
(180.6) (181.3) (180.6)

Temperature 90.1 90.3 90.5 90.6 89.9 90.1

30 541.5
(180.4) (181.1) (180.0)

90.5 90.7 90.8 90.9 90.4 90.1

40 543.4
(181.2) (181.7) (180.5)

Total 542.2 544.1 541.1 1627.4

o

Pressure
200 215 230 Total

90.4 90.2 90.7 90.6 90.2 90.4

20 𝑌 1 .. 542.5
𝑌 11. (180.6) 𝑌 12 . (181.3) 𝑌 13 . (180.6)
90.1 90.3 90.5 90.6 89.9 90.1
Temperature 30 𝑌 2 .. 541.5
𝑌 21 . (180.4) 𝑌 22 . (181.1) 𝑌 23 . (180.0)
90.5 90.7 90.8 90.9 90.4 90.1
40 𝑌 3 .. 543.4
𝑌 31 . (181.2) 𝑌 32 . (181.7) 𝑌 33 . (180.5)

Total 𝑌 .1. 542.2 𝑌 .2. 544.1 𝑌 .3. 541.1 𝑌 .. . 1627.4

o is the total number of observations

 In constructing the graph in order to check for the presence of an interaction, the 3 different levels of pressure are
represented on the x-axis.
 The average responses are represented on the y-axis using an appropriate scale.

 For each level of temperature, the average response is plotted corresponding to each level of pressure.

Average Response
91.00

Pressure
90.80
200 215 230

20 90.30 90.65 90.30 90.60

Temperature 30 90.20 90.55 90.00

90.40
40 90.60 90.85 90.25
90.20

90.00
200 215 230
Pressure
 Continue to plot these points for each temperature level…

Average Response
91.00
Temp = 40
Pressure
90.80
200 215 230

20 90.30 90.65 90.30 90.60

Temperature 30 90.20 90.55 90.00 90.40

Temp = 20
40 90.60 90.85 90.25
90.20

Temp = 30
90.00
200 215 230
Pressure
 Average Response
91.00
Temp = 40
90.80

90.60

90.40
Temp = 20

90.20

Temp = 30
90.00
200 215 230
Pressure

 From this plot, we can see that there is a pressure by temperature interaction effect.
 If no interaction was present, the lines for each temperature level would never intersect.
For example:
Average Response
91.00
Temp = 40
90.80

90.60

90.40
Temp = 20

90.20
Temp = 30
90.00
200 215 230
Pressure
The Statistical Model
The statistical model for a factorial design in a CRD (a fixed effects model) has the following form:

𝑌 𝑖𝑗𝑘=¿𝜇 + 𝜏¿ 𝑖 +¿𝛽 𝑗 + ¿𝜏𝛽𝑖𝑗 +¿𝜖 𝑖𝑗𝑘

¿ 𝜇 𝑖𝑗 +𝜖 𝑖𝑗𝑘
Where:

 is the usual overall population mean;

 is the effect of the level of factor A (also known as the main effect of factor A);

 is the effect of the level of factor B (also known as the main effect of factor B);

 is the interaction effect of the level of factor A and level of factor B;

 is the mean of the combination of the level of factor A and the level of factor B, i.e., the treatment mean;

 The are the random error terms which are assumed to be independently and normally distributed with a mean of
0 and constant variance .
 Since this is a fixed effects model, the usual assumptions apply:
 The parameters , and are fixed constants;
𝑎 𝑏 𝑎 𝑏
 ∑ 𝜏 𝑖 =∑ 𝛽 𝑗 =∑ (𝜏𝛽 )𝑖𝑗 =∑ (𝜏𝛽 )𝑖𝑗 =0
𝑖=1 𝑗 =1 𝑖=1 𝑗 =1

Treatment Effects:

For a factorial design, multiple hypothesis tests are carried out starting with a test concerning the treatment effects
(factor level combinations):

The treatments (factor level combinations) have no effect on the response

The treatments (factor level combinations) have a significant effect on the response

or

All the are equal versus At least two differ

 If the null hypothesis concerning the treatment effects is not rejected, then there is no need to continue with
additional hypothesis tests concerning the main and interaction effects of the factors.
 However, if the null hypothesis is rejected in favour of the alternative, we can continue to carry out hypothesis tests
concerning the main and interaction effects.

Interaction Effects:

Before carrying out a hypothesis test concerning the main effects of each factor, we first need to determine if there is
an interaction effect between the factors:

All the equal zero (there are no interaction effects)

Not all the equal zero (there are interaction effects)

 Only if the null hypothesis concerning the interaction effects is NOT rejected we can carry out hypothesis tests
concerning the main effects of each of the factors.
Main Effects:

If no interaction effects between the two factors are present, then we can carry out hypothesis tests concerning the
effects of factor A as well as factor B:

Main effect of factor A: All the equal zero

Not all the equal zero

Main effect of factor B: All the equal zero

Not all the equal zero

ANOVA
The procedure of carrying out the hypothesis test concerning the treatment effects for the analysis of variance is the
same as the previous chapters.
𝑎 𝑏 𝑟
1 2
SSTO¿ ∑ ∑ ∑ 𝑌 − 𝑎𝑏𝑟 𝑌 . ..
2
𝑖𝑗𝑘
df
𝑖=1 𝑗=1 𝑘=1
 As usual, SSTO can be decomposed into
𝑎 𝑏
1 2 1 2
¿
 SST ∑ ∑ 𝑟 𝑌 𝑖 𝑗. − 𝑎𝑏𝑟 𝑌 . . . df
𝑖=1 𝑗=1

 SSE SSTO – SST df

Thus, SST can be decomposed into:

 SSA (measure of variation due to factor A effects)

 SSB (measure of variation due to factor B effects)

 SSAB (measure of variation due to the interaction effects)

These sums of squares are calculated as follows:

𝑎
1 2 1
¿∑
2
 SSA 𝑌 𝑖 .. − 𝑌 .. . df
𝑖=1 𝑏𝑟 𝑎𝑏𝑟

𝑏
1 2 1 2
¿
 SSB ∑ 𝑎𝑟 𝑌 . 𝑗 . − 𝑎𝑏𝑟 𝑌 . .. df
𝑗=1

 SSAB SST – SSA – SSB df

The ANOVA table for a factorial design in a CRD:

Sum of Mean sum of

Source of Variation df F-statistic
Squares squares

Treatment SST 𝑎𝑏 −1 MST F=

F=
Factor A SSA 𝑎− 1 MSA

F=
Factor B SSB 𝑏 −1 MSB

F=
Interaction AB SSAB (𝑎 −1)(𝑏 −1) MSAB

Error SSE 𝑎 𝑏(𝑟 −1) MSE

Total SSTO 𝑎𝑏𝑟 −1

 Let us go back to the temperature/pressure example:
The yield of a chemical process is being studied. The two most important variables that are thought to affect the yield of
the chemical process are pressure and temperature. Three levels of each factor are selected and a factorial experiment in
a CRD with two replications is performed.

 Using the data from this example, let us obtain the ANOVA table in the form of that on the previous slide.
 Recall for this example:
 Pressure
200 215 230 Total

90.4 90.2 90.7 90.6 90.2 90.4

20 542.5
(180.6) (181.3) (180.6)

Temperature 90.1 90.3 90.5 90.6 89.9 90.1

30 541.5
(180.4) (181.1) (180.0)

90.5 90.7 90.8 90.9 90.4 90.1

40 543.4
(181.2) (181.7) (180.5)

Total 542.2 544.1 541.1 1627.4

First, calculate SSTO:

𝑎 𝑏 𝑟
SSTO¿ ∑ ∑ ∑ 𝑌 −
1 22 ( 1627.4 )2
𝑌 . ¿.. ( 90.4 ) + ( 90.2 ) + …+ ( 90.1 ) − 18
𝑖𝑗𝑘
2 2 2

𝑖=1 𝑗=1 𝑘=1 𝑎𝑏𝑟

¿ 1.298
 To calculate SST, ignore the rows and columns and just consider the totals in the cells:
Pressure
200 215 230 Total

90.4 90.2 90.7 90.6 90.2 90.4

20 542.5
(180.6) (181.3) (180.6)

Temperature 90.1 90.3 90.5 90.6 89.9 90.1

30 541.5
(180.4) (181.1) (180.0)

90.5 90.7 90.8 90.9 90.4 90.1

40 543.4
(181.2) (181.7) (180.5)

Total 542.2 544.1 541.1 1627.4

𝑎 𝑏
1 1 2
( 181.3 )2 ( 180.5 )2 ( 1627.4 )2
¿ ∑ ∑ 𝑌 𝑖 𝑗. −
SST
2 2 ( 180.6 )
𝑌 .¿. . 2 + 2 + …+ 2 − 18
𝑖=1 𝑗=1 𝑟 𝑎𝑏𝑟
¿ 1.138
 SSE SSTO – SST ¿ 1.298 −1.138¿ 0.160
Now we need to calculate the decomposed parts of SST:
Pressure
200 215 230 Total

90.4 90.2 90.7 90.6 90.2 90.4

20 542.5
(180.6) (181.3) (180.6)

Temperature 90.1 90.3 90.5 90.6 89.9 90.1

30 541.5
(180.4) (181.1) (180.0)

90.5 90.7 90.8 90.9 90.4 90.1

40 543.4
(181.2) (181.7) (180.5)

𝑎
1 2 1 2( 542.5 )
2
( 541.5 )2 ( 543.4 )2 ( 1627.4 )2
¿∑
SSA = SSTemp 𝑌 𝑖 .. − 𝑌¿ .. . + + −
𝑖=1 𝑏 𝑟 𝑎𝑏𝑟 6 6 6 18
¿ 0.301
Next we calculate SSB (SS for Pressure):
Pressure
200 215 230

90.4 90.2 90.7 90.6 90.2 90.4

20
(180.6) (181.3) (180.6)

Temperature 90.1 90.3 90.5 90.6 89.9 90.1

30
(180.4) (181.1) (180.0)

90.5 90.7 90.8 90.9 90.4 90.1

40
(181.2) (181.7) (180.5)

Total 542.2 544.1 541.1

𝑏
1 1 (2542.2 )2 ( 544.1 )2 ( 541.1 )2 ( 1627.4 )2
¿∑
SSB =SSPressure
2
𝑌. 𝑗 .− ¿𝑌 . .. + + −
𝑗=1 𝑎𝑟 𝑎𝑏𝑟 6 6 6 18
¿ 0.768
 Lastly, we can find the sum of squares for the interaction between temperature and pressure:

SSAB SSTemp*Press SST – SSA – SSB

1.138 – 0.301 – 0.768

0.069

The degrees of freedom for each sum of squares:

df(SSTO) ¿ 18 −1=17
df(SST) ¿ 9 − 1=8
These 2 sum to df(SSTO)
df(SSE) ¿9
df(SSTemp) ¿ 3 − 1=2
df(SSPress) ¿ 3 − 1=2 These 3 sum to df(SST)

df(SSTemp*Press) ¿4
 The ANOVA table for this factorial design example of a CRD:

Sum of Mean sum of

Source of Variation df F-statistic
Squares squares

1.138
Treatment 1.138 8 =0.142 F
8
0.301 F
Temperature 0.301 2 2
=0.151

0.768 F
Pressure 0.768 2 2
=0.384

0.069
Temp*Press 0.069 4 4
=0.017 F

0.160
Error 0.160 9 9
=0 .018

Total 1.298 17
Now, we start the ANOVA test by performing a hypothesis test concerning the treatments using a level of
significance of 5%:

Treatment Effects:

The treatments (factor level combinations) have no effect on the response

The treatments (factor level combinations) have a significant effect on the response
or

All the are equal versus At least two differ

The critical value for this test: (from the statistical F-tables).

Degrees of freedom of Degrees of freedom of

MST MSE

Therefore, (from the ANOVA table) since is rejected a 5% l.o.s., therefore we can conclude that the treatments have a
significant effect on the response (or that at least two treatment means differ).
 Since on the previous slide was rejected, we can then perform a hypothesis test for the presence of an interaction effect

Interaction Effects:

All the equal zero (there are no interaction effects)

Not all the equal zero (there are interaction effects)

The critical value for this test: (from the statistical F-tables).

Degrees of freedom of Degrees of freedom of

MSTemp*Press MSE

Therefore, (from the ANOVA table) since is NOT rejected a 5% l.o.s., therefore we can conclude that there are no interaction
effects present between temperature and pressure.

 Since there are NO interaction effects present between temperature and pressure, we are now able to perform hypothesis
tests concerning the effects of temperature and pressure separately.
 Main Effects:

Main effect of factor A (Temperature):

All the equal zero (Temperature has no effect on the response)

Not all the equal zero (Temperature has a significant effect on the response)

The critical value for this test: (from the statistical F-tables).

Degrees of freedom of Degrees of freedom of

MSTemp MSE

Therefore, (from the ANOVA table) since is rejected a 5% l.o.s., therefore we can conclude that factor ‘temperature’ has a
significant effect on the response.

Main effect of factor B (Pressure):

All the equal zero (Pressure has no effect on the response)

Not all the equal zero (Pressure has a significant effect on the response)
The critical value for this test: (from the statistical F-tables).

Degrees of freedom of Degrees of freedom of

MSPress MSE

Therefore, (from the ANOVA table) since is rejected a 5% l.o.s., therefore we can conclude that factor ‘pressure’ has a
significant effect on the response.

Pairwise Comparisons

 Recall from previous chapters: If the null hypothesis concerning the treatment effects is rejected, and thus we can
conclude at least two treatment means differ, we may wish to determine which treatment means differ by carrying out a
pairwise comparison of the means.

 However, since the treatment effects for a factorial design are made up of the effects of TWO factors (in this course) and
potential interaction effects, the process of a pairwise comparison differs to what we have done in previous
chapters.
Let us calculate the mean for each treatment:
Pressure
200 215 230

20 180.6 ÷ 2 = 90.30 181.3 ÷ 2 = 90.65 180.6 ÷ 2 = 90.30

Temperature 30 180.4 ÷ 2 = 90.20 181.1 ÷ 2 = 90.55 180.0 ÷ 2 = 90.00

40 181.2 ÷ 2 = 90.60 181.7 ÷ 2 = 90.85 180.5 ÷ 2 = 90.25

Suppose there was an interaction effect present between temperature and pressure on the response variable. Let us carry out
a pairwise comparison on the factor level means for each factor.
Pairwise Comparison in the PRESENCE of an Interaction Effect:

Assuming there is an interaction effect, we can only compare the means between the rows within the SAME column
when doing a pairwise comparison for Temperature.
Pressure
200 215 230

20 180.6 ÷ 2 = 90.30 181.3 ÷ 2 = 90.65 180.6 ÷ 2 = 90.30

Temperature 30 180.4 ÷ 2 = 90.20 181.1 ÷ 2 = 90.55 180.0 ÷ 2 = 90.00

40 181.2 ÷ 2 = 90.60 181.7 ÷ 2 = 90.85 180.5 ÷ 2 = 90.25

 The LSD value for a pairwise comparison when an interaction effect is present:

𝐿𝑆𝐷=𝑡
𝑎𝑏 ( 𝑟 −1 ) ;1 −
𝛼
2 √ 2 𝑀𝑆𝐸
𝑟 √
¿ 𝑡 9 ;0.975
2(0.018 )
2
¿ 2.262
√
2(0.018 )
2
¿ 0.303

Now, comparing the temperature factor level means when pressure = 200:

Pressure
200 |90.3 −90.2|=0.1 ¿ 𝐿𝑆𝐷

20 180.6 ÷ 2 = 90.30 |90.3 −90.6|=0.3 ¿ 𝐿𝑆𝐷

30 180.4 ÷ 2 = 90.20
Temperature |90.2− 90.6|=0.4 ¿ 𝐿𝑆𝐷
40 181.2 ÷ 2 = 90.60
Therefore, and are significantly different.
Similarly, comparing the temperature factor level means when pressure = 215:
Pressure
215
|90.65 −90.55|=0.1 ¿ 𝐿𝑆𝐷

20 181.3 ÷ 2 = 90.65 |90.65 −90.85|=0.2 ¿ 𝐿𝑆𝐷

|90.55 −90.85|=0.3 ¿ 𝐿𝑆𝐷

Temperature 30 181.1 ÷ 2 = 90.55

Therefore, none of the temperature factor level means are

40 181.7 ÷ 2 = 90.85
significantly different when pressure is at 215.

And when pressure = 230: 230

|90.30 −90.00|=0.3 ¿ 𝐿𝑆𝐷

20 180.6 ÷ 2 = 90.30 |90.30 −90.25|=0.05 ¿ 𝐿𝑆𝐷

|90.00 −90.25|=0.25 ¿ 𝐿𝑆𝐷

Temperature 30 180.0 ÷ 2 = 90.00
Therefore, none of the temperature factor level means are
40 180.5 ÷ 2 = 90.25 significantly different when pressure is at 230.
We now follow a similar procedure for pressure, where we can only compare the pressure factor level means that are at the
same temperature level.

This means we only compare the means in the same ROW:

Pressure
200 215 230

20 180.6 ÷ 2 = 90.30 181.3 ÷ 2 = 90.65 180.6 ÷ 2 = 90.30

Temperature 30 180.4 ÷ 2 = 90.20 181.1 ÷ 2 = 90.55 180.0 ÷ 2 = 90.00

40 181.2 ÷ 2 = 90.60 181.7 ÷ 2 = 90.85 180.5 ÷ 2 = 90.25

The LSD value remains the same for both factors (when an interaction effect is present): 𝐿𝑆𝐷=0.303

Let us compare the pressure level means when temperature = 20:

Pressure
200 215 230

Temperature 20 180.6 ÷ 2 = 90.30 181.3 ÷ 2 = 90.65 180.6 ÷ 2 = 90.30

|𝟗𝟎 . 𝟑𝟎 −𝟗𝟎 .𝟔𝟓|=𝟎 .𝟑𝟓

¿ 𝑳𝑺𝑫 |90.30 −90.30|=0 ¿ 𝐿𝑆𝐷 |𝟗𝟎 . 𝟔𝟓 −𝟗𝟎 .𝟑𝟎|=𝟎 .𝟑𝟓¿ 𝑳𝑺𝑫

Therefore, and are significantly different, as well as and .

And when temperature = 30: 200 215 230

Temperature 30 180.4 ÷ 2 = 90.20 181.1 ÷ 2 = 90.55 180.0 ÷ 2 = 90.00

|𝟗𝟎 . 𝟐𝟎 −𝟗𝟎 .𝟓𝟓|=𝟎 .𝟑𝟓

¿ 𝑳𝑺𝑫 |90.20 −90.00|=0.20¿ 𝐿𝑆𝐷 |𝟗𝟎 . 𝟓𝟓 −𝟗𝟎 .𝟎𝟎|=𝟎 .𝟓𝟓¿ 𝑳𝑺𝑫

Therefore, and are significantly different, as well as and .

And lastly, when temperature = 40:
Pressure
200 215 230

Temperature 40 181.2 ÷ 2 = 90.60 181.7 ÷ 2 = 90.85 180.5 ÷ 2 = 90.25

|90.60 −90.85|=0.25 ¿ 𝐿𝑆𝐷 |𝟗𝟎 . 𝟔𝟎 −𝟗𝟎 .𝟐𝟓|=𝟎 .𝟑𝟓 ¿ 𝐿𝑆𝐷 |𝟗𝟎 . 𝟖𝟓 −𝟗𝟎 .𝟐𝟓|=𝟎 .𝟔𝟎¿ 𝐿𝑆𝐷

Therefore, and are significantly different, as well as and .

 N.B: To demonstrate the pairwise comparison procedure when an interaction effect is significant, we assumed there was
an interaction effect in this example.

 However, in this temperature/pressure example, we found the interaction effect to be insignificant, and thus were able to
perform hypothesis tests concerning the main effects of each factor.
 Pairwise Comparison in the ABSENCE of an Interaction Effect:

Since no interaction effect is present, we can compare the overall factor level means for temperature, ignoring the factor
levels of pressure.

Thus, we need to determine the mean of ALL the responses in each level of temperature:

Pressure
200 215 230 Total Mean

90.4 90.2 90.7 90.6 90.2 90.4

20 542.5
542.5
÷6 90.42
(180.6) (181.3) (180.6)

Temperature 90.1 90.3 90.5 90.6 89.9 90.1

30 541.5
541.5
÷6 90.25
(180.4) (181.1) (180.0)

90.5 90.7 90.8 90.9 90.4 90.1

40 543.4
543.4
÷6 90.57
(181.2) (181.7) (180.5)
The LSD value will now be: 𝐿𝑆𝐷=𝑡
𝑎𝑏 ( 𝑟 −1 ) ;1 −
𝛼
2 √ 2 𝑀𝑆𝐸
𝑏𝑟

¿ 𝑡 9 ;0.975
√ 2(0.018 )
2(3)
¿ 2.262
2(0.018 )
2(3) √
¿ 0.1752

Mean |90.42− 90.25|=0.17 ¿ 𝐿𝑆𝐷

20 90.42
|90.42− 90.57|=0.15 ¿ 𝐿𝑆𝐷

Temperature 30 90.25
|90.25 −90.57|=0.32 ¿ 𝐿𝑆𝐷

40 90.57
Therefore, and are significantly different.
Similarly, for pressure: Pressure
200 215 230

90.4 90.2 90.7 90.6 90.2 90.4

20
(180.6) (181.3) (180.6)

Temperature 90.1 90.3 90.5 90.6 89.9 90.1

30
(180.4) (181.1) (180.0)

90.5 90.7 90.8 90.9 90.4 90.1

40
(181.2) (181.7) (180.5)

Total 542.2 544.1 541.1

Mean 90.37 90.68 90.18

 Since both factors have the same number of factor levels (), the LSD value for both factors will be the same…

𝐿𝑆𝐷=𝑡
𝑎𝑏 ( 𝑟 −1 ) ;1 −
𝛼
2 √ 2 𝑀𝑆𝐸
𝑎𝑟 ¿ 𝑡 9 ;0.975
√ 2(0.018 )
2(3) √
¿ 2.262
2(0.018 )
2(3)
¿ 0.1752

Pressure
200 215 230

Mean 90.37 90.68 90.18

|90.37 −90.68|=0.31 ¿ 𝐿𝑆𝐷 |90.37 −90.18|=0.19 ¿ 𝐿𝑆𝐷 |90.68 −90.18|=0.50 ¿ 𝐿𝑆𝐷

Therefore, and are significantly different, as well as and . That is, all the means differ from each other
significantly.