257 8

Principal Component and

Factor Analysis
8.1 Introduction – 259
8.2 Understanding Principal Component and Factor
Analysis – 261
8.2.1 Why Use Principal Component and Factor Analysis? – 261
8.2.2 Analysis Steps – 262

8.3 Principal Component Analysis – 263

8.3.1 Check Requirements and Conduct Preliminary
Analyses – 263
8.3.2 Extract the Factors – 266
8.3.3 Determine the Number of Factors – 271
8.3.4 Interpret the Factor Solution – 272
8.3.5 Evaluate the Goodness-of-Fit of the Factor Solution – 275
8.3.6 Compute the Factor Scores – 275

8.4 Confirmatory Factor Analysis and Reliability

Analysis – 276
8.5 Structural Equation Modeling – 280
8.6 Example – 282
8.6.1 Principal Component Analysis – 282
8.6.2 Reliability Analysis – 293

Electronic supplementary material

The online version of this chapter (https://doi.org/10.1007/978-3-662-56707-4_8) contains
additional material that is available to authorized users. You can also download the “Springer
Nature More Media App” from the iOS or Android App Store to stream the videos and scan the
image containing the “Play button”.

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

M. Sarstedt, E. Mooi, A Concise Guide to Market Research, Springer Texts in Business
and Economics, https://doi.org/10.1007/978-3-662-56707-4_8
8.7 Customer Satisfaction at Haver & Boecker
(Case Study) – 296
8.8 Review Questions – 297
References – 298
8.1 · Introduction
259 8

Learning Objectives
After reading this chapter, you should understand:
55The basics of principal component and factor analysis.
55The principles of exploratory and confirmatory factor analysis.
55Key terms, such as communality, eigenvalues, factor loadings, and factor scores.
55What factor rotation is.
55How to determine whether data are suitable for carrying out an exploratory factor
analysis.
55How to interpret SPSS principal component analysis output.
55The principles of reliability analysis and its execution in SPSS.
55The concept of structural equation modeling.

Keywords
Anti-image • Bartlett method • Bartlett’s test of sphericity • Communality • Components • Confirmatory
factor analysis • Correlation residuals • Covariance-based structural equation modeling • Cronbach’s
Alpha • Direct oblimin rotation • Eigenvalue • Eigenvectors • Exploratory factor analysis • Factor analy-
sis • Factor loading • Factor rotation • Factor scores • Factor weights • Factors • Internal consistency reliabil-
ity • Kaiser criterion • Kaiser–Meyer–Olkin criterion • Latent root criterion • Measure of sampling adequa-
cy • Oblique rotation • Orthogonal rotation • Parallel analysis • Partial least squares structural equation mod-
eling • Path diagram • Principal axis factoring • Principal component analysis • Principal components • Prin-
cipal factor analysis • Promax rotation • Quartimax rotation • Regression method • Reliability analysis • Scree
plot • Split-half reliability • Structural equation modeling • Test-retest reliability • Varimax rotation

8.1 Introduction

Principal component analysis (PCA) and factor analysis (also called principal factor analysis
or principal axis factoring) are two methods for identifying structure within a set of vari-
ables. Many analyses involve large numbers of variables that are difficult to interpret. Using
PCA or factor analysis helps find interrelationships between variables (usually called items)
to identify a smaller number of unifying variables called factors. Consider the example of a
soccer club whose management wants to measure the satisfaction of the fans. The manage-
ment could, for instance, measure fan satisfaction by asking how satisfied the fans are with
the (1) assortment of merchandise, (2) quality of merchandise, and (3) prices of merchan-
dise. It is likely that these three items together measure satisfaction with the merchandise.
Through the application of PCA or factor analysis, we can determine whether a single factor
represents the three satisfaction items well. Practically, PCA and factor analysis are applied
to understand much larger sets of variables, tens or even hundreds, when just reading the
variables’ descriptions does not determine an obvious or immediate number of factors.
PCA and factor analysis both explain patterns of correlations within a set of observed
variables. That is, they identify sets of highly correlated variables and infer an underlying
factor structure. While PCA and factor analysis are very similar in the way they arrive at a
solution, they differ fundamentally in their assumptions of the variables’ nature and their
treatment in the analysis. Due to these differences, the methods follow different research
objectives, which dictate their areas of application. While the PCA’s objective is to reproduce
 260 Chapter 8 · Principal Component and Factor Analysis

a data structure, as well as possible only using a few factors, factor analysis aims to explain
the variables’ correlations using factors (e.g., Hair et al. 2010; Matsunaga 2010; Mulaik
2009).1 We will discuss these differences and their implications in this chapter.
Both PCA and factor analysis can be used for exploratory or confirmatory purposes.
What are exploratory and confirmatory factor analyses? Comparing the left and right panels
of . Fig 8.1 shows us the difference. Exploratory factor analysis, often simply referred to as
EFA, does not rely on previous ideas on the factor structure we may find. That is, there may
be relationships (indicated by the arrows) between each factor (indicated by ovals) and each
item. While some of these relationships may be weak (indicated by the dotted arrows), others
are more pronounced, suggesting that these items represent an underlying factor well. The
left panel of . Fig. 8.1 illustrates this point. Thus, an exploratory factor analysis reveals the
number of factors and the items belonging to a specific factor. In a confirmatory factor anal-
ysis, usually simply referred to as CFA, there may only be relationships between a factor and
specific items. In the right panel of . Fig. 8.1, the first three items relate to factor 1, whereas
the last two items relate to factor 2. Different from the exploratory factor analysis, in a con-
firmatory factor analysis, we have clear expectations of the factor structure (e.g., because
researchers have proposed a scale that we want to adapt for our study) and we want to test
for the expected structure.
8 In this chapter, we primarily deal with exploratory factor analysis, as it conveys the
principles that underlie all factor analytic procedures and because the two techniques are
(almost) identical from a statistical point of view. Nevertheless, we will also discuss an
important aspect of confirmatory factor analysis, namely reliability analysis, which tests
the consistency of a measurement scale (see 7 Chap. 3). We will also briefly introduce a
specific confirmatory factor analysis approach called structural equation modeling (often
simply referred to as SEM). Structural equation modeling differs statistically and practi-
cally from PCA and factor analysis. It is not only used to evaluate how well observed vari-
ables relate to factors but also to analyze hypothesized relationships between factors that
the researcher specifies prior to the analysis based on theory and logic.

Satisfaction with the Satisfaction with the

condition of the condition of the
stadium (x1) stadium (x1)

Satisfaction with the Satisfaction with Satisfaction with the Satisfaction with
outer appearance of the the stadium outer appearance of the the stadium
stadium (x2) (factor 1) stadium (x2) (factor 1)
Satisfaction with the Satisfaction with the
interior design of the interior design of the
stadium (x3) stadium (x3)

Assortment of Assortment of
merchandise (x4) Satisfaction with merchandise (x4) Satisfaction with
the merchandise the merchandise
Quality of (factor 2) Quality of (factor 2)
merchandise (x5) merchandise (x5)

. Fig. 8.1 Exploratory factor analysis (left) and confirmatory factor analysis (right)

1 Other methods for carrying out factor analyses include, for example, unweighted least squares, gen-
eralized least squares, or maximum likelihood but these are statistically complex.
8.2 · Understanding Principal Component and Factor Analysis
261 8
8.2 Understanding Principal Component and Factor Analysis

8.2.1 Why Use Principal Component and Factor Analysis?

Researchers often face the problem of large questionnaires comprising many items. For
example, in a survey of a major German soccer club, the management was particularly inter-
ested in identifying and evaluating performance features that relate to soccer fans’ satisfaction
(Sarstedt et al. 2014). Examples of relevant features include the stadium, the team composi-
tion and their success, the trainer, and the management. The club therefore commissioned a
questionnaire comprising 99 previously identified items by means of literature databases and
focus groups of fans. All the items were measured on scales ranging from 1 (“very dissatisfied”)
to 7 (“very satisfied”). . Table 8.1 shows an overview of some items considered in the study.

. Table 8.1 Items in the soccer fan satisfaction study

Satisfaction with …

Condition of the stadium Public appearances of the players

Interior design of the stadium Number of stars in the team

Outer appearance of the stadium Interaction of players with fans

Signposting outside the stadium Volume of the loudspeakers in the stadium

Signposting inside the stadium Choice of music in the stadium

Roofing inside the stadium Entertainment program in the stadium

Comfort of the seats Stadium speaker

Video score boards in the stadium Newsmagazine of the stadium

Condition of the restrooms Price of annual season ticket

Tidiness within the stadium Entry fees

Size of the stadium Offers of reduced tickets

View onto the playing field Design of the home jersey

Number of restrooms Design of the away jersey

Sponsors’ advertisements in the stadium Assortment of merchandise

Location of the stadium Quality of merchandise

Name of the stadium Prices of merchandise

Determination and commitment of the players Pre-sale of tickets

Current success regarding matches Online-shop

Identification of the players with the club Opening times of the fan-shops

Quality of the team composition Accessibility of the fan-shops

Presence of a player with whom fans can identify Behavior of the sales persons in the fan shops
 262 Chapter 8 · Principal Component and Factor Analysis

As you can imagine, tackling such a large set of items is problematic, because it pro-
vides quite complex data. Given the task of identifying and evaluating performance fea-
tures that relate to soccer fans’ satisfaction (measured by “Overall, how satisfied are you
with your soccer club”), we cannot simply compare the items on a pairwise basis. It is far
more reasonable to consider the factor structure first. For example, satisfaction with the
condition of the stadium (x1), outer appearance of the stadium (x2), and interior design
of the stadium (x3) cover similar aspects that relate to the respondents’ satisfaction with
the stadium. If a soccer fan is generally very satisfied with the stadium, he/she will most
likely answer all three items positively. Conversely, if a respondent is generally dissatis-
fied with the stadium, he/she is most likely to be rather dissatisfied with all the perfor-
mance aspects of the stadium, such as the outer appearance and interior design. Conse-
quently, these three items are likely to be highly correlated—they cover related aspects
of the respondents’ overall satisfaction with the stadium. More precisely, these items can
be interpreted as manifestations of the factor capturing the “joint meaning” of the items
related to it. The arrows pointing from the factor to the items in . Fig. 8.1 indicate this point.
In our example, the “joint meaning” of the three items could be described as satisfaction
with the stadium, since the items represent somewhat different, yet related, aspects of the
stadium. Likewise, there is a second factor that relates to the two items x4 and x5, which,
8 like the first factor, shares a common meaning, namely satisfaction with the merchandise.
PCA and factor analysis are two statistical procedures that draw on item correlations
in order to find a small number of factors. Having conducted the analysis, we can make
use of few (uncorrelated) factors instead of many variables, thus significantly reducing the
analysis’s complexity. For example, if we find six factors, we only need to consider six cor-
relations between the factors and overall satisfaction, which means that the recommen-
dations will rely on six factors.

8.2.2 Analysis Steps

Like any multivariate analysis method, PCA and factor analysis are subject to certain
requirements, which need to be met for the analysis to be meaningful. A crucial require-
ment is that the variables need to exhibit a certain degree of correlation. In our example in
. Fig. 8.1, this is probably the case, as we expect increased correlations between x1, x2, and
x3, on the one hand, and between x4 and x5 on the other. Other items, such as x1 and x4, are
probably somewhat correlated, but to a lesser degree than the group of items x1, x2, and x3
and the pair x4 and x5. Several methods allow for testing whether the item correlations are
sufficiently high.
Both PCA and factor analysis strive to reduce the overall item set to a smaller set of
factors. More precisely, PCA extracts factors such that they account for variables’ vari-
ance, whereas factor analysis attempts to explain the correlations between the variables.
Whichever approach you apply, using only a few factors instead of many items reduces its
precision, because the factors cannot represent all the information included in the items.
Consequently, there is a trade-off between simplicity and accuracy. In order to make the
analysis as simple as possible, we want to extract only a few factors. At the same time, we
do not want to lose too much information by having too few factors. This trade-off has to
be addressed in any PCA and factor analysis when deciding how many factors to extract
from the data.
8.3 · Principal Component Analysis
263 8

Check requirements and conduct preliminary analyses

Extract the factors

Determine the number of factors

Interpret the factor solution

Evaluate the goodness-of-fit of the factor solution

Compute the factor scores (optional)

. Fig. 8.2 Steps involved in a PCA

Once the number of factors to retain from the data has been identified, we can proceed
with the interpretation of the factor solution. This step requires us to produce a label for each
factor that best characterizes the joint meaning of all the variables associated with it. This step
is often challenging, but there are ways of facilitating the interpretation of the factor solution.
Finally, we have to assess how well the factors reproduce the data. This is done by examining
the solution’s goodness-of-fit, which completes the standard analysis. However, if we wish to
continue using the results in further analyses, we need to calculate the factor scores. Factor
scores are linear combinations of the items and can be used as variables in follow-up analyses.
. Figure 8.2 illustrates the steps involved in the analysis; we will discuss these in more
detail in the following sections. In doing so, our theoretical descriptions and illustrations
will focus on the PCA, as this method is easier to grasp. However, most of our descriptions
also apply to factor analysis.

8.3 Principal Component Analysis

8.3.1 Check Requirements and Conduct Preliminary Analyses

Before carrying out a PCA, we have to consider several requirements, which we can test
by answering the following questions:
44Are the measurement scales appropriate?
44Is the sample size sufficiently large?
44Are the observations independent?
44Are the variables sufficiently correlated?
 264 Chapter 8 · Principal Component and Factor Analysis

z Are the measurement scales appropriate?

For a PCA, it is best to have data measured on an interval or ratio scale. In practical appli-
cations, items measured on an ordinal scale level have become common. Ordinal scales
can be used if:
44the scale points are equidistant, which means that the difference in the wording
between scale steps is the same (see 7 Chap. 3), and
44there are five or more response categories.

z Is the sample size sufficiently large?

Another point of concern is the sample size. As a rule of thumb, the number of (valid)
observations should be at least ten times the number of items used for analysis. This only
provides a rough indication of the necessary sample size. Fortunately, researchers have
conducted studies to determine minimum sample size requirements, which depend on
other aspects of the study. MacCallum et al. (1999) suggest the following:
44When all communalities (we will discuss this term in 7 Sect. 8.3.2.4) are above 0.60,
small sample sizes of below 100 are adequate.
44With communalities around 0.50, sample sizes between 100 and 200 are sufficient.
44When communalities are consistently low, with many or all under 0.50, a sample size
8 between 100 and 200 is adequate if the number of factors is small and each of these is
measured with six or more items.
44When communalities are consistently low and the factors numbers are high or are
measured with only few items (i.e., 3 or less), 300 observations are recommended.

z Are the observations independent?

We have to ensure that the observations are independent. This means that the observations
need to be completely unrelated (see 7 Chap. 3). If we use dependent observations, we would
introduce “artificial” correlations, which are not due to an underlying factor structure,
but simply to the same respondents having answered the same questions multiple times.

z Are the variables sufficiently correlated?

As indicated before, PCA is based on correlations between items. Consequently, conduct-
ing a PCA only makes sense if the items correlate sufficiently. The problem is deciding what
“sufficient” actually means.
An obvious step is to examine the correlation matrix (7 Chap. 5). Naturally, we want
the correlations between different items to be as high as possible, but they will not always
be. In our previous example, we expect high correlations between x1, x2, and x3, on the
one hand, and x4 and x5 on the other. Conversely, we might expect lower correlations
between, for example, x1 and x4 and between x3 and x5. Thus, not all of the correlation
matrix’s elements need to have high values. The PCA depends on the relative size of the
correlations. Therefore, if single correlations are very low, this is not necessarily problem-
atic! Only when all the correlations are around zero, PCA is no longer useful. In addition,
the statistical significance of each correlation coefficient helps decide whether it differs
significantly from zero.
There are additional measures to determine whether the items correlate sufficiently.
One is the anti-image. The anti-image describes the portion of an item’s variance that is
independent of another item in the analysis. Obviously, we want all items to be highly
correlated, so that the anti-images of an item set are as small as possible. Initially, we do
8.3 · Principal Component Analysis
265 8

. Table 8.2 Threshold values for KMO and MSA

KMO/MSA value Adequacy of the correlations

Below 0.50 Unacceptable

0.50–0.59 Miserable

0.60–0.69 Mediocre

0.70–0.79 Middling

0.80–0.89 Meritorious

0.90 and higher Marvelous

not interpret the anti-image values directly, but use a measure based on the anti-image
concept: The Kaiser–Meyer–Olkin (KMO) criterion. The KMO criterion, also called the
measure of sampling adequacy (MSA), indicates whether the other variables in the dataset
can explain the correlations between variables. Kaiser (1974), who introduced the sta-
tistic, recommends a set of nicely labeled threshold values for KMO and MSA, which
. Table 8.2 presents.
The Bartlett’s test of sphericity can be used to test the null hypothesis that the correla-
tion matrix is a diagonal matrix (i.e., all non-diagonal elements are zero) in the population.
Since we need high correlations for PCA, we want to reject the null hypothesis. A large test
statistic value and corresponding a small p-value will favor the rejection of the hypoth-
esis. In practical applications, it is virtually impossible not to reject this null hypothesis,
as typically there are some correlations, particularly in larger sets of items. In addition,
PCA is typically used with large samples, a situation, which favors the rejection of the null
hypothesis. Thus, Bartlett’s test is of rather limited value for assessing whether the vari-
ables are sufficiently correlated.
To summarize, the correlation matrix with the associated significance levels provides a
first insight into the correlation structures. However, the final decision of whether the data
are appropriate for PCA should be primarily based on the KMO statistic. If this measure
indicates sufficiently correlated variables, we can continue the analysis of the results. If
not, we should try to identify items that correlate only weakly with the remaining items
and remove them. In Box 8.1, we discuss how to do this.

Box 8.1 Identifying problematic items

Examining the correlation matrix and the significance levels of correlations allows identifying
items that correlate only weakly with the remaining items. An even better approach is examining
the variable-specific MSA values, which are interpreted like the overall KMO statistic (see
. Table 8.2). In fact, the KMO statistic is simply the overall mean of all variable-specific MSA
values. Consequently, all the MSA values should also lie above the threshold level of 0.50. If
this is not the case, consider removing this item from the analysis. An item’s communality (see
7 Sect. 8.3.2.4) can also serve as a useful indicator of how well the factors extracted represent
an item. However, communalities are mostly considered when evaluating the solution’s
goodness-of-fit.
 266 Chapter 8 · Principal Component and Factor Analysis

8.3.2 Extract the Factors

8.3.2.1 Principal Component Analysis vs. Factor Analysis

Factor analysis assumes that each variable’s variance can be divided into common vari-
ance (i.e., variance shared with all the other variables in the analysis) and unique variance
(. Fig. 8.3), the latter of which can be further broken down into specific variance (i.e.,
variance associated with only one specific variable) and error variance (i.e., variance
due to measurement error). The method, however, can only reproduce common vari-
ance. Thereby factor analysis explicitly recognizes the presence of error. Conversely, PCA
assumes that all variance is common variance, which factor extraction can fully explain
(e.g., Preacher and MacCallum 2003). These differences entail different interpretations of
the analysis’s outcomes. PCA asks:
» Which umbrella term can we use to summarize a set of variables that loads highly on a
specific factor?

Conversely, factor analysis asks:

» What is the common reason for the strong correlations between a set of variables?
8 From a theoretical perspective, the assumption that there is a unique variance for which the
factors cannot fully account, is generally more realistic, but simultaneously more restric-
tive. Although theoretically sound, this restriction can sometimes lead to complications
in the analysis, which have contributed to the widespread use of PCA, especially in market
research practice.
Researchers usually suggest using PCA when data reduction is the primary concern;
that is, when the focus is to extract a minimum number of factors that account for a
maximum proportion of the variables’ total variance. In contrast, if the primary concern
is to identify latent dimensions represented in the variables, factor analysis should be
applied. However, prior research has shown that both approaches arrive at essentially the
same result when

Principal component analysis Factor analysis

Unique
variance

Common variance / Common variance /

communality communality

Variance extracted
Variance excluded

. Fig. 8.3 Principal component analysis vs. factor analysis

8.3 · Principal Component Analysis
267 8
44more than 30 variables are used, or
44most of the variables’ communalities exceed 0.60.

With 20 or fewer variables and communalities below 0.40—which are clearly undesirable
in empirical research—the differences are probably pronounced (Stevens 2009).
Apart from these conceptual differences in the variables’ nature, PCA and factor
­analysis differ in the aim of their analysis. Whereas the goal of factor analysis is to explain
the correlations between the variables, PCA focuses on explaining the variables’ variances.
That is, the PCA’s objective is to determine the linear combinations of the variables that
retain as much information from the original variables as possible. Strictly speaking, PCA
does not extract factors, but components, which are labeled as such in SPSS.
Despite these differences, which have very little relevance in many common research
settings in practice, PCA and factor analysis have many points in common. For example,
the methods follow very similar ways to arrive at a solution and their interpretations of
statistical measures, such as KMO, eigenvalues, or factor loadings, are (almost) identical.
In fact, SPSS blends these two procedures when running a PCA as the program initially
applies a factor analysis but rescales the estimates such that they conform to a PCA. That
way, the analysis assumes that the entire variance is common but produces (rotated) load-
ings (we will discuss factor rotation in 7 Sect. 8.3.4.1), which facilitate the interpretation
of the factors.
Despite the small differences of PCA and factor analysis in most research settings,
researchers have strong feelings about the choice of PCA or factor analysis. Cliff (1987, p.
349) summarizes this issue well, by noting that proponents of factor analysis “insist that
components analysis is at best a common factor analysis with some error added and at
worst an unrecognizable hodgepodge of things from which nothing can be determined.”
For further discussions on this topic, see also Velicer and Jackson (1990) and Widaman
(1993).2

8.3.2.2 How Does Factor Extraction Work?

When extracting factors, PCA’s objective is to reproduce a data structure with only a
few factors. PCA does this by generating a new set of factors as linear composites of the
original variables, which reproduces the original variables’ variance as best as possi-
ble. These linear composites are called principal components, but, for simplicity’s sake,
we refer to them as factors. More precisely, PCA computes eigenvectors. These eigen-
vectors include so called factor weights, which extract the maximum possible variance
of all the variables, with successive factoring continuing until a significant share of the
variance is explained.
Operationally, the first factor is extracted in such a way that it maximizes the vari-
ance accounted for in the variables. We can visualize this easily by examining the vector
space illustrated in . Fig. 8.4. In this example, we have five variables (x1−x5) represented

2 Related discussions have been raised in structural equation modeling, where researchers have
heatedly discussed the strengths and limitations of factor-based and component-based approaches
(e.g. Sarstedt et al. 2016a; Hair et al. 2017b).
 268 Chapter 8 · Principal Component and Factor Analysis

. Fig. 8.4 Factor extraction3

F1
x1 x4
x2 x5
x3

90°

F2

by five vectors starting at the zero point, with each vector’s length standardized to one.
To maximize the variance accounted for, the first factor F1 is fitted into this vector space
in such a way that the sum of all the angles between this factor and the five variables in
the vector space is minimized. We do this to interpret the angle between two vectors
as correlations. For example, if the factor’s vector and a variable’s vector are congruent,
8 the angle between these two is zero, indicating that the factor and the variable correlate
perfectly. On the other hand, if the factor and the variable are uncorrelated, the angle
between these two is 90°. This correlation between a (unit-scaled) factor and a vari-
able is called the factor loading. Note that factor weights and factor loadings essentially
express the same thing—the relationships between variables and factors—but they are
based on different scales.
After extracting F1, a second factor (F2) is extracted, which maximizes the remain-
ing variance accounted for. The second factor is fitted at a 90° angle into the vector space
(. Fig. 8.4) and is therefore uncorrelated with the first factor.4 If we extract a third factor,
it will explain the maximum amount of variance for which factors 1 and 2 have hitherto
not accounted. This factor will also be fitted at a 90° angle to the first two factors, making it
independent from the first two factors (we don’t illustrate this third factor in . Fig. 8.4, as
this is a three-dimensional space). The fact that the factors are uncorrelated is an import-
ant feature, as we can use them to replace many highly correlated variables in follow-up
analyses. For example, using uncorrelated factors as independent variables in a regression
analysis helps solve potential collinearity issues (7 Chap. 7).

3 Note that . Fig. 8.3 describes a special case, as the five variables are scaled down into a two-dimen-
sional space. In this set-up, it would be possible for the two factors to explain all five items. However,
in real-life, the five items span a five-dimensional vector space.
4 Note that this changes when oblique rotation is used. We will discuss factor rotation later in this
chapter.
8.3 · Principal Component Analysis
269 8

The Explained Visually webpage offers an excellent illustration of two- and three-dimensional
factor extraction.

© intheskies/stock.adobe.com
http://setosa.io/ev/principal-component-analysis/

An important PCA feature is that it works with standardized variables (see 7 Chap. 5 for
an explanation of what standardized variables are). Standardizing variables has import-
ant implications for our analysis in two respects. First, we can assess each factor’s eigen-
value, which indicates how much a specific factor extracts all of the variables’ variance
(see 7 Sect. 8.3.2.3). Second, the standardization of variables allows for assessing each
variable’s communality, which describes how much the factors extracted capture or repro-
duce each variable’s variance (see 7 Sect. 8.3.2.4).

8.3.2.3 What Are Eigenvalues?

To understand the concept of eigenvalues, think of the soccer fan satisfaction study
(. Fig. 8.1). In this example, there are five variables. As all the variables are standardized
prior to the analysis, each has a variance of 1. In a simplified way, we could say that the
overall information (i.e., variance) that we want to reproduce through factor extraction is
5 units. Let’s assume that we extract the two factors presented above.
The first factor’s eigenvalue indicates how much of the total variance the factor accounts
for. Each variable has a variance of 1, which means the number of variables is the total vari-
ance (i.e., 5 variables = 5 units of variance). If a factor has an eigenvalue of, let’s say 2.10, it
covers the information of 2.10 variables or, put differently, accounts for 2.10/5.00 = 42 %
of the overall variance (. Fig. 8.5).
Extracting a second factor will allow us to explain another part of the remaining vari-
ance (i.e., 5.00 − 2.10 = 2.90 units, . Fig. 8.5). However, the eigenvalue of the second
factor will always be smaller than that of the first factor. Assume that the second factor
has an eigenvalue of 1.30 units. The second factor then accounts for 1.30/5.00 = 26 % of
the overall variance. Together, these two factors explain (2.10 + 1.30)/5.00 = 68 % of the
 270 Chapter 8 · Principal Component and Factor Analysis

100% 100%
0.30
0.50

0.80
68%
% of variance

1.30 1.30
5.00

2.10 2.10

Total variance Total variance Variance

reproduced by reproduced by
five factors two factors
8
. Fig. 8.5 Total variance explained by variables and factors

overall variance. Every additional factor extracted increases the variance accounted for
until we have extracted as many factors as there are variables. In this case, the factors
account for 100 % of the overall variance, which means that the factors reproduce the
complete variance.

For readers interested in the statistical principles, the Explained Visually webpage illustrates the
concepts of eigenvalues and eigenvectors.

© Alexander Vasilyev/stock.adobe.com
http://setosa.io/ev/eigenvectors-and-eigenvalues/
8.3 · Principal Component Analysis
271 8
Following the PCA approach, we assume that factor extraction can reproduce each vari-
able’s entire variance. In other words, we assume that each variable’s variance is common;
that is, the variance is shared with other variables. This differs in factor analysis, in which
each variable can also have a unique variance.

8.3.2.4 What Is Communality?

Whereas the eigenvalue tells us how much variance each factor accounts for, the commu-
nality indicates how much variance of each variable factor extraction can reproduce. There
is no commonly agreed threshold for a variable’s communality, as this depends strongly
on the complexity of the analysis at hand. However, generally, the extracted factors should
account for at least 50 % of a variable’s variance. Thus, the communalities should be above
0.50. Every additional factor extracted will increase the explained variance, and if we
extract as many factors as there are variables (in our example five), each variable’s commu-
nality would be 1.00. The factors extracted would then fully explain each variable; that is,
the first factor will explain a certain amount of each variable’s variance, the second factor
another part, and so on.
However, since our overall objective is to reduce the number of variables through factor
extraction, we should extract only a few factors that account for a high degree of overall
variance. This raises the question of how to decide on the number of factors to extract from
the data, which we discuss in the following section.

8.3.3 Determine the Number of Factors

Determining the number of factors to extract from the data is a crucial and challenging
step in any PCA. Several approaches offer guidance in this respect, but most researchers
do not pick just one method, but use multiple ones. If different methods suggest the same
number of factors, this leads to greater confidence in the results.

8.3.3.1 The Kaiser Criterion

An intuitive way to decide on the number of factors is to extract all the factors with an
eigenvalue greater than 1. The reason for this is that each factor with an eigenvalue greater
than 1 accounts for more variance than a single variable (remember, we are looking at stan-
dardized variables, which is why each variable’s variance is exactly 1). As the objective of
PCA is to reduce the overall number of variables, each factor should of course account
for more variance than a single variable can. If this occurs, then this factor is useful for
reducing the set of variables. Extracting all the factors with an eigenvalue greater than 1
is frequently called the Kaiser criterion or latent root criterion and is commonly used to
determine the number of factors. However, the Kaiser criterion is well known for over-
specifying the number of factors; that is, the criterion suggests more factors than it should
(e.g., Russell 2002; Zwick and Velicer 1986).

8.3.3.2 The Scree Plot

Another popular way to decide on the number of factors to extract is to plot each factor’s
eigenvalue (y-axis) against the factor with which it is associated (x-axis). This results
in a scree plot, which typically has a distinct break in it, thereby showing the “correct”
 272 Chapter 8 · Principal Component and Factor Analysis

number of factors (Cattell 1966). This distinct break is called the “elbow.” Researchers
typically recommend retaining all factors above this break, as they contribute most to
the explanation of the variance in the dataset. Thus, we select one factor less than indi-
cated by the elbow.

8.3.3.3 Parallel Analysis

A large body of review papers and simulation studies has produced a prescriptive con-
sensus that Horn’s (1965) parallel analysis is the best method for deciding how many
factors to extract (e.g., Dinno 2009; Hayton et al. 2004; Henson and Roberts 2006;
Matsunga 2010; Zwick and Velicer 1986). The rationale underlying parallel analysis is
that factors from real data with a valid underlying factor structure should have larger
eigenvalues than those derived from randomly generated data (actually pseudorandom
deviates) with the same sample size and number of variables. Parallel analysis involves
several steps. First, a large number of datasets are randomly generated that have the
same number of observations and variables as the original dataset. Parallel PCAs are
then run on each of the datasets (hence, parallel analysis), resulting in many slightly
different sets of randomly generated eigenvalues. We then compare the randomly gen-
erated eigenvalues with those from the original analysis. Only factors whose original
8 eigenvalues are larger than the 95th percentile of the eigenvalues should be retained
(Longman et al. 1989).

8.3.3.4 Expectations
When, for example, replicating a previous market research study, we might have a priori
information on the number of factors we want to find. For example, if a previous study
suggests that a certain item set comprises five factors, we should extract the same number
of factors, even if statistical criteria, such as the scree plot, suggest a different number.
Similarly, theory might suggest that a certain number of factors should be extracted from
the data.
Strictly speaking, these are confirmatory approaches to PCA, which blur the distinc-
tion between these two factor analysis types. Ultimately however, we should not only rely
on the data, but keep in mind that the research results should be interpretable and action-
able for market research practice. Once we have decided on the number of factors to retain
from the data, we can start interpreting the factor solution.

> Whatever combination of approaches we use to determine the number of factors,

the factors extracted should account for at least 50 % of the total variance
explained (75 % or more is recommended).

8.3.4 Interpret the Factor Solution

8.3.4.1 Rotate the Factors

To interpret the solution, we have to determine which variables relate to each of the factors
extracted. We do this by examining the factor loadings, which represent the correlations
between the factors and the variables and can take values ranging from −1 to +1. A high
8.3 · Principal Component Analysis
273 8
factor loading indicates that a certain factor represents a variable well. Subsequently, we
look for high absolute values, because the correlation between a variable and a factor
can also be negative. Using the highest absolute factor loadings, we “assign” each vari-
able to a certain factor and then produce a label for each factor that best characterizes
the joint meaning of all the variables associated with it. This labeling is subjective, but
a key PCA step. An example of a label is the respondents’ satisfaction with the stadium,
which r­ epresents the items referring to its condition, outer appearance, and interior design
(. Fig. 8.1).
We can make use of factor rotation to facilitate the factors’ interpretation. We do not
have to rotate the factor solution, but it will facilitate interpreting the findings, partic-
ularly if we have a reasonably large number of items (say six or more). To understand
what factor rotation is all about, once again consider the factor structure described in
. Fig. 8.4. Here, we see that both factors relate to the variables in the set. However, the
first factor appears to generally correlate more strongly with the variables, whereas the
second factor only correlates weakly with the variables (to clarify, we look for small
angles between the factors and variables). This implies that we “assign” all variables
to the first factor without taking the second into consideration. This does not appear
to be very meaningful, as we want both factors to represent certain facets of the vari-
able set. Factor rotation can resolve this problem. By rotating the factor axes, we can
create a situation in which a set of variables loads highly on only one specific factor,
whereas another set loads highly on another. . Figure 8.6 illustrates the factor rota-
tion graphically.
On the left side of the figure, we see that both factors are orthogonally rotated 49°,
meaning that a 90° angle is maintained between the factors during the rotation procedure.
Consequently, the factors remain uncorrelated, which is in line with the PCA’s initial objec-
tive. By rotating the first factor from F1 to F1′, it is now strongly related to variables x1, x2,
and x3, but weakly related to x4 and x5. Conversely, by rotating the second factor from F2
to F2′, it is now strongly related to x4 and x5, but weakly related to the remaining variables.
The assignment of the variables is now much clearer, which facilitates the interpretation
of the factors significantly.
Various orthogonal rotation methods exist, all of which differ with regard to their treat-
ment of the loading structure. The varimax rotation is the best-known and this procedure

F1 F1
x1 x4 x1 x4 F 2
x2 x5 x2
F1 F1 x5
x3 F2 x3

49º 45º

49º 66º
F2 F2

. Fig. 8.6 Orthogonal and oblique factor rotation

 274 Chapter 8 · Principal Component and Factor Analysis

aims at maximizing the dispersion of loadings within factors, which means a few vari-
ables will have high loadings, while the remaining variables’ loadings will be considerably
smaller (Kaiser 1958)
Alternatively, we can choose between several oblique rotation techniques. In oblique
rotation, the 90° angle between the factors is not maintained during rotation, and the
resulting factors are therefore correlated. . Figure 8.6 (right side) illustrates an example
of an oblique factor rotation. Promax rotation is a commonly used oblique rotation tech-
nique. The promax rotation allows for setting an exponent (referred to as kappa) that
needs to be greater than 1. Higher values make the loadings even more extreme (i.e.,
high loadings are amplified and weak loadings are reduced even further), which is at
the cost of stronger correlations between the factors and less total variance explained
(Hamilton 2013). A kappa value of 3 works well for most applications. Direct oblimin
rotation is a popular alternative oblique rotation type, which allows specifying the
maximum degree of obliqueness. This degree is the delta, which determines the level
of the correlation allowed between the factors. A delta of zero (the default) ensures
that the factors are—if at all—only moderately correlated, which is acceptable for most
analyses. Oblique rotation is used when factors are possibly related. It is, for example,
very likely that the respondents’ satisfaction with the stadium is related to their satis-
8 faction with other aspects of the soccer club, such as the number of stars in the team or
the quality of the merchandise. However, relinquishing the initial objective of extract-
ing uncorrelated factors can diminish the factors’ interpretability. We therefore recom-
mend using the varimax rotation to enhance the interpretability of the results. Only if
the results are difficult to interpret, an oblique rotation should be applied. Among the
oblique rotation methods, researchers generally recommend the promax (Gorsuch
1983) or oblimin (Kim and Mueller 1978) methods but differences between the rota-
tion types are typically marginal (Brown 2009).

8.3.4.2 Assign the Variables to the Factors

After rotating the factors, we need to interpret them and give each factor a name, which
has some descriptive value. Interpreting factors involves assigning each variable to a
specific factor based on the highest absolute (!) loading. For example, if a variable has
a 0.60 loading with the first factor and a 0.20 loading with the second, we would assign
this variable to the first factor. Loadings may nevertheless be very similar (e.g., 0.50 for
the first factor and 0.55 for the second one), making the assignment ambiguous. In such
a situation, we could assign the variable to another factor, even though it does not have
the highest loading on this specific factor. While this step can help increase the results’
face validity (see 7 Chap. 3), we should make sure that the variable’s factor loading with
the designated factor is above an acceptable level. If very few factors have been extracted,
the loading should be at least 0.50, but with a high number of factors, lower loadings
of above 0.30 are acceptable. Alternatively, some simply ignore a certain variable if it
does not fit with the factor structure. In such a situation, we should re-run the analy-
sis without variables that do not load highly on a specific factor. In the end, the results
should be interpretable and actionable, but keep in mind that this technique is, first and
foremost, exploratory!
8.3 · Principal Component Analysis
275 8
8.3.5 Evaluate the Goodness-of-Fit of the Factor Solution

8.3.5.1  heck the Congruence of the Initial and Reproduced

C
Correlations
While PCA focuses on explaining the variables’ variances, checking how well the method
approximates the correlation matrix allows for assessing the quality of the solution (i.e.,
the goodness-of-fit) (Graffelman 2013). More precisely, to assess the solution’s goodness-
of-fit, we can make use of the differences between the correlations in the data and those
that the factors imply. These differences are also called correlation residuals and should
be as small as possible.
In practice, we check the proportion of correlation residuals with an absolute value
higher than 0.05. Even though there is no strict rule of thumb regarding the maximum
proportion, a proportion of more than 50 % should raise concern. However, high residuals
usually go hand in hand with an unsatisfactory KMO measure; consequently, this problem
already surfaces when testing the assumptions.

8.3.5.2  heck How Much of Each Variable’s Variance Is Reproduced

C
by Means of Factor Extraction
Another way to check the solution’s goodness-of-fit is by evaluating how much of each
variable’s variance the factors reproduce (i.e., the communality). If several communal-
ities exhibit low values, we should consider removing these variables. Considering the
variable-specific MSA measures could help us make this decision (see Box 8.1). If there
are more variables in the dataset, communalities usually become smaller; however, if the
factor solution accounts for less than 50 % of a variable’s variance (i.e., the variable’s com-
munality is less than 0.50), it is worthwhile reconsidering the set-up.

8.3.6 Compute the Factor Scores

After the rotation and interpretation of the factors, we can compute the factor scores,
another element of the analysis. Factor scores are linear combinations of the items and can
be used as separate variables in subsequent analyses. For example, instead of using many
highly correlated independent variables in a regression analysis, we can use few uncor-
related factors to overcome collinearity problems.
The simplest ways to compute factor scores for each observation is to sum all the scores
of the items assigned to a factor. While easy to compute, this approach neglects the poten-
tial differences in each item’s contribution to each factor (Sarstedt et al. 2016).
Drawing on the item weights produced by the PCA is a more elaborate way of com-
puting factor scores (Hershberger 2005). These weights indicate each item’s relative con-
tribution to forming the factor; we simply multiply the standardized variables’ values
with the weights to get the factor scores. Factor scores computed on the basis of item
weights have a zero mean. This means that if a respondent has a value greater than zero
for a certain factor, he/she scores above the average in terms of the characteristic that this
factor describes. Conversely, if a factor score is below zero, then this respondent exhibits
the characteristic below average.
 276 Chapter 8 · Principal Component and Factor Analysis

Different from the PCA, a factor analysis does not produce determinate factor scores.
In other words, the factor is indeterminate, which means that part of it remains an arbi-
trary quantity, capable of taking on an infinite range of values (e.g., Grice 2001; Steiger
1979). Thus, we have to rely on other approaches to compute factor scores. The use of these
approaches is, however, not restricted to factor analysis but extends to PCA, because of the
specific way SPSS (and other programs) have implemented the method. The most promi-
nent of these approaches is the regression method. This method takes into account (1) the
correlation between the factors and variables (via the variable loadings), (2) the correla-
tion between the variables, and (3) the correlation between the factors if oblique rotation
has been used (DiStefano et al. 2009). The regression method z-standardizes each factor to
zero mean and unit standard deviation.5 We can therefore interpret an observation’s score
in relation to the mean and in terms of the units of standard deviation from this mean. For
example, an observation’s factor score of 0.79 implies that this observation is 0.79 standard
deviations above the average with regard to the corresponding factor.
Another popular approach is the Bartlett method, which is similar to the regression
method. In SPSS, the method produces factor scores with zero mean and standard devia-
tions larger than one. Owing to the way they are estimated, the factor scores that the Bartlett
method produces are considered more accurate (Hershberger 2005). However, in practical
8 applications, both methods produce very similar results. Because of the z-standardization
of the scores, which facilitates the comparison of scores across factors, we recommend
using the regression method.
In . Table 8.3 we summarize the main steps that need to be taken when conducting a
PCA or factor analysis in SPSS.

8.4 Confirmatory Factor Analysis and Reliability Analysis

Many researchers and practitioners acknowledge the prominent role that exploratory
factor analysis plays in exploring data structures. Data can be analyzed without precon-
ceived ideas of the number of factors or how these relate to the variables under consider-
ation. Whereas this approach is, as its name implies, exploratory in nature, the confirma-
tory factor analysis allows for testing hypothesized structures underlying a set of variables.
In a confirmatory factor analysis, the researcher needs to first specify the constructs
and their associations with variables, which should be based on previous measurements
or theory. Instead of allowing the procedure to determine the number of factors, as is
done in an exploratory factor analysis, a confirmatory factor analysis tells us how well
the actual data fit the pre-specified structure. Returning to our introductory example, we
could, for example, assume that the construct satisfaction with the stadium can be mea-
sured by the three items x1 (condition of the stadium), x2 (appearance of the stadium),
and x3 (interior design of the stadium). Likewise, we could hypothesize that satisfaction
with the merchandise can be adequately measured using the items x4 and x5. In a confir-
matory factor analysis, we set up a theoretical model linking the items with the respective

5 Note that this is only the case in PCA. When using factor analysis, the standard deviations are differ-
ent from one (DiStefano et al. 2009).
8.4 · Confirmatory Factor Analysis and Reliability Analysis
277 8

. Table 8.3 Steps involved in carrying out a PCA or factor analysis in SPSS

Theory Action
Check assumptions and carry out preliminary analyses

Select variables that should be ► Analyze ► Dimension Reduction ► Factor. Enter the
reduced to a set of underlying variables in the Variables box.
factors (PCA) or should be used to
identify underlying dimensions
(factor analysis)

Are the variables interval or ratio Determine the measurement level of your variables
scaled? (see 7 Chap. 3). If ordinal variables are used, make sure that
the scale steps are equidistant.

Is the sample size sufficiently Check MacCallum et al.’s (1999) guidelines for minimum
large? sample size requirements, dependent on the variables’
communality. For example, if all the communalities are
above 0.60, small sample sizes of below 100 are adequate.
With communalities around 0.50, sample sizes between 100
and 200 are sufficient. Ensure that your dataset meets these
thresholds after handling missing values.

Are the observations Determine whether the observations are dependent or

independent? independent (see 7 Chap. 3).

Are the variables sufficiently ► Analyze ► Dimension Reduction ► Factor ► Descriptives.

correlated? Tick Coefficients, Significance levels, KMO and Bartlett’s
test of sphericity, and Anti-image.
Check whether the KMO ≥ 0.50. Also consider the following
measures:
–A re the correlation coefficients different from zero and
significant (indicated in the lower part of the correlation
matrix under Sig. (1-tailed)?)
– Is the p-value of the Bartlett’s test (indicated by Sig.) ≤ 0.05?
To identify items that are too lowly correlated, check the
variable-specific MSA values (indicated in the Anti-Image
Correlation table). Are the MSA values ≥ 0.50?

Extract the factors

Choose the method of factor If the goal is to reduce the number of variables to a set of
analysis underlying factors (i.e., principal component analysis):
► Analyze ► Dimension Reduction ► Factor ► Extraction
► Principal components.
If the goal is to identify underlying dimensions (i.e., factor
analysis):
► Analyze ► Dimension Reduction ► Factor ► Extraction
►Principal axis factoring
 278 Chapter 8 · Principal Component and Factor Analysis

. Table 8.3 (Continued)

Theory Action
Determine the number of factors

Determine the number of factors Kaiser criterion: Extract all factor with an eigenvalue greater
than 1 (default).
Create a scree plot and select the number left of the
distinctive break (elbow).
► Analyze ► Dimension Reduction ► Factor ► Extraction
► Scree plot
Run parallel analysis: Download the syntax file Parallel
analysis.sps from the book’s website (⤓Web Appendix
→ Downloads) and open it in SPSS. Specify the number
of observations under compute Ncases (line 10) and
the number of variables under compute NVars (line 11).
Go to ► Run ► All. Extract those factors whose original
eigenvalues are greater than those indicated under Prcntyle.
Pre-specify the number of factors based on a priori
information: ► Analyze ► Data Reduction ► Factor
8 ► Extraction ► Fixed number of factors: Factors to extract
Check the Cumulative % column in the Total Variance
Explained column.

Interpret the factor solution

Rotate the factors Use the varimax procedure or, if necessary, the promax
procedure with kappa set to 3: ► Analyze ► Dimension
Reduction ► Factor ► Rotation.

Assign variables to factors and Use the rotated solution to assign each variable to a certain
interpret the factors factor based on the highest absolute loading. To facilitate
interpretation, you may also assign a variable to a different
factor but check that the loading lies at an acceptable level
(0.50 if only few factors are extracted, 0.30 if many factors are
extracted).
Find an umbrella term for clusters of items assigned to each
factor.

Compute factor scores Save factor scores as new variables using the regression
method: ► Analyze ► Dimension Reduction ► Factor
► Scores ► Save as variables: Regression.

Evaluate the goodness-of-fit of the factor solution

Check the congruence of Create a reproduced correlation matrix: ► Analyze

the initial and reproduced ► Dimension Reduction ► Factor ► Descriptives
correlations ► Reproduced. Is the proportion of residuals greater than
0.05 ≤ 50 %?

Check how much of each Examine the communalities from the reproduced correlation
variable’s variance is reproduced matrix. Check if reproduced communalities (on the diagonal)
by means of factor extraction are ≥ 0.50.
8.4 · Confirmatory Factor Analysis and Reliability Analysis
279 8
construct (note that in confirmatory factor analysis, researchers generally use the term
construct rather than factor). This process is also called operationalization (see 7 Chap. 3)
and usually involves drawing a visual representation (called a path diagram) indicating
the expected relationships.
. Figure 8.7 shows a path diagram—you will notice the similarity to the diagram in
. Fig. 8.1. Ovals represent the constructs (e.g., Y1, satisfaction with the stadium) and boxes
represent the items (x1 to x5). Other elements include the relationships between the con-
structs and respective items (the loadings l1 to l5), the error terms (e1 to e5) that capture the
extent to which a construct does not explain a specific item, and the correlations between
the constructs of interest (r12).
Having defined the individual constructs and developed the path diagram, we can
estimate the model. The relationships between the constructs and items (the loadings l1
to l5) and the item correlations (not shown in . Fig. 8.7) are of particular interest, as they
indicate whether the construct has been reliably and validly measured.
Reliability analysis is an important element of a confirmatory factor analysis and essen-
tial when working with measurement scales. The preferred way to evaluate reliability is by
taking two independent measurements (using the same subjects) and comparing these
using correlations. This is also called test-retest reliability (see 7 Chap. 3). However, prac-
ticalities often prevent researchers from surveying their subjects a second time.
An alternative is to estimate the split-half reliability. In the split-half reliability, scale
items are divided into halves and the scores of the halves are correlated to obtain an esti-
mate of reliability. Since all items should be consistent regarding what they indicate about
the construct, the halves can be considered approximations of alternative forms of the

Satisfaction with the

e1
condition of the stadium (x1)
l1

Satisfaction with the outer Satisfaction with

e2 l2
appearance of the stadium (x2) the stadium (Y1)

l3
Satisfaction with the interior
e3
design of the stadium (x3)
r 12

Assortment of
e4
merchandise (x4)
l4
Satisfaction with
the merchandise (Y2)
l5
Quality of
e5
merchandise (x5)

. Fig. 8.7 Path diagram (confirmatory factor analysis)

 280 Chapter 8 · Principal Component and Factor Analysis

Box 8.2 Things to consider when calculating Cronbach’s Alpha

When calculating Cronbach’s Alpha, ensure that all items are formulated in the same direction
(positively or negatively worded). For example, in psychological measurement, it is common
to use both negatively and positively worded items in a questionnaire. These need to be
reversed prior to the reliability analysis. In SPSS, this can be achieved using the Recode option
discussed in 7 Chap. 5. Furthermore, we have to be aware of there being multiple factors in our
item set. Some multi-item scales comprise subsets of items that measure different facets of a
multidimensional construct. For example, soccer fan satisfaction is a multidimensional construct
that includes aspects such as satisfaction with the stadium, the merchandise (as described
above), the team, and the coach. Each of these factors is measured using different sets of items,
whose consistency has to be evaluated separately. Calculating one Cronbach’s Alpha over 99
items would certainly be inappropriate. Cronbach’s Alpha is always calculated over the items
belonging to one construct and not all items in the dataset!

same scale. Consequently, instead of looking at the scale’s test-retest reliability, research-
ers consider the scale’s equivalence, thus showing the extent to which two measures of the
same general trait agree. We call this type of reliability the internal consistency reliability.
In the example of satisfaction with the stadium, we compute this scale’s split-half reli-
8 ability manually by, for example, splitting up the scale into x1 on the one side and x2 and
x3 on the other. We then compute the sum of x2 and x3 (or calculate the items’ average)
to form a total score and correlate this score with x1. A high correlation indicates that the
two subsets of items measure related aspects of the same underlying construct and, thus,
suggests a high degree of internal consistency. However, with many items, there are many
different ways to split the variables into two groups.
Cronbach (1951) proposed calculating the average of all possible split-half coefficients
resulting from different ways of splitting the scale items. The Cronbach’s Alpha coefficient
has become by far the most popular measure of internal consistency. In the example above,
this would comprise calculating the average of the correlations between (1) x1 and x2 + x3,
(2) x2 and x1 + x3, as well as (3) x3 and x1 + x2. The Cronbach’s Alpha coefficient gener-
ally varies from 0 to 1, whereas a generally agreed lower limit for the coefficient is 0.70.
However, in exploratory studies, a value of 0.60 is acceptable, while values of 0.80 or higher
are regarded as satisfactory in the more advanced stages of research (Hair et al. 2011). In
Box 8.2, we provide more advice on the use of Cronbach’s Alpha. We will illustrate a reli-
ability analysis using the standard SPSS module in the example at the end of this chapter.

8.5 Structural Equation Modeling

Whereas a confirmatory factor analysis involves testing if and how items relate to specific
constructs, structural equation modeling involves the estimation of relations between these
constructs. It has become one of the most important methods in social sciences, includ-
ing marketing research.
There are broadly two approaches to structural equation modeling: Covariance-based
structural equation modeling (e.g., Jöreskog 1971) and partial least squares structural equa-
tion modeling (e.g., Wold 1982), simply referred to as CB-SEM and PLS-SEM. Both estima-
tion methods are based on the idea of an underlying model that allows the researcher to
test relationships between multiple items and constructs.
8.5 · Structural Equation Modeling
281 8
. Figure 8.8 shows an example path diagram with four constructs (represented by circles
or ovals) and their respective items (represented by boxes).6 A path model incorporates
two types of constructs: (1) exogenous constructs (here, satisfaction with the stadium (Y1)
and satisfaction with the merchandise (Y2)) that do not depend on other constructs, and
(2) endogenous constructs (here, overall satisfaction (Y3) and loyalty (Y4)) that depend on
one or more exogenous (or other endogenous) constructs. The relations between the con-
structs (indicated with p) are called path coefficients, while the relations between the con-
structs and their respective items (indicated with l) are the item loadings. One can distin-
guish between the structural model that incorporates the relations between the constructs
and the (exogenous and endogenous) measurement models that represent the relationships
between the constructs and their related items. Items that measure constructs are labeled x.
In the model in . Fig. 8.8, we assume that the two exogenous constructs satisfaction with
the stadium and satisfaction with the merchandise relate to the endogenous construct overall
satisfaction and that overall satisfaction relates to loyalty. Depending on the research question,
we could of course incorporate additional exogenous and endogenous constructs. Using
empirical data, we could then test this model and, thus, evaluate the relationships between
all the constructs and between each construct and its items. We could, for example, assess
which of the two constructs, Y1 or Y2, relates more strongly to Y3. The result helps us when
developing marketing plans to increase overall satisfaction and, ultimately, loyalty.
The evaluation of a path model analysis requires several steps that include the assess-
ment of both measurement models and the structural model. Diamantopoulos and Siguaw
(2000) and Hair et al. (2019) provide thorough descriptions of the covariance-based struc-
tural equation modeling approach and its application. Hair et al. (2017a, 2018) provide a
step-by-step introduction on how to set up and test path models using partial least squares
structural equation modeling.

x1 x6
l1 l6
Satisfaction with p11 Overall satisfaction
x2 l2 the stadium (Y1) l7 x7
(Y3)
l3 l8
x3 x8
p34
p21

x4 x9
l4 Satisfaction with l9
Loyalty (Y4)
the merchandise (Y2)
l5 l10
x5 x10
Structural model

Measurement models of the exogenous constructs Measurement models of the endogenous constructs

. Fig. 8.8 Path diagram (structural equation modeling)

6 Note that we omitted the error terms for clarity.

 282 Chapter 8 · Principal Component and Factor Analysis

8.6 Example

In this example, we take a closer look at some of the items from the Oddjob Airways dataset
(⤓ Web Appendix → Downloads). This dataset contains eight items that relate to the cus-
tomers’ experience when flying with Oddjob Airways. For each of the following items, the
respondents had to rate their degree of agreement from 1 (“completely disagree”) to 100
(“completely agree”). The variable names are included below:
44with Oddjob Airways you will arrive on time (s1),
44the entire journey with Oddjob Airways will occur as booked (s2),
44in case something does not work out as planned, Oddjob Airways will find a good
solution (s3),
44the flight schedules of Oddjob Airways are reliable (s4),
44Oddjob Airways provides you with a very pleasant travel experience (s5),
44Oddjob Airways’s on board facilities are of high quality (s6),
44Oddjob Airways’s cabin seats are comfortable (s7), and
44Oddjob Airways offers a comfortable on-board experience (s8).

Our aim is to reduce the complexity of this item set by extracting several factors. Hence,
8 we use these items to run a PCA.

8.6.1 Principal Component Analysis

8.6.1.1 Check Requirements and Conduct Preliminary Analyses

Before initiating the analysis, we need to check the assumptions of PCA and carry out
preliminary analyses. We find that all eight variables are interval scaled from 1 (“very
unsatisfied”) to 100 (“very satisfied”), therefore meeting the requirements in terms of the
measurement scale.
Determining whether the variables are sufficiently correlated is part of the PCA. To
initiate the analysis, go to ► Analyze ► Dimension Reduction ► Factor. In the dialog box
that opens (. Fig. 8.9), enter all eight variables into the Variables box.
Next, click on Descriptives (. Fig. 8.10) and check the boxes next to Coefficients, Sig-
nificance levels, Anti-image, and KMO and Bartlett’s test of sphericity. Also make sure to
select Reproduced, which requests the reproduced correlation matrix that we will use
for assessing the goodness-of-fit of the factor solution. Finally, select Initial solution and
Univariate descriptives in the upper part of the dialog box to display useful summary
statistics of your data. All other options are of minor importance, so skip these and click
Continue.
Now click on Extraction (. Fig. 8.11) and choose Principal components in the Method
drop-down menu. Under Extract, you can determine the rule for factor extraction. The
default option in SPSS is the Kaiser criterion as indicated by the option Eigenvalues greater
than 1. In case you have prior information regarding the number of factors to extract, select
Fixed number of factors and specify the corresponding number next to Factors to extract.
Finally, under Display, make sure to select Unrotated factor solution and Scree plot. Click
on Continue.
8.6 · Example
283 8

. Fig. 8.9 Factor analysis dialog box

. Fig. 8.10 Descriptives option

(factor analysis)

Under Rotation, you can choose between several orthogonal and oblique rotation
methods. Select the Varimax procedure and click on Continue. Finally, under Options, you
can decide how missing values should be handled and specify the display format of the
coefficients in the component matrix. Select Exclude cases listwise to eliminate observations
that have missing values in any of the variables used in any of the analyses. Avoid replacing
missing values with the mean as this will diminish the variation in the data, especially if
there are many missing values in your dataset. You should always check the option Sorted
by size under Coefficient Display Format, as this greatly increases the clarity of the display
of results. If you wish, you can suppress low loadings of say less than 0.20 by selecting the
option Suppress small coefficients. Particularly when analyzing many items, this option
makes the output much easier to interpret. After having specified all the options, you can
initiate the analysis by clicking on Continue, followed by OK.
 284 Chapter 8 · Principal Component and Factor Analysis

8 . Fig. 8.11 Extraction option (factor analysis)

The descriptive statistics in . Table 8.4 reveal that there are several observations with
missing values in the dataset. However, with 921 valid observations, the sample size
requirements are clearly met, even if the analysis produces very low communality values.
The correlation matrix in the upper part of . Table 8.5 indicates that there are several
pairs of highly correlated variables. The values in the diagonal are all 1.000, which is logical,
as this is the correlation between a variable and itself! The off-diagonal cells correspond
to the pairwise correlations. For example, the pairwise correlation between s1 and s2 is
.754. The corresponding value in the lower part of . Table 8.5 shows that all correlations
are significant (.000). As an absolute minimum standard, we need at least one correlation
in the off-diagonal cells to be significant and we clearly meet this minimum.The correla-
tion matrix in . Table 8.5 also shows that there are several pairs of highly correlated vari-
ables. For example, not only s1 is highly correlated with s2 (correlation = .754), but also s3
is highly correlated with s1 (correlation = .622), just like s4 (correlation = .733). As these
variables’ correlations with the remaining correlations is much lower, it is likely that these
four variables form one factor. As you can see from the correlation matrix, we can already
identify a likely factor structure.
The results in . Table 8.6 indicate a KMO value of .907, which is “marvelous”
(. Table 8.2). Correspondingly, all MSA values shown on the diagonal in the lower part of
the Anti-image Matrices output (. Table 8.7) are high. For example, s1 has an MSA value
of .914. Not surprisingly the Bartlett’s test shown in . Table 8.6 is significant (Sig. = .000),
which means that we can reject the null hypothesis of uncorrelated variables. Summariz-
ing these results, we conclude that the data are appropriate for PCA. Hence, we can con-
tinue with the interpretation of the PCA.
8.6 · Example
285 8

. Table 8.4 Descriptive statistics

Descriptive Statistics

Mean Std. Deviation Analysis N

s1 60.36 26.300 921

s2 59.34 25.963 921

s3 55.83 25.150 921

s4 56.89 27.808 921

s5 56.48 22.269 921

s6 56.10 22.070 921

s7 51.60 24.568 921

s8 57.08 21.286 921

. Table 8.5 Correlation matrix

Correlation Matrix

s1 s2 s3 s4 s5 s6 s7 s8

Correlation s1 1.000 .754 .622 .733 .525 .500 .455 .542

s2 .754 1.000 .697 .771 .536 .493 .453 .535

s3 .622 .697 1.000 .648 .550 .492 .452 .553

s4 .733 .771 .648 1.000 .484 .436 .367 .487

s5 .525 .536 .550 .484 1.000 .818 .788 .811

s6 .500 .493 .492 .436 .818 1.000 .832 .842

s7 .455 .453 .452 .367 .788 .832 1.000 .780

s8 .542 .535 .553 .487 .811 .842 .780 1.000

Sig. s1 .000 .000 .000 .000 .000 .000 .000

(1-tailed)
s2 .000 .000 .000 .000 .000 .000 .000

s3 .000 .000 .000 .000 .000 .000 .000

s4 .000 .000 .000 .000 .000 .000 .000

s5 .000 .000 .000 .000 .000 .000 .000

s6 .000 .000 .000 .000 .000 .000 .000

s7 .000 .000 .000 .000 .000 .000 .000

s8 .000 .000 .000 .000 .000 .000 .000

 286 Chapter 8 · Principal Component and Factor Analysis

. Table 8.6 KMO and Bartlett’s test statistics

KMO and Bartlett's Test

Kaiser-Meyer-Olkin Measure of Sampling Adequacy. .907

Bartlett's Test of Sphericity Approx. Chi-Square 6421.354

df 28

Sig. .000

8.6.1.2 Extract the Factors and Determine the Number of Factors

. Table 8.8 lists the eigenvalues associated with each factor before extraction, after
extraction, and after rotation. In the columns labeled Initial Eigenvalues, we see the results
before extraction. SPSS lists all eight factors (we know that there are potentially as many
factors as there are variables) in this column. Most of these factors are only of minor
importance. This is reflected in each factor’s eigenvalue, which is displayed in the table’s
second column. Here, we see that the first factor has an eigenvalue of 5.249. As there are
eight variables in our dataset, this factor accounts for 5.249/8.00 = 65.611 % of the overall
8 variance, as indicated in the third column. It is quite remarkable that by using only one
factor instead of eight variables, we can account for over 65 % of the overall variance! The
second factor has an eigenvalue of 1.328 and, thus, still covers more variance than a single
variable. In contrast, factors 3–8 only marginally account for the total variance explained,
as their eigenvalues are considerably smaller than 1.
The second set of columns, labeled Extraction Sums of Squared Loadings, contains the
factor solutions after extraction. Since we chose the Kaiser criterion as default option, SPSS
extracts all factors with an eigenvalue greater than 1. As a consequence, SPSS extracts two
factors, which jointly account for 82.215 % of the overall variance. The final part of the
table, labeled Rotation Sums of Squared Loadings, displays the factors after rotation. The
rotation typically changes the factors’ eigenvalues, but not the total variance explained. For
example, before rotation, the second factor accounted for 16.604 % of the overall variance
but after rotation, it accounts for 39.582 %.
Looking at the scree plot in . Fig. 8.12, we find an “elbow” in the line at three factors.
As the number of factors that the scree plot suggests is one factor less than the elbow indi-
cates, we conclude that two factors are appropriate. This finding therefore supports the
conclusion as the Kaiser criterion.
While the Kaiser criterion and the scree plot are helpful for determining the number
of factors to extract, parallel analysis is a more robust criterion. Parallel analysis can only
be run using SPSS syntax. To do so, go to the book’s ⤓ Web Appendix (→ Downloads) and
download the syntax file Parallel analysis.sps (O’Connor 2000). Next, go to ► File ► Open
►Syntax, locate the syntax file on your hard drive and open it. SPSS will now show the
syntax editor as in . Fig. 8.13.
Under compute Ncases in line 10, you need to specify the number of observations,
which is 921 in our case study. Similarly, under compute NVars (line 11), specify the number
. Table 8.7 Anti-image matrices

Anti-image Matrices
8.6 · Example

s1 s2 s3 s4 s5 s6 s7 s8

Anti-image Covariance s1 .354 −.107 −.029 −.112 −8.946E−5 −.007 −.011 −.019

s2 −.107 .289 −.102 −.122 −.009 −.001 −.013 .002

s3 −.029 −.102 .440 −.066 −.036 .010 .004 −.036

s4 −.112 −.122 −.066 .334 −.016 −.004 .037 −.011

s5 −8.946E−5 −.009 −.036 −.016 .247 −.057 −.068 −.063

s6 −.007 −.001 .010 −.004 −.057 .197 −.093 −.085

s7 −.011 −.013 .004 .037 −.068 −.093 .262 −.040

s8 −.019 .002 −.036 −.011 −.063 −.085 −.040 .226

Anti-image Correlation s1 .917a −.334 −.072 −.325 .000 −.025 −.035 −.068

s2 −.334 .884a −.285 −.393 −.034 −.003 −.049 .006

s3 −.072 −.285 .943a −.171 −.111 .034 .011 −.115

s4 −.325 −.393 −.171 .883a −.057 −.014 .125 −.039

s5 .000 −.034 −.111 −.057 .931a −.259 −.269 −.265

287

s6 −.025 −.003 .034 −.014 −.259 .883a −.408 −.400

s7 −.035 −.049 .011 .125 −.269 −.408 .904a −.164

s8 −.068 .006 −.115 −.039 −.265 −.400 −.164 .918a

a Measures of Sampling Adequacy(MSA)
8
 8
288

. Table 8.8 Factor extraction

Total Variance Explained

Initial Eigenvalues Extraction Sums of Squared Loadings Rotation Sums of Squared Loadings

Component Total % of Variance Cumulative % Total % of Variance Cumulative % Total % of Variance Cumulative %
1 5.249 65.611 65.611 5.249 65.611 65.611 3.411 42.633 42.633

2 1.328 16.604 82.215 1.328 16.604 82.215 3.167 39.582 82.215

3 .397 4.966 87.181

4 .263 3.290 90.471

Chapter 8 · Principal Component and Factor Analysis

5 .231 2.890 93.361

6 .196 2.454 95.815

7 .193 2.409 98.224

8 .142 1.776 100.000

Extraction Method: Principal Component Analysis.

8.6 · Example
289 8
. Fig. 8.12 Scree plot
Scree Plot
4

Eigenvalue
2

Elbow
1

1 2 3 4 5 6 7 8
Component Number

. Fig. 8.13 Syntax editor

 290 Chapter 8 · Principal Component and Factor Analysis

. Fig. 8.14 Parallel analysis output 5XQ0$75,;SURFHGXUH

6SHFLILFDWLRQVIRUWKLV5XQ
1FDVHV
1YDUV
1GDWVHWV
3HUFHQW

5DQGRP'DWD(LJHQYDOXHV
5RRW0HDQV3UFQW\OH









 (1'0$75,;

8 of variables used in the analysis, which is 8. Next, initiate the analysis by going to ► Run
► All and SPSS will show an output similar to . Fig. 8.14.
The column labeled Prcntyle shows the 95th percentile for each factor’s eigenvalue
resulting from the randomly generated data. Note that because of this random process,
your numbers are going to look different. However, deviations typically occur at the third
decimal place. We can now compare the original eigenvalues from . Table 8.8 with the ran-
domly generated eigenvalues from . Fig. 8.14. We learn that the first two factors produce
eigenvalues larger than the randomly generated eigenvalues. Whereas the first original
Eigenvalue is clearly higher (5.249; . Table 8.7) than the randomly generated one (1.185;
. Fig. 8.14), the difference is much less pronounced for the second factor (1.328 vs. 1.122).
The third factor’s original eigenvalue (.397) is clearly lower than its randomly generated
counterpart (1.076). Hence, based on the parallel analysis results, we would also opt for
a two-factor solution.

8.6.1.3 Interpret the Factor Solution

We continue with the interpretation of the factors. To do so, take a look at the Rotated Com-
ponent Matrix (. Table 8.9), which shows the item loadings after rotation. To interpret the
factors, we first “assign” each variable to a certain factor based on its maximum absolute
factor loading. That is, if the highest absolute loading is negative, higher values of a par-
ticular variable relate negatively to the assigned factor. After that, we should find a label
for each factor that best describes the set of variables associated with that factor. Looking
at . Table 8.9, we see that s1–s4 load highly on the second factor, whereas s5–s8 load on
the first factor. For example, s1 has a .829 loading on the second factor, while its loading is
much weaker on the first factor (.299). If you compare these results with the regular Com-
ponent Matrix, you will see that prior to rotation, all items load highly on the first factor
and much weaker on the second.
Having identified which variables load highly on which factor in the rotated solution,
we now need to identify labels for each factor. Looking at the variable labels, we learn
8.6 · Example
291 8

. Table 8.9 Rotated component matrix

Rotated Component Matrixa

Component

1 2
s6 .903 .272

s7 .902 .208

s8 .856 .352

s5 .852 .347

s4 .198 .885

s2 .282 .871

s1 .299 .829

s3 .340 .759

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.
a Rotation converged in 3 iterations.

that the first set of variables (s1–s4) relate to reliability aspects of the journey and related
processes, such as the booking. We could therefore label this factor (i.e., factor 2) reli-
ability. The second set of variables (s5–s8) relate to different aspects of the onboard facil-
ities and the travel experience. Hence, we could label this factor (i.e., factor 1) onboard
experience. The labeling of factors is subjective and you could provide different labels.

8.6.1.4 Evaluate the Goodness-of-Fit of the Factor Solution

The last step involves assessing the analysis’s goodness-of-fit. To do so, we first look at the
residuals (i.e., the differences between observed and reproduced correlations) in the repro-
duced correlation matrix (. Table 8.10). Examining the lower part of the table, we see that
there are several residuals with absolute values larger than 0.05. However, we do not have
to count every single value in the matrix (this could be quite exhausting if there are over
100 variables in the dataset!). Instead, SPSS counts the proportion of residuals with high
residuals, which is reported in the first part of the table. As we can see in point b beneath
the table, 28.0 % of the residuals have absolute values greater than 0.05. As the percentage
of increased residuals is well below 50 %, we can presume a good model fit.
This result is supported by the variables’ communalities and the total variance explained.
All communalities in . Table 8.11 are very high, indicating that the factors reproduce the
variables’ variance well. For example, with a value of .763, s1 has a communality value clearly
above the 0.50 threshold, suggesting that the two factors account for 76.3 % of this variable’s
variation. Finally, the two factors explain over 80 % of each variable’s variance (. Table 8.8).
Jointly these results provide support for the solution’s goodness-of-fit.

> In case the analysis indicates a poor goodness-of-fit, you should reconsider the
set-up by eliminating items that have low communalities and MSA values.
 8
292

. Table 8.10 Reproduced correlations and residual matrices

Reproduced Correlations

s1 s2 s3 s4 s5 s6 s7 s8

Reproduced Correlation s1 .777a .806 .731 .793 .542 .495 .442 .548

s2 .806 .838a .757 .827 .542 .491 .435 .548

s3 .731 .757 .691a .739 .553 .513 .464 .558

s4 .793 .827 .739 .822a .476 .420 .363 .482

s5 .542 .542 .553 .476 .847a .864 .840 .852

s6 .495 .491 .513 .420 .864 .890a .871 .869

s7 .442 .435 .464 .363 .840 .871 .856a .845

s8 .548 .548 .558 .482 .852 .869 .845 .856a

Residualb s1 −.053 −.109 −.060 −.017 .005 .014 −.006

Chapter 8 · Principal Component and Factor Analysis

s2 −.053 −.060 −.056 −.006 .002 .018 −.013

s3 −.109 −.060 −.091 −.003 −.022 −.012 −.006

s4 −.060 −.056 −.091 .008 .016 .004 .005

s5 −.017 −.006 −.003 .008 −.046 −.053 −.040

s6 .005 .002 −.022 .016 −.046 −.039 −.026

s7 .014 .018 −.012 .004 −.053 −.039 −.065

s8 −.006 −.013 −.006 .005 −.040 −.026 −.065

Extraction Method: Principal Component Analysis.

a Reproduced communalities
b Residuals are computed between observed and reproduced correlations. There are 8 (28.0 %) nonredundant residuals with absolute values greater than 0.05.
8.6 · Example
293 8
. Table 8.11 Communalities

Communalities

Initial Extraction
s1 1.000 .777

s2 1.000 .838

s3 1.000 .691

s4 1.000 .822

s5 1.000 .847

s6 1.000 .890

s7 1.000 .856

s8 1.000 .856

Extraction Method: Principal Component Analysis.

8.6.1.5 Compute the Factor Scores

If you wish to use factor scores in subsequent analyses, you can rerun the analysis by going
to ► Analyze ► Dimension Reduction ► Factor and selecting the Scores option. Since we
recommend using the regression method, choose Save as variables and Regression. When
clicking on Continue and OK, SPSS creates two new variables labeled FAC1_1 and FAC2_1,
which include the scores of the two factors (. Fig. 8.15). These are z-standardized, meaning
that the newly generated variables have mean values of 0 and standard deviations of 1. This
entails the factor scores are estimated in units of standard deviations from their means.
For example, the first observation is about 1.911 standard deviations below average on the
onboard experience factor (i.e., factor 1) and about 0.914 standard deviations above the reli-
ability factor (i.e., factor 2). In contrast, the second observation is clearly above average in
terms of onboard experience (1.684) and reliability (1.297). Using these variables as input
we could, for example, evaluate whether male and female customers differ significantly
with regard to their satisfaction with the airline’s reliability, and the onboard experience.

! SPSS can only calculate factor scores if it has information on all the variables
included in the analysis. If SPSS does not have all the information, it only shows a “.”
(dot) in the data view window, indicating a system-missing value.

8.6.2 Reliability Analysis

To illustrate its usage, let’s carry out a reliability analysis of the first factor onboard experi-
ence by calculating Cronbach’s Alpha as a function of variables s5 to s8. To run the reliabil-
ity analysis, click on ► Analyze ► Scale ► Reliability Analysis. Next, enter variables s5,
s6, s7, and s8 into the Items box (again, you may have to right-click on the items and select
Display Variable Names to show the names instead of the variable labels) and type in the
 294 Chapter 8 · Principal Component and Factor Analysis

. Fig. 8.15 SPSS data view window

8 scale’s name (e.g., satisfaction with the onboard experience). Make sure that Alpha is selected
in the Model drop-down list (. Fig. 8.16).
Next, click on Statistics and choose Scale if item deleted (. Fig 8.17). You can also
request several descriptive statistics for each item (including correlations) and the entire
scale. However, for the sake of simplicity, we will work with the default settings. Click on
Continue, followed by OK.
The results in . Table 8.12 show that the scale exhibits a high degree of internal consis-
tency reliability. With a value of .944, the Cronbach’s Alpha coefficient lies well above the
commonly suggested threshold of .70. This result is not surprising, since we are simply
testing a scale previously established using item correlations. Keep in mind that we should

. Fig. 8.16 Reliability analysis dialog box

8.6 · Example
295 8

. Table 8.12 Reliability statistics

Reliability Statistics

Cronbach's Alpha No of Items

.944 4

carry out a reliability analysis to test a scale using a different sample—this example is only
for illustration purposes! The rightmost column of . Table 8.13 indicates what the Cron-
bach’s Alpha would be if we deleted the item indicated in that row. When we compare
each of the values with the overall Cronbach’s Alpha value, we can see that any change in
the scale’s set-up would reduce the Cronbach’s Alpha value. For example, by removing s5
from the scale, the Cronbach’s Alpha of the new scale comprising only s6, s7, and s8 would
be reduced to .928. In the column labeled Corrected Item-Total Correlation of . Table 8.13,
SPSS indicates the correlation between the item and the scale that is composed of other
items. This information is useful for determining whether reverse-coded items were also
identified as such. Reverse-coded items should have a minus sign.

. Table 8.13 Item-total statistics

Item-Total Statistics

Scale Mean if Item Scale Variance if Corrected Item-­ Cronbach's Alpha

Deleted Item Deleted Total Correlation if Item Deleted
s5 165.53 4084.951 .861 .928

s6 165.91 4045.200 .896 .917

s7 170.42 3859.797 .852 .933

s8 164.96 4215.151 .861 .929

. Fig. 8.17 Statistics option

(reliability analysis)
 296 Chapter 8 · Principal Component and Factor Analysis

8.7 Customer Satisfaction at Haver & Boecker (Case Study)

Case Study

© Haver & Boecker OHG

© Haver & Boecker OHG

Haver & Boecker (http:// and complex. Since the importance of customer
www.haverboecker.com) is company’s philosophy satisfaction and decided
one of the world’s leading is to help customers and to commission a market
and most renowned business partners solve their research project in order to
machine producers in the challenges or problems, identify marketing activities
fields of mineral processing, they often customize their that can positively contribute
as well as the storing, products and services to to the business’s overall
conveying, packing, and meet the buyers’ needs. success. Based on a thorough
loading of bulk material. Therefore, the customer literature review, as well
The family-owned group is not a passive buyer, but as interviews with experts,
operates through its global an active partner. Given the company developed a
network of facilities, with this background, the short survey to explore their
manufacturing units, among customers’ satisfaction customers’ satisfaction with
others, in Germany, the UK, plays an important role in specific performance features
Belgium, US, Canada, Brazil, establishing, developing, and their overall satisfaction.
China, and India. and maintaining successful All the items were measured
The company’s relationships customer relationships. on seven-point scales, with
with its customers are Early on, the company’s higher scores denoting
usually long-term oriented management realized the higher levels of satisfaction.
8.8 · Review Questions
297 8

A standardized survey was 44 Overall, how satisfied are data are sufficiently
mailed to customers in you with the supplier? correlated?
12 countries worldwide, (overall) (b) How many factors
which resulted in 281 fully would you extract?
completed questionnaires. Your task is to analyze the Base your decision on
The following items (names dataset to provide the the Kaiser criterion,
in parentheses) were listed in management of Haver the scree plot, and
the survey: & Boecker with advice parallel analysis. Do
44 Reliability of the machines for effective customer these three methods
and systems. (s1) satisfaction management. The suggest the same
44 Life-time of the machines dataset is labeled Haver and number of factors?
and systems. (s2) Boecker.sav (⤓ Web Appendix (c) Find suitable labels for
44 Functionality and → Downloads). the extracted factors.
user-friendliness 1. Using regression analysis, (d) Evaluate the
operation of the machines locate those variables factor solution’s
and systems. (s3) that best explain the goodness-of-fit.
44 Appearance of the customers’ overall 3. Use the factor scores and
machines and systems. (s4) satisfaction (overall). regress the customers’
44 Accuracy of the machines Evaluate the model fit overall satisfaction
and systems. (s5) and assess the impact (overall) on these.
44 Timely availability of the of each variable on the Evaluate the strength of
after-sales service. (s6) dependent variable. the model and compare it
44 Local availability of the Remember to consider with the initial regression.
after-sales service. (s7) collinearity diagnostics. What should Haver &
44 Fast processing of 2. Determine the factors Boecker’s management
complaints. (s8) that characterize the do to increase their
44 Composition of respondents using factor customers’ satisfaction?
quotations. (s9) analysis. Use items s1–s12 4. Calculate the Cronbach’s
44 Transparency of for this. Run a PCA with Alpha over items s1–s5
quotations. (s10) varimax rotation to help and interpret the results.
44 Fixed product prize interpretation. Consider
for the machines and the following aspects: For further information on
systems. (s11) (a) Are all assumptions the dataset and the study,
44 Cost/performance ratio for carrying out a see Festge and Schwaiger
of the machines and PCA met? Are the (2007), as well as Sarstedt
systems. (s12) et al. (2009).

8.8 Review Questions

1. What is factor analysis? Try to explain what factor analysis is in your own words.
2. What is the difference between exploratory factor analysis and confirmatory factor
analysis?
3. What is the difference between PCA and factor analysis?
4. Describe the terms communality, eigenvalue, and factor loading. How do these
concepts relate to one another?
5. Describe the Kaiser criterion, the scree plot, and parallel analysis to determine
the number of factors. What are there similarities and differences between these
methods?
 298 Chapter 8 · Principal Component and Factor Analysis

6. What is the purpose of a varimax rotation? Does a rotation alter eigenvalues or

factor loadings?
7. Re-run the Oddjob Airways case study by carrying out a factor analysis and compare
the results to the PCA analysis described in 7 Sect. 8.6. Are there any differences?
8. What is reliability analysis and why is it important?
9. Explain the basic principle of structural equation modeling.

References
Brown, J. D. (2009). Choosing the right type of rotation in PCA and EFA. JALT Testing & Evaluation SIG News-
letter, 13(3), 20–25.
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2),
245–276.
Cliff, N. (1987). Analyzing multivariate data. New York, NJ: Harcourt Brace Jovanovich.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
Diamantopoulos, A., & Siguaw, J. A. (2000). Introducing LISREL: A guide for the uninitiated. London: Sage.
Dinno, A. (2009). Exploring the sensitivity of Horn’s parallel analysis to the distributional form of random
8 data. Multivariate Behavioral Research, 44(3), 362–388.
DiStefano, C., Zhu, M., & Mîndriă, D. (2009). Understanding and using factor scores: Considerations fort he
applied researcher. Practical Assessment, Research & Evaluation, 14(20), 1–11.
Festge, F., & Schwaiger, M. (2007). The drivers of customer satisfaction with industrial goods: An interna-
tional study. Advances in International Marketing, 18, 179–207.
Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Graffelman, J. (2013). Linear-angle correlation plots: New graphs for revealing correlation structure. Jour-
nal of Computational and Graphical Statistics, 22(1), 92–106.
Grice, J. W. (2001). Computing and evaluating factor scores. Psychological Methods, 6(4), 430–450.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis. A global perspective
(8th ed.). Boston. MA: Cengage.
Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. Journal of Marketing Theory
and Practice, 19(2), 139–151.
Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017a). A primer on partial least squares structural equa-
tion modeling (PLS-SEM) (2nd ed.). Thousand Oaks, CA: Sage.
Hair, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., & Thiele, K. O. (2017b). Mirror, mirror on the wall. A com-
parative evaluation of composite-based structural equation modeling methods. Journal of the Acad-
emy of Marketing Science, 45(5), 616–632.
Hair, J. F., Sarstedt, M., Ringle, C. M., & Gudergan, S. P. (2018). Advanced issues in partial least squares struc-
tural equation modeling (PLS-SEM). Thousand Oaks, CA: Sage.
Hamilton, L. C. (2013), Statistics with Stata: Version 12: Cengage Learning.
Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis:
A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191–205.
Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common
errors and some comment on improved practice. Educational and Psychological Measurement, 66(3),
393–416.
Hershberger, S. L. (2005). Factor scores. In: B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in
behavioral science (pp. 636–644). New York, NJ: John Wiley.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2),
179–185.
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36(4), 409–426.
Kaiser, H. F. (1958). The varimax criterion for factor analytic rotation in factor analysis. Educational and Psy-
chological Measurement, 23(3), 770–773.
Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36.
References
299 8
Kim, J. O., & Mueller, C. W. (1978). Introduction to factor analysis: What it is and how to do it. Thousand Oaks,
CA: Sage.
Longman, R. S., Cota, A. A., Holden, R. R., & Fekken, G. C. (1989). A regression equation for the parallel anal-
ysis criterion in principal components analysis: Mean and 95th percentile Eigenvalues. Multivariate
Behavioral Research, 24(1), 59–69.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological
Methods, 4(1), 84–99.
Matsunga, M. (2010). How to factor-analyze your data right: Do’s and don’ts and how to’s. International
Journal of Psychological Research, 3(1), 97–110.
Mulaik, S. A. (2009). Foundations of factor analysis (2nd ed.). London: Chapman & Hall.
O'Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel
analysis and Velicer's MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396–402.
Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift’s electric factor analysis machine. Under-
standing Statistics, 2(1), 13–43.
Russell, D. W. (2002). In search of underlying dimensions: The use (and abuse) of factor analysis in Person-
ality and Social Psychology Bulletin. Personality and Social Psychology Bulletin, 28(12), 1629–1646.
Sarstedt, M., Schwaiger, M., & Ringle, C. M. (2009). Do we fully understand the critical success factors of
customer satisfaction with industrial goods? Extending Festge and Schwaiger’s model to account for
unobserved heterogeneity. Journal of Business Market Management, 3(3), 185–206.
Sarstedt, M., Ringle, C. M., Raithel, S., & Gudergan, S. (2014). In pursuit of understanding what drives fan
satisfaction. Journal of Leisure Research, 46(4), 419–447.
Sarstedt, M., Hair, J. F., Ringle, C. M., Thiele, K. O., & Gudergan, S. P. (2016). Estimation issues with PLS and
CBSEM: Where the bias lies!. Journal of Business Research, 69(10), 3998–4010.
Steiger, J. H. (1979). Factor indeterminacy in the 1930's and the 1970's some interesting parallels. Psycho-
metrika, 44(2), 157–167.
Stevens, J. P. (2009). Applied multivariate statistics for the social sciences (5th ed.). Hillsdale: Erlbaum.
Velicer, W. F., & Jackson, D. N. (1990). Component analysis versus common factor analysis: Some issues in
selecting an appropriate procedure. Multivariate Behavioral Research, 25(1), 1–28.
Widaman, K. F. (1993). Common factor analysis versus principal component analysis: Differential bias in
representing model parameters?. Multivariate Behavioral Research, 28(3), 263–311.
Wold, H. O. A. (1982). Soft modeling: The basic design and some extensions. In: K. G. Jöreskog & H. O. A.
Wold (Eds.), Systems under indirect observations: Part II (pp. 1–54). Amsterdam: North-Holland.
Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components
to retain. Psychological Bulletin, 99(3), 432–442.

Further Reading
Nunnally, J. C., & Bernstein, I. H. (1993). Psychometric theory (3rd ed.). New York: McGraw-Hill.
Stewart, D. W. (1981). The application and misapplication of factor analysis in marketing research. Journal
of Marketing Research, 18(1), 51–62.