JGASKIN || April 9th, 2013
QUANT EXAMPLE ANALYSIS
This example does not include background/introduction sections, theoretical support for
hypotheses, discussion of findings, limitations, future research directions, conclusions, etc. This
is just an example of how one might slice up the analysis and report and interpret the findings.
Data Screening
Univariate:
- Missing Data:
o RD1 had one missing value, which we imputed with the median. We used median
imputation because RD1 is an ordinal variable (was measured using a Likert scale).
o Two controls had missing values—representing 5% or less of the sample size, so we
imputed the missing values for these continuous (scale) variables (income – 2 missing;
and customer interactions – 16 missing) using the mean of all available values.
- Outliers:
o All variables but one (customer interactions) were on ordinal scales with seven or fewer
intervals, thus extreme value outliers do not exist. For customer interactions, we
examined a box plot for outliers and found two respondents with exceptionally high
values, however, we had no reason to believe these were incorrect values, and we have no
theoretical basis for removing them. Thus they remain simply as high responses.
- Normality
o Once again, since nearly all our variables are based on Likert-type scales, we have no
reason to exclude variables based on skewness unless they exhibit no variance. Thus
rather than testing skewness, we focused on kurtosis. Kurtosis greater than or less than
+/- 1.00 indicates potentially problematic kurtosis (and therefore, lack of sufficient
variance). All of the burnout from management items had borderline kurtosis issues (abs
value between 1 and 2). These are fairly borderline values and we will simply flag them for
potential future issues in subsequent analyses. BC3 and BC4 however, had kurtosis values
around 3.0; therefore there is insufficient variance in those items to retain them.
Accordingly, we have dropped those two items.
Multivariate (tested after measurement model):
- Linearity
o We tested linearity by performing curve estimation regression for all direct effects in our
model. The results show that the relationships between variables are sufficiently linear
(i.e., all p-values were less than 0.05), except between Autonomy and Productivity;
however, no curve estimation was significant either. Accordingly, we have left the
relationship in our model, subject to trimming during subsequent analyses.
- Homoscedasticity
o The results of the homoscedasticity test (scatter plot of zPred on zResid) indicate that the
mediators and SatW are homoscedastic, but Reliability is slightly more heteroskedastic.
As we will be moderating by gender and job category, we retested reliability for each
subgroup (male, female, csr, bcr) and found it to be homoscedastic within each.
- Multicollinearity
o We tested the Variable Inflation Factor for all of the exogenous variables simultaneously.
The VIFs were all less than 2.0, indicating that the exogenous variables are all distinct. (If
1
JGASKIN || April 9th, 2013
you find that they are not all within a good range, you can cite O’Brian 2007 who says that
high VIFs aren’t necessarily a cause of alarm.)
Exploratory Factor Analysis
We conducted an EFA using Maximum Likelihood 1 with Promax rotation 2 to see if the observed variables
loaded together as expected, were adequately correlated, and met criteria of reliability and validity. We
address each of these below for the final seven-factor model depicted in the pattern matrix below:
- Adequacy:
o The KMO and Bartlett’s test for sampling adequacy was significant and the
communalities for each variable were sufficiently high (all above 0.300 and most above
0.600), thus indicating the chosen variables were adequately correlated for a factor
analysis. Additionally, the reproduced matrix had only 2% non-redundant residuals
greater than 0.05, further confirming the adequacy of the variables and 7-factor model.
(If individual items have low communalities (like less than 0.200), you might do yourself
a favor by removing them. These items are probably the ones that also had kurtosis
issues.)
- Reliability:
o The Cronbach’s alphas for the extracted factors are shown below, along with their labels
and specification. All alpha’s were above 0.70 except Unsupportive Coworkers which was
very close at 0.691. The factors are all reflective because their indicators are highly
correlated and are largely interchangeable (Jarvis et al. 2003).
Factor Label Cronbach’s alpha Specification
Feedback 0.907 Reflective
Reliability 0.795 Reflective
Resource Demand Gap 0.800 Reflective
Learning Orientation 0.875 Reflective
Autonomy 0.864 Reflective
Unsupportive coworkers 0.691 Reflective
Satisfaction with work 0.774 Reflective
1
Maximum Likelihood Estimation was chosen in order to determine unique variance among items and the
correlation between factors, and also to remain consistent with our subsequent CFA. Maximum Likelihood also
provides a goodness of fit test for the factor solution.
2
Promax was chosen because the dataset is quite large (n=304) and promax can account for the correlated factors.
2
JGASKIN || April 9th, 2013
- Validity:
o The factors demonstrate sufficient convergent validity, as their loadings were all above
the recommended minimum threshold of 0.350 for a samples size of 300 (Hair et al.,
2010). The factors also demonstrate sufficient discriminant validity, as the correlation
matrix shows no correlations above 0.700, and there are no problematic cross-loadings.
a
Pattern Matrix
Factor
FB RL RD LO AU UC SW
f1 .884
f2 .881
f3 .861
f4 .734
q2 .806
q5 .754
q1 .712
q4 .597
q3 .547
rd3 .895
rd4 .741
rd2 .698
rd1 .601
l3 .894
l1 .831
l2 .806
a1 .897
a2 .844
a3 .753
uc2 .830
uc1 .633
uc3 .528
sw1 .908
sw3 .609
sw2 .472
Extraction Method: Maximum Likelihood Estimation.
Rotation Method: Promax with Kaiser Normalization.
3
JGASKIN || April 9th, 2013
This seven-factor model had a total variance explained of 60%, with all extracted factors having
eigenvalues above 1.0 except one, which was close at 0.989.
Confirmatory Factor Analysis
- Model Fit
o We removed RD3 due to poor loading. UC3 also was somewhat low (0.58); however, we
did not remove it because the factor only had three indicators, and a two-indicator factor
often results in instability. Modification indices were consulted to determine if there was
opportunity to improve the model. Accordingly, we covaried the error terms between f3
and f4. The table below indicates that the goodness of fit for our measurement model is
sufficient.
Metric Observed value Recommended
cmin/df 1.508 Between 1 and 3
CFI 0.965 >0.950
RMSEA 0.041 <0.060
PCLOSE 0.959 >0.050
SRMR 0.051 <0.090
- Validity and Reliability
o To test for convergent validity we calculated the AVE. For all factors, the AVE was above
0.50 except for Unsupportive Coworkers, which was close at 0.460. However, as this
factor is minimally correlated with the other factors in the model, and because the
reliability score (0.716) was greater than 0.700, we felt this was admissible ( i.e., while it
is not especially strong internally, it is, at least, a reliable and distinct construct within
our model).
o To test for discriminant validity we compared the square root of the AVE (on the diagonal
in the matrix below) to all inter-factor correlations. All factors demonstrated adequate
discriminant validity because the diagonal values are greater than the correlations.
o We also computed the composite reliability for each factor. In all cases the CR was above
the minimum threshold of 0.70, indicating we have reliability in our factors. (If you
experience problems during this phase with AVE or CR, it is probably because you did not
have a good EFA solution. I would return to the EFA to work that out first.)
4
JGASKIN || April 9th, 2013
CR AVE 1 2 3 4 5 6 7
1. LearningO 0.876 0.703 0.838
2. Feedback 0.900 0.693 0.197 0.833
3. Reliability 0.804 0.509 0.227 0.007 0.713
4. RDGap 0.805 0.510 -0.180 -0.402 0.021 0.714
5. UnsCoW 0.716 0.460 -0.252 -0.391 -0.008 0.316 0.678
6. Autonomy 0.865 0.680 0.338 0.384 -0.142 -0.425 -0.399 0.825
7. SatW 0.772 0.533 0.486 0.377 -0.092 -0.575 -0.470 0.625 0.730
- Common Method Bias
o Because the data for both IVs and DVs was collected using a single instrument (a survey),
we conducted a common method bias test to determine if a method bias was affecting the
results of our measurement model. The test we used was the “unmeasured latent factor”
method recommended by Podsakoff et al. (2003) for studies that do not explicitly
measure a common factor (as in this study). Comparing the standardized regression
weights before and after adding the Common Latent Factor (CLF) shows that none of the
regression weights are dramatically affected by the CLF—i.e., the deltas are less than
0.200 and the CR and AVE for each construct still meet minimum thresholds.
Nevertheless, to err on the conservative side, we have opted to retain the CLF for our
structural model (by imputing composites in AMOS while the CLF is present), and thus
we have CMB-adjusted values. (Retaining the CLF is not required if you find no CMB.)
- Invariance Tests
o Since we are planning on moderating the structural model with two categorical variables,
we conducted configural and metric invariance tests.
o Gender:
The model fit of the unconstrained measurement models (with groups loaded
separately) had adequate fit (cmin/df = 1.423; CFI 0.942), indicating that the
model is configurally invariant. After constraining the models to be equal, we
found the chi-square difference test to be non-significant (pval>0.05); thus, our
measurement model meets criteria for metric invariance across gender as well.
[note to students] Had it not met the criteria for metric invariance, you would
need to look at the differences between regression weights for the two groups to
see which regression weight was most different. This might then need to be
removed if possible. If not possible, you might rely on MacKenzie et al. 2011
“Construct Measurement and Validation Procedures in MIS and Behavioral
5
JGASKIN || April 9th, 2013
Research: Integrating New and Existing Techniques”, who say that as long as one
item per construct (aside from the constrained one) is metrically invariant, then
you can proceed with further invariance tests (like multi-group moderation). You
can test this using the critical ratios approach described in the video called:
“multigroup moderation in amos – made easy”.
o Job category
The model fit for job category was equally good (cmin/df = 1.356; CFI 0.952). The
chi-square difference test was again non-significant (pval>0.05).
Hypotheses
All hypotheses were tested while controlling for Education, Income, and Number of customers handled
per day. Mediation tests were conducted without the presence of moderators. Multi-group moderation
tests were conducted using the full model, but prior to adding the interaction variables. Interaction effects
were tested using the full dataset, rather than the moderated dataset. These procedures were necessary in
order to have enough power to test each set of hypotheses, and in order to maintain theoretical clarity and
parsimony.
[note to students] You would of course also provide here some theoretical logic for why you included
the controls you included and for why you expect the hypothesized relationships to be observed as
hypothesized.
Mediation
H1a. Learning Orientation mediates the negative relationship between Resources demand gap and
Satisfaction with work.
H1b. Learning Orientation mediates the negative relationship between Resources demand gap and
Reliability.
Multi-group moderation
H2a. The positive relationship between Autonomy and Satisfaction with work will be stronger for males
than for females.
H2b. The positive relationship between Autonomy and Reliability will be stronger for males than for
females.
Interaction
H3a. An increase in Unsupportive Coworkers will strengthen the negative relationship between
Resource Demand Gap and Learning Orientation.
H3b. An increase in Unsupportive Coworkers will weaken the positive relationship between Feedback
and Learning Orientation.
6
JGASKIN || April 9th, 2013
Structural Model
- Create Composites from factor scores
o Composite variables were created using factor scores in AMOS while the CLF was
present. (This is not necessary, but optional. You may retain the full structural model if
you desire – it just gets a bit unwieldy with interactions.)
o Interaction terms were created by standardizing the appropriate variables, and then
multiplying them.
- Model Fit (of initial structural model after fitting – i.e., not during moderation tests).
o The fitted structural model demonstrates adequate fit. In order to achieve good fit, we
were required to add a direct path between resource demand gap and satisfaction with
work, as well as between unsupportive coworkers and satisfaction with work. We felt
these additions were theoretically logical, and probably indicate that the hypothesized
mediation is actually partial rather than full. We additionally covaried the error terms of
the mediators, as we wanted to account for their correlation without adding theoretical
complexity to our model. To remain consistent, 3 we covaried the error terms of the
dependent variables. While there may exist causal relationships between these variables,
this is not the focus of this model. The actions we have taken allow us to account for these
potential correlations without having to explicitly theorize and test them.
Metric Observed value Recommended
cmin/df 1.393 Between 1 and 3
CFI 0.965 >0.950
RMSEA 0.041 <0.060
PCLOSE 0.959 >0.050
SRMR 0.0396 <0.090
- Controls
o The controls did not have a significant impact on either dependent variable, except the
number of customers handled per day had a slight negative effect on Satisfaction with
work (standardized beta = -0.061*).
3
This issue of consistently applying theoretical reasoning when covarying error terms is advocated by David Kenny:
[Link] He also recommends this action be considered especially when the
modification indices indicate that such an action would significantly reduce the chi-square. This second criteria was
also true for this model.
7
JGASKIN || April 9th, 2013
- Hypothesis testing
o Mediation
Mediation was tested using 2000 bias corrected bootstrapping resamples in AMOS. The
direct and indirect effects were analyzed for potential partial mediation (discovered while
fitting the model). Just indirect effects were analyzed for establishing full mediation. The
results are summarized in the Hypotheses Summary table below.
[note to students] In addition to bootstrapping, you may want to follow the Baron and
Kenny approach (direct effects tested, then add mediator, then see if direct effects drop).
o Multi-group Moderation
To test the categorical moderation hypotheses, we produced the critical ratios for the
differences in regression weights between groups. From these critical ratios we calculated
p-values to determine the significance of the difference. The results are summarized in
the Hypothesis Summary table below.
o Interaction
To test the interaction hypotheses we first standardized the IVs and then created product
variables. We then trimmed non-significant interaction regressions one at a time until
only significant paths remained. In this case, only one significant path remained, from
RDxUC to LO. We plotted this interaction as shown below. The results of the interaction
tests are summarized in the Hypothesis Summary table below. Additionally, we observed
that model fit was good (cmin/df = 1.644; CFI 0.981) for the final moderated model.
8
JGASKIN || April 9th, 2013
Hypothesis Summary Table
Mediation Evidence Supported?
H1a. Learning Orientation mediates the negative Direct w/o Med: -.372*** Yes:
relationship between Resources demand gap and Direct w/ Med:0.237*** Partial
Satisfaction with work. Indirect: -.124*** Mediation
H1b. Learning Orientation mediates the negative Direct w/o Med: -.182***
Yes:
relationship between Resources demand gap and Direct w/ Med: 0.056(ns)
Full Mediation
Reliability. Indirect: -.088***
Multi-group moderation
H2a. The positive relationship between Autonomy and Males: 0.486*** Yes:
Satisfaction with work will be stronger for males than Females: 0.267*** Stronger for
for females. ∆Zscore: -2.62*** males
Males: -0.394***
H2b. The positive relationship between Autonomy and No:
Females: -0.274***
Reliability will be stronger for males than for females. No difference
∆Zscore: 0.545(ns)
Interaction
H3a. An increase in Unsupportive Coworkers will Yes:
Interaction effect:
strengthen the negative relationship between Resource Stronger
-0.074*
Demand Gap and Learning Orientation. negative effect
H3b. An increase in Unsupportive Coworkers will
Interaction effect: No:
weaken the positive relationship between Feedback and
0.037(ns) No Effect
Learning Orientation.
