Factors Affecting
Measures of Fit
David A. Kenny
January 25, 2014
Background
Introduction
Measures of Fit
2
Factors that Affect
Fit Indices
Number of Variables
Model Complexity
Sample Size
Non-normality
Click to go
to a specific
topic.
3
Number of Variables (v)
Anecdotal evidence that models with many
variables have poor fit.
Kenny and McCoach (2003) show that the RMSEA
gets lower as more variables are added to the model
and that the TLI and CFI are relatively stable, but
tend to decline slightly.
We still do not understand why it is that models
with more variables tend to have poor fit.
4
Model Complexity (df)
One way to think about model complexity
is the number of parameters estimated or
conversely the model df. So for a given
dataset, a model with more parameters and so
smaller df is more complex than a model with
fewer parameters and a larger df.
The question asked for a given fit index,
especially if the question is model comparison,
is how much c2 needs to change per df for that
5
fit index not to change.
How much c2 needs to change per df
for the fit index not to change
(theoretical values and for two studies)
Measure
Theoretical Value Study 1b Study 2c
Bentler & Bonett
0
0
0
CFI
1
1
1
AIC
2 √
2
2
Tucker-Lewisa
c2/df
3.56
2.62
RMSEAa
c2/df
3.56
2.62
SABIC
ln[(N +2)/24]
2.54
2.79
BIC
ln(N)
5.71
5.96
ac2 value used from the final model.
bN = 301, v = 9, c2(24) = 88.31 (Holzinger-Swineford)
cN = 388, v = 8, c2(19) = 49.87 (Moreland & Beech)
6
Implications of the
Previous Slide
The different fit indices change by a
constant amount, regardless of the df
change.
Larger values reward parsimony and
smaller values reward complexity. The BIC
rewards parsimony most, and the CFI (after
the Bentler and Bonett) the least.
7
SABIC Penalty
If the RMSEA equals
√[(ln[(N + 2)/24] – 1)/(N - 1)]
then the penalty for the SABIC equals the
same value as the RMSEA or the TLI.
Also the penalty for the SABIC is
greater than the AIC if N > 175.
8
Sample Size (N)
Bentler-Bonett fails to adjust for
sample size: Models with larger sample
sizes have smaller values.
The TLI and CFI do not vary much
with sample size.
The RMSEA and the SRMR are
larger with smaller sample sizes.
Differences in the AIC get larger as
N increases and so is affected by N.
9
Non-normality
Non-normal data (especially high
kurtosis) inflate c2.
Absolute measures of fit, RMSEA
and SRMR, are likely inflated.
Presumably, incremental and
perhaps comparative measures of fit are
less affected by non-normality.
10
References
Click here to go to references.
11