Factor Analysis

Can be used to determine if related subtests “cluster” together

Can be used in a MTMM study to determine if traits “cluster” together

Can be used in scale development to determine if related items “cluster” together
Extracting Factors

Factor analysis starts with a correlation matrix for all individual variables, subtests, or
items.

The algorithm initially assumes that only one underlying factor can adequately account
for the association among variables, subtests, or items. In other words, it begins with the
assumption that a one factor model can account for the correlations among item
responses.

To test this assumption, the algorithm must estimate the correlation between the
underlying factor and each item to determine if the correlation between the items is
equivalent to the product of the path coefficients.

Of course, although we know the correlation between items we do not know the path
coefficients. Earlier we stated that one way to determine those path coefficients was to
use the correlation among items.

One approach to estimate the path coefficients is to use the total scale score as a proxy for
the factor or latent variable. This makes sense because the actual item scores are
presumed to be determined from this one latent variable.

We can then use the item-total correlation as a proxy for the correlation between the
observed items and the unobserved latent variable or factor. Furthermore, we can
estimate what the correlation between items should be if the one factor model fits the data
using what we know about path diagrams and we can compare that to what the actual
correlation between items actually is. In this example, under the one factor model the
correlation between items should be 0.12 or (0.3)(0.4)

The residual, or difference between the actual and predicted correlation between items,
indicates how well the one factor model fits. A substantial residual implies that the one
factor model doesn't fit the data well. In this example, the residual is 0.18, which
indicates a poor fit.

Of course, in practice the computer does this operation for the whole correlation matrix
simultaneously and rather than ending up with a single residual we end up with a residual
matrix with each entry representing the amount of correlation or covariance between
pairs of items that exists over and beyond what a single factor model could predict.

The algorithm then repeats this procedure using the residual matrix, rather than the
correlation matrix, as input. In other words, the next factor tries to account for variation
in the residual matrix which is variation that is not explained by the first factor.

This process continues until the residual matrix is small enough. It is possible to continue
the procedure until the residual matrix is composed of all zeros. This will occur when the
number of factors is equivalent to the number of items. For a k item scale, k factors will
explain all of the covariance among the k items.
When to Stop a.k.a. How many factors to extract

One of the goals of factor analysis is to create a smaller set of variables (the factors) that
capture the original information nearly as well as the larger set of variables (the items)

Some factor analytic methods, primarily those based on maximum likelihood estimation
and confirmatory models use a statistical criterion which amounts to conducting an
inferential test to determine whether the residual matrix contains an amount of
covariation that is statistically greater than zero. If so the process continues until this is
no longer the case, if not the process stops.

Two widely used non-inferential procedures to determine when enough factors have been
extracted are the eigenvalue rule and the scree test.

The eigenvalue rule makes use of the fact that an eigenvalue represents the amount of
information captured by a factor. In fact, when principal components analysis is used to
extract factors from a k item scale (as is commonly done) an eigenvalue of 1 corresponds
to 1/k % of the total information available in the items. Therefore, a factor with an
eigenvalue of 1 contains the same proportion of total information as does the typical
single item. For this reason, the eigenvalue rule states that only factors with eigenvalues
greater than one should be retained. Of course, this rule is subject to interpretation.
What should one do with an eigenvalue that is 1.09? More likely than not, factors with
eigenvalues close to one can also be eliminated.

The scree test rule is also based on eigenvalues but this rule uses relative, as opposed to
absolute, values as a criterion. It is based on a plot of eigenvalues associated with
successive factors, each of which will diminish in value because they are based on
smaller and smaller residual matrices. This rule calls for retaining factors that lie above
the "elbow" of the plot which is the point at which the information decreases suddently.
This rule is more subjective than the eigenvalue rule.

Do not be afraid to see what happens when you force the number of factors to be
equivalent to the number of subscales on your measure. However, be aware that this
hypothesized structure may not be optimal mathematically. The final solution should be
based both upon your hypothesized structure and the eigenvalues.
Rotating Factors

Typically, the raw unrotated factors are difficult to interpret. For m uncorrelated factors,
the relationship between intercorrelations and factor loadings is:
m
 ij   aik a jk
k 1
where i and j refer to the items, a refers to the factor loadings, and k refers
to the particular factor

This relationship can be obtained in an infinite number of ways. For example, consider
the case of a two factor solution
12 = a11 a21 + a12 a22
Suppose that 12 = 0.25 then we could have
0.25 = 0.3 * 0.6 + 0.7*(0.1)
0.25 = 0.75 * 0.6 + (-.35)(0.57)
OR
or an infinite number of other
solutions

Rotations allow us to mathematically transform factor loadings to aid in interpretation.
Factor rotations increase interpretability by identifying clusters of variables (items) that
can be characterized predominantly by a single factor or latent variable. In other words,
rotation of factors helps to which underlying factor a set of items is most strongly
associated with.

The transformed set of factors will have the same number of factors and will account for
the correlations as well as the original set of factors. The items are not changed nor are
the relationships among the items. The only thing that will change is the magnitude of
the factors.

The ultimate goal is to have each item load primarily on a single factor, called simple
structure. Rotation algorithms use mathematical criteria that are optimized when the
closest possible interpretation to simple structure has been obtained.

Rotations can either be orthogonal (assuming the correlation between the underlying
factors is zero) or oblique (assuming the underlying factors are correlated). When
rotations are orthogonal the reference lines are perpendicular which corresponds to
factors that are statistically independent of each other.

When factors are rotated orthogonally they have nice mathematical properties. For
example, the combined effect of uncorrelated factors is the simple sum of their separate
effects. With oblique factors there is a redundancy in the information provided in the
factors so that the amount of variation explained by two correlated factors is less than the
sum of the parts.

The choice between orthogonal and oblique rotations is largely determined by how one
views the factors conceptually. If theory strongly suggests that the factors are correlated
than the rotation method chosen should reflect this. If the underlying factors are thought
to be minimally related than it makes sense to choose the simpler rotation method, which
is an orthogonal rotation,

Orthogonal Rotations
Quartimax : This rotation tries to load each variable mainly on one factor to try
and “clean up” the variables. The problem with this method is that most of the
variables tend to load on a single factor
Varimax : This rotations tries to “clean up” the factors and is usually easiest to
interpret

Oblique Rotations include the following : Oblimax, Oblimin, Quartimin, Maxplane,
Orthoblique, and Promax.
Steps to running factor analysis in SPSS
1. Select "Factor" under "Data Reduction" under "Analyze" on the menu bar in SPSS.
2. For the initial analyses you can use the defaults provided by SPSS. However if you want
to change the criterion for extracting factors you do so using the extraction button.
3. Be sure to select a rotation method to ease interpretation. In my experience the varimax
rotation usually results in the most interpretable solution, but feel free to try other
methods. If your right click with your mouse, while pointing at a rotational method or
parameter (i.e. delta or kappa_ SPSS gives you a brief description of the rotational
method or parameter. Also be sure to specify that you want the rotated solution. No
sense in doing a rotation and then interpreting the unrotated solution!
4. For interpretation purposes it is helpful to have the variables sorted by size as well as to
suppress factor loading that are less than a pre-specified number (usually between 0.1 and
0.3). These specifications can be found under "Options", as can specifications for dealing
with missing data.
5. This is what the output will look like…
Communalities
DSE1
DSE2
DSE3
DSE4
DSE5
DSE6
DSE7
DSE8
DSE9
DSE10
DSE11
DSE12
DSE13
DSE14
DSE15
DSE16
DSE17
DSE18
DSE19
DSE20
DSE21
DSE22
DSE23
DSE24
DSE25
DSE26
DSE27
DSE28
DSE29
DSE30
DSE31
DSE32
DSE33
DSE34
DSE35
DSE36
DSE37
DSE38
DSE39
DSE40
DSE41
DSE42
Initial
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Extraction
.522
.740
.675
.737
.695
.490
.588
.771
.538
.732
.581
.535
.549
.387
.498
.613
.623
.594
.657
.214
.615
.559
.727
.605
.582
.765
.629
.623
.639
.654
.622
.682
.559
.691
.505
.543
.742
.537
.559
.598
.665
.599
Extraction Method: Principal Component Analysis.
Total Variance Explained
Component
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Total
13.879
3.719
2.959
1.888
1.727
1.268
.915
.870
.837
.814
.717
.683
.650
.633
.587
.579
.570
.548
.502
.473
.455
.449
.439
.425
.425
.404
.389
.365
.357
.329
.321
.306
.305
.299
.290
.270
.253
.245
.229
.223
.211
.194
Initial Eigenvalues
% of Variance
Cumulative %
33.045
33.045
8.855
41.900
7.046
48.946
4.496
53.442
4.112
57.554
3.019
60.573
2.179
62.752
2.072
64.824
1.993
66.817
1.937
68.754
1.706
70.461
1.625
72.086
1.548
73.634
1.507
75.141
1.396
76.538
1.378
77.916
1.357
79.273
1.305
80.578
1.196
81.774
1.125
82.899
1.082
83.982
1.070
85.051
1.045
86.096
1.013
87.109
1.012
88.121
.963
89.084
.927
90.011
.868
90.879
.851
91.730
.783
92.512
.763
93.276
.729
94.005
.726
94.731
.711
95.442
.689
96.132
.642
96.774
.602
97.376
.583
97.959
.545
98.504
.532
99.036
.503
99.539
.461
100.000
Extraction Method: Principal Component Analysis.
Extraction Sums of Squared Loadings
Total
% of Variance
Cumulative %
13.879
33.045
33.045
3.719
8.855
41.900
2.959
7.046
48.946
1.888
4.496
53.442
1.727
4.112
57.554
1.268
3.019
60.573
Rotation Sums of Squared Loadings
Total
% of Variance
Cumulative %
5.264
12.534
12.534
5.145
12.249
24.783
4.824
11.485
36.268
4.333
10.316
46.584
3.895
9.273
55.857
1.981
4.717
60.573
Scree Plot
16
14
12
10
8
Eigenvalue
6
4
2
0
1
4
7
10
13
Component Number
16
19
22
25
28
31
34
37
40
Component Matrixa
DSE34
DSE24
DSE18
DSE38
DSE4
DSE42
DSE6
DSE19
DSE35
DSE10
DSE29
DSE39
DSE25
DSE12
DSE16
DSE23
DSE27
DSE22
DSE30
DSE17
DSE3
DSE37
DSE33
DSE5
DSE41
DSE32
DSE26
DSE1
DSE36
DSE21
DSE7
DSE13
DSE9
DSE31
DSE15
DSE20
DSE14
DSE40
DSE2
DSE8
DSE28
DSE11
1
.776
.732
.719
.706
.689
.676
.668
.668
.655
.635
.628
.626
.621
.618
.587
.585
.577
.574
.574
.573
.568
.564
.559
.556
.555
.555
.535
.530
.521
.520
.503
.501
.470
.469
.434
.409
.394
.418
.477
.521
.435
.448
-.130
Component
3
4
-.102
-.184
-.165
-.105
-.189
-.118
-.286
-.164
-.184
-.273
-.142
-.170
-.265
-.266
-.178
-.242
-.183
-.340
.360
-.131
.490
2
-.136
-.105
-.234
-.309
.408
.412
-.217
-.356
.361
.327
.446
-.369
.303
.353
-.351
-.258
.391
.450
.378
.295
.464
.436
.468
-.287
-.372
.112
-.382
-.102
-.404
-.469
.151
-.427
-.487
.523
-.151
.411
5
-.206
-.108
-.416
-.118
.228
.169
-.488
-.273
.243
.256
-.178
-.420
.131
-.459
.139
.134
.413
.453
.144
.110
.115
.135
-.242
.202
-.234
-.257
.395
.394
-.117
.135
Extraction Method: Principal Component Analys is.
a. 6 components extracted.
-.184
-.151
-.191
.352
-.125
-.165
.390
.343
-.202
-.121
.467
-.162
-.153
.120
.135
.421
-.392
-.320
.432
-.327
.118
.362
.423
.554
.522
6
-.127
-.237
-.110
-.103
.218
-.111
.204
-.121
.170
.120
.477
.463
.350
Rotated Component Matrixa
DSE37
DSE32
DSE3
DSE41
DSE27
DSE21
DSE9
DSE15
DSE7
DSE19
DSE13
DSE25
DSE1
DSE39
DSE35
DSE30
DSE38
DSE8
DSE26
DSE2
DSE31
DSE40
DSE36
DSE14
DSE20
DSE10
DSE4
DSE16
DSE22
DSE34
DSE24
DSE18
DSE23
DSE5
DSE11
DSE17
DSE33
DSE28
DSE6
DSE29
DSE42
DSE12
1
.833
.788
.780
.779
.735
.730
.603
.597
.127
.105
.142
.177
.212
.147
.348
.141
.133
.175
.131
.117
.134
.169
.179
.267
.263
.291
2
.116
.172
.134
.130
.205
.102
.741
.720
.715
.687
.655
.650
.564
.550
.398
.113
.124
.127
.142
.269
.188
.243
.183
.124
.184
.372
.434
.416
.217
.252
.129
.217
.245
.201
.198
.244
.197
.357
.230
.262
.252
Component
3
4
.121
.166
.119
.162
.117
.169
.142
.120
.113
.253
.163
.225
.209
.227
.242
.308
.852
.843
.842
.748
.734
.681
.581
.276
.123
.153
.109
.109
.173
.181
.187
.109
.120
6
.115
.135
.114
.197
.161
.129
.365
-.106
.342
.321
.175
.157
.165
.169
.216
.205
.134
.173
.145
.112
.134
.123
.549
.122
.147
.196
.141
.139
.195
.795
.770
.722
.634
.608
.524
.520
.195
.172
.145
.246
.103
.252
.121
.245
.201
.327
.399
.273
.326
Extraction Method: Principal Component Analys is.
Rotation Method: Varimax with Kaiser Normalization.
a. Rotation converged in 6 iterations .
5
.101
.105
.141
.108
.168
.153
.190
.121
.259
.167
.175
.787
.764
.703
.702
.659
.630
.388
.161
.263
.162
.144
.124
.299
.122
.110
.211
.461
.589
.517
.510
Component Transformation Matrix
Component
1
2
3
4
5
6
1
.443
.542
-.676
.161
.132
-.094
2
.499
-.418
-.013
-.485
.583
.022
3
.386
.569
.720
-.009
.084
.024
Extraction Method: Principal Component Analys is.
Rotation Method: Varimax with Kaiser Normalization.
4
.461
-.179
-.010
-.288
-.751
-.330
5
.374
-.414
.133
.809
.063
-.108
6
.229
-.061
-.075
.020
-.261
.933