Factor Analysis Can be used to determine if related subtests “cluster” together Can be used in a MTMM study to determine if traits “cluster” together Can be used in scale development to determine if related items “cluster” together Extracting Factors Factor analysis starts with a correlation matrix for all individual variables, subtests, or items. The algorithm initially assumes that only one underlying factor can adequately account for the association among variables, subtests, or items. In other words, it begins with the assumption that a one factor model can account for the correlations among item responses. To test this assumption, the algorithm must estimate the correlation between the underlying factor and each item to determine if the correlation between the items is equivalent to the product of the path coefficients. Of course, although we know the correlation between items we do not know the path coefficients. Earlier we stated that one way to determine those path coefficients was to use the correlation among items. One approach to estimate the path coefficients is to use the total scale score as a proxy for the factor or latent variable. This makes sense because the actual item scores are presumed to be determined from this one latent variable. We can then use the item-total correlation as a proxy for the correlation between the observed items and the unobserved latent variable or factor. Furthermore, we can estimate what the correlation between items should be if the one factor model fits the data using what we know about path diagrams and we can compare that to what the actual correlation between items actually is. In this example, under the one factor model the correlation between items should be 0.12 or (0.3)(0.4) The residual, or difference between the actual and predicted correlation between items, indicates how well the one factor model fits. A substantial residual implies that the one factor model doesn't fit the data well. In this example, the residual is 0.18, which indicates a poor fit. Of course, in practice the computer does this operation for the whole correlation matrix simultaneously and rather than ending up with a single residual we end up with a residual matrix with each entry representing the amount of correlation or covariance between pairs of items that exists over and beyond what a single factor model could predict. The algorithm then repeats this procedure using the residual matrix, rather than the correlation matrix, as input. In other words, the next factor tries to account for variation in the residual matrix which is variation that is not explained by the first factor. This process continues until the residual matrix is small enough. It is possible to continue the procedure until the residual matrix is composed of all zeros. This will occur when the number of factors is equivalent to the number of items. For a k item scale, k factors will explain all of the covariance among the k items. When to Stop a.k.a. How many factors to extract One of the goals of factor analysis is to create a smaller set of variables (the factors) that capture the original information nearly as well as the larger set of variables (the items) Some factor analytic methods, primarily those based on maximum likelihood estimation and confirmatory models use a statistical criterion which amounts to conducting an inferential test to determine whether the residual matrix contains an amount of covariation that is statistically greater than zero. If so the process continues until this is no longer the case, if not the process stops. Two widely used non-inferential procedures to determine when enough factors have been extracted are the eigenvalue rule and the scree test. The eigenvalue rule makes use of the fact that an eigenvalue represents the amount of information captured by a factor. In fact, when principal components analysis is used to extract factors from a k item scale (as is commonly done) an eigenvalue of 1 corresponds to 1/k % of the total information available in the items. Therefore, a factor with an eigenvalue of 1 contains the same proportion of total information as does the typical single item. For this reason, the eigenvalue rule states that only factors with eigenvalues greater than one should be retained. Of course, this rule is subject to interpretation. What should one do with an eigenvalue that is 1.09? More likely than not, factors with eigenvalues close to one can also be eliminated. The scree test rule is also based on eigenvalues but this rule uses relative, as opposed to absolute, values as a criterion. It is based on a plot of eigenvalues associated with successive factors, each of which will diminish in value because they are based on smaller and smaller residual matrices. This rule calls for retaining factors that lie above the "elbow" of the plot which is the point at which the information decreases suddently. This rule is more subjective than the eigenvalue rule. Do not be afraid to see what happens when you force the number of factors to be equivalent to the number of subscales on your measure. However, be aware that this hypothesized structure may not be optimal mathematically. The final solution should be based both upon your hypothesized structure and the eigenvalues. Rotating Factors Typically, the raw unrotated factors are difficult to interpret. For m uncorrelated factors, the relationship between intercorrelations and factor loadings is: m ij aik a jk k 1 where i and j refer to the items, a refers to the factor loadings, and k refers to the particular factor This relationship can be obtained in an infinite number of ways. For example, consider the case of a two factor solution 12 = a11 a21 + a12 a22 Suppose that 12 = 0.25 then we could have 0.25 = 0.3 * 0.6 + 0.7*(0.1) 0.25 = 0.75 * 0.6 + (-.35)(0.57) OR or an infinite number of other solutions Rotations allow us to mathematically transform factor loadings to aid in interpretation. Factor rotations increase interpretability by identifying clusters of variables (items) that can be characterized predominantly by a single factor or latent variable. In other words, rotation of factors helps to which underlying factor a set of items is most strongly associated with. The transformed set of factors will have the same number of factors and will account for the correlations as well as the original set of factors. The items are not changed nor are the relationships among the items. The only thing that will change is the magnitude of the factors. The ultimate goal is to have each item load primarily on a single factor, called simple structure. Rotation algorithms use mathematical criteria that are optimized when the closest possible interpretation to simple structure has been obtained. Rotations can either be orthogonal (assuming the correlation between the underlying factors is zero) or oblique (assuming the underlying factors are correlated). When rotations are orthogonal the reference lines are perpendicular which corresponds to factors that are statistically independent of each other. When factors are rotated orthogonally they have nice mathematical properties. For example, the combined effect of uncorrelated factors is the simple sum of their separate effects. With oblique factors there is a redundancy in the information provided in the factors so that the amount of variation explained by two correlated factors is less than the sum of the parts. The choice between orthogonal and oblique rotations is largely determined by how one views the factors conceptually. If theory strongly suggests that the factors are correlated than the rotation method chosen should reflect this. If the underlying factors are thought to be minimally related than it makes sense to choose the simpler rotation method, which is an orthogonal rotation, Orthogonal Rotations Quartimax : This rotation tries to load each variable mainly on one factor to try and “clean up” the variables. The problem with this method is that most of the variables tend to load on a single factor Varimax : This rotations tries to “clean up” the factors and is usually easiest to interpret Oblique Rotations include the following : Oblimax, Oblimin, Quartimin, Maxplane, Orthoblique, and Promax. Steps to running factor analysis in SPSS 1. Select "Factor" under "Data Reduction" under "Analyze" on the menu bar in SPSS. 2. For the initial analyses you can use the defaults provided by SPSS. However if you want to change the criterion for extracting factors you do so using the extraction button. 3. Be sure to select a rotation method to ease interpretation. In my experience the varimax rotation usually results in the most interpretable solution, but feel free to try other methods. If your right click with your mouse, while pointing at a rotational method or parameter (i.e. delta or kappa_ SPSS gives you a brief description of the rotational method or parameter. Also be sure to specify that you want the rotated solution. No sense in doing a rotation and then interpreting the unrotated solution! 4. For interpretation purposes it is helpful to have the variables sorted by size as well as to suppress factor loading that are less than a pre-specified number (usually between 0.1 and 0.3). These specifications can be found under "Options", as can specifications for dealing with missing data. 5. This is what the output will look like… Communalities DSE1 DSE2 DSE3 DSE4 DSE5 DSE6 DSE7 DSE8 DSE9 DSE10 DSE11 DSE12 DSE13 DSE14 DSE15 DSE16 DSE17 DSE18 DSE19 DSE20 DSE21 DSE22 DSE23 DSE24 DSE25 DSE26 DSE27 DSE28 DSE29 DSE30 DSE31 DSE32 DSE33 DSE34 DSE35 DSE36 DSE37 DSE38 DSE39 DSE40 DSE41 DSE42 Initial 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Extraction .522 .740 .675 .737 .695 .490 .588 .771 .538 .732 .581 .535 .549 .387 .498 .613 .623 .594 .657 .214 .615 .559 .727 .605 .582 .765 .629 .623 .639 .654 .622 .682 .559 .691 .505 .543 .742 .537 .559 .598 .665 .599 Extraction Method: Principal Component Analysis. Total Variance Explained Component 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 Total 13.879 3.719 2.959 1.888 1.727 1.268 .915 .870 .837 .814 .717 .683 .650 .633 .587 .579 .570 .548 .502 .473 .455 .449 .439 .425 .425 .404 .389 .365 .357 .329 .321 .306 .305 .299 .290 .270 .253 .245 .229 .223 .211 .194 Initial Eigenvalues % of Variance Cumulative % 33.045 33.045 8.855 41.900 7.046 48.946 4.496 53.442 4.112 57.554 3.019 60.573 2.179 62.752 2.072 64.824 1.993 66.817 1.937 68.754 1.706 70.461 1.625 72.086 1.548 73.634 1.507 75.141 1.396 76.538 1.378 77.916 1.357 79.273 1.305 80.578 1.196 81.774 1.125 82.899 1.082 83.982 1.070 85.051 1.045 86.096 1.013 87.109 1.012 88.121 .963 89.084 .927 90.011 .868 90.879 .851 91.730 .783 92.512 .763 93.276 .729 94.005 .726 94.731 .711 95.442 .689 96.132 .642 96.774 .602 97.376 .583 97.959 .545 98.504 .532 99.036 .503 99.539 .461 100.000 Extraction Method: Principal Component Analysis. Extraction Sums of Squared Loadings Total % of Variance Cumulative % 13.879 33.045 33.045 3.719 8.855 41.900 2.959 7.046 48.946 1.888 4.496 53.442 1.727 4.112 57.554 1.268 3.019 60.573 Rotation Sums of Squared Loadings Total % of Variance Cumulative % 5.264 12.534 12.534 5.145 12.249 24.783 4.824 11.485 36.268 4.333 10.316 46.584 3.895 9.273 55.857 1.981 4.717 60.573 Scree Plot 16 14 12 10 8 Eigenvalue 6 4 2 0 1 4 7 10 13 Component Number 16 19 22 25 28 31 34 37 40 Component Matrixa DSE34 DSE24 DSE18 DSE38 DSE4 DSE42 DSE6 DSE19 DSE35 DSE10 DSE29 DSE39 DSE25 DSE12 DSE16 DSE23 DSE27 DSE22 DSE30 DSE17 DSE3 DSE37 DSE33 DSE5 DSE41 DSE32 DSE26 DSE1 DSE36 DSE21 DSE7 DSE13 DSE9 DSE31 DSE15 DSE20 DSE14 DSE40 DSE2 DSE8 DSE28 DSE11 1 .776 .732 .719 .706 .689 .676 .668 .668 .655 .635 .628 .626 .621 .618 .587 .585 .577 .574 .574 .573 .568 .564 .559 .556 .555 .555 .535 .530 .521 .520 .503 .501 .470 .469 .434 .409 .394 .418 .477 .521 .435 .448 -.130 Component 3 4 -.102 -.184 -.165 -.105 -.189 -.118 -.286 -.164 -.184 -.273 -.142 -.170 -.265 -.266 -.178 -.242 -.183 -.340 .360 -.131 .490 2 -.136 -.105 -.234 -.309 .408 .412 -.217 -.356 .361 .327 .446 -.369 .303 .353 -.351 -.258 .391 .450 .378 .295 .464 .436 .468 -.287 -.372 .112 -.382 -.102 -.404 -.469 .151 -.427 -.487 .523 -.151 .411 5 -.206 -.108 -.416 -.118 .228 .169 -.488 -.273 .243 .256 -.178 -.420 .131 -.459 .139 .134 .413 .453 .144 .110 .115 .135 -.242 .202 -.234 -.257 .395 .394 -.117 .135 Extraction Method: Principal Component Analys is. a. 6 components extracted. -.184 -.151 -.191 .352 -.125 -.165 .390 .343 -.202 -.121 .467 -.162 -.153 .120 .135 .421 -.392 -.320 .432 -.327 .118 .362 .423 .554 .522 6 -.127 -.237 -.110 -.103 .218 -.111 .204 -.121 .170 .120 .477 .463 .350 Rotated Component Matrixa DSE37 DSE32 DSE3 DSE41 DSE27 DSE21 DSE9 DSE15 DSE7 DSE19 DSE13 DSE25 DSE1 DSE39 DSE35 DSE30 DSE38 DSE8 DSE26 DSE2 DSE31 DSE40 DSE36 DSE14 DSE20 DSE10 DSE4 DSE16 DSE22 DSE34 DSE24 DSE18 DSE23 DSE5 DSE11 DSE17 DSE33 DSE28 DSE6 DSE29 DSE42 DSE12 1 .833 .788 .780 .779 .735 .730 .603 .597 .127 .105 .142 .177 .212 .147 .348 .141 .133 .175 .131 .117 .134 .169 .179 .267 .263 .291 2 .116 .172 .134 .130 .205 .102 .741 .720 .715 .687 .655 .650 .564 .550 .398 .113 .124 .127 .142 .269 .188 .243 .183 .124 .184 .372 .434 .416 .217 .252 .129 .217 .245 .201 .198 .244 .197 .357 .230 .262 .252 Component 3 4 .121 .166 .119 .162 .117 .169 .142 .120 .113 .253 .163 .225 .209 .227 .242 .308 .852 .843 .842 .748 .734 .681 .581 .276 .123 .153 .109 .109 .173 .181 .187 .109 .120 6 .115 .135 .114 .197 .161 .129 .365 -.106 .342 .321 .175 .157 .165 .169 .216 .205 .134 .173 .145 .112 .134 .123 .549 .122 .147 .196 .141 .139 .195 .795 .770 .722 .634 .608 .524 .520 .195 .172 .145 .246 .103 .252 .121 .245 .201 .327 .399 .273 .326 Extraction Method: Principal Component Analys is. Rotation Method: Varimax with Kaiser Normalization. a. Rotation converged in 6 iterations . 5 .101 .105 .141 .108 .168 .153 .190 .121 .259 .167 .175 .787 .764 .703 .702 .659 .630 .388 .161 .263 .162 .144 .124 .299 .122 .110 .211 .461 .589 .517 .510 Component Transformation Matrix Component 1 2 3 4 5 6 1 .443 .542 -.676 .161 .132 -.094 2 .499 -.418 -.013 -.485 .583 .022 3 .386 .569 .720 -.009 .084 .024 Extraction Method: Principal Component Analys is. Rotation Method: Varimax with Kaiser Normalization. 4 .461 -.179 -.010 -.288 -.751 -.330 5 .374 -.414 .133 .809 .063 -.108 6 .229 -.061 -.075 .020 -.261 .933