SW 983 LECTURE NOTES
CLUSTER ANALYSIS
Definition: Any of several procedures in multivariate analysis designed to determine whether
individuals (or other units of analysis) are similar or dissimilar enough to fall into groups or
clusters.
Aka: Q analysis, typology construction, classification analysis, and numerical taxonomy.
More an art than a science
The cluster variate is the set of variables representing the characteristics used to compare objects
in the cluster analysis. Cluster analysis is the only multivariate technique that does not estimate
the variate empirically but instead uses the variate as specified by the researcher. The focus of
cluster analysis is on the comparison of objects based on the variate, not on the estimation of the
variate itself.
Cluster analysis can be characterized as descriptive, atheoretical, and noninferential. Cluster
analysis has no statistical basis upon which to draw statistical inferences from a sample to a
population, and it is used primarily as an exploratory technique. The solutions are not unique,
as the cluster membership for any number of solutions is dependent upon many elements of the
procedure and many different solutions can be obtained by varying one or more elements.
Moreover, cluster analysis will always create clusters regardless of the “true existence of any
structure in the data. Finally, the cluster solution is totally dependent upon the variables used
as the basis for the similarity measure.
Related (but significantly different) to two other techniques:
1. Factor analysis – FA groups variables while CA groups subjects. Having said that you
will see that the SPSS program allows you to perform cluster analysis on variables or
cases. FA is sometimes done prior to CA with the factor scores used to perform CA.
Hair, et al, cites some research suggesting this may not be appropriate (p. 491). FA
generally has a more theoretical basis and provides statistical tests. CA is more ad hoc.
2. Discriminant analysis - DA is similar in that the goal is to classify a set of cases into
groups or categories, but in CA neither the number nor the members of the groups are
known. DA can be performed in conjunction with CA after the clusters are identified to
derive weights for the variate or clustering variables.
Steps:
1. Look for outliers – Requires multivariate test. Mahalanobis D2 provides such a test (see
p. 66 of text). Available as output variable in SPSS regression module.
2. Choose a distance measure – Squared Euclidian distance most commonly used.
3. Standardize the data (z scores) to avoid weighting importance of variables with larger
variance.
4. Choose a clustering method – hierarchical agglomerative method most commonly used.
5. Determine how many groups – Could be guided by theory but is mostly a subjective
judgment. Want clusters with fair representation (numbers of items), maximize between
group differences and minimize within cluster differences. Distance measure provides
more objective measure of this but is still relative.
6. Describe the clusters – May involve naming. What variables are hanging together for a
particular cluster? Run mean or proportion comparisons across clusters. Ability to make
sense of the clusters may influence your decision about how many clusters should exist.
Cluster Analysis with SPSS
SPSS provides three modules for doing cluster analysis. The TwoStep procedure is new and the
easiest to run. Also available are Hierarchical and K-Means cluster procedure.
The TwoStep Cluster Analysis procedure is an exploratory tool designed to reveal natural
groupings (or clusters) within a data set that would otherwise not be apparent. The algorithm
employed by this procedure has several desirable features that differentiate it from traditional
clustering techniques:



Handling of categorical and continuous variables. By assuming variables to be
independent, a joint multinomial-normal distribution can be placed on categorical and
continuous variables.
Automatic selection of number of clusters. By comparing the values of a model-choice
criterion across different clustering solutions, the procedure can automatically determine
the optimal number of clusters.
Scalability. By constructing a cluster features (CF) tree that summarizes the records, the
TwoStep algorithm allows you to analyze large data files.
Example. Retail and consumer product companies regularly apply clustering techniques to data
that describe their customers' buying habits, gender, age, income level, etc. These companies
tailor their marketing and product development strategies to each consumer group to increase
sales and build brand loyalty.
Statistics. The procedure produces information criteria (AIC or BIC) by numbers of clusters in
the solution, cluster frequencies for the final clustering, and descriptive statistics by cluster for
the final clustering.
Plots. The procedure produces bar charts of cluster frequencies, pie charts of cluster frequencies,
and variable importance charts.
Distance Measure. This selection determines how the similarity between two clusters is
computed.


Log-likelihood. The likelihood measure places a probability distribution on the variables.
Continuous variables are assumed to be normally distributed, while categorical variables
are assumed to be multinomial. All variables are assumed to be independent.
Euclidean. The Euclidean measure is the "straight line" distance between two clusters. It
can be used only when all of the variables are continuous.
Number of Clusters. This selection allows you to specify how the number of clusters is to be
determined.


Determine automatically. The procedure will automatically determine the "best" number
of clusters, using the criterion specified in the Clustering Criterion group. Optionally,
enter a positive integer specifying the maximum numbers of clusters that the procedure
should consider.
Specify fixed. Allows you to fix the number of clusters in the solution. Enter a positive
integer.
Count of Continuous Variables. This group provides a summary of the continuous variable
standardization specifications made in the Options dialog box.
Clustering Criterion. This selection determines how the automatic clustering algorithm
determines the number of clusters. Either the Bayesian Information Criterion (BIC) or the
Akaike Information Criterion (AIC) can be specified.
If you have a small number of cases, and want to choose between several methods for cluster
formation, variable transformation, and measuring the dissimilarity between clusters, try the
Hierarchical Cluster Analysis procedure. The Hierarchical Cluster Analysis procedure also
allows you to cluster variables instead of cases.
The K-Means Cluster Analysis procedure is limited to continuous variables, but can be used to
analyze large data sets and allows you to save the distances from cluster centers for each object.
Unlike the Hierarchical Cluster Analysis procedure which results in a series of solutions
corresponding to different numbers of clusters, the K-Means CA procedure produces only one
solution for the number of clusters requested by the user.