KV Petrides
Lecture 1
Psychometrics
Dr. K. V. Petrides
www.psychometriclab.com
KV Petrides
Individual differences
• Most branches of psychology are interested in
human (or animal) behaviour under different
experimental conditions.
• The assumption is that all people (or animals) are
pretty much the same.
• This assumption is patently wrong:
– Some children develop faster than others
– Some adults are less conscientious than others
– Some dogs are more intelligent than others
KV Petrides
Correlational vs experimental psychology
• There has always existed a divide between
correlational and experimental psychology.
• In what is perhaps the most famous presidential
address to the annual convention of the American
Psychological Association (APA), Cronbach
(1957) pointed out the existence of two opposing
camps in psychology:
– correlationists versus experimentalists
• This module concerns primarily
correlational psychology.
L J Cronbach
KV Petrides
Sir Francis Galton (1822-1911)
• Sir Francis Galton was one of the
greatest polymaths of all time.
His credits include:
– Father of individual differences.
– Pioneer of correlation and regression.
– Discoverer of ‘regression to the mean’
effect.
– Pioneer of fingerprint identification and
classification.
– Inventor of the weather map.
– Inventor of the Galton whistle (produces
sounds into the ultrasonic range).
– Founder (with Pearson and Weldon) of
Biometrika.
KV Petrides
Measurement & psychometrics I
• “Everything that exists, exists in some quantity and can
therefore be measured.”
- E L Thorndike
• Measurement concerns the systematic assignment of numbers to
represent quantitative attributes of objects or events.
• The concept of measurement is fundamentally important to the
study of individual differences and of psychology, more
generally.
• The ability to measure and quantify constructs allows us to test
falsifiable theoretical hypotheses, which is what establishes
psychology as a science and distinguishes it from other
disciplines (e.g., literature).
KV Petrides
Measurement & psychometrics II
• Psychometrics is the branch of psychology concerned
with the scientific measurement of individual
differences.
• The accurate assessment of individual differences is
important because theories can be tested if and only if
the constructs involved can be measured.
• In addition, a working knowledge of psychometrics is
a sine qua non for the understanding of:
– Advanced personality and intelligence theories.
– Advanced psychological applications (occupational,
educational, clinical, medical, military, etc.).
KV Petrides
Psychometrics
K G JÖreskog
H Goldstein
• Boosted by cheap computing power and some
truly gifted mathematical statisticians, the field
of psychometrics and behavioural statistics has
been expanding with unprecedented speed.
• These rapid and technically rigorous
developments have created a need for a new
breed of quantitative scientist who can work at
the interface between mathematical statistics and
applied research.
KV Petrides
Reliability and validity
• Two fundamental notions in psychometrics and
measurement theory more generally (e.g., blood
pressure) are reliability and validity.
• Reliability:Very broadly, the dependability of a
measurement instrument.
• Validity: The property that a measurement
instrument measures what it claims to measure.
• A reliable test may be invalid but a valid test may
not be unreliable.
KV Petrides
Reliability I
• There are several different types of
reliability:
– Internal consistency (Cronbach’s α):
• measures the homogeneity of the test (the degree to
which the various parts of an instrument measure the
same variable).
• technically, the proportion of true score variance
accounted for by a fallible measure. Its square root
indicates the correlation between observed and true
scores on a variable.
KV Petrides
Interpretation of Cronbach’s alpha
• The sampling distribution of Cronbach’s alpha is
unknown and, therefore, it is not possible to establish
significance levels, rejection regions, etc.
• Certain benchmark values have been proposed in
order to interpret Cronbach’s alpha:
–
–
–
–
–
α < .60 poor
.60 < α < .70 adequate
.70 < α < .80 good
.80 < α < .90 very good
α > .90 perhaps too good
KV Petrides
Reliability II
• Alternate-forms reliability: Involves the
construction of two tests of equal length by
randomly sampling from the same domain
(‘universe of items’). The Pearson correlation
between these two tests (forms) is an estimate of
the reliability.
• Parallel-forms reliability: Involves the
construction of two tests comprising items of the
same difficulty, which leads to similar score
distributions. The Pearson correlation between
these two tests (forms) is an estimate of the
reliability.
KV Petrides
Reliability III
• Split-half reliability: Randomly splits the test into
two halves, calculates the Pearson correlation
between them, and applies the Spearman-Brown
prophecy formula to estimate reliability. Still
used, but not recommended.
• Test-retest reliability: Also known as ‘temporal
stability’. Same test administered on two different
occasions spaced about one month or more apart.
The Pearson correlation between the two scores is
an estimate of the reliability. Note this is very
different from the foregoing types of reliability.
KV Petrides
Validity I
• There are very many different types of validity
and, in some cases, there is more than one label to
describe them. Here are the most common and
important ones:
– Construct validity: The degree to which a test measures
a specified construct as determined by the interpretation
of the psychological meaning of test scores and the
implications of this interpretation. Term introduced by
Cronbach and Meehl (1955). Construct validity is
important from a scientific perspective and rests on the
psychological theory underpinning an instrument.
KV Petrides
Validity II
• Criterion validity: the degree to which scores on a test
correlate with scores on a relevant external criterion. It is a
broad type of validity encompassing several more specific
types (e.g., convergent, predictive, concurrent).
• Convergent validity: the degree to which scores on a test
correlate with variables they are supposed to correlate
with, given the nature of the construct.
• Discriminant validity: the degree to which scores on a test
do NOT correlate with (are ‘independent of’ or
‘orthogonal to’) variables they are NOT supposed to
correlate with, given the nature of the construct.
KV Petrides
Validity III
• Predictive validity: the degree to which scores on a test
predict future behavior on a criterion variable.
• Concurrent validity: is based on the correlation between
predictor (test) and criterion scores obtained at
approximately the same time (e.g., self-reported and
clinically diagnosed depression).
• Congruent validity: the degree to which a new test
correlates with extant measurement instruments of the
construct. It is a weak type of validity because extant
measures may themselves have low validity.
• Face validity: ‘experts’ review test contents to determine if
they are appropriate ‘on their face’.
KV Petrides
Internal validity
• So far, we have looked at types of validity that are
relevant to individual differences constructs,
specifically, and correlational (nonexperimental)
research, more generally.
• There is another type of validity that is important
mainly in experimental research, viz., internal
validity.
• Internal validity: The extent to which the observed
effect on the dependent variable is caused only by
the experimental treatment condition.
KV Petrides
Threats to internal validity
• There are six well-known extraneous and,
therefore, potentially confounding variables that
must be controlled.
– History
– Maturation
– Mortality
– Instrumentation
– Statistical regression
– Selection
• Christensen, L. B. (2006). Experimental methodology
(10th ed.). Needham heights: Allyn and Bacon.
KV Petrides
Factor analysis
KV Petrides
Factor analysis I
• Principal component analysis (PCA) and principal
factor analysis (PFA) are statistical methods used
to reduce the dimensionality of a correlation (more
often) or a covariance (less often) matrix.
• Typically, the researcher’s interest is to find which
of the variables in a particular set (items, scales,
response latencies, etc.) form coherent and
relatively independent subsets.
• Variables that correlate highly together, but lower
with other variables in the set, are grouped into
components (PCA) or factors (PFA).
KV Petrides
Factor analysis II
KV Petrides
Factor analysis in SPSS
Variables to
be factored
Do NOT change
this default
Click ‘extraction’
to get menu on the
right
Click ‘Scree plot’ and run the analysis directly afterwards
KV Petrides
How many factors?
• A crucial consideration in PFA concerns the number of
factors to be extracted from the matrix. There are a host
of extraction criteria available, two of which are
especially common:
• Kaiser eigenvalue criterion: This is the default in SPSS
and extracts all factors with eigenvalues > 1. It is often
very misleading and should never be adopted
uncritically.
• Scree plot: Eigenvalues are plotted against the principal
components. The cut-off point for factor extraction is
where the line changes slope. We extract all the factors
that lie above the break. If there are two breaks in the
slope, we extract all the factors above the leftmost break.
KV Petrides
Scree plot
Scree Plot
6
The scree test is one of the best criteria for extracting the
right number of factors. Its major disadvantage is that it
often requires considerable experience and expertise to
interpret it correctly. In many cases, it should be
supplemented by additional criteria (e.g., parallel
analysis). In the example below, the scree points to a 4factor solution. However, the correct number of factors
happens to be 3 in this case (I know this from additional
analyses).
5
4
3
2
1
0
1
3
5
7
Component Number
9
11
13
15
17
19
21
KV Petrides
Factor rotation I
• The main aim of factor rotation is to maximize
interpretability.
• Factor solutions with multiple variables having
moderate loadings on multiple factors are virtually
uninterpretable.
• Rotational algorithms try to identify solutions in which:
– Each factor comprises a few variables with high loadings and
many variables with low loadings.
– Each variable has a high loading on one factor and low
loadings on all the other factors.
– Such solutions are called ‘simple structure’ solutions.
KV Petrides
Unrotated matrix
KV Petrides
PCA/PFA II
KV Petrides
Factor rotation II
• There are two types of rotations:
– Orthogonal rotations (e.g., VARIMAX) keep all factors
at right angles (i.e., they do not allow them to
intercorrelate).
• Advantages: simple and easily interpretable solutions.
• Disadvantages: solutions are often unrealistic.
– Oblique rotations (e.g., OBLIMIN) allow factors to
intercorrelate, if necessary.
• Advantages: Solutions are precise, flexible, and realistic.
• Disadvantages: Solutions are more difficult to interpret due to
factor intercorrelations and the involvement of several matrices.
KV Petrides
The structure of cognitive abilities
Cooper (2002)
• Cattell’s gc
(crystallized) and gf
(fluid) are second-order
factors.
– gc: consolidated
knowledge arising
from educational
opportunities.
– gf: the capacity to
figure out new
problems. Most loaded
on tests with virtually
no scholastic or
cultural content (e.g,
Raven’s).
KV Petrides
The structure of personality
KV Petrides
FA output I
Percentage of item variance
accounted for by factors.
Percentage of total variance
accounted for by each factor.
KV Petrides
FA output II
• We ignore the part of the output labelled ‘component matrix’ and focus
on the ‘rotated component matrix’.
Factor loadings
KV Petrides
On the Web
• http://issid.org/issid.html
– International Society for the Study of
Individual Differences (ISSID).
• http://www.personality-project.org/
– Maintained by William Revelle.